diff mbox series

test: rely on EAL detection for core list

Message ID 20211018170136.5189-1-david.marchand@redhat.com (mailing list archive)
State Superseded, archived
Delegated to: David Marchand
Headers
Series test: rely on EAL detection for core list |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/github-robot: build success github build: passed
ci/iol-broadcom-Functional success Functional Testing PASS
ci/iol-broadcom-Performance success Performance Testing PASS
ci/iol-x86_64-unit-testing success Testing PASS
ci/iol-x86_64-compile-testing success Testing PASS
ci/iol-intel-Performance success Performance Testing PASS
ci/iol-aarch64-compile-testing success Testing PASS
ci/iol-intel-Functional success Functional Testing PASS
ci/iol-mellanox-Performance success Performance Testing PASS

Commit Message

David Marchand Oct. 18, 2021, 5:01 p.m. UTC 
  Cores count has a direct impact on the time needed to complete unit
tests.

Currently, the core list used for unit test is enforced to "all cores on
the system" with no way for (CI) users to adapt it.
On the other hand, EAL default behavior (when no -c/-l option gets passed)
is to start threads on as many cores available in the process cpu
affinity.

Remove logic from meson: users can then select where to run the tests by
either running meson with a custom cpu affinity (using taskset/cpuset
depending on OS) or by passing a --test-args option to meson.

Example:
$ sudo meson test -C build --suite fast-tests -t 3 --test-args "-l 0-3"

Signed-off-by: David Marchand <david.marchand@redhat.com>
---
I wanted to post this as a RFC, but now, I wonder if all CI test RFC
patches, so sending as a normal patch.

---
 app/test/get-coremask.sh | 13 -------------
 app/test/meson.build     | 10 +---------
 2 files changed, 1 insertion(+), 22 deletions(-)
 delete mode 100755 app/test/get-coremask.sh

Comments

Bruce Richardson Oct. 18, 2021, 5:53 p.m. UTC | #1 
On Mon, Oct 18, 2021 at 07:01:36PM +0200, David Marchand wrote:
> Cores count has a direct impact on the time needed to complete unit
> tests.
> 
> Currently, the core list used for unit test is enforced to "all cores on
> the system" with no way for (CI) users to adapt it.
> On the other hand, EAL default behavior (when no -c/-l option gets passed)
> is to start threads on as many cores available in the process cpu
> affinity.
> 
> Remove logic from meson: users can then select where to run the tests by
> either running meson with a custom cpu affinity (using taskset/cpuset
> depending on OS) or by passing a --test-args option to meson.
> 
> Example:
> $ sudo meson test -C build --suite fast-tests -t 3 --test-args "-l 0-3"
> 
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---
> I wanted to post this as a RFC, but now, I wonder if all CI test RFC
> patches, so sending as a normal patch.
> 
> ---
I really like this idea! Patch looks good other than it needs some doc
changes.
David Marchand Oct. 19, 2021, 9:52 a.m. UTC | #2 
On Mon, Oct 18, 2021 at 7:53 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> On Mon, Oct 18, 2021 at 07:01:36PM +0200, David Marchand wrote:
> > Cores count has a direct impact on the time needed to complete unit
> > tests.
> >
> > Currently, the core list used for unit test is enforced to "all cores on
> > the system" with no way for (CI) users to adapt it.
> > On the other hand, EAL default behavior (when no -c/-l option gets passed)
> > is to start threads on as many cores available in the process cpu
> > affinity.
> >
> > Remove logic from meson: users can then select where to run the tests by
> > either running meson with a custom cpu affinity (using taskset/cpuset
> > depending on OS) or by passing a --test-args option to meson.
> >
> > Example:
> > $ sudo meson test -C build --suite fast-tests -t 3 --test-args "-l 0-3"
> >
> > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > ---
> > I wanted to post this as a RFC, but now, I wonder if all CI test RFC
> > patches, so sending as a normal patch.
> >
> > ---
> I really like this idea! Patch looks good other than it needs some doc
> changes.

Wdyt of:

diff --git a/doc/guides/prog_guide/meson_ut.rst
b/doc/guides/prog_guide/meson_ut.rst
index fff88655dd..d35e0577c8 100644
--- a/doc/guides/prog_guide/meson_ut.rst
+++ b/doc/guides/prog_guide/meson_ut.rst
@@ -47,9 +47,15 @@ Arguments of ``test()`` that can be provided in
meson.build are as below:

 * ``is_parallel`` is used to run test case either in parallel or
non-parallel mode.
 * ``timeout`` is used to specify the timeout of test case.
-* ``args`` is used to specify test specific parameters.
+* ``args`` is used to specify test specific parameters (see note below).
 * ``env`` is used to specify test specific environment parameters.

+Note: the content of meson ``--test-args`` option and the content of ``args``
+are appended when invoking the DPDK test binary.
+Because of this, it is recommended not to set any default coremask or memory
+configuration in per test ``args`` and rather let users select what best fits
+their environment. If a test can't run, then it should be skipped, as described
+below.

 Dealing with skipped test cases
 -------------------------------
Bruce Richardson Oct. 19, 2021, 10:04 a.m. UTC | #3 
On Tue, Oct 19, 2021 at 11:52:32AM +0200, David Marchand wrote:
> On Mon, Oct 18, 2021 at 7:53 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > On Mon, Oct 18, 2021 at 07:01:36PM +0200, David Marchand wrote:
> > > Cores count has a direct impact on the time needed to complete unit
> > > tests.
> > >
> > > Currently, the core list used for unit test is enforced to "all cores on
> > > the system" with no way for (CI) users to adapt it.
> > > On the other hand, EAL default behavior (when no -c/-l option gets passed)
> > > is to start threads on as many cores available in the process cpu
> > > affinity.
> > >
> > > Remove logic from meson: users can then select where to run the tests by
> > > either running meson with a custom cpu affinity (using taskset/cpuset
> > > depending on OS) or by passing a --test-args option to meson.
> > >
> > > Example:
> > > $ sudo meson test -C build --suite fast-tests -t 3 --test-args "-l 0-3"
> > >
> > > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > > ---
> > > I wanted to post this as a RFC, but now, I wonder if all CI test RFC
> > > patches, so sending as a normal patch.
> > >
> > > ---
> > I really like this idea! Patch looks good other than it needs some doc
> > changes.
> 
> Wdyt of:
> 
> diff --git a/doc/guides/prog_guide/meson_ut.rst
> b/doc/guides/prog_guide/meson_ut.rst
> index fff88655dd..d35e0577c8 100644
> --- a/doc/guides/prog_guide/meson_ut.rst
> +++ b/doc/guides/prog_guide/meson_ut.rst
> @@ -47,9 +47,15 @@ Arguments of ``test()`` that can be provided in
> meson.build are as below:
> 
>  * ``is_parallel`` is used to run test case either in parallel or
> non-parallel mode.
>  * ``timeout`` is used to specify the timeout of test case.
> -* ``args`` is used to specify test specific parameters.
> +* ``args`` is used to specify test specific parameters (see note below).
>  * ``env`` is used to specify test specific environment parameters.
> 
> +Note: the content of meson ``--test-args`` option and the content of ``args``
> +are appended when invoking the DPDK test binary.
> +Because of this, it is recommended not to set any default coremask or memory
> +configuration in per test ``args`` and rather let users select what best fits
> +their environment. If a test can't run, then it should be skipped, as described
> +below.
> 
>  Dealing with skipped test cases
>  -------------------------------

That text is good. However, I also think we need an example above of using
test-args to limit core masks. How about also adding something like:

diff --git a/doc/guides/prog_guide/meson_ut.rst b/doc/guides/prog_guide/meson_ut.rst
index fff88655dd..4924066556 100644
--- a/doc/guides/prog_guide/meson_ut.rst
+++ b/doc/guides/prog_guide/meson_ut.rst
@@ -37,6 +37,13 @@ command::

     $ meson test --suite fast-tests

+If desired, additional arguments can be passed to the test run via the meson ``--test-args`` option.
+For example, tests will by default run on as many available cores as is needed for the test,
+starting with the lowest number core - generally core 0.
+To run the fast-tests suite using only cores 8 through 16, one can use::
+
+   $ meson test --suite fast-tests --test-args="-l 8-16"
+
 The meson command to list all available tests::

     $ meson test --list
diff mbox series

Patch

diff --git a/app/test/get-coremask.sh b/app/test/get-coremask.sh
deleted file mode 100755
index bb8cf404d2..0000000000
--- a/app/test/get-coremask.sh
+++ /dev/null
@@ -1,13 +0,0 @@ 
-#! /bin/sh -e
-# SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2019 Intel Corporation
-
-if [ "$(uname)" = "Linux" ] ; then
-	cat /sys/devices/system/cpu/present
-elif [ "$(uname)" = "FreeBSD" ] ; then
-	ncpus=$(/sbin/sysctl -n hw.ncpu)
-	echo 0-$(expr $ncpus - 1)
-else
-# fallback
-	echo 0-3
-fi
diff --git a/app/test/meson.build b/app/test/meson.build
index a16374b7a1..c7b377d52d 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -463,13 +463,8 @@  message('hugepage availability: @0@'.format(has_hugepage))
 timeout_seconds = 600
 timeout_seconds_fast = 10
 
-get_coremask = find_program('get-coremask.sh')
-num_cores_arg = '-l ' + run_command(get_coremask).stdout().strip()
-
-default_test_args = [num_cores_arg]
-
 foreach arg : fast_tests
-    test_args = default_test_args
+    test_args = []
     run_test = true
     if not has_hugepage
         if arg[1]
@@ -502,7 +497,6 @@  endforeach
 foreach arg : perf_test_names
     test(arg, dpdk_test,
             env : ['DPDK_TEST=' + arg],
-            args : default_test_args,
             timeout : timeout_seconds,
             is_parallel : false,
             suite : 'perf-tests')
@@ -511,7 +505,6 @@  endforeach
 foreach arg : driver_test_names
     test(arg, dpdk_test,
             env : ['DPDK_TEST=' + arg],
-            args : default_test_args,
             timeout : timeout_seconds,
             is_parallel : false,
             suite : 'driver-tests')
@@ -520,7 +513,6 @@  endforeach
 foreach arg : dump_test_names
     test(arg, dpdk_test,
             env : ['DPDK_TEST=' + arg],
-            args : default_test_args,
             timeout : timeout_seconds,
             is_parallel : false,
             suite : 'debug-tests')