[v4,0/8] expand list of optional libraries

  This patchset contains changes to DPDK to make the build more configurable
overall. The changes can be grouped into two areas:

* Firstly, there are changes to make the build of the unit tests more
  flexible and maintainable. These 5 patches switch the unit tests from
  having separate hard-coded lists of files and tests, with lots of
  conditional logic to add/remove items from the list, to instead being
  a single list of files and dependencies per-file. The actual lists of
  test-suites are built up dynamically from the list of files whose
  dependencies are met and therefore will be part of the build.
  This rework enables...
* The second set of patches which expands the list of libraries which
  can be disabled at build-time, while still giving a successful build
  of DPDK. Overall, after this set is applied, the list of optional
  libraries in DPDK is approx doubled in size. [15 additions giving
  31 optional libs]

NOTES:
* as stated in the RFC, the test binary moves in the build as a result
  of this set - from app/test/dpdk-test to app/dpdk-test.
* running the suites of tests no longer works after patch 4 until patch
  5 is applied. If this is an issue, the two patches can be merged on
  apply. However, the individual tests are still runnable through the
  test binary - it's only through "meson test" that there are issues, so
  I'm keeping the patches separate since each is relatively complex in its
  own right.
* A number of files in the test directory were not being built, and seem
  to have never been built with meson. This requires separate work, outside
  the scope of the patchset to fix.
* Similarly, some unit tests were never added to test suites. These can
  now be found by using grep to find the old REGISTER_TEST_COMMAND macro.
  Again, adding orphan tests to suites is outside the scope of this set.

V4:
- improve test suite definition, hopefully improving test pass rate:
  - drop unnecessary file-prefix for fast tests, since tests are not parallel
  - always pass driver path flag in "meson test" for shared builds.

V3:
- Create symlink to old path to test binary, to ensure all automated CI,
  and any other use of the test binary still works.

V2:
- Changes since RFC:
   -  merged two previous sets - reworking test build, and disabling
      libs - into one, since they depend on each other
   -  Reworked the list of dependencies for each file in the tests,
      so that files that depend on other files have those deps called
      out.
   -  Added list of external dependencies to the test builds
   -  For disabling libs, merged patches disabling individual libs
      together into one patch, since all unit test build changes were
      unnecessary. This made each patch a single-line addition to
      the optional list in lib/meson.build.

Bruce Richardson (8):
  app/test: add new macros for various test types
  app/test: tag tests with the test type
  app/test: make telemetry data test buildable on windows
  app/test: build using per-file dependency matrix
  app/test: define unit tests suites based on test macros
  examples/l3fwd: make eventdev an optional dependency
  build: make most device classes optional
  build: expand list of optional libraries

 app/meson.build                               |  15 +-
 app/test/meson.build                          | 823 +++++-------------
 app/test/suites/meson.build                   |  72 ++
 app/test/test.h                               |   9 +-
 app/test/test_acl.c                           |   2 +-
 app/test/test_atomic.c                        |   2 +-
 app/test/test_barrier.c                       |   2 +-
 app/test/test_bitmap.c                        |   2 +-
 app/test/test_bitops.c                        |   2 +-
 app/test/test_bpf.c                           |   4 +-
 app/test/test_byteorder.c                     |   2 +-
 app/test/test_cksum.c                         |   2 +-
 app/test/test_cmdline.c                       |   2 +-
 app/test/test_common.c                        |   2 +-
 app/test/test_cpuflags.c                      |   2 +-
 app/test/test_crc.c                           |   2 +-
 app/test/test_cryptodev.c                     |  38 +-
 app/test/test_cryptodev_asym.c                |   2 +-
 app/test/test_cycles.c                        |   2 +-
 app/test/test_debug.c                         |   2 +-
 app/test/test_devargs.c                       |   2 +-
 app/test/test_distributor.c                   |   2 +-
 app/test/test_distributor_perf.c              |   2 +-
 app/test/test_dmadev.c                        |   2 +-
 app/test/test_eal_flags.c                     |  24 +-
 app/test/test_eal_fs.c                        |   2 +-
 app/test/test_efd.c                           |   2 +-
 app/test/test_efd_perf.c                      |   2 +-
 app/test/test_errno.c                         |   2 +-
 app/test/test_ethdev_link.c                   |   2 +-
 app/test/test_event_ring.c                    |   2 +-
 app/test/test_eventdev.c                      |   2 +-
 app/test/test_fbarray.c                       |   2 +-
 app/test/test_fib.c                           |   4 +-
 app/test/test_fib6.c                          |   4 +-
 app/test/test_fib6_perf.c                     |   2 +-
 app/test/test_fib_perf.c                      |   2 +-
 app/test/test_func_reentrancy.c               |   2 +-
 app/test/test_hash.c                          |   2 +-
 app/test/test_hash_functions.c                |   2 +-
 app/test/test_hash_multiwriter.c              |   2 +-
 app/test/test_hash_perf.c                     |   2 +-
 app/test/test_hash_readwrite.c                |   4 +-
 app/test/test_hash_readwrite_lf_perf.c        |   2 +-
 app/test/test_interrupts.c                    |   2 +-
 app/test/test_ipfrag.c                        |   2 +-
 app/test/test_ipsec.c                         |   2 +-
 app/test/test_ipsec_perf.c                    |   2 +-
 app/test/test_kvargs.c                        |   2 +-
 app/test/test_lcores.c                        |   2 +-
 app/test/test_logs.c                          |   2 +-
 app/test/test_lpm.c                           |   2 +-
 app/test/test_lpm6.c                          |   2 +-
 app/test/test_lpm6_perf.c                     |   2 +-
 app/test/test_lpm_perf.c                      |   2 +-
 app/test/test_malloc.c                        |   2 +-
 app/test/test_malloc_perf.c                   |   2 +-
 app/test/test_mbuf.c                          |   2 +-
 app/test/test_mcslock.c                       |   2 +-
 app/test/test_member.c                        |   2 +-
 app/test/test_member_perf.c                   |   2 +-
 app/test/test_memcpy.c                        |   2 +-
 app/test/test_memcpy_perf.c                   |   2 +-
 app/test/test_memory.c                        |   2 +-
 app/test/test_mempool.c                       |   2 +-
 app/test/test_mempool_perf.c                  |   2 +-
 app/test/test_memzone.c                       |   2 +-
 app/test/test_meter.c                         |   2 +-
 app/test/test_mp_secondary.c                  |   2 +-
 app/test/test_per_lcore.c                     |   2 +-
 app/test/test_pflock.c                        |   2 +-
 app/test/test_pie.c                           |   6 +-
 app/test/test_pmd_perf.c                      |   2 +-
 app/test/test_power.c                         |   2 +-
 app/test/test_power_cpufreq.c                 |   2 +-
 app/test/test_power_intel_uncore.c            |   2 +-
 app/test/test_power_kvm_vm.c                  |   2 +-
 app/test/test_prefetch.c                      |   2 +-
 app/test/test_rand_perf.c                     |   2 +-
 app/test/test_rcu_qsbr.c                      |   2 +-
 app/test/test_rcu_qsbr_perf.c                 |   2 +-
 app/test/test_reassembly_perf.c               |   2 +-
 app/test/test_reciprocal_division.c           |   2 +-
 app/test/test_reciprocal_division_perf.c      |   2 +-
 app/test/test_red.c                           |   4 +-
 app/test/test_reorder.c                       |   2 +-
 app/test/test_rib.c                           |   4 +-
 app/test/test_rib6.c                          |   4 +-
 app/test/test_ring.c                          |   2 +-
 app/test/test_ring_perf.c                     |   2 +-
 app/test/test_rwlock.c                        |   8 +-
 app/test/test_sched.c                         |   2 +-
 app/test/test_security.c                      |   2 +-
 app/test/test_seqlock.c                       |   2 +-
 app/test/test_service_cores.c                 |   4 +-
 app/test/test_spinlock.c                      |   2 +-
 app/test/test_stack.c                         |   4 +-
 app/test/test_stack_perf.c                    |   4 +-
 app/test/test_string_fns.c                    |   2 +-
 app/test/test_tailq.c                         |   2 +-
 app/test/test_telemetry_data.c                |  14 +-
 app/test/test_thash.c                         |   2 +-
 app/test/test_thash_perf.c                    |   2 +-
 app/test/test_threads.c                       |   2 +-
 app/test/test_ticketlock.c                    |   2 +-
 app/test/test_timer.c                         |   2 +-
 app/test/test_timer_perf.c                    |   2 +-
 app/test/test_timer_racecond.c                |   2 +-
 app/test/test_trace.c                         |   2 +-
 app/test/test_trace_perf.c                    |   2 +-
 app/test/test_version.c                       |   2 +-
 buildtools/get-test-suites.py                 |  33 +
 .../has-hugepages.py                          |   0
 buildtools/meson.build                        |   2 +
 examples/l3fwd/l3fwd_em.c                     |   2 +
 examples/l3fwd/l3fwd_em_hlm.h                 |   2 +
 examples/l3fwd/l3fwd_event.c                  |   2 +
 examples/l3fwd/l3fwd_event.h                  |   7 +-
 examples/l3fwd/l3fwd_event_generic.c          |   2 +
 examples/l3fwd/l3fwd_event_internal_port.c    |   2 +
 examples/l3fwd/l3fwd_fib.c                    |   2 +
 examples/l3fwd/l3fwd_lpm.c                    |   2 +
 examples/l3fwd/main.c                         |  65 +-
 examples/l3fwd/meson.build                    |   5 +-
 lib/meson.build                               |  15 +
 125 files changed, 610 insertions(+), 764 deletions(-)
 create mode 100644 app/test/suites/meson.build
 create mode 100644 buildtools/get-test-suites.py
 rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%)

--
2.39.2
  

On Tue, Aug 15, 2023 at 02:13:12PM +0100, Bruce Richardson wrote:
> This patchset contains changes to DPDK to make the build more configurable
> overall. The changes can be grouped into two areas:
> 
> * Firstly, there are changes to make the build of the unit tests more
>   flexible and maintainable. These 5 patches switch the unit tests from
>   having separate hard-coded lists of files and tests, with lots of
>   conditional logic to add/remove items from the list, to instead being
>   a single list of files and dependencies per-file. The actual lists of
>   test-suites are built up dynamically from the list of files whose
>   dependencies are met and therefore will be part of the build.
>   This rework enables...
> * The second set of patches which expands the list of libraries which
>   can be disabled at build-time, while still giving a successful build
>   of DPDK. Overall, after this set is applied, the list of optional
>   libraries in DPDK is approx doubled in size. [15 additions giving
>   31 optional libs]
> 
> NOTES:
> * as stated in the RFC, the test binary moves in the build as a result
>   of this set - from app/test/dpdk-test to app/dpdk-test.
> * running the suites of tests no longer works after patch 4 until patch
>   5 is applied. If this is an issue, the two patches can be merged on
>   apply. However, the individual tests are still runnable through the
>   test binary - it's only through "meson test" that there are issues, so
>   I'm keeping the patches separate since each is relatively complex in its
>   own right.
> * A number of files in the test directory were not being built, and seem
>   to have never been built with meson. This requires separate work, outside
>   the scope of the patchset to fix.
> * Similarly, some unit tests were never added to test suites. These can
>   now be found by using grep to find the old REGISTER_TEST_COMMAND macro.
>   Again, adding orphan tests to suites is outside the scope of this set.
> 
> V4:
> - improve test suite definition, hopefully improving test pass rate:
>   - drop unnecessary file-prefix for fast tests, since tests are not parallel
>   - always pass driver path flag in "meson test" for shared builds.
> 
Missed applying Morten's ack to the series in V4. Adding it here.

Series-acked-by: Morten Brørup <mb@smartsharesystems.com>