Message ID | 20230814151609.3077077-1-bruce.richardson@intel.com (mailing list archive) |
---|---|
Headers |
Return-Path: <ci-bounces@dpdk.org> X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DA83243063; Mon, 14 Aug 2023 17:16:22 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CAC974329B; Mon, 14 Aug 2023 17:16:22 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.20]) by mails.dpdk.org (Postfix) with ESMTP id 3093D40A7F; Mon, 14 Aug 2023 17:16:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1692026180; x=1723562180; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=u5SOVpMgaQ3A96zpNxJcDAB+6+yBxdzje2zJz2VYsPY=; b=b1+Lf1bTEMLGDSV3dHBFQ+n2lfJc5vIu0N3cbWMWGD0xustaug2Rsnhu kMLaq+ZiQLkkQpzMj5j1Y/JA2ZZe0XxYdUY+KRpry+FzVWW8ijOr4BXP/ maFWxLDqSKHSuse04eJ+MiLe/HIqlDXlIPCtWM1TB4NtnQGRzG7XKSTUA NfgejefXMy23DyAbVuCqJrp5IkS425w8uHe5xTWsHbr/DU54AxOIajnYi 2BMv8BpmpdlfysjCQLcXFLQKSmcep+ZZkuQbCsDo8gNfuSIEznhDO4xmo ZAGwdqVc+YjcUg+zrbJQPZvkESXxNwRZ0aF/HfYB430khGPoeUXudqtDp Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10802"; a="362211241" X-IronPort-AV: E=Sophos;i="6.01,172,1684825200"; d="scan'208";a="362211241" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Aug 2023 08:16:19 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10802"; a="803499898" X-IronPort-AV: E=Sophos;i="6.01,172,1684825200"; d="scan'208";a="803499898" Received: from silpixa00401385.ir.intel.com ([10.237.214.14]) by fmsmga004.fm.intel.com with ESMTP; 14 Aug 2023 08:16:17 -0700 From: Bruce Richardson <bruce.richardson@intel.com> To: dev@dpdk.org Cc: ci@dpdk.org, Bruce Richardson <bruce.richardson@intel.com> Subject: [PATCH v2 0/8] expand list of optional libraries Date: Mon, 14 Aug 2023 16:16:01 +0100 Message-Id: <20230814151609.3077077-1-bruce.richardson@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230721115125.55137-1-bruce.richardson@intel.com> References: <20230721115125.55137-1-bruce.richardson@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions <ci.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/ci>, <mailto:ci-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/ci/> List-Post: <mailto:ci@dpdk.org> List-Help: <mailto:ci-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/ci>, <mailto:ci-request@dpdk.org?subject=subscribe> Errors-To: ci-bounces@dpdk.org |
Series |
expand list of optional libraries
|
|
Message
Bruce Richardson
Aug. 14, 2023, 3:16 p.m. UTC
This patchset contains changes to DPDK to make the build more configurable overall. The changes can be grouped into two areas: * Firstly, there are changes to make the build of the unit tests more flexible and maintainable. These 5 patches switch the unit tests from having separate hard-coded lists of files and tests, with lots of conditional logic to add/remove items from the list, to instead being a single list of files and dependencies per-file. The actual lists of test-suites are built up dynamically from the list of files whose dependencies are met and therefore will be part of the build. This rework enables... * The second set of patches which expands the list of libraries which can be disabled at build-time, while still giving a successful build of DPDK. Overall, after this set is applied, the list of optional libraries in DPDK is approx doubled in size. [15 additions giving 31 optional libs] NOTES: * as stated in the RFC, the test binary moves in the build as a result of this set - from app/test/dpdk-test to app/dpdk-test. * running the suites of tests no longer works after patch 4 until patch 5 is applied. If this is an issue, the two patches can be merged on apply. However, the individual tests are still runnable through the test binary - it's only through "meson test" that there are issues, so I'm keeping the patches separate since each is relatively complex in its own right. V2: - Changes since RFC: - merged two previous sets - reworking test build, and disabling libs - into one, since they depend on each other - Reworked the list of dependencies for each file in the tests, so that files that depend on other files have those deps called out. - Added list of external dependencies to the test builds - For disabling libs, merged patches disabling individual libs together into one patch, since all unit test build changes were unnecessary. This made each patch a single-line addition to the optional list in lib/meson.build. Bruce Richardson (8): app/test: add new macros for various test types app/test: tag tests with the test type app/test: make telemetry data test buildable on windows app/test: build using per-file dependency matrix app/test: define unit tests suites based on test macros examples/l3fwd: make eventdev an optional dependency build: make most device classes optional build: expand list of optional libraries app/meson.build | 15 +- app/test/meson.build | 818 +++++------------- app/test/suites/meson.build | 76 ++ app/test/test.h | 9 +- app/test/test_acl.c | 2 +- app/test/test_atomic.c | 2 +- app/test/test_barrier.c | 2 +- app/test/test_bitmap.c | 2 +- app/test/test_bitops.c | 2 +- app/test/test_bpf.c | 4 +- app/test/test_byteorder.c | 2 +- app/test/test_cksum.c | 2 +- app/test/test_cmdline.c | 2 +- app/test/test_common.c | 2 +- app/test/test_cpuflags.c | 2 +- app/test/test_crc.c | 2 +- app/test/test_cryptodev.c | 38 +- app/test/test_cryptodev_asym.c | 2 +- app/test/test_cycles.c | 2 +- app/test/test_debug.c | 2 +- app/test/test_devargs.c | 2 +- app/test/test_distributor.c | 2 +- app/test/test_distributor_perf.c | 2 +- app/test/test_dmadev.c | 2 +- app/test/test_eal_flags.c | 24 +- app/test/test_eal_fs.c | 2 +- app/test/test_efd.c | 2 +- app/test/test_efd_perf.c | 2 +- app/test/test_errno.c | 2 +- app/test/test_ethdev_link.c | 2 +- app/test/test_event_ring.c | 2 +- app/test/test_eventdev.c | 2 +- app/test/test_fbarray.c | 2 +- app/test/test_fib.c | 4 +- app/test/test_fib6.c | 4 +- app/test/test_fib6_perf.c | 2 +- app/test/test_fib_perf.c | 2 +- app/test/test_func_reentrancy.c | 2 +- app/test/test_hash.c | 2 +- app/test/test_hash_functions.c | 2 +- app/test/test_hash_multiwriter.c | 2 +- app/test/test_hash_perf.c | 2 +- app/test/test_hash_readwrite.c | 4 +- app/test/test_hash_readwrite_lf_perf.c | 2 +- app/test/test_interrupts.c | 2 +- app/test/test_ipfrag.c | 2 +- app/test/test_ipsec.c | 2 +- app/test/test_ipsec_perf.c | 2 +- app/test/test_kvargs.c | 2 +- app/test/test_lcores.c | 2 +- app/test/test_logs.c | 2 +- app/test/test_lpm.c | 2 +- app/test/test_lpm6.c | 2 +- app/test/test_lpm6_perf.c | 2 +- app/test/test_lpm_perf.c | 2 +- app/test/test_malloc.c | 2 +- app/test/test_malloc_perf.c | 2 +- app/test/test_mbuf.c | 2 +- app/test/test_mcslock.c | 2 +- app/test/test_member.c | 2 +- app/test/test_member_perf.c | 2 +- app/test/test_memcpy.c | 2 +- app/test/test_memcpy_perf.c | 2 +- app/test/test_memory.c | 2 +- app/test/test_mempool.c | 2 +- app/test/test_mempool_perf.c | 2 +- app/test/test_memzone.c | 2 +- app/test/test_meter.c | 2 +- app/test/test_mp_secondary.c | 2 +- app/test/test_per_lcore.c | 2 +- app/test/test_pflock.c | 2 +- app/test/test_pie.c | 6 +- app/test/test_pmd_perf.c | 2 +- app/test/test_power.c | 2 +- app/test/test_power_cpufreq.c | 2 +- app/test/test_power_intel_uncore.c | 2 +- app/test/test_power_kvm_vm.c | 2 +- app/test/test_prefetch.c | 2 +- app/test/test_rand_perf.c | 2 +- app/test/test_rcu_qsbr.c | 2 +- app/test/test_rcu_qsbr_perf.c | 2 +- app/test/test_reassembly_perf.c | 2 +- app/test/test_reciprocal_division.c | 2 +- app/test/test_reciprocal_division_perf.c | 2 +- app/test/test_red.c | 4 +- app/test/test_reorder.c | 2 +- app/test/test_rib.c | 4 +- app/test/test_rib6.c | 4 +- app/test/test_ring.c | 2 +- app/test/test_ring_perf.c | 2 +- app/test/test_rwlock.c | 8 +- app/test/test_sched.c | 2 +- app/test/test_security.c | 2 +- app/test/test_seqlock.c | 2 +- app/test/test_service_cores.c | 4 +- app/test/test_spinlock.c | 2 +- app/test/test_stack.c | 4 +- app/test/test_stack_perf.c | 4 +- app/test/test_string_fns.c | 2 +- app/test/test_tailq.c | 2 +- app/test/test_telemetry_data.c | 14 +- app/test/test_thash.c | 2 +- app/test/test_thash_perf.c | 2 +- app/test/test_threads.c | 2 +- app/test/test_ticketlock.c | 2 +- app/test/test_timer.c | 2 +- app/test/test_timer_perf.c | 2 +- app/test/test_timer_racecond.c | 2 +- app/test/test_trace.c | 2 +- app/test/test_trace_perf.c | 2 +- app/test/test_version.c | 2 +- buildtools/get-test-suites.py | 33 + .../has-hugepages.py | 0 buildtools/meson.build | 2 + examples/l3fwd/l3fwd_em.c | 2 + examples/l3fwd/l3fwd_em_hlm.h | 2 + examples/l3fwd/l3fwd_event.c | 2 + examples/l3fwd/l3fwd_event.h | 7 +- examples/l3fwd/l3fwd_event_generic.c | 2 + examples/l3fwd/l3fwd_event_internal_port.c | 2 + examples/l3fwd/l3fwd_fib.c | 2 + examples/l3fwd/l3fwd_lpm.c | 2 + examples/l3fwd/main.c | 65 +- examples/l3fwd/meson.build | 5 +- lib/meson.build | 15 + 125 files changed, 607 insertions(+), 766 deletions(-) create mode 100644 app/test/suites/meson.build create mode 100644 buildtools/get-test-suites.py rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%) -- 2.39.2
Comments
On Mon, Aug 14, 2023 at 04:16:01PM +0100, Bruce Richardson wrote: > This patchset contains changes to DPDK to make the build more configurable > overall. The changes can be grouped into two areas: > > * Firstly, there are changes to make the build of the unit tests more > flexible and maintainable. These 5 patches switch the unit tests from > having separate hard-coded lists of files and tests, with lots of > conditional logic to add/remove items from the list, to instead being > a single list of files and dependencies per-file. The actual lists of > test-suites are built up dynamically from the list of files whose > dependencies are met and therefore will be part of the build. > This rework enables... > * The second set of patches which expands the list of libraries which > can be disabled at build-time, while still giving a successful build > of DPDK. Overall, after this set is applied, the list of optional > libraries in DPDK is approx doubled in size. [15 additions giving > 31 optional libs] > > NOTES: > * as stated in the RFC, the test binary moves in the build as a result > of this set - from app/test/dpdk-test to app/dpdk-test. > * running the suites of tests no longer works after patch 4 until patch > 5 is applied. If this is an issue, the two patches can be merged on > apply. However, the individual tests are still runnable through the > test binary - it's only through "meson test" that there are issues, so > I'm keeping the patches separate since each is relatively complex in its > own right. > Apologies, some other things I neglected to mention in the notes above. This patchset has revealed some issues with the building and running of unit tests: * Some files/tests were not getting built, and look to have been missed years ago in the move from make to meson. Specifically those tests using external resources, e.g. the cfgfile tests, are not buildable and have not been for years. * There are quite a number of tests defined that are not registered in a test suite in meson and which cannot be run through meson test. Since the test cases were automatically tagged via script when generating this patchset, the offending test cases can now be found by grepping the sources for use of the old "REGISTER_TEST_COMMAND" macro. The fixing of these issues I consider outside the scope of this set, and won't be looking at in subsequent revisions. Hopefully #2 should be fairly easy to rectify in future, but #1 may be more complex. /Bruce