From patchwork Wed Aug 16 15:34:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 130423 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5350343081; Wed, 16 Aug 2023 17:36:13 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CAB3243274; Wed, 16 Aug 2023 17:35:42 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.43]) by mails.dpdk.org (Postfix) with ESMTP id 0B0DE43269 for ; Wed, 16 Aug 2023 17:35:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1692200141; x=1723736141; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=FLjWuoHFXCWLiqqNhZV7SA8/gq5E/j9uEWQQaginUTI=; b=D3gTphOBewvAXlNOzO+yvLq4ifgJNDLAkFyI70SrYOJQ43SfHJciDLBJ /kiiiLkKgoUkcknT15s96xZHChtjtLNohZ8S0hWq6aZ4oYOpNmfeszl7m XYzUMIqCF3sKBpK7Jwe3BV4iHyEo7KZh6NXsLJi8XKb7YOH9tbZXginVC /kCPDLbI3qMrqFeuTSAUHlccO+WhcilJfNfrv68x4cFxSvOH6WbpgpwlX 02Hu6UPPrPxnBimJglDgzDDpTmepyt0nQhF5n6uQKmqshi3vLNCOY8gFP WMy6aFxqHPzbU4kc3mEqdZqJlWXh3kIDixR3Trq6lOlDHaOtuViuAg+2X g==; X-IronPort-AV: E=McAfee;i="6600,9927,10803"; a="458916004" X-IronPort-AV: E=Sophos;i="6.01,177,1684825200"; d="scan'208";a="458916004" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Aug 2023 08:35:40 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10803"; a="857856275" X-IronPort-AV: E=Sophos;i="6.01,177,1684825200"; d="scan'208";a="857856275" Received: from silpixa00401385.ir.intel.com ([10.237.214.14]) by orsmga004.jf.intel.com with ESMTP; 16 Aug 2023 08:35:38 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: david.marchand@redhat.com, probb@iol.unh.edu, mb@smartsharesystems.com, Bruce Richardson Subject: [PATCH v6 06/11] app/test: define unit tests suites based on test macros Date: Wed, 16 Aug 2023 16:34:34 +0100 Message-Id: <20230816153439.551501-7-bruce.richardson@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230816153439.551501-1-bruce.richardson@intel.com> References: <20230721115125.55137-1-bruce.richardson@intel.com> <20230816153439.551501-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Rather than having the test suites listed out in the meson.build files and having to have them enabled/disabled selectively based on what libs are being built, pull the tests to run from the source files which were added to the build. Most test suites require no additional info other than the list of test names in the suite. However the fast-test are special that they have additional parameters associated with them. This requires some additional work in the test extraction script and in the meson.build file for processing the output. Signed-off-by: Bruce Richardson Acked-by: Morten Brørup --- MAINTAINERS | 3 +- app/meson.build | 7 +- app/test/suites/meson.build | 74 +++++++++++++++++++ buildtools/get-test-suites.py | 33 +++++++++ .../has-hugepages.py | 0 buildtools/meson.build | 2 + 6 files changed, 117 insertions(+), 2 deletions(-) create mode 100644 app/test/suites/meson.build create mode 100644 buildtools/get-test-suites.py rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%) diff --git a/MAINTAINERS b/MAINTAINERS index 8c3f2c993f..c4ce661b6e 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1731,7 +1731,6 @@ Test Applications Unit tests framework F: app/test/commands.c -F: app/test/has_hugepage.py F: app/test/packet_burst_generator.c F: app/test/packet_burst_generator.h F: app/test/process.h @@ -1742,6 +1741,8 @@ F: app/test/test_pmd_perf.c F: app/test/test_resource.c F: app/test/virtual_pmd.c F: app/test/virtual_pmd.h +F: buildtools/has-hugepages.py +F: buildtools/get-test-suites.py Sample packet helper functions for unit test M: Reshma Pattan diff --git a/app/meson.build b/app/meson.build index 0d8b618e7f..c14dc80892 100644 --- a/app/meson.build +++ b/app/meson.build @@ -101,7 +101,7 @@ foreach app:apps link_libs = dpdk_static_libraries + dpdk_drivers endif - executable('dpdk-' + name, + exec = executable('dpdk-' + name, sources, c_args: cflags, link_args: ldflags, @@ -110,4 +110,9 @@ foreach app:apps include_directories: includes, install_rpath: join_paths(get_option('prefix'), driver_install_path), install: true) + if name == 'test' + dpdk_test = exec + autotest_sources = sources + subdir('test/suites') # define the pre-canned test suites + endif endforeach diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build new file mode 100644 index 0000000000..5cdf7c200c --- /dev/null +++ b/app/test/suites/meson.build @@ -0,0 +1,74 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2023 Intel Corporation + +# some perf tests (eg: memcpy perf autotest) take a very long +# to complete, so timeout to 10 minutes +timeout_seconds = 600 +timeout_seconds_fast = 10 + +test_no_huge_args = ['--no-huge', '-m', '2048'] +has_hugepage = run_command(has_hugepages_cmd, check: true).stdout().strip() != '0' +message('hugepage availability: @0@'.format(has_hugepage)) + +# process source files to determine the different unit test suites +# - fast_tests +# - perf_tests +# - driver_tests +test_suites = run_command(get_test_suites_cmd, autotest_sources, + check: true).stdout().strip().split() +foreach suite:test_suites + # simple cases - tests without parameters or special handling + suite = suite.split('=') + suite_name = suite[0] + suite_tests = suite[1].split(',') + if suite_name != 'fast-tests' + # simple cases - tests without parameters or special handling + foreach t: suite_tests + test(t, dpdk_test, + env: ['DPDK_TEST=' + t], + timeout: timeout_seconds, + is_parallel: false, + suite: suite_name) + endforeach + else + # special fast-test handling here + foreach t: suite_tests + params = t.split(':') + test_name = params[0] + nohuge = params[1] == 'true' + asan = params[2] == 'true' + + test_args = [] + if nohuge + test_args += test_no_huge_args + elif not has_hugepage + continue #skip this tests + endif + if not asan and (get_option('b_sanitize') == 'address' + or get_option('b_sanitize') == 'address,undefined') + continue # skip this test + endif + + if get_option('default_library') == 'shared' + test_args += ['-d', dpdk_drivers_build_dir] + endif + + test(test_name, dpdk_test, + args : test_args, + env: ['DPDK_TEST=' + test_name], + timeout : timeout_seconds_fast, + is_parallel : false, + suite : 'fast-tests') + if not is_windows and test_name == 'trace_autotest' + test_args += ['--trace=.*'] + test_args += ['--trace-dir=@0@'.format(meson.current_build_dir())] + test(test_name + '_with_traces', dpdk_test, + args : test_args, + env: ['DPDK_TEST=' + test_name], + timeout : timeout_seconds_fast, + is_parallel : false, + suite : 'fast-tests') + endif + endforeach + endif +endforeach diff --git a/buildtools/get-test-suites.py b/buildtools/get-test-suites.py new file mode 100644 index 0000000000..95a9cad4c8 --- /dev/null +++ b/buildtools/get-test-suites.py @@ -0,0 +1,33 @@ +#! /usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2023 Intel Corporation + +import sys +import re + +input_list = sys.argv[1:] +test_def_regex = re.compile("REGISTER_([A-Z]+)_TEST\s*\(\s*([a-z0-9_]+)") +test_suites = {} + +def get_fast_test_params(test_name, ln): + "Extract the extra fast-test parameters from the line" + #print(f"ln: {ln.rstrip()}, test_name: {test_name}, split: {ln.split(test_name, 1)}") + (_, rest_of_line) = ln.split(test_name, 1) + (_, nohuge, asan, _func) = rest_of_line.split(',', 3) + return f":{nohuge.strip().lower()}:{asan.strip().lower()}" + +for fname in input_list: + with open(fname) as f: + contents = [ln for ln in f.readlines() if test_def_regex.match(ln.strip())] + for ln in contents: + (test_suite, test_name) = test_def_regex.match(ln).group(1, 2) + suite_name = f"{test_suite.lower()}-tests" + if suite_name in test_suites: + test_suites[suite_name].append(test_name) + else: + test_suites[suite_name] = [test_name] + if suite_name == "fast-tests": + test_suites["fast-tests"][-1] += get_fast_test_params(test_name, ln) + +for suite in test_suites.keys(): + print(f"{suite}={','.join(test_suites[suite])}") diff --git a/app/test/has_hugepage.py b/buildtools/has-hugepages.py similarity index 100% rename from app/test/has_hugepage.py rename to buildtools/has-hugepages.py diff --git a/buildtools/meson.build b/buildtools/meson.build index e1c600e40f..ac5e4dcf08 100644 --- a/buildtools/meson.build +++ b/buildtools/meson.build @@ -18,6 +18,8 @@ map_to_win_cmd = py3 + files('map_to_win.py') sphinx_wrapper = py3 + files('call-sphinx-build.py') get_cpu_count_cmd = py3 + files('get-cpu-count.py') get_numa_count_cmd = py3 + files('get-numa-count.py') +get_test_suites_cmd = py3 + files('get-test-suites.py') +has_hugepages_cmd = py3 + files('has-hugepages.py') binutils_avx512_check = (py3 + files('binutils-avx512-check.py') + [objdump] + cc.cmd_array())