From patchwork Thu Sep 14 15:16:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 131429 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 98EF342598; Thu, 14 Sep 2023 17:17:08 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 38C7E40293; Thu, 14 Sep 2023 17:17:08 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 17C9340289 for ; Thu, 14 Sep 2023 17:17:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1694704627; x=1726240627; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=H45rtnbqsjy2COy5ZWNRolbwWLGMGckLwqAbEao54h8=; b=B58Ph3rhz08XbQxyzM0jxnMHcpdJ7BvtfKq0IxtdQjbXDdZxubpBHTA2 qanedXdlIT5M3/pgPQtEFJSCDBtsMFiqZJl0/+NlRRYutfu9e+A5cezGM 001/yufTvcDuOThBg5bXBglyVtSyITk3dyUFkX7cTYHsgohpVLHKboGHu eC8MmYQ3kk7z0eyvzSgWsk1XRDh3FrkdhtykDJ7qihMIHhN4/XSrkTefu ZmpVcnWvF4B+P9e4OxyCk+jkzlwXPYDJ144h2xWyCY85sS69Blb/uuqtx jXXvoomi54M8ejhKE6U6gQKxSDYEpzK1EgFeOnjLJZQ8F2almF36JeIX1 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10833"; a="443030202" X-IronPort-AV: E=Sophos;i="6.02,146,1688454000"; d="scan'208";a="443030202" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Sep 2023 08:16:47 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10833"; a="744578506" X-IronPort-AV: E=Sophos;i="6.02,146,1688454000"; d="scan'208";a="744578506" Received: from silpixa00401385.ir.intel.com ([10.237.214.14]) by orsmga002.jf.intel.com with ESMTP; 14 Sep 2023 08:16:45 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: aconole@redhat.com, Bruce Richardson , Thomas Monjalon Subject: [PATCH v4] app/test: add support for skipping tests Date: Thu, 14 Sep 2023 16:16:36 +0100 Message-Id: <20230914151636.278641-1-bruce.richardson@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230817105851.501947-1-bruce.richardson@intel.com> References: <20230817105851.501947-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org When called from automated tools, like meson test, it is often useful to skip tests in a test suite, without having to alter the test build. To do so, we add support for DPDK_TEST_SKIP environment variable, where one can provide a comma-separated list of tests. When the test binary is called to run one of the tests on the list via either cmdline parameter or environment variable (as done with meson test), the test will not actually be run, but will be reported skipped. Example run: $ DPDK_TEST_SKIP=dump_devargs,dump_ring meson test --suite=debug-tests ... 1/9 DPDK:debug-tests / dump_devargs SKIP 1.11s 2/9 DPDK:debug-tests / dump_log_types OK 1.06s 3/9 DPDK:debug-tests / dump_malloc_heaps OK 1.11s 4/9 DPDK:debug-tests / dump_malloc_stats OK 1.07s 5/9 DPDK:debug-tests / dump_mempool OK 1.11s 6/9 DPDK:debug-tests / dump_memzone OK 1.06s 7/9 DPDK:debug-tests / dump_physmem OK 1.13s 8/9 DPDK:debug-tests / dump_ring SKIP 1.04s 9/9 DPDK:debug-tests / dump_struct_sizes OK 1.10s Ok: 7 Expected Fail: 0 Fail: 0 Unexpected Pass: 0 Skipped: 2 Timeout: 0 Signed-off-by: Bruce Richardson Acked-by: Thomas Monjalon Acked-by: Aaron Conole --- V4: Add os shim header, to avoid issues with use of strdup on windows V3: Fix checkpatch whitespace issue V2: Check return value from rte_strsplit, just in case. --- app/test/test.c | 32 ++++++++++++++++++++++++++++++++ app/test/test.h | 1 + 2 files changed, 33 insertions(+) diff --git a/app/test/test.c b/app/test/test.c index fb073ff795..bfa9ea52e3 100644 --- a/app/test/test.c +++ b/app/test/test.c @@ -193,6 +193,25 @@ main(int argc, char **argv) if (test_count > 0) { char buf[1024]; + char *dpdk_test_skip = getenv("DPDK_TEST_SKIP"); + char *skip_tests[128] = {0}; + size_t n_skip_tests = 0; + + if (dpdk_test_skip != NULL && strlen(dpdk_test_skip) > 0) { + int split_ret; + char *dpdk_test_skip_cp = strdup(dpdk_test_skip); + if (dpdk_test_skip_cp == NULL) { + ret = -1; + goto out; + } + dpdk_test_skip = dpdk_test_skip_cp; + split_ret = rte_strsplit(dpdk_test_skip, strlen(dpdk_test_skip), + skip_tests, RTE_DIM(skip_tests), ','); + if (split_ret > 0) + n_skip_tests = split_ret; + else + free(dpdk_test_skip); + } cl = cmdline_new(main_ctx, "RTE>>", 0, 1); if (cl == NULL) { @@ -201,6 +220,15 @@ main(int argc, char **argv) } for (i = 0; i < test_count; i++) { + /* check if test is to be skipped */ + for (size_t j = 0; j < n_skip_tests; j++) { + if (strcmp(tests[i], skip_tests[j]) == 0) { + fprintf(stderr, "Skipping %s [DPDK_TEST_SKIP]\n", tests[i]); + ret = TEST_SKIPPED; + goto end_of_cmd; + } + } + snprintf(buf, sizeof(buf), "%s\n", tests[i]); if (cmdline_parse_check(cl, buf) < 0) { printf("Error: invalid test command: '%s'\n", tests[i]); @@ -211,9 +239,13 @@ main(int argc, char **argv) } else ret = last_test_result; +end_of_cmd: if (ret != 0) break; } + if (n_skip_tests > 0) + free(dpdk_test_skip); + cmdline_free(cl); goto out; } else { diff --git a/app/test/test.h b/app/test/test.h index a91ded76af..eede583cf9 100644 --- a/app/test/test.h +++ b/app/test/test.h @@ -12,6 +12,7 @@ #include #include +#include #define TEST_SUCCESS EXIT_SUCCESS #define TEST_FAILED -1