From patchwork Mon Nov 13 15:05:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 134162 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3E0BE4331C; Mon, 13 Nov 2023 16:05:44 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C4EA2402B0; Mon, 13 Nov 2023 16:05:43 +0100 (CET) Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.126]) by mails.dpdk.org (Postfix) with ESMTP id F11A0402AB for ; Mon, 13 Nov 2023 16:05:41 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1699887942; x=1731423942; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=JoVG8y5KvkUl6TxV6r68z88UHupwedfWJqlaIDQaAso=; b=f4SnBYz7itXd2xo1UJz3rQVK+a+2ZvV0mfZLqezymDVBDSpABVvQO5Nx t8YOUENdQwv+he7DfmSfVdRECgQ7p+7YtAK6dynlwA4WwNBcl5dtR/dsR tk6uqYg/z2+sy2MCwExWIwlGRaLYskeZ41b6ELtS9JoVa7cc9s29JUPM2 mMRAFfr81kGRhWBf70oHZEyPCA2hmzCPfJBSjVjJJLgePfwA3Fjkb2m29 ThQoNwVurDrSGiWEKtP7KWK840zfi9X6JPBBlJGnn8WRMXcMW/e/o66Ce PFEQ6WR7Xh1WdKppSRGyCOX4lNvh9y5+NiSUrLK1YYORJUzZbu7yOYTrg w==; X-IronPort-AV: E=McAfee;i="6600,9927,10893"; a="375475707" X-IronPort-AV: E=Sophos;i="6.03,299,1694761200"; d="scan'208";a="375475707" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Nov 2023 07:05:40 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10893"; a="767946847" X-IronPort-AV: E=Sophos;i="6.03,299,1694761200"; d="scan'208";a="767946847" Received: from unknown (HELO silpixa00401385.ir.intel.com) ([10.237.214.22]) by fmsmga007.fm.intel.com with ESMTP; 13 Nov 2023 07:05:39 -0800 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson Subject: [PATCH] app/test: don't count skipped tests as executed Date: Mon, 13 Nov 2023 15:05:33 +0000 Message-Id: <20231113150533.249808-1-bruce.richardson@intel.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The logic around skipped tests is a little confusing in the unit test runner. * Any explicitly disabled tests are counted as skipped but not executed. * Any tests that return TEST_SKIPPED are counted as both skipped and executed, using the same statistics counters. This makes the stats very strange and hard to correlate, since the totals don't add up. One would expect that SKIPPED + EXECUTED + UNSUPPORTED == TOTAL, and that PASSED + FAILED == EXECUTED. To achieve this, mark any tests returning TEST_SKIPPED, or ENOTSUP as not having executed. Signed-off-by: Bruce Richardson Acked-by: Akhil Goyal Acked-by: Ciara Power Acked-by: Tyler Retzlaff --- app/test/test.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/app/test/test.c b/app/test/test.c index bfa9ea52e3..7b882a59de 100644 --- a/app/test/test.c +++ b/app/test/test.c @@ -375,11 +375,13 @@ unit_test_suite_runner(struct unit_test_suite *suite) if (test_success == TEST_SUCCESS) suite->succeeded++; - else if (test_success == TEST_SKIPPED) + else if (test_success == TEST_SKIPPED) { suite->skipped++; - else if (test_success == -ENOTSUP) + suite->executed--; + } else if (test_success == -ENOTSUP) { suite->unsupported++; - else + suite->executed--; + } else suite->failed++; } else if (test_success == -ENOTSUP) { suite->unsupported++;