From patchwork Tue Apr 26 03:07:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jun Dong X-Patchwork-Id: 110242 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 98555A0093; Tue, 26 Apr 2022 05:07:22 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6954D406A2; Tue, 26 Apr 2022 05:07:22 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id 9358040691 for ; Tue, 26 Apr 2022 05:07:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1650942440; x=1682478440; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=FxiEgGpuuwmc2q0KSYyPeTEwfueynLr5qT2nS2E48KM=; b=JoAz4vl+FHpyzDSnypY2k/2r6NuyZtbYP8UIZ9db60TYsO2HdXxb2OBL dUBWsmT6ahPC1LnP/10DzbOiuSNMPgfALzHe9gu3Z3sXlE0kj5qVelEce f9hpcKYkh8hLqqql0x0ukyJoOjRujiDJ10LBoBrNjZH5Y9NkLpDl/ls6R uA1mb/bOP3Gy9D0TBxNJXGoD7nkENXsF4r2kwuRt5h/ukkQDJJi1zIWpc ex4fnnEpNKldv6rdD3GzgkOxujf++reyXJcg3FRO0LOAOxdOLUOQmFWkL 0ZdYeHN/LtW0PEt1DooZydOu1t1OfINlCuRoZ2pvsNUD9V0t1555Srt15 Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10328"; a="264953044" X-IronPort-AV: E=Sophos;i="5.90,290,1643702400"; d="scan'208";a="264953044" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Apr 2022 20:07:19 -0700 X-IronPort-AV: E=Sophos;i="5.90,290,1643702400"; d="scan'208";a="616792935" Received: from shwdenpg197.ccr.corp.intel.com ([10.253.109.70]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Apr 2022 20:07:17 -0700 From: Jun Dong To: dts@dpdk.org Cc: lijuan.tu@intel.com, qingx.sun@intel.com, junx.dong@intel.com Subject: [dts] [V1] framework/*: fix bug of excel/json report is empty Date: Tue, 26 Apr 2022 11:07:11 +0800 Message-Id: <20220426030711.6754-1-junx.dong@intel.com> X-Mailer: git-send-email 2.33.1.windows.1 MIME-Version: 1.0 X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org In execution.cfg, assume that we defined multiple section of configs that they can have different driver,crbs,target and suite list. If one target seted failure(e.g.: driver loaded failure ), then the related dut and target will be recorded in a failure target collect(like a blacklist), and the excel/json test result report about the dut and target that be in blacklist will be empty, even if other section that have the same dut and target executed normally. so, if one dut and target related section executed normally, we can try to removing it over from the blacklist, then the excel/json report will generated normally. last, if the lastted section's target seted failure, and framework try to generated the report again in the end point of the process. the excel/json report also be empty, the empty report will replace that already normally generated report, and the end point save operation actually is meaningless because every suite have saved result to report. Signed-off-by: Jun Dong Acked-by: Lijuan Tu --- framework/dts.py | 8 +++++--- framework/test_result.py | 15 +++++++++++++++ 2 files changed, 20 insertions(+), 3 deletions(-) diff --git a/framework/dts.py b/framework/dts.py index d31beccb..4003d5c3 100644 --- a/framework/dts.py +++ b/framework/dts.py @@ -71,7 +71,7 @@ from .utils import ( imp.reload(sys) requested_tests = None -result = None +result: Result = None excel_report = None json_report = None stats_report = None @@ -407,6 +407,8 @@ def dts_run_prerequisties(duts, tester, pkgName, patch, dts_commands, serializer serializer.discard_cache() settings.report_error("DUT_SETUP_ERR") return False + else: + result.remove_failed_dut(duts[0]) def dts_run_target(duts, tester, targets, test_suites, subtitle): @@ -435,6 +437,8 @@ def dts_run_target(duts, tester, targets, test_suites, subtitle): log_handler.error(" !!! DEBUG IT: " + traceback.format_exc()) result.add_failed_target(result.dut, target, str(ex)) continue + else: + result.remove_failed_target(result.dut, target) dts_run_suite(duts, tester, test_suites, target, subtitle) @@ -675,8 +679,6 @@ def run_all( dts_crbs_exit(duts, tester) - save_all_results() - def show_speedup_options_messages(read_cache, skip_setup): if read_cache: diff --git a/framework/test_result.py b/framework/test_result.py index abf8edb9..2d7cda4b 100644 --- a/framework/test_result.py +++ b/framework/test_result.py @@ -386,6 +386,13 @@ class Result(object): """ self.__failed_duts[dut] = msg + def remove_failed_dut(self, dut): + """ + Remove the given DUT from failed duts collection + """ + if dut in self.__failed_duts: + self.__failed_duts.pop(dut) + def is_dut_failed(self, dut): """ True if the given DUT was marked as failing @@ -404,6 +411,14 @@ class Result(object): """ self.__failed_targets[dut + target] = msg + def remove_failed_target(self, dut, target): + """ + Remove the given DUT, target from failed targets collection + """ + key_word = dut + target + if key_word in self.__failed_targets: + self.__failed_targets.pop(key_word) + def is_target_failed(self, dut, target): """ True if the given DUT,target were marked as failing