[V1] framework/*: fix bug of excel/json report is empty
Commit Message
In execution.cfg, assume that we defined multiple section
of configs that they can have different driver,crbs,target
and suite list. If one target seted failure(e.g.: driver
loaded failure ), then the related dut and target will be
recorded in a failure target collect(like a blacklist),
and the excel/json test result report about the dut and
target that be in blacklist will be empty, even if other
section that have the same dut and target executed normally.
so, if one dut and target related section executed normally,
we can try to removing it over from the blacklist, then the
excel/json report will generated normally.
last, if the lastted section's target seted failure, and
framework try to generated the report again in the end
point of the process. the excel/json report also be empty, the
empty report will replace that already normally generated
report, and the end point save operation actually is
meaningless because every suite have saved result to report.
Signed-off-by: Jun Dong <junx.dong@intel.com>
---
framework/dts.py | 8 +++++---
framework/test_result.py | 15 +++++++++++++++
2 files changed, 20 insertions(+), 3 deletions(-)
Comments
On Tue, 26 Apr 2022 11:07:11 +0800, Jun Dong <junx.dong@intel.com> wrote:
> In execution.cfg, assume that we defined multiple section
> of configs that they can have different driver,crbs,target
> and suite list. If one target seted failure(e.g.: driver
> loaded failure ), then the related dut and target will be
> recorded in a failure target collect(like a blacklist),
> and the excel/json test result report about the dut and
> target that be in blacklist will be empty, even if other
> section that have the same dut and target executed normally.
>
> so, if one dut and target related section executed normally,
> we can try to removing it over from the blacklist, then the
> excel/json report will generated normally.
>
> last, if the lastted section's target seted failure, and
> framework try to generated the report again in the end
> point of the process. the excel/json report also be empty, the
> empty report will replace that already normally generated
> report, and the end point save operation actually is
> meaningless because every suite have saved result to report.
>
> Signed-off-by: Jun Dong <junx.dong@intel.com>
Acked-by: Lijuan Tu <lijuan.tu@intel.com>
Applied, thanks
@@ -71,7 +71,7 @@ from .utils import (
imp.reload(sys)
requested_tests = None
-result = None
+result: Result = None
excel_report = None
json_report = None
stats_report = None
@@ -407,6 +407,8 @@ def dts_run_prerequisties(duts, tester, pkgName, patch, dts_commands, serializer
serializer.discard_cache()
settings.report_error("DUT_SETUP_ERR")
return False
+ else:
+ result.remove_failed_dut(duts[0])
def dts_run_target(duts, tester, targets, test_suites, subtitle):
@@ -435,6 +437,8 @@ def dts_run_target(duts, tester, targets, test_suites, subtitle):
log_handler.error(" !!! DEBUG IT: " + traceback.format_exc())
result.add_failed_target(result.dut, target, str(ex))
continue
+ else:
+ result.remove_failed_target(result.dut, target)
dts_run_suite(duts, tester, test_suites, target, subtitle)
@@ -675,8 +679,6 @@ def run_all(
dts_crbs_exit(duts, tester)
- save_all_results()
-
def show_speedup_options_messages(read_cache, skip_setup):
if read_cache:
@@ -386,6 +386,13 @@ class Result(object):
"""
self.__failed_duts[dut] = msg
+ def remove_failed_dut(self, dut):
+ """
+ Remove the given DUT from failed duts collection
+ """
+ if dut in self.__failed_duts:
+ self.__failed_duts.pop(dut)
+
def is_dut_failed(self, dut):
"""
True if the given DUT was marked as failing
@@ -404,6 +411,14 @@ class Result(object):
"""
self.__failed_targets[dut + target] = msg
+ def remove_failed_target(self, dut, target):
+ """
+ Remove the given DUT, target from failed targets collection
+ """
+ key_word = dut + target
+ if key_word in self.__failed_targets:
+ self.__failed_targets.pop(key_word)
+
def is_target_failed(self, dut, target):
"""
True if the given DUT,target were marked as failing