diff mbox series

[v3,7/7] dts: run all test suites by default

Message ID 20250115141809.3898708-8-luca.vizzarro@arm.com (mailing list archive)
State Superseded
Delegated to: Paul Szczepanek
Headers
Series dts: refactor configuration |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation warning apply issues

Commit Message

Luca Vizzarro Jan. 15, 2025, 2:18 p.m. UTC 
The configuration requires the user to explicitly set the requested test
suites in the files. Sometimes we want to run all the test suites and
don't want to manually specify all of them. It is therefore reasonable
to change the default behaviour to automatically run all the available
test suites if none are specified.

Bugzilla ID: 1360

Signed-off-by: Luca Vizzarro <luca.vizzarro@arm.com>
Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>
---
 dts/framework/config/test_run.py | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

Comments

Dean Marx Jan. 16, 2025, 9:01 p.m. UTC | #1 
On Wed, Jan 15, 2025 at 9:19 AM Luca Vizzarro <luca.vizzarro@arm.com> wrote:
>
> The configuration requires the user to explicitly set the requested test
> suites in the files. Sometimes we want to run all the test suites and
> don't want to manually specify all of them. It is therefore reasonable
> to change the default behaviour to automatically run all the available
> test suites if none are specified.
>
> Bugzilla ID: 1360
>
> Signed-off-by: Luca Vizzarro <luca.vizzarro@arm.com>
> Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>

Looks good, would it be worth mentioning in the comment above the
test_suites field that removing it will default to running all suites?
Otherwise I'm not sure how a user would know this without digging
through the docs or framework.

i.e.: # the following test suites will be run in their entirety. If
removed, all test suites will be run.
test_suites: etc.

Reviewed-by: Dean Marx <dmarx@iol.unh.edu>
Luca Vizzarro Jan. 20, 2025, 10 a.m. UTC | #2 
On 16/01/2025 21:01, Dean Marx wrote:
> Looks good, would it be worth mentioning in the comment above the
> test_suites field that removing it will default to running all suites?
> Otherwise I'm not sure how a user would know this without digging
> through the docs or framework.
> 
> i.e.: # the following test suites will be run in their entirety. If
> removed, all test suites will be run.
> test_suites: etc.

Good point, thanks. Will add it.

And thank you for the reviews!
diff mbox series

Patch

diff --git a/dts/framework/config/test_run.py b/dts/framework/config/test_run.py
index f08e034e25..6ea05ceeba 100644
--- a/dts/framework/config/test_run.py
+++ b/dts/framework/config/test_run.py
@@ -259,6 +259,20 @@  def validate_names(self) -> Self:
         return self
 
 
+def fetch_all_test_suites() -> list[TestSuiteConfig]:
+    """Returns all the available test suites as configuration objects.
+
+    This function does not include the smoke tests.
+    """
+    from framework.test_suite import AVAILABLE_TEST_SUITES
+
+    return [
+        TestSuiteConfig(test_suite=test_suite.name)
+        for test_suite in AVAILABLE_TEST_SUITES
+        if test_suite.name != "smoke_tests"
+    ]
+
+
 class TestRunConfiguration(FrozenModel):
     """The configuration of a test run.
 
@@ -275,7 +289,7 @@  class TestRunConfiguration(FrozenModel):
     #: Whether to skip smoke tests.
     skip_smoke_tests: bool = False
     #: The names of test suites and/or test cases to execute.
-    test_suites: list[TestSuiteConfig] = Field(min_length=1)
+    test_suites: list[TestSuiteConfig] = Field(min_length=1, default_factory=fetch_all_test_suites)
     #: The SUT node name to use in this test run.
     system_under_test_node: str
     #: The TG node name to use in this test run.