From patchwork Mon Dec 4 10:24:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Juraj_Linke=C5=A1?= X-Patchwork-Id: 134809 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1E2DE4366A; Mon, 4 Dec 2023 11:27:08 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CEF7242D9A; Mon, 4 Dec 2023 11:25:01 +0100 (CET) Received: from mail-wm1-f47.google.com (mail-wm1-f47.google.com [209.85.128.47]) by mails.dpdk.org (Postfix) with ESMTP id 8F03E42D28 for ; Mon, 4 Dec 2023 11:24:52 +0100 (CET) Received: by mail-wm1-f47.google.com with SMTP id 5b1f17b1804b1-40bd5eaa66cso26118765e9.2 for ; Mon, 04 Dec 2023 02:24:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pantheon.tech; s=google; t=1701685492; x=1702290292; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Gu+4JrC2IuPtEiL7DfKPBV14u/4QgUSAAuxdak3zylg=; b=B6sIv+F8AsHCyzuuxQXPxP2tq96LUoMjrYDInwfWfmKZAyR27THgdhxOozCAWB7XDZ fMoQOp3nC4mXAC9iS8fxfFHJZNTgkvrTbSnTNRQg9iwv8OIaS58ISA6/+kDxvxF6EdnT GM34ok0AmZJIxzfAxh38HGfQf7n/W/yMb7Scg1eB90B3bfVtNyJAV3y4hgaPN+bqS9PH +m5eIk47V1HGAQdFZqExPxobCOI0urwU7TwoHMmvQDP7g4VM2ye0G5Nrr7dWzoRWQLHQ 8ZjIGRVe/UW9F9ZT5i0IuQkjhYy/TJK9nS29Jj6+EcgssYhlpui1ILo0SgDrVp8BUxpb nEQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701685492; x=1702290292; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Gu+4JrC2IuPtEiL7DfKPBV14u/4QgUSAAuxdak3zylg=; b=xMecA+UgM8tlt7Wlp5qryziz3hWCyw3e55dcUBnXi6LPNvhIJ/owzF8EFmn4jPNgid cpZi5nS73S4SPMmhJlOiSt+f9t9/tBJxmD3sNg2Y7hZvugf0PFhNpLhSgWF8BEW5AiDt 5uoROdxBAw4QCIE5TKPjqiXlHEeb0BSKyuws8/FvpGPWH/TqziHGg9CAqYdSrooxxfTy p4Kr0m2/nzK4vqp6qcp1HNnfv31Vb6JEQX3Uj7ibV9c4MTtN7/PJQpqvUbJWr6SJgDBG uIEuYfpQOZJNhX+9C+cinefUokhtbiUXNapg8d8iT/43ppDto4oyOKwix1tYjktYIn0t ny3A== X-Gm-Message-State: AOJu0Yy9eGUHQsLntD1MYoTHYXaBbj15PVc9g65qsswwXy5XnI0G2hHS F53SdTTqc2dAGyDtJoaiwU013A== X-Google-Smtp-Source: AGHT+IFNfuafLMTrnzuZhtSHAi1wJfByk8cu/8OswzuPQJa99i4NBxxd370wDKhG9B+5xT78ocdBhQ== X-Received: by 2002:a05:600c:11d2:b0:409:19a0:d247 with SMTP id b18-20020a05600c11d200b0040919a0d247mr2725139wmi.18.1701685492200; Mon, 04 Dec 2023 02:24:52 -0800 (PST) Received: from jlinkes-PT-Latitude-5530.pantheon.local ([81.89.53.154]) by smtp.gmail.com with ESMTPSA id m28-20020a05600c3b1c00b0040b2b38a1fasm14255415wms.4.2023.12.04.02.24.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Dec 2023 02:24:51 -0800 (PST) From: =?utf-8?q?Juraj_Linke=C5=A1?= To: thomas@monjalon.net, Honnappa.Nagarahalli@arm.com, jspewock@iol.unh.edu, probb@iol.unh.edu, paul.szczepanek@arm.com, yoan.picchi@foss.arm.com, Luca.Vizzarro@arm.com Cc: dev@dpdk.org, =?utf-8?q?Juraj_Linke=C5=A1?= Subject: [PATCH v9 21/21] dts: test suites docstring update Date: Mon, 4 Dec 2023 11:24:29 +0100 Message-Id: <20231204102429.106709-22-juraj.linkes@pantheon.tech> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231204102429.106709-1-juraj.linkes@pantheon.tech> References: <20231123151344.162812-1-juraj.linkes@pantheon.tech> <20231204102429.106709-1-juraj.linkes@pantheon.tech> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Format according to the Google format and PEP257, with slight deviations. Signed-off-by: Juraj Linkeš Reviewed-by: Jeremy Spewock --- dts/tests/TestSuite_hello_world.py | 16 +++++--- dts/tests/TestSuite_os_udp.py | 20 ++++++---- dts/tests/TestSuite_smoke_tests.py | 61 ++++++++++++++++++++++++------ 3 files changed, 72 insertions(+), 25 deletions(-) diff --git a/dts/tests/TestSuite_hello_world.py b/dts/tests/TestSuite_hello_world.py index 768ba1cfa8..fd7ff1534d 100644 --- a/dts/tests/TestSuite_hello_world.py +++ b/dts/tests/TestSuite_hello_world.py @@ -1,7 +1,8 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2010-2014 Intel Corporation -""" +"""The DPDK hello world app test suite. + Run the helloworld example app and verify it prints a message for each used core. No other EAL parameters apart from cores are used. """ @@ -15,22 +16,25 @@ class TestHelloWorld(TestSuite): + """DPDK hello world app test suite.""" + def set_up_suite(self) -> None: - """ + """Set up the test suite. + Setup: Build the app we're about to test - helloworld. """ self.app_helloworld_path = self.sut_node.build_dpdk_app("helloworld") def test_hello_world_single_core(self) -> None: - """ + """Single core test case. + Steps: Run the helloworld app on the first usable logical core. Verify: The app prints a message from the used core: "hello from core " """ - # get the first usable core lcore_amount = LogicalCoreCount(1, 1, 1) lcores = LogicalCoreCountFilter(self.sut_node.lcores, lcore_amount).filter() @@ -42,14 +46,14 @@ def test_hello_world_single_core(self) -> None: ) def test_hello_world_all_cores(self) -> None: - """ + """All cores test case. + Steps: Run the helloworld app on all usable logical cores. Verify: The app prints a message from all used cores: "hello from core " """ - # get the maximum logical core number eal_para = self.sut_node.create_eal_parameters( lcore_filter_specifier=LogicalCoreList(self.sut_node.lcores) diff --git a/dts/tests/TestSuite_os_udp.py b/dts/tests/TestSuite_os_udp.py index bf6b93deb5..2cf29d37bb 100644 --- a/dts/tests/TestSuite_os_udp.py +++ b/dts/tests/TestSuite_os_udp.py @@ -1,7 +1,8 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2023 PANTHEON.tech s.r.o. -""" +"""Basic IPv4 OS routing test suite. + Configure SUT node to route traffic from if1 to if2. Send a packet to the SUT node, verify it comes back on the second port on the TG node. """ @@ -13,24 +14,26 @@ class TestOSUdp(TestSuite): + """IPv4 UDP OS routing test suite.""" + def set_up_suite(self) -> None: - """ + """Set up the test suite. + Setup: - Configure SUT ports and SUT to route traffic from if1 to if2. + Bind the SUT ports to the OS driver, configure the ports and configure the SUT + to route traffic from if1 to if2. """ - - # This test uses kernel drivers self.sut_node.bind_ports_to_driver(for_dpdk=False) self.configure_testbed_ipv4() def test_os_udp(self) -> None: - """ + """Basic UDP IPv4 traffic test case. + Steps: Send a UDP packet. Verify: The packet with proper addresses arrives at the other TG port. """ - packet = Ether() / IP() / UDP() received_packets = self.send_packet_and_capture(packet) @@ -40,7 +43,8 @@ def test_os_udp(self) -> None: self.verify_packets(expected_packet, received_packets) def tear_down_suite(self) -> None: - """ + """Tear down the test suite. + Teardown: Remove the SUT port configuration configured in setup. """ diff --git a/dts/tests/TestSuite_smoke_tests.py b/dts/tests/TestSuite_smoke_tests.py index 8958f58dac..5e2bac14bd 100644 --- a/dts/tests/TestSuite_smoke_tests.py +++ b/dts/tests/TestSuite_smoke_tests.py @@ -1,6 +1,17 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2023 University of New Hampshire +"""Smoke test suite. + +Smoke tests are a class of tests which are used for validating a minimal set of important features. +These are the most important features without which (or when they're faulty) the software wouldn't +work properly. Thus, if any failure occurs while testing these features, +there isn't that much of a reason to continue testing, as the software is fundamentally broken. + +These tests don't have to include only DPDK tests, as the reason for failures could be +in the infrastructure (a faulty link between NICs or a misconfiguration). +""" + import re from framework.config import PortConfig @@ -11,23 +22,39 @@ class SmokeTests(TestSuite): + """DPDK and infrastructure smoke test suite. + + The test cases validate the most basic DPDK functionality needed for all other test suites. + The infrastructure also needs to be tested, as that is also used by all other test suites. + + Attributes: + is_blocking: This test suite will block the execution of all other test suites + in the build target after it. + nics_in_node: The NICs present on the SUT node. + """ + is_blocking = True # dicts in this list are expected to have two keys: # "pci_address" and "current_driver" nics_in_node: list[PortConfig] = [] def set_up_suite(self) -> None: - """ + """Set up the test suite. + Setup: - Set the build directory path and generate a list of NICs in the SUT node. + Set the build directory path and a list of NICs in the SUT node. """ self.dpdk_build_dir_path = self.sut_node.remote_dpdk_build_dir self.nics_in_node = self.sut_node.config.ports def test_unit_tests(self) -> None: - """ + """DPDK meson ``fast-tests`` unit tests. + + Test that all unit test from the ``fast-tests`` suite pass. + The suite is a subset with only the most basic tests. + Test: - Run the fast-test unit-test suite through meson. + Run the ``fast-tests`` unit test suite through meson. """ self.sut_node.main_session.send_command( f"meson test -C {self.dpdk_build_dir_path} --suite fast-tests -t 60", @@ -37,9 +64,14 @@ def test_unit_tests(self) -> None: ) def test_driver_tests(self) -> None: - """ + """DPDK meson ``driver-tests`` unit tests. + + Test that all unit test from the ``driver-tests`` suite pass. + The suite is a subset with driver tests. This suite may be run with virtual devices + configured in the test run configuration. + Test: - Run the driver-test unit-test suite through meson. + Run the ``driver-tests`` unit test suite through meson. """ vdev_args = "" for dev in self.sut_node.virtual_devices: @@ -60,9 +92,12 @@ def test_driver_tests(self) -> None: ) def test_devices_listed_in_testpmd(self) -> None: - """ + """Testpmd device discovery. + + Test that the devices configured in the test run configuration are found in testpmd. + Test: - Uses testpmd driver to verify that devices have been found by testpmd. + List all devices found in testpmd and verify the configured devices are among them. """ testpmd_driver = self.sut_node.create_interactive_shell(TestPmdShell, privileged=True) dev_list = [str(x) for x in testpmd_driver.get_devices()] @@ -74,10 +109,14 @@ def test_devices_listed_in_testpmd(self) -> None: ) def test_device_bound_to_driver(self) -> None: - """ + """Device driver in OS. + + Test that the devices configured in the test run configuration are bound to + the proper driver. + Test: - Ensure that all drivers listed in the config are bound to the correct - driver. + List all devices with the ``dpdk-devbind.py`` script and verify that + the configured devices are bound to the proper driver. """ path_to_devbind = self.sut_node.path_to_devbind_script