From patchwork Fri May 12 19:25:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeremy Spewock X-Patchwork-Id: 126841 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7C43642AE8; Fri, 12 May 2023 21:27:56 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6E843406BA; Fri, 12 May 2023 21:27:56 +0200 (CEST) Received: from mail-pl1-f226.google.com (mail-pl1-f226.google.com [209.85.214.226]) by mails.dpdk.org (Postfix) with ESMTP id 32DDF406B3 for ; Fri, 12 May 2023 21:27:55 +0200 (CEST) Received: by mail-pl1-f226.google.com with SMTP id d9443c01a7336-1ab05018381so95667585ad.2 for ; Fri, 12 May 2023 12:27:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1683919674; x=1686511674; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eFi9qByGtlFnJiQN83k5R0gUjmkvYgFEcabbCTe2qB8=; b=evK+wgQ/PB1rMXTHfCyQp/fDPaxPK/mZx48IDTzg9hZR6QRLwzZSEpevwUv+HmTTTk SHRfcgIFOT3Uw7w1ck6KMzMvbu6652sNh+n94sjlj5ayC2VsCXhzdMG8N60HsxKuO4Wn Zpb2vqcRRoKQYQzJFff9gjIb+xjYOgq5K183Q= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683919674; x=1686511674; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eFi9qByGtlFnJiQN83k5R0gUjmkvYgFEcabbCTe2qB8=; b=YrRJAtUB2zxVO6fUTjfIV7cxCif6eJ7Tz7pT4hxVUQXyE08WB1c/dmSGNNidcq/riV 5FWL6lcXUdA3av+VtAy3vRDn5+Pv5ff+Kb2+/5RPAwRD8HFmb1OMHjo1CNsVy/1LFrF7 4LXBWBa1ic5xanvp/S4thboNrKVlXvH/eGdZXYzN1n15oix/ZT7zjbhdg0XAjJLEmkdi 6e0fFg3Ui3rKPUEHCeQrGt5cbfVsufIFqtVHrh9GZxChFNw9hR8rIH6ge9CUe6tf/nUH 2U6CYpJFxoPVyGjU9j+/Im9O6W+3a/ZHUFChiEzx3nX5x0LvVzRwV+98LDe/Vx7MpOQx Ga4Q== X-Gm-Message-State: AC+VfDx0aNy+1OkHKIwhLBLlMlTyUl3vnqFxydwu23eRj2JhiSChis6m JNgnAG9Qo5ORBgZ/XvgMursJuHU4LeFUVrJUa5AldRrvnHq+ogYLYnEtPxPxoOs216CDhCI5e7k 6wRRO1Q9CrTIG9ScgMbsuednSe4pig8/oeOOcdNOcfdEkQxDUCRHFIE2kHhbAsIO9XmP1/omLXX IqD2cL0A== X-Google-Smtp-Source: ACHHUZ5fPK/VH8ZLeYdYjvijUXeex19+reKLltAbZQP+pTaAkYbP2oHs21k33oFHpcjzeqiAZxv1pPJ+Jw5G X-Received: by 2002:a17:903:22c7:b0:1a6:71b1:a0b9 with SMTP id y7-20020a17090322c700b001a671b1a0b9mr32345308plg.47.1683919674260; Fri, 12 May 2023 12:27:54 -0700 (PDT) Received: from postal.iol.unh.edu (postal.iol.unh.edu. [132.177.123.84]) by smtp-relay.gmail.com with ESMTPS id ju13-20020a170903428d00b001a6484fd026sm680934plb.115.2023.05.12.12.27.54 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 12 May 2023 12:27:54 -0700 (PDT) X-Relaying-Domain: iol.unh.edu Received: from iol.unh.edu (unknown [IPv6:2606:4100:3880:1271:90f9:1b64:f6e6:867f]) by postal.iol.unh.edu (Postfix) with ESMTP id 72ADD605246B; Fri, 12 May 2023 15:27:53 -0400 (EDT) From: jspewock@iol.unh.edu To: dev@dpdk.org Cc: Jeremy Spewock Subject: [RFC v2 1/2] dts: add smoke tests Date: Fri, 12 May 2023 15:25:43 -0400 Message-Id: <20230512192540.401-3-jspewock@iol.unh.edu> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230512192540.401-2-jspewock@iol.unh.edu> References: <20230512192540.401-2-jspewock@iol.unh.edu> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Jeremy Spewock Adds a new test suite for running smoke tests that verify general configuration aspects of the system under test. If any of these tests fail, the DTS execution terminates as part of a "fail-fast" model. Signed-off-by: Jeremy Spewock --- dts/conf.yaml | 9 ++ dts/framework/config/__init__.py | 21 +++++ dts/framework/config/conf_yaml_schema.json | 32 ++++++- dts/framework/dts.py | 19 +++- dts/framework/exception.py | 11 +++ dts/framework/remote_session/os_session.py | 6 +- .../remote_session/remote/__init__.py | 28 ++++++ dts/framework/test_result.py | 13 ++- dts/framework/test_suite.py | 24 ++++- dts/framework/testbed_model/__init__.py | 5 + .../interactive_apps/__init__.py | 6 ++ .../interactive_apps/interactive_command.py | 57 +++++++++++ .../interactive_apps/testpmd_driver.py | 24 +++++ dts/framework/testbed_model/node.py | 2 + dts/framework/testbed_model/sut_node.py | 6 ++ dts/tests/TestSuite_smoke_tests.py | 94 +++++++++++++++++++ 16 files changed, 348 insertions(+), 9 deletions(-) create mode 100644 dts/framework/testbed_model/interactive_apps/__init__.py create mode 100644 dts/framework/testbed_model/interactive_apps/interactive_command.py create mode 100644 dts/framework/testbed_model/interactive_apps/testpmd_driver.py create mode 100644 dts/tests/TestSuite_smoke_tests.py diff --git a/dts/conf.yaml b/dts/conf.yaml index a9bd8a3e..042ef954 100644 --- a/dts/conf.yaml +++ b/dts/conf.yaml @@ -10,13 +10,22 @@ executions: compiler_wrapper: ccache perf: false func: true + nics: #physical devices to be used for testing + - addresses: + - "0000:11:00.0" + - "0000:11:00.1" + driver: "i40e" + vdevs: #names of virtual devices to be used for testing + - "crypto_openssl" test_suites: + - smoke_tests - hello_world system_under_test: "SUT 1" nodes: - name: "SUT 1" hostname: sut1.change.me.localhost user: root + password: "" arch: x86_64 os: linux lcores: "" diff --git a/dts/framework/config/__init__.py b/dts/framework/config/__init__.py index ebb0823f..f3b8b6e3 100644 --- a/dts/framework/config/__init__.py +++ b/dts/framework/config/__init__.py @@ -106,6 +106,21 @@ def from_dict(d: dict) -> "NodeConfiguration": hugepages=hugepage_config, ) +@dataclass(slots=True, frozen=True) +class NICConfiguration: + addresses: list[str] + driver: str + + @staticmethod + def from_dict(d:dict) -> "NICConfiguration": + return NICConfiguration( + addresses=[addr for addr in d.get("addresses", [])], + driver=d.get("driver") + ) + @staticmethod + def from_list(l:list[dict]) -> list["NICConfiguration"]: + return [] + [NICConfiguration.from_dict(x) for x in l] + @dataclass(slots=True, frozen=True) class BuildTargetConfiguration: @@ -157,6 +172,8 @@ class ExecutionConfiguration: func: bool test_suites: list[TestSuiteConfig] system_under_test: NodeConfiguration + nics: list[NICConfiguration] + vdevs: list[str] @staticmethod def from_dict(d: dict, node_map: dict) -> "ExecutionConfiguration": @@ -166,7 +183,9 @@ def from_dict(d: dict, node_map: dict) -> "ExecutionConfiguration": test_suites: list[TestSuiteConfig] = list( map(TestSuiteConfig.from_dict, d["test_suites"]) ) + nic_conf: NICConfiguration = NICConfiguration.from_list(d['nics']) sut_name = d["system_under_test"] + list_of_vdevs = d["vdevs"] assert sut_name in node_map, f"Unknown SUT {sut_name} in execution {d}" return ExecutionConfiguration( @@ -174,7 +193,9 @@ def from_dict(d: dict, node_map: dict) -> "ExecutionConfiguration": perf=d["perf"], func=d["func"], test_suites=test_suites, + nics=nic_conf, system_under_test=node_map[sut_name], + vdevs=list_of_vdevs ) diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json index ca2d4a1e..603859de 100644 --- a/dts/framework/config/conf_yaml_schema.json +++ b/dts/framework/config/conf_yaml_schema.json @@ -40,6 +40,18 @@ "mscv" ] }, + "single_nic" : { + "type":"object", + "description": "an object that holds nic information", + "properties": { + "addresses": { + "type":"array", + "items": { + "type":"string" + } + } + } + }, "build_target": { "type": "object", "description": "Targets supported by DTS", @@ -97,7 +109,8 @@ "test_suite": { "type": "string", "enum": [ - "hello_world" + "hello_world", + "smoke_tests" ] }, "test_target": { @@ -211,6 +224,23 @@ ] } }, + "nics": { + "type": "array", + "items": { + "oneOf": [ + { + "$ref": "#/definitions/single_nic" + } + ] + } + }, + "vdevs": { + "description": "Names of vdevs to be used in execution", + "type": "array", + "items": { + "type":"string" + } + }, "system_under_test": { "$ref": "#/definitions/node_name" } diff --git a/dts/framework/dts.py b/dts/framework/dts.py index 05022845..0d03e158 100644 --- a/dts/framework/dts.py +++ b/dts/framework/dts.py @@ -5,6 +5,8 @@ import sys +from .exception import BlockingTestSuiteError + from .config import CONFIGURATION, BuildTargetConfiguration, ExecutionConfiguration from .logger import DTSLOG, getLogger from .test_result import BuildTargetResult, DTSResult, ExecutionResult, Result @@ -49,6 +51,7 @@ def run_all() -> None: nodes[sut_node.name] = sut_node if sut_node: + #SMOKE TEST EXECUTION GOES HERE! _run_execution(sut_node, execution, result) except Exception as e: @@ -118,7 +121,7 @@ def _run_build_target( try: sut_node.set_up_build_target(build_target) - result.dpdk_version = sut_node.dpdk_version + # result.dpdk_version = sut_node.dpdk_version build_target_result.update_setup(Result.PASS) except Exception as e: dts_logger.exception("Build target setup failed.") @@ -146,6 +149,7 @@ def _run_suites( with possibly only a subset of test cases. If no subset is specified, run all test cases. """ + end_execution = False for test_suite_config in execution.test_suites: try: full_suite_path = f"tests.TestSuite_{test_suite_config.test_suite}" @@ -160,13 +164,24 @@ def _run_suites( else: for test_suite_class in test_suite_classes: + #HERE NEEDS CHANGING test_suite = test_suite_class( sut_node, test_suite_config.test_cases, execution.func, build_target_result, + sut_node._build_target_config, + result ) - test_suite.run() + try: + test_suite.run() + except BlockingTestSuiteError as e: + dts_logger.exception("An error occurred within a blocking TestSuite, execution will now end.") + result.add_error(e) + end_execution = True + #if a blocking test failed and we need to bail out of suite executions + if end_execution: + break def _exit_dts() -> None: diff --git a/dts/framework/exception.py b/dts/framework/exception.py index ca353d98..4e3f63d1 100644 --- a/dts/framework/exception.py +++ b/dts/framework/exception.py @@ -25,6 +25,7 @@ class ErrorSeverity(IntEnum): SSH_ERR = 4 DPDK_BUILD_ERR = 10 TESTCASE_VERIFY_ERR = 20 + BLOCKING_TESTSUITE_ERR = 25 class DTSError(Exception): @@ -144,3 +145,13 @@ def __init__(self, value: str): def __str__(self) -> str: return repr(self.value) + +class BlockingTestSuiteError(DTSError): + suite_name: str + severity: ClassVar[ErrorSeverity] = ErrorSeverity.BLOCKING_TESTSUITE_ERR + + def __init__(self, suite_name:str) -> None: + self.suite_name = suite_name + + def __str__(self) -> str: + return f"Blocking suite {self.suite_name} failed." diff --git a/dts/framework/remote_session/os_session.py b/dts/framework/remote_session/os_session.py index 4c48ae25..22776bc1 100644 --- a/dts/framework/remote_session/os_session.py +++ b/dts/framework/remote_session/os_session.py @@ -12,7 +12,9 @@ from framework.testbed_model import LogicalCore from framework.utils import EnvVarsDict, MesonArgs -from .remote import CommandResult, RemoteSession, create_remote_session +from .remote import CommandResult, RemoteSession, create_remote_session, create_interactive_session + +from paramiko import SSHClient class OSSession(ABC): @@ -26,6 +28,7 @@ class OSSession(ABC): name: str _logger: DTSLOG remote_session: RemoteSession + _interactive_session: SSHClient def __init__( self, @@ -37,6 +40,7 @@ def __init__( self.name = name self._logger = logger self.remote_session = create_remote_session(node_config, name, logger) + self._interactive_session = create_interactive_session(node_config, name, logger) def close(self, force: bool = False) -> None: """ diff --git a/dts/framework/remote_session/remote/__init__.py b/dts/framework/remote_session/remote/__init__.py index 8a151221..abca8edc 100644 --- a/dts/framework/remote_session/remote/__init__.py +++ b/dts/framework/remote_session/remote/__init__.py @@ -9,8 +9,36 @@ from .remote_session import CommandResult, RemoteSession from .ssh_session import SSHSession +from paramiko import SSHClient, AutoAddPolicy +from framework.utils import GREEN def create_remote_session( node_config: NodeConfiguration, name: str, logger: DTSLOG ) -> RemoteSession: return SSHSession(node_config, name, logger) + +def create_interactive_session( + node_config: NodeConfiguration, name: str, logger: DTSLOG +) -> SSHClient: + """ + Creates a paramiko SSH session that is designed to be used for interactive shells + + This session is meant to be used on an "as needed" basis and may never be utilized + """ + client: SSHClient = SSHClient() + client.set_missing_host_key_policy(AutoAddPolicy) + ip: str = node_config.hostname + logger.info(GREEN(f"Connecting to host {ip}")) + #Preset to 22 because paramiko doesn't accept None + port: int = 22 + if ":" in node_config.hostname: + ip, port = node_config.hostname.split(":") + port = int(port) + client.connect( + ip, + username=node_config.user, + port=port, + password=node_config.password or "", + timeout=20 if port else 10 + ) + return client diff --git a/dts/framework/test_result.py b/dts/framework/test_result.py index 74391982..77202ae2 100644 --- a/dts/framework/test_result.py +++ b/dts/framework/test_result.py @@ -8,6 +8,7 @@ import os.path from collections.abc import MutableSequence from enum import Enum, auto +from typing import Dict from .config import ( OS, @@ -67,12 +68,13 @@ class Statistics(dict): Using a dict provides a convenient way to format the data. """ - def __init__(self, dpdk_version): + def __init__(self, output_info: Dict[str, str] | None): super(Statistics, self).__init__() for result in Result: self[result.name] = 0 self["PASS RATE"] = 0.0 - self["DPDK VERSION"] = dpdk_version + if output_info: + for info_key, info_val in output_info.items(): self[info_key] = info_val def __iadd__(self, other: Result) -> "Statistics": """ @@ -258,6 +260,7 @@ class DTSResult(BaseResult): """ dpdk_version: str | None + output: dict | None _logger: DTSLOG _errors: list[Exception] _return_code: ErrorSeverity @@ -267,6 +270,7 @@ class DTSResult(BaseResult): def __init__(self, logger: DTSLOG): super(DTSResult, self).__init__() self.dpdk_version = None + self.output = None self._logger = logger self._errors = [] self._return_code = ErrorSeverity.NO_ERR @@ -296,7 +300,10 @@ def process(self) -> None: for error in self._errors: self._logger.debug(repr(error)) - self._stats_result = Statistics(self.dpdk_version) + self._stats_result = Statistics(self.output) + #add information gathered from the smoke tests to the statistics + # for info_key, info_val in smoke_test_info.items(): self._stats_result[info_key] = info_val + # print(self._stats_result) self.add_stats(self._stats_result) with open(self._stats_filename, "w+") as stats_file: stats_file.write(str(self._stats_result)) diff --git a/dts/framework/test_suite.py b/dts/framework/test_suite.py index 0705f38f..1518fb8a 100644 --- a/dts/framework/test_suite.py +++ b/dts/framework/test_suite.py @@ -10,11 +10,14 @@ import inspect import re from types import MethodType +from typing import Dict -from .exception import ConfigurationError, SSHTimeoutError, TestCaseVerifyError +from .config import BuildTargetConfiguration + +from .exception import BlockingTestSuiteError, ConfigurationError, SSHTimeoutError, TestCaseVerifyError from .logger import DTSLOG, getLogger from .settings import SETTINGS -from .test_result import BuildTargetResult, Result, TestCaseResult, TestSuiteResult +from .test_result import BuildTargetResult, DTSResult, Result, TestCaseResult, TestSuiteResult from .testbed_model import SutNode @@ -37,10 +40,12 @@ class TestSuite(object): """ sut_node: SutNode + is_blocking = False _logger: DTSLOG _test_cases_to_run: list[str] _func: bool _result: TestSuiteResult + _dts_result: DTSResult def __init__( self, @@ -48,6 +53,8 @@ def __init__( test_cases: list[str], func: bool, build_target_result: BuildTargetResult, + build_target_conf: BuildTargetConfiguration, + dts_result: DTSResult ): self.sut_node = sut_node self._logger = getLogger(self.__class__.__name__) @@ -55,6 +62,8 @@ def __init__( self._test_cases_to_run.extend(SETTINGS.test_cases) self._func = func self._result = build_target_result.add_test_suite(self.__class__.__name__) + self.build_target_info = build_target_conf + self._dts_result = dts_result def set_up_suite(self) -> None: """ @@ -118,6 +127,9 @@ def run(self) -> None: f"the next test suite may be affected." ) self._result.update_setup(Result.ERROR, e) + if len(self._result.get_errors()) > 0 and self.is_blocking: + raise BlockingTestSuiteError(test_suite_name) + def _execute_test_suite(self) -> None: """ @@ -137,6 +149,7 @@ def _execute_test_suite(self) -> None: f"Attempt number {attempt_nr} out of {all_attempts}." ) self._run_test_case(test_case_method, test_case_result) + def _get_functional_test_cases(self) -> list[MethodType]: """ @@ -232,6 +245,11 @@ def _execute_test_case( test_case_result.update(Result.SKIP) raise KeyboardInterrupt("Stop DTS") + def write_to_statistics_file(self, output: Dict[str, str]): + if self._dts_result.output != None: + self._dts_result.output.update(output) + else: + self._dts_result.output = output def get_test_suites(testsuite_module_path: str) -> list[type[TestSuite]]: def is_test_suite(object) -> bool: @@ -252,3 +270,5 @@ def is_test_suite(object) -> bool: test_suite_class for _, test_suite_class in inspect.getmembers(testcase_module, is_test_suite) ] + + diff --git a/dts/framework/testbed_model/__init__.py b/dts/framework/testbed_model/__init__.py index f54a9470..63f17cc3 100644 --- a/dts/framework/testbed_model/__init__.py +++ b/dts/framework/testbed_model/__init__.py @@ -20,3 +20,8 @@ ) from .node import Node from .sut_node import SutNode + +from .interactive_apps import ( + InteractiveScriptHandler, + TestpmdDriver +) diff --git a/dts/framework/testbed_model/interactive_apps/__init__.py b/dts/framework/testbed_model/interactive_apps/__init__.py new file mode 100644 index 00000000..0382d7e0 --- /dev/null +++ b/dts/framework/testbed_model/interactive_apps/__init__.py @@ -0,0 +1,6 @@ +from .interactive_command import ( + InteractiveScriptHandler +) +from .testpmd_driver import ( + TestpmdDriver +) \ No newline at end of file diff --git a/dts/framework/testbed_model/interactive_apps/interactive_command.py b/dts/framework/testbed_model/interactive_apps/interactive_command.py new file mode 100644 index 00000000..7467911b --- /dev/null +++ b/dts/framework/testbed_model/interactive_apps/interactive_command.py @@ -0,0 +1,57 @@ +# import paramiko +from paramiko import SSHClient, Channel, channel +from framework.settings import SETTINGS + +class InteractiveScriptHandler: + + _ssh_client: SSHClient + _stdin: channel.ChannelStdinFile + _ssh_channel: Channel + + def __init__(self, ssh_client: SSHClient, timeout:float = SETTINGS.timeout) -> None: + self._ssh_client = ssh_client + self._ssh_channel = self._ssh_client.invoke_shell() + self._stdin = self._ssh_channel.makefile_stdin("wb") + self._ssh_channel.settimeout(timeout) + + def send_command(self, command:str) -> None: + """ + Send command to channel without recording output. + + This method will not verify any input or output, it will + simply assume the command succeeded + """ + self._stdin.write(command + '\n') + self._stdin.flush() + + def send_command_get_output(self, command:str, expect:str) -> str: + """ + Send a command and get all output before the expected ending string. + + **NOTE** + Lines that expect input are not included in the stdout buffer so they cannot be + used for expect. For example, if you were prompted to log into something + with a username and password, you cannot expect "username:" because it wont + yet be in the stdout buffer. A work around for this could be consuming an + extra newline character to force the current prompt into the stdout buffer. + + *Return* + All output before expected string + """ + stdout = self._ssh_channel.makefile("r") + self._stdin.write(command + '\n') + self._stdin.flush() + out:str = "" + for line in stdout: + out += str(line) + if expect in str(line): + break + stdout.close() #close the buffer to flush the output + return out + + def close(self): + self._stdin.close() + self._ssh_channel.close() + + def __del__(self): + self.close() diff --git a/dts/framework/testbed_model/interactive_apps/testpmd_driver.py b/dts/framework/testbed_model/interactive_apps/testpmd_driver.py new file mode 100644 index 00000000..1993eae6 --- /dev/null +++ b/dts/framework/testbed_model/interactive_apps/testpmd_driver.py @@ -0,0 +1,24 @@ +from framework.testbed_model.interactive_apps import InteractiveScriptHandler + +from pathlib import PurePath + +class TestpmdDriver: + prompt:str = "testpmd>" + interactive_handler: InteractiveScriptHandler + + def __init__(self, handler: InteractiveScriptHandler, dpdk_build_dir:PurePath, eal_flags:str = "", cmd_line_options:str = "") -> None: + """ + Sets the handler to drive the SSH session and starts testpmd + """ + self.interactive_handler = handler + # self.interactive_handler.send_command("sudo su") + # self.interactive_handler.send_command("cd /root/testpmd-testing/dpdk/build") + self.interactive_handler.send_command_get_output(f"{dpdk_build_dir}/app/dpdk-testpmd {eal_flags} -- -i {cmd_line_options}\n", self.prompt) + + def send_command(self, command:str, expect:str = prompt) -> str: + """ + Specific way of handling the command for testpmd + + An extra newline character is consumed in order to force the current line into the stdout buffer + """ + return self.interactive_handler.send_command_get_output(command + "\n", expect) \ No newline at end of file diff --git a/dts/framework/testbed_model/node.py b/dts/framework/testbed_model/node.py index d48fafe6..c5147e0e 100644 --- a/dts/framework/testbed_model/node.py +++ b/dts/framework/testbed_model/node.py @@ -40,6 +40,7 @@ class Node(object): lcores: list[LogicalCore] _logger: DTSLOG _other_sessions: list[OSSession] + _execution_config: ExecutionConfiguration def __init__(self, node_config: NodeConfiguration): self.config = node_config @@ -64,6 +65,7 @@ def set_up_execution(self, execution_config: ExecutionConfiguration) -> None: """ self._setup_hugepages() self._set_up_execution(execution_config) + self._execution_config = execution_config def _set_up_execution(self, execution_config: ExecutionConfiguration) -> None: """ diff --git a/dts/framework/testbed_model/sut_node.py b/dts/framework/testbed_model/sut_node.py index 2b2b50d9..8c39a66d 100644 --- a/dts/framework/testbed_model/sut_node.py +++ b/dts/framework/testbed_model/sut_node.py @@ -14,6 +14,7 @@ from .hw import LogicalCoreCount, LogicalCoreList, VirtualDevice from .node import Node +from .interactive_apps import InteractiveScriptHandler class SutNode(Node): @@ -261,6 +262,11 @@ def run_dpdk_app( return self.main_session.send_command( f"{app_path} {eal_args}", timeout, verify=True ) + def create_interactive_session_handler(self) -> InteractiveScriptHandler: + """ + Create a handler for interactive sessions + """ + return InteractiveScriptHandler(self.main_session._interactive_session) class EalParameters(object): diff --git a/dts/tests/TestSuite_smoke_tests.py b/dts/tests/TestSuite_smoke_tests.py new file mode 100644 index 00000000..bacf289d --- /dev/null +++ b/dts/tests/TestSuite_smoke_tests.py @@ -0,0 +1,94 @@ +from framework.test_suite import TestSuite +from framework.testbed_model.sut_node import SutNode + +from framework.testbed_model.interactive_apps import TestpmdDriver + +def get_compiler_version(compiler_name: str, sut_node: SutNode) -> str: + match compiler_name: + case "gcc": + return sut_node.main_session.send_command(f"{compiler_name} --version", 60).stdout.split("\n")[0] + case "clang": + return sut_node.main_session.send_command(f"{compiler_name} --version", 60).stdout.split("\n")[0] + case "msvc": + return sut_node.main_session.send_command(f"cl", 60).stdout + case "icc": + return sut_node.main_session.send_command(f"{compiler_name} -V", 60).stdout + +class SmokeTests(TestSuite): + is_blocking = True + + def set_up_suite(self) -> None: + """ + Setup: + build all DPDK + """ + self.dpdk_build_dir_path = self.sut_node.remote_dpdk_build_dir + + + def test_unit_tests(self) -> None: + """ + Test: + run the fast-test unit-test suite through meson + """ + self.sut_node.main_session.send_command(f"meson test -C {self.dpdk_build_dir_path} --suite fast-tests", 300) + + def test_driver_tests(self) -> None: + """ + Test: + run the driver-test unit-test suite through meson + """ + list_of_vdevs = "" + for dev in self.sut_node._execution_config.vdevs: + list_of_vdevs += f"{dev}," + print(list_of_vdevs) + if len(list_of_vdevs) > 0: + self.sut_node.main_session.send_command(f"meson test -C {self.dpdk_build_dir_path} --suite driver-tests --test-args \"--vdev {list_of_vdevs}\"", 300) + else: + self.sut_node.main_session.send_command(f"meson test -C {self.dpdk_build_dir_path} --suite driver-tests", 300) + + def test_gather_info(self) -> None: + """ + Test: + gather information about the system and send output to statistics.txt + """ + out = {} + + out['OS'] = self.sut_node.main_session.send_command("awk -F= '$1==\"NAME\" {print $2}' /etc/os-release", 60).stdout + out["OS VERSION"] = self.sut_node.main_session.send_command("awk -F= '$1==\"VERSION\" {print $2}' /etc/os-release", 60, True).stdout + out["COMPILER VERSION"] = get_compiler_version(self.build_target_info.compiler.name, self.sut_node) + out["DPDK VERSION"] = self.sut_node.dpdk_version + if self.build_target_info.os.name == "linux": + out['KERNEL VERSION'] = self.sut_node.main_session.send_command("uname -r", 60).stdout + elif self.build_target_info.os.name == "windows": + out['KERNEL VERSION'] = self.sut_node.main_session.send_command("uname -a", 60).stdout + self.write_to_statistics_file(out) + + + + def test_start_testpmd(self) -> None: + """ + Creates and instance of the testpmd driver to run the testpmd app + """ + driver: TestpmdDriver = TestpmdDriver(self.sut_node.create_interactive_session_handler(), self.dpdk_build_dir_path) + + print(driver.send_command("show port summary all")) + + def test_device_bound_to_driver(self) -> None: + """ + Test: + ensure that all drivers listed in the config are bound to the correct drivers + """ + for nic in self.sut_node._execution_config.nics: + for address in nic.addresses: + out = self.sut_node.main_session.send_command(f"{self.dpdk_build_dir_path}/../usertools/dpdk-devbind.py --status | grep {address}", 60) + self.verify( + len(out.stdout) != 0, + f"Failed to find configured device ({address}) using dpdk-devbind.py", + ) + for string in out.stdout.split(" "): + if 'drv=' in string: + self.verify( + string.split("=")[1] == nic.driver.strip(), + f'Driver for device {address} does not match driver listed in configuration (bound to {string.split("=")[1]})', + ) + \ No newline at end of file From patchwork Fri May 12 19:25:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeremy Spewock X-Patchwork-Id: 126842 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1819342AE8; Fri, 12 May 2023 21:28:20 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F3BDD410D7; Fri, 12 May 2023 21:28:19 +0200 (CEST) Received: from mail-yw1-f225.google.com (mail-yw1-f225.google.com [209.85.128.225]) by mails.dpdk.org (Postfix) with ESMTP id 4F14940ED7 for ; Fri, 12 May 2023 21:28:18 +0200 (CEST) Received: by mail-yw1-f225.google.com with SMTP id 00721157ae682-55a20a56a01so182761157b3.3 for ; Fri, 12 May 2023 12:28:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1683919697; x=1686511697; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IiJUisML/iaL/0qfyd2n7J3hfEVTR09QKS+NMzziXD4=; b=LmXSm9upbOheHid+KeYHSeGZ9ix0EXQYOweIMa5Mkufsosimi3nJ+r9tsTHXLu2jHa 94ZzVCrWfAXKSUCl3upY3CVJueifo5QJgRqA37v3K6ZSAwzrbGI7NNgNYuvdmFjsWPoN 28K9IrGHT3oN3AU7maLE4qbi0Q8J1TUBMxdCk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683919697; x=1686511697; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IiJUisML/iaL/0qfyd2n7J3hfEVTR09QKS+NMzziXD4=; b=VOUq6t3rCpEqvl9HnEU9MfXhlMMJkqqMeDbyFsMtTII8QZlXF3mbMmFiOk8Rl+zwT1 0Qoz4g8MFE/Q3jzR4qs3NEHvqCZBLdAr3fzE8NX4pZA36ikLlM0QVMT0NjhE60IKXxGI NQMdk9t5p2JOYHkxrhziKOm8FIGrVjIHboIoq7CS9Ur2uNp65ODAY5Bq03qiB+b2UNGD Ck0eVZxhoYj5P/zBFtl6VAZUBFBIHMbuoduvWp+M70Dr+Q4SK722SG+De4uGUAJoa7kR wrxJB5m6wpIx8sqTMFdmpcM3dl2A1EL3Y8ubB4hcIFgjtWAsVuuheckF7ics+LWbuTTl V04Q== X-Gm-Message-State: AC+VfDw1IaB5SxAxF/mVEePcoLgssZOdhBLQirU2UC2Lk+Vq55DzMKxu rEuKP75aTOblirTI1wujbXJhOceDMec5aU5x/Cz9xzJ+ueYsKeKtJQ1Zz3z2hx3oIAGF5dZe90w /qCpxiUYd5ah81Ug8O1CJHGsMEQIBwFX0v9SzetMhL0NJttB9iGbh9rVspFKXcwspoQwNl+m711 Zttr0zmtZxA87WDypmRYOvPQ== X-Google-Smtp-Source: ACHHUZ7z5cB5rbN+j3bX/v8o3ilO3SGdoyR+O+eimXmBWfutWt2WMtVNmJA4/SmnDm+QCVcyyJMEjPr0G55V X-Received: by 2002:a0d:ddd0:0:b0:560:f684:a8a6 with SMTP id g199-20020a0dddd0000000b00560f684a8a6mr6265692ywe.28.1683919697643; Fri, 12 May 2023 12:28:17 -0700 (PDT) Received: from postal.iol.unh.edu (postal.iol.unh.edu. [2606:4100:3880:1234::84]) by smtp-relay.gmail.com with ESMTPS id m187-20020a0dcac4000000b00545aa677bedsm1956822ywd.50.2023.05.12.12.28.17 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 12 May 2023 12:28:17 -0700 (PDT) X-Relaying-Domain: iol.unh.edu Received: from iol.unh.edu (unknown [IPv6:2606:4100:3880:1271:90f9:1b64:f6e6:867f]) by postal.iol.unh.edu (Postfix) with ESMTP id 4020C605246B; Fri, 12 May 2023 15:28:17 -0400 (EDT) From: jspewock@iol.unh.edu To: dev@dpdk.org Cc: Jeremy Spewock Subject: [RFC v2 2/2] dts: added paramiko to dependencies Date: Fri, 12 May 2023 15:25:45 -0400 Message-Id: <20230512192540.401-4-jspewock@iol.unh.edu> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230512192540.401-2-jspewock@iol.unh.edu> References: <20230512192540.401-2-jspewock@iol.unh.edu> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Jeremy Spewock added paramiko to the dependency files Signed-off-by: Jeremy Spewock --- dts/poetry.lock | 160 ++++++++++++++++++++++++++++++++++----------- dts/pyproject.toml | 1 + 2 files changed, 124 insertions(+), 37 deletions(-) diff --git a/dts/poetry.lock b/dts/poetry.lock index 0b2a007d..d5b41550 100644 --- a/dts/poetry.lock +++ b/dts/poetry.lock @@ -1,20 +1,33 @@ [[package]] name = "attrs" -version = "22.1.0" +version = "23.1.0" description = "Classes Without Boilerplate" category = "main" optional = false -python-versions = ">=3.5" +python-versions = ">=3.7" [package.extras] -dev = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "mypy (>=0.900,!=0.940)", "pytest-mypy-plugins", "zope.interface", "furo", "sphinx", "sphinx-notfound-page", "pre-commit", "cloudpickle"] -docs = ["furo", "sphinx", "zope.interface", "sphinx-notfound-page"] -tests = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "mypy (>=0.900,!=0.940)", "pytest-mypy-plugins", "zope.interface", "cloudpickle"] -tests_no_zope = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "mypy (>=0.900,!=0.940)", "pytest-mypy-plugins", "cloudpickle"] +cov = ["attrs", "coverage[toml] (>=5.3)"] +dev = ["attrs", "pre-commit"] +docs = ["furo", "myst-parser", "sphinx", "sphinx-notfound-page", "sphinxcontrib-towncrier", "towncrier", "zope-interface"] +tests = ["attrs", "zope-interface"] +tests-no-zope = ["cloudpickle", "hypothesis", "mypy (>=1.1.1)", "pympler", "pytest-mypy-plugins", "pytest-xdist", "pytest (>=4.3.0)"] + +[[package]] +name = "bcrypt" +version = "4.0.1" +description = "Modern password hashing for your software and your servers" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.extras] +tests = ["pytest (>=3.2.1,!=3.3.0)"] +typecheck = ["mypy"] [[package]] name = "black" -version = "22.10.0" +version = "22.12.0" description = "The uncompromising code formatter." category = "dev" optional = false @@ -33,6 +46,17 @@ d = ["aiohttp (>=3.7.4)"] jupyter = ["ipython (>=7.8.0)", "tokenize-rt (>=3.2.0)"] uvloop = ["uvloop (>=0.15.2)"] +[[package]] +name = "cffi" +version = "1.15.1" +description = "Foreign Function Interface for Python calling C code." +category = "main" +optional = false +python-versions = "*" + +[package.dependencies] +pycparser = "*" + [[package]] name = "click" version = "8.1.3" @@ -52,18 +76,39 @@ category = "dev" optional = false python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,!=3.6.*,>=2.7" +[[package]] +name = "cryptography" +version = "40.0.2" +description = "cryptography is a package which provides cryptographic recipes and primitives to Python developers." +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +cffi = ">=1.12" + +[package.extras] +docs = ["sphinx (>=5.3.0)", "sphinx-rtd-theme (>=1.1.1)"] +docstest = ["pyenchant (>=1.6.11)", "twine (>=1.12.0)", "sphinxcontrib-spelling (>=4.0.1)"] +pep8test = ["black", "ruff", "mypy", "check-manifest"] +sdist = ["setuptools-rust (>=0.11.4)"] +ssh = ["bcrypt (>=3.1.5)"] +test = ["pytest (>=6.2.0)", "pytest-shard (>=0.1.2)", "pytest-benchmark", "pytest-cov", "pytest-subtests", "pytest-xdist", "pretend", "iso8601"] +test-randomorder = ["pytest-randomly"] +tox = ["tox"] + [[package]] name = "isort" -version = "5.10.1" +version = "5.12.0" description = "A Python utility / library to sort Python imports." category = "dev" optional = false -python-versions = ">=3.6.1,<4.0" +python-versions = ">=3.8.0" [package.extras] -pipfile_deprecated_finder = ["pipreqs", "requirementslib"] -requirements_deprecated_finder = ["pipreqs", "pip-api"] -colors = ["colorama (>=0.4.3,<0.5.0)"] +colors = ["colorama (>=0.4.3)"] +requirements-deprecated-finder = ["pip-api", "pipreqs"] +pipfile-deprecated-finder = ["pip-shims (>=0.5.2)", "pipreqs", "requirementslib"] plugins = ["setuptools"] [[package]] @@ -87,7 +132,7 @@ python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" [[package]] name = "jsonschema" -version = "4.17.0" +version = "4.17.3" description = "An implementation of JSON Schema validation for Python" category = "main" optional = false @@ -129,15 +174,33 @@ reports = ["lxml"] [[package]] name = "mypy-extensions" -version = "0.4.3" -description = "Experimental type system extensions for programs checked with the mypy typechecker." +version = "1.0.0" +description = "Type system extensions for programs checked with the mypy type checker." category = "dev" optional = false -python-versions = "*" +python-versions = ">=3.5" + +[[package]] +name = "paramiko" +version = "3.1.0" +description = "SSH2 protocol library" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +bcrypt = ">=3.2" +cryptography = ">=3.3" +pynacl = ">=1.5" + +[package.extras] +all = ["pyasn1 (>=0.1.7)", "invoke (>=2.0)", "gssapi (>=1.4.1)", "pywin32 (>=2.1.8)"] +gssapi = ["pyasn1 (>=0.1.7)", "gssapi (>=1.4.1)", "pywin32 (>=2.1.8)"] +invoke = ["invoke (>=2.0)"] [[package]] name = "pathspec" -version = "0.10.1" +version = "0.11.1" description = "Utility library for gitignore style pattern matching of file paths." category = "dev" optional = false @@ -156,15 +219,15 @@ ptyprocess = ">=0.5" [[package]] name = "platformdirs" -version = "2.5.2" -description = "A small Python module for determining appropriate platform-specific dirs, e.g. a \"user data dir\"." +version = "3.5.1" +description = "A small Python package for determining appropriate platform-specific dirs, e.g. a \"user data dir\"." category = "dev" optional = false python-versions = ">=3.7" [package.extras] -docs = ["furo (>=2021.7.5b38)", "proselint (>=0.10.2)", "sphinx-autodoc-typehints (>=1.12)", "sphinx (>=4)"] -test = ["appdirs (==1.4.4)", "pytest-cov (>=2.7)", "pytest-mock (>=3.6)", "pytest (>=6)"] +docs = ["furo (>=2023.3.27)", "proselint (>=0.13)", "sphinx-autodoc-typehints (>=1.23,!=1.23.4)", "sphinx (>=6.2.1)"] +test = ["appdirs (==1.4.4)", "covdefaults (>=2.3)", "pytest-cov (>=4)", "pytest-mock (>=3.10)", "pytest (>=7.3.1)"] [[package]] name = "ptyprocess" @@ -176,25 +239,33 @@ python-versions = "*" [[package]] name = "pycodestyle" -version = "2.9.1" +version = "2.10.0" description = "Python style guide checker" category = "dev" optional = false python-versions = ">=3.6" +[[package]] +name = "pycparser" +version = "2.21" +description = "C parser in Python" +category = "main" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" + [[package]] name = "pydocstyle" -version = "6.1.1" +version = "6.3.0" description = "Python docstring style checker" category = "dev" optional = false python-versions = ">=3.6" [package.dependencies] -snowballstemmer = "*" +snowballstemmer = ">=2.2.0" [package.extras] -toml = ["toml"] +toml = ["tomli (>=1.2.3)"] [[package]] name = "pyflakes" @@ -228,9 +299,24 @@ tests = ["pytest (>=7.1.2)", "pytest-mypy", "eradicate (>=2.0.0)", "radon (>=5.1 toml = ["toml (>=0.10.2)"] vulture = ["vulture"] +[[package]] +name = "pynacl" +version = "1.5.0" +description = "Python binding to the Networking and Cryptography (NaCl) library" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +cffi = ">=1.4.1" + +[package.extras] +docs = ["sphinx (>=1.6.5)", "sphinx-rtd-theme"] +tests = ["pytest (>=3.2.1,!=3.3.0)", "hypothesis (>=3.27.0)"] + [[package]] name = "pyrsistent" -version = "0.19.1" +version = "0.19.3" description = "Persistent/Functional/Immutable data structures" category = "main" optional = false @@ -270,7 +356,7 @@ python-versions = ">=3.7" [[package]] name = "types-pyyaml" -version = "6.0.12.1" +version = "6.0.12.9" description = "Typing stubs for PyYAML" category = "main" optional = false @@ -278,7 +364,7 @@ python-versions = "*" [[package]] name = "typing-extensions" -version = "4.4.0" +version = "4.5.0" description = "Backported and Experimental Type Hints for Python 3.7+" category = "dev" optional = false @@ -299,13 +385,16 @@ jsonschema = ">=4,<5" [metadata] lock-version = "1.1" python-versions = "^3.10" -content-hash = "a0f040b07fc6ce4deb0be078b9a88c2a465cb6bccb9e260a67e92c2403e2319f" +content-hash = "c119901b1c13b14adf9b53624430aff8720d9a3a180b028579b82c65d49474df" [metadata.files] attrs = [] +bcrypt = [] black = [] +cffi = [] click = [] colorama = [] +cryptography = [] isort = [] jsonpatch = [] jsonpointer = [] @@ -313,20 +402,17 @@ jsonschema = [] mccabe = [] mypy = [] mypy-extensions = [] +paramiko = [] pathspec = [] -pexpect = [ - {file = "pexpect-4.8.0-py2.py3-none-any.whl", hash = "sha256:0b48a55dcb3c05f3329815901ea4fc1537514d6ba867a152b581d69ae3710937"}, - {file = "pexpect-4.8.0.tar.gz", hash = "sha256:fc65a43959d153d0114afe13997d439c22823a27cefceb5ff35c2178c6784c0c"}, -] -platformdirs = [ - {file = "platformdirs-2.5.2-py3-none-any.whl", hash = "sha256:027d8e83a2d7de06bbac4e5ef7e023c02b863d7ea5d079477e722bb41ab25788"}, - {file = "platformdirs-2.5.2.tar.gz", hash = "sha256:58c8abb07dcb441e6ee4b11d8df0ac856038f944ab98b7be6b27b2a3c7feef19"}, -] +pexpect = [] +platformdirs = [] ptyprocess = [] pycodestyle = [] +pycparser = [] pydocstyle = [] pyflakes = [] pylama = [] +pynacl = [] pyrsistent = [] pyyaml = [] snowballstemmer = [] diff --git a/dts/pyproject.toml b/dts/pyproject.toml index a136c91e..928837cb 100644 --- a/dts/pyproject.toml +++ b/dts/pyproject.toml @@ -13,6 +13,7 @@ pexpect = "^4.8.0" warlock = "^2.0.1" PyYAML = "^6.0" types-PyYAML = "^6.0.8" +paramiko = "^3.1.0" [tool.poetry.dev-dependencies] mypy = "^0.961"