From patchwork Thu Jun 6 21:34:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Luca Vizzarro X-Patchwork-Id: 140839 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3602244178; Thu, 6 Jun 2024 23:34:34 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9D00A427D9; Thu, 6 Jun 2024 23:34:30 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 054CE40EAB for ; Thu, 6 Jun 2024 23:34:29 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D9554339; Thu, 6 Jun 2024 14:34:52 -0700 (PDT) Received: from localhost.localdomain (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 68D693F792; Thu, 6 Jun 2024 14:34:27 -0700 (PDT) From: Luca Vizzarro To: dev@dpdk.org Cc: Jeremy Spewock , =?utf-8?q?Juraj_Linke=C5=A1?= , Luca Vizzarro , Paul Szczepanek Subject: [PATCH v5 1/5] dts: fix InteractiveShell command prompt filtering Date: Thu, 6 Jun 2024 22:34:16 +0100 Message-Id: <20240606213420.254260-2-luca.vizzarro@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240606213420.254260-1-luca.vizzarro@arm.com> References: <20240412111136.3470304-1-luca.vizzarro@arm.com> <20240606213420.254260-1-luca.vizzarro@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org When sending a command using an instance of InteractiveShell the output should filter out the trailing shell prompt when returning it. After every command two shell prompts are summoned. One is consumed as it is used as a delimiter for the command output. The second one is not consumed and left for the next command to be sent. Given that the consumed prompt is merely a delimiter, this should not be added to the returned output, as it may be mistakenly be interpreted as the command's own output. Bugzilla ID: 1411 Fixes: 88489c0501af ("dts: add smoke tests") Signed-off-by: Luca Vizzarro Reviewed-by: Paul Szczepanek Reviewed-by: Juraj Linkeš Reviewed-by: Jeremy Spewock Tested-by: Nicholas Pratte Reviewed-by: Nicholas Pratte --- dts/framework/remote_session/interactive_shell.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dts/framework/remote_session/interactive_shell.py b/dts/framework/remote_session/interactive_shell.py index 074a541279..aa5d2d9be8 100644 --- a/dts/framework/remote_session/interactive_shell.py +++ b/dts/framework/remote_session/interactive_shell.py @@ -132,11 +132,11 @@ def send_command(self, command: str, prompt: str | None = None) -> str: self._stdin.flush() out: str = "" for line in self._stdout: - out += line if prompt in line and not line.rstrip().endswith( command.rstrip() ): # ignore line that sent command break + out += line self._logger.debug(f"Got output: {out}") return out From patchwork Thu Jun 6 21:34:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Luca Vizzarro X-Patchwork-Id: 140840 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 02EDF44178; Thu, 6 Jun 2024 23:34:42 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4A642427E8; Thu, 6 Jun 2024 23:34:32 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 95ACA4161A for ; Thu, 6 Jun 2024 23:34:30 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 77F102F4; Thu, 6 Jun 2024 14:34:54 -0700 (PDT) Received: from localhost.localdomain (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id CDCBB3F792; Thu, 6 Jun 2024 14:34:28 -0700 (PDT) From: Luca Vizzarro To: dev@dpdk.org Cc: Jeremy Spewock , =?utf-8?q?Juraj_Linke=C5=A1?= , Luca Vizzarro , Paul Szczepanek Subject: [PATCH v5 2/5] dts: skip first line of send command output Date: Thu, 6 Jun 2024 22:34:17 +0100 Message-Id: <20240606213420.254260-3-luca.vizzarro@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240606213420.254260-1-luca.vizzarro@arm.com> References: <20240412111136.3470304-1-luca.vizzarro@arm.com> <20240606213420.254260-1-luca.vizzarro@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The first line of the InteractiveShell send_command method is generally the command input field. This sometimes is unwanted, therefore this commit enables the possibility of omitting the first line from the returned output. Signed-off-by: Luca Vizzarro Reviewed-by: Paul Szczepanek Reviewed-by: Juraj Linkeš Reviewed-by: Jeremy Spewock --- dts/framework/remote_session/interactive_shell.py | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/dts/framework/remote_session/interactive_shell.py b/dts/framework/remote_session/interactive_shell.py index aa5d2d9be8..c025c52ba3 100644 --- a/dts/framework/remote_session/interactive_shell.py +++ b/dts/framework/remote_session/interactive_shell.py @@ -105,7 +105,9 @@ def _start_application(self, get_privileged_command: Callable[[str], str] | None start_command = get_privileged_command(start_command) self.send_command(start_command) - def send_command(self, command: str, prompt: str | None = None) -> str: + def send_command( + self, command: str, prompt: str | None = None, skip_first_line: bool = False + ) -> str: """Send `command` and get all output before the expected ending string. Lines that expect input are not included in the stdout buffer, so they cannot @@ -121,6 +123,7 @@ def send_command(self, command: str, prompt: str | None = None) -> str: command: The command to send. prompt: After sending the command, `send_command` will be expecting this string. If :data:`None`, will use the class's default prompt. + skip_first_line: Skip the first line when capturing the output. Returns: All output in the buffer before expected string. @@ -132,6 +135,9 @@ def send_command(self, command: str, prompt: str | None = None) -> str: self._stdin.flush() out: str = "" for line in self._stdout: + if skip_first_line: + skip_first_line = False + continue if prompt in line and not line.rstrip().endswith( command.rstrip() ): # ignore line that sent command From patchwork Thu Jun 6 21:34:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Luca Vizzarro X-Patchwork-Id: 140841 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A6B4244178; Thu, 6 Jun 2024 23:34:47 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 80534427E0; Thu, 6 Jun 2024 23:34:34 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 8E8B14281D for ; Thu, 6 Jun 2024 23:34:32 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8A2842F4; Thu, 6 Jun 2024 14:34:56 -0700 (PDT) Received: from localhost.localdomain (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 888A63F792; Thu, 6 Jun 2024 14:34:30 -0700 (PDT) From: Luca Vizzarro To: dev@dpdk.org Cc: Jeremy Spewock , =?utf-8?q?Juraj_Linke=C5=A1?= , Luca Vizzarro , Paul Szczepanek Subject: [PATCH v5 3/5] dts: add parsing utility module Date: Thu, 6 Jun 2024 22:34:18 +0100 Message-Id: <20240606213420.254260-4-luca.vizzarro@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240606213420.254260-1-luca.vizzarro@arm.com> References: <20240412111136.3470304-1-luca.vizzarro@arm.com> <20240606213420.254260-1-luca.vizzarro@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Adds parsing text into a custom dataclass. It provides a new `TextParser` dataclass to be inherited. This implements the `parse` method, which combined with the parser functions, it can automatically parse the value for each field. This new utility will facilitate and simplify the parsing of complex command outputs, while ensuring that the codebase does not get bloated and stays flexible. Signed-off-by: Luca Vizzarro Reviewed-by: Paul Szczepanek Reviewed-by: Juraj Linkeš Reviewed-by: Jeremy Spewock Tested-by: Nicholas Pratte Reviewed-by: Nicholas Pratte --- dts/framework/exception.py | 9 ++ dts/framework/parser.py | 229 +++++++++++++++++++++++++++++++++++++ 2 files changed, 238 insertions(+) create mode 100644 dts/framework/parser.py diff --git a/dts/framework/exception.py b/dts/framework/exception.py index cce1e0231a..d9d690037d 100644 --- a/dts/framework/exception.py +++ b/dts/framework/exception.py @@ -31,6 +31,8 @@ class ErrorSeverity(IntEnum): #: SSH_ERR = 4 #: + INTERNAL_ERR = 5 + #: DPDK_BUILD_ERR = 10 #: TESTCASE_VERIFY_ERR = 20 @@ -192,3 +194,10 @@ def __init__(self, suite_name: str) -> None: def __str__(self) -> str: """Add some context to the string representation.""" return f"Blocking suite {self._suite_name} failed." + + +class InternalError(DTSError): + """An internal error or bug has occurred in DTS.""" + + #: + severity: ClassVar[ErrorSeverity] = ErrorSeverity.INTERNAL_ERR diff --git a/dts/framework/parser.py b/dts/framework/parser.py new file mode 100644 index 0000000000..741dfff821 --- /dev/null +++ b/dts/framework/parser.py @@ -0,0 +1,229 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2024 Arm Limited + +"""Parsing utility module. + +This module provides :class:`~TextParser` which can be used to model any dataclass to a block of +text. +""" + +import re +from abc import ABC +from dataclasses import MISSING, dataclass, fields +from functools import partial +from typing import Any, Callable, TypedDict, cast + +from typing_extensions import Self + +from framework.exception import InternalError + + +class ParserFn(TypedDict): + """Parser function in a dict compatible with the :func:`dataclasses.field` metadata param.""" + + #: + TextParser_fn: Callable[[str], Any] + + +@dataclass +class TextParser(ABC): + r"""Helper abstract dataclass that parses a text according to the fields' rules. + + In order to enable text parsing in a dataclass, subclass it with :class:`TextParser`. + + The provided `parse` method is a factory which parses the supplied text and creates an instance + with populated dataclass fields. This takes text as an argument and for each field in the + dataclass, the field's parser function is run against the whole text. The returned value is then + assigned to the field of the new instance. If the field does not have a parser function its + default value or factory is used instead. If no default is available either, an exception is + raised. + + This class provides a selection of parser functions and a function to wrap parser functions with + generic functions. Parser functions are designed to be passed to the fields' metadata param. The + most commonly used parser function is expected to be the `find` method, which runs a regular + expression against the text to find matches. + + Example: + The following example makes use of and demonstrates every parser function available: + + ..code:: python + + from dataclasses import dataclass, field + from enum import Enum + from framework.parser import TextParser + + class Colour(Enum): + BLACK = 1 + WHITE = 2 + + @classmethod + def from_str(cls, text: str): + match text: + case "black": + return cls.BLACK + case "white": + return cls.WHITE + case _: + return None # unsupported colour + + @classmethod + def make_parser(cls): + # make a parser function that finds a match and + # then makes it a Colour object through Colour.from_str + return TextParser.wrap(TextParser.find(r"is a (\w+)"), cls.from_str) + + @dataclass + class Animal(TextParser): + kind: str = field(metadata=TextParser.find(r"is a \w+ (\w+)")) + name: str = field(metadata=TextParser.find(r"^(\w+)")) + colour: Colour = field(metadata=Colour.make_parser()) + age: int = field(metadata=TextParser.find_int(r"aged (\d+)")) + + steph = Animal.parse("Stephanie is a white cat aged 10") + print(steph) # Animal(kind='cat', name='Stephanie', colour=, age=10) + """ + + """============ BEGIN PARSER FUNCTIONS ============""" + + @staticmethod + def wrap(parser_fn: ParserFn, wrapper_fn: Callable) -> ParserFn: + """Makes a wrapped parser function. + + `parser_fn` is called and if a non-None value is returned, `wrapper_function` is called with + it. Otherwise the function returns early with None. In pseudo-code: + + intermediate_value := parser_fn(input) + if intermediary_value is None then + output := None + else + output := wrapper_fn(intermediate_value) + + Args: + parser_fn: The dictionary storing the parser function to be wrapped. + wrapper_fn: The function that wraps `parser_fn`. + + Returns: + ParserFn: A dictionary for the `dataclasses.field` metadata argument containing the + newly wrapped parser function. + """ + inner_fn = parser_fn["TextParser_fn"] + + def _composite_parser_fn(text: str) -> Any: + intermediate_value = inner_fn(text) + if intermediate_value is None: + return None + return wrapper_fn(intermediate_value) + + return ParserFn(TextParser_fn=_composite_parser_fn) + + @staticmethod + def find( + pattern: str | re.Pattern[str], + flags: re.RegexFlag = re.RegexFlag(0), + named: bool = False, + ) -> ParserFn: + """Makes a parser function that finds a regular expression match in the text. + + If the pattern has any capturing groups, it returns None if no match was found, otherwise a + tuple containing the values per each group is returned. If the pattern has only one + capturing group and a match was found, its value is returned. If the pattern has no + capturing groups then either True or False is returned if the pattern had a match or not. + + Args: + pattern: The regular expression pattern. + flags: The regular expression flags. Ignored if the given pattern is already compiled. + named: If set to True only the named capturing groups will be returned, as a dictionary. + + Returns: + ParserFn: A dictionary for the `dataclasses.field` metadata argument containing the find + parser function. + """ + if isinstance(pattern, str): + pattern = re.compile(pattern, flags) + + def _find(text: str) -> Any: + m = pattern.search(text) + if m is None: + return None if pattern.groups > 0 else False + + if pattern.groups == 0: + return True + + if named: + return m.groupdict() + + matches = m.groups() + if len(matches) == 1: + return matches[0] + + return matches + + return ParserFn(TextParser_fn=_find) + + @staticmethod + def find_int( + pattern: str | re.Pattern[str], + flags: re.RegexFlag = re.RegexFlag(0), + int_base: int = 0, + ) -> ParserFn: + """Makes a parser function that converts the match of :meth:`~find` to int. + + This function is compatible only with a pattern containing one capturing group. + + Args: + pattern: The regular expression pattern. + flags: The regular expression flags. Ignored if the given pattern is already compiled. + int_base: The base of the number to convert from. + + Raises: + InternalError: If the pattern does not have exactly one capturing group. + + Returns: + ParserFn: A dictionary for the `dataclasses.field` metadata argument containing the + :meth:`~find` parser function wrapped by the int built-in. + """ + if isinstance(pattern, str): + pattern = re.compile(pattern, flags) + + if pattern.groups != 1: + raise InternalError("only one capturing group is allowed with this parser function") + + return TextParser.wrap(TextParser.find(pattern), partial(int, base=int_base)) + + """============ END PARSER FUNCTIONS ============""" + + @classmethod + def parse(cls, text: str) -> Self: + """Creates a new instance of the class from the given text. + + A new class instance is created with all the fields that have a parser function in their + metadata. Fields without one are ignored and are expected to have a default value, otherwise + the class initialization will fail. + + A field is populated with the value returned by its corresponding parser function. + + Args: + text: the text to parse + + Raises: + InternalError: if the parser did not find a match and the field does not have a default + value or default factory. + + Returns: + A new instance of the class. + """ + fields_values = {} + for field in fields(cls): + parse = cast(ParserFn, field.metadata).get("TextParser_fn") + if parse is None: + continue + + value = parse(text) + if value is not None: + fields_values[field.name] = value + elif field.default is MISSING and field.default_factory is MISSING: + raise InternalError( + f"parser for field {field.name} returned None, but the field has no default" + ) + + return cls(**fields_values) From patchwork Thu Jun 6 21:34:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Luca Vizzarro X-Patchwork-Id: 140842 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D78C344178; Thu, 6 Jun 2024 23:34:53 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C99C542D0B; Thu, 6 Jun 2024 23:34:35 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id B22C3427DE for ; Thu, 6 Jun 2024 23:34:34 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9D8B82F4; Thu, 6 Jun 2024 14:34:58 -0700 (PDT) Received: from localhost.localdomain (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id BD7B23F792; Thu, 6 Jun 2024 14:34:32 -0700 (PDT) From: Luca Vizzarro To: dev@dpdk.org Cc: Jeremy Spewock , =?utf-8?q?Juraj_Linke=C5=A1?= , Luca Vizzarro , Paul Szczepanek Subject: [PATCH v5 4/5] dts: add `show port info` command to TestPmdShell Date: Thu, 6 Jun 2024 22:34:19 +0100 Message-Id: <20240606213420.254260-5-luca.vizzarro@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240606213420.254260-1-luca.vizzarro@arm.com> References: <20240412111136.3470304-1-luca.vizzarro@arm.com> <20240606213420.254260-1-luca.vizzarro@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add a new TestPmdPort data structure to represent the output returned by `show port info`, which is implemented as part of TestPmdShell. The TestPmdPort data structure and its derived classes are modelled based on the relevant testpmd source code. This implementation makes extensive use of regular expressions, which all parse individually. The rationale behind this is to lower the risk of the testpmd output changing as part of development. Therefore minimising breakage. Bugzilla ID: 1407 Signed-off-by: Luca Vizzarro Reviewed-by: Paul Szczepanek Reviewed-by: Juraj Linkeš Reviewed-by: Jeremy Spewock Tested-by: Nicholas Pratte Reviewed-by: Nicholas Pratte --- dts/framework/remote_session/testpmd_shell.py | 537 +++++++++++++++++- 1 file changed, 536 insertions(+), 1 deletion(-) diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py index cb2ab6bd00..ab9a1f86a9 100644 --- a/dts/framework/remote_session/testpmd_shell.py +++ b/dts/framework/remote_session/testpmd_shell.py @@ -1,6 +1,7 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2023 University of New Hampshire # Copyright(c) 2023 PANTHEON.tech s.r.o. +# Copyright(c) 2024 Arm Limited """Testpmd interactive shell. @@ -15,12 +16,17 @@ testpmd_shell.close() """ +import re import time -from enum import auto +from dataclasses import dataclass, field +from enum import Flag, auto from pathlib import PurePath from typing import Callable, ClassVar +from typing_extensions import Self + from framework.exception import InteractiveCommandExecutionError +from framework.parser import ParserFn, TextParser from framework.settings import SETTINGS from framework.utils import StrEnum @@ -80,6 +86,491 @@ class TestPmdForwardingModes(StrEnum): recycle_mbufs = auto() +class VLANOffloadFlag(Flag): + """Flag representing the VLAN offload settings of a NIC port.""" + + #: + STRIP = auto() + #: + FILTER = auto() + #: + EXTEND = auto() + #: + QINQ_STRIP = auto() + + @classmethod + def from_str_dict(cls, d): + """Makes an instance from a dict containing the flag member names with an "on" value. + + Args: + d: A dictionary containing the flag members as keys and any string value. + + Returns: + A new instance of the flag. + """ + flag = cls(0) + for name in cls.__members__: + if d.get(name) == "on": + flag |= cls[name] + return flag + + @classmethod + def make_parser(cls) -> ParserFn: + """Makes a parser function. + + Returns: + ParserFn: A dictionary for the `dataclasses.field` metadata argument containing a + parser function that makes an instance of this flag from text. + """ + return TextParser.wrap( + TextParser.find( + r"VLAN offload:\s+" + r"strip (?Pon|off), " + r"filter (?Pon|off), " + r"extend (?Pon|off), " + r"qinq strip (?Pon|off)$", + re.MULTILINE, + named=True, + ), + cls.from_str_dict, + ) + + +class RSSOffloadTypesFlag(Flag): + """Flag representing the RSS offload flow types supported by the NIC port.""" + + #: + ipv4 = auto() + #: + ipv4_frag = auto() + #: + ipv4_tcp = auto() + #: + ipv4_udp = auto() + #: + ipv4_sctp = auto() + #: + ipv4_other = auto() + #: + ipv6 = auto() + #: + ipv6_frag = auto() + #: + ipv6_tcp = auto() + #: + ipv6_udp = auto() + #: + ipv6_sctp = auto() + #: + ipv6_other = auto() + #: + l2_payload = auto() + #: + ipv6_ex = auto() + #: + ipv6_tcp_ex = auto() + #: + ipv6_udp_ex = auto() + #: + port = auto() + #: + vxlan = auto() + #: + geneve = auto() + #: + nvgre = auto() + #: + user_defined_22 = auto() + #: + gtpu = auto() + #: + eth = auto() + #: + s_vlan = auto() + #: + c_vlan = auto() + #: + esp = auto() + #: + ah = auto() + #: + l2tpv3 = auto() + #: + pfcp = auto() + #: + pppoe = auto() + #: + ecpri = auto() + #: + mpls = auto() + #: + ipv4_chksum = auto() + #: + l4_chksum = auto() + #: + l2tpv2 = auto() + #: + ipv6_flow_label = auto() + #: + user_defined_38 = auto() + #: + user_defined_39 = auto() + #: + user_defined_40 = auto() + #: + user_defined_41 = auto() + #: + user_defined_42 = auto() + #: + user_defined_43 = auto() + #: + user_defined_44 = auto() + #: + user_defined_45 = auto() + #: + user_defined_46 = auto() + #: + user_defined_47 = auto() + #: + user_defined_48 = auto() + #: + user_defined_49 = auto() + #: + user_defined_50 = auto() + #: + user_defined_51 = auto() + #: + l3_pre96 = auto() + #: + l3_pre64 = auto() + #: + l3_pre56 = auto() + #: + l3_pre48 = auto() + #: + l3_pre40 = auto() + #: + l3_pre32 = auto() + #: + l2_dst_only = auto() + #: + l2_src_only = auto() + #: + l4_dst_only = auto() + #: + l4_src_only = auto() + #: + l3_dst_only = auto() + #: + l3_src_only = auto() + + #: + ip = ipv4 | ipv4_frag | ipv4_other | ipv6 | ipv6_frag | ipv6_other | ipv6_ex + #: + udp = ipv4_udp | ipv6_udp | ipv6_udp_ex + #: + tcp = ipv4_tcp | ipv6_tcp | ipv6_tcp_ex + #: + sctp = ipv4_sctp | ipv6_sctp + #: + tunnel = vxlan | geneve | nvgre + #: + vlan = s_vlan | c_vlan + #: + all = ( + eth + | vlan + | ip + | tcp + | udp + | sctp + | l2_payload + | l2tpv3 + | esp + | ah + | pfcp + | gtpu + | ecpri + | mpls + | l2tpv2 + ) + + @classmethod + def from_list_string(cls, names: str) -> Self: + """Makes a flag from a whitespace-separated list of names. + + Args: + names: a whitespace-separated list containing the members of this flag. + + Returns: + An instance of this flag. + """ + flag = cls(0) + for name in names.split(): + flag |= cls.from_str(name) + return flag + + @classmethod + def from_str(cls, name: str) -> Self: + """Makes a flag matching the supplied name. + + Args: + name: a valid member of this flag in text + Returns: + An instance of this flag. + """ + member_name = name.strip().replace("-", "_") + return cls[member_name] + + @classmethod + def make_parser(cls) -> ParserFn: + """Makes a parser function. + + Returns: + ParserFn: A dictionary for the `dataclasses.field` metadata argument containing a + parser function that makes an instance of this flag from text. + """ + return TextParser.wrap( + TextParser.find(r"Supported RSS offload flow types:((?:\r?\n? \S+)+)", re.MULTILINE), + RSSOffloadTypesFlag.from_list_string, + ) + + +class DeviceCapabilitiesFlag(Flag): + """Flag representing the device capabilities.""" + + #: Device supports Rx queue setup after device started. + RUNTIME_RX_QUEUE_SETUP = auto() + #: Device supports Tx queue setup after device started. + RUNTIME_TX_QUEUE_SETUP = auto() + #: Device supports shared Rx queue among ports within Rx domain and switch domain. + RXQ_SHARE = auto() + #: Device supports keeping flow rules across restart. + FLOW_RULE_KEEP = auto() + #: Device supports keeping shared flow objects across restart. + FLOW_SHARED_OBJECT_KEEP = auto() + + @classmethod + def make_parser(cls) -> ParserFn: + """Makes a parser function. + + Returns: + ParserFn: A dictionary for the `dataclasses.field` metadata argument containing a + parser function that makes an instance of this flag from text. + """ + return TextParser.wrap( + TextParser.find_int(r"Device capabilities: (0x[A-Fa-f\d]+)"), + cls, + ) + + +class DeviceErrorHandlingMode(StrEnum): + """Enum representing the device error handling mode.""" + + #: + none = auto() + #: + passive = auto() + #: + proactive = auto() + #: + unknown = auto() + + @classmethod + def make_parser(cls) -> ParserFn: + """Makes a parser function. + + Returns: + ParserFn: A dictionary for the `dataclasses.field` metadata argument containing a + parser function that makes an instance of this enum from text. + """ + return TextParser.wrap(TextParser.find(r"Device error handling mode: (\w+)"), cls) + + +def make_device_private_info_parser() -> ParserFn: + """Device private information parser. + + Ensures that we are not parsing invalid device private info output. + + Returns: + ParserFn: A dictionary for the `dataclasses.field` metadata argument containing a parser + function that parses the device private info from the TestPmd port info output. + """ + + def _validate(info: str): + info = info.strip() + if info == "none" or info.startswith("Invalid file") or info.startswith("Failed to dump"): + return None + return info + + return TextParser.wrap(TextParser.find(r"Device private info:\s+([\s\S]+)"), _validate) + + +@dataclass +class TestPmdPort(TextParser): + """Dataclass representing the result of testpmd's ``show port info`` command.""" + + #: + id: int = field(metadata=TextParser.find_int(r"Infos for port (\d+)\b")) + #: + device_name: str = field(metadata=TextParser.find(r"Device name: ([^\r\n]+)")) + #: + driver_name: str = field(metadata=TextParser.find(r"Driver name: ([^\r\n]+)")) + #: + socket_id: int = field(metadata=TextParser.find_int(r"Connect to socket: (\d+)")) + #: + is_link_up: bool = field(metadata=TextParser.find("Link status: up")) + #: + link_speed: str = field(metadata=TextParser.find(r"Link speed: ([^\r\n]+)")) + #: + is_link_full_duplex: bool = field(metadata=TextParser.find("Link duplex: full-duplex")) + #: + is_link_autonegotiated: bool = field(metadata=TextParser.find("Autoneg status: On")) + #: + is_promiscuous_mode_enabled: bool = field(metadata=TextParser.find("Promiscuous mode: enabled")) + #: + is_allmulticast_mode_enabled: bool = field( + metadata=TextParser.find("Allmulticast mode: enabled") + ) + #: Maximum number of MAC addresses + max_mac_addresses_num: int = field( + metadata=TextParser.find_int(r"Maximum number of MAC addresses: (\d+)") + ) + #: Maximum configurable length of RX packet + max_hash_mac_addresses_num: int = field( + metadata=TextParser.find_int(r"Maximum number of MAC addresses of hash filtering: (\d+)") + ) + #: Minimum size of RX buffer + min_rx_bufsize: int = field(metadata=TextParser.find_int(r"Minimum size of RX buffer: (\d+)")) + #: Maximum configurable length of RX packet + max_rx_packet_length: int = field( + metadata=TextParser.find_int(r"Maximum configurable length of RX packet: (\d+)") + ) + #: Maximum configurable size of LRO aggregated packet + max_lro_packet_size: int = field( + metadata=TextParser.find_int(r"Maximum configurable size of LRO aggregated packet: (\d+)") + ) + + #: Current number of RX queues + rx_queues_num: int = field(metadata=TextParser.find_int(r"Current number of RX queues: (\d+)")) + #: Max possible RX queues + max_rx_queues_num: int = field(metadata=TextParser.find_int(r"Max possible RX queues: (\d+)")) + #: Max possible number of RXDs per queue + max_queue_rxd_num: int = field( + metadata=TextParser.find_int(r"Max possible number of RXDs per queue: (\d+)") + ) + #: Min possible number of RXDs per queue + min_queue_rxd_num: int = field( + metadata=TextParser.find_int(r"Min possible number of RXDs per queue: (\d+)") + ) + #: RXDs number alignment + rxd_alignment_num: int = field(metadata=TextParser.find_int(r"RXDs number alignment: (\d+)")) + + #: Current number of TX queues + tx_queues_num: int = field(metadata=TextParser.find_int(r"Current number of TX queues: (\d+)")) + #: Max possible TX queues + max_tx_queues_num: int = field(metadata=TextParser.find_int(r"Max possible TX queues: (\d+)")) + #: Max possible number of TXDs per queue + max_queue_txd_num: int = field( + metadata=TextParser.find_int(r"Max possible number of TXDs per queue: (\d+)") + ) + #: Min possible number of TXDs per queue + min_queue_txd_num: int = field( + metadata=TextParser.find_int(r"Min possible number of TXDs per queue: (\d+)") + ) + #: TXDs number alignment + txd_alignment_num: int = field(metadata=TextParser.find_int(r"TXDs number alignment: (\d+)")) + #: Max segment number per packet + max_packet_segment_num: int = field( + metadata=TextParser.find_int(r"Max segment number per packet: (\d+)") + ) + #: Max segment number per MTU/TSO + max_mtu_segment_num: int = field( + metadata=TextParser.find_int(r"Max segment number per MTU\/TSO: (\d+)") + ) + + #: + device_capabilities: DeviceCapabilitiesFlag = field( + metadata=DeviceCapabilitiesFlag.make_parser(), + ) + #: + device_error_handling_mode: DeviceErrorHandlingMode = field( + metadata=DeviceErrorHandlingMode.make_parser() + ) + #: + device_private_info: str | None = field( + default=None, + metadata=make_device_private_info_parser(), + ) + + #: + hash_key_size: int | None = field( + default=None, metadata=TextParser.find_int(r"Hash key size in bytes: (\d+)") + ) + #: + redirection_table_size: int | None = field( + default=None, metadata=TextParser.find_int(r"Redirection table size: (\d+)") + ) + #: + supported_rss_offload_flow_types: RSSOffloadTypesFlag = field( + default=RSSOffloadTypesFlag(0), metadata=RSSOffloadTypesFlag.make_parser() + ) + + #: + mac_address: str | None = field( + default=None, metadata=TextParser.find(r"MAC address: ([A-Fa-f0-9:]+)") + ) + #: + fw_version: str | None = field( + default=None, metadata=TextParser.find(r"Firmware-version: ([^\r\n]+)") + ) + #: + dev_args: str | None = field(default=None, metadata=TextParser.find(r"Devargs: ([^\r\n]+)")) + #: Socket id of the memory allocation + mem_alloc_socket_id: int | None = field( + default=None, + metadata=TextParser.find_int(r"memory allocation on the socket: (\d+)"), + ) + #: + mtu: int | None = field(default=None, metadata=TextParser.find_int(r"MTU: (\d+)")) + + #: + vlan_offload: VLANOffloadFlag | None = field( + default=None, + metadata=VLANOffloadFlag.make_parser(), + ) + + #: Maximum size of RX buffer + max_rx_bufsize: int | None = field( + default=None, metadata=TextParser.find_int(r"Maximum size of RX buffer: (\d+)") + ) + #: Maximum number of VFs + max_vfs_num: int | None = field( + default=None, metadata=TextParser.find_int(r"Maximum number of VFs: (\d+)") + ) + #: Maximum number of VMDq pools + max_vmdq_pools_num: int | None = field( + default=None, metadata=TextParser.find_int(r"Maximum number of VMDq pools: (\d+)") + ) + + #: + switch_name: str | None = field( + default=None, metadata=TextParser.find(r"Switch name: ([\r\n]+)") + ) + #: + switch_domain_id: int | None = field( + default=None, metadata=TextParser.find_int(r"Switch domain Id: (\d+)") + ) + #: + switch_port_id: int | None = field( + default=None, metadata=TextParser.find_int(r"Switch Port Id: (\d+)") + ) + #: + switch_rx_domain: int | None = field( + default=None, metadata=TextParser.find_int(r"Switch Rx domain: (\d+)") + ) + + class TestPmdShell(InteractiveShell): """Testpmd interactive shell. @@ -225,6 +716,50 @@ def set_forward_mode(self, mode: TestPmdForwardingModes, verify: bool = True): f"Test pmd failed to set fwd mode to {mode.value}" ) + def show_port_info_all(self) -> list[TestPmdPort]: + """Returns the information of all the ports. + + Returns: + list[TestPmdPort]: A list containing all the ports information as `TestPmdPort`. + """ + output = self.send_command("show port info all") + + # Sample output of the "all" command looks like: + # + # + # + # ********************* Infos for port 0 ********************* + # Key: value + # + # ********************* Infos for port 1 ********************* + # Key: value + # + # + # Takes advantage of the double new line in between ports as end delimiter. But we need to + # artificially add a new line at the end to pick up the last port. Because commands are + # executed on a pseudo-terminal created by paramiko on the remote node, lines end with CRLF. + # Therefore we also need to take the carriage return into account. + iter = re.finditer(r"\*{21}.*?[\r\n]{4}", output + "\r\n", re.S) + return [TestPmdPort.parse(block.group(0)) for block in iter] + + def show_port_info(self, port_id: int) -> TestPmdPort: + """Returns the given port information. + + Args: + port_id: The port ID to gather information for. + + Raises: + InteractiveCommandExecutionError: If `port_id` is invalid. + + Returns: + TestPmdPort: An instance of `TestPmdPort` containing the given port's information. + """ + output = self.send_command(f"show port info {port_id}", skip_first_line=True) + if output.startswith("Invalid port"): + raise InteractiveCommandExecutionError("invalid port given") + + return TestPmdPort.parse(output) + def close(self) -> None: """Overrides :meth:`~.interactive_shell.close`.""" self.send_command("quit", "") From patchwork Thu Jun 6 21:34:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Luca Vizzarro X-Patchwork-Id: 140843 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B1D7744178; Thu, 6 Jun 2024 23:35:01 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B10C542D7F; Thu, 6 Jun 2024 23:34:38 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 1023442D3F for ; Thu, 6 Jun 2024 23:34:37 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E0C0C2F4; Thu, 6 Jun 2024 14:35:00 -0700 (PDT) Received: from localhost.localdomain (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id A5F8B3F792; Thu, 6 Jun 2024 14:34:34 -0700 (PDT) From: Luca Vizzarro To: dev@dpdk.org Cc: Jeremy Spewock , =?utf-8?q?Juraj_Linke=C5=A1?= , Luca Vizzarro , Paul Szczepanek Subject: [PATCH v5 5/5] dts: add `show port stats` command to TestPmdShell Date: Thu, 6 Jun 2024 22:34:20 +0100 Message-Id: <20240606213420.254260-6-luca.vizzarro@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240606213420.254260-1-luca.vizzarro@arm.com> References: <20240412111136.3470304-1-luca.vizzarro@arm.com> <20240606213420.254260-1-luca.vizzarro@arm.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add a new TestPmdPortStats data structure to represent the output returned by `show port stats`, which is implemented as part of TestPmdShell. Bugzilla ID: 1407 Signed-off-by: Luca Vizzarro Reviewed-by: Paul Szczepanek Reviewed-by: Juraj Linkeš Reviewed-by: Jeremy Spewock Tested-by: Nicholas Pratte Reviewed-by: Nicholas Pratte TestPmdPort: return TestPmdPort.parse(output) + def show_port_stats_all(self) -> list[TestPmdPortStats]: + """Returns the statistics of all the ports. + + Returns: + list[TestPmdPortStats]: A list containing all the ports stats as `TestPmdPortStats`. + """ + output = self.send_command("show port stats all") + + # Sample output of the "all" command looks like: + # + # ########### NIC statistics for port 0 ########### + # values... + # ################################################# + # + # ########### NIC statistics for port 1 ########### + # values... + # ################################################# + # + iter = re.finditer(r"(^ #*.+#*$[^#]+)^ #*\r$", output, re.MULTILINE) + return [TestPmdPortStats.parse(block.group(1)) for block in iter] + + def show_port_stats(self, port_id: int) -> TestPmdPortStats: + """Returns the given port statistics. + + Args: + port_id: The port ID to gather information for. + + Raises: + InteractiveCommandExecutionError: If `port_id` is invalid. + + Returns: + TestPmdPortStats: An instance of `TestPmdPortStats` containing the given port's stats. + """ + output = self.send_command(f"show port stats {port_id}", skip_first_line=True) + if output.startswith("Invalid port"): + raise InteractiveCommandExecutionError("invalid port given") + + return TestPmdPortStats.parse(output) + def close(self) -> None: """Overrides :meth:`~.interactive_shell.close`.""" self.send_command("quit", "")