From patchwork Wed Jul 24 18:39:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jeremy Spewock X-Patchwork-Id: 142704 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2FC26456A2; Wed, 24 Jul 2024 20:39:38 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 71DF1427D2; Wed, 24 Jul 2024 20:39:37 +0200 (CEST) Received: from mail-vk1-f231.google.com (mail-vk1-f231.google.com [209.85.221.231]) by mails.dpdk.org (Postfix) with ESMTP id 7B895427C2 for ; Wed, 24 Jul 2024 20:39:36 +0200 (CEST) Received: by mail-vk1-f231.google.com with SMTP id 71dfb90a1353d-4f50c673e83so54873e0c.0 for ; Wed, 24 Jul 2024 11:39:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1721846376; x=1722451176; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RtHvppNB1eGZkwkuor6cB1R1Cf1wfWaUz0MBbOT+zVM=; b=dejgkHLcoswbE2NTospBQ20BjPkRkUmcJO/ZSFbNP9xcozoaI/+HkEijg6YVyE2B46 grWl8yiExgN9GOKZjrt7V/wZt8fny3oi2e0rOppRQfMAJ+ZLcP8PhfDgTOcJnQPsepOX s3Q5cwpfAcZzt8C5Licdn5moHQ7Tw9AgtjGVM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721846376; x=1722451176; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RtHvppNB1eGZkwkuor6cB1R1Cf1wfWaUz0MBbOT+zVM=; b=RhPpI16mPZh2o9WKJDIWzkOrg3nx2+BBbEeoNKmxPf497kCD8id/Do9Jp6tnlFU3qt iOTj2C2g5BYysGO4beurDflagJ6+OXYaVnDYl0pbn4v5JSAhv3Kpoa+MHQw2FgowLiQO botLRCHrdFOIQyATFaTTCWhqYndRjYa0BpuhBw+JkAVlzEYFXPDL375IoFm/zCdMNRTy Faabw8Wki73S0vhIaVS293+UthyVg77V86QkitzrTsS63AfjILccH6Z4ahKXHNV9bl1B RMJM0nLWoIxzH+SvKrmHdcFWyLlgRkhfYbFG3FU4sdDOt7R3U33hBc3tcwWAbS7gv4CV aqsQ== X-Gm-Message-State: AOJu0Yx/nmPcpgg5TZcTFlpeMxW9DTTTUkePo8rAvjCkYA7TX0N9xDDf VtT+8wfAYvjmgR5yNaw18pRpTUU/UqI3zbYt2rMZL5Y3KlvOw5QJFsIclN8DmliiBG/D5WoqyrM t35C7aeMnhOklPgxpYT5gLd4zeiQRw8kY X-Google-Smtp-Source: AGHT+IGGCUY77xyDea2cwEWEtV9DFGWPy4TDSOQnUc8wnFWnnaJkD5za6wpkjuprDnhicvcJm6xycUOPF+rv X-Received: by 2002:a05:6122:7d1:b0:4f5:27ac:ce6e with SMTP id 71dfb90a1353d-4f6c5b953f0mr1051999e0c.7.1721846375586; Wed, 24 Jul 2024 11:39:35 -0700 (PDT) Received: from postal.iol.unh.edu (postal.iol.unh.edu. [132.177.123.84]) by smtp-relay.gmail.com with ESMTPS id 71dfb90a1353d-4f4fa004e1csm724195e0c.11.2024.07.24.11.39.35 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 Jul 2024 11:39:35 -0700 (PDT) X-Relaying-Domain: iol.unh.edu Received: from iol.unh.edu (unknown [IPv6:2606:4100:3880:1257::1083]) by postal.iol.unh.edu (Postfix) with ESMTP id B1563605C351; Wed, 24 Jul 2024 14:39:34 -0400 (EDT) From: jspewock@iol.unh.edu To: juraj.linkes@pantheon.tech, probb@iol.unh.edu, yoan.picchi@foss.arm.com, wathsala.vithanage@arm.com, Honnappa.Nagarahalli@arm.com, npratte@iol.unh.edu, paul.szczepanek@arm.com, thomas@monjalon.net, Luca.Vizzarro@arm.com Cc: dev@dpdk.org, Jeremy Spewock , Luca Vizzarro Subject: [PATCH v6 1/3] dts: Improve output gathering in interactive shells Date: Wed, 24 Jul 2024 14:39:12 -0400 Message-ID: <20240724183914.436188-2-jspewock@iol.unh.edu> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240724183914.436188-1-jspewock@iol.unh.edu> References: <20240501161623.26672-1-jspewock@iol.unh.edu> <20240724183914.436188-1-jspewock@iol.unh.edu> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Jeremy Spewock The current implementation of consuming output from interactive shells relies on being able to find an expected prompt somewhere within the output buffer after sending the command. This is useful in situations where the prompt does not appear in the output itself, but in some practical cases (such as the starting of an XML-RPC server for scapy) the prompt exists in one of the commands sent to the shell and this can cause the command to exit early and creates a race condition between the server starting and the first command being sent to the server. This patch addresses this problem by searching for a line that strictly ends with the provided prompt, rather than one that simply contains it, so that the detection that a command is finished is more consistent. It also adds a catch to detect when a command times out before finding the prompt or the underlying SSH session dies so that the exception can be wrapped into a more explicit one and be more consistent with the non-interactive shells. Bugzilla ID: 1359 Fixes: 88489c0501af ("dts: add smoke tests") Signed-off-by: Jeremy Spewock Reviewed-by: Juraj Linkeš Reviewed-by: Luca Vizzarro Reviewed-by: Nicholas Pratte --- dts/framework/exception.py | 66 ++++++++++++------- .../single_active_interactive_shell.py | 51 +++++++++----- 2 files changed, 80 insertions(+), 37 deletions(-) diff --git a/dts/framework/exception.py b/dts/framework/exception.py index 74fd2af3b6..f45f789825 100644 --- a/dts/framework/exception.py +++ b/dts/framework/exception.py @@ -51,26 +51,6 @@ class DTSError(Exception): severity: ClassVar[ErrorSeverity] = ErrorSeverity.GENERIC_ERR -class SSHTimeoutError(DTSError): - """The SSH execution of a command timed out.""" - - #: - severity: ClassVar[ErrorSeverity] = ErrorSeverity.SSH_ERR - _command: str - - def __init__(self, command: str): - """Define the meaning of the first argument. - - Args: - command: The executed command. - """ - self._command = command - - def __str__(self) -> str: - """Add some context to the string representation.""" - return f"{self._command} execution timed out." - - class SSHConnectionError(DTSError): """An unsuccessful SSH connection.""" @@ -98,8 +78,42 @@ def __str__(self) -> str: return message -class SSHSessionDeadError(DTSError): - """The SSH session is no longer alive.""" +class _SSHTimeoutError(DTSError): + """The execution of a command via SSH timed out. + + This class is private and meant to be raised as its interactive and non-interactive variants. + """ + + #: + severity: ClassVar[ErrorSeverity] = ErrorSeverity.SSH_ERR + _command: str + + def __init__(self, command: str): + """Define the meaning of the first argument. + + Args: + command: The executed command. + """ + self._command = command + + def __str__(self) -> str: + """Add some context to the string representation.""" + return f"{self._command} execution timed out." + + +class SSHTimeoutError(_SSHTimeoutError): + """The execution of a command on a non-interactive SSH session timed out.""" + + +class InteractiveSSHTimeoutError(_SSHTimeoutError): + """The execution of a command on an interactive SSH session timed out.""" + + +class _SSHSessionDeadError(DTSError): + """The SSH session is no longer alive. + + This class is private and meant to be raised as its interactive and non-interactive variants. + """ #: severity: ClassVar[ErrorSeverity] = ErrorSeverity.SSH_ERR @@ -118,6 +132,14 @@ def __str__(self) -> str: return f"SSH session with {self._host} has died." +class SSHSessionDeadError(_SSHSessionDeadError): + """Non-interactive SSH session has died.""" + + +class InteractiveSSHSessionDeadError(_SSHSessionDeadError): + """Interactive SSH session as died.""" + + class ConfigurationError(DTSError): """An invalid configuration.""" diff --git a/dts/framework/remote_session/single_active_interactive_shell.py b/dts/framework/remote_session/single_active_interactive_shell.py index 38094c0fe2..38318aa764 100644 --- a/dts/framework/remote_session/single_active_interactive_shell.py +++ b/dts/framework/remote_session/single_active_interactive_shell.py @@ -27,7 +27,11 @@ from paramiko import Channel, channel # type: ignore[import-untyped] from typing_extensions import Self -from framework.exception import InteractiveCommandExecutionError +from framework.exception import ( + InteractiveCommandExecutionError, + InteractiveSSHSessionDeadError, + InteractiveSSHTimeoutError, +) from framework.logger import DTSLogger from framework.params import Params from framework.settings import SETTINGS @@ -71,7 +75,10 @@ class SingleActiveInteractiveShell(ABC): #: Extra characters to add to the end of every command #: before sending them. This is often overridden by subclasses and is - #: most commonly an additional newline character. + #: most commonly an additional newline character. This additional newline + #: character is used to force the line that is currently awaiting input + #: into the stdout buffer so that it can be consumed and checked against + #: the expected prompt. _command_extra_chars: ClassVar[str] = "" #: Path to the executable to start the interactive application. @@ -138,7 +145,7 @@ def _start_application(self) -> None: try: self.send_command(start_command) break - except TimeoutError: + except InteractiveSSHTimeoutError: self._logger.info( f"Interactive shell failed to start (attempt {attempt+1} out of " f"{self._init_attempts})" @@ -175,6 +182,9 @@ def send_command( Raises: InteractiveCommandExecutionError: If attempting to send a command to a shell that is not currently running. + InteractiveSSHSessionDeadError: The session died while executing the command. + InteractiveSSHTimeoutError: If command was sent but prompt could not be found in + the output before the timeout. """ if not self.is_alive: raise InteractiveCommandExecutionError( @@ -183,19 +193,30 @@ def send_command( self._logger.info(f"Sending: '{command}'") if prompt is None: prompt = self._default_prompt - self._stdin.write(f"{command}{self._command_extra_chars}\n") - self._stdin.flush() out: str = "" - for line in self._stdout: - if skip_first_line: - skip_first_line = False - continue - if prompt in line and not line.rstrip().endswith( - command.rstrip() - ): # ignore line that sent command - break - out += line - self._logger.debug(f"Got output: {out}") + try: + self._stdin.write(f"{command}{self._command_extra_chars}\n") + self._stdin.flush() + for line in self._stdout: + if skip_first_line: + skip_first_line = False + continue + if line.rstrip().endswith(prompt): + break + out += line + except TimeoutError as e: + self._logger.exception(e) + self._logger.debug( + f"Prompt ({prompt}) was not found in output from command before timeout." + ) + raise InteractiveSSHTimeoutError(command) from e + except OSError as e: + self._logger.exception(e) + raise InteractiveSSHSessionDeadError( + self._node.main_session.interactive_session.hostname + ) from e + finally: + self._logger.debug(f"Got output: {out}") return out def _close(self) -> None: From patchwork Wed Jul 24 18:39:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jeremy Spewock X-Patchwork-Id: 142705 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 495B3456A2; Wed, 24 Jul 2024 20:39:44 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B331A42E76; Wed, 24 Jul 2024 20:39:39 +0200 (CEST) Received: from mail-qv1-f100.google.com (mail-qv1-f100.google.com [209.85.219.100]) by mails.dpdk.org (Postfix) with ESMTP id DA2B040B97 for ; Wed, 24 Jul 2024 20:39:37 +0200 (CEST) Received: by mail-qv1-f100.google.com with SMTP id 6a1803df08f44-6b7a36f26f3so11311586d6.1 for ; Wed, 24 Jul 2024 11:39:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1721846377; x=1722451177; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ap735j3cLYE3m1ei6abxO38EK1Sj/3VyuWAOmiZB5+8=; b=GczmPdQHYIS+eQP9bsDLw2Zc6i+z8VLGzuN7hivtEXg2Iwi34OXsLPB2QZ821nX1f9 1MpR+d06NGR71a3ksNJLgZ88svGCIWmAsuxxoN8QL2833LiO89BEWd6i9iEwkNJByGGZ qJIwiqeQ3WYe1Cc7SxminD0N0Uwc+rsaYTJfk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721846377; x=1722451177; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ap735j3cLYE3m1ei6abxO38EK1Sj/3VyuWAOmiZB5+8=; b=Cufs1RFUPkoCRgADbVG++YOrhPfTt1e+IpJEPs38+LnuRpjh4MKgLoQVQ46tRT5Ime YP5tqKxREj2D9zaBIPDWjPHM42XsdUSjBmIOhufFKhobLobhaZVq6iuElLOoLDgYPz/S QsY1BzvJn9bpmVgcIC1+P5O6qE+nIas9ssWL+XNAOiehhJHUuPFLLifbxvuVywK7kQUr gf8cueZmfSlVyp6NTDoJViA6l5uQFozY58D0nYFw5xjXMR45AKKH5A9AZUVZttFXCsny EBv1upq7/B8fagHWJb8Y3IsZ949x44BSQZLdOEsXOmhvFYrcGlRfhfbVXU/gyVrb37bd /7xg== X-Gm-Message-State: AOJu0YxFQojQGqOyDE73jUtP4wFWoLO1S9BIGl+6YqV26nVIIsg1csl7 gMOT41IcptMgyG2+HtpuywHPoM/oWh6rskqnGsGvhYqkWR/pmDhcIHAwXugouVCEkZx3Ecl4mik 0iCGba2yF1DowPpOWqascoTFH8YE/NjCb X-Google-Smtp-Source: AGHT+IGoXAuU/spIZg13V2qr5h9IxWF4jkDCupa/CKrH2HYaZ2UGtI3eC9/Urcf+FD0XUDzTvbSb99nUYybf X-Received: by 2002:a0c:fa52:0:b0:6b5:2aa3:3a7f with SMTP id 6a1803df08f44-6b99154d6c9mr44211076d6.20.1721846377160; Wed, 24 Jul 2024 11:39:37 -0700 (PDT) Received: from postal.iol.unh.edu (postal.iol.unh.edu. [132.177.123.84]) by smtp-relay.gmail.com with ESMTPS id 6a1803df08f44-6b96ce6ffd8sm4742646d6.24.2024.07.24.11.39.37 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 Jul 2024 11:39:37 -0700 (PDT) X-Relaying-Domain: iol.unh.edu Received: from iol.unh.edu (unknown [IPv6:2606:4100:3880:1257::1083]) by postal.iol.unh.edu (Postfix) with ESMTP id 520B2605C351; Wed, 24 Jul 2024 14:39:36 -0400 (EDT) From: jspewock@iol.unh.edu To: juraj.linkes@pantheon.tech, probb@iol.unh.edu, yoan.picchi@foss.arm.com, wathsala.vithanage@arm.com, Honnappa.Nagarahalli@arm.com, npratte@iol.unh.edu, paul.szczepanek@arm.com, thomas@monjalon.net, Luca.Vizzarro@arm.com Cc: dev@dpdk.org, Jeremy Spewock , Luca Vizzarro Subject: [PATCH v6 2/3] dts: Add missing docstring from XML-RPC server Date: Wed, 24 Jul 2024 14:39:13 -0400 Message-ID: <20240724183914.436188-3-jspewock@iol.unh.edu> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240724183914.436188-1-jspewock@iol.unh.edu> References: <20240501161623.26672-1-jspewock@iol.unh.edu> <20240724183914.436188-1-jspewock@iol.unh.edu> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Jeremy Spewock When this XML-RPC server implementation was added, the docstring had to be shortened in order to reduce the chances of this race condition being encountered. Now that this race condition issue is resolved, the full docstring can be restored. Signed-off-by: Jeremy Spewock Reviewed-by: Juraj Linkeš Reviewed-by: Luca Vizzarro Reviewed-by: Nicholas Pratte --- .../testbed_model/traffic_generator/scapy.py | 46 ++++++++++++++++++- 1 file changed, 45 insertions(+), 1 deletion(-) diff --git a/dts/framework/testbed_model/traffic_generator/scapy.py b/dts/framework/testbed_model/traffic_generator/scapy.py index 7f0cc2bc18..08e1f4ae7e 100644 --- a/dts/framework/testbed_model/traffic_generator/scapy.py +++ b/dts/framework/testbed_model/traffic_generator/scapy.py @@ -128,9 +128,53 @@ def scapy_send_packets(xmlrpc_packets: list[xmlrpc.client.Binary], send_iface: s class QuittableXMLRPCServer(SimpleXMLRPCServer): - """Basic XML-RPC server. + r"""Basic XML-RPC server. The server may be augmented by functions serializable by the :mod:`marshal` module. + + Example: + :: + + def hello_world(): + # to be sent to the XML-RPC server + print("Hello World!") + + # start the XML-RPC server on the remote node + # this is done by starting a Python shell on the remote node + from framework.remote_session import PythonShell + # the example assumes you're already connected to a tg_node + session = tg_node.create_interactive_shell(PythonShell, timeout=5, privileged=True) + + # then importing the modules needed to run the server + # and the modules for any functions later added to the server + session.send_command("import xmlrpc") + session.send_command("from xmlrpc.server import SimpleXMLRPCServer") + + # sending the source code of this class to the Python shell + from xmlrpc.server import SimpleXMLRPCServer + src = inspect.getsource(QuittableXMLRPCServer) + src = "\n".join([l for l in src.splitlines() if not l.isspace() and l != ""]) + spacing = "\n" * 4 + session.send_command(spacing + src + spacing) + + # then starting the server with: + command = "s = QuittableXMLRPCServer(('0.0.0.0', {listen_port}));s.serve_forever()" + session.send_command(command, "XMLRPC OK") + + # now the server is running on the remote node and we can add functions to it + # first connect to the server from the execution node + import xmlrpc.client + server_url = f"http://{tg_node.config.hostname}:8000" + rpc_server_proxy = xmlrpc.client.ServerProxy(server_url) + + # get the function bytes to send + import marshal + function_bytes = marshal.dumps(hello_world.__code__) + rpc_server_proxy.add_rpc_function(hello_world.__name__, function_bytes) + + # now we can execute the function on the server + xmlrpc_binary_recv: xmlrpc.client.Binary = rpc_server_proxy.hello_world() + print(str(xmlrpc_binary_recv)) """ def __init__(self, *args, **kwargs): From patchwork Wed Jul 24 18:39:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jeremy Spewock X-Patchwork-Id: 142706 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4E34F456A2; Wed, 24 Jul 2024 20:39:50 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0007A42EDE; Wed, 24 Jul 2024 20:39:41 +0200 (CEST) Received: from mail-pj1-f100.google.com (mail-pj1-f100.google.com [209.85.216.100]) by mails.dpdk.org (Postfix) with ESMTP id 92F0842D55 for ; Wed, 24 Jul 2024 20:39:39 +0200 (CEST) Received: by mail-pj1-f100.google.com with SMTP id 98e67ed59e1d1-2cb7cd6f5f2so100195a91.2 for ; Wed, 24 Jul 2024 11:39:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1721846379; x=1722451179; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8lFlTaDFvIejpGFVOGx0BVlFciRudhGUD20pgoyrnKA=; b=Qc+5/q8o224SjGO3Evtv15tuwPhFironbx714P6B8jHFo6ZXOcrQmgkexKn6snImXb OjkhAbwXHblvEykmWB5XFoWgF12AfcXr6Xtq0EwD7g+WwGk/RVNTqyE3JjPGonKxmPbv QyrvXUdYEjxL9PaqQ/io7G7p0dW8Q83UsatAU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721846379; x=1722451179; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8lFlTaDFvIejpGFVOGx0BVlFciRudhGUD20pgoyrnKA=; b=sXUjo93BKp/iw+if33o3j833XuceX/212VOY1tVcLgKEF0yerWig++1anGuBoSqbhY 1OtDaH5UXDQA1OSNGB33zPZOJLUq56mlmxMmnYyew/BTa2Y99EuKCJUuaWvl6yDGtfnN MsVWDuBqRCkjXCwc/Lvb6YkMW2kZAix/zxXNjvuZfzo5YAIc756ygqgNWvlUi8szlrJP WCZKtkvBnZlGfIzzqJ19OqtGYnN/zpGvJDfbL1gJt0HKAUoc18Oy+beRTxCK12UK8ejI OZMRn5RuPK5Jf4HKcQUMQQQph6zD7tP5Db/nL6W8LJqiekarlqZerLjeZHhv4gc5HZJR 7lkA== X-Gm-Message-State: AOJu0YyduJ/NzW8yM3Li7wSlLJBgOCnlO+kr5OUnAUoqmJX2qA9AX2Ig o7vdLekYlvoqaO4iIbaOB3kJ+V5bXPWfVLTtdl3qwz9Z2Y3rtGM8IlpJDcO1TVJzYXe1hVUYUTe efe9LcV/iWONiCI7QIFHjBc/dzjuVtN8JtlfzyLvBRL8qHn88 X-Google-Smtp-Source: AGHT+IEMKfmXHJ07HRQVXETlqPbkCLpknUnq0r7oezKppQbXP6kkDGyuenVxNx6DX5z+C/X9eD8CQAfeAFmE X-Received: by 2002:a17:90b:2242:b0:2c9:98c2:f6d7 with SMTP id 98e67ed59e1d1-2cf2376fd94mr469158a91.5.1721846378729; Wed, 24 Jul 2024 11:39:38 -0700 (PDT) Received: from postal.iol.unh.edu (postal.iol.unh.edu. [2606:4100:3880:1234::84]) by smtp-relay.gmail.com with ESMTPS id 98e67ed59e1d1-2cdb73d2594sm100749a91.5.2024.07.24.11.39.38 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 Jul 2024 11:39:38 -0700 (PDT) X-Relaying-Domain: iol.unh.edu Received: from iol.unh.edu (unknown [IPv6:2606:4100:3880:1257::1083]) by postal.iol.unh.edu (Postfix) with ESMTP id 6990F605C351; Wed, 24 Jul 2024 14:39:37 -0400 (EDT) From: jspewock@iol.unh.edu To: juraj.linkes@pantheon.tech, probb@iol.unh.edu, yoan.picchi@foss.arm.com, wathsala.vithanage@arm.com, Honnappa.Nagarahalli@arm.com, npratte@iol.unh.edu, paul.szczepanek@arm.com, thomas@monjalon.net, Luca.Vizzarro@arm.com Cc: dev@dpdk.org, Jeremy Spewock , Luca Vizzarro Subject: [PATCH v6 3/3] dts: Improve logging for interactive shells Date: Wed, 24 Jul 2024 14:39:14 -0400 Message-ID: <20240724183914.436188-4-jspewock@iol.unh.edu> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240724183914.436188-1-jspewock@iol.unh.edu> References: <20240501161623.26672-1-jspewock@iol.unh.edu> <20240724183914.436188-1-jspewock@iol.unh.edu> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Jeremy Spewock The messages being logged by interactive shells currently are using the same logger as the node they were created from. Because of this, when sending interactive commands, the logs make no distinction between when you are sending a command directly to the host and when you are using an interactive shell on the host. This change adds names to interactive shells so that they are able to use their own loggers with distinct names. Signed-off-by: Jeremy Spewock Reviewed-by: Juraj Linkeš Tested-by: Nicholas Pratte Reviewed-by: Nicholas Pratte Reviewed-by: Luca Vizzarro --- dts/framework/remote_session/dpdk_shell.py | 3 ++- .../remote_session/single_active_interactive_shell.py | 9 +++++++-- dts/framework/remote_session/testpmd_shell.py | 2 ++ dts/framework/testbed_model/traffic_generator/scapy.py | 4 +++- 4 files changed, 14 insertions(+), 4 deletions(-) diff --git a/dts/framework/remote_session/dpdk_shell.py b/dts/framework/remote_session/dpdk_shell.py index 950c6ca670..c5f5c2d116 100644 --- a/dts/framework/remote_session/dpdk_shell.py +++ b/dts/framework/remote_session/dpdk_shell.py @@ -82,6 +82,7 @@ def __init__( ascending_cores: bool = True, append_prefix_timestamp: bool = True, app_params: EalParams = EalParams(), + name: str | None = None, ) -> None: """Extends :meth:`~.interactive_shell.InteractiveShell.__init__`. @@ -96,7 +97,7 @@ def __init__( append_prefix_timestamp, ) - super().__init__(node, privileged, timeout, app_params) + super().__init__(node, privileged, timeout, app_params, name) def _update_real_path(self, path: PurePath) -> None: """Extends :meth:`~.interactive_shell.InteractiveShell._update_real_path`. diff --git a/dts/framework/remote_session/single_active_interactive_shell.py b/dts/framework/remote_session/single_active_interactive_shell.py index 38318aa764..77a4dcefdf 100644 --- a/dts/framework/remote_session/single_active_interactive_shell.py +++ b/dts/framework/remote_session/single_active_interactive_shell.py @@ -32,7 +32,7 @@ InteractiveSSHSessionDeadError, InteractiveSSHTimeoutError, ) -from framework.logger import DTSLogger +from framework.logger import DTSLogger, get_dts_logger from framework.params import Params from framework.settings import SETTINGS from framework.testbed_model.node import Node @@ -92,6 +92,7 @@ def __init__( privileged: bool = False, timeout: float = SETTINGS.timeout, app_params: Params = Params(), + name: str | None = None, ) -> None: """Create an SSH channel during initialization. @@ -102,9 +103,13 @@ def __init__( shell. This timeout is for collecting output, so if reading from the buffer and no output is gathered within the timeout, an exception is thrown. app_params: The command line parameters to be passed to the application on startup. + name: Name for the interactive shell to use for logging. This name will be appended to + the name of the underlying node which it is running on. """ self._node = node - self._logger = node._logger + if name is None: + name = type(self).__name__ + self._logger = get_dts_logger(f"{node.name}.{name}") self._app_params = app_params self._privileged = privileged self._timeout = timeout diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py index eda6eb320f..43e9f56517 100644 --- a/dts/framework/remote_session/testpmd_shell.py +++ b/dts/framework/remote_session/testpmd_shell.py @@ -604,6 +604,7 @@ def __init__( lcore_filter_specifier: LogicalCoreCount | LogicalCoreList = LogicalCoreCount(), ascending_cores: bool = True, append_prefix_timestamp: bool = True, + name: str | None = None, **app_params: Unpack[TestPmdParamsDict], ) -> None: """Overrides :meth:`~.dpdk_shell.DPDKShell.__init__`. Changes app_params to kwargs.""" @@ -615,6 +616,7 @@ def __init__( ascending_cores, append_prefix_timestamp, TestPmdParams(**app_params), + name, ) def start(self, verify: bool = True) -> None: diff --git a/dts/framework/testbed_model/traffic_generator/scapy.py b/dts/framework/testbed_model/traffic_generator/scapy.py index 08e1f4ae7e..13fc1107aa 100644 --- a/dts/framework/testbed_model/traffic_generator/scapy.py +++ b/dts/framework/testbed_model/traffic_generator/scapy.py @@ -261,7 +261,9 @@ def __init__(self, tg_node: Node, config: ScapyTrafficGeneratorConfig): self._tg_node.config.os == OS.linux ), "Linux is the only supported OS for scapy traffic generation" - self.session = PythonShell(self._tg_node, timeout=5, privileged=True) + self.session = PythonShell( + self._tg_node, timeout=5, privileged=True, name="ScapyXMLRPCServer" + ) self.session.start_application()