From patchwork Wed May 1 16:16:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jeremy Spewock X-Patchwork-Id: 139770 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 070F843F5D; Wed, 1 May 2024 18:17:28 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CA7BB402C7; Wed, 1 May 2024 18:17:27 +0200 (CEST) Received: from mail-il1-f228.google.com (mail-il1-f228.google.com [209.85.166.228]) by mails.dpdk.org (Postfix) with ESMTP id 143DF402C7 for ; Wed, 1 May 2024 18:17:27 +0200 (CEST) Received: by mail-il1-f228.google.com with SMTP id e9e14a558f8ab-36c6b4d7dc8so2213275ab.2 for ; Wed, 01 May 2024 09:17:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1714580246; x=1715185046; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EtlyOdDnX/Jty1BYDfb2e3EDOlPtLNciyHxYKgLd28E=; b=P7FKdWeSBwjpnhrMsc6ZouLhShZrYTuTbG/m6bK49T32m3R0VrPsmcn95YS2Dd/MjM abjIDpoqb8YFTx8kLfjknkUUZXHRkaJeXIttmxb/StnHJVyR+L5IlQgdXqGfugi9dUqE 17hLwnK1kbeWbpo2MWESvAXcuNe+hOGEZbwQQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714580246; x=1715185046; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EtlyOdDnX/Jty1BYDfb2e3EDOlPtLNciyHxYKgLd28E=; b=Nf9VE+lRXoqTuNABbEXHVyN1VMJ/fyxtgzyC2xgQQglxXAeVojtZruMxaJ9bO3AgZi YiqzA/uzvTc7m/77H8p5DuYaL+BNO5qHUJxncMqaGNUrWzyKgsoJ2EwqBSVDuxAamnSg 1ahMkk3648igBOhcRFeKkSFS61HOURY1SOBXYtnMhuCfyola13ux9zz2X8cubQqKEova M+g3tkSBOvGnFhGtohXmnFgAVvzJL8qajqlbfJSrWG9liCPB0mj+nCFeCnnamWv0GVro qZ+biaHdm5xkILOUI9cNTQFVTbh6chWmac63gwcuI1adUSJPmtNy5RBlPSHUQOjRb6GM diPQ== X-Gm-Message-State: AOJu0Yw6QYf68QHBOPPsz+9X8bRBX6hgLlTtT+w0JCHR/IfljPNZ5NJm m+2YRfxfHm+WWzkzLcWt9wXWDMWBGffUPFiQLRDVdHZITbYxPb6Cqobp5CpM8O0TIIc0Klrq+YY uSkNulg2wPnKYM/Rp25PVOodpOck/5w+H X-Google-Smtp-Source: AGHT+IEd0STBNdEhAXxUvOsTLXJWQ7PgJhTiiRzC7jvPqtAPBCOUCmBbch9G7whpKxCcyHwzdCT90ciaqX/J X-Received: by 2002:a92:cdaf:0:b0:36c:3ad4:ef5a with SMTP id g15-20020a92cdaf000000b0036c3ad4ef5amr3726170ild.27.1714580246242; Wed, 01 May 2024 09:17:26 -0700 (PDT) Received: from postal.iol.unh.edu (postal.iol.unh.edu. [132.177.123.84]) by smtp-relay.gmail.com with ESMTPS id r28-20020a638f5c000000b005dc4202b40bsm1863183pgn.6.2024.05.01.09.17.25 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 01 May 2024 09:17:26 -0700 (PDT) X-Relaying-Domain: iol.unh.edu Received: from iol.unh.edu (unknown [IPv6:2606:4100:3880:1271:90f9:1b64:f6e6:867f]) by postal.iol.unh.edu (Postfix) with ESMTP id 3CEBE6052514; Wed, 1 May 2024 12:17:25 -0400 (EDT) From: jspewock@iol.unh.edu To: Luca.Vizzarro@arm.com, wathsala.vithanage@arm.com, yoan.picchi@foss.arm.com, juraj.linkes@pantheon.tech, paul.szczepanek@arm.com, probb@iol.unh.edu, thomas@monjalon.net, Honnappa.Nagarahalli@arm.com Cc: dev@dpdk.org, Jeremy Spewock Subject: [PATCH v2 1/3] dts: Improve output gathering in interactive shells Date: Wed, 1 May 2024 12:16:21 -0400 Message-ID: <20240501161623.26672-2-jspewock@iol.unh.edu> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240501161623.26672-1-jspewock@iol.unh.edu> References: <20240312172558.11844-1-jspewock@iol.unh.edu> <20240501161623.26672-1-jspewock@iol.unh.edu> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Jeremy Spewock The current implementation of consuming output from interactive shells relies on being able to find an expected prompt somewhere within the output buffer after sending the command. This is useful in situations where the prompt does not appear in the output itself, but in some practical cases (such as the starting of an XML-RPC server for scapy) the prompt exists in one of the commands sent to the shell and this can cause the command to exit early and creates a race condition between the server starting and the first command being sent to the server. This patch addresses this problem by searching for a line that strictly ends with the provided prompt, rather than one that simply contains it, so that the detection that a command is finished is more consistent. It also adds a catch to detect when a command times out before finding the prompt or the underlying SSH session dies so that the exception can be wrapped into a more explicit one and be more consistent with the non-interactive shells. Bugzilla ID: 1359 Fixes: 88489c0501af ("dts: add smoke tests") Signed-off-by: Jeremy Spewock Reviewed-by: Luca Vizzarro Reviewed-by: Juraj Linkeš --- dts/framework/exception.py | 66 ++++++++++++------- .../remote_session/interactive_shell.py | 46 +++++++++---- 2 files changed, 77 insertions(+), 35 deletions(-) diff --git a/dts/framework/exception.py b/dts/framework/exception.py index cce1e0231a..627190c781 100644 --- a/dts/framework/exception.py +++ b/dts/framework/exception.py @@ -48,26 +48,6 @@ class DTSError(Exception): severity: ClassVar[ErrorSeverity] = ErrorSeverity.GENERIC_ERR -class SSHTimeoutError(DTSError): - """The SSH execution of a command timed out.""" - - #: - severity: ClassVar[ErrorSeverity] = ErrorSeverity.SSH_ERR - _command: str - - def __init__(self, command: str): - """Define the meaning of the first argument. - - Args: - command: The executed command. - """ - self._command = command - - def __str__(self) -> str: - """Add some context to the string representation.""" - return f"{self._command} execution timed out." - - class SSHConnectionError(DTSError): """An unsuccessful SSH connection.""" @@ -95,8 +75,42 @@ def __str__(self) -> str: return message -class SSHSessionDeadError(DTSError): - """The SSH session is no longer alive.""" +class _SSHTimeoutError(DTSError): + """The execution of a command via SSH timed out. + + This class is private and meant to be raised as its interactive and non-interactive variants. + """ + + #: + severity: ClassVar[ErrorSeverity] = ErrorSeverity.SSH_ERR + _command: str + + def __init__(self, command: str): + """Define the meaning of the first argument. + + Args: + command: The executed command. + """ + self._command = command + + def __str__(self) -> str: + """Add some context to the string representation.""" + return f"{self._command} execution timed out." + + +class SSHTimeoutError(_SSHTimeoutError): + """The execution of a command on a non-interactive SSH session timed out.""" + + +class InteractiveSSHTimeoutError(_SSHTimeoutError): + """The execution of a command on an interactive SSH session timed out.""" + + +class _SSHSessionDeadError(DTSError): + """The SSH session is no longer alive. + + This class is private and meant to be raised as its interactive and non-interactive variants. + """ #: severity: ClassVar[ErrorSeverity] = ErrorSeverity.SSH_ERR @@ -115,6 +129,14 @@ def __str__(self) -> str: return f"SSH session with {self._host} has died." +class SSHSessionDeadError(_SSHSessionDeadError): + """Non-interactive SSH session has died.""" + + +class InteractiveSSHSessionDeadError(_SSHSessionDeadError): + """Interactive SSH session as died.""" + + class ConfigurationError(DTSError): """An invalid configuration.""" diff --git a/dts/framework/remote_session/interactive_shell.py b/dts/framework/remote_session/interactive_shell.py index 5cfe202e15..0b0ccdb545 100644 --- a/dts/framework/remote_session/interactive_shell.py +++ b/dts/framework/remote_session/interactive_shell.py @@ -18,11 +18,17 @@ from pathlib import PurePath from typing import Callable, ClassVar -from paramiko import Channel, SSHClient, channel # type: ignore[import] +from paramiko import Channel, channel # type: ignore[import] +from framework.exception import ( + InteractiveSSHSessionDeadError, + InteractiveSSHTimeoutError, +) from framework.logger import DTSLogger from framework.settings import SETTINGS +from .interactive_remote_session import InteractiveRemoteSession + class InteractiveShell(ABC): """The base class for managing interactive shells. @@ -34,7 +40,7 @@ class InteractiveShell(ABC): session. """ - _interactive_session: SSHClient + _interactive_session: InteractiveRemoteSession _stdin: channel.ChannelStdinFile _stdout: channel.ChannelFile _ssh_channel: Channel @@ -60,7 +66,7 @@ class InteractiveShell(ABC): def __init__( self, - interactive_session: SSHClient, + interactive_session: InteractiveRemoteSession, logger: DTSLogger, get_privileged_command: Callable[[str], str] | None, app_args: str = "", @@ -80,7 +86,7 @@ def __init__( and no output is gathered within the timeout, an exception is thrown. """ self._interactive_session = interactive_session - self._ssh_channel = self._interactive_session.invoke_shell() + self._ssh_channel = self._interactive_session.session.invoke_shell() self._stdin = self._ssh_channel.makefile_stdin("w") self._stdout = self._ssh_channel.makefile("r") self._ssh_channel.settimeout(timeout) @@ -124,20 +130,34 @@ def send_command(self, command: str, prompt: str | None = None) -> str: Returns: All output in the buffer before expected string. + + Raises: + InteractiveSSHSessionDeadError: The session died while executing the command. + InteractiveSSHTimeoutError: If command was sent but prompt could not be found in + the output before the timeout. """ self._logger.info(f"Sending: '{command}'") if prompt is None: prompt = self._default_prompt - self._stdin.write(f"{command}{self._command_extra_chars}\n") - self._stdin.flush() out: str = "" - for line in self._stdout: - out += line - if prompt in line and not line.rstrip().endswith( - command.rstrip() - ): # ignore line that sent command - break - self._logger.debug(f"Got output: {out}") + try: + self._stdin.write(f"{command}{self._command_extra_chars}\n") + self._stdin.flush() + for line in self._stdout: + out += line + if line.rstrip().endswith(prompt): + break + except TimeoutError as e: + self._logger.exception(e) + self._logger.debug( + f"Prompt ({prompt}) was not found in output from command before timeout." + ) + raise InteractiveSSHTimeoutError(command) from e + except OSError as e: + self._logger.exception(e) + raise InteractiveSSHSessionDeadError(self._interactive_session.hostname) from e + finally: + self._logger.debug(f"Got output: {out}") return out def close(self) -> None: From patchwork Wed May 1 16:16:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jeremy Spewock X-Patchwork-Id: 139771 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DEBE743F5D; Wed, 1 May 2024 18:17:33 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2ABCD402E4; Wed, 1 May 2024 18:17:30 +0200 (CEST) Received: from mail-qk1-f227.google.com (mail-qk1-f227.google.com [209.85.222.227]) by mails.dpdk.org (Postfix) with ESMTP id 2ACC8402E1 for ; Wed, 1 May 2024 18:17:29 +0200 (CEST) Received: by mail-qk1-f227.google.com with SMTP id af79cd13be357-78f03917484so477818785a.3 for ; Wed, 01 May 2024 09:17:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1714580248; x=1715185048; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=W43tcSoEIDmhOS6XRsVDQlQAWz7Lh27HnfQeI3st7ic=; b=ddRoPKX+A5+KtxRA3jZYx37aq/OxbiMEKjnzpA9Mg5DaJQ/cwAyK5F+tQztgwmLVx9 A8n77bx5Ao7IzFnLFlhP4/GOTga8dKTLJOayX6PLWsCD5u/8LkUvBg0reIZgh2jNfiQ6 54LAE3wggHxicMJ9XqxY5ls5K2r0/vw/LUWc0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714580248; x=1715185048; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=W43tcSoEIDmhOS6XRsVDQlQAWz7Lh27HnfQeI3st7ic=; b=IeG8U8SLvedog7i79px9K58NWTwcttClb4izbTuFSNZ2wpYXk8XsG5blJe/+K5svxc dUggd9o6jqCNyiniZfJPAeWhD2sGh0B1qp47nDRjbmh8/svFGrHshBHw0lwVtZbgs1ks 04CIfX7rObyS4j4eOLtP1wy+6yn69Aok22uNaX3G8/bIClwlb46M9vsRlRTPdVzHgxb8 qE++sbuNHVUj1rSwjg5sb1RN8flvX0vGThPMfxeDaCtzj5St7lMkqQWwAuotRh85Ijdp 3W0DY0c5Gg4POC554taVKhfomObmEZSIQkAzT7Rnh27QfPN0rduA8FPjZVFY4AFOlKis hmKw== X-Gm-Message-State: AOJu0YyTcGttmg+S9YrgeUr0e9vLXeFLarCgD7ZQpDE7zMkb7fidUhT+ eM0CRDTkkDYxXwwI6ORblk2eUCJgFipwK41jMiygdUKlqYlyccBZgAM5QjHHqRSS1tBpfOoJlU3 D74lvceK54ioqhfzPFFkX88QX0gR4GGKB X-Google-Smtp-Source: AGHT+IFlyGp9mw1I3iIxx44ym18Y8NqO1ZSgQaXWVIK42a9rOjw+GzOJe2tj/QY5zbxAqtAkef6NUPh1fa/b X-Received: by 2002:a05:6214:ca3:b0:6a0:9770:39c2 with SMTP id s3-20020a0562140ca300b006a0977039c2mr2546732qvs.54.1714580248495; Wed, 01 May 2024 09:17:28 -0700 (PDT) Received: from postal.iol.unh.edu (postal.iol.unh.edu. [132.177.123.84]) by smtp-relay.gmail.com with ESMTPS id r5-20020a0cf605000000b006a0f7e9e0a5sm74321qvm.23.2024.05.01.09.17.28 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 01 May 2024 09:17:28 -0700 (PDT) X-Relaying-Domain: iol.unh.edu Received: from iol.unh.edu (unknown [IPv6:2606:4100:3880:1271:90f9:1b64:f6e6:867f]) by postal.iol.unh.edu (Postfix) with ESMTP id 0D3936052514; Wed, 1 May 2024 12:17:28 -0400 (EDT) From: jspewock@iol.unh.edu To: Luca.Vizzarro@arm.com, wathsala.vithanage@arm.com, yoan.picchi@foss.arm.com, juraj.linkes@pantheon.tech, paul.szczepanek@arm.com, probb@iol.unh.edu, thomas@monjalon.net, Honnappa.Nagarahalli@arm.com Cc: dev@dpdk.org, Jeremy Spewock Subject: [PATCH v2 2/3] dts: Add missing docstring from XML-RPC server Date: Wed, 1 May 2024 12:16:22 -0400 Message-ID: <20240501161623.26672-3-jspewock@iol.unh.edu> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240501161623.26672-1-jspewock@iol.unh.edu> References: <20240312172558.11844-1-jspewock@iol.unh.edu> <20240501161623.26672-1-jspewock@iol.unh.edu> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Jeremy Spewock When this XML-RPC server implementation was added, the docstring had to be shortened in order to reduce the chances of this race condition being encountered. Now that this race condition issue is resolved, the full docstring can be restored. Signed-off-by: Jeremy Spewock Reviewed-by: Luca Vizzarro Reviewed-by: Juraj Linkeš --- .../testbed_model/traffic_generator/scapy.py | 46 ++++++++++++++++++- 1 file changed, 45 insertions(+), 1 deletion(-) diff --git a/dts/framework/testbed_model/traffic_generator/scapy.py b/dts/framework/testbed_model/traffic_generator/scapy.py index df3069d516..d0e0a7c64e 100644 --- a/dts/framework/testbed_model/traffic_generator/scapy.py +++ b/dts/framework/testbed_model/traffic_generator/scapy.py @@ -128,9 +128,53 @@ def scapy_send_packets(xmlrpc_packets: list[xmlrpc.client.Binary], send_iface: s class QuittableXMLRPCServer(SimpleXMLRPCServer): - """Basic XML-RPC server. + r"""Basic XML-RPC server. The server may be augmented by functions serializable by the :mod:`marshal` module. + + Example: + :: + + def hello_world(): + # to be sent to the XML-RPC server + print("Hello World!") + + # start the XML-RPC server on the remote node + # the example assumes you're already connect to a tg_node + # this is done by starting a Python shell on the remote node + from framework.remote_session import PythonShell + session = tg_node.create_interactive_shell(PythonShell, timeout=5, privileged=True) + + # then importing the modules needed to run the server + # and the modules for any functions later added to the server + session.send_command("import xmlrpc") + session.send_command("from xmlrpc.server import SimpleXMLRPCServer") + + # sending the source code of this class to the Python shell + from xmlrpc.server import SimpleXMLRPCServer + src = inspect.getsource(QuittableXMLRPCServer) + src = "\n".join([l for l in src.splitlines() if not l.isspace() and l != ""]) + spacing = "\n" * 4 + session.send_command(spacing + src + spacing) + + # then starting the server with: + command = "s = QuittableXMLRPCServer(('0.0.0.0', {listen_port}));s.serve_forever()" + session.send_command(command, "XMLRPC OK") + + # now the server is running on the remote node and we can add functions to it + # first connect to the server from the execution node + import xmlrpc.client + server_url = f"http://{tg_node.config.hostname}:8000" + rpc_server_proxy = xmlrpc.client.ServerProxy(server_url) + + # get the function bytes to send + import marshal + function_bytes = marshal.dumps(hello_world.__code__) + rpc_server_proxy.add_rpc_function(hello_world.__name__, function_bytes) + + # now we can execute the function on the server + xmlrpc_binary_recv: xmlrpc.client.Binary = rpc_server_proxy.hello_world() + print(str(xmlrpc_binary_recv)) """ def __init__(self, *args, **kwargs): From patchwork Wed May 1 16:16:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jeremy Spewock X-Patchwork-Id: 139772 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 59B7843F5D; Wed, 1 May 2024 18:17:41 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 698F340633; Wed, 1 May 2024 18:17:33 +0200 (CEST) Received: from mail-qv1-f97.google.com (mail-qv1-f97.google.com [209.85.219.97]) by mails.dpdk.org (Postfix) with ESMTP id C5472402A7 for ; Wed, 1 May 2024 18:17:31 +0200 (CEST) Received: by mail-qv1-f97.google.com with SMTP id 6a1803df08f44-69b514d3cf4so75967166d6.0 for ; Wed, 01 May 2024 09:17:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1714580251; x=1715185051; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fyUAKes1V0sUW115fIJqfu/xF5FBEoeRWIDtsRAc5gI=; b=ZbY+ylTO1dpqK5dHWIC7BXKOghIbrk7JeGTuTXhUB3ZfGPtr3DC/o+dBptPtpuIfcv siKxXZJFHLeDVvFxQgbZ5SdhMC6F8FSRc6KR4FjFjC84Oz3JNUncdCHGCo/sCCznZAb1 AODfIRtmUYswophGig4xLn5tvqJ9FMkyd9UMw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714580251; x=1715185051; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fyUAKes1V0sUW115fIJqfu/xF5FBEoeRWIDtsRAc5gI=; b=NaYThfTneGhwrgzZf/JFGkwCI3hwW3KK3PvxZ+0C7ie8T6VCZifne49mTOXj5oOzYn 8XW95X4sPtw3oFkVaPSyXB/aZ9AsJx1mfAsE4i9JwMcnoszhyIxM9+9HBn/yfnVUQw1z D9EoYowjjyHSWgrOrTB1QjOaR6WDml62fXEBUvNubK7jRnXx7DqYLx7xCaEkh9jUTjgE Ms0DHuc7TaOKrDII2+sxxnlg65VhqNejblxRXHUKSzZSC+A9JwcPcyqMpBv/4DMhhXBf rJmZ5394fRQbleTlC6jJfDRbexxeXHBL5t9OmEc+TbYiWHimSa1PXcYP5LAxh9tKrtRE 4lBQ== X-Gm-Message-State: AOJu0YxMjx9wDbxPVgPLKe2j3GrAVgVmlvV8HKMCsFSM+Fu7T9TJC5nn 7y4wSyVOu+8yc+7I7HE+kYUoMGEDkdInaak/NveHy8G5hbY1jvDOivl/ehDdte1xdKr9oPEfes/ 0O2tSICxZu/iJr3GsDXVPTSXj8xcjR/W7 X-Google-Smtp-Source: AGHT+IGZbOtz8MI9dN00EPplcExVa4hmN6eu8W//whKKe6gkjm+u4PowNpanKrEfctGY8we1W1L3Sv9+XGjN X-Received: by 2002:a05:6214:21ab:b0:6a0:cb4d:7948 with SMTP id t11-20020a05621421ab00b006a0cb4d7948mr2738077qvc.40.1714580251245; Wed, 01 May 2024 09:17:31 -0700 (PDT) Received: from postal.iol.unh.edu (postal.iol.unh.edu. [132.177.123.84]) by smtp-relay.gmail.com with ESMTPS id b6-20020a0cfe66000000b006a0acd9db7fsm1139541qvv.69.2024.05.01.09.17.31 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 01 May 2024 09:17:31 -0700 (PDT) X-Relaying-Domain: iol.unh.edu Received: from iol.unh.edu (unknown [IPv6:2606:4100:3880:1271:90f9:1b64:f6e6:867f]) by postal.iol.unh.edu (Postfix) with ESMTP id CB66D6052514; Wed, 1 May 2024 12:17:30 -0400 (EDT) From: jspewock@iol.unh.edu To: Luca.Vizzarro@arm.com, wathsala.vithanage@arm.com, yoan.picchi@foss.arm.com, juraj.linkes@pantheon.tech, paul.szczepanek@arm.com, probb@iol.unh.edu, thomas@monjalon.net, Honnappa.Nagarahalli@arm.com Cc: dev@dpdk.org, Jeremy Spewock Subject: [PATCH v2 3/3] dts: Improve logging for interactive shells Date: Wed, 1 May 2024 12:16:23 -0400 Message-ID: <20240501161623.26672-4-jspewock@iol.unh.edu> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240501161623.26672-1-jspewock@iol.unh.edu> References: <20240312172558.11844-1-jspewock@iol.unh.edu> <20240501161623.26672-1-jspewock@iol.unh.edu> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Jeremy Spewock The messages being logged by interactive shells currently are using the same logger as the node they were created from. Because of this, when sending interactive commands, the logs make no distinction between when you are sending a command directly to the host and when you are using an interactive shell on the host. This change adds names to interactive shells so that they are able to use their own loggers with distinct names. Signed-off-by: Jeremy Spewock Reviewed-by: Luca Vizzarro Reviewed-by: Juraj Linkeš --- dts/framework/remote_session/interactive_shell.py | 9 +++++---- dts/framework/testbed_model/node.py | 7 +++++++ dts/framework/testbed_model/os_session.py | 6 ++++-- dts/framework/testbed_model/sut_node.py | 7 ++++++- dts/framework/testbed_model/traffic_generator/scapy.py | 2 +- 5 files changed, 23 insertions(+), 8 deletions(-) diff --git a/dts/framework/remote_session/interactive_shell.py b/dts/framework/remote_session/interactive_shell.py index 0b0ccdb545..eb9c9b6843 100644 --- a/dts/framework/remote_session/interactive_shell.py +++ b/dts/framework/remote_session/interactive_shell.py @@ -24,7 +24,7 @@ InteractiveSSHSessionDeadError, InteractiveSSHTimeoutError, ) -from framework.logger import DTSLogger +from framework.logger import DTSLogger, get_dts_logger from framework.settings import SETTINGS from .interactive_remote_session import InteractiveRemoteSession @@ -66,8 +66,8 @@ class InteractiveShell(ABC): def __init__( self, + name: str, interactive_session: InteractiveRemoteSession, - logger: DTSLogger, get_privileged_command: Callable[[str], str] | None, app_args: str = "", timeout: float = SETTINGS.timeout, @@ -75,8 +75,9 @@ def __init__( """Create an SSH channel during initialization. Args: + name: Name for the interactive shell to use for logging. This name will be appended to + the name of the underlying node which it is running on. interactive_session: The SSH session dedicated to interactive shells. - logger: The logger instance this session will use. get_privileged_command: A method for modifying a command to allow it to use elevated privileges. If :data:`None`, the application will not be started with elevated privileges. @@ -91,7 +92,7 @@ def __init__( self._stdout = self._ssh_channel.makefile("r") self._ssh_channel.settimeout(timeout) self._ssh_channel.set_combine_stderr(True) # combines stdout and stderr streams - self._logger = logger + self._logger = get_dts_logger(f"{interactive_session._node_config.name}.{name}") self._timeout = timeout self._app_args = app_args self._start_application(get_privileged_command) diff --git a/dts/framework/testbed_model/node.py b/dts/framework/testbed_model/node.py index 74061f6262..a5beeae7e7 100644 --- a/dts/framework/testbed_model/node.py +++ b/dts/framework/testbed_model/node.py @@ -199,6 +199,7 @@ def create_interactive_shell( shell_cls: Type[InteractiveShellType], timeout: float = SETTINGS.timeout, privileged: bool = False, + name: str = "", app_args: str = "", ) -> InteractiveShellType: """Factory for interactive session handlers. @@ -210,6 +211,8 @@ def create_interactive_shell( timeout: Timeout for reading output from the SSH channel. If you are reading from the buffer and don't receive any data within the timeout it will throw an error. privileged: Whether to run the shell with administrative privileges. + name: The name to use for the interactive application in the logs. If not specified, + this will default to the name of `shell_cls`. app_args: The arguments to be passed to the application. Returns: @@ -218,10 +221,14 @@ def create_interactive_shell( if not shell_cls.dpdk_app: shell_cls.path = self.main_session.join_remote_path(shell_cls.path) + if name == "": + name = shell_cls.__name__ + return self.main_session.create_interactive_shell( shell_cls, timeout, privileged, + name, app_args, ) diff --git a/dts/framework/testbed_model/os_session.py b/dts/framework/testbed_model/os_session.py index d5bf7e0401..1f6d8257ed 100644 --- a/dts/framework/testbed_model/os_session.py +++ b/dts/framework/testbed_model/os_session.py @@ -134,6 +134,7 @@ def create_interactive_shell( shell_cls: Type[InteractiveShellType], timeout: float, privileged: bool, + name: str, app_args: str, ) -> InteractiveShellType: """Factory for interactive session handlers. @@ -146,14 +147,15 @@ def create_interactive_shell( reading from the buffer and don't receive any data within the timeout it will throw an error. privileged: Whether to run the shell with administrative privileges. + name: Name for the shell to use in the logs. app_args: The arguments to be passed to the application. Returns: An instance of the desired interactive application shell. """ return shell_cls( - self.interactive_session.session, - self._logger, + name, + self.interactive_session, self._get_privileged_command if privileged else None, app_args, timeout, diff --git a/dts/framework/testbed_model/sut_node.py b/dts/framework/testbed_model/sut_node.py index 97aa26d419..0bddef6f44 100644 --- a/dts/framework/testbed_model/sut_node.py +++ b/dts/framework/testbed_model/sut_node.py @@ -442,6 +442,7 @@ def create_interactive_shell( shell_cls: Type[InteractiveShellType], timeout: float = SETTINGS.timeout, privileged: bool = False, + name: str = "", app_parameters: str = "", eal_parameters: EalParameters | None = None, ) -> InteractiveShellType: @@ -459,6 +460,8 @@ def create_interactive_shell( reading from the buffer and don't receive any data within the timeout it will throw an error. privileged: Whether to run the shell with administrative privileges. + name: The name to use for the interactive application in the logs. If not specified, + this will default to the name `shell_cls`. eal_parameters: List of EAL parameters to use to launch the app. If this isn't provided or an empty string is passed, it will default to calling :meth:`create_eal_parameters`. @@ -478,7 +481,9 @@ def create_interactive_shell( self.remote_dpdk_build_dir, shell_cls.path ) - return super().create_interactive_shell(shell_cls, timeout, privileged, app_parameters) + return super().create_interactive_shell( + shell_cls, timeout, privileged, name, app_parameters + ) def bind_ports_to_driver(self, for_dpdk: bool = True) -> None: """Bind all ports on the SUT to a driver. diff --git a/dts/framework/testbed_model/traffic_generator/scapy.py b/dts/framework/testbed_model/traffic_generator/scapy.py index d0e0a7c64e..5676235119 100644 --- a/dts/framework/testbed_model/traffic_generator/scapy.py +++ b/dts/framework/testbed_model/traffic_generator/scapy.py @@ -262,7 +262,7 @@ def __init__(self, tg_node: Node, config: ScapyTrafficGeneratorConfig): ), "Linux is the only supported OS for scapy traffic generation" self.session = self._tg_node.create_interactive_shell( - PythonShell, timeout=5, privileged=True + PythonShell, timeout=5, privileged=True, name="ScapyXMLRPCServer" ) # import libs in remote python console