From patchwork Fri Nov 4 11:05:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Juraj_Linke=C5=A1?= X-Patchwork-Id: 119478 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9EFDCA00C5; Fri, 4 Nov 2022 12:05:32 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 84EB442D10; Fri, 4 Nov 2022 12:05:32 +0100 (CET) Received: from lb.pantheon.sk (lb.pantheon.sk [46.229.239.20]) by mails.dpdk.org (Postfix) with ESMTP id 9665042D0E for ; Fri, 4 Nov 2022 12:05:31 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by lb.pantheon.sk (Postfix) with ESMTP id D0B181BA5B4; Fri, 4 Nov 2022 12:05:29 +0100 (CET) X-Virus-Scanned: amavisd-new at siecit.sk Received: from lb.pantheon.sk ([127.0.0.1]) by localhost (lb.pantheon.sk [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LMHqchizwRnk; Fri, 4 Nov 2022 12:05:26 +0100 (CET) Received: from entguard.lab.pantheon.local (unknown [46.229.239.141]) by lb.pantheon.sk (Postfix) with ESMTP id D4ECB1B68D8; Fri, 4 Nov 2022 12:05:25 +0100 (CET) From: =?utf-8?q?Juraj_Linke=C5=A1?= To: thomas@monjalon.net, Honnappa.Nagarahalli@arm.com, ohilyard@iol.unh.edu, lijuan.tu@intel.com, kda@semihalf.com, bruce.richardson@intel.com Cc: dev@dpdk.org, =?utf-8?q?Juraj_Linke=C5=A1?= Subject: [PATCH v8 0/9] dts: ssh connection to a node Date: Fri, 4 Nov 2022 11:05:14 +0000 Message-Id: <20221104110523.511367-1-juraj.linkes@pantheon.tech> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221103151934.450887-1-juraj.linkes@pantheon.tech> References: <20221103151934.450887-1-juraj.linkes@pantheon.tech> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org All the necessary code needed to connect to a node in a topology with a bit more, such as basic logging and some extra useful methods. To run the code, modify the config file, conf.yaml and execute ./main.py from the root dts folder. Here's an example config: executions: - system_under_test: "SUT 1" nodes: - name: "SUT 1" hostname: 127.0.0.1 user: root The framework will use the user's SSH key to authenticate. User password can be specified, in which case it will be used, but it's strongly discouraged. There are configuration files with documentation that help with setting up the execution/development environment. The code only connects to a node. You'll see logs emitted to console saying where DTS connected. There's only a bit of documentation, as there's not much to document. We'll add some real docs when there's enough functionality to document, when the HelloWorld testcases is in (point 4 in our roadmap below). What will be documented later is runtime dependencies and how to set up the DTS control node environment. This is our current roadmap: 1. Review this patchset and do the rest of the items in parallel, if possible. 2. We have extracted the code needed to run the most basic testcase, HelloWorld, which runs the DPDK Hello World application. We'll split this along logical/functional boundaries and send after 1 is done. 3. Once we have 2 applied, we're planning on adding a basic functional testcase - pf_smoke. This send a bit of traffic, so the big addition is the software traffic generator, Scapy. There's some work already done on Traffic generators we'll be sending as a dependence on this patch series. 4. After 3, we'll add a basic performance testcase which doesn't use Scapy, but Trex or Ixia instead. 5. This is far in the future, but at this point we should have all of the core functionality in place. What then remains is adding the rest of the testcases. We're already working on items 2-4 and we may send more patches even before this patch series is accepted if that's beneficial. The new patches would then depend on this patch. This patch, as well as all others in the pipeline, are the result of extensive DTS workgroup review which happens internally. If you'd like us to make it more public we'd have no problem with that. v3: Added project config files and developer tools. Removed locks for parallel nodes, which are not needed now and will be implemented much later (in a different patch). v4: Minor fixes - added missing Exception and utils function. v5: Reordered commits because the dependencies between commits changed. Added more developer tools. Added definitions of DTS testbed elements. Reworked SSH implementation - split it so that the split between an abstraction and the actual implementation is clearer. Modified the directory structure to better organize the current and the future code. v6: Minor code/grammar/style changes and a minor bugfix suggested by Stanislaw. v7: Removed non-DTS Python specific configuration as well as Docker configuration to review outside of this series. Ported documentation to .rst and integrated it with DPDK docs. Merged the three dev scripts into one. Fixed ssh timeout and output issues. v8: Replaced toml lexer in docs with cfg. Juraj Linkeš (8): dts: add project tools config dts: add developer tools dts: add basic logging facility dts: add remote session abstraction dts: add ssh session module dts: add node base class dts: add dts workflow module dts: add dts executable script Owen Hilyard (1): dts: add config parser module .gitignore | 3 + MAINTAINERS | 5 + devtools/dts-check-format.sh | 87 +++++ doc/guides/tools/dts.rst | 123 +++++++ doc/guides/tools/index.rst | 1 + dts/conf.yaml | 6 + dts/framework/__init__.py | 3 + dts/framework/config/__init__.py | 99 +++++ dts/framework/config/conf_yaml_schema.json | 65 ++++ dts/framework/dts.py | 67 ++++ dts/framework/exception.py | 56 +++ dts/framework/logger.py | 113 ++++++ dts/framework/remote_session/__init__.py | 14 + .../remote_session/remote_session.py | 95 +++++ dts/framework/remote_session/ssh_session.py | 184 ++++++++++ dts/framework/settings.py | 119 +++++++ dts/framework/testbed_model/__init__.py | 7 + dts/framework/testbed_model/node.py | 62 ++++ dts/framework/utils.py | 30 ++ dts/main.py | 23 ++ dts/poetry.lock | 337 ++++++++++++++++++ dts/pyproject.toml | 46 +++ 22 files changed, 1545 insertions(+) create mode 100755 devtools/dts-check-format.sh create mode 100644 doc/guides/tools/dts.rst create mode 100644 dts/conf.yaml create mode 100644 dts/framework/__init__.py create mode 100644 dts/framework/config/__init__.py create mode 100644 dts/framework/config/conf_yaml_schema.json create mode 100644 dts/framework/dts.py create mode 100644 dts/framework/exception.py create mode 100644 dts/framework/logger.py create mode 100644 dts/framework/remote_session/__init__.py create mode 100644 dts/framework/remote_session/remote_session.py create mode 100644 dts/framework/remote_session/ssh_session.py create mode 100644 dts/framework/settings.py create mode 100644 dts/framework/testbed_model/__init__.py create mode 100644 dts/framework/testbed_model/node.py create mode 100644 dts/framework/utils.py create mode 100755 dts/main.py create mode 100644 dts/poetry.lock create mode 100644 dts/pyproject.toml