[v8,0/9] dts: ssh connection to a node

All the necessary code needed to connect to a node in a topology with
a bit more, such as basic logging and some extra useful methods.

To run the code, modify the config file, conf.yaml and execute ./main.py
from the root dts folder. Here's an example config:
executions:
  - system_under_test: "SUT 1"
nodes:
  - name: "SUT 1"
    hostname: 127.0.0.1
    user: root

The framework will use the user's SSH key to authenticate. User password
can be specified, in which case it will be used, but it's strongly
discouraged.

There are configuration files with documentation that help with setting up
the execution/development environment.

The code only connects to a node. You'll see logs emitted to console
saying where DTS connected.

There's only a bit of documentation, as there's not much to document.
We'll add some real docs when there's enough functionality to document,
when the HelloWorld testcases is in (point 4 in our roadmap below). What
will be documented later is runtime dependencies and how to set up the DTS
control node environment.

This is our current roadmap:
1. Review this patchset and do the rest of the items in parallel, if
possible.
2. We have extracted the code needed to run the most basic testcase,
HelloWorld, which runs the DPDK Hello World application. We'll split
this along logical/functional boundaries and send after 1 is done.
3. Once we have 2 applied, we're planning on adding a basic functional
testcase - pf_smoke. This send a bit of traffic, so the big addition is
the software traffic generator, Scapy. There's some work already done on
Traffic generators we'll be sending as a dependence on this patch
series.
4. After 3, we'll add a basic performance testcase which doesn't use
Scapy, but Trex or Ixia instead.
5. This is far in the future, but at this point we should have all of
the core functionality in place. What then remains is adding the rest of
the testcases.

We're already working on items 2-4 and we may send more patches even
before this patch series is accepted if that's beneficial. The new
patches would then depend on this patch.

This patch, as well as all others in the pipeline, are the result of
extensive DTS workgroup review which happens internally. If you'd like
us to make it more public we'd have no problem with that.

v3:
Added project config files and developer tools.
Removed locks for parallel nodes, which are not needed now and will be
implemented much later (in a different patch).

v4:
Minor fixes - added missing Exception and utils function.

v5:
Reordered commits because the dependencies between commits changed.
Added more developer tools.
Added definitions of DTS testbed elements.
Reworked SSH implementation - split it so that the split between an
abstraction and the actual implementation is clearer.
Modified the directory structure to better organize the current and the
future code.

v6:
Minor code/grammar/style changes and a minor bugfix suggested by
Stanislaw.

v7:
Removed non-DTS Python specific configuration as well as Docker
configuration to review outside of this series.
Ported documentation to .rst and integrated it with DPDK docs.
Merged the three dev scripts into one.
Fixed ssh timeout and output issues.

v8:
Replaced toml lexer in docs with cfg.

Juraj Linkeš (8):
  dts: add project tools config
  dts: add developer tools
  dts: add basic logging facility
  dts: add remote session abstraction
  dts: add ssh session module
  dts: add node base class
  dts: add dts workflow module
  dts: add dts executable script

Owen Hilyard (1):
  dts: add config parser module

 .gitignore                                    |   3 +
 MAINTAINERS                                   |   5 +
 devtools/dts-check-format.sh                  |  87 +++++
 doc/guides/tools/dts.rst                      | 123 +++++++
 doc/guides/tools/index.rst                    |   1 +
 dts/conf.yaml                                 |   6 +
 dts/framework/__init__.py                     |   3 +
 dts/framework/config/__init__.py              |  99 +++++
 dts/framework/config/conf_yaml_schema.json    |  65 ++++
 dts/framework/dts.py                          |  67 ++++
 dts/framework/exception.py                    |  56 +++
 dts/framework/logger.py                       | 113 ++++++
 dts/framework/remote_session/__init__.py      |  14 +
 .../remote_session/remote_session.py          |  95 +++++
 dts/framework/remote_session/ssh_session.py   | 184 ++++++++++
 dts/framework/settings.py                     | 119 +++++++
 dts/framework/testbed_model/__init__.py       |   7 +
 dts/framework/testbed_model/node.py           |  62 ++++
 dts/framework/utils.py                        |  30 ++
 dts/main.py                                   |  23 ++
 dts/poetry.lock                               | 337 ++++++++++++++++++
 dts/pyproject.toml                            |  46 +++
 22 files changed, 1545 insertions(+)
 create mode 100755 devtools/dts-check-format.sh
 create mode 100644 doc/guides/tools/dts.rst
 create mode 100644 dts/conf.yaml
 create mode 100644 dts/framework/__init__.py
 create mode 100644 dts/framework/config/__init__.py
 create mode 100644 dts/framework/config/conf_yaml_schema.json
 create mode 100644 dts/framework/dts.py
 create mode 100644 dts/framework/exception.py
 create mode 100644 dts/framework/logger.py
 create mode 100644 dts/framework/remote_session/__init__.py
 create mode 100644 dts/framework/remote_session/remote_session.py
 create mode 100644 dts/framework/remote_session/ssh_session.py
 create mode 100644 dts/framework/settings.py
 create mode 100644 dts/framework/testbed_model/__init__.py
 create mode 100644 dts/framework/testbed_model/node.py
 create mode 100644 dts/framework/utils.py
 create mode 100755 dts/main.py
 create mode 100644 dts/poetry.lock
 create mode 100644 dts/pyproject.toml
  

04/11/2022 12:05, Juraj Linkeš:
> All the necessary code needed to connect to a node in a topology with
> a bit more, such as basic logging and some extra useful methods.

Applied, thanks.

That's the first step towards integration of DTS in DPDK repository.
Nice to see this becoming a reality.

[...]
> This is our current roadmap:
> 1. Review this patchset and do the rest of the items in parallel, if
> possible.
> 2. We have extracted the code needed to run the most basic testcase,
> HelloWorld, which runs the DPDK Hello World application. We'll split
> this along logical/functional boundaries and send after 1 is done.
> 3. Once we have 2 applied, we're planning on adding a basic functional
> testcase - pf_smoke. This send a bit of traffic, so the big addition is
> the software traffic generator, Scapy. There's some work already done on
> Traffic generators we'll be sending as a dependence on this patch
> series.
> 4. After 3, we'll add a basic performance testcase which doesn't use
> Scapy, but Trex or Ixia instead.
> 5. This is far in the future, but at this point we should have all of
> the core functionality in place. What then remains is adding the rest of
> the testcases.

Let's join the force and help making this project a pleasant tool.
  

<snip>

> 
> 04/11/2022 12:05, Juraj Linkeš:
> > All the necessary code needed to connect to a node in a topology with
> > a bit more, such as basic logging and some extra useful methods.
> 
> Applied, thanks.
> 
> That's the first step towards integration of DTS in DPDK repository.
> Nice to see this becoming a reality.
Thanks Thomas and the community for the reviews and suggestions. This is the first important step, hopefully we can continue the collaboration in the future releases.

> 
> [...]
> > This is our current roadmap:
> > 1. Review this patchset and do the rest of the items in parallel, if
> > possible.
> > 2. We have extracted the code needed to run the most basic testcase,
> > HelloWorld, which runs the DPDK Hello World application. We'll split
> > this along logical/functional boundaries and send after 1 is done.
> > 3. Once we have 2 applied, we're planning on adding a basic functional
> > testcase - pf_smoke. This send a bit of traffic, so the big addition
> > is the software traffic generator, Scapy. There's some work already
> > done on Traffic generators we'll be sending as a dependence on this
> > patch series.
> > 4. After 3, we'll add a basic performance testcase which doesn't use
> > Scapy, but Trex or Ixia instead.
> > 5. This is far in the future, but at this point we should have all of
> > the core functionality in place. What then remains is adding the rest
> > of the testcases.
> 
> Let's join the force and help making this project a pleasant tool.
>
  

On Wed, Nov 9, 2022 at 11:23 AM Honnappa Nagarahalli <
Honnappa.Nagarahalli@arm.com> wrote:

> <snip>
>
> >
> > 04/11/2022 12:05, Juraj Linkeš:
> > > All the necessary code needed to connect to a node in a topology with
> > > a bit more, such as basic logging and some extra useful methods.
> >
> > Applied, thanks.
> >
> > That's the first step towards integration of DTS in DPDK repository.
> > Nice to see this becoming a reality.
> Thanks Thomas and the community for the reviews and suggestions. This is
> the first important step, hopefully we can continue the collaboration in
> the future releases.
>
> >
> > [...]
> > > This is our current roadmap:
> > > 1. Review this patchset and do the rest of the items in parallel, if
> > > possible.
> > > 2. We have extracted the code needed to run the most basic testcase,
> > > HelloWorld, which runs the DPDK Hello World application. We'll split
> > > this along logical/functional boundaries and send after 1 is done.
> > > 3. Once we have 2 applied, we're planning on adding a basic functional
> > > testcase - pf_smoke. This send a bit of traffic, so the big addition
> > > is the software traffic generator, Scapy. There's some work already
> > > done on Traffic generators we'll be sending as a dependence on this
> > > patch series.
> > > 4. After 3, we'll add a basic performance testcase which doesn't use
> > > Scapy, but Trex or Ixia instead.
> > > 5. This is far in the future, but at this point we should have all of
> > > the core functionality in place. What then remains is adding the rest
> > > of the testcases.
> >
> > Let's join the force and help making this project a pleasant tool.
> >
>
>
I would also like to thank everyone for their reviews and suggestions
toward DTS. The primary goal of this rewrite is make it as easy as possible
to write robust tests so that DPDK can be heavily and reliably tested. This
rewrite allows us at the DTS Working Group to leverage more than 8 years of
experience with the prior DTS to help make it much easier for anyone to
write tests or set up their own testing, but we also want feedback from the
community, so please take a look and give us more feedback.

I would also like to remind everyone that if you have any features that you
want to see in DTS you should bring them to us as soon as possible. We are
starting with support for NICs, but want to be able to branch out to test
all DPDK-supported devices. If you have concerns or required features that
you don't want to discuss on the mailing list for some reason, you can send
your concerns directly to me and I will make sure your needs are
represented in discussions around DTS.

Planned but Unimplemented Features:
* telnet connections for the DUT
* scapy traffic generator (low performance but easy to use)
* trex traffic generator (DPDK-based software traffic generator)
* IXIA traffic generator (hardware traffic generator)
* Traffic generator abstraction layer
* Automatic skipping of tests based on the hardware under test, os and
other factors
* Abstractions to test development to allow simple tests (send this list of
packets then expect this list, etc) to be written as easily as possible.
* Automatic setup and teardown of virtual machines for virtio testing
* The ability to cross compile for a given target on the system running DTS
(assuming relevant libraries/compilers are installed), then install onto
the system under test (intended for embedded systems or SOCs).
* Structured logging for automated analysis
* and many more