[v3,00/10] dts: add hello world testcase

Add code needed to run the HelloWorld testcase which just runs the hello
world dpdk application.

The patchset currently heavily refactors this original DTS code needed
to run the testcase:
* The whole architecture has been redone into more sensible class
  hierarchy
* DPDK build on the System under Test
* DPDK eal args construction, app running and shutting down
* SUT hugepage memory configuration
* Test runner
* Test results
* TestSuite class
* Test runner parts interfacing with TestSuite
* The HelloWorld testsuite itself

The code is divided into sub-packages, some of which are divided
further.

There patch may need to be divided into smaller chunks. If so, proposals
on where exactly to split it would be very helpful.

v3:
Finished refactoring everything in this patch, with test suite and test
results being the last parts.
Also changed the directory structure. It's now simplified and the
imports look much better.
I've also many many minor changes such as renaming variables here and
there.

Juraj Linkeš (10):
  dts: add node and os abstractions
  dts: add ssh command verification
  dts: add dpdk build on sut
  dts: add dpdk execution handling
  dts: add node memory setup
  dts: add test suite module
  dts: add hello world testplan
  dts: add hello world testsuite
  dts: add test suite config and runner
  dts: add test results module

 dts/conf.yaml                                 |  19 +-
 dts/framework/config/__init__.py              | 132 +++++++-
 dts/framework/config/arch.py                  |  57 ++++
 dts/framework/config/conf_yaml_schema.json    | 150 ++++++++-
 dts/framework/dts.py                          | 185 ++++++++--
 dts/framework/exception.py                    | 100 +++++-
 dts/framework/logger.py                       |  24 +-
 dts/framework/remote_session/__init__.py      |  30 +-
 dts/framework/remote_session/linux_session.py | 114 +++++++
 dts/framework/remote_session/os_session.py    | 177 ++++++++++
 dts/framework/remote_session/posix_session.py | 221 ++++++++++++
 .../remote_session/remote/__init__.py         |  16 +
 .../remote_session/remote/remote_session.py   | 155 +++++++++
 .../{ => remote}/ssh_session.py               |  91 ++++-
 .../remote_session/remote_session.py          |  95 ------
 dts/framework/settings.py                     |  79 ++++-
 dts/framework/test_result.py                  | 316 ++++++++++++++++++
 dts/framework/test_suite.py                   | 254 ++++++++++++++
 dts/framework/testbed_model/__init__.py       |  20 +-
 dts/framework/testbed_model/dpdk.py           |  78 +++++
 dts/framework/testbed_model/hw/__init__.py    |  27 ++
 dts/framework/testbed_model/hw/cpu.py         | 253 ++++++++++++++
 .../testbed_model/hw/virtual_device.py        |  16 +
 dts/framework/testbed_model/node.py           | 165 +++++++--
 dts/framework/testbed_model/sut_node.py       | 261 +++++++++++++++
 dts/framework/utils.py                        |  39 ++-
 dts/test_plans/hello_world_test_plan.rst      |  68 ++++
 dts/tests/TestSuite_hello_world.py            |  59 ++++
 28 files changed, 2998 insertions(+), 203 deletions(-)
 create mode 100644 dts/framework/config/arch.py
 create mode 100644 dts/framework/remote_session/linux_session.py
 create mode 100644 dts/framework/remote_session/os_session.py
 create mode 100644 dts/framework/remote_session/posix_session.py
 create mode 100644 dts/framework/remote_session/remote/__init__.py
 create mode 100644 dts/framework/remote_session/remote/remote_session.py
 rename dts/framework/remote_session/{ => remote}/ssh_session.py (65%)
 delete mode 100644 dts/framework/remote_session/remote_session.py
 create mode 100644 dts/framework/test_result.py
 create mode 100644 dts/framework/test_suite.py
 create mode 100644 dts/framework/testbed_model/dpdk.py
 create mode 100644 dts/framework/testbed_model/hw/__init__.py
 create mode 100644 dts/framework/testbed_model/hw/cpu.py
 create mode 100644 dts/framework/testbed_model/hw/virtual_device.py
 create mode 100644 dts/framework/testbed_model/sut_node.py
 create mode 100644 dts/test_plans/hello_world_test_plan.rst
 create mode 100644 dts/tests/TestSuite_hello_world.py
  

Everything looks good to me with the exception of some issues I ran into
with terminal codes. Setting TERM=dumb before running fixed it, but we
might want to set that inside of DTS since I can't think of a reason why we
would need colors or any of the other "fancy" features of the vt220, and
setting everything to teletype mode should make our lives easier when
parsing output.

I think that later on it might make sense to have "CPU" be a device class
like NIC or Cryptodev, but that can be revisited once we get closer to
interacting with hardware.

On Tue, Jan 17, 2023 at 10:49 AM Juraj Linkeš <juraj.linkes@pantheon.tech>
wrote:

> Add code needed to run the HelloWorld testcase which just runs the hello
> world dpdk application.
>
> The patchset currently heavily refactors this original DTS code needed
> to run the testcase:
> * The whole architecture has been redone into more sensible class
>   hierarchy
> * DPDK build on the System under Test
> * DPDK eal args construction, app running and shutting down
> * SUT hugepage memory configuration
> * Test runner
> * Test results
> * TestSuite class
> * Test runner parts interfacing with TestSuite
> * The HelloWorld testsuite itself
>
> The code is divided into sub-packages, some of which are divided
> further.
>
> There patch may need to be divided into smaller chunks. If so, proposals
> on where exactly to split it would be very helpful.
>
> v3:
> Finished refactoring everything in this patch, with test suite and test
> results being the last parts.
> Also changed the directory structure. It's now simplified and the
> imports look much better.
> I've also many many minor changes such as renaming variables here and
> there.
>
> Juraj Linkeš (10):
>   dts: add node and os abstractions
>   dts: add ssh command verification
>   dts: add dpdk build on sut
>   dts: add dpdk execution handling
>   dts: add node memory setup
>   dts: add test suite module
>   dts: add hello world testplan
>   dts: add hello world testsuite
>   dts: add test suite config and runner
>   dts: add test results module
>
>  dts/conf.yaml                                 |  19 +-
>  dts/framework/config/__init__.py              | 132 +++++++-
>  dts/framework/config/arch.py                  |  57 ++++
>  dts/framework/config/conf_yaml_schema.json    | 150 ++++++++-
>  dts/framework/dts.py                          | 185 ++++++++--
>  dts/framework/exception.py                    | 100 +++++-
>  dts/framework/logger.py                       |  24 +-
>  dts/framework/remote_session/__init__.py      |  30 +-
>  dts/framework/remote_session/linux_session.py | 114 +++++++
>  dts/framework/remote_session/os_session.py    | 177 ++++++++++
>  dts/framework/remote_session/posix_session.py | 221 ++++++++++++
>  .../remote_session/remote/__init__.py         |  16 +
>  .../remote_session/remote/remote_session.py   | 155 +++++++++
>  .../{ => remote}/ssh_session.py               |  91 ++++-
>  .../remote_session/remote_session.py          |  95 ------
>  dts/framework/settings.py                     |  79 ++++-
>  dts/framework/test_result.py                  | 316 ++++++++++++++++++
>  dts/framework/test_suite.py                   | 254 ++++++++++++++
>  dts/framework/testbed_model/__init__.py       |  20 +-
>  dts/framework/testbed_model/dpdk.py           |  78 +++++
>  dts/framework/testbed_model/hw/__init__.py    |  27 ++
>  dts/framework/testbed_model/hw/cpu.py         | 253 ++++++++++++++
>  .../testbed_model/hw/virtual_device.py        |  16 +
>  dts/framework/testbed_model/node.py           | 165 +++++++--
>  dts/framework/testbed_model/sut_node.py       | 261 +++++++++++++++
>  dts/framework/utils.py                        |  39 ++-
>  dts/test_plans/hello_world_test_plan.rst      |  68 ++++
>  dts/tests/TestSuite_hello_world.py            |  59 ++++
>  28 files changed, 2998 insertions(+), 203 deletions(-)
>  create mode 100644 dts/framework/config/arch.py
>  create mode 100644 dts/framework/remote_session/linux_session.py
>  create mode 100644 dts/framework/remote_session/os_session.py
>  create mode 100644 dts/framework/remote_session/posix_session.py
>  create mode 100644 dts/framework/remote_session/remote/__init__.py
>  create mode 100644 dts/framework/remote_session/remote/remote_session.py
>  rename dts/framework/remote_session/{ => remote}/ssh_session.py (65%)
>  delete mode 100644 dts/framework/remote_session/remote_session.py
>  create mode 100644 dts/framework/test_result.py
>  create mode 100644 dts/framework/test_suite.py
>  create mode 100644 dts/framework/testbed_model/dpdk.py
>  create mode 100644 dts/framework/testbed_model/hw/__init__.py
>  create mode 100644 dts/framework/testbed_model/hw/cpu.py
>  create mode 100644 dts/framework/testbed_model/hw/virtual_device.py
>  create mode 100644 dts/framework/testbed_model/sut_node.py
>  create mode 100644 dts/test_plans/hello_world_test_plan.rst
>  create mode 100644 dts/tests/TestSuite_hello_world.py
>
> --
> 2.30.2
>
>
  

On Thu, Jan 19, 2023 at 11:16 AM Owen Hilyard <ohilyard@iol.unh.edu> wrote:

> Everything looks good to me with the exception of some issues I ran into
> with terminal codes. Setting TERM=dumb before running fixed it, but we
> might want to set that inside of DTS since I can't think of a reason why we
> would need colors or any of the other "fancy" features of the vt220, and
> setting everything to teletype mode should make our lives easier when
> parsing output.
>
> I think that later on it might make sense to have "CPU" be a device class
> like NIC or Cryptodev, but that can be revisited once we get closer to
> interacting with hardware.
>
> On Tue, Jan 17, 2023 at 10:49 AM Juraj Linkeš <juraj.linkes@pantheon.tech>
> wrote:
>
>> Add code needed to run the HelloWorld testcase which just runs the hello
>> world dpdk application.
>>
>> The patchset currently heavily refactors this original DTS code needed
>> to run the testcase:
>> * The whole architecture has been redone into more sensible class
>>   hierarchy
>> * DPDK build on the System under Test
>> * DPDK eal args construction, app running and shutting down
>> * SUT hugepage memory configuration
>> * Test runner
>> * Test results
>> * TestSuite class
>> * Test runner parts interfacing with TestSuite
>> * The HelloWorld testsuite itself
>>
>> The code is divided into sub-packages, some of which are divided
>> further.
>>
>> There patch may need to be divided into smaller chunks. If so, proposals
>> on where exactly to split it would be very helpful.
>>
>> v3:
>> Finished refactoring everything in this patch, with test suite and test
>> results being the last parts.
>> Also changed the directory structure. It's now simplified and the
>> imports look much better.
>> I've also many many minor changes such as renaming variables here and
>> there.
>>
>> Juraj Linkeš (10):
>>   dts: add node and os abstractions
>>   dts: add ssh command verification
>>   dts: add dpdk build on sut
>>   dts: add dpdk execution handling
>>   dts: add node memory setup
>>   dts: add test suite module
>>   dts: add hello world testplan
>>   dts: add hello world testsuite
>>   dts: add test suite config and runner
>>   dts: add test results module
>>
>>  dts/conf.yaml                                 |  19 +-
>>  dts/framework/config/__init__.py              | 132 +++++++-
>>  dts/framework/config/arch.py                  |  57 ++++
>>  dts/framework/config/conf_yaml_schema.json    | 150 ++++++++-
>>  dts/framework/dts.py                          | 185 ++++++++--
>>  dts/framework/exception.py                    | 100 +++++-
>>  dts/framework/logger.py                       |  24 +-
>>  dts/framework/remote_session/__init__.py      |  30 +-
>>  dts/framework/remote_session/linux_session.py | 114 +++++++
>>  dts/framework/remote_session/os_session.py    | 177 ++++++++++
>>  dts/framework/remote_session/posix_session.py | 221 ++++++++++++
>>  .../remote_session/remote/__init__.py         |  16 +
>>  .../remote_session/remote/remote_session.py   | 155 +++++++++
>>  .../{ => remote}/ssh_session.py               |  91 ++++-
>>  .../remote_session/remote_session.py          |  95 ------
>>  dts/framework/settings.py                     |  79 ++++-
>>  dts/framework/test_result.py                  | 316 ++++++++++++++++++
>>  dts/framework/test_suite.py                   | 254 ++++++++++++++
>>  dts/framework/testbed_model/__init__.py       |  20 +-
>>  dts/framework/testbed_model/dpdk.py           |  78 +++++
>>  dts/framework/testbed_model/hw/__init__.py    |  27 ++
>>  dts/framework/testbed_model/hw/cpu.py         | 253 ++++++++++++++
>>  .../testbed_model/hw/virtual_device.py        |  16 +
>>  dts/framework/testbed_model/node.py           | 165 +++++++--
>>  dts/framework/testbed_model/sut_node.py       | 261 +++++++++++++++
>>  dts/framework/utils.py                        |  39 ++-
>>  dts/test_plans/hello_world_test_plan.rst      |  68 ++++
>>  dts/tests/TestSuite_hello_world.py            |  59 ++++
>>  28 files changed, 2998 insertions(+), 203 deletions(-)
>>  create mode 100644 dts/framework/config/arch.py
>>  create mode 100644 dts/framework/remote_session/linux_session.py
>>  create mode 100644 dts/framework/remote_session/os_session.py
>>  create mode 100644 dts/framework/remote_session/posix_session.py
>>  create mode 100644 dts/framework/remote_session/remote/__init__.py
>>  create mode 100644 dts/framework/remote_session/remote/remote_session.py
>>  rename dts/framework/remote_session/{ => remote}/ssh_session.py (65%)
>>  delete mode 100644 dts/framework/remote_session/remote_session.py
>>  create mode 100644 dts/framework/test_result.py
>>  create mode 100644 dts/framework/test_suite.py
>>  create mode 100644 dts/framework/testbed_model/dpdk.py
>>  create mode 100644 dts/framework/testbed_model/hw/__init__.py
>>  create mode 100644 dts/framework/testbed_model/hw/cpu.py
>>  create mode 100644 dts/framework/testbed_model/hw/virtual_device.py
>>  create mode 100644 dts/framework/testbed_model/sut_node.py
>>  create mode 100644 dts/test_plans/hello_world_test_plan.rst
>>  create mode 100644 dts/tests/TestSuite_hello_world.py
>>
>> --
>> 2.30.2
>>
>>
Tested-by: Patrick Robb <probb@iol.unh.edu>