[V1] tests/checksum_offload: Bugfix for bug 529

  added missing quote in template for scapy packets.
rearraged closing down scapy and testpmd to avoid issues with other test
cases.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
---
 tests/TestSuite_checksum_offload.py | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
  

Hello Ma,lihong,

While I was re-testing the entire suite to get the test log file for you, I
found a few more issues that occur when running the entire test suite at
once. I've included fixes for these but apparently checkpatch doesn't like
something about the patch, so I'll resubmit. Sorry about that. I've
included the log file after stripping some of the output. Due to how the
test was constructed, I wasn't able to use the normal method to run scapy
and had to do it through send_expect, which means that all of the colorings
in the shell were logged. I've done my best to remove all of the lines with
color symbols in them since they aren't needed to understand the output.

Sorry about messing up the submission again,
Owen

On Wed, Sep 9, 2020 at 5:08 PM Owen Hilyard <ohilyard@iol.unh.edu> wrote:

> added missing quote in template for scapy packets.
> rearraged closing down scapy and testpmd to avoid issues with other test
> cases.
>
> Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> ---
>  tests/TestSuite_checksum_offload.py | 11 ++++++-----
>  1 file changed, 6 insertions(+), 5 deletions(-)
>
> diff --git a/tests/TestSuite_checksum_offload.py
> b/tests/TestSuite_checksum_offload.py
> index c4a877d..f2e9878 100644
> --- a/tests/TestSuite_checksum_offload.py
> +++ b/tests/TestSuite_checksum_offload.py
> @@ -374,7 +374,7 @@ class TestChecksumOffload(TestCase):
>      def replay_pcap_file_on_tester(self, iface, packet_file_path):
>          self.tester.send_expect("scapy", ">>>")
>          self.scapy_exec(f"packets = rdpcap('{packet_file_path}')")
> -        self.scapy_exec(f"sendp(packets, iface={iface})")
> +        self.scapy_exec(f"sendp(packets, iface='{iface}')")
>          self.tester.send_expect("quit()", "# ")
>
>      def validate_packet_list_checksums(self, packets):
> @@ -703,7 +703,7 @@ class TestChecksumOffload(TestCase):
>          for l4 in l4_protos:
>              for chksum in "", "chksum=0xf":
>                  vf =
> self.send_pkt_expect_good_bad_from_flag_catch_failure(
> -                    f"eth/IP({chksum})/{l4}()/(X'*50)",
> +                    f"eth/IP({chksum})/{l4}()/('X'*50)",
>                      "PKT_RX_IP_CKSUM_", f"{l4}",
>                      should_pass=(chksum == ""))
>                  if vf is not None:
> @@ -846,14 +846,13 @@ class TestChecksumOffload(TestCase):
>          #                             if vf is not None:
>          #                                 verification_errors.append(vf)
>
> -        # tunneled inner
> +        self.tester.send_expect("quit", "#")
> +        self.dut.send_expect("stop", "testpmd>")
>
>          for err in verification_errors:
>              self.logger.error(str(err))
>          self.verify(len(verification_errors) == 0, "See previous output")
>
>         -        self.tester.send_expect("quit", "#")
> -        self.dut.send_expect("stop", "testpmd>")
>
>      def test_hardware_checksum_check_l4_tx(self):
>          self.checksum_enablehw(self.dut_ports[0])
> @@ -887,7 +886,9 @@ class TestChecksumOffload(TestCase):
>
>          error_messages =
> self.validate_packet_list_checksums(captured_packets)
>
> +        self.tester.send_expect("quit", "#")
>          self.dut.send_expect("stop", "testpmd>")
> +
>          if len(error_messages) != 0:
>              for error_msg in error_messages:
>                  self.logger.error(error_msg)
> --
> 2.25.1
>
>
  

Thanks response of that, in the log file, I find the result of new cases test_hardware_checksum_check_l4_rx and test_hardware_checksum_check_l4_tx are failed,
Is it expected result of now or there is bug of it ?

Regards,
Ma,lihong

From: Owen Hilyard <ohilyard@iol.unh.edu>
Sent: Thursday, September 10, 2020 5:14 AM
To: dts@dpdk.org; Ma, LihongX <lihongx.ma@intel.com>
Cc: Lincoln Lavoie <lylavoie@iol.unh.edu>
Subject: Re: [PATCH V1] tests/checksum_offload: Bugfix for bug 529

Hello Ma,lihong,

While I was re-testing the entire suite to get the test log file for you, I found a few more issues that occur when running the entire test suite at once. I've included fixes for these but apparently checkpatch doesn't like something about the patch, so I'll resubmit. Sorry about that. I've included the log file after stripping some of the output. Due to how the test was constructed, I wasn't able to use the normal method to run scapy and had to do it through send_expect, which means that all of the colorings in the shell were logged. I've done my best to remove all of the lines with color symbols in them since they aren't needed to understand the output.

Sorry about messing up the submission again,
Owen

On Wed, Sep 9, 2020 at 5:08 PM Owen Hilyard <ohilyard@iol.unh.edu<mailto:ohilyard@iol.unh.edu>> wrote:
added missing quote in template for scapy packets.
rearraged closing down scapy and testpmd to avoid issues with other test
cases.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu<mailto:ohilyard@iol.unh.edu>>
---
 tests/TestSuite_checksum_offload.py | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/tests/TestSuite_checksum_offload.py b/tests/TestSuite_checksum_offload.py
index c4a877d..f2e9878 100644
--- a/tests/TestSuite_checksum_offload.py
+++ b/tests/TestSuite_checksum_offload.py
@@ -374,7 +374,7 @@ class TestChecksumOffload(TestCase):
     def replay_pcap_file_on_tester(self, iface, packet_file_path):
         self.tester.send_expect("scapy", ">>>")
         self.scapy_exec(f"packets = rdpcap('{packet_file_path}')")
-        self.scapy_exec(f"sendp(packets, iface={iface})")
+        self.scapy_exec(f"sendp(packets, iface='{iface}')")
         self.tester.send_expect("quit()", "# ")

     def validate_packet_list_checksums(self, packets):
@@ -703,7 +703,7 @@ class TestChecksumOffload(TestCase):
         for l4 in l4_protos:
             for chksum in "", "chksum=0xf":
                 vf = self.send_pkt_expect_good_bad_from_flag_catch_failure(
-                    f"eth/IP({chksum})/{l4}()/(X'*50)",
+                    f"eth/IP({chksum})/{l4}()/('X'*50)",
                     "PKT_RX_IP_CKSUM_", f"{l4}",
                     should_pass=(chksum == ""))
                 if vf is not None:
@@ -846,14 +846,13 @@ class TestChecksumOffload(TestCase):
         #                             if vf is not None:
         #                                 verification_errors.append(vf)

-        # tunneled inner
+        self.tester.send_expect("quit", "#")
+        self.dut.send_expect("stop", "testpmd>")

         for err in verification_errors:
             self.logger.error(str(err))
         self.verify(len(verification_errors) == 0, "See previous output")

        -        self.tester.send_expect("quit", "#")
-        self.dut.send_expect("stop", "testpmd>")

     def test_hardware_checksum_check_l4_tx(self):
         self.checksum_enablehw(self.dut_ports[0])
@@ -887,7 +886,9 @@ class TestChecksumOffload(TestCase):

         error_messages = self.validate_packet_list_checksums(captured_packets)

+        self.tester.send_expect("quit", "#")
         self.dut.send_expect("stop", "testpmd>")
+
         if len(error_messages) != 0:
             for error_msg in error_messages:
                 self.logger.error(error_msg)
--
2.25.1
  

BTW, it is better increase the version number when we submit a new version of the patch, it will easy to know the latest version of the patch. eg: V1 ,V2, V3 ….

Regards,
Ma,lihong

From: Ma, LihongX
Sent: Thursday, September 10, 2020 9:46 AM
To: Owen Hilyard <ohilyard@iol.unh.edu>; dts@dpdk.org
Cc: Lincoln Lavoie <lylavoie@iol.unh.edu>
Subject: RE: [PATCH V1] tests/checksum_offload: Bugfix for bug 529

Thanks response of that, in the log file, I find the result of new cases test_hardware_checksum_check_l4_rx and test_hardware_checksum_check_l4_tx are failed,
Is it expected result of now or there is bug of it ?

Regards,
Ma,lihong

From: Owen Hilyard <ohilyard@iol.unh.edu<mailto:ohilyard@iol.unh.edu>>
Sent: Thursday, September 10, 2020 5:14 AM
To: dts@dpdk.org<mailto:dts@dpdk.org>; Ma, LihongX <lihongx.ma@intel.com<mailto:lihongx.ma@intel.com>>
Cc: Lincoln Lavoie <lylavoie@iol.unh.edu<mailto:lylavoie@iol.unh.edu>>
Subject: Re: [PATCH V1] tests/checksum_offload: Bugfix for bug 529

Hello Ma,lihong,

While I was re-testing the entire suite to get the test log file for you, I found a few more issues that occur when running the entire test suite at once. I've included fixes for these but apparently checkpatch doesn't like something about the patch, so I'll resubmit. Sorry about that. I've included the log file after stripping some of the output. Due to how the test was constructed, I wasn't able to use the normal method to run scapy and had to do it through send_expect, which means that all of the colorings in the shell were logged. I've done my best to remove all of the lines with color symbols in them since they aren't needed to understand the output.

Sorry about messing up the submission again,
Owen

On Wed, Sep 9, 2020 at 5:08 PM Owen Hilyard <ohilyard@iol.unh.edu<mailto:ohilyard@iol.unh.edu>> wrote:
added missing quote in template for scapy packets.
rearraged closing down scapy and testpmd to avoid issues with other test
cases.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu<mailto:ohilyard@iol.unh.edu>>
---
 tests/TestSuite_checksum_offload.py | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/tests/TestSuite_checksum_offload.py b/tests/TestSuite_checksum_offload.py
index c4a877d..f2e9878 100644
--- a/tests/TestSuite_checksum_offload.py
+++ b/tests/TestSuite_checksum_offload.py
@@ -374,7 +374,7 @@ class TestChecksumOffload(TestCase):
     def replay_pcap_file_on_tester(self, iface, packet_file_path):
         self.tester.send_expect("scapy", ">>>")
         self.scapy_exec(f"packets = rdpcap('{packet_file_path}')")
-        self.scapy_exec(f"sendp(packets, iface={iface})")
+        self.scapy_exec(f"sendp(packets, iface='{iface}')")
         self.tester.send_expect("quit()", "# ")

     def validate_packet_list_checksums(self, packets):
@@ -703,7 +703,7 @@ class TestChecksumOffload(TestCase):
         for l4 in l4_protos:
             for chksum in "", "chksum=0xf":
                 vf = self.send_pkt_expect_good_bad_from_flag_catch_failure(
-                    f"eth/IP({chksum})/{l4}()/(X'*50)",
+                    f"eth/IP({chksum})/{l4}()/('X'*50)",
                     "PKT_RX_IP_CKSUM_", f"{l4}",
                     should_pass=(chksum == ""))
                 if vf is not None:
@@ -846,14 +846,13 @@ class TestChecksumOffload(TestCase):
         #                             if vf is not None:
         #                                 verification_errors.append(vf)

-        # tunneled inner
+        self.tester.send_expect("quit", "#")
+        self.dut.send_expect("stop", "testpmd>")

         for err in verification_errors:
             self.logger.error(str(err))
         self.verify(len(verification_errors) == 0, "See previous output")

        -        self.tester.send_expect("quit", "#")
-        self.dut.send_expect("stop", "testpmd>")

     def test_hardware_checksum_check_l4_tx(self):
         self.checksum_enablehw(self.dut_ports[0])
@@ -887,7 +886,9 @@ class TestChecksumOffload(TestCase):

         error_messages = self.validate_packet_list_checksums(captured_packets)

+        self.tester.send_expect("quit", "#")
         self.dut.send_expect("stop", "testpmd>")
+
         if len(error_messages) != 0:
             for error_msg in error_messages:
                 self.logger.error(error_msg)
--
2.25.1
  

With regards to the failed test cases, that is expected behavior for the
machine I was running it on. The NIC in the dev machine the IOL team is
using is running i40e and the support is P in the compatibility table, and
I'm fairly sure our NIC doesn't support inner checksum offloads but does
support outer checksum offloads. I'll try to remember to increment the
version number if I need more than 1 patch again.

Thanks,
Owen Hilyard

On Wed, Sep 9, 2020 at 9:51 PM Ma, LihongX <lihongx.ma@intel.com> wrote:

> BTW, it is better increase the version number when we submit a new version
> of the patch, it will easy to know the latest version of the patch. eg: V1
> ,V2, V3 ….
>
>
>
> Regards,
>
> Ma,lihong
>
>
>
> *From:* Ma, LihongX
> *Sent:* Thursday, September 10, 2020 9:46 AM
> *To:* Owen Hilyard <ohilyard@iol.unh.edu>; dts@dpdk.org
> *Cc:* Lincoln Lavoie <lylavoie@iol.unh.edu>
> *Subject:* RE: [PATCH V1] tests/checksum_offload: Bugfix for bug 529
>
>
>
> Thanks response of that, in the log file, I find the result of new cases
> test_hardware_checksum_check_l4_rx and test_hardware_checksum_check_l4_tx
> are failed,
>
> Is it expected result of now or there is bug of it ?
>
>
>
> Regards,
>
> Ma,lihong
>
>
>
> *From:* Owen Hilyard <ohilyard@iol.unh.edu>
> *Sent:* Thursday, September 10, 2020 5:14 AM
> *To:* dts@dpdk.org; Ma, LihongX <lihongx.ma@intel.com>
> *Cc:* Lincoln Lavoie <lylavoie@iol.unh.edu>
> *Subject:* Re: [PATCH V1] tests/checksum_offload: Bugfix for bug 529
>
>
>
> Hello Ma,lihong,
>
>
>
> While I was re-testing the entire suite to get the test log file for you,
> I found a few more issues that occur when running the entire test suite at
> once. I've included fixes for these but apparently checkpatch doesn't like
> something about the patch, so I'll resubmit. Sorry about that. I've
> included the log file after stripping some of the output. Due to how the
> test was constructed, I wasn't able to use the normal method to run scapy
> and had to do it through send_expect, which means that all of the colorings
> in the shell were logged. I've done my best to remove all of the lines with
> color symbols in them since they aren't needed to understand the output.
>
>
>
> Sorry about messing up the submission again,
>
> Owen
>
>
>
> On Wed, Sep 9, 2020 at 5:08 PM Owen Hilyard <ohilyard@iol.unh.edu> wrote:
>
> added missing quote in template for scapy packets.
> rearraged closing down scapy and testpmd to avoid issues with other test
> cases.
>
> Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> ---
>  tests/TestSuite_checksum_offload.py | 11 ++++++-----
>  1 file changed, 6 insertions(+), 5 deletions(-)
>
> diff --git a/tests/TestSuite_checksum_offload.py
> b/tests/TestSuite_checksum_offload.py
> index c4a877d..f2e9878 100644
> --- a/tests/TestSuite_checksum_offload.py
> +++ b/tests/TestSuite_checksum_offload.py
> @@ -374,7 +374,7 @@ class TestChecksumOffload(TestCase):
>      def replay_pcap_file_on_tester(self, iface, packet_file_path):
>          self.tester.send_expect("scapy", ">>>")
>          self.scapy_exec(f"packets = rdpcap('{packet_file_path}')")
> -        self.scapy_exec(f"sendp(packets, iface={iface})")
> +        self.scapy_exec(f"sendp(packets, iface='{iface}')")
>          self.tester.send_expect("quit()", "# ")
>
>      def validate_packet_list_checksums(self, packets):
> @@ -703,7 +703,7 @@ class TestChecksumOffload(TestCase):
>          for l4 in l4_protos:
>              for chksum in "", "chksum=0xf":
>                  vf =
> self.send_pkt_expect_good_bad_from_flag_catch_failure(
> -                    f"eth/IP({chksum})/{l4}()/(X'*50)",
> +                    f"eth/IP({chksum})/{l4}()/('X'*50)",
>                      "PKT_RX_IP_CKSUM_", f"{l4}",
>                      should_pass=(chksum == ""))
>                  if vf is not None:
> @@ -846,14 +846,13 @@ class TestChecksumOffload(TestCase):
>          #                             if vf is not None:
>          #                                 verification_errors.append(vf)
>
> -        # tunneled inner
> +        self.tester.send_expect("quit", "#")
> +        self.dut.send_expect("stop", "testpmd>")
>
>          for err in verification_errors:
>              self.logger.error(str(err))
>          self.verify(len(verification_errors) == 0, "See previous output")
>
>         -        self.tester.send_expect("quit", "#")
> -        self.dut.send_expect("stop", "testpmd>")
>
>      def test_hardware_checksum_check_l4_tx(self):
>          self.checksum_enablehw(self.dut_ports[0])
> @@ -887,7 +886,9 @@ class TestChecksumOffload(TestCase):
>
>          error_messages =
> self.validate_packet_list_checksums(captured_packets)
>
> +        self.tester.send_expect("quit", "#")
>          self.dut.send_expect("stop", "testpmd>")
> +
>          if len(error_messages) != 0:
>              for error_msg in error_messages:
>                  self.logger.error(error_msg)
> --
> 2.25.1
>
>