[V3,5/6] test_plans/vf_offload: Refactor checksum_offload case.

Message ID 20230718023311.2480869-6-ke1.xu@intel.com (mailing list archive)
State New
Headers
Series Rewrite test suite for checksum_offload, tso and vf_offload |

Commit Message

Ke Xu July 18, 2023, 2:33 a.m. UTC
  Update vf_offload testplan. Synced with PF checksum and TSO testplans.

1. Add Common test steps and packet organization table to better show the test steps.

2. Update test cases and test scopes. Using HW and SW to describe checksum configuration. Add cases for enabling HW IP UDP TCP SCTP seperately. Add cases for inner HW enabled with enabling HW outer IP UDP seperately.

3. Updated test cases and test scopes. Covering TSO tunnel, non-tunnel and combined scenario seperately.

4. Update cases for NIC comaptibility. Add description for VLAN, DCF VxLAN and AVX512 to fit specific testing scenarios required by several NICs.

5. RX scopes and offload flags are combined with varied packet type TX cases for better covering and more common validation.

Signed-off-by: Ke Xu <ke1.xu@intel.com>
---
 test_plans/vf_offload_test_plan.rst | 1356 ++++++++++++++++-----------
 1 file changed, 794 insertions(+), 562 deletions(-)
  

Patch

diff --git a/test_plans/vf_offload_test_plan.rst b/test_plans/vf_offload_test_plan.rst
index 1c929371..daf9e7c0 100644
--- a/test_plans/vf_offload_test_plan.rst
+++ b/test_plans/vf_offload_test_plan.rst
@@ -1,13 +1,78 @@ 
 .. SPDX-License-Identifier: BSD-3-Clause
-   Copyright(c) 2015-2017 Intel Corporation
+   Copyright(c) 2015-2023 Intel Corporation
 
 ==========
 VF Offload
 ==========
 
+Description
+===========
 
-Prerequisites for checksum offload
-==================================
+The support of RX/TX L3/L4 checksum offload features by Poll Mode Drivers consists in:
+
+On the RX side:
+
+- Verify IPv4 checksum by hardware for received packets.
+- Verify UDP/TCP/SCTP checksum by hardware for received packets.
+- Verify tunneled outer IPv4 checksum by hardware for recieved packets.
+- Verify tunneled outer UDP checksum by hardware for recieved packets.
+
+On the TX side:
+
+- IPv4 checksum insertion by hardware/software in transmitted packets.
+- UDP/TCP/SCTP checksum insertion by hardware/software in transmitted packets (sctp length in 4 bytes).
+- Tunneled outer IPv4 checksum insertion by hardware/software in transmitted packets.
+- Tunneled outer UDP checksum insertion by hardware/software in transmitted packets.
+
+RX/TX side, the insertion of a L3/L4 or tunneled outer L3/L4 checksum by hardware can be enabled with the
+following command of the ``testpmd`` application and running in a dedicated
+tx checksum mode::
+
+    set fwd csum
+    csum set ip|tcp|udp|sctp|outer-ip|outer-udp hw|sw port_id
+
+The support of TX TSO features by Poll Mode Drivers consists in:
+
+On the TX side:
+
+- TCP/UDP segmentation.
+- Tunneled inner TCP/UDP segmentation.
+
+RX/TX side, the TSO can be enabled with the following command of the ``testpmd`` application and running in tx
+checksum mode with enabled checksum support::
+
+    set fwd csum
+    csum set ...
+    tso set <segment_length> <port_id>
+    tunnel_tso set <segment_length> <port_id>
+
+Please notice here that in DPDK TSO refers to both TCP Segmentation Offload (TSO)
+and UDP Fragmentation Offload (UFO). DPDK TSO is also different from Generic Segmentation
+Offload (GSO) for there is a GSO lib implemented elsewhere in DPDK. We will use TSO and UFO
+to descripe packets and test steps, but only TSO configurations on ``testpmd`` are involved.
+
+The transmission of packet is done with the ``start`` command of the ``testpmd``
+application that will receive packets and then transmit the packet out on all
+configured ports.
+
+Prerequisites
+=============
+
+Topology
+--------
+
+    dut_port_0 <---> tester_vm_port_0
+
+    dut_port_1 <---> tester_vm_port_1
+
+Software
+--------
+
+    - dpdk: http://dpdk.org/git/dpdk
+    - scapy: http://www.secdev.org/projects/scapy/
+
+Configuration and Setup
+-----------------------
 
 If using vfio the kernel must be >= 3.6+ and VT-d must be enabled in bios. When
 using vfio, use the following commands to load the vfio driver and bind it
@@ -15,32 +80,27 @@  to the device under test::
 
    modprobe vfio
    modprobe vfio-pci
-   usertools/dpdk-devbind.py --bind=vfio-pci device_bus_id
+   usertools/dpdk-devbind.py --bind=vfio-pci <device_bus_id>
 
 IP link set VF trust on and spoofchk off on DUT::
 
    ip link set $PF_INTF vf 0 trust on
    ip link set $PF_INTF vf 0 spoofchk off
 
-Assuming that ports ``0`` and ``1`` are connected to a traffic generator,
-enable hardware rx checksum offload with "--enable-rx-cksum",
-launch the ``testpmd`` with the following arguments:
+If running TSO, on tester, all offload feature should be disabled on tx port, and start rx port capture::
 
- if test IAVF, start up VF port::
-
-  ./build/app/dpdk-testpmd -cffffff -n 1 -- -i --burst=1 --txpt`=32 \
-  --txht=8 --txwt=0 --txfreet=0 --rxfreet=64 --mbcache=250 --portmask=0x5
-  --enable-rx-cksum
+  ifconfig <tx port> mtu 9000
+  ethtool -K <tx port> rx off tx off tso off gso off gro off lro off
+  ip l set <tx port> up
+  tcpdump -n -e -i <rx port> -s 0 -w /tmp/cap
 
- if test DCF, set VF port to dcf and start up::
+Based on the topology, launch the ``testpmd`` with the following arguments::
 
-   Enable kernel trust mode:
+  ./build/app/dpdk-testpmd -c <...> -n <...> -- -i <...> --enable-rx-cksum
 
-       ip link set $PF_INTF vf 0 trust on
+For DCF mode::
 
-   dpdk-testpmd -c 0x0f -n 4 -a 00:04.0,cap=dcf -a 00:05.0,cap=dcf -- -i --burst=1 --txpt=32 \
-   --txht=8 --txwt=0 --txfreet=0 --rxfreet=64 --mbcache=250 --portmask=0x5
-   --enable-rx-cksum
+  ./build/app/dpdk-testpmd -c <...> -n <...> -a <...>,cap=dcf -a <...>,cap=dcf -- -i <...> --enable-rx-cksum
 
 .. note::
 
@@ -50,621 +110,793 @@  Set the verbose level to 1 to display information for each received packet::
 
   testpmd> set verbose 1
 
-Setup the ``csum`` forwarding mode::
+Due to DPDK 236bc417e2da(app/testpmd: fix MAC header in checksum forward engine) changes the checksum 
+functions adds switches to control whether to exchange MAC address.
+Currently, our test scripts are based on not exchanging MAC addresses, mac-swap needs to be disabled:
 
-  testpmd> set fwd csum
-  Set csum packet forwarding mode
+  testpmd> csum mac-swap off 0
 
-Start the packet forwarding::
+Common test steps
+=================
+
+This test uses ``Scapy`` to send out packages. The dut forwards packages with checksum offload enabled on
+rx/tx port and TSO enabled on tx port. After packages send out by tx port, the tester receives packages
+and verify them along with the verbose and stats.
+
+1. launch test-pmd to enable RX Checksum offload and for some NIC it enables a general set of RX Offloads::
+
+    ./dpdk/x86_64-native-linuxapp-gcc/app/dpdk-testpmd -c <...> -n <...> -a <...> -- -i --enable-rx-cksum
+
+2. config testpmd::
+
+    set csum fwd mode and stop all port::
+
+        testpmd> set fwd csum
+        testpmd> port stop all
+
+    Modify the following commands to configure checksum offload according to the test scenario of each case:
+
+        testpmd> csum set ip|tcp|udp|sctp|outer-ip|outer-udp hw|sw port_id
+
+    start all ports and fwd:
+
+        testpmd> port start all
+        testpmd> set verbose 1
+        testpmd> start
+
+3. start to capture packets on tester::
+
+    tester: tcpdump -i <iface> -Q in -e ether src 52:00:00:00:00:00 -w /tmp/ens10.pcap
+
+4. send packets with scapy::
+
+    Configure the packet generator to send multiple combinations of inner and outer packet parts
+    according to the test scenario of each case, refer to the following table:
+
++------------------+---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+| Packet Types     | Packet Names              |                                                    Packet organizations                                                      |
++==================+===========================+==============================================================================================================================+
+|                  | IP/UDP                    | IP(src="10.0.0.1") / UDP(sport=29999, dport=30000) / Raw(randstring(PAYLOAD_SIZE))                                           |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  | IP/TCP                    | IP(src="10.0.0.1") / TCP(sport=29999, dport=30000) / Raw(randstring(PAYLOAD_SIZE))                                           |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  | IP/SCTP                   | IP(src="10.0.0.1") / SCTP(sport=29999, dport=30000, chksum=0x0000) / Raw(randstring(PAYLOAD_SIZE))                           |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  | IPv6/UDP                  | IPv6(src="::1") / UDP(sport=29999, dport=30000) / Raw(randstring(PAYLOAD_SIZE))                                              |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  | IPv6/TCP                  | IPv6(src="::1") / TCP(sport=29999, dport=30000) / Raw(randstring(PAYLOAD_SIZE))                                              |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+| Inner Parts      | IPv6/SCTP                 | IPv6(src="::1") / SCTP(sport=29999, dport=30000, chksum=0x0000) / Raw(randstring(PAYLOAD_SIZE))                              |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  | VLAN/IP/UDP               | Dot1Q(vlan=100) / IP(src="10.0.0.1") / UDP(sport=29999, dport=30000) / Raw(randstring(PAYLOAD_SIZE))                         |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  | VLAN/IP/TCP               | Dot1Q(vlan=100) / IP(src="10.0.0.1") / TCP(sport=29999, dport=30000) / Raw(randstring(PAYLOAD_SIZE))                         |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  | VLAN/IP/SCTP              | Dot1Q(vlan=100) / IP(src="10.0.0.1") / SCTP(sport=29999, dport=30000, chksum=0x0000) / Raw(randstring(PAYLOAD_SIZE))         |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  | VLAN/IPv6/UDP             | Dot1Q(vlan=100) / IPv6(src="::1") / UDP(sport=29999, dport=30000) / Raw(randstring(PAYLOAD_SIZE))                            |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  | VLAN/IPv6/TCP             | Dot1Q(vlan=100) / IPv6(src="::1") / TCP(sport=29999, dport=30000) / Raw(randstring(PAYLOAD_SIZE))                            |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  | VLAN/IPv6/SCTP            | Dot1Q(vlan=100) / IPv6(src="::1") / SCTP(sport=29999, dport=30000, chksum=0x0000) / Raw(randstring(PAYLOAD_SIZE))            |
++------------------+---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IP/VXLAN                   | IP(src="10.0.0.1") / UDP(dport=4789) / VXLAN() / Ether()                                                                     |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IPv6/VXLAN                 | IPv6() / UDP(dport=4789) / VXLAN() / Ether()                                                                                 |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IP/VXLAN-GPE               | IP(src="10.0.0.1") / UDP(sport=4790, dport=4790) / VXLAN()                                                                   |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IPv6/VXLAN-GPE             | IPv6() / UDP(sport=4790, dport=4790) / VXLAN()                                                                               |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IP/VXLAN-GPE/Ether         | IP(src="10.0.0.1") / UDP(sport=4790, dport=4790) / VXLAN() / Ether()                                                         |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IPv6/VXLAN-GPE/Ether       | IPv6() / UDP(sport=4790, dport=4790) / VXLAN() / Ether()                                                                     |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IP/GRE                     | IP(src="10.0.0.1", proto=47) / GRE()                                                                                         |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IPv6/GRE                   | IPv6(nh=47) / GRE()                                                                                                          |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IP/GRE/Ether               | IP(src="10.0.0.1", proto=47) / GRE() / Ether()                                                                               |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IPv6/GRE/Ether             | IPv6(nh=47) / GRE() / Ether()                                                                                                |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IP/NVGRE                   | IP(src="10.0.0.1", proto=47) / GRE(key_present=1, proto=0x6558, key=0x00000100) / Ether()                                    |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IPv6/NVGRE                 | IPv6(nh=47) / GRE(key_present=1, proto=0x6558, key=0x00000100) / Ether()                                                     |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IP/GTPU                    | IP(src="10.0.0.1") / UDP(dport=2152) / GTP_U_Header(gtp_type=255, teid=0x123456)                                             |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IPv6/GTPU                  | IPv6() / UDP(dport=2152) / GTP_U_Header(gtp_type=255, teid=0x123456)                                                         |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IP/GENEVE                  | IP(src="10.0.0.1") / UDP(dport=6081, sport=29999) / GENEVE()                                                                 |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IPv6/GENEVE                | IPv6() / UDP(dport=6081, sport=29999) / GENEVE()                                                                             |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IP                         | IP(src="10.0.0.1")                                                                                                           |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+| Outer Parts      |IPv6                       | IPv6()                                                                                                                       |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IP/VXLAN              | Dot1Q(vlan=100) / IP(src="10.0.0.1") / UDP(dport=4789) / VXLAN() / Ether()                                                   |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IPv6/VXLAN            | Dot1Q(vlan=100) / IPv6() / UDP(dport=4789) / VXLAN() / Ether()                                                               |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IP/VXLAN-GPE          | Dot1Q(vlan=100) / IP(src="10.0.0.1") / UDP(sport=4790, dport=4790) / VXLAN()                                                 |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IPv6/VXLAN-GPE        | Dot1Q(vlan=100) / IPv6() / UDP(sport=4790, dport=4790) / VXLAN()                                                             |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IP/VXLAN-GPE/Ether    | Dot1Q(vlan=100) / IP(src="10.0.0.1") / UDP(sport=4790, dport=4790) / VXLAN() / Ether()                                       |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IPv6/VXLAN-GPE/Ether  | Dot1Q(vlan=100) / IPv6() / UDP(sport=4790, dport=4790) / VXLAN() / Ether()                                                   |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IP/GRE                | Dot1Q(vlan=100) / IP(src="10.0.0.1", proto=47) / GRE()                                                                       |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IPv6/GRE              | Dot1Q(vlan=100) / IPv6(nh=47) / GRE()                                                                                        |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IP/GRE/Ether          | Dot1Q(vlan=100) / IP(src="10.0.0.1", proto=47) / GRE() / Ether()                                                             |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IPv6/GRE/Ether        | Dot1Q(vlan=100) / IPv6(nh=47) / GRE() / Ether()                                                                              |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IP/NVGRE              | Dot1Q(vlan=100) / IP(src="10.0.0.1", proto=47) / GRE(key_present=1, proto=0x6558, key=0x00000100) / Ether()                  |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IPv6/NVGRE            | Dot1Q(vlan=100) / IPv6(nh=47) / GRE(key_present=1, proto=0x6558, key=0x00000100) / Ether()                                   |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IP/GTPU               | Dot1Q(vlan=100) / IP(src="10.0.0.1") / UDP(dport=2152) / GTP_U_Header(gtp_type=255, teid=0x123456)                           |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IPv6/GTPU             | Dot1Q(vlan=100) / IPv6() / UDP(dport=2152) / GTP_U_Header(gtp_type=255, teid=0x123456)                                       |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IP/GENEVE             | Dot1Q(vlan=100) / IP(src="10.0.0.1") / UDP(dport=6081, sport=29999) / GENEVE()                                               |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IPv6/GENEVE           | Dot1Q(vlan=100) / IPv6() / UDP(dport=6081, sport=29999) / GENEVE()                                                           |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IP                    | Dot1Q(vlan=100) / IP(src="10.0.0.1")                                                                                         |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IPv6                  | Dot1Q(vlan=100) / IPv6()                                                                                                     |
++------------------+---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+
++------------------+------------------+----------------------------+
+| Scenarios        | Packet Types     | Packet Names               |
++==================+==================+============================+
+|                  |                  | IP/UDP                     |
+|                  |                  +----------------------------+
+|                  |                  | IP/TCP                     |
+|                  |                  +----------------------------+
+| CSUM NON-TUNNEL  | Inner parts      | IP/SCTP                    |
+| PACKETS          |                  +----------------------------+
+|                  |                  | IPv6/UDP                   |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/TCP                   |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/SCTP                  |
++------------------+------------------+----------------------------+
+|                  |                  | IP/VXLAN                   |
+|                  | Outer parts      +----------------------------+
+|                  |                  | IPv6/VXLAN                 |
+|                  +------------------+----------------------------+
+| CSUM TUNNEL      |                  | IP/UDP                     |
+| PACKETS          |                  +----------------------------+
+| OF VXLAN         |                  | IP/TCP                     |
+|                  |                  +----------------------------+
+|                  | Inner parts      | IP/SCTP                    |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/UDP                   |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/TCP                   |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/SCTP                  |
++------------------+------------------+----------------------------+
+|                  |                  | IP/VXLAN                   |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/VXLAN                 |
+|                  |                  +----------------------------+
+|                  |                  | IP/VXLAN-GPE               |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/VXLAN-GPE             |
+|                  |                  +----------------------------+
+|                  |                  | IP/VXLAN-GPE/Ether         |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/VXLAN-GPE/Ether       |
+|                  |                  +----------------------------+
+|                  |                  | IP/GRE                     |
+|                  |                  +----------------------------+
+| CSUM TUNNEL      | Outer parts      | IPv6/GRE                   |
+| PACKETS          |                  +----------------------------+
+|                  |                  | IP/GRE/Ether               |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/GRE/Ether             |
+|                  |                  +----------------------------+
+|                  |                  | IP/NVGRE                   |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/NVGRE                 |
+|                  |                  +----------------------------+
+|                  |                  | IP/GTPU                    |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/GTPU                  |
+|                  +------------------+----------------------------+
+|                  |                  | IP/UDP                     |
+|                  |                  +----------------------------+
+|                  |                  | IP/TCP                     |
+|                  |                  +----------------------------+
+|                  | Inner parts      | IP/SCTP                    |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/UDP                   |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/TCP                   |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/SCTP                  |
++------------------+------------------+----------------------------+
+|                  |                  | VLAN/IP/UDP                |
+|                  |                  +----------------------------+
+|                  |                  | VLAN/IP/TCP                |
+|                  |                  +----------------------------+
+| CSUM NON-TUNNEL  | Inner parts      | VLAN/IP/SCTP               |
+| VLAN PACKETS     |                  +----------------------------+
+|                  |                  | VLAN/IPv6/UDP              |
+|                  |                  +----------------------------+
+|                  |                  | VLAN/IPv6/TCP              |
+|                  |                  +----------------------------+
+|                  |                  | VLAN/IPv6/SCTP             |
++------------------+------------------+----------------------------+
+|                  |                  | VLAN/IP/VXLAN              |
+|                  | Outer parts      +----------------------------+
+|                  |                  | VLAN/IPv6/VXLAN            |
+|                  +------------------+----------------------------+
+| CSUM TUNNEL      |                  | VLAN/IP/UDP                |
+| VLAN PACKETS     |                  +----------------------------+
+| OF VXLAN         |                  | VLAN/IP/TCP                |
+|                  |                  +----------------------------+
+|                  | Inner parts      | VLAN/IP/SCTP               |
+|                  |                  +----------------------------+
+|                  |                  | VLAN/IPv6/UDP              |
+|                  |                  +----------------------------+
+|                  |                  | VLAN/IPv6/TCP              |
+|                  |                  +----------------------------+
+|                  |                  | VLAN/IPv6/SCTP             |
++------------------+------------------+----------------------------+
+|                  |                  | VLAN/IP/VXLAN              |
+|                  |                  +----------------------------+
+|                  |                  | VLAN/IPv6/VXLAN            |
+|                  |                  +----------------------------+
+|                  |                  | VLAN/IP/VXLAN-GPE          |
+|                  |                  +----------------------------+
+|                  |                  | VLAN/IPv6/VXLAN-GPE        |
+|                  |                  +----------------------------+
+|                  |                  | VLAN/IP/VXLAN-GPE/Ether    |
+|                  |                  +----------------------------+
+|                  |                  | VLAN/IPv6/VXLAN-GPE/Ether  |
+|                  |                  +----------------------------+
+|                  |                  | VLAN/IP/GRE                |
+|                  |                  +----------------------------+
+| CSUM TUNNEL      | Outer parts      | VLAN/IPv6/GRE              |
+| VLAN PACKETS     |                  +----------------------------+
+|                  |                  | VLAN/IP/GRE/Ether          |
+|                  |                  +----------------------------+
+|                  |                  | VLAN/IPv6/GRE/Ether        |
+|                  |                  +----------------------------+
+|                  |                  | VLAN/IP/NVGRE              |
+|                  |                  +----------------------------+
+|                  |                  | VLAN/IPv6/NVGRE            |
+|                  |                  +----------------------------+
+|                  |                  | VLAN/IP/GTPU               |
+|                  |                  +----------------------------+
+|                  |                  | VLAN/IPv6/GTPU             |
+|                  +------------------+----------------------------+
+|                  |                  | VLAN/IP/UDP                |
+|                  |                  +----------------------------+
+|                  |                  | VLAN/IP/TCP                |
+|                  |                  +----------------------------+
+|                  | Inner parts      | VLAN/IP/SCTP               |
+|                  |                  +----------------------------+
+|                  |                  | VLAN/IPv6/UDP              |
+|                  |                  +----------------------------+
+|                  |                  | VLAN/IPv6/TCP              |
+|                  |                  +----------------------------+
+|                  |                  | VLAN/IPv6/SCTP             |
++------------------+------------------+----------------------------+
+| TSO NON-TUNNEL   |                  | IP/TCP                     |
+| PACKETS          | Inner parts      +----------------------------+
+|                  |                  | IPv6/TCP                   |
++------------------+------------------+----------------------------+
+|                  |                  | IP/VXLAN-GPE               |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/VXLAN-GPE             |
+|                  |                  +----------------------------+
+|                  |                  | IP/VXLAN-GPE/Ether         |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/VXLAN-GPE/Ether       |
+|                  |                  +----------------------------+
+|                  |                  | IP/GRE                     |
+|                  |                  +----------------------------+
+| TSO TUNNEL       | Outer parts      | IPv6/GRE                   |
+| PACKETS          |                  +----------------------------+
+|                  |                  | IP/GRE/Ether               |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/GRE/Ether             |
+|                  |                  +----------------------------+
+|                  |                  | IP/NVGRE                   |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/NVGRE                 |
+|                  |                  +----------------------------+
+|                  |                  | IP/GTPU                    |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/GTPU                  |
+|                  +------------------+----------------------------+
+|                  |                  | IP/TCP                     |
+|                  | Inner parts      +----------------------------+
+|                  |                  | IPv6/TCP                   |
++------------------+------------------+----------------------------+
+| UFO NON-TUNNEL   |                  | IP/UDP                     |
+| PACKETS          | Inner parts      +----------------------------+
+|                  |                  | IPv6/UDP                   |
++------------------+------------------+----------------------------+
+|                  |                  | IP/VXLAN-GPE               |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/VXLAN-GPE             |
+|                  |                  +----------------------------+
+|                  |                  | IP/VXLAN-GPE/Ether         |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/VXLAN-GPE/Ether       |
+|                  |                  +----------------------------+
+|                  |                  | IP/GRE                     |
+|                  |                  +----------------------------+
+| UFO TUNNEL       | Outer parts      | IPv6/GRE                   |
+| PACKETS          |                  +----------------------------+
+|                  |                  | IP/GRE/Ether               |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/GRE/Ether             |
+|                  |                  +----------------------------+
+|                  |                  | IP/NVGRE                   |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/NVGRE                 |
+|                  |                  +----------------------------+
+|                  |                  | IP/GTPU                    |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/GTPU                  |
+|                  +------------------+----------------------------+
+|                  |                  | IP/UDP                     |
+|                  | Inner parts      +----------------------------+
+|                  |                  | IPv6/UDP                   |
++------------------+------------------+----------------------------+
+
+5. stop capture packets and then stop fwd::
+
+    tester:     pkill tcpdump
+    dut:        testpmd> stop
+
+6. check rx and tx results::
+    
+    Check if ``ol_flag`` contains these fields based on the checksum values of L3 and L4 in the sent packets, refer to the following table.
+
++--------------+---------------------------------+--------------------------+
+| packet type  |         checksum value          |      check ol_flag       |
++==============+=================================+==========================+
+| Inner        | IPv4 (bad checksum)             | `RX_IP_CKSUM_BAD`        |
+|              +---------------------------------+--------------------------+
+|              | IPv4 (good checksum)            | `RX_IP_CKSUM GOOD`       |
+|              +---------------------------------+--------------------------+
+|              | TCP / UDP / SCTP (bad checksum) | `RX_L4_CKSUM_BAD`        |
+|              +---------------------------------+--------------------------+
+|              | TCP / UDP / SCTP (bad checksum) | `RX_L4_CKSUM_GOOD`       |
++--------------+---------------------------------+--------------------------+
+| Outer        | IPv4 (bad checksum)             | `RX_OUTER_IP_CKSUM_BAD`  |
+|              +---------------------------------+--------------------------+
+|              | IPv4 (good checksum)            | `RX_OUTER_IP_CKSUM GOOD` |
+|              +---------------------------------+--------------------------+
+|              | UDP (bad checksum)              | `RX_OUTER_L4_CKSUM_BAD`  |
+|              +---------------------------------+--------------------------+
+|              | UDP (good checksum)             | `RX_OUTER_L4_CKSUM_GOOD` |
++--------------+---------------------------------+--------------------------+
+
+
+    Check rx and tx statistics according to the checksum values of L3 and L4 in the sent packets.
+
++------------------+-------------------------------------------------------------------+
+|      entry       |                            checkpoint                             |
++==================+===================================================================+
+| RX-packets       | `RX-packets` =  Num of sent packets                               |
++------------------+-------------------------------------------------------------------+
+| RX-total         | `RX-total` = Num of sent packets                                  |
++------------------+-------------------------------------------------------------------+
+| Bad-ipcsum       | `Bad-ipcsum` = Sum of bad inner IP checksum in sent packets       |
++------------------+-------------------------------------------------------------------+
+| Bad-l4csum       | `Bad-l4csum` = Sum of bad inner L4 checksum in sent packets       |
++------------------+-------------------------------------------------------------------+
+| Bad-outer-ipcsum | `Bad-outer-ipcsum` = Sum of bad outer IP checksum in sent packets |
++------------------+-------------------------------------------------------------------+
+| Bad-outer-l4csum | `Bad-outer-l4csum` = Sum of bad outer L4 checksum in sent packets |
++------------------+-------------------------------------------------------------------+
+| TX-packets       | `TX-packets` =  `RX-packets`   (exclude tso cases)                |
++------------------+-------------------------------------------------------------------+
+| TX-total         | `TX-total` =  `RX-total`       (exclude tso cases)                |
++------------------+-------------------------------------------------------------------+
+
+    Read pcap files with scapy and check if configured checksum values to insert of tx_packets are all correct::
+
+        >>> tx_pkts = rdpcap("/tmp/ens10.pcap") # tx_packets is of list of all tx pkts
+        >>> tx_pkts[0].show2() # will print all info about the first packet, compare the checksum with the ref checksum.
+
+1. For TSO cases, we verify if the segmented payload matches the send payload::
+        
+        >>> tx_pkts[0].load == send_pkt.load[ : SEG_LENGTH ]
+        >>> tx_pkts[1].load == send_pkt.load[ SEG_LENGTH : 2*SEG_LENGTH ]
+        >>> ...
+
+Test case 1: HW IP non-tunneling
+================================
+
+This test is used to test non-tunneling packets and will enable hw ip checksum offload.
+Configure the TX HW Offload::
+
+    csum set ip hw 1
+    
+Send packets of below with bad IP checksum, bad L4 checksum, bad all checksum, good all checksum,
+each packet for multiple times, 10 times usually::
+
+    CSUM NON-TUNNEL PACKETS
+    CSUM NON-TUNNEL VLAN PACKETS (for i40e and ice)
+
+
+Test case 2: HW UDP non-tunneling
+=================================
+
+This test is used to test non-tunneling packets and will enable hw udp checksum offload.
+Configure the TX HW Offload::
+
+    csum set udp hw 1
+    
+Send packets of below with bad IP checksum, bad L4 checksum, bad all checksum, good all checksum,
+each packet for multiple times, 10 times usually::
+
+    CSUM NON-TUNNEL PACKETS
+    CSUM NON-TUNNEL VLAN PACKETS (for i40e and ice)
+
+Check for the correct checksum insertion at the correct place.
+
+
+Test case 3: HW TCP non-tunneling
+=================================
+
+This test is used to test non-tunneling packets and will enable hw tcp checksum offload.
+Configure the TX HW Offload::
+
+    csum set tcp hw 1
+    
+Send packets of below with bad IP checksum, bad L4 checksum, bad all checksum, good all checksum,
+each packet for multiple times, 10 times usually::
+
+    CSUM NON-TUNNEL PACKETS
+    CSUM NON-TUNNEL VLAN PACKETS (for i40e and ice)
+
+Check for the correct checksum insertion at the correct place.
+
+
+Test case 4: HW SCTP non-tunneling
+==================================
 
-  testpmd> start
-    csum packet forwarding - CRC stripping disabled - packets/burst=32
-    nb forwarding cores=1 - nb forwarding ports=10
-    RX queues=1 - RX desc=128 - RX free threshold=64
-    RX threshold registers: pthresh=8 hthresh=8 wthresh=4
-    TX queues=1 - TX desc=512 - TX free threshold=0
-    TX threshold registers: pthresh=32 hthresh=8 wthresh=8
+This test is used to test non-tunneling packets and will enable hw sctp checksum offload.
+Configure the TX HW Offload::
 
-Verify that how many packets found with Bad-ipcsum or Bad-l4csum::
+    csum set sctp hw 1
+    
+Send packets of below with bad IP checksum, bad L4 checksum, bad all checksum, good all checksum,
+each packet for multiple times, 10 times usually::
 
-  testpmd> stop
-  ---------------------- Forward statistics for port 0  ----------------------
-  RX-packets: 0              RX-dropped: 0             RX-total: 0
-  Bad-ipcsum: 0              Bad-l4csum: 0
-  TX-packets: 0              TX-dropped: 0             TX-total: 0
-  ----------------------------------------------------------------------------
+    CSUM NON-TUNNEL PACKETS
+    CSUM NON-TUNNEL VLAN PACKETS (for i40e and ice)
 
+Check for the correct checksum insertion at the correct place.
 
-Test Case: HW checksum offload check
-====================================
-Start testpmd and enable checksum offload on rx port.
 
-Setup the ``csum`` forwarding mode::
+Test case 5: HW IP + L4 non-tunneling (w/o VLAN)
+================================================
 
-  testpmd> set fwd csum
-  Set csum packet forwarding mode
+This test is used to test non-tunneling packets and will enable IP and all L4 hw checksum offload.
+Configure the TX HW Offload::
 
-Enable the IPv4/UDP/TCP/SCTP HW checksum offload on port 0::
+    csum set ip hw 1
+    csum set udp hw 1
+    csum set tcp hw 1
+    csum set sctp hw 1
+    
+Send packets of below with bad IP checksum, bad L4 checksum, bad all checksum, good all checksum,
+each packet for multiple times, 10 times usually::
 
-  testpmd> port stop all
-  testpmd> csum set ip hw 0
-  testpmd> csum set tcp hw 0
-  testpmd> csum set udp hw 0
-  testpmd> csum set sctp hw 0
-  testpmd> port start all
-  testpmd> set promisc 0 
-  
-  Due to DPDK 236bc417e2da(app/testpmd: fix MAC header in checksum forward engine) changes the checksum 
-  functions adds switches to control whether to exchange MAC address.
-  Currently, our test scripts are based on not exchanging MAC addresses, mac-swap needs to be disabled:
-  testpmd> csum mac-swap off 0
+    CSUM NON-TUNNEL PACKETS
+    CSUM NON-TUNNEL VLAN PACKETS (for i40e and ice)
 
-  testpmd> start
-    csum packet forwarding - CRC stripping disabled - packets/burst=32
-    nb forwarding cores=1 - nb forwarding ports=10
-    RX queues=1 - RX desc=128 - RX free threshold=64
-    RX threshold registers: pthresh=8 hthresh=8 wthresh=4
-    TX queues=1 - TX desc=512 - TX free threshold=0
-    TX threshold registers: pthresh=32 hthresh=8 wthresh=8
-
-Configure the traffic generator to send the multiple packets for the following
-combination:
-
-  +----------------+----------------------------------------+
-  | packet type    | packet organization                    |
-  +================+========================================+
-  |                | Ether / IPv4 / UDP / payload           |
-  |                +----------------------------------------+
-  |                | Ether / IPv4 / TCP / payload           |
-  |                +----------------------------------------+
-  | packets        | Ether / IPv4 / SCTP / payload          |
-  | for checksum   +----------------------------------------+
-  | offload test   | Ether / IPv6 / UDP / payload           |
-  |                +----------------------------------------+
-  |                | Ether / IPv6 / TCP / payload           |
-  +----------------+----------------------------------------+
-
-Send packets with incorrect checksum,
-verify dpdk can rx it and report the checksum error,
-verify that the same number of packet are correctly received on the traffic
-generator side. And IPv4 checksum, TCP checksum, UDP checksum, SCTP checksum need
-be validated as pass by the tester.
-
-The IPv4 source address will not be changed by testpmd.
-
-
-Test Case: HW tunneling checksum offload check
-==============================================
-In DPDK 22.11 release, Intel® Ethernet 800 Series NIC with ICE supports HW
-checksum offload for tunneling packets for checking both inner and outer
-checksum. For the packets involved in this case, a ICE COMMON DDP Package
-is required.
-
-Start testpmd and enable checksum offload on rx port.
-
-Setup the ``csum`` forwarding mode::
-
-  testpmd> set fwd csum
-  Set csum packet forwarding mode
-
-Enable the IPv4/UDP/TCP/SCTP HW checksum offload on port 0::
-
-  testpmd> port stop all
-  testpmd> csum set ip hw 0
-  testpmd> csum set tcp hw 0
-  testpmd> csum set udp hw 0
-  testpmd> csum set sctp hw 0
-  testpmd> csum set outer-ip hw 0
-  testpmd> csum set outer-udp hw 0
-  testpmd> csum parse-tunnel on 0
-  testpmd> port start all
-  testpmd> set promisc 0 on
-  testpmd> start
-    csum packet forwarding - CRC stripping disabled - packets/burst=32
-    nb forwarding cores=1 - nb forwarding ports=10
-    RX queues=1 - RX desc=128 - RX free threshold=64
-    RX threshold registers: pthresh=8 hthresh=8 wthresh=4
-    TX queues=1 - TX desc=512 - TX free threshold=0
-    TX threshold registers: pthresh=32 hthresh=8 wthresh=8
-
-Configure the traffic generator to send the multiple packets for the following
-combination with inner package of:
-
-  +----------------+----------------------------------------+
-  | packet type    | packet organization                    |
-  +================+========================================+
-  |                | IPv4 / UDP / payload                   |
-  |                +----------------------------------------+
-  |                | IPv4 / TCP / payload                   |
-  |                +----------------------------------------+
-  | inner packets  | IPv4 / SCTP / payload                  |
-  | for checksum   +----------------------------------------+
-  | offload test   | IPv6 / UDP / payload                   |
-  |                +----------------------------------------+
-  |                | IPv6 / TCP / payload                   |
-  |                +----------------------------------------+
-  |                | IPv6 / SCTP / payload                  |
-  +----------------+----------------------------------------+
-
-And outer or tunneling package of :
-
-  +----------------+----------------------------------------+
-  | packet type    | packet organization                    |
-  +================+========================================+
-  |                | Ether / IPv4 / UDP / VXLAN / Ether     |
-  |                +----------------------------------------+
-  |                | Ether / IPv6 / UDP / VXLAN / Ether     |
-  |                +----------------------------------------+
-  |                | Ether / IPv4 / GRE                     |
-  | outer and      +----------------------------------------+
-  | tunneling      | Ether / IPv4 / GRE / Ether             |
-  | packets        +----------------------------------------+
-  | for checksum   | Ether / IPv6 / GRE                     |
-  | offload test   +----------------------------------------+
-  |                | Ether / IPv6 / GRE / Ether             |
-  |                +----------------------------------------+
-  |                | Ether / IPv4 / NVGRE                   |
-  |                +----------------------------------------+
-  |                | Ether / IPv4 / NVGRE / Ether           |
-  |                +----------------------------------------+
-  |                | Ether / IPv6 / NVGRE                   |
-  |                +----------------------------------------+
-  |                | Ether / IPv6 / NVGRE / Ether           |
-  |                +----------------------------------------+
-  |                | Ether / IPv4 / UDP / GTPU              |
-  |                +----------------------------------------+
-  |                | Ether / IPv6 / UDP / GTPU              |
-  +----------------+----------------------------------------+
-  
-Notice that VxLAN needs DCF to configure, so testing of VxLAN may need to perform
-on DCF.
-
-Send packets with incorrect checksum on outer IPv4, outer UDP (if exists), inner
-IP, inner L4, verify dpdk can rx it and report the checksum error,
-verify that the same number of packet are correctly received on the traffic
-generator side. And IPv4 checksum, TCP checksum, UDP checksum, SCTP checksum need
-be validated as pass by the tester.
-
-The IPv4 source address will not be changed by testpmd.
-
-
-Test Case: SW checksum offload check
-====================================
-
-Enable SW checksum offload, send same packet with incorrect checksum
-and verify checksum is valid.
-
-Setup the ``csum`` forwarding mode::
-
-  testpmd> set fwd csum
-  Set csum packet forwarding mode
-
-Enable the IPv4/UDP/TCP/SCTP SW checksum offload on port 0::
-
-  testpmd> port stop all
-  testpmd> csum set ip sw 0
-  testpmd> csum set tcp sw 0
-  testpmd> csum set udp sw 0
-  testpmd> csum set sctp sw 0
-  testpmd> port start all
-  testpmd> set promisc 0 on
-
-  Due to DPDK 236bc417e2da(app/testpmd: fix MAC header in checksum forward engine) changes the checksum 
-  functions adds switches to control whether to exchange MAC address.
-  Currently, our test scripts are based on not exchanging MAC addresses, mac-swap needs to be disabled:
-  testpmd> csum mac-swap off 0
+Check for the correct checksum insertion at the correct place.
 
-  testpmd> start
-    csum packet forwarding - CRC stripping disabled - packets/burst=32
-    nb forwarding cores=1 - nb forwarding ports=10
-    RX queues=1 - RX desc=128 - RX free threshold=64
-    RX threshold registers: pthresh=8 hthresh=8 wthresh=4
-    TX queues=1 - TX desc=512 - TX free threshold=0
-    TX threshold registers: pthresh=32 hthresh=8 wthresh=8
-
-Configure the traffic generator to send the multiple packets for the following
-combination:
-
-  +----------------+----------------------------------------+
-  | packet type    | packet organization                    |
-  +================+========================================+
-  |                | Ether / IPv4 / UDP / payload           |
-  |                +----------------------------------------+
-  | packets        | Ether / IPv4 / TCP / payload           |
-  | for checksum   +----------------------------------------+
-  | offload test   | Ether / IPv6 / UDP / payload           |
-  |                +----------------------------------------+
-  |                | Ether / IPv6 / TCP / payload           |
-  +----------------+----------------------------------------+
-
-Send packets with incorrect checksum,
-verify dpdk can rx it and report the checksum error,
-verify that the same number of packet are correctly received on the traffic
-generator side. And IPv4 checksum, TCP checksum, UDP checksum need
-be validated as pass by the tester.
-
-The first byte of source IPv4 address will be increased by testpmd. The checksum
-is indeed recalculated by software algorithms.
-
-
-Test Case: HW checksum offload check with vlan
-==============================================
-Start testpmd and enable checksum offload on rx port. Based on test steps of
-HW checksum offload check, configure the traffic generator to send the multiple
-packets for the following combination:
-
-  +----------------+----------------------------------------+
-  | packet type    | packet organization                    |
-  +================+========================================+
-  |                | Ether / VLAN / IPv4 / UDP / payload    |
-  |                +----------------------------------------+
-  |                | Ether / VLAN / IPv4 / TCP / payload    |
-  |                +----------------------------------------+
-  | packets        | Ether / VLAN / IPv4 / SCTP / payload   |
-  | for checksum   +----------------------------------------+
-  | offload test   | Ether / VLAN / IPv6 / UDP / payload    |
-  |                +----------------------------------------+
-  |                | Ether / VLAN / IPv6 / TCP / payload    |
-  +----------------+----------------------------------------+
-
-
-Test Case: HW tunneling checksum offload check with vlan
-========================================================
-Based on test steps of HW tunneling checksum offload check, configure the
-traffic generator to send the multiple packets combination with outer or
-tunneling package of:
-
-  +----------------+--------------------------------------------+
-  | packet type    | packet organization                        |
-  +================+============================================+
-  |                | Ether / VLAN / IPv4 / UDP / VXLAN / Ether  |
-  |                +--------------------------------------------+
-  |                | Ether / VLAN / IPv6 / UDP / VXLAN / Ether  |
-  |                +--------------------------------------------+
-  |                | Ether / VLAN / IPv4 / GRE                  |
-  | outer and      +--------------------------------------------+
-  | tunneling      | Ether / VLAN / IPv4 / GRE / Ether          |
-  | packets        +--------------------------------------------+
-  | for checksum   | Ether / VLAN / IPv6 / GRE                  |
-  | offload test   +--------------------------------------------+
-  |                | Ether / VLAN / IPv6 / GRE / Ether          |
-  |                +--------------------------------------------+
-  |                | Ether / VLAN / IPv4 / NVGRE                |
-  |                +--------------------------------------------+
-  |                | Ether / VLAN / IPv4 / NVGRE / Ether        |
-  |                +--------------------------------------------+
-  |                | Ether / VLAN / IPv6 / NVGRE                |
-  |                +--------------------------------------------+
-  |                | Ether / VLAN / IPv6 / NVGRE / Ether        |
-  |                +--------------------------------------------+
-  |                | Ether / VLAN / IPv4 / UDP / GTPU           |
-  |                +--------------------------------------------+
-  |                | Ether / VLAN / IPv6 / UDP / GTPU           |
-  +----------------+--------------------------------------------+
-
-
-Test Case: HW checksum offload check on scalar path
-===================================================
-These set of cases based on existing cases are designed for better case managment for
-regression test.
-
-Start testpmd with eal parameter --force-max-simd-bitwidth=64. Based on test steps of
-'HW checksum offload check'.
-
-Test Case: HW checksum offload check on sse path
-================================================
-Start testpmd with eal parameter --force-max-simd-bitwidth=128. Based on test steps of
-'HW checksum offload check'.
+For PMD not supporting VLAN defaultly, we run an extra case::
+    
+    csum set ip hw 1
+    csum set udp hw 1
+    csum set tcp hw 1
+    csum set sctp hw 1
+    vlan set filter on 0
+    rx_vlan add 100 0
 
-Test Case: HW checksum offload check on avx2 path
-=================================================
-Start testpmd with eal parameter --force-max-simd-bitwidth=256. Based on test steps of
-'HW checksum offload check'.
+Send packets of below with bad IP checksum, bad L4 checksum, bad all checksum, good all checksum,
+each packet for multiple times, 10 times usually::
 
-Test Case: HW checksum offload check on avx512 path
-===================================================
-Start testpmd with eal parameter --force-max-simd-bitwidth=512. Based on test steps of
-'HW checksum offload check'.
+    CSUM NON-TUNNEL VLAN PACKETS
 
-Test Case: HW checksum offload check with vlan on scalar path
-=============================================================
+Check for the correct checksum insertion at the correct place.
 
-Test Case: HW checksum offload check with vlan on sse path
-=============================================================
 
-Test Case: HW checksum offload check with vlan on avx2 path
-=============================================================
+Test case 6: HW Outer-IP tunneling
+==================================
 
-Test Case: HW checksum offload check with vlan on avx512 path
-=============================================================
+After DPDK v22.11, Intel® Ethernet 800 Series NIC with ICE supports tunneling packets on VF.
+This test is used to test tunneling scenario and will enable Outer-IP checksum offload.
+Configure the TX HW Offload::
 
-Test Case: HW tunneling checksum offload check on scalar path
-=============================================================
+    csum set ip hw 1
+    csum set udp hw 1
+    csum set tcp hw 1
+    csum set sctp hw 1
+    csum set outer-ip hw 1
+    csum parse-tunnel on 1
+    
+Send packets of below with bad IP checksum, bad L4 checksum, bad all checksum, good all checksum,
+each packet for multiple times, 10 times usually::
 
-Test Case: HW tunneling checksum offload check on sse path
-==========================================================
+    CSUM NON-TUNNEL PACKETS
+    CSUM NON-TUNNEL VLAN PACKETS
 
-Test Case: HW tunneling checksum offload check on avx2 path
-===========================================================
+Send packets of below with bad Outer-IP checksum, bad Outer-L4 checksum, bad IP checksum, bad L4
+checksum, bad all checksum, good all checksum, each packet for multiple times, 10 times usually::
 
-Test Case: HW tunneling checksum offload check on avx512 path
-=============================================================
+    CSUM TUNNEL PACKETS
+    CSUM TUNNEL VLAN PACKETS
 
-Test Case: HW tunneling checksum offload check with vlan on scalar path
-=======================================================================
+Check for the correct checksum insertion at the correct place.
 
-Test Case: HW tunneling checksum offload check with vlan on sse path
-====================================================================
 
-Test Case: HW tunneling checksum offload check with vlan on avx2 path
-=====================================================================
+Test case 7: HW Outer-UDP tunneling
+===================================
 
-Test Case: HW tunneling checksum offload check with vlan on avx512 path
-=======================================================================
+After DPDK v22.11, Intel® Ethernet 800 Series NIC with ICE supports tunneling packets on VF.
+This test is used to test tunneling scenario and will enable Outer-UDP checksum offload.
+Configure the TX HW Offload::
 
-Test Case: SW checksum offload check on scalar path
-===================================================
+    csum set ip hw 1
+    csum set udp hw 1
+    csum set tcp hw 1
+    csum set sctp hw 1
+    csum set outer-udp hw 1
+    csum parse-tunnel on 1
+    
+Send packets of below with bad IP checksum, bad L4 checksum, bad all checksum, good all checksum,
+each packet for multiple times, 10 times usually::
 
-Test Case: SW checksum offload check on sse path
-================================================
+    CSUM NON-TUNNEL PACKETS
+    CSUM NON-TUNNEL VLAN PACKETS
 
-Test Case: SW checksum offload check on avx2 path
-=================================================
+Send packets of below with bad Outer-IP checksum, bad Outer-L4 checksum, bad IP checksum, bad L4
+checksum, bad all checksum, good all checksum, each packet for multiple times, 10 times usually::
 
-Test Case: SW checksum offload check on avx512 path
-===================================================
+    CSUM TUNNEL PACKETS
+    CSUM TUNNEL VLAN PACKETS
 
+Check for the correct checksum insertion at the correct place.
 
-Prerequisites for TSO
-=====================
 
-The DUT must take one of the Ethernet controller ports connected to a port on another
-device that is controlled by the Scapy packet generator.
+Test case 8: HW Outer-IP + Outer-UDP tunneling with SW Inner
+============================================================
 
-The Ethernet interface identifier of the port that Scapy will use must be known.
-On tester, all offload feature should be disabled on tx port, and start rx port capture::
+After DPDK v22.11, Intel® Ethernet 800 Series NIC with ICE supports tunneling packets on VF.
+This test is used to test tunneling scenario and will enable Outer-IP and Outer-UDP checksum offload.
+Configure the TX HW Offload::
 
-  ethtool -K <tx port> rx off tx off tso off gso off gro off lro off
-  ip l set <tx port> up
-  tcpdump -n -e -i <rx port> -s 0 -w /tmp/cap
+    csum set outer-ip hw 1
+    csum set outer-udp hw 1
+    csum parse-tunnel on 1
+    
+Send packets of below with bad IP checksum, bad L4 checksum, bad all checksum, good all checksum,
+each packet for multiple times, 10 times usually::
 
+    CSUM NON-TUNNEL PACKETS
+    CSUM NON-TUNNEL VLAN PACKETS
 
-On DUT, run pmd with parameter "--enable-rx-cksum". Then enable TSO on tx port
-and checksum on rx port. The test commands is below::
-
-  # Enable hw checksum on rx port
-  testpmd> port stop all
-  testpmd> csum set ip hw 0
-  testpmd> csum set tcp hw 0
-  testpmd> csum set udp hw 0
-  testpmd> csum set sctp hw 0
-  testpmd> port start all
-  testpmd> set promisc 0 on
-  testpmd> set fwd csum
-
-  # Enable TSO on tx port
-  testpmd> tso set 800 1
-
-For tunneling cases on Intel® Ethernet 800 Series NIC with ICE, add tunneling support
-on csum and enable tunnel tso as below::
-
-  # Enable hw checksum for tunneling on rx port
-  testpmd> port stop all
-  testpmd> csum set outer-ip hw 0
-  testpmd> csum set outer-udp hw 0
-  testpmd> csum parse-tunnel on 0
-  testpmd> port start all
-  testpmd> tunnel_tso set 800 1
-
-Configure the traffic generator to send the multiple packets for the following
-combination, each combination for several times:
-
-  +----------------+----------------------------------------+---------------+
-  | packet type    | packet organization                    | packet count  |
-  +================+========================================+===============+
-  |                | Ether / IPv4 / TCP / payload len 128   | 10            |
-  |                +----------------------------------------+---------------+
-  |                | Ether / IPv4 / TCP / payload len 800   | 10            |
-  |                +----------------------------------------+---------------+
-  |                | Ether / IPv4 / TCP / payload len 801   | 10            |
-  |                +----------------------------------------+---------------+
-  |                | Ether / IPv4 / TCP / payload len 1700  | 10            |
-  |                +----------------------------------------+---------------+
-  |                | Ether / IPv4 / TCP / payload len 2500  | 10            |
-  | non-tunneling  +----------------------------------------+---------------+
-  | packets for    | Ether / IPv4 / TCP / payload len 8500  | 1000          |
-  | TSO test       +----------------------------------------+---------------+
-  |                | Ether / IPv6 / TCP / payload len 128   | 10            |
-  |                +----------------------------------------+---------------+
-  |                | Ether / IPv6 / TCP / payload len 800   | 10            |
-  |                +----------------------------------------+---------------+
-  |                | Ether / IPv6 / TCP / payload len 801   | 10            |
-  |                +----------------------------------------+---------------+
-  |                | Ether / IPv6 / TCP / payload len 1700  | 10            |
-  |                +----------------------------------------+---------------+
-  |                | Ether / IPv6 / TCP / payload len 2500  | 10            |
-  |                +----------------------------------------+---------------+
-  |                | Ether / IPv6 / TCP / payload len 8500  | 1000          |
-  +----------------+----------------------------------------+---------------+
-  |                | Ether / IPv4 / UDP / VXLAN / Ether     | N/A           |
-  |                +----------------------------------------+---------------+
-  |                | Ether / IPv6 / UDP / VXLAN / Ether     | N/A           |
-  |                +----------------------------------------+---------------+
-  |                | Ether / IPv4 / GRE                     | N/A           |
-  | outer and      +----------------------------------------+---------------+
-  | tunneling      | Ether / IPv4 / GRE / Ether             | N/A           |
-  | packets        +----------------------------------------+---------------+
-  | for tso test   | Ether / IPv6 / GRE                     | N/A           |
-  |                +----------------------------------------+---------------+
-  |                | Ether / IPv6 / GRE / Ether             | N/A           |
-  |                +----------------------------------------+---------------+
-  |                | Ether / IPv4 / NVGRE                   | N/A           |
-  |                +----------------------------------------+---------------+
-  |                | Ether / IPv4 / NVGRE / Ether           | N/A           |
-  |                +----------------------------------------+---------------+
-  |                | Ether / IPv6 / NVGRE                   | N/A           |
-  |                +----------------------------------------+---------------+
-  |                | Ether / IPv6 / NVGRE / Ether           | N/A           |
-  |                +----------------------------------------+---------------+
-  |                | Ether / IPv4 / UDP / GTPU              | N/A           |
-  |                +----------------------------------------+---------------+
-  |                | Ether / IPv6 / UDP / GTPU              | N/A           |
-  +----------------+----------------------------------------+---------------+
-  |                | IPv4 / TCP / payload len 128           | 10            |
-  |                +----------------------------------------+---------------+
-  |                | IPv4 / TCP / payload len 800           | 10            |
-  |                +----------------------------------------+---------------+
-  |                | IPv4 / TCP / payload len 801           | 10            |
-  |                +----------------------------------------+---------------+
-  |                | IPv4 / TCP / payload len 1700          | 10            |
-  |                +----------------------------------------+---------------+
-  |                | IPv4 / TCP / payload len 2500          | 10            |
-  |                +----------------------------------------+---------------+
-  | inner packets  | IPv4 / TCP / payload len 8500          | 1000          |
-  | for TSO test   +----------------------------------------+---------------+
-  |                | IPv6 / TCP / payload len 128           | 10            |
-  |                +----------------------------------------+---------------+
-  |                | IPv6 / TCP / payload len 800           | 10            |
-  |                +----------------------------------------+---------------+
-  |                | IPv6 / TCP / payload len 801           | 10            |
-  |                +----------------------------------------+---------------+
-  |                | IPv6 / TCP / payload len 1700          | 10            |
-  |                +----------------------------------------+---------------+
-  |                | IPv6 / TCP / payload len 2500          | 10            |
-  |                +----------------------------------------+---------------+
-  |                | IPv6 / TCP / payload len 8500          | 1000          |
-  +----------------+----------------------------------------+---------------+
-  
-Notice that VxLAN needs DCF to configure, so testing of VxLAN may need to perform
-on DCF.
-
-
-Test case: csum fwd engine, use TSO
-===================================
+Send packets of below with bad Outer-IP checksum, bad Outer-L4 checksum, bad IP checksum, bad L4
+checksum, bad all checksum, good all checksum, each packet for multiple times, 10 times usually::
 
-This test uses ``Scapy`` to send out one large TCP package. The dut forwards package
-with TSO enable on tx port while rx port turns checksum on. After package send out
-by TSO on tx port, the tester receives multiple small TCP package.
+    CSUM TUNNEL PACKETS
+    CSUM TUNNEL VLAN PACKETS
 
-Turn off tx port by ethtool on tester::
+Check for the correct checksum insertion at the correct place.
 
-  ethtool -K <tx port> rx off tx off tso off gso off gro off lro off
-  ip l set <tx port> up
 
-Capture package rx port on tester::
+Test case 9: HW Outer-IP + Outer-UDP tunneling with HW Inner (w/o DCF, w/o AVX512)
+===================================================================================
 
-  tcpdump -n -e -i <rx port> -s 0 -w /tmp/cap
+Test for both default path and for AVX512 path by adding EAL Parameter '--force-max-simd-bitwidth=512'.
 
-Launch the userland ``testpmd`` application on DUT as follows::
+After DPDK v22.11, Intel® Ethernet 800 Series NIC with ICE supports tunneling packets on VF.
+This test is used to test tunneling scenario and will enable Outer-IP and Outer-UDP checksum offload.
+Configure the TX HW Offload::
 
-  testpmd> set verbose 1
-  # Enable hw checksum on rx port
-  testpmd> port stop all
-  testpmd> csum set ip hw 0
-  testpmd> csum set tcp hw 0
-  testpmd> csum set udp hw 0
-  testpmd> csum set sctp hw 0
-  testpmd> set promisc 0 on
-  testpmd> port start all
-
-  # Enable TSO on tx port
-  testpmd> tso set 800 1
-  # Set fwd engine and start
-
-  testpmd> set fwd csum
-
-  Due to DPDK 236bc417e2da(app/testpmd: fix MAC header in checksum forward engine) changes the checksum 
-  functions adds switches to control whether to exchange MAC address.
-  Currently, our test scripts are based on not exchanging MAC addresses, mac-swap needs to be disabled:
-  testpmd> csum mac-swap off 0
+    csum set ip hw 1
+    csum set udp hw 1
+    csum set tcp hw 1
+    csum set sctp hw 1
+    csum set outer-ip hw 1
+    csum set outer-udp hw 1
+    csum parse-tunnel on 1
+    
+Send packets of below with bad IP checksum, bad L4 checksum, bad all checksum, good all checksum,
+each packet for multiple times, 10 times usually::
 
-  testpmd> start
+    CSUM NON-TUNNEL PACKETS
+    CSUM NON-TUNNEL VLAN PACKETS
 
-Test IPv4() in scapy::
+Send packets of below with bad Outer-IP checksum, bad Outer-L4 checksum, bad IP checksum, bad L4
+checksum, bad all checksum, good all checksum, each packet for multiple times, 10 times usually::
 
-    sendp([Ether(dst="%s", src="52:00:00:00:00:00")/IP(src="192.168.1.1",dst="192.168.1.2")/TCP(sport=1021,dport=1021)/Raw(load=RandString(size=%s))], iface="%s", count = %s)
+    CSUM TUNNEL PACKETS
+    CSUM TUNNEL VLAN PACKETS
 
-Test IPv6() in scapy::
+Check for the correct checksum insertion at the correct place.
 
-    sendp([Ether(dst="%s", src="52:00:00:00:00:00")/IPv6(src="FE80:0:0:0:200:1FF:FE00:200", dst="3555:5555:6666:6666:7777:7777:8888:8888")/TCP(sport=1021,dport=1021)/Raw(load=RandString(size=%s))], iface="%s", count = %s)
+For VXLAN packets, we run an extra case on DCF::
+    
+    csum set ip hw 1
+    csum set udp hw 1
+    csum set tcp hw 1
+    csum set sctp hw 1
+    csum set outer-ip hw 1
+    csum set outer-udp hw 1
+    csum parse-tunnel on 1
+    rx_vxlan_port add 4789 0
 
-Test case: csum fwd engine, use tunnel TSO
-==========================================
-In DPDK 22.11 release, Intel® Ethernet 800 Series NIC with ICE supports HW
-TSO for tunneling packets. For the packets involved in this case, a ICE COMMON
-DDP Package is required.
+Send packets of below with bad Outer-IP checksum, bad Outer-L4 checksum, bad IP checksum, bad L4
+checksum, bad all checksum, good all checksum, each packet for multiple times, 10 times usually::
 
-This test uses ``Scapy`` to send out one large tunneled TCP package. The dut
-forwards package with tunnel TSO enable on tx port while rx port turns checksum
-on. After package send out by TSO on tx port, the tester receives multiple small
-TCP package.
+    CSUM TUNNEL PACKETS OF VXLAN
+    CSUM TUNNEL VLAN PACKETS OF VXLAN
 
-Turn off tx port by ethtool on tester::
+Check for the correct checksum insertion at the correct place.
 
-  ethtool -K <tx port> rx off tx off tso off gso off gro off lro off
-  ip l set <tx port> up
 
-Capture package rx port on tester::
+Test case 10: SW non-tunneling (w/o VLAN)
+=========================================
 
-  tcpdump -n -e -i <rx port> -s 0 -w /tmp/cap
+This test is used to test tunneling scenario and will enable no checksum offload.
+Configured in default.
+Send packets of below with bad IP checksum, bad L4 checksum, bad all checksum, good all checksum,
+each packet for multiple times, 10 times usually::
 
-Launch the userland ``testpmd`` application on DUT as follows::
+    CSUM NON-TUNNEL PACKETS
+    CSUM NON-TUNNEL VLAN PACKETS (for i40e and ice)
 
-  testpmd> set verbose 1
-  # Enable hw checksum on rx port
-  testpmd> port stop all
-  testpmd> csum set ip hw 0
-  testpmd> csum set tcp hw 0
-  testpmd> csum set udp hw 0
-  testpmd> csum set sctp hw 0
-  testpmd> csum set outer-ip hw 0
-  testpmd> csum set outer-udp hw 0
-  testpmd> csum parse-tunnel on 0
-  testpmd> set promisc 0 on
-  testpmd> port start all
+Check for the correct checksum insertion at the correct place.
+
+For PMD not supporting VLAN defaultly, we run an extra case::
+    
+    vlan set filter on 0
+    rx_vlan add 100 0
 
-  # Enable TSO on tx port
-  testpmd> tunnel_tso set 800 1
-  # Set fwd engine and start
+Send packets of below with bad IP checksum, bad L4 checksum, bad all checksum, good all checksum,
+each packet for multiple times, 10 times usually::
 
-  testpmd> set fwd csum
-  testpmd> start
+    CSUM NON-TUNNEL VLAN PACKETS
 
-Test IPv4() in scapy::
+Check for the correct checksum insertion at the correct place.
 
-  for one_outer_packet in outer_packet_list:
-    sendp([Ether(dst="%s", src="52:00:00:00:00:00")/one_outer_packet/IP(src="192.168.1.1",dst="192.168.1.2")/TCP(sport=1021,dport=1021)/Raw(load=RandString(size=%s))], iface="%s", count = %s)
 
-Test IPv6() in scapy::
+Test case 11: SW tunneling (w/o DCF, w/o AVX512)
+=================================================
 
-  for one_outer_packet in outer_packet_list:
-    sendp([Ether(dst="%s", src="52:00:00:00:00:00")/one_outer_packet/IPv6(src="FE80:0:0:0:200:1FF:FE00:200", dst="3555:5555:6666:6666:7777:7777:8888:8888")/TCP(sport=1021,dport=1021)/Raw(load=RandString(size=%s))], iface="%s", count = %s)
+Test for both default path and for AVX512 path by adding EAL Parameter '--force-max-simd-bitwidth=512'.
 
-Test case: csum fwd engine, use TSO, on scalar path
-===================================================
-These set of cases based on existing cases are designed for better case managment for
-regression test.
+After DPDK v22.11, Intel® Ethernet 800 Series NIC with ICE supports tunneling packets on VF.
+This test is used to test tunneling scenario and will enable no checksum offload.
+Configure the TX HW Offload::
 
-Start testpmd with eal parameter --force-max-simd-bitwidth=64. Based on test steps of
-'csum fwd engine, use TSO'.
+    csum parse-tunnel on 1
+    
+Send packets of below with bad IP checksum, bad L4 checksum, bad all checksum, good all checksum,
+each packet for multiple times, 10 times usually::
 
-Test case: csum fwd engine, use TSO, on sse path
-================================================
+    CSUM NON-TUNNEL PACKETS
+    CSUM NON-TUNNEL VLAN PACKETS (for i40e and ice)
+
+Send packets of below with bad Outer-IP checksum, bad Outer-L4 checksum, bad IP checksum, bad L4
+checksum, bad all checksum, good all checksum, each packet for multiple times, 10 times usually::
+
+    CSUM TUNNEL PACKETS (common packets for common DDP)
+    CSUM TUNNEL VLAN PACKETS (for i40e and ice, common packets for common DDP)
+
+Check for the correct checksum insertion at the correct place.
+
+For VXLAN packets, we run an extra case on DCF::
+    
+    csum parse-tunnel on 1
+    rx_vxlan_port add 4789 0
 
-Test case: csum fwd engine, use TSO, on avx2 path
+Send packets of below with bad Outer-IP checksum, bad Outer-L4 checksum, bad IP checksum, bad L4
+checksum, bad all checksum, good all checksum, each packet for multiple times, 10 times usually::
+
+    CSUM TUNNEL PACKETS OF VXLAN
+    CSUM TUNNEL VLAN PACKETS OF VXLAN
+
+Check for the correct checksum insertion at the correct place.
+
+
+Test case 12: TSO (w/o VLAN, w/o DCF)
 =================================================
 
-Test case: csum fwd engine, use TSO, on avx512 path
-===================================================
+This test is used to test TSO and will enable all HW checksum offload with TSO.
+Configure the TX HW Offload::
 
-Test case: csum fwd engine, use tunnel TSO, on scalar path
-==========================================================
+    csum set ip hw 1
+    csum set udp hw 1
+    csum set tcp hw 1
+    csum set sctp hw 1
+    csum set outer-ip hw 1
+    csum set outer-udp hw 1
+    csum parse-tunnel on 1
+    tso set 800 1
+    
+Send packets of below with bad all checksum, good all checksum, payload size ranges
+in 128, 800, 801, 1700, 2500, 8500, each packet for multiple times, 10 times usually::
 
-Test case: csum fwd engine, use tunnel TSO, on sse path
-=======================================================
+    TSO NON-TUNNEL PACKETS
+    TSO TUNNEL PACKETS
+    UFO NON-TUNNEL PACKETS
+    UFO TUNNEL PACKETS
+
+Check for the correct checksum insertion at the correct place, check for the correct payload segmentation.
+
+
+Test case 13: TSO tunneling (w/o VLAN, w/o DCF)
+===========================================================
 
-Test case: csum fwd engine, use tunnel TSO, on avx2 path
-========================================================
+This test is used to test Tunnel TSO and will enable all HW checksum offload with Tunnel TSO.
+Configure the TX HW Offload::
+
+    csum set ip hw 1
+    csum set udp hw 1
+    csum set tcp hw 1
+    csum set sctp hw 1
+    csum set outer-ip hw 1
+    csum set outer-udp hw 1
+    csum parse-tunnel on 1
+    tunnel_tso set 800 1
+    
+Send packets of below with bad all checksum, good all checksum, payload size ranges
+in 128, 800, 801, 1700, 2500, 8500, each packet for multiple times, 10 times usually::
+
+    TSO NON-TUNNEL PACKETS
+    TSO TUNNEL PACKETS
+    UFO NON-TUNNEL PACKETS
+    UFO TUNNEL PACKETS
+
+Check for the correct checksum insertion at the correct place, check for the correct payload segmentation.
+
+
+Test case 14: TSO combine (w/o VLAN, w/o DCF, w/o AVX512)
+=========================================================
+
+This test is used to test TSO and Tunnel TSO, will enable all HW checksum offload with TSO and Tunnel TSO.
+Configure the TX HW Offload::
+
+    csum set ip hw 1
+    csum set udp hw 1
+    csum set tcp hw 1
+    csum set sctp hw 1
+    csum set outer-ip hw 1
+    csum set outer-udp hw 1
+    csum parse-tunnel on 1
+    tso set 800 1
+    tunnel_tso set 800 1
+    
+Send packets of below with bad all checksum, good all checksum, payload size ranges
+in 128, 800, 801, 1700, 2500, 8500, each packet for multiple times, 10 times usually::
+
+    TSO NON-TUNNEL PACKETS
+    TSO TUNNEL PACKETS
+    UFO NON-TUNNEL PACKETS
+    UFO TUNNEL PACKETS
+
+Check for the correct checksum insertion at the correct place, check for the correct payload segmentation.
 
-Test case: csum fwd engine, use tunnel TSO, on avx512 path
-==========================================================