[V2,3/6] test_plans/tso: Refactor TSO case.

Message ID 20230718012629.2471811-4-ke1.xu@intel.com (mailing list archive)
State Superseded
Headers
Series Rewrite test suite for checksum_offload, tso and vf_offload |

Commit Message

Ke Xu July 18, 2023, 1:26 a.m. UTC
Update tso testplan.

1. Add Common test steps and packet organization table to better show the test steps.

2. Updated test cases and test scopes. Covering TSO tunnel, non-tunnel and combined scenario seperately.

3. Update cases for NIC comaptibility. Add description for VLAN, DCF VxLAN and AVX512 to fit specific testing scenarios required by several NICs.

4. RX scopes and offload flags are combined with varied packet type TX cases for better covering and more common validation.

Signed-off-by: Ke Xu <ke1.xu@intel.com>
---
 test_plans/tso_test_plan.rst | 489 ++++++++++++++++++++++++++---------
 1 file changed, 366 insertions(+), 123 deletions(-)
  

Patch

diff --git a/test_plans/tso_test_plan.rst b/test_plans/tso_test_plan.rst
index e5122bb9..19b6a116 100644
--- a/test_plans/tso_test_plan.rst
+++ b/test_plans/tso_test_plan.rst
@@ -1,5 +1,5 @@ 
 .. SPDX-License-Identifier: BSD-3-Clause
-   Copyright(c) 2015-2017 Intel Corporation
+   Copyright(c) 2015-2023 Intel Corporation
 
 =========================================
 Transmit Segmentation Offload (TSO) Tests
@@ -10,23 +10,56 @@  Description
 
 This document provides the plan for testing the TSO (Transmit Segmentation
 Offload, also called Large Send offload - LSO) feature of
-Intel Ethernet Controller, including Intel 82599 10GbE Ethernet Controller and
-Intel® Ethernet Converged Network Adapter XL710-QDA2. TSO enables the TCP/IP stack to
+Intel Ethernet Controller, including Intel 82599 10GbE Ethernet Controller,
+Intel® Ethernet Converged Network Adapter 700 Series and Intel® Ethernet Converged
+Network Adapter 800 Series. TSO enables the TCP/IP stack to
 pass to the network device a larger ULP datagram than the Maximum Transmit
 Unit Size (MTU). NIC divides the large ULP datagram to multiple segments
 according to the MTU size.
 
+The support of TX TSO features by Poll Mode Drivers consists in:
+
+On the TX side:
+
+- TCP/UDP segmentation.
+- Tunneled inner TCP/UDP segmentation.
+
+RX/TX side, the TSO can be enabled with the following command of the ``testpmd`` application and running in tx
+checksum mode with enabled checksum support::
+
+    set fwd csum
+    csum set ...
+    tso set <segment_length> <port_id>
+    tunnel_tso set <segment_length> <port_id>
+
+Please notice here that in DPDK TSO refers to both TCP Segmentation Offload (TSO)
+and UDP Fragmentation Offload (UFO). DPDK TSO is also different from Generic Segmentation
+Offload (GSO) for there is a GSO lib implemented elsewhere in DPDK. We will use TSO and UFO
+to descripe packets and test steps, but only TSO configurations on ``testpmd`` are involved.
+
+The transmission of packet is done with the ``start`` command of the ``testpmd``
+application that will receive packets and then transmit the packet out on all
+configured ports.
 
 Prerequisites
 =============
 
-Hardware:
-   Intel® Ethernet 700 Series, Intel® Ethernet 800 Series and 82599/500 Series
+Topology
+--------
+
+    dut_port_0 <---> tester_port_0
+
+    dut_port_1 <---> tester_port_1
+
+Software
+--------
 
-The DUT must take one of the Ethernet controller ports connected to a port on another
-device that is controlled by the Scapy packet generator.
+    - dpdk: http://dpdk.org/git/dpdk
+    - scapy: http://www.secdev.org/projects/scapy/
+
+Configuration and Setup
+-----------------------
 
-The Ethernet interface identifier of the port that Scapy will use must be known.
 On tester, all offload feature should be disabled on tx port, and start rx port capture::
 
   ifconfig <tx port> mtu 9000
@@ -48,121 +81,331 @@  and checksum on rx port. The test commands is below::
   # enable TSO on tx port
   *tso set 800 1
 
-
-Test case: csum fwd engine, use TSO
-===================================
-
-This test uses ``Scapy`` to send out one large TCP package. The dut forwards package
-with TSO enable on tx port while rx port turns checksum on. After package send out
-by TSO on tx port, the tester receives multiple small TCP package.
-
-Turn off tx port by ethtool on tester::
-
-  ethtool -K <tx port> rx off tx off tso off gso off gro off lro off
-  ip l set <tx port> up
-
-capture package rx port on tester::
-
-  tcpdump -n -e -i <rx port> -s 0 -w /tmp/cap
-
-Launch the userland ``testpmd`` application on DUT as follows::
-
-   ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -c 0xffffffff -n 2 -- -i --rxd=512 --txd=512
-   --burst=32 --rxfreet=64 --mbcache=128 --portmask=0x3 --txpt=36 --txht=0 --txwt=0
-   --txfreet=32 --txrst=32 --enable-rx-cksum
-     testpmd> set verbose 1
-     # should stop ports before set csum and start ports after the settings
-     testpmd> port stop all
-   # enable hw checksum on rx port
-   testpmd> csum set ip hw 0
-   testpmd> csum set udp hw 0
-   testpmd> csum set tcp hw 0
-   testpmd> csum set sctp hw 0
-   testpmd> csum set outer-ip hw 0
-   testpmd> csum parse-tunnel on 0
-   # enable TSO on tx port
-   testpmd> tso set 800 1
-   # set fwd engine and start
-   testpmd> set fwd csum
-   testpmd> port start all
-   testpmd> set promisc all off
-   testpmd> start
-
-Test IPv4() in scapy::
-
-    sendp([Ether(dst="%s", src="52:00:00:00:00:00")/IP(src="192.168.1.1",dst="192.168.1.2")/UDP(sport=1021,dport=1021)/Raw(load="\x50"*%s)], iface="%s")
-
-Test IPv6() in scapy::
-
-    sendp([Ether(dst="%s", src="52:00:00:00:00:00")/IPv6(src="FE80:0:0:0:200:1FF:FE00:200", dst="3555:5555:6666:6666:7777:7777:8888:8888")/UDP(sport=1021,dport=1021)/Raw(load="\x50"*%s)], iface="%s"
-
-Test case: csum fwd engine, use TSO tunneling
-=============================================
-not support nic: IXGBE_10G-82599_SFP, IGC-I225_LM, IGC-I226_LM.
-
-This test uses ``Scapy`` to send out one large TCP package. The dut forwards package
-with TSO enable on tx port while rx port turns checksum on. After package send out
-by TSO on tx port, the tester receives multiple small TCP package.
-
-Turn off tx port by ethtool on tester::
-
-  ethtool -K <tx port> rx off tx off tso off gso off gro off lro off
-  ip l set <tx port> up
-
-capture package rx port on tester::
-
-  tcpdump -n -e -i <rx port> -s 0 -w /tmp/cap
-
-Launch the userland ``testpmd`` application on DUT as follows::
-
-   ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -c 0xffffffff -n 2 -- -i --rxd=512 --txd=512
-   --burst=32 --rxfreet=64 --mbcache=128 --portmask=0x3 --txpt=36 --txht=0 --txwt=0
-   --txfreet=32 --txrst=32 --enable-rx-cksum
-     testpmd> set verbose 1
-
-     testpmd> port stop all
-   # enable hw checksum on rx port
-   testpmd> csum set ip hw 0
-   testpmd> csum set udp hw 0
-   testpmd> csum set tcp hw 0
-   testpmd> csum set sctp hw 0
-   testpmd> csum set outer-ip hw 0
-   #Intel® Ethernet 700 Series not support outer udp
-   testpmd> csum set outer-udp hw 0
-   testpmd> csum parse-tunnel on 0
-
-   # enable hw checksum on tx port
-   testpmd> csum set ip hw 1
-   testpmd> csum set udp hw 1
-   testpmd> csum set tcp hw 1
-   testpmd> csum set sctp hw 1
-   #csum set outer-ip must be set to hw if outer L3 is IPv4
-   testpmd> csum set outer-ip hw 1
-   #csum parse-tunnel must be set so that tunneled packets are recognized
-   testpmd> csum parse-tunnel on 1
-   #Intel® Ethernet 700 Series not support outer udp
-   testpmd> csum set outer-udp hw 1
-
-   # enable TSO on tx port
-   testpmd> tunnel_tso set 800 1
-   # enable VXLAN protocol on ports
-   testpmd> rx_vxlan_port add 4789 0
-   # set fwd engine and start
-   testpmd> set fwd csum
-   testpmd> port start all
-   testpmd> set promisc all off
-   testpmd> start
-
-Test vxlan() in scapy::
-
-    sendp([Ether(dst="%s",src="52:00:00:00:00:00")/IP(src="192.168.1.1",dst="192.168.1.2")/UDP(sport=1021,dport=4789)/VXLAN(vni=1234)/Ether(dst=%s,src="52:00:00:00:00:00")/IP(src="192.168.1.1",dst="192.168.1.2")/UDP(sport=1021,dport=1021)/Raw(load="\x50"*%s)], iface="%s"
-
-Test nvgre() in scapy::
-
-    sendp([Ether(dst="%s",src="52:00:00:00:00:00")/IP(src="192.168.1.1",dst="192.168.1.2",proto=47)/GRE(key_present=1,proto=0x6558,key=0x00001000)/Ether(dst="%s",src="52:00:00:00:00:00")/IP(src="192.168.1.1",dst="192.168.1.2")/TCP(sport=1021,dport=1021)/("X"*%s)], iface="%s")
-
-Test case: TSO performance
-==========================
+Common test steps
+=================
+
+This test uses ``Scapy`` to send out large TCP/UDP packages. The dut forwards packages
+with checksum offload and TSO enabled on tx port. After package send out by TSO on tx port,
+the tester receives multiple small TCP/UDP packages and verify them along with the verbose
+and stats.
+
+1. launch test-pmd to enable RX Checksum offload and for some NIC it enables a general set of RX Offloads::
+
+    ./dpdk/x86_64-native-linuxapp-gcc/app/dpdk-testpmd -c <...> -n <...> -a <...> -- -i --enable-rx-cksum
+
+2. config testpmd::
+
+    set csum fwd mode and stop all port::
+
+        testpmd> set fwd csum
+        testpmd> port stop all
+
+    Modify the following commands to configure checksum offload according to the test scenario of each case:
+
+        testpmd> csum set ip|tcp|udp|sctp|outer-ip|outer-udp hw|sw port_id
+
+    start all ports and fwd:
+
+        testpmd> port start all
+        testpmd> set verbose 1
+        testpmd> start
+
+3. start to capture packets on tester::
+
+    tester: tcpdump -i <iface> -Q in -e ether src 52:00:00:00:00:00 -w /tmp/ens10.pcap
+
+4. send packets with scapy::
+
+    Configure the packet generator to send multiple combinations of inner and outer packet parts
+    according to the test scenario of each case, refer to the following table:
+
++------------------+---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+| Packet Types     | Packet Names              |                                                    Packet organizations                                                      |
++==================+===========================+==============================================================================================================================+
+|                  | IP/UDP                    | IP(src="10.0.0.1") / UDP(sport=29999, dport=30000) / Raw(randstring(PAYLOAD_SIZE))                                           |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  | IP/TCP                    | IP(src="10.0.0.1") / TCP(sport=29999, dport=30000) / Raw(randstring(PAYLOAD_SIZE))                                           |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  | IP/SCTP                   | IP(src="10.0.0.1") / SCTP(sport=29999, dport=30000, chksum=0x0000) / Raw(randstring(PAYLOAD_SIZE))                           |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  | IPv6/UDP                  | IPv6(src="::1") / UDP(sport=29999, dport=30000) / Raw(randstring(PAYLOAD_SIZE))                                              |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  | IPv6/TCP                  | IPv6(src="::1") / TCP(sport=29999, dport=30000) / Raw(randstring(PAYLOAD_SIZE))                                              |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+| Inner Parts      | IPv6/SCTP                 | IPv6(src="::1") / SCTP(sport=29999, dport=30000, chksum=0x0000) / Raw(randstring(PAYLOAD_SIZE))                              |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  | VLAN/IP/UDP               | Dot1Q(vlan=100) / IP(src="10.0.0.1") / UDP(sport=29999, dport=30000) / Raw(randstring(PAYLOAD_SIZE))                         |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  | VLAN/IP/TCP               | Dot1Q(vlan=100) / IP(src="10.0.0.1") / TCP(sport=29999, dport=30000) / Raw(randstring(PAYLOAD_SIZE))                         |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  | VLAN/IP/SCTP              | Dot1Q(vlan=100) / IP(src="10.0.0.1") / SCTP(sport=29999, dport=30000, chksum=0x0000) / Raw(randstring(PAYLOAD_SIZE))         |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  | VLAN/IPv6/UDP             | Dot1Q(vlan=100) / IPv6(src="::1") / UDP(sport=29999, dport=30000) / Raw(randstring(PAYLOAD_SIZE))                            |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  | VLAN/IPv6/TCP             | Dot1Q(vlan=100) / IPv6(src="::1") / TCP(sport=29999, dport=30000) / Raw(randstring(PAYLOAD_SIZE))                            |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  | VLAN/IPv6/SCTP            | Dot1Q(vlan=100) / IPv6(src="::1") / SCTP(sport=29999, dport=30000, chksum=0x0000) / Raw(randstring(PAYLOAD_SIZE))            |
++------------------+---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IP/VXLAN                   | IP(src="10.0.0.1") / UDP(dport=4789) / VXLAN() / Ether()                                                                     |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IPv6/VXLAN                 | IPv6() / UDP(dport=4789) / VXLAN() / Ether()                                                                                 |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IP/VXLAN-GPE               | IP(src="10.0.0.1") / UDP(sport=4790, dport=4790) / VXLAN()                                                                   |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IPv6/VXLAN-GPE             | IPv6() / UDP(sport=4790, dport=4790) / VXLAN()                                                                               |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IP/VXLAN-GPE/Ether         | IP(src="10.0.0.1") / UDP(sport=4790, dport=4790) / VXLAN() / Ether()                                                         |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IPv6/VXLAN-GPE/Ether       | IPv6() / UDP(sport=4790, dport=4790) / VXLAN() / Ether()                                                                     |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IP/GRE                     | IP(src="10.0.0.1", proto=47) / GRE()                                                                                         |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IPv6/GRE                   | IPv6(nh=47) / GRE()                                                                                                          |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IP/GRE/Ether               | IP(src="10.0.0.1", proto=47) / GRE() / Ether()                                                                               |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IPv6/GRE/Ether             | IPv6(nh=47) / GRE() / Ether()                                                                                                |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IP/NVGRE                   | IP(src="10.0.0.1", proto=47) / GRE(key_present=1, proto=0x6558, key=0x00000100) / Ether()                                    |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IPv6/NVGRE                 | IPv6(nh=47) / GRE(key_present=1, proto=0x6558, key=0x00000100) / Ether()                                                     |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IP/GTPU                    | IP(src="10.0.0.1") / UDP(dport=2152) / GTP_U_Header(gtp_type=255, teid=0x123456)                                             |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IPv6/GTPU                  | IPv6() / UDP(dport=2152) / GTP_U_Header(gtp_type=255, teid=0x123456)                                                         |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IP/GENEVE                  | IP(src="10.0.0.1") / UDP(dport=6081, sport=29999) / GENEVE()                                                                 |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IPv6/GENEVE                | IPv6() / UDP(dport=6081, sport=29999) / GENEVE()                                                                             |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |IP                         | IP(src="10.0.0.1")                                                                                                           |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+| Outer Parts      |IPv6                       | IPv6()                                                                                                                       |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IP/VXLAN              | Dot1Q(vlan=100) / IP(src="10.0.0.1") / UDP(dport=4789) / VXLAN() / Ether()                                                   |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IPv6/VXLAN            | Dot1Q(vlan=100) / IPv6() / UDP(dport=4789) / VXLAN() / Ether()                                                               |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IP/VXLAN-GPE          | Dot1Q(vlan=100) / IP(src="10.0.0.1") / UDP(sport=4790, dport=4790) / VXLAN()                                                 |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IPv6/VXLAN-GPE        | Dot1Q(vlan=100) / IPv6() / UDP(sport=4790, dport=4790) / VXLAN()                                                             |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IP/VXLAN-GPE/Ether    | Dot1Q(vlan=100) / IP(src="10.0.0.1") / UDP(sport=4790, dport=4790) / VXLAN() / Ether()                                       |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IPv6/VXLAN-GPE/Ether  | Dot1Q(vlan=100) / IPv6() / UDP(sport=4790, dport=4790) / VXLAN() / Ether()                                                   |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IP/GRE                | Dot1Q(vlan=100) / IP(src="10.0.0.1", proto=47) / GRE()                                                                       |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IPv6/GRE              | Dot1Q(vlan=100) / IPv6(nh=47) / GRE()                                                                                        |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IP/GRE/Ether          | Dot1Q(vlan=100) / IP(src="10.0.0.1", proto=47) / GRE() / Ether()                                                             |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IPv6/GRE/Ether        | Dot1Q(vlan=100) / IPv6(nh=47) / GRE() / Ether()                                                                              |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IP/NVGRE              | Dot1Q(vlan=100) / IP(src="10.0.0.1", proto=47) / GRE(key_present=1, proto=0x6558, key=0x00000100) / Ether()                  |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IPv6/NVGRE            | Dot1Q(vlan=100) / IPv6(nh=47) / GRE(key_present=1, proto=0x6558, key=0x00000100) / Ether()                                   |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IP/GTPU               | Dot1Q(vlan=100) / IP(src="10.0.0.1") / UDP(dport=2152) / GTP_U_Header(gtp_type=255, teid=0x123456)                           |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IPv6/GTPU             | Dot1Q(vlan=100) / IPv6() / UDP(dport=2152) / GTP_U_Header(gtp_type=255, teid=0x123456)                                       |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IP/GENEVE             | Dot1Q(vlan=100) / IP(src="10.0.0.1") / UDP(dport=6081, sport=29999) / GENEVE()                                               |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IPv6/GENEVE           | Dot1Q(vlan=100) / IPv6() / UDP(dport=6081, sport=29999) / GENEVE()                                                           |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IP                    | Dot1Q(vlan=100) / IP(src="10.0.0.1")                                                                                         |
+|                  +---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+|                  |VLAN/IPv6                  | Dot1Q(vlan=100) / IPv6()                                                                                                     |
++------------------+---------------------------+------------------------------------------------------------------------------------------------------------------------------+
+
++------------------+------------------+----------------------------+
+| Scenarios        | Packet Types     | Packet Names               |
++==================+==================+============================+
+| TSO NON-TUNNEL   |                  | IP/TCP                     |
+| PACKETS          | Inner parts      +----------------------------+
+|                  |                  | IPv6/TCP                   |
++------------------+------------------+----------------------------+
+|                  |                  | IP/VXLAN-GPE               |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/VXLAN-GPE             |
+|                  |                  +----------------------------+
+|                  |                  | IP/VXLAN-GPE/Ether         |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/VXLAN-GPE/Ether       |
+|                  |                  +----------------------------+
+|                  |                  | IP/GRE                     |
+|                  |                  +----------------------------+
+| TSO TUNNEL       | Outer parts      | IPv6/GRE                   |
+| PACKETS          |                  +----------------------------+
+|                  |                  | IP/GRE/Ether               |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/GRE/Ether             |
+|                  |                  +----------------------------+
+|                  |                  | IP/NVGRE                   |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/NVGRE                 |
+|                  |                  +----------------------------+
+|                  |                  | IP/GTPU                    |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/GTPU                  |
+|                  +------------------+----------------------------+
+|                  |                  | IP/TCP                     |
+|                  | Inner parts      +----------------------------+
+|                  |                  | IPv6/TCP                   |
++------------------+------------------+----------------------------+
+| UFO NON-TUNNEL   |                  | IP/UDP                     |
+| PACKETS          | Inner parts      +----------------------------+
+|                  |                  | IPv6/UDP                   |
++------------------+------------------+----------------------------+
+|                  |                  | IP/VXLAN-GPE               |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/VXLAN-GPE             |
+|                  |                  +----------------------------+
+|                  |                  | IP/VXLAN-GPE/Ether         |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/VXLAN-GPE/Ether       |
+|                  |                  +----------------------------+
+|                  |                  | IP/GRE                     |
+|                  |                  +----------------------------+
+| UFO TUNNEL       | Outer parts      | IPv6/GRE                   |
+| PACKETS          |                  +----------------------------+
+|                  |                  | IP/GRE/Ether               |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/GRE/Ether             |
+|                  |                  +----------------------------+
+|                  |                  | IP/NVGRE                   |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/NVGRE                 |
+|                  |                  +----------------------------+
+|                  |                  | IP/GTPU                    |
+|                  |                  +----------------------------+
+|                  |                  | IPv6/GTPU                  |
+|                  +------------------+----------------------------+
+|                  |                  | IP/UDP                     |
+|                  | Inner parts      +----------------------------+
+|                  |                  | IPv6/UDP                   |
++------------------+------------------+----------------------------+
+
+5. stop capture packets and then stop fwd::
+
+    tester:     pkill tcpdump
+    dut:        testpmd> stop
+
+6. check rx and tx statistics according to the expected segmented packets.
+
++------------------+-------------------------------------------------------------------+
+|      entry       |                            checkpoint                             |
++==================+===================================================================+
+| RX-packets       | `RX-packets` =  Num of sent packets                               |
++------------------+-------------------------------------------------------------------+
+| RX-total         | `RX-total` = Num of sent packets                                  |
++------------------+-------------------------------------------------------------------+
+| Bad-ipcsum       | `Bad-ipcsum` = Sum of bad inner IP checksum in sent packets       |
++------------------+-------------------------------------------------------------------+
+| Bad-l4csum       | `Bad-l4csum` = Sum of bad inner L4 checksum in sent packets       |
++------------------+-------------------------------------------------------------------+
+| Bad-outer-ipcsum | `Bad-outer-ipcsum` = Sum of bad outer IP checksum in sent packets |
++------------------+-------------------------------------------------------------------+
+| Bad-outer-l4csum | `Bad-outer-l4csum` = Sum of bad outer L4 checksum in sent packets |
++------------------+-------------------------------------------------------------------+
+| TX-packets       | `TX-packets` =  Sum of segmented packets                          |
++------------------+-------------------------------------------------------------------+
+| TX-total         | `TX-total` =  Sum of segmented packets                            |
++------------------+-------------------------------------------------------------------+
+
+    Read pcap files with scapy and check if configured checksum values to insert of tx_packets are all correct::
+
+        >>> tx_pkts = rdpcap("/tmp/ens10.pcap") # tx_packets is of list of all tx pkts
+        >>> tx_pkts[0].show2() # will print all info about the first packet, compare the checksum with the ref checksum.
+
+    Verify if the segmented payload matches the send payload::
+        
+        >>> tx_pkts[0].load == send_pkt.load[ : SEG_LENGTH ]
+        >>> tx_pkts[1].load == send_pkt.load[ SEG_LENGTH : 2*SEG_LENGTH ]
+        >>> ...
+
+
+Test case 1: TSO (w/o VLAN)
+===========================
+
+This test is used to test TSO and will enable all HW checksum offload with TSO.
+Configure the TX HW Offload::
+
+    csum set ip hw 1
+    csum set udp hw 1
+    csum set tcp hw 1
+    csum set sctp hw 1
+    csum set outer-ip hw 1
+    csum set outer-udp hw 1
+    csum parse-tunnel on 1
+    tso set 800 1
+    
+Send packets of below with bad all checksum, good all checksum, payload size ranges
+in 128, 800, 801, 1700, 2500, 8500, each packet for multiple times, 10 times usually::
+
+    TSO NON-TUNNEL PACKETS
+    TSO TUNNEL PACKETS
+    UFO NON-TUNNEL PACKETS
+    UFO TUNNEL PACKETS
+
+Check for the correct checksum insertion at the correct place, check for the correct payload segmentation.
+
+
+Test case 2: TSO tunneling (w/o VLAN)
+=====================================
+
+This test is used to test Tunnel TSO and will enable all HW checksum offload with Tunnel TSO.
+Configure the TX HW Offload::
+
+    csum set ip hw 1
+    csum set udp hw 1
+    csum set tcp hw 1
+    csum set sctp hw 1
+    csum set outer-ip hw 1
+    csum set outer-udp hw 1
+    csum parse-tunnel on 1
+    tunnel_tso set 800 1
+    
+Send packets of below with bad all checksum, good all checksum, payload size ranges
+in 128, 800, 801, 1700, 2500, 8500, each packet for multiple times, 10 times usually::
+
+    TSO NON-TUNNEL PACKETS
+    TSO TUNNEL PACKETS
+    UFO NON-TUNNEL PACKETS
+    UFO TUNNEL PACKETS
+
+Check for the correct checksum insertion at the correct place, check for the correct payload segmentation.
+
+
+Test case 3: TSO combine (w/o VLAN, w/o AVX512)
+===============================================
+
+This test is used to test TSO and Tunnel TSO, will enable all HW checksum offload with TSO and Tunnel TSO.
+Configure the TX HW Offload::
+
+    csum set ip hw 1
+    csum set udp hw 1
+    csum set tcp hw 1
+    csum set sctp hw 1
+    csum set outer-ip hw 1
+    csum set outer-udp hw 1
+    csum parse-tunnel on 1
+    tso set 800 1
+    tunnel_tso set 800 1
+    
+Send packets of below with bad all checksum, good all checksum, payload size ranges
+in 128, 800, 801, 1700, 2500, 8500, each packet for multiple times, 10 times usually::
+
+    TSO NON-TUNNEL PACKETS
+    TSO TUNNEL PACKETS
+    UFO NON-TUNNEL PACKETS
+    UFO TUNNEL PACKETS
+
+Check for the correct checksum insertion at the correct place, check for the correct payload segmentation.
+
+Test case 4: TSO performance
+============================
 
 Set the packet stream to be sent out from packet generator before testing as
 below.