[v1,2/2] dts: add mtu update and jumbo frames test suite

Message ID 20250117145838.40206-3-npratte@iol.unh.edu (mailing list archive)
State Superseded
Delegated to: Paul Szczepanek
Headers
Series dts: mtu update and jumbo frames test suite |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation warning apply issues

Commit Message

Nicholas Pratte Jan. 17, 2025, 2:58 p.m. UTC
A functional test suite that assesses MTU updating and forwarding within
a DPDK application.

This suite consolidates the previous 'mtu_update' and 'jumbo_frames' test
suites from the old dts framework into a single, comprehensive test suite,
and it covers all of mtu the  adjustment options within the ethdev api.

Bugzilla ID: 1421

Signed-off-by: Nicholas Pratte <npratte@iol.unh.edu>
Signed-off-by: Alex Chapman <alex.chapman@arm.com>
---
 dts/tests/TestSuite_mtu_update_fwding.py | 278 +++++++++++++++++++++++
 1 file changed, 278 insertions(+)
 create mode 100644 dts/tests/TestSuite_mtu_update_fwding.py
  

Comments

Patrick Robb Jan. 20, 2025, 1:47 a.m. UTC | #1
On Fri, Jan 17, 2025 at 9:58 AM Nicholas Pratte <npratte@iol.unh.edu> wrote:

>
> +    def assess_mtu_boundary(self, testpmd_shell: TestPmdShell, mtu: int)
> -> None:
> +        """Sets the new MTU and verifies packets at the set boundary.
> +
> +        Ensure that packets smaller than or equal to a set MTU will be
> received and packets larger
> +        will not.
> +
> +        First, start testpmd and update the MTU. Then ensure the new
> value appears
> +        on port info for all ports.
> +        Next, start packet capturing and send 3 different lengths of
> packet and verify
> +        they are handled correctly.
> +            # 1. VENDOR_AGNOSTIC_PADDING units smaller than the MTU
> specified.
> +            # 2. Equal to the MTU specified.
> +            # 3. VENDOR_AGNOSTIC_PADDING units larger than the MTU
> specified (should be fragmented).
> +        Finally, stop packet capturing.
> +
> +        Args:
> +            testpmd_shell: Active testpmd shell of a given test case.
> +            mtu: New Maximum Transmission Unit to be tested.
> +        """
> +        # Send 3 packets of different sizes (accounting for vendor
> inconsistencies).
> +        # 1. VENDOR_AGNOSTIC_PADDING units smaller than the MTU specified.
> +        # 2. Equal to the MTU specified.
> +        # 3. VENDOR_AGNOSTIC_PADDING units larger than the MTU specified.
> +        smaller_frame_size: int = mtu - VENDOR_AGNOSTIC_PADDING
> +        equal_frame_size: int = mtu
> +        larger_frame_size: int = mtu + VENDOR_AGNOSTIC_PADDING
> +
> +        self.send_packet_and_verify(pkt_size=smaller_frame_size,
> should_receive=True)
> +        self.send_packet_and_verify(pkt_size=equal_frame_size,
> should_receive=True)
> +
> +        current_mtu = testpmd_shell.show_port_info(0).mtu
> +        self.verify(current_mtu is not None, "Error grabbing testpmd MTU
> value.")
> +        if current_mtu and (
> +            current_mtu >= STANDARD_MTU + VENDOR_AGNOSTIC_PADDING and mtu
> == STANDARD_MTU
> +        ):
> +            self.send_packet_and_verify(pkt_size=larger_frame_size,
> should_receive=True)
>

I don't understand when this condition may be true - can you explain?
Thanks!


> +        else:
> +            self.send_packet_and_verify(pkt_size=larger_frame_size,
> should_receive=False)
> +
> +    @func_test
> +    def test_runtime_mtu_updating_and_forwarding(self) -> None:
> +        """Verify runtime MTU adjustments and assess packet forwarding
> behavior.
> +
> +        Test:
> +            Start TestPMD in a paired topology.
> +            Set port MTU to 1500.
> +            Send packets of 1493, 1500 and 1509 bytes.
>

I think 1493 should be 1491.


> --
> 2.47.1
>
>
Thanks, other than a couple questions here and in the associated patch this
looks good. I can merge on Tuesday.

Reviewed-by: Patrick Robb <probb@iol.unh.edu>
Tested-by: Patrick Robb <probb@iol.unh.edu>
  
Nicholas Pratte Jan. 24, 2025, 8:33 p.m. UTC | #2
Thank you for the feedback, see my comments below!

<snip>
>> +        current_mtu = testpmd_shell.show_port_info(0).mtu
>> +        self.verify(current_mtu is not None, "Error grabbing testpmd MTU value.")
>> +        if current_mtu and (
>> +            current_mtu >= STANDARD_MTU + VENDOR_AGNOSTIC_PADDING and mtu == STANDARD_MTU
>> +        ):
>> +            self.send_packet_and_verify(pkt_size=larger_frame_size, should_receive=True)
>
>
> I don't understand when this condition may be true - can you explain? Thanks!

I added some additional testing to Alex's MTU update tests to check
the forwarding of standard 1500 byte packets for MTU sizes greater
than 1500. Practically speaking this means that the suite will check
1500 byte packets for 2400, 4800 and 9000 byte MTU sizes. In all these
cases, packets of size 1509 should be forwarded. In the case of a 1500
byte MTU, this 1509 byte packet should be dropped.

Now the reason I decided to put these test 1500 byte packet tests in
the test suite at all is because the 'jumbo_frames' suite performs a
similar test; an MTU size of 9000 bytes is set, and standard 1500 byte
packets are checked for forwarding. The original 'mtu_update' doesn't
include this sort of testing in its test plan, but I figured I'd add
on this testing so that both MTU tests are more uniform in their
functional tests. It's a good question because this exact bit of
change in the test is what transformed this suite into an MTU test for
both '--max-pkt-len' and 'port config mtu' since each of these options
use different components of ethdev.

>
>>
>> +        else:
>> +            self.send_packet_and_verify(pkt_size=larger_frame_size, should_receive=False)
>> +
>> +    @func_test
>> +    def test_runtime_mtu_updating_and_forwarding(self) -> None:
>> +        """Verify runtime MTU adjustments and assess packet forwarding behavior.
>> +
>> +        Test:
>> +            Start TestPMD in a paired topology.
>> +            Set port MTU to 1500.
>> +            Send packets of 1493, 1500 and 1509 bytes.
>
>
> I think 1493 should be 1491.

Nice one! I'll fix that.

>
>>
>> --
>> 2.47.1
>>
>
> Thanks, other than a couple questions here and in the associated patch this looks good. I can merge on Tuesday.
>
> Reviewed-by: Patrick Robb <probb@iol.unh.edu>
> Tested-by: Patrick Robb <probb@iol.unh.edu>
  

Patch

diff --git a/dts/tests/TestSuite_mtu_update_fwding.py b/dts/tests/TestSuite_mtu_update_fwding.py
new file mode 100644
index 0000000000..caafc41576
--- /dev/null
+++ b/dts/tests/TestSuite_mtu_update_fwding.py
@@ -0,0 +1,278 @@ 
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 Arm Limited
+# Copyright(c) 2023-2024 University of New Hampshire
+"""MTU update and jumbo frame forwarding test suite.
+
+A suite of tests to ensures the consistency of jumbo and standard frame transmission within a DPDK
+application. If a NIC receives a packet that is greater than its assigned MTU length, then that
+packet should be dropped. Likewise, if a NIC receives a packet that is less than or equal to a
+designated MTU length, the packet should be accepted.
+
+The definition of MTU between individual vendors varies with a +/- difference of 9 bytes, at most.
+To universally test MTU functionality, and not concern over individual vendor behavior, this test
+suite compensates using a 9 byte upper and lower bound when testing a set MTU boundary.
+"""
+
+from scapy.layers.inet import IP
+from scapy.layers.l2 import Ether
+from scapy.packet import Raw
+
+from framework.remote_session.testpmd_shell import TestPmdShell
+from framework.test_suite import TestSuite, func_test
+
+STANDARD_FRAME = 1518  # --max-pkt-len will subtract l2 information at a minimum of 18 bytes.
+JUMBO_FRAME = 9018
+
+STANDARD_MTU = 1500
+JUMBO_MTU = 9000
+
+IP_HEADER_LEN = 20
+VENDOR_AGNOSTIC_PADDING = 9  # Used as a work around for varying MTU definitions between vendors.
+
+
+class TestMtuUpdateFwding(TestSuite):
+    """DPDK PMD jumbo frames and MTU update test suite.
+
+    Assess the expected behavior of frames greater than, less then, or equal to a designated MTU
+    size in a DPDK application.
+
+    Verify the behavior of both runtime MTU and pre-runtime MTU adjustments within DPDK
+    applications. (TestPMD's CLI and runtime MTU adjustment options leverage different logical
+    in components within ethdev to set a value).
+
+    Test cases will verify that any frame greater than an assigned MTU are dropped and packets
+    less than or equal to a designated MTU are forwarded and fully intact.
+    """
+
+    def set_up_suite(self) -> None:
+        """Set up the test suite.
+
+        Setup:
+            Set traffic generator MTU lengths to a size greater than scope of all
+            test cases.
+        """
+        self.tg_node.main_session.configure_port_mtu(JUMBO_MTU + 200, self._tg_port_egress)
+        self.tg_node.main_session.configure_port_mtu(JUMBO_MTU + 200, self._tg_port_ingress)
+
+    def send_packet_and_verify(self, pkt_size: int, should_receive: bool) -> None:
+        """Generate, send a packet, and assess its behavior based on a given packet size.
+
+        Generates a packet based on a specified size and sends it to the SUT. The desired packet's
+        payload size is calculated, and a string of arbrity size, containing a single character,
+        is placed in the packet as payload. This method assesses whether or not it was forwarded,
+        depending on the test case, and does so via a check of the previously-inserted packet
+        payload.
+
+        Args:
+            pkt_size: Size of packet to be generated and sent.
+            should_receive: Indicate whether the test case expects to receive the packet or not.
+        """
+        padding = pkt_size - IP_HEADER_LEN
+        # Insert '    ' as placeholder 'CRC' error correction.
+        packet = Ether() / Raw(load="    ") / IP(len=pkt_size) / Raw(load="X" * padding)
+        received_packets = self.send_packet_and_capture(packet)
+        found = any(
+            ("X" * padding) in str(packets.load)
+            for packets in received_packets
+            if hasattr(packets, "load")
+        )
+
+        if should_receive:
+            self.verify(found, "Did not receive packet.")
+        else:
+            self.verify(not found, "Received packet.")
+
+    def assess_mtu_boundary(self, testpmd_shell: TestPmdShell, mtu: int) -> None:
+        """Sets the new MTU and verifies packets at the set boundary.
+
+        Ensure that packets smaller than or equal to a set MTU will be received and packets larger
+        will not.
+
+        First, start testpmd and update the MTU. Then ensure the new value appears
+        on port info for all ports.
+        Next, start packet capturing and send 3 different lengths of packet and verify
+        they are handled correctly.
+            # 1. VENDOR_AGNOSTIC_PADDING units smaller than the MTU specified.
+            # 2. Equal to the MTU specified.
+            # 3. VENDOR_AGNOSTIC_PADDING units larger than the MTU specified (should be fragmented).
+        Finally, stop packet capturing.
+
+        Args:
+            testpmd_shell: Active testpmd shell of a given test case.
+            mtu: New Maximum Transmission Unit to be tested.
+        """
+        # Send 3 packets of different sizes (accounting for vendor inconsistencies).
+        # 1. VENDOR_AGNOSTIC_PADDING units smaller than the MTU specified.
+        # 2. Equal to the MTU specified.
+        # 3. VENDOR_AGNOSTIC_PADDING units larger than the MTU specified.
+        smaller_frame_size: int = mtu - VENDOR_AGNOSTIC_PADDING
+        equal_frame_size: int = mtu
+        larger_frame_size: int = mtu + VENDOR_AGNOSTIC_PADDING
+
+        self.send_packet_and_verify(pkt_size=smaller_frame_size, should_receive=True)
+        self.send_packet_and_verify(pkt_size=equal_frame_size, should_receive=True)
+
+        current_mtu = testpmd_shell.show_port_info(0).mtu
+        self.verify(current_mtu is not None, "Error grabbing testpmd MTU value.")
+        if current_mtu and (
+            current_mtu >= STANDARD_MTU + VENDOR_AGNOSTIC_PADDING and mtu == STANDARD_MTU
+        ):
+            self.send_packet_and_verify(pkt_size=larger_frame_size, should_receive=True)
+        else:
+            self.send_packet_and_verify(pkt_size=larger_frame_size, should_receive=False)
+
+    @func_test
+    def test_runtime_mtu_updating_and_forwarding(self) -> None:
+        """Verify runtime MTU adjustments and assess packet forwarding behavior.
+
+        Test:
+            Start TestPMD in a paired topology.
+            Set port MTU to 1500.
+            Send packets of 1493, 1500 and 1509 bytes.
+                Verify the first two packets are forwarded and the last is dropped.
+
+            Set port MTU to 2400.
+            Send packets of 1493, 1500 and 1509 bytes.
+                Verify all three packets are forwarded.
+            Send packets of 2393, 2400 and 2409 bytes.
+                Verify the first two packets are forwarded and the last is dropped.
+
+            Set port MTU to 4800.
+            Send packets of 1493, 1500 and 1509 bytes.
+                Verify all three packets are forwarded.
+            Send packets of 4793, 4800 and 4809 bytes.
+                Verify the first two packets are forwarded and the last is dropped.
+
+            Set port MTU to 9000.
+            Send packets of 1493, 1500 and 1509 bytes.
+                Verify all three packets are forwarded.
+            Send packets of 8993, 9000 and 9009 bytes.
+                Verify the first two packets are forwarded and the last is dropped.
+        Verify:
+            Verifies the successful forwarding of packets via a search for an inserted payload.
+            If the payload is found, the packet was transmitted successfully. Otherwise, the packet
+            is considered dropped.
+
+            Verify that standard MTU packets forward, in addition to packets within the limits of
+            an MTU size set during runtime.
+        """
+        with TestPmdShell(
+            self.sut_node,
+            tx_offloads=0x8000,
+            mbuf_size=[JUMBO_MTU + 200],
+        ) as testpmd:
+            # Configure the new MTU.
+
+            # Start packet capturing.
+            testpmd.start()
+
+            testpmd.set_port_mtu_all(1500, verify=True)
+            self.assess_mtu_boundary(testpmd, 1500)
+
+            testpmd.set_port_mtu_all(2400, verify=True)
+            self.assess_mtu_boundary(testpmd, 1500)
+            self.assess_mtu_boundary(testpmd, 2400)
+
+            testpmd.set_port_mtu_all(4800, verify=True)
+            self.assess_mtu_boundary(testpmd, 1500)
+            self.assess_mtu_boundary(testpmd, 4800)
+
+            testpmd.set_port_mtu_all(9000, verify=True)
+            self.assess_mtu_boundary(testpmd, 1500)
+            self.assess_mtu_boundary(testpmd, 9000)
+
+    @func_test
+    def test_cli_mtu_forwarding_for_std_packets(self) -> None:
+        """Assesses packet forwarding of standard MTU packets after pre-runtime MTU adjustments.
+
+        Test:
+            Start TestPMD with MTU size of 1518 bytes, set pre-runtime.
+            Send packets of size 1493, 1500 and 1509 bytes.
+            Verify the first two packets are forwarded and the last is dropped.
+        Verify:
+            Verifies the successful forwarding of packets via a search for an inserted payload.
+            If the payload is found, the packet was transmitted successfully. Otherwise, the packet
+            is considered dropped.
+
+            Verify the first two packets are forwarded and the last is dropped after pre-runtime
+            MTU modification.
+        """
+        with TestPmdShell(
+            self.sut_node,
+            tx_offloads=0x8000,
+            mbuf_size=[JUMBO_MTU + 200],
+            mbcache=200,
+            max_pkt_len=STANDARD_FRAME,
+        ) as testpmd:
+            testpmd.start()
+
+            self.send_packet_and_verify(STANDARD_MTU - VENDOR_AGNOSTIC_PADDING, should_receive=True)
+            self.send_packet_and_verify(STANDARD_MTU, should_receive=True)
+            self.send_packet_and_verify(
+                STANDARD_MTU + VENDOR_AGNOSTIC_PADDING, should_receive=False
+            )
+
+    @func_test
+    def test_cli_jumbo_forwarding_for_jumbo_mtu(self) -> None:
+        """Assess packet forwarding of packets within the bounds of a pre-runtime MTU adjustment.
+
+        Test:
+            Start TestPMD with MTU size of 9018 bytes, set pre-runtime.
+            Send packets of size 8993, 9000 and 1509 bytes.
+        Verify:
+            Verifies the successful forwarding of packets via a search for an inserted payload.
+            If the payload is found, the packet was transmitted successfully. Otherwise, the packet
+            is considered dropped.
+
+            Verify that all packets are forwarded after pre-runtime MTU modification.
+        """
+        with TestPmdShell(
+            self.sut_node,
+            tx_offloads=0x8000,
+            mbuf_size=[JUMBO_MTU + 200],
+            mbcache=200,
+            max_pkt_len=JUMBO_FRAME,
+        ) as testpmd:
+            testpmd.start()
+
+            self.send_packet_and_verify(JUMBO_MTU - VENDOR_AGNOSTIC_PADDING, should_receive=True)
+            self.send_packet_and_verify(JUMBO_MTU, should_receive=True)
+            self.send_packet_and_verify(STANDARD_MTU + VENDOR_AGNOSTIC_PADDING, should_receive=True)
+
+    @func_test
+    def test_cli_mtu_std_packets_for_jumbo_mtu(self) -> None:
+        """Assess boundary of jumbo MTU value set pre-runtime.
+
+        Test:
+            Start TestPMD with MTU size of 9018 bytes, set pre-runtime.
+            Send a packets of size 8993, 9000 and 9009 bytes.
+            Verify the first two packets are forwarded and the last is dropped.
+        Verify:
+            Verifies the successful forwarding of packets via a search for an inserted payload.
+            If the payload is found, the packet was transmitted successfully. Otherwise, the packet
+            is considered dropped.
+
+            Verify the first two packets are forwarded and the last is dropped after pre-runtime
+            MTU modification.
+        """
+        with TestPmdShell(
+            self.sut_node,
+            tx_offloads=0x8000,
+            mbuf_size=[JUMBO_MTU + 200],
+            mbcache=200,
+            max_pkt_len=JUMBO_FRAME,
+        ) as testpmd:
+            testpmd.start()
+
+            self.send_packet_and_verify(JUMBO_MTU - VENDOR_AGNOSTIC_PADDING, should_receive=True)
+            self.send_packet_and_verify(JUMBO_MTU, should_receive=True)
+            self.send_packet_and_verify(JUMBO_MTU + VENDOR_AGNOSTIC_PADDING, should_receive=False)
+
+    def tear_down_suite(self) -> None:
+        """Tear down the test suite.
+
+        Teardown:
+            Set the MTU size of the traffic generator back to the standard 1518 byte size.
+        """
+        self.tg_node.main_session.configure_port_mtu(STANDARD_MTU, self._tg_port_egress)
+        self.tg_node.main_session.configure_port_mtu(STANDARD_MTU, self._tg_port_ingress)