From patchwork Fri Aug 2 17:29:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeremy Spewock X-Patchwork-Id: 142873 X-Patchwork-Delegate: juraj.linkes@pantheon.tech Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D51974571D; Fri, 2 Aug 2024 19:29:59 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D040F4279B; Fri, 2 Aug 2024 19:29:50 +0200 (CEST) Received: from mail-qv1-f100.google.com (mail-qv1-f100.google.com [209.85.219.100]) by mails.dpdk.org (Postfix) with ESMTP id 628EF415D7 for ; Fri, 2 Aug 2024 19:29:48 +0200 (CEST) Received: by mail-qv1-f100.google.com with SMTP id 6a1803df08f44-6b78c980981so44711196d6.2 for ; Fri, 02 Aug 2024 10:29:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1722619788; x=1723224588; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Li2b4eh/xvYBLhveSS+JXRyFjbmyzoNu9BvswS1uIiA=; b=BelSP3d4HOiMHHlK3N1yD2VJhLLGtdmig9uwYAXVMsZfyfKOMbbvAWBlzWk236t0L1 UWaEFGAkRpEuuegNeNsoHp9RlE/TOg+BNF17NWpw6X7gsK/X8pVYAFT+l5GUDzM8T2dS SP9jCxWDqUNdDWpy2Jr7IfbEy4N5KUxgY5rws= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722619788; x=1723224588; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Li2b4eh/xvYBLhveSS+JXRyFjbmyzoNu9BvswS1uIiA=; b=tTucO13u4B90SX69NvKTwjriwTH1WdYCpL9Xchbc+BaZFEiBSO3zEwUPkCbCn80lrx 2NB86RYqeubwR3Rvdo99QSBcOEsUyyFJKPmZTWRHmGNUliEn00m+Q3jS0Qin/jUsMYXS Hf+sRnrBUWg2IJPTUk4p4oJSmvJEFDWBiKldU2LgoWCZbRFwVEaHB9bfXC4WfNowIeC2 rSngw90XSN9iUs3uwDNVbffpC/CoKQ5jYN7ywbEzuYVfy7MYxb+No51wSkmHZ2wBHmCN kbK1gjvC1Fb2oOxBAfbLh6fKpzfRVlfZ2HD0o2EzBM0ZqKIYCCymQIynzyedPZUSxzhU N42A== X-Gm-Message-State: AOJu0Yy0KkyRO+vHcQSaHCasChYgc4lVsiBqoejI5RJZU4wcu8JTzD6h dzSuCeVaG1gKlGFmA6knq774daT9wtJ1eibvatcZqzTdMp/iB4n2sgrU0wzGCRsC7KA8E5Ps9Mu 7kDbAFNSJQw0Z6mjxu2CdJqi9r6x8cc2HBfTQFqIThXzf5+MB X-Google-Smtp-Source: AGHT+IFsOFxwU1TAUvl7gx2fshdWs73n7C2ap5oRUZ7/fA6XJl8AwdIzUST3nUnu73ef6y76oFT1xJ5Rz70+ X-Received: by 2002:a05:6214:4985:b0:6b5:e1fb:68ee with SMTP id 6a1803df08f44-6bb983b8d74mr51078156d6.25.1722619787619; Fri, 02 Aug 2024 10:29:47 -0700 (PDT) Received: from postal.iol.unh.edu (postal.iol.unh.edu. [2606:4100:3880:1234::84]) by smtp-relay.gmail.com with ESMTPS id 6a1803df08f44-6bb9c875b21sm468546d6.63.2024.08.02.10.29.47 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 02 Aug 2024 10:29:47 -0700 (PDT) X-Relaying-Domain: iol.unh.edu Received: from iol.unh.edu (unknown [IPv6:2606:4100:3880:1257::1083]) by postal.iol.unh.edu (Postfix) with ESMTP id D7940605C351; Fri, 2 Aug 2024 13:29:46 -0400 (EDT) From: jspewock@iol.unh.edu To: thomas@monjalon.net, wathsala.vithanage@arm.com, Luca.Vizzarro@arm.com, npratte@iol.unh.edu, yoan.picchi@foss.arm.com, probb@iol.unh.edu, Honnappa.Nagarahalli@arm.com, paul.szczepanek@arm.com, alex.chapman@arm.com, juraj.linkes@pantheon.tech Cc: dev@dpdk.org, Jeremy Spewock Subject: [RFC PATCH v1 2/3] dts: add port stats checks test suite Date: Fri, 2 Aug 2024 13:29:27 -0400 Message-ID: <20240802172928.212277-3-jspewock@iol.unh.edu> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240802172928.212277-1-jspewock@iol.unh.edu> References: <20240802172928.212277-1-jspewock@iol.unh.edu> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Jeremy Spewock This patch adds a new test suite to DTS that validates the accuracy of the port statistics using testpmd. The functionality is tested by sending a packet of a fixed side to the SUT and verifying that the statistic for packets received, received bytes, packets sent, and sent bytes all update accordingly. Depends-on: patch-142762 ("dts: add text parser for testpmd verbose output") Signed-off-by: Jeremy Spewock --- dts/tests/TestSuite_port_stats_checks.py | 156 +++++++++++++++++++++++ 1 file changed, 156 insertions(+) create mode 100644 dts/tests/TestSuite_port_stats_checks.py diff --git a/dts/tests/TestSuite_port_stats_checks.py b/dts/tests/TestSuite_port_stats_checks.py new file mode 100644 index 0000000000..71e1c7906f --- /dev/null +++ b/dts/tests/TestSuite_port_stats_checks.py @@ -0,0 +1,156 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2024 University of New Hampshire + +"""Port Statistics testing suite. + +This test suite tests the functionality of querying the statistics of a port and verifies that the +values provided in the statistics accurately reflect the traffic that has been handled on the port. +This is shown by sending a packet of a fixed size to the SUT and verifying that the number of RX +packets has increased by 1, the number of RX bytes has increased by the specified size, the number +of TX packets has also increased by 1 (since we expect the packet to be forwarded), and the number +of TX bytes has also increased by the same fixed amount. +""" + +from typing import ClassVar, Tuple + +from scapy.layers.inet import IP # type: ignore[import-untyped] +from scapy.layers.l2 import Ether # type: ignore[import-untyped] +from scapy.packet import Packet, Raw # type: ignore[import-untyped] + +from framework.params.testpmd import SimpleForwardingModes +from framework.remote_session.testpmd_shell import TestPmdShell, TestPmdVerboseOutput +from framework.test_suite import TestSuite + + +class TestPortStatsChecks(TestSuite): + """DPDK Port statistics testing suite. + + Support for port statistics is tested by sending a packet of a fixed size denoted by + `total_packet_len` and verifying the that TX/RX packets of the TX/RX ports updated by exactly + 1 and the TX/RX bytes of the TX/RX ports updated by exactly `total_packet_len`. This is done by + finding the total amount of packets that were sent/received which did not originate from this + test suite and taking the sum of the lengths of each of these "noise" packets and subtracting + it from the total values in the port statistics so that all that is left are relevant values. + """ + + #: Port where traffic will be received on the SUT. + recv_port: ClassVar[int] = 0 + #: Port where traffic will be sent from on the SUT. + send_port: ClassVar[int] = 1 + + #: + ip_header_len: ClassVar[int] = 20 + #: + ether_header_len: ClassVar[int] = 14 + + #: Length of the packet being sent including the IP and frame headers. + total_packet_len: ClassVar[int] = 100 + #: Packet to send during testing. + send_pkt: ClassVar[Packet] = ( + Ether() / IP() / Raw("X" * (total_packet_len - ip_header_len - ether_header_len)) + ) + + def extract_noise_information( + self, verbose_out: list[TestPmdVerboseOutput] + ) -> Tuple[int, int, int, int]: + """Extract information about packets that were not sent by the framework in `verbose_out`. + + Extract the number of sent/received packets that did not originate from this test suite as + well as the sum of the lengths of said "noise" packets. Note that received packets are only + examined on the port with the ID `self.recv_port` since these are the receive stats that + will be analyzed in this suite. Sent packets are also only examined on the port with the ID + `self.send_port`. + + Packets are considered to be "noise" when they don't match the expected structure of the + packets that are being sent by this test suite. Specifically, the source and destination + mac addresses as well as the software packet type are checked on packets received by + testpmd to ensure they match the proper addresses of the TG and SUT nodes. Packets that are + sent by testpmd however only check the source mac address and the software packet type. + This is because MAC forwarding mode adjusts both addresses, but only the source will belong + to the TG or SUT node. + + Args: + verbose_out: Parsed testpmd verbose output to collect the noise information from. + + Returns: + A tuple containing the total size of received noise in bytes, the number of received + noise packets, size of all noise packets sent by testpmd in bytes, and the number of + noise packets sent by testpmd. + """ + recv_noise_bytes = 0 + recv_noise_packets = 0 + sent_noise_bytes = 0 + num_sent_packets = 0 + for verbose_block in verbose_out: + for p in verbose_block.packets: + if verbose_block.was_received and verbose_block.port_id == self.recv_port: + if ( + p.src_mac.lower() != self._tg_port_egress.mac_address.lower() + or p.dst_mac.lower() != self._sut_port_ingress.mac_address.lower() + or "L2_ETHER L3_IPV4" != p.sw_ptype.strip() + ): + recv_noise_bytes += p.length + recv_noise_packets += 1 + elif not verbose_block.was_received and verbose_block.port_id == self.send_port: + if ( + p.src_mac.lower() != self._sut_port_egress.mac_address.lower() + or "L2_ETHER L3_IPV4" != p.sw_ptype.strip() + ): + sent_noise_bytes += p.length + num_sent_packets += 1 + + return recv_noise_bytes, recv_noise_packets, sent_noise_bytes, num_sent_packets + + def test_stats_updates(self) -> None: + """Send a packet with a fixed length and verify port stats updated properly. + + Send a packet with a total length of `self.total_packet_len` and verify that the rx port + only received one packet and the number of rx_bytes increased by exactly + `self.total_packet_len`. Also verify that the tx port only sent one packet and that the + tx_bytes of the port increase by exactly `self.total_packet_len`. + + Noise on the wire is ignored by extracting the total number of noise packets and the sum of + the lengths of those packets and subtracting them from the totals that are provided by the + testpmd command `show port info all`. + + Test: + Start testpmd in MAC forwarding mode and set verbose mode to 3 (RX and TX). + Start packet forwarding and then clear all port statistics. + Send a packet, then stop packet forwarding and collect the port stats. + Parse verbose info from stopping packet forwarding and verify values in port stats. + """ + with TestPmdShell(self.sut_node, forward_mode=SimpleForwardingModes.mac) as testpmd: + testpmd.set_verbose(3) + testpmd.start() + testpmd.clear_port_stats_all() + self.send_packet_and_capture(self.send_pkt) + forwarding_output = testpmd.stop() + port_stats_all = testpmd.show_port_stats_all() + verbose_information = TestPmdShell.extract_verbose_output(forwarding_output) + + # Gather information from irrelevant packets sent/ received on the same port. + rx_irr_bytes, rx_irr_pakts, tx_irr_bytes, tx_irr_pakts = self.extract_noise_information( + verbose_information + ) + recv_relevant_packets = port_stats_all[self.recv_port].rx_packets - rx_irr_pakts + sent_relevant_packets = port_stats_all[self.send_port].tx_packets - tx_irr_pakts + + self.verify( + recv_relevant_packets == 1, + f"Port {self.recv_port} received {recv_relevant_packets} packets but expected to only " + "receive 1.", + ) + self.verify( + port_stats_all[self.recv_port].rx_bytes - rx_irr_bytes == self.total_packet_len, + f"Number of bytes received by port {self.recv_port} did not match the amount sent.", + ) + self.verify( + sent_relevant_packets == 1, + f"Number was packets sent by port {self.send_port} was not equal to the number " + f"received by port {self.recv_port}.", + ) + self.verify( + port_stats_all[self.send_port].tx_bytes - tx_irr_bytes == self.total_packet_len, + f"Number of bytes sent by port {self.send_port} did not match the number of bytes " + f"received by port {self.recv_port}.", + )