From patchwork Thu Mar 16 15:14:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Harivel X-Patchwork-Id: 125191 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5B0C841EB1; Thu, 16 Mar 2023 16:14:51 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3B6EF40DF6; Thu, 16 Mar 2023 16:14:51 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id B447740DDC for ; Thu, 16 Mar 2023 16:14:48 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678979688; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=0PFxjRXsiahlfQ3+MOJVOUehVPem+9WTBTv9VYtWtC8=; b=I2n35irgPjkzNpAdLTZoqGZhgD6GlCV6d/rgjIKAmMIVPIgw5adKVdcIGI+O8QlIC2hClM i6AbIwh8gGrXGoSbuailN+cQNCUmL+75Eah/mDNviSUMAwBANMxqkeOdR7sM3xq2hk2Jkb 5f7ElR/dq0d81JX2b9r1Tub2AwIZ9TE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-456-m6K4grmcO2q2QR0fMtWSlg-1; Thu, 16 Mar 2023 11:14:45 -0400 X-MC-Unique: m6K4grmcO2q2QR0fMtWSlg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D5A6A87A9EA; Thu, 16 Mar 2023 15:14:44 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.39.208.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id A56772166B2A; Thu, 16 Mar 2023 15:14:42 +0000 (UTC) From: Anthony Harivel To: Thomas Monjalon , Aman Singh , Yuying Zhang Cc: dev@dpdk.org, rjarry@redhat.com, david.marchand@redhat.com, ktraynor@redhat.com, Anthony Harivel Subject: [PATCH 1/2] app/testpmd: fwd engines report processed packets Date: Thu, 16 Mar 2023 16:14:37 +0100 Message-Id: <20230316151438.186241-1-aharivel@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Change packet_fwd_t to return the number of processed packets instead of a boolean. This will be used in the next commit to determine the load ratio of an lcore and adjust the sleep duration accordingly Signed-off-by: Anthony Harivel --- .mailmap | 1 + app/test-pmd/5tswap.c | 6 +++--- app/test-pmd/csumonly.c | 6 +++--- app/test-pmd/flowgen.c | 4 ++-- app/test-pmd/icmpecho.c | 6 +++--- app/test-pmd/ieee1588fwd.c | 18 +++++++++--------- app/test-pmd/iofwd.c | 6 +++--- app/test-pmd/macfwd.c | 6 +++--- app/test-pmd/macswap.c | 6 +++--- app/test-pmd/noisy_vnf.c | 4 ++-- app/test-pmd/rxonly.c | 6 +++--- app/test-pmd/shared_rxq_fwd.c | 6 +++--- app/test-pmd/testpmd.c | 6 +++--- app/test-pmd/testpmd.h | 2 +- app/test-pmd/txonly.c | 6 +++--- 15 files changed, 45 insertions(+), 44 deletions(-) diff --git a/.mailmap b/.mailmap index 156b2278ce84..97b5bdfe1cfd 100644 --- a/.mailmap +++ b/.mailmap @@ -111,6 +111,7 @@ Anna Lukin Anoob Joseph Antara Ganesh Kolar Anthony Fee +Anthony Harivel Antonio Fischetti Anupam Kapoor Apeksha Gupta diff --git a/app/test-pmd/5tswap.c b/app/test-pmd/5tswap.c index ff8c2dcde59e..f55d4547e3d6 100644 --- a/app/test-pmd/5tswap.c +++ b/app/test-pmd/5tswap.c @@ -81,7 +81,7 @@ swap_udp(struct rte_udp_hdr *udp_hdr) * 2,3,4. Swaps source and destination for MAC, IPv4/IPv6, UDP/TCP. * Parses each layer and swaps it. When the next layer doesn't match it stops. */ -static bool +static size_t pkt_burst_5tuple_swap(struct fwd_stream *fs) { struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; @@ -107,7 +107,7 @@ pkt_burst_5tuple_swap(struct fwd_stream *fs) */ nb_rx = common_fwd_stream_receive(fs, pkts_burst, nb_pkt_per_burst); if (unlikely(nb_rx == 0)) - return false; + return 0; txp = &ports[fs->tx_port]; ol_flags = ol_flags_init(txp->dev_conf.txmode.offloads); @@ -154,7 +154,7 @@ pkt_burst_5tuple_swap(struct fwd_stream *fs) } common_fwd_stream_transmit(fs, pkts_burst, nb_rx); - return true; + return nb_rx; } struct fwd_engine five_tuple_swap_fwd_engine = { diff --git a/app/test-pmd/csumonly.c b/app/test-pmd/csumonly.c index fc85c22a7757..2632894a29d1 100644 --- a/app/test-pmd/csumonly.c +++ b/app/test-pmd/csumonly.c @@ -828,7 +828,7 @@ pkts_ip_csum_recalc(struct rte_mbuf **pkts_burst, const uint16_t nb_pkts, uint64 * IP, UDP, TCP and SCTP flags always concern the inner layer. The * OUTER_IP is only useful for tunnel packets. */ -static bool +static size_t pkt_burst_checksum_forward(struct fwd_stream *fs) { struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; @@ -860,7 +860,7 @@ pkt_burst_checksum_forward(struct fwd_stream *fs) /* receive a burst of packet */ nb_rx = common_fwd_stream_receive(fs, pkts_burst, nb_pkt_per_burst); if (unlikely(nb_rx == 0)) - return false; + return 0; rx_bad_ip_csum = 0; rx_bad_l4_csum = 0; @@ -1174,7 +1174,7 @@ pkt_burst_checksum_forward(struct fwd_stream *fs) fs->rx_bad_outer_l4_csum += rx_bad_outer_l4_csum; fs->rx_bad_outer_ip_csum += rx_bad_outer_ip_csum; - return true; + return nb_rx; } struct fwd_engine csum_fwd_engine = { diff --git a/app/test-pmd/flowgen.c b/app/test-pmd/flowgen.c index 53b5f24f11e7..a89de81ce615 100644 --- a/app/test-pmd/flowgen.c +++ b/app/test-pmd/flowgen.c @@ -58,7 +58,7 @@ RTE_DEFINE_PER_LCORE(int, _next_flow); * terminate receive traffic. Received traffic is simply discarded, but we * still do so in order to maintain traffic statistics. */ -static bool +static size_t pkt_burst_flow_gen(struct fwd_stream *fs) { unsigned pkt_size = tx_pkt_length - 4; /* Adjust FCS */ @@ -168,7 +168,7 @@ pkt_burst_flow_gen(struct fwd_stream *fs) RTE_PER_LCORE(_next_flow) = next_flow; - return true; + return nb_pkt; } static int diff --git a/app/test-pmd/icmpecho.c b/app/test-pmd/icmpecho.c index 68524484e305..3c08d3cb311e 100644 --- a/app/test-pmd/icmpecho.c +++ b/app/test-pmd/icmpecho.c @@ -269,7 +269,7 @@ ipv4_hdr_cksum(struct rte_ipv4_hdr *ip_h) * Receive a burst of packets, lookup for ICMP echo requests, and, if any, * send back ICMP echo replies. */ -static bool +static size_t reply_to_icmp_echo_rqsts(struct fwd_stream *fs) { struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; @@ -296,7 +296,7 @@ reply_to_icmp_echo_rqsts(struct fwd_stream *fs) */ nb_rx = common_fwd_stream_receive(fs, pkts_burst, nb_pkt_per_burst); if (unlikely(nb_rx == 0)) - return false; + return 0; nb_replies = 0; for (i = 0; i < nb_rx; i++) { @@ -477,7 +477,7 @@ reply_to_icmp_echo_rqsts(struct fwd_stream *fs) if (nb_replies > 0) common_fwd_stream_transmit(fs, pkts_burst, nb_replies); - return true; + return nb_rx; } struct fwd_engine icmp_echo_engine = { diff --git a/app/test-pmd/ieee1588fwd.c b/app/test-pmd/ieee1588fwd.c index 386d9f10e642..9f3f4b665f2f 100644 --- a/app/test-pmd/ieee1588fwd.c +++ b/app/test-pmd/ieee1588fwd.c @@ -89,7 +89,7 @@ port_ieee1588_tx_timestamp_check(portid_t pi) (wait_us == 1) ? "" : "s"); } -static bool +static size_t ieee1588_packet_fwd(struct fwd_stream *fs) { struct rte_mbuf *mb; @@ -103,7 +103,7 @@ ieee1588_packet_fwd(struct fwd_stream *fs) * Receive 1 packet at a time. */ if (common_fwd_stream_receive(fs, &mb, 1) == 0) - return false; + return 0; /* * Check that the received packet is a PTP packet that was detected @@ -124,14 +124,14 @@ ieee1588_packet_fwd(struct fwd_stream *fs) (unsigned) mb->pkt_len); } rte_pktmbuf_free(mb); - return false; + return 0; } if (eth_type != RTE_ETHER_TYPE_1588) { printf("Port %u Received NON PTP packet incorrectly" " detected by hardware\n", fs->rx_port); rte_pktmbuf_free(mb); - return false; + return 0; } /* @@ -145,14 +145,14 @@ ieee1588_packet_fwd(struct fwd_stream *fs) " protocol version 0x%x (should be 0x02)\n", fs->rx_port, ptp_hdr->version); rte_pktmbuf_free(mb); - return false; + return 0; } if (ptp_hdr->msg_id != PTP_SYNC_MESSAGE) { printf("Port %u Received PTP V2 Ethernet frame with unexpected" " message ID 0x%x (expected 0x0 - PTP_SYNC_MESSAGE)\n", fs->rx_port, ptp_hdr->msg_id); rte_pktmbuf_free(mb); - return false; + return 0; } printf("Port %u IEEE1588 PTP V2 SYNC Message filtered by hardware\n", fs->rx_port); @@ -166,7 +166,7 @@ ieee1588_packet_fwd(struct fwd_stream *fs) " by hardware\n", fs->rx_port); rte_pktmbuf_free(mb); - return false; + return 0; } /* For i40e we need the timesync register index. It is ignored for the @@ -184,14 +184,14 @@ ieee1588_packet_fwd(struct fwd_stream *fs) mb->ol_flags |= RTE_MBUF_F_TX_IEEE1588_TMST; if (common_fwd_stream_transmit(fs, &mb, 1) == 0) { printf("Port %u sent PTP packet dropped\n", fs->tx_port); - return false; + return 0; } /* * Check the TX timestamp. */ port_ieee1588_tx_timestamp_check(fs->tx_port); - return true; + return 1; } static int diff --git a/app/test-pmd/iofwd.c b/app/test-pmd/iofwd.c index ba06fae4a6b0..77814326291d 100644 --- a/app/test-pmd/iofwd.c +++ b/app/test-pmd/iofwd.c @@ -41,7 +41,7 @@ * This is the fastest possible forwarding operation, as it does not access * to packets data. */ -static bool +static size_t pkt_burst_io_forward(struct fwd_stream *fs) { struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; @@ -52,11 +52,11 @@ pkt_burst_io_forward(struct fwd_stream *fs) */ nb_rx = common_fwd_stream_receive(fs, pkts_burst, nb_pkt_per_burst); if (unlikely(nb_rx == 0)) - return false; + return 0; common_fwd_stream_transmit(fs, pkts_burst, nb_rx); - return true; + return nb_rx; } struct fwd_engine io_fwd_engine = { diff --git a/app/test-pmd/macfwd.c b/app/test-pmd/macfwd.c index 7316d73315b1..c1c601c3cb83 100644 --- a/app/test-pmd/macfwd.c +++ b/app/test-pmd/macfwd.c @@ -41,7 +41,7 @@ * Change the source and the destination Ethernet addressed of packets * before forwarding them. */ -static bool +static size_t pkt_burst_mac_forward(struct fwd_stream *fs) { struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; @@ -58,7 +58,7 @@ pkt_burst_mac_forward(struct fwd_stream *fs) */ nb_rx = common_fwd_stream_receive(fs, pkts_burst, nb_pkt_per_burst); if (unlikely(nb_rx == 0)) - return false; + return 0; txp = &ports[fs->tx_port]; tx_offloads = txp->dev_conf.txmode.offloads; @@ -88,7 +88,7 @@ pkt_burst_mac_forward(struct fwd_stream *fs) common_fwd_stream_transmit(fs, pkts_burst, nb_rx); - return true; + return nb_rx; } struct fwd_engine mac_fwd_engine = { diff --git a/app/test-pmd/macswap.c b/app/test-pmd/macswap.c index 57f77003fe44..cad39b1a7369 100644 --- a/app/test-pmd/macswap.c +++ b/app/test-pmd/macswap.c @@ -47,7 +47,7 @@ * MAC swap forwarding mode: Swap the source and the destination Ethernet * addresses of packets before forwarding them. */ -static bool +static size_t pkt_burst_mac_swap(struct fwd_stream *fs) { struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; @@ -58,12 +58,12 @@ pkt_burst_mac_swap(struct fwd_stream *fs) */ nb_rx = common_fwd_stream_receive(fs, pkts_burst, nb_pkt_per_burst); if (unlikely(nb_rx == 0)) - return false; + return 0; do_macswap(pkts_burst, nb_rx, &ports[fs->tx_port]); common_fwd_stream_transmit(fs, pkts_burst, nb_rx); - return true; + return nb_rx; } struct fwd_engine mac_swap_engine = { diff --git a/app/test-pmd/noisy_vnf.c b/app/test-pmd/noisy_vnf.c index 2bf90a983c7d..13660fbad338 100644 --- a/app/test-pmd/noisy_vnf.c +++ b/app/test-pmd/noisy_vnf.c @@ -110,7 +110,7 @@ sim_memory_lookups(struct noisy_config *ncf, uint16_t nb_pkts) * out of the FIFO * 4. Cases 2 and 3 combined */ -static bool +static size_t pkt_burst_noisy_vnf(struct fwd_stream *fs) { const uint64_t freq_khz = rte_get_timer_hz() / 1000; @@ -169,7 +169,7 @@ pkt_burst_noisy_vnf(struct fwd_stream *fs) ncf->prev_time = rte_get_timer_cycles(); } end: - return nb_rx > 0 || nb_tx > 0; + return (nb_rx > nb_tx) ? nb_rx : nb_tx; } #define NOISY_STRSIZE 256 diff --git a/app/test-pmd/rxonly.c b/app/test-pmd/rxonly.c index 315f9286cdbe..0583bbac87bb 100644 --- a/app/test-pmd/rxonly.c +++ b/app/test-pmd/rxonly.c @@ -41,7 +41,7 @@ /* * Received a burst of packets. */ -static bool +static size_t pkt_burst_receive(struct fwd_stream *fs) { struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; @@ -52,11 +52,11 @@ pkt_burst_receive(struct fwd_stream *fs) */ nb_rx = common_fwd_stream_receive(fs, pkts_burst, nb_pkt_per_burst); if (unlikely(nb_rx == 0)) - return false; + return 0; rte_pktmbuf_free_bulk(pkts_burst, nb_rx); - return true; + return nb_rx; } static void diff --git a/app/test-pmd/shared_rxq_fwd.c b/app/test-pmd/shared_rxq_fwd.c index 623d62da88cc..1ff846db48ca 100644 --- a/app/test-pmd/shared_rxq_fwd.c +++ b/app/test-pmd/shared_rxq_fwd.c @@ -89,7 +89,7 @@ forward_shared_rxq(struct fwd_stream *fs, uint16_t nb_rx, &pkts_burst[nb_rx - nb_sub_burst]); } -static bool +static size_t shared_rxq_fwd(struct fwd_stream *fs) { struct rte_mbuf *pkts_burst[nb_pkt_per_burst]; @@ -97,10 +97,10 @@ shared_rxq_fwd(struct fwd_stream *fs) nb_rx = common_fwd_stream_receive(fs, pkts_burst, nb_pkt_per_burst); if (unlikely(nb_rx == 0)) - return false; + return 0; forward_shared_rxq(fs, nb_rx, pkts_burst); - return true; + return nb_rx; } static void diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 003269660801..e25c082387f5 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -2274,14 +2274,14 @@ run_pkt_fwd_on_lcore(struct fwd_lcore *fc, packet_fwd_t pkt_fwd) for (sm_id = 0; sm_id < nb_fs; sm_id++) { struct fwd_stream *fs = fsm[sm_id]; uint64_t start_fs_tsc = 0; - bool busy; + size_t pkts; if (fs->disabled) continue; if (record_core_cycles) start_fs_tsc = rte_rdtsc(); - busy = (*pkt_fwd)(fs); - if (record_core_cycles && busy) + pkts = (*pkt_fwd)(fs); + if (record_core_cycles && pkts > 0) fs->busy_cycles += rte_rdtsc() - start_fs_tsc; } #ifdef RTE_LIB_BITRATESTATS diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index b9215720b615..b399a0dc7f84 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -382,7 +382,7 @@ struct fwd_lcore { typedef int (*port_fwd_begin_t)(portid_t pi); typedef void (*port_fwd_end_t)(portid_t pi); typedef void (*stream_init_t)(struct fwd_stream *fs); -typedef bool (*packet_fwd_t)(struct fwd_stream *fs); +typedef size_t (*packet_fwd_t)(struct fwd_stream *fs); struct fwd_engine { const char *fwd_mode_name; /**< Forwarding mode name. */ diff --git a/app/test-pmd/txonly.c b/app/test-pmd/txonly.c index b3d68731047e..ecdf9c54d3ea 100644 --- a/app/test-pmd/txonly.c +++ b/app/test-pmd/txonly.c @@ -323,7 +323,7 @@ pkt_burst_prepare(struct rte_mbuf *pkt, struct rte_mempool *mbp, /* * Transmit a burst of multi-segments packets. */ -static bool +static size_t pkt_burst_transmit(struct fwd_stream *fs) { struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; @@ -388,7 +388,7 @@ pkt_burst_transmit(struct fwd_stream *fs) } if (nb_pkt == 0) - return false; + return 0; nb_tx = common_fwd_stream_transmit(fs, pkts_burst, nb_pkt); @@ -404,7 +404,7 @@ pkt_burst_transmit(struct fwd_stream *fs) (unsigned) (nb_pkt - nb_tx)); } - return true; + return nb_tx; } static int From patchwork Thu Mar 16 15:14:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Harivel X-Patchwork-Id: 125192 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7464C41EB1; Thu, 16 Mar 2023 16:14:56 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3D4E942D53; Thu, 16 Mar 2023 16:14:52 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 42A1E40DDC for ; Thu, 16 Mar 2023 16:14:50 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678979689; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fDXrBGzEeZPc+WseW9qG6CsaF3hsZ3Dzc82Wu44SMNA=; b=EdprIf7VkjRM2ZsTQy+hf8IiQa+m+I06WwQNPebEmRrDWFbd5d9McHkBi8LmqBoIK+PXcl PZK4uXRwUiG1sdgndNPG1CYSj8vielMTFcUJi1pq/RU+hFyYr2m0qAJulqqrKqKUwcjIa8 O/Npw7q0GuZbRPSyprS1D0h7iyyMYXc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-619-QzykoLDMOGmVkHjSQIap8g-1; Thu, 16 Mar 2023 11:14:47 -0400 X-MC-Unique: QzykoLDMOGmVkHjSQIap8g-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 15D0D800D8E; Thu, 16 Mar 2023 15:14:47 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.39.208.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id 43DB72166B26; Thu, 16 Mar 2023 15:14:45 +0000 (UTC) From: Anthony Harivel To: Aman Singh , Yuying Zhang Cc: dev@dpdk.org, rjarry@redhat.com, david.marchand@redhat.com, ktraynor@redhat.com, Anthony Harivel Subject: [PATCH 2/2] app/testpmd: add testpmd based sleeping Date: Thu, 16 Mar 2023 16:14:38 +0100 Message-Id: <20230316151438.186241-2-aharivel@redhat.com> In-Reply-To: <20230316151438.186241-1-aharivel@redhat.com> References: <20230316151438.186241-1-aharivel@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sleep for an incremental amount of time if the fwd engine has processed less than at least half a burst of packets (i.e 16pkts with default setting) on a polling iteration of testpmd. Upon detecting the threshold of >= 16 pkts on an Rxq, reset the sleep time to zero (i.e. no sleep). Sleep time will be increased on each iteration where the low load conditions remain up to a total of the max sleep time which is set by the user with the "--max-sleep-us NUM" command line argument or when in interactive "mode set max_sleep NUM". The default max_sleep value is 0, which means that no sleeps will occur and the default behaviour is unchanged from previously. Testing has been performed on AMD EPYC 7702 server with --nb-cores 12. The results were obtained via turbostat for each individual lcore: max_sleep 0 ====== ======== ======== ======== idle 4Mpps 16Mpps Bursts =============== ====== ======== ======== ======== C1-state % 0 0 0 0 C2-state % 0 0 0 0 % usage 100 100 100 100 Watt / core 1.14 1.18 1.19 1.14 =============== ====== ======== ======== ======== max_sleep 500 ====== ======== ======== ======== idle 4Mpps 16Mpps Bursts =============== ====== ======== ======== ======== C1-state % 99 85 74 98.6 C2-state % 0 0 0 0 % usage 1 15 26 1 Watt / core 0.04 0.18 0.28 0 04 =============== ====== ======== ======== ======== max_sleep 1000 ====== ======== ======== ======== idle 4Mpps 16Mpps Bursts =============== ====== ======== ======== ======== C1-state % 0 85 74 0.3 C2-state % 99 0 0 97.6 % usage 1 15 25 1 Watt / core 0.02 0.18 0.28 0 02 =============== ====== ======== ======== ======== On most cases, the consumption of the cores is greatly improved while still performing zero packet loss. Latency test has been performed on each tests above. The CPU has a C1 latency of 1us and a C2 latency of 400us. On the worst case scenario, Tx Burst of thousands packets every seconds, the following latency in us (micro seconds) has been observed: =========== ==== ===== ====== max_sleep 0 500 1000 ----------- ---- ----- ------ max latency 14 560 1260 min latency 5 5 6 Avg latency 7 305 617 =========== ==== ===== ====== link: https://www.github.com/torvalds/linux/tree/master/tools/power/x86/turbostat Signed-off-by: Anthony Harivel Reviewed-by: Robin Jarry --- app/test-pmd/cmdline.c | 78 +++++++++++++++++++++ app/test-pmd/parameters.c | 14 ++++ app/test-pmd/testpmd.c | 24 ++++++- app/test-pmd/testpmd.h | 2 + doc/guides/testpmd_app_ug/run_app.rst | 19 +++++ doc/guides/testpmd_app_ug/testpmd_funcs.rst | 13 ++++ 6 files changed, 149 insertions(+), 1 deletion(-) diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index 02c72d06b768..b89aae1c3414 100644 --- a/app/test-pmd/cmdline.c +++ b/app/test-pmd/cmdline.c @@ -5522,6 +5522,82 @@ static cmdline_parse_inst_t cmd_set_link_check = { }, }; +/* *** SET MAX SLEEP *** */ +struct cmd_max_sleep_result { + cmdline_fixed_string_t set; + cmdline_fixed_string_t max_sleep; + uint64_t value; +}; + +static void +cmd_max_sleep_set_parsed(void *parsed_result, + __rte_unused struct cmdline *cl, + __rte_unused void *data) +{ + struct cmd_max_sleep_result *res = parsed_result; + if ((!strcmp(res->set, "set")) + && (!strcmp(res->max_sleep, "max_sleep"))) + max_sleep_us = res->value; +} + +static cmdline_parse_token_string_t cmd_max_sleep_set = + TOKEN_STRING_INITIALIZER(struct cmd_max_sleep_result, + set, "set"); +static cmdline_parse_token_string_t cmd_max_sleep = + TOKEN_STRING_INITIALIZER(struct cmd_max_sleep_result, + max_sleep, "max_sleep"); +static cmdline_parse_token_num_t cmd_max_sleep_value = + TOKEN_NUM_INITIALIZER(struct cmd_max_sleep_result, + value, RTE_UINT64); + +static cmdline_parse_inst_t cmd_set_max_sleep = { + .f = cmd_max_sleep_set_parsed, + .data = NULL, + .help_str = "set max_sleep : Set the maximum sleep in micro seconds.", + .tokens = { + (void *)&cmd_max_sleep_set, + (void *)&cmd_max_sleep, + (void *)&cmd_max_sleep_value, + NULL, + }, +}; + +/* *** SHOW MAX SLEEP *** */ +struct cmd_max_sleep_show_result { + cmdline_fixed_string_t show; + cmdline_fixed_string_t max_sleep; + uint64_t value; +}; + +static void +cmd_max_sleep_show_parsed(void *parsed_result, + __rte_unused struct cmdline *cl, + __rte_unused void *data) +{ + struct cmd_max_sleep_show_result *res = parsed_result; + if ((!strcmp(res->show, "show")) + && (!strcmp(res->max_sleep, "max_sleep"))) + printf("max_sleep %"PRIu64" us\n", max_sleep_us); +} + +static cmdline_parse_token_string_t cmd_show_max_sleep_show = + TOKEN_STRING_INITIALIZER(struct cmd_max_sleep_show_result, + show, "show"); +static cmdline_parse_token_string_t cmd_max_sleep_show = + TOKEN_STRING_INITIALIZER(struct cmd_max_sleep_show_result, + max_sleep, "max_sleep"); + +static cmdline_parse_inst_t cmd_show_max_sleep = { + .f = cmd_max_sleep_show_parsed, + .data = NULL, + .help_str = "show max_sleep: Display the maximun sleep in micro seconds.", + .tokens = { + (void *)&cmd_show_max_sleep_show, + (void *)&cmd_max_sleep_show, + NULL, + }, +}; + /* *** SET FORWARDING MODE *** */ struct cmd_set_fwd_mode_result { cmdline_fixed_string_t set; @@ -12770,6 +12846,8 @@ static cmdline_parse_ctx_t builtin_ctx[] = { (cmdline_parse_inst_t *)&cmd_set_allmulti_mode_all, (cmdline_parse_inst_t *)&cmd_set_flush_rx, (cmdline_parse_inst_t *)&cmd_set_link_check, + (cmdline_parse_inst_t *)&cmd_set_max_sleep, + (cmdline_parse_inst_t *)&cmd_show_max_sleep, (cmdline_parse_inst_t *)&cmd_vlan_offload, (cmdline_parse_inst_t *)&cmd_vlan_tpid, (cmdline_parse_inst_t *)&cmd_rx_vlan_filter_all, diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c index 3b37809baf00..6cb4ce9edb0a 100644 --- a/app/test-pmd/parameters.c +++ b/app/test-pmd/parameters.c @@ -201,6 +201,9 @@ usage(char* progname) printf(" --rx-mq-mode=0xX: hexadecimal bitmask of RX mq mode can be " "enabled\n"); printf(" --record-core-cycles: enable measurement of CPU cycles.\n"); + printf(" --max-sleep-us: maximum sleep time that will be requested in\n" + " microseconds per iteration of packet processing which has received zero\n" + " or a small amount of packets from the Rx queues it is polling.\n"); printf(" --record-burst-stats: enable display of RX and TX bursts.\n"); printf(" --hairpin-mode=0xXX: bitmask set the hairpin port mode.\n" " 0x10 - explicit Tx rule, 0x02 - hairpin ports paired\n" @@ -707,6 +710,7 @@ launch_args_parse(int argc, char** argv) { "no-iova-contig", 0, 0, 0 }, { "rx-mq-mode", 1, 0, 0 }, { "record-core-cycles", 0, 0, 0 }, + { "max-sleep-us", 1, 0, 0 }, { "record-burst-stats", 0, 0, 0 }, { PARAM_NUM_PROCS, 1, 0, 0 }, { PARAM_PROC_ID, 1, 0, 0 }, @@ -1459,6 +1463,16 @@ launch_args_parse(int argc, char** argv) } if (!strcmp(lgopts[opt_idx].name, "record-core-cycles")) record_core_cycles = 1; + if (!strcmp(lgopts[opt_idx].name, "max-sleep-us")) { + char *end = NULL; + errno = 0; + unsigned long sleep = strtoul(optarg, &end, 10); + + if (errno != 0 || *optarg == '\0' || *end != '\0' || sleep == 0) + rte_exit(EXIT_FAILURE, "max-sleep-us must be > 0\n"); + + max_sleep_us = sleep; + } if (!strcmp(lgopts[opt_idx].name, "record-burst-stats")) record_burst_stats = 1; if (!strcmp(lgopts[opt_idx].name, PARAM_NUM_PROCS)) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index e25c082387f5..ccd9be175c59 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -524,6 +524,11 @@ uint8_t gro_flush_cycles = GRO_DEFAULT_FLUSH_CYCLES; */ enum rte_eth_rx_mq_mode rx_mq_mode = RTE_ETH_MQ_RX_VMDQ_DCB_RSS; +/* + * Max sleep time requested in microseconds per iteration + */ +uint64_t max_sleep_us; + /* * Used to set forced link speed */ @@ -2254,6 +2259,8 @@ static void run_pkt_fwd_on_lcore(struct fwd_lcore *fc, packet_fwd_t pkt_fwd) { struct fwd_stream **fsm; + uint64_t sleep_us = 0; + uint64_t sleep_cycles; uint64_t prev_tsc; streamid_t nb_fs; streamid_t sm_id; @@ -2283,6 +2290,8 @@ run_pkt_fwd_on_lcore(struct fwd_lcore *fc, packet_fwd_t pkt_fwd) pkts = (*pkt_fwd)(fs); if (record_core_cycles && pkts > 0) fs->busy_cycles += rte_rdtsc() - start_fs_tsc; + if (pkts > nb_pkt_per_burst / 2) + sleep_us = 0; } #ifdef RTE_LIB_BITRATESTATS if (bitrate_enabled != 0 && @@ -2302,10 +2311,23 @@ run_pkt_fwd_on_lcore(struct fwd_lcore *fc, packet_fwd_t pkt_fwd) latencystats_lcore_id == rte_lcore_id()) rte_latencystats_update(); #endif + sleep_cycles = 0; + if (max_sleep_us) { + /* Check if a sleep should happen on this iteration. */ + if (sleep_us > 0) { + uint64_t tsc = rte_rdtsc(); + + rte_delay_us_sleep(sleep_us); + sleep_cycles = rte_rdtsc() - tsc; + } + if (sleep_us < max_sleep_us) + /* Increase sleep time for next iteration. */ + sleep_us += 1; + } if (record_core_cycles) { uint64_t tsc = rte_rdtsc(); - fc->total_cycles += tsc - prev_tsc; + fc->total_cycles += tsc - prev_tsc - sleep_cycles; prev_tsc = tsc; } } while (! fc->stopped); diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index b399a0dc7f84..63758831d6c9 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -779,6 +779,8 @@ extern struct mplsoudp_decap_conf mplsoudp_decap_conf; extern enum rte_eth_rx_mq_mode rx_mq_mode; +extern uint64_t max_sleep_us; + extern struct rte_flow_action_conntrack conntrack_context; extern int proc_id; diff --git a/doc/guides/testpmd_app_ug/run_app.rst b/doc/guides/testpmd_app_ug/run_app.rst index 57b23241cf36..c5afe44e680f 100644 --- a/doc/guides/testpmd_app_ug/run_app.rst +++ b/doc/guides/testpmd_app_ug/run_app.rst @@ -555,6 +555,25 @@ The command line options are: The default value is 0. Hairpin will use single port mode and implicit Tx flow mode. +* ``--max-sleep-us=N`` + + Set the maximum sleep in micro seconds. The default value is 0. + + When `max-sleep-us` is set, the lcores running the packet forwarding may stop active polling and + go to sleep for an incrementing amount of time. Each time the forwarding engine processes less + than half a burst of packets, the sleep time will be incremented by 1 micro second, up to the + maximum value set by the user. + + At any point, if the forwarding engine returns more than half a burst of packets, the sleep time + will be reset to 0. + + Sleeping in the packet processing path yields back control to the kernel scheduler. The actual + sleep/wakeup times are not guaranteed and may differ significantly depending on system + configuration, allowed C-states and scheduler timer resolution (on Linux, this is controlled by + ``prctl(PR_SET_TIMERSLACK, nanoseconds)`` and it defaults to 10 micro seconds). + + In interactive mode, the maximum sleep time can be set with ``set max_sleep N`` and displayed + with ``show max_sleep``. Testpmd Multi-Process Command-line Options ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst index 8f2384785930..53a67f02b702 100644 --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst @@ -637,6 +637,12 @@ Show fec mode of a port:: testpmd> show port (port_id) fec_mode +show max_sleep +~~~~~~~~~~~~~~ + +Show max_sleep in micro seconds:: + + testpmd> show max_sleep Configuration Functions ----------------------- @@ -1802,6 +1808,13 @@ Set fec mode for a specific port:: testpmd> set port (port_id) fec_mode auto|off|rs|baser +Set max_sleep +~~~~~~~~~~~~ + +Set max_sleep in micro seconds:: + + testpmd> set max_sleep + Config Sample actions list ~~~~~~~~~~~~~~~~~~~~~~~~~~