From patchwork Fri Sep 24 11:44:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tudor Cornea X-Patchwork-Id: 99590 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C7EA1A0548; Fri, 24 Sep 2021 13:44:59 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 876C24126F; Fri, 24 Sep 2021 13:44:59 +0200 (CEST) Received: from mail-wr1-f53.google.com (mail-wr1-f53.google.com [209.85.221.53]) by mails.dpdk.org (Postfix) with ESMTP id 799104122D for ; Fri, 24 Sep 2021 13:44:58 +0200 (CEST) Received: by mail-wr1-f53.google.com with SMTP id d21so26472175wra.12 for ; Fri, 24 Sep 2021 04:44:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=hvqY6cDpvegSWbWhwOWhGA8EvKoptpaS+C6CyPo8PWU=; b=lGLHLTblIAseNEVpS1ElmNzYe2i+iXLJwkuBAlFEYZ6XPHFf6YjsB9K9onebgt995r IeYuwfsHPD9uiKMrZOSrQxzXd3XFsacE91rfQzLMzz8h3R4Piebl05wzIG8hL/osOhxo MJjRxgW7WjM6TJk0a6v9IlQIcBLxdtwEunLFV0UuBfl499V5PGpEq9ScSgG+uZAnRSjZ HZVntMjcCKTWHDvMMMjY7wSD2lgqs/BtDFRj7gpgHVPp1hi/aRFb+5J3FLEA2XY7znwz dc082eIly0WF2HPRvMbSyo6ua96Vg1yqSUnacXr80kotON7f3rjd30uP7MwUQHfv9Q6M zv7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=hvqY6cDpvegSWbWhwOWhGA8EvKoptpaS+C6CyPo8PWU=; b=2i2T8JTS+ZPs9WyIe7h8WZVMvN5Qxs5GUpsx1q779Ka+f4j/7FwOSf3VYOTCa9aE+V e9VQooryZ+MjRJ8jSnQ5yTJ3cCpLymnyZI4Z/Sy9k/DwuJUQzWdnbX5LgCGTbk0Hy8/6 Sw4lZ1CxhUj+FradnIfbvyaGdUUn7UcTg4seXuuO2WhPxe9AcWEl+UqkqAqLjVY6HOLh AHXHc6rxmp7m9DPsMqSd2oV3Opp8MYaK1SEORrI9ImGvuxfH+eb59uxZLx/R49fTwFhy cvLu7cocn5hiAp1tB378c4YSLpq+MU1sN8Ongl4awRbBxfHUhVJft2/0oCzFyRzn4BVy nJvw== X-Gm-Message-State: AOAM530kKuMJDPfBLWkFIiPU3ZmPJ8FUSiN1MyvNU6bJMQ83o6ueil1C +lDqvq+tL4UpnIPL+Nav22Q= X-Google-Smtp-Source: ABdhPJwUVw20N3aPRZiDX8leEWuIODt450ojYW7TFkfPr0mZYHAZ84GRpRBdyJ1EF4L50p+UgM048w== X-Received: by 2002:a1c:ac03:: with SMTP id v3mr1637643wme.127.1632483898130; Fri, 24 Sep 2021 04:44:58 -0700 (PDT) Received: from tucornea-dev-machine.localdomain ([193.226.172.43]) by smtp.gmail.com with ESMTPSA id b6sm7834895wrh.4.2021.09.24.04.44.57 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 24 Sep 2021 04:44:57 -0700 (PDT) From: Tudor Cornea To: ferruh.yigit@intel.com Cc: linville@tuxdriver.com, stephen@networkplumber.org, andrew.rybchenko@oktetlabs.ru, thomas@monjalon.net, jerinj@marvell.com, dev@dpdk.org, Tudor Cornea Date: Fri, 24 Sep 2021 14:44:45 +0300 Message-Id: <1632483885-84732-1-git-send-email-tudor.cornea@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1631091558-63337-1-git-send-email-tudor.cornea@gmail.com> References: <1631091558-63337-1-git-send-email-tudor.cornea@gmail.com> Subject: [dpdk-dev] [PATCH v3] net/af_packet: reinsert the stripped vlan tag X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The af_packet pmd driver binds to a raw socket and allows sending and receiving of packets through the kernel. Since commit [1], the kernel strips the vlan tags early in __netif_receive_skb_core(), so we receive untagged packets while running with the af_packet pmd. Luckily for us, the skb vlan-related fields are still populated from the stripped vlan tags, so we end up having all the information that we need in the mbuf. Having the pmd driver support DEV_RX_OFFLOAD_VLAN_STRIP allows the application to control the desired vlan stripping behavior, until we have a way to describe offloads that can't be disabled by pmd drivers. This patch will cause a change in the default way that the af_packet pmd treats received vlan-tagged frames. While previously, the application was required to check the PKT_RX_VLAN_STRIPPED flag, after this patch, the pmd will re-insert the vlan tag transparently to the user, unless the DEV_RX_OFFLOAD_VLAN_STRIP is enabled in rxmode.offloads. I've attempted a preliminary benchmark to understand if the change could cause a sizable performance hit. Setup: Two virtual machines running on top of an ESXi hypervisor Tx: DPDK app (running on top of vmxnet3 PMD) Rx: af_packet (running on top of a kernel vmxnet3 interface) Packet size :68 (packet contains a vlan tag) Rates: Tx - 1.419 Mpps Rx (without vlan insertion) - 1227636 pps Rx (with vlan insertion) - 1220081 pps At a first glance, we don't seem to have a large degradation in terms of packet rate [1] https://github.com/torvalds/linux/commit/bcc6d47903612c3861201cc3a866fb604f26b8b2 Signed-off-by: Tudor Cornea --- v3: * Updated release note and documentation * Updated commit with performance measurements v2: * Added DEV_RX_OFFLOAD_VLAN_STRIP to rxmode->offloads --- doc/guides/nics/af_packet.rst | 38 +++++++++++++++++++++++++++++++ doc/guides/rel_notes/release_21_11.rst | 4 ++++ drivers/net/af_packet/rte_eth_af_packet.c | 12 ++++++++++ 3 files changed, 54 insertions(+) diff --git a/doc/guides/nics/af_packet.rst b/doc/guides/nics/af_packet.rst index efd6f1c..97d5502 100644 --- a/doc/guides/nics/af_packet.rst +++ b/doc/guides/nics/af_packet.rst @@ -65,3 +65,41 @@ framecnt=512): .. code-block:: console --vdev=eth_af_packet0,iface=tap0,blocksz=4096,framesz=2048,framecnt=512,qpairs=1,qdisc_bypass=0 + +Features and Limitations of the af_packet PMD +--------------------------------------------- + +Since the following commit, the Linux kernel strips the vlan tag + +.. code-block:: console + + commit bcc6d47903612c3861201cc3a866fb604f26b8b2 + Author: Jiri Pirko + Date: Thu Apr 7 19:48:33 2011 +0000 + + net: vlan: make non-hw-accel rx path similar to hw-accel + +Running on such a kernel results in receiving untagged frames while using +the af_packet PMD. Fortunately, the stripped information is still available +for use in ``mbuf->vlan_tci``, and applications could check ``PKT_RX_VLAN_STRIPPED``. + +However, we currently don't have a way to describe offloads which can't be +disabled by PMDs, and this creates an inconsistency with the way applications +expect the PMD offloads to work, and requires them to be aware of which +underlying driver they use. + +Since release 21.11 the af_packet PMD will implement support for the +``DEV_RX_OFFLOAD_VLAN_STRIP`` offload, and users can control the desired vlan +stripping behavior. + +It's important to note that the default case will change. If previously, +the vlan tag was stripped, if the application now requires the same behavior, +it will need to configure ``rxmode.offloads`` with ``DEV_RX_OFFLOAD_VLAN_STRIP``. + +The PMD driver will re-insert the vlan tag transparently to the application +if the kernel strips it, as long as the ``DEV_RX_OFFLOAD_VLAN_STRIP`` is not +enabled. + +.. code-block:: console + + port_conf.rxmode.offloads |= DEV_RX_OFFLOAD_VLAN_STRIP diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst index ad7c1af..095fd5b 100644 --- a/doc/guides/rel_notes/release_21_11.rst +++ b/doc/guides/rel_notes/release_21_11.rst @@ -66,6 +66,10 @@ New Features * Added rte_flow support for dual VLAN insert and strip actions. +* **Updated af_packet ethdev driver.** + + * Added DEV_RX_OFFLOAD_VLAN_STRIP capability. + * **Updated Marvell cnxk crypto PMD.** * Added AES-CBC SHA1-HMAC support in lookaside protocol (IPsec) for CN10K. diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c index b73b211..5ed9dd6 100644 --- a/drivers/net/af_packet/rte_eth_af_packet.c +++ b/drivers/net/af_packet/rte_eth_af_packet.c @@ -48,6 +48,7 @@ struct pkt_rx_queue { struct rte_mempool *mb_pool; uint16_t in_port; + uint8_t vlan_strip; volatile unsigned long rx_pkts; volatile unsigned long rx_bytes; @@ -78,6 +79,7 @@ struct pmd_internals { struct pkt_rx_queue *rx_queue; struct pkt_tx_queue *tx_queue; + uint8_t vlan_strip; }; static const char *valid_arguments[] = { @@ -148,6 +150,9 @@ eth_af_packet_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) if (ppd->tp_status & TP_STATUS_VLAN_VALID) { mbuf->vlan_tci = ppd->tp_vlan_tci; mbuf->ol_flags |= (PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED); + + if (!pkt_q->vlan_strip && rte_vlan_insert(&mbuf)) + PMD_LOG(ERR, "Failed to reinsert VLAN tag"); } /* release incoming frame and advance ring buffer */ @@ -302,6 +307,11 @@ eth_dev_stop(struct rte_eth_dev *dev) static int eth_dev_configure(struct rte_eth_dev *dev __rte_unused) { + struct rte_eth_conf *dev_conf = &dev->data->dev_conf; + const struct rte_eth_rxmode *rxmode = &dev_conf->rxmode; + struct pmd_internals *internals = dev->data->dev_private; + + internals->vlan_strip = !!(rxmode->offloads & DEV_RX_OFFLOAD_VLAN_STRIP); return 0; } @@ -318,6 +328,7 @@ eth_dev_info(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->min_rx_bufsize = 0; dev_info->tx_offload_capa = DEV_TX_OFFLOAD_MULTI_SEGS | DEV_TX_OFFLOAD_VLAN_INSERT; + dev_info->rx_offload_capa = DEV_RX_OFFLOAD_VLAN_STRIP; return 0; } @@ -448,6 +459,7 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, dev->data->rx_queues[rx_queue_id] = pkt_q; pkt_q->in_port = dev->data->port_id; + pkt_q->vlan_strip = internals->vlan_strip; return 0; }