From patchwork Wed May 15 18:08:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ajit Khaparde X-Patchwork-Id: 53443 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 18C5B5F1C; Wed, 15 May 2019 20:08:28 +0200 (CEST) Received: from rnd-relay.smtp.broadcom.com (rnd-relay.smtp.broadcom.com [192.19.229.170]) by dpdk.org (Postfix) with ESMTP id 75E935B20 for ; Wed, 15 May 2019 20:08:24 +0200 (CEST) Received: from nis-sj1-27.broadcom.com (nis-sj1-27.lvn.broadcom.net [10.75.144.136]) by rnd-relay.smtp.broadcom.com (Postfix) with ESMTP id 187F730C033; Wed, 15 May 2019 11:08:22 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.10.3 rnd-relay.smtp.broadcom.com 187F730C033 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=broadcom.com; s=dkimrelay; t=1557943702; bh=C9YrNrk8nidWJfIi0sHUUsYcEVk5ocmx0QXztDPq2bI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AnXX7ls/r3C7qSY5h5X46QEgRC7vpAmrA1QH3FQOv+mJ1c3pAN1jXV/e5FaigAqyO Ou40yM14kk7Jhs0BSodlz+FMo5rWkYL8Hd/cKcENIIk6RzHNpmuutpJ7/p+3aU3TEF pyXWkE5C4bo7ubFNuDbwi13YHrpNgH7ilSSfzjuI= Received: from C02VPB22HTD6.wifi.broadcom.net (c02vpb22htd6.wifi.broadcom.net [10.69.74.102]) by nis-sj1-27.broadcom.com (Postfix) with ESMTP id E268DAC0709; Wed, 15 May 2019 11:08:22 -0700 (PDT) From: Ajit Khaparde To: dev@dpdk.org Cc: ferruh.yigit@intel.com, Somnath Kotur , Sriharsha Basavapatna Date: Wed, 15 May 2019 11:08:12 -0700 Message-Id: <20190515180817.71523-2-ajit.khaparde@broadcom.com> X-Mailer: git-send-email 2.20.1 (Apple Git-117) In-Reply-To: <20190515180817.71523-1-ajit.khaparde@broadcom.com> References: <20190515180817.71523-1-ajit.khaparde@broadcom.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 1/6] net/bnxt: fix an issue seen with TSO X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" We wrongly update lflags in the Tx descriptor; avoid it. Also, instead of calculating the last producer index to see if mbuf segments are chained, check if the pointer is NULL to iterate through the segment list. Fixes: 6eb3cc2294fd ("net/bnxt: add initial Tx code") Signed-off-by: Ajit Khaparde Reviewed-by: Somnath Kotur Signed-off-by: Sriharsha Basavapatna --- drivers/net/bnxt/bnxt_txr.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/drivers/net/bnxt/bnxt_txr.c b/drivers/net/bnxt/bnxt_txr.c index 39be7bdfa..6f55f3c53 100644 --- a/drivers/net/bnxt/bnxt_txr.c +++ b/drivers/net/bnxt/bnxt_txr.c @@ -123,7 +123,6 @@ static uint16_t bnxt_start_xmit(struct rte_mbuf *tx_pkt, struct tx_bd_long_hi *txbd1 = NULL; uint32_t vlan_tag_flags, cfa_action; bool long_bd = false; - uint16_t last_prod = 0; struct rte_mbuf *m_seg; struct bnxt_sw_tx_bd *tx_buf; static const uint32_t lhint_arr[4] = { @@ -143,8 +142,6 @@ static uint16_t bnxt_start_xmit(struct rte_mbuf *tx_pkt, tx_buf = &txr->tx_buf_ring[txr->tx_prod]; tx_buf->mbuf = tx_pkt; tx_buf->nr_bds = long_bd + tx_pkt->nb_segs; - last_prod = (txr->tx_prod + tx_buf->nr_bds - 1) & - txr->tx_ring_struct->ring_mask; if (unlikely(bnxt_tx_avail(txr) < tx_buf->nr_bds)) return -ENOMEM; @@ -193,11 +190,17 @@ static uint16_t bnxt_start_xmit(struct rte_mbuf *tx_pkt, txbd1->cfa_action = cfa_action; if (tx_pkt->ol_flags & PKT_TX_TCP_SEG) { + uint16_t hdr_size; + /* TSO */ txbd1->lflags |= TX_BD_LONG_LFLAGS_LSO; - txbd1->hdr_size = tx_pkt->l2_len + tx_pkt->l3_len + + hdr_size = tx_pkt->l2_len + tx_pkt->l3_len + tx_pkt->l4_len + tx_pkt->outer_l2_len + tx_pkt->outer_l3_len; + /* The hdr_size is multiple of 16bit units not 8bit. + * Hence divide by 2. + */ + txbd1->hdr_size = hdr_size >> 1; txbd1->mss = tx_pkt->tso_segsz; } else if ((tx_pkt->ol_flags & PKT_TX_OIP_IIP_TCP_UDP_CKSUM) == @@ -282,7 +285,7 @@ static uint16_t bnxt_start_xmit(struct rte_mbuf *tx_pkt, m_seg = tx_pkt->next; /* i is set at the end of the if(long_bd) block */ - while (txr->tx_prod != last_prod) { + while (m_seg) { txr->tx_prod = RING_NEXT(txr->tx_ring_struct, txr->tx_prod); tx_buf = &txr->tx_buf_ring[txr->tx_prod]; @@ -295,8 +298,6 @@ static uint16_t bnxt_start_xmit(struct rte_mbuf *tx_pkt, } txbd->flags_type |= TX_BD_LONG_FLAGS_PACKET_END; - if (txbd1) - txbd1->lflags = rte_cpu_to_le_32(txbd1->lflags); txr->tx_prod = RING_NEXT(txr->tx_ring_struct, txr->tx_prod); From patchwork Wed May 15 18:08:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ajit Khaparde X-Patchwork-Id: 53442 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1F2B75B2C; Wed, 15 May 2019 20:08:26 +0200 (CEST) Received: from rnd-relay.smtp.broadcom.com (rnd-relay.smtp.broadcom.com [192.19.229.170]) by dpdk.org (Postfix) with ESMTP id 97D065B2A for ; Wed, 15 May 2019 20:08:24 +0200 (CEST) Received: from nis-sj1-27.broadcom.com (nis-sj1-27.lvn.broadcom.net [10.75.144.136]) by rnd-relay.smtp.broadcom.com (Postfix) with ESMTP id 499F330C09C; Wed, 15 May 2019 11:08:22 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.10.3 rnd-relay.smtp.broadcom.com 499F330C09C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=broadcom.com; s=dkimrelay; t=1557943702; bh=JcyXOUDnMlouCcYScs6bci57gH3iElAhropllVmqFc8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oY+MPUFh0d6B3Swdy3wXSWWjVC5dzxA7lhHEQE4HUPj16jjYBFuhpnN1yyoQEyX4d 8QpLu6ugTk9pX9ETNkZ1qgHt2FydpJ4yZ8qkMzPlH1hkN64Zwz6vxUlIWPDvzoy5Ws RuaAsp4yNdHziBoUEPBiUnvpbfdh17zuICEbn8sE= Received: from C02VPB22HTD6.wifi.broadcom.net (c02vpb22htd6.wifi.broadcom.net [10.69.74.102]) by nis-sj1-27.broadcom.com (Postfix) with ESMTP id 104DCAC077A; Wed, 15 May 2019 11:08:23 -0700 (PDT) From: Ajit Khaparde To: dev@dpdk.org Cc: ferruh.yigit@intel.com, Somnath Kotur , Sriharsha Basavapatna Date: Wed, 15 May 2019 11:08:13 -0700 Message-Id: <20190515180817.71523-3-ajit.khaparde@broadcom.com> X-Mailer: git-send-email 2.20.1 (Apple Git-117) In-Reply-To: <20190515180817.71523-1-ajit.khaparde@broadcom.com> References: <20190515180817.71523-1-ajit.khaparde@broadcom.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 2/6] net/bnxt: add support to update ipid X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add support to update ipid during TSO. Signed-off-by: Ajit Khaparde Reviewed-by: Somnath Kotur Signed-off-by: Sriharsha Basavapatna --- drivers/net/bnxt/bnxt_txr.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/net/bnxt/bnxt_txr.c b/drivers/net/bnxt/bnxt_txr.c index 6f55f3c53..3a0d73af2 100644 --- a/drivers/net/bnxt/bnxt_txr.c +++ b/drivers/net/bnxt/bnxt_txr.c @@ -193,7 +193,8 @@ static uint16_t bnxt_start_xmit(struct rte_mbuf *tx_pkt, uint16_t hdr_size; /* TSO */ - txbd1->lflags |= TX_BD_LONG_LFLAGS_LSO; + txbd1->lflags |= TX_BD_LONG_LFLAGS_LSO | + TX_BD_LONG_LFLAGS_T_IPID; hdr_size = tx_pkt->l2_len + tx_pkt->l3_len + tx_pkt->l4_len + tx_pkt->outer_l2_len + tx_pkt->outer_l3_len; From patchwork Wed May 15 18:08:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ajit Khaparde X-Patchwork-Id: 53445 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 06D875F2E; Wed, 15 May 2019 20:08:32 +0200 (CEST) Received: from rnd-relay.smtp.broadcom.com (rnd-relay.smtp.broadcom.com [192.19.229.170]) by dpdk.org (Postfix) with ESMTP id AE0CB5B2E for ; Wed, 15 May 2019 20:08:24 +0200 (CEST) Received: from nis-sj1-27.broadcom.com (nis-sj1-27.lvn.broadcom.net [10.75.144.136]) by rnd-relay.smtp.broadcom.com (Postfix) with ESMTP id 64EB330C0AB; Wed, 15 May 2019 11:08:22 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.10.3 rnd-relay.smtp.broadcom.com 64EB330C0AB DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=broadcom.com; s=dkimrelay; t=1557943702; bh=Og7h7DMCpjW+uuGgPsLArnjmlC5jJjlgSDvN4wn1i2Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ea3yKSP+jGI4AW/eYBf6HjSdS4XLFvZ5OWS0ca9eor120gkATxKW9A24pr+9IWcNb uFXNbu8NdyT3kMp6gCUa7jjry4Ceg1Tl7T8SXg8IyANLd15Eh6T/34cxe0V4b5xGWo MicQPdXxkD7UD9A06xZxQOgj+p6/FG96B4ERayXQ= Received: from C02VPB22HTD6.wifi.broadcom.net (c02vpb22htd6.wifi.broadcom.net [10.69.74.102]) by nis-sj1-27.broadcom.com (Postfix) with ESMTP id 41D52AC0798; Wed, 15 May 2019 11:08:23 -0700 (PDT) From: Ajit Khaparde To: dev@dpdk.org Cc: ferruh.yigit@intel.com, Somnath Kotur , Sriharsha Basavapatna Date: Wed, 15 May 2019 11:08:14 -0700 Message-Id: <20190515180817.71523-4-ajit.khaparde@broadcom.com> X-Mailer: git-send-email 2.20.1 (Apple Git-117) In-Reply-To: <20190515180817.71523-1-ajit.khaparde@broadcom.com> References: <20190515180817.71523-1-ajit.khaparde@broadcom.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 3/6] net/bnxt: check for some error conditions in Tx path X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The HW can have limits on the minimum packet size it can support, or the maximum number of segments it can support. Check for such possibilities. Also check if we are going to have a 0 length buffer. Fixes: 6eb3cc2294fd ("net/bnxt: add initial Tx code") Signed-off-by: Ajit Khaparde Reviewed-by: Somnath Kotur Signed-off-by: Sriharsha Basavapatna --- drivers/net/bnxt/bnxt_txr.c | 31 ++++++++++++++++++++++++++++++- drivers/net/bnxt/bnxt_txr.h | 2 ++ 2 files changed, 32 insertions(+), 1 deletion(-) diff --git a/drivers/net/bnxt/bnxt_txr.c b/drivers/net/bnxt/bnxt_txr.c index 3a0d73af2..9684fb177 100644 --- a/drivers/net/bnxt/bnxt_txr.c +++ b/drivers/net/bnxt/bnxt_txr.c @@ -143,6 +143,33 @@ static uint16_t bnxt_start_xmit(struct rte_mbuf *tx_pkt, tx_buf->mbuf = tx_pkt; tx_buf->nr_bds = long_bd + tx_pkt->nb_segs; + /* Check if number of Tx descriptors is above HW limit */ + if (unlikely(tx_buf->nr_bds > BNXT_MAX_TSO_SEGS)) { + PMD_DRV_LOG(ERR, + "Num descriptors %d exceeds HW limit\n", + tx_buf->nr_bds); + return -ENOSPC; + } + + /* If packet length is less than minimum packet size, pad it */ + if (unlikely(rte_pktmbuf_pkt_len(tx_pkt) < BNXT_MIN_PKT_SIZE)) { + uint8_t pad = BNXT_MIN_PKT_SIZE - rte_pktmbuf_pkt_len(tx_pkt); + char *seg = rte_pktmbuf_append(tx_pkt, pad); + + if (!seg) { + PMD_DRV_LOG(ERR, + "Failed to pad mbuf by %d bytes\n", + pad); + return -ENOMEM; + } + + /* Note: data_len, pkt len are updated in rte_pktmbuf_append */ + memset(seg, 0, pad); + } + + /* Check non zero data_len */ + RTE_VERIFY(tx_pkt->data_len); + if (unlikely(bnxt_tx_avail(txr) < tx_buf->nr_bds)) return -ENOMEM; @@ -203,6 +230,7 @@ static uint16_t bnxt_start_xmit(struct rte_mbuf *tx_pkt, */ txbd1->hdr_size = hdr_size >> 1; txbd1->mss = tx_pkt->tso_segsz; + RTE_VERIFY(txbd1->mss); } else if ((tx_pkt->ol_flags & PKT_TX_OIP_IIP_TCP_UDP_CKSUM) == PKT_TX_OIP_IIP_TCP_UDP_CKSUM) { @@ -285,8 +313,9 @@ static uint16_t bnxt_start_xmit(struct rte_mbuf *tx_pkt, } m_seg = tx_pkt->next; - /* i is set at the end of the if(long_bd) block */ while (m_seg) { + /* Check non zero data_len */ + RTE_VERIFY(m_seg->data_len); txr->tx_prod = RING_NEXT(txr->tx_ring_struct, txr->tx_prod); tx_buf = &txr->tx_buf_ring[txr->tx_prod]; diff --git a/drivers/net/bnxt/bnxt_txr.h b/drivers/net/bnxt/bnxt_txr.h index 7f3c7cdb0..f802d5080 100644 --- a/drivers/net/bnxt/bnxt_txr.h +++ b/drivers/net/bnxt/bnxt_txr.h @@ -10,6 +10,8 @@ #define MAX_TX_RINGS 16 #define BNXT_TX_PUSH_THRESH 92 +#define BNXT_MAX_TSO_SEGS 32 +#define BNXT_MIN_PKT_SIZE 52 #define B_TX_DB(db, prod) rte_write32((DB_KEY_TX | (prod)), db) From patchwork Wed May 15 18:08:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ajit Khaparde X-Patchwork-Id: 53444 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 217965F28; Wed, 15 May 2019 20:08:30 +0200 (CEST) Received: from rnd-relay.smtp.broadcom.com (rnd-relay.smtp.broadcom.com [192.19.229.170]) by dpdk.org (Postfix) with ESMTP id BBC9D5B36 for ; Wed, 15 May 2019 20:08:24 +0200 (CEST) Received: from nis-sj1-27.broadcom.com (nis-sj1-27.lvn.broadcom.net [10.75.144.136]) by rnd-relay.smtp.broadcom.com (Postfix) with ESMTP id 7FF3D30C0B1; Wed, 15 May 2019 11:08:22 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.10.3 rnd-relay.smtp.broadcom.com 7FF3D30C0B1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=broadcom.com; s=dkimrelay; t=1557943702; bh=bndTDeJfGFvI3ctLU1bh9giJHGBCV7VOo4H6B7NNYEk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lUKzmWo/2iN4ShpVr2QETWrLAxA8qqZn/zuACeZ5jLiJM/XCojXKhlNvnrVZLrx2q LYL07uSFXp2J8AOcM2D8VVZ46USHmZsBB8z4SrleUbgxwXrIn9aehyfAhfYH6jixAD hg7NEMN3S+RYw/SaHpyAzWmQa2rmJg21tNdA+1bA= Received: from C02VPB22HTD6.wifi.broadcom.net (c02vpb22htd6.wifi.broadcom.net [10.69.74.102]) by nis-sj1-27.broadcom.com (Postfix) with ESMTP id 5CD9BAC07C0; Wed, 15 May 2019 11:08:23 -0700 (PDT) From: Ajit Khaparde To: dev@dpdk.org Cc: ferruh.yigit@intel.com, Sriharsha Basavapatna , Ajit Kumar Khaparde Date: Wed, 15 May 2019 11:08:15 -0700 Message-Id: <20190515180817.71523-5-ajit.khaparde@broadcom.com> X-Mailer: git-send-email 2.20.1 (Apple Git-117) In-Reply-To: <20190515180817.71523-1-ajit.khaparde@broadcom.com> References: <20190515180817.71523-1-ajit.khaparde@broadcom.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 4/6] net/bnxt: fix a couple of issues with Tx batching X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Sriharsha Basavapatna This patch addresses the following issues with Tx batching: 1. Tx stall observed in some conditions: The batching code doesn't request for a completion when only a partial chain of packets is transmitted due to mbuf allocation errors. Because of this, Tx consumer index is not updated correctly and it eventually leads to qfull condition. Fix this by requesting a completion for the last packet in the partial chain that is transmitted successfully. 2. Tx stall seen with Jumbo frames: With jumbo frames, number of TxBDs is > 1. While setting up these additional BDs in bnxt_start_xmit(), the flags field is being set using the OR-assignment operator. We end up using a stale value of the flags field (from a previous use of that descriptor). This results in an invalid completion and eventually leads to tx stall. Fix this to just assign the flags field with the right value. Fixes: 5735eb241947 ("net/bnxt: support Tx batching") Signed-off-by: Sriharsha Basavapatna Reviewed-by: Ajit Kumar Khaparde Signed-off-by: Ajit Khaparde --- drivers/net/bnxt/bnxt_txq.h | 1 - drivers/net/bnxt/bnxt_txr.c | 57 ++++++++++++++++++++++--------------- 2 files changed, 34 insertions(+), 24 deletions(-) diff --git a/drivers/net/bnxt/bnxt_txq.h b/drivers/net/bnxt/bnxt_txq.h index f2c712a75..720ca90cf 100644 --- a/drivers/net/bnxt/bnxt_txq.h +++ b/drivers/net/bnxt/bnxt_txq.h @@ -24,7 +24,6 @@ struct bnxt_tx_queue { uint8_t wthresh; /* Write-back threshold reg */ uint32_t ctx_curr; /* Hardware context states */ uint8_t tx_deferred_start; /* not in global dev start */ - uint8_t cmpl_next; /* Next BD to trigger a compl */ struct bnxt *bp; int index; diff --git a/drivers/net/bnxt/bnxt_txr.c b/drivers/net/bnxt/bnxt_txr.c index 9684fb177..186934136 100644 --- a/drivers/net/bnxt/bnxt_txr.c +++ b/drivers/net/bnxt/bnxt_txr.c @@ -103,26 +103,33 @@ int bnxt_init_tx_ring_struct(struct bnxt_tx_queue *txq, unsigned int socket_id) return 0; } -static inline uint32_t bnxt_tx_avail(struct bnxt_tx_ring_info *txr) +static inline uint32_t bnxt_tx_bds_in_hw(struct bnxt_tx_queue *txq) +{ + return ((txq->tx_ring->tx_prod - txq->tx_ring->tx_cons) & + txq->tx_ring->tx_ring_struct->ring_mask); +} + +static inline uint32_t bnxt_tx_avail(struct bnxt_tx_queue *txq) { /* Tell compiler to fetch tx indices from memory. */ rte_compiler_barrier(); - return txr->tx_ring_struct->ring_size - - ((txr->tx_prod - txr->tx_cons) & - txr->tx_ring_struct->ring_mask) - 1; + return ((txq->tx_ring->tx_ring_struct->ring_size - + bnxt_tx_bds_in_hw(txq)) - 1); } static uint16_t bnxt_start_xmit(struct rte_mbuf *tx_pkt, struct bnxt_tx_queue *txq, uint16_t *coal_pkts, - uint16_t *cmpl_next) + uint16_t *cmpl_next, + struct tx_bd_long **last_txbd) { struct bnxt_tx_ring_info *txr = txq->tx_ring; struct tx_bd_long *txbd; struct tx_bd_long_hi *txbd1 = NULL; uint32_t vlan_tag_flags, cfa_action; bool long_bd = false; + unsigned short nr_bds = 0; struct rte_mbuf *m_seg; struct bnxt_sw_tx_bd *tx_buf; static const uint32_t lhint_arr[4] = { @@ -139,15 +146,14 @@ static uint16_t bnxt_start_xmit(struct rte_mbuf *tx_pkt, PKT_TX_TUNNEL_GENEVE)) long_bd = true; - tx_buf = &txr->tx_buf_ring[txr->tx_prod]; - tx_buf->mbuf = tx_pkt; - tx_buf->nr_bds = long_bd + tx_pkt->nb_segs; + nr_bds = long_bd + tx_pkt->nb_segs; + if (unlikely(bnxt_tx_avail(txq) < nr_bds)) + return -ENOMEM; /* Check if number of Tx descriptors is above HW limit */ - if (unlikely(tx_buf->nr_bds > BNXT_MAX_TSO_SEGS)) { + if (unlikely(nr_bds > BNXT_MAX_TSO_SEGS)) { PMD_DRV_LOG(ERR, - "Num descriptors %d exceeds HW limit\n", - tx_buf->nr_bds); + "Num descriptors %d exceeds HW limit\n", nr_bds); return -ENOSPC; } @@ -170,12 +176,13 @@ static uint16_t bnxt_start_xmit(struct rte_mbuf *tx_pkt, /* Check non zero data_len */ RTE_VERIFY(tx_pkt->data_len); - if (unlikely(bnxt_tx_avail(txr) < tx_buf->nr_bds)) - return -ENOMEM; + tx_buf = &txr->tx_buf_ring[txr->tx_prod]; + tx_buf->mbuf = tx_pkt; + tx_buf->nr_bds = nr_bds; txbd = &txr->tx_desc_ring[txr->tx_prod]; txbd->opaque = *coal_pkts; - txbd->flags_type = tx_buf->nr_bds << TX_BD_LONG_FLAGS_BD_CNT_SFT; + txbd->flags_type = nr_bds << TX_BD_LONG_FLAGS_BD_CNT_SFT; txbd->flags_type |= TX_BD_SHORT_FLAGS_COAL_NOW; if (!*cmpl_next) { txbd->flags_type |= TX_BD_LONG_FLAGS_NO_CMPL; @@ -189,6 +196,7 @@ static uint16_t bnxt_start_xmit(struct rte_mbuf *tx_pkt, else txbd->flags_type |= lhint_arr[tx_pkt->pkt_len >> 9]; txbd->address = rte_cpu_to_le_64(rte_mbuf_data_iova(tx_buf->mbuf)); + *last_txbd = txbd; if (long_bd) { txbd->flags_type |= TX_BD_LONG_TYPE_TX_BD_LONG; @@ -321,7 +329,7 @@ static uint16_t bnxt_start_xmit(struct rte_mbuf *tx_pkt, txbd = &txr->tx_desc_ring[txr->tx_prod]; txbd->address = rte_cpu_to_le_64(rte_mbuf_data_iova(m_seg)); - txbd->flags_type |= TX_BD_SHORT_TYPE_TX_BD_SHORT; + txbd->flags_type = TX_BD_SHORT_TYPE_TX_BD_SHORT; txbd->len = m_seg->data_len; m_seg = m_seg->next; @@ -371,8 +379,7 @@ static int bnxt_handle_tx_cp(struct bnxt_tx_queue *txq) uint32_t ring_mask = cp_ring_struct->ring_mask; uint32_t opaque = 0; - if (((txq->tx_ring->tx_prod - txq->tx_ring->tx_cons) & - txq->tx_ring->tx_ring_struct->ring_mask) < txq->tx_free_thresh) + if (bnxt_tx_bds_in_hw(txq) < txq->tx_free_thresh) return 0; do { @@ -411,7 +418,8 @@ uint16_t bnxt_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, struct bnxt_tx_queue *txq = tx_queue; uint16_t nb_tx_pkts = 0; uint16_t coal_pkts = 0; - uint16_t cmpl_next = txq->cmpl_next; + uint16_t cmpl_next = 0; + struct tx_bd_long *last_txbd = NULL; /* Handle TX completions */ bnxt_handle_tx_cp(txq); @@ -422,20 +430,23 @@ uint16_t bnxt_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, return 0; } - txq->cmpl_next = 0; /* Handle TX burst request */ for (nb_tx_pkts = 0; nb_tx_pkts < nb_pkts; nb_tx_pkts++) { int rc; - /* Request a completion on first and last packet */ + /* Request a completion on the last packet */ cmpl_next |= (nb_pkts == nb_tx_pkts + 1); coal_pkts++; rc = bnxt_start_xmit(tx_pkts[nb_tx_pkts], txq, - &coal_pkts, &cmpl_next); + &coal_pkts, &cmpl_next, &last_txbd); if (unlikely(rc)) { - /* Request a completion in next cycle */ - txq->cmpl_next = 1; + /* Request a completion on the last successfully + * enqueued packet + */ + if (last_txbd) + last_txbd->flags_type &= + ~TX_BD_LONG_FLAGS_NO_CMPL; break; } } From patchwork Wed May 15 18:08:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ajit Khaparde X-Patchwork-Id: 53446 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 877E05F72; Wed, 15 May 2019 20:08:33 +0200 (CEST) Received: from rnd-relay.smtp.broadcom.com (rnd-relay.smtp.broadcom.com [192.19.229.170]) by dpdk.org (Postfix) with ESMTP id D2E7F5F14 for ; Wed, 15 May 2019 20:08:24 +0200 (CEST) Received: from nis-sj1-27.broadcom.com (nis-sj1-27.lvn.broadcom.net [10.75.144.136]) by rnd-relay.smtp.broadcom.com (Postfix) with ESMTP id 9BBAA30C0B7; Wed, 15 May 2019 11:08:22 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.10.3 rnd-relay.smtp.broadcom.com 9BBAA30C0B7 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=broadcom.com; s=dkimrelay; t=1557943702; bh=Gq42Kz7Xo2g9dOVWAvDWrAc8Z+a9xqD16CN9Ue88F28=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vVy3CGGV84mJ9C7LevpwLnBXU/WU8HFvuw91EbWK6gcPDkIkTGlgPE1Qd0qLRWxDN mo9+YOQ/BECytJPJL832kquud0I0Jc1+vmE1WNzNVajHAePTINYzbBseZa5PDF3c05 jIf9ZUBS6uW5HmJBHDyk6tIyFYr9mWwP9BXGjDHg= Received: from C02VPB22HTD6.wifi.broadcom.net (c02vpb22htd6.wifi.broadcom.net [10.69.74.102]) by nis-sj1-27.broadcom.com (Postfix) with ESMTP id 77CDCAC0709; Wed, 15 May 2019 11:08:23 -0700 (PDT) From: Ajit Khaparde To: dev@dpdk.org Cc: ferruh.yigit@intel.com, Sriharsha Basavapatna , Ajit Kumar Khaparde Date: Wed, 15 May 2019 11:08:16 -0700 Message-Id: <20190515180817.71523-6-ajit.khaparde@broadcom.com> X-Mailer: git-send-email 2.20.1 (Apple Git-117) In-Reply-To: <20190515180817.71523-1-ajit.khaparde@broadcom.com> References: <20190515180817.71523-1-ajit.khaparde@broadcom.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 5/6] net/bnxt: optimize Tx batching code X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Sriharsha Basavapatna This patch adds a few optimizations to Tx batching. Instead of specifying 'cmpl_next' to bnxt_start_xmit() as a hint to enable completion for a given packet, request for completion on the last successfully enqueued TxBD. This takes care of both success and error cases. It eliminates 'cmpl_next' arg and related per-packet condition checks around it. Fixes: 5735eb241947 ("net/bnxt: support Tx batching") Signed-off-by: Sriharsha Basavapatna Reviewed-by: Ajit Kumar Khaparde Signed-off-by: Ajit Khaparde --- drivers/net/bnxt/bnxt_txr.c | 32 +++++++++----------------------- 1 file changed, 9 insertions(+), 23 deletions(-) diff --git a/drivers/net/bnxt/bnxt_txr.c b/drivers/net/bnxt/bnxt_txr.c index 186934136..b15778b39 100644 --- a/drivers/net/bnxt/bnxt_txr.c +++ b/drivers/net/bnxt/bnxt_txr.c @@ -121,7 +121,6 @@ static inline uint32_t bnxt_tx_avail(struct bnxt_tx_queue *txq) static uint16_t bnxt_start_xmit(struct rte_mbuf *tx_pkt, struct bnxt_tx_queue *txq, uint16_t *coal_pkts, - uint16_t *cmpl_next, struct tx_bd_long **last_txbd) { struct bnxt_tx_ring_info *txr = txq->tx_ring; @@ -184,12 +183,7 @@ static uint16_t bnxt_start_xmit(struct rte_mbuf *tx_pkt, txbd->opaque = *coal_pkts; txbd->flags_type = nr_bds << TX_BD_LONG_FLAGS_BD_CNT_SFT; txbd->flags_type |= TX_BD_SHORT_FLAGS_COAL_NOW; - if (!*cmpl_next) { - txbd->flags_type |= TX_BD_LONG_FLAGS_NO_CMPL; - } else { - *coal_pkts = 0; - *cmpl_next = false; - } + txbd->flags_type |= TX_BD_LONG_FLAGS_NO_CMPL; txbd->len = tx_pkt->data_len; if (tx_pkt->pkt_len >= 2014) txbd->flags_type |= TX_BD_LONG_FLAGS_LHINT_GTE2K; @@ -415,10 +409,10 @@ static int bnxt_handle_tx_cp(struct bnxt_tx_queue *txq) uint16_t bnxt_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) { - struct bnxt_tx_queue *txq = tx_queue; + int rc; uint16_t nb_tx_pkts = 0; uint16_t coal_pkts = 0; - uint16_t cmpl_next = 0; + struct bnxt_tx_queue *txq = tx_queue; struct tx_bd_long *last_txbd = NULL; /* Handle TX completions */ @@ -432,27 +426,19 @@ uint16_t bnxt_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, /* Handle TX burst request */ for (nb_tx_pkts = 0; nb_tx_pkts < nb_pkts; nb_tx_pkts++) { - int rc; - - /* Request a completion on the last packet */ - cmpl_next |= (nb_pkts == nb_tx_pkts + 1); coal_pkts++; rc = bnxt_start_xmit(tx_pkts[nb_tx_pkts], txq, - &coal_pkts, &cmpl_next, &last_txbd); + &coal_pkts, &last_txbd); - if (unlikely(rc)) { - /* Request a completion on the last successfully - * enqueued packet - */ - if (last_txbd) - last_txbd->flags_type &= - ~TX_BD_LONG_FLAGS_NO_CMPL; + if (unlikely(rc)) break; - } } - if (nb_tx_pkts) + if (likely(nb_tx_pkts)) { + /* Request a completion on the last packet */ + last_txbd->flags_type &= ~TX_BD_LONG_FLAGS_NO_CMPL; B_TX_DB(txq->tx_ring->tx_doorbell, txq->tx_ring->tx_prod); + } return nb_tx_pkts; } From patchwork Wed May 15 18:08:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ajit Khaparde X-Patchwork-Id: 53447 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4F0CF69D4; Wed, 15 May 2019 20:08:35 +0200 (CEST) Received: from rnd-relay.smtp.broadcom.com (rnd-relay.smtp.broadcom.com [192.19.229.170]) by dpdk.org (Postfix) with ESMTP id AFF055A6A for ; Wed, 15 May 2019 20:08:25 +0200 (CEST) Received: from nis-sj1-27.broadcom.com (nis-sj1-27.lvn.broadcom.net [10.75.144.136]) by rnd-relay.smtp.broadcom.com (Postfix) with ESMTP id B47EF30C0BB; Wed, 15 May 2019 11:08:22 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.10.3 rnd-relay.smtp.broadcom.com B47EF30C0BB DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=broadcom.com; s=dkimrelay; t=1557943702; bh=EV3OWFFZVn29XyU5y2c55a+H40GNeBkQ1cDtuMaA+xc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mN6Rm+E69SAP7G54ba8ihxTCNjw65BfCaHJFJwQzNt0L05NNKfF5mH8G3AUflb+ud DcsxgPiKt0pcvCdw4SWmqnbsJxRulrTXT1lO/V6gh+pubmK46HiS3r9yWvZv2nfmOG AEb7K5T2t8UTArNYBsLEf9orR7PntlWvyziV08aA= Received: from C02VPB22HTD6.wifi.broadcom.net (c02vpb22htd6.wifi.broadcom.net [10.69.74.102]) by nis-sj1-27.broadcom.com (Postfix) with ESMTP id 95919AC07C2; Wed, 15 May 2019 11:08:23 -0700 (PDT) From: Ajit Khaparde To: dev@dpdk.org Cc: ferruh.yigit@intel.com, Sriharsha Basavapatna Date: Wed, 15 May 2019 11:08:17 -0700 Message-Id: <20190515180817.71523-7-ajit.khaparde@broadcom.com> X-Mailer: git-send-email 2.20.1 (Apple Git-117) In-Reply-To: <20190515180817.71523-1-ajit.khaparde@broadcom.com> References: <20190515180817.71523-1-ajit.khaparde@broadcom.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 6/6] net/bnxt: support bulk free of Tx mbufs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The driver currently uses rte_pktmbuf_free() to free each mbuf after transmit completion. This is optimized to free multiple mbufs using rte_mempool_put_bulk(). Signed-off-by: Ajit Khaparde Signed-off-by: Sriharsha Basavapatna --- drivers/net/bnxt/bnxt_txq.c | 11 ++++++++ drivers/net/bnxt/bnxt_txq.h | 1 + drivers/net/bnxt/bnxt_txr.c | 50 +++++++++++++++++++++++++++++-------- 3 files changed, 52 insertions(+), 10 deletions(-) diff --git a/drivers/net/bnxt/bnxt_txq.c b/drivers/net/bnxt/bnxt_txq.c index b9b975e4c..5a7bfaf3e 100644 --- a/drivers/net/bnxt/bnxt_txq.c +++ b/drivers/net/bnxt/bnxt_txq.c @@ -69,6 +69,7 @@ void bnxt_tx_queue_release_op(void *tx_queue) rte_memzone_free(txq->mz); txq->mz = NULL; + rte_free(txq->free); rte_free(txq); } } @@ -110,6 +111,16 @@ int bnxt_tx_queue_setup_op(struct rte_eth_dev *eth_dev, rc = -ENOMEM; goto out; } + + txq->free = rte_zmalloc_socket(NULL, + sizeof(struct rte_mbuf *) * nb_desc, + RTE_CACHE_LINE_SIZE, socket_id); + if (!txq->free) { + PMD_DRV_LOG(ERR, "allocation of tx mbuf free array failed!"); + rte_free(txq); + rc = -ENOMEM; + goto out; + } txq->bp = bp; txq->nb_tx_desc = nb_desc; txq->tx_free_thresh = tx_conf->tx_free_thresh; diff --git a/drivers/net/bnxt/bnxt_txq.h b/drivers/net/bnxt/bnxt_txq.h index 720ca90cf..a0d4678d9 100644 --- a/drivers/net/bnxt/bnxt_txq.h +++ b/drivers/net/bnxt/bnxt_txq.h @@ -33,6 +33,7 @@ struct bnxt_tx_queue { unsigned int cp_nr_rings; struct bnxt_cp_ring_info *cp_ring; const struct rte_memzone *mz; + struct rte_mbuf **free; }; void bnxt_free_txq_stats(struct bnxt_tx_queue *txq); diff --git a/drivers/net/bnxt/bnxt_txr.c b/drivers/net/bnxt/bnxt_txr.c index b15778b39..9de12e0d0 100644 --- a/drivers/net/bnxt/bnxt_txr.c +++ b/drivers/net/bnxt/bnxt_txr.c @@ -320,6 +320,7 @@ static uint16_t bnxt_start_xmit(struct rte_mbuf *tx_pkt, RTE_VERIFY(m_seg->data_len); txr->tx_prod = RING_NEXT(txr->tx_ring_struct, txr->tx_prod); tx_buf = &txr->tx_buf_ring[txr->tx_prod]; + tx_buf->mbuf = m_seg; txbd = &txr->tx_desc_ring[txr->tx_prod]; txbd->address = rte_cpu_to_le_64(rte_mbuf_data_iova(m_seg)); @@ -339,24 +340,53 @@ static uint16_t bnxt_start_xmit(struct rte_mbuf *tx_pkt, static void bnxt_tx_cmp(struct bnxt_tx_queue *txq, int nr_pkts) { struct bnxt_tx_ring_info *txr = txq->tx_ring; + struct rte_mempool *pool = NULL; + struct rte_mbuf **free = txq->free; uint16_t cons = txr->tx_cons; + unsigned int blk = 0; int i, j; for (i = 0; i < nr_pkts; i++) { - struct bnxt_sw_tx_bd *tx_buf; struct rte_mbuf *mbuf; + struct bnxt_sw_tx_bd *tx_buf = &txr->tx_buf_ring[cons]; + unsigned short nr_bds = tx_buf->nr_bds; - tx_buf = &txr->tx_buf_ring[cons]; - cons = RING_NEXT(txr->tx_ring_struct, cons); - mbuf = tx_buf->mbuf; - tx_buf->mbuf = NULL; - - /* EW - no need to unmap DMA memory? */ - - for (j = 1; j < tx_buf->nr_bds; j++) + for (j = 0; j < nr_bds; j++) { + mbuf = tx_buf->mbuf; + tx_buf->mbuf = NULL; cons = RING_NEXT(txr->tx_ring_struct, cons); - rte_pktmbuf_free(mbuf); + tx_buf = &txr->tx_buf_ring[cons]; + if (!mbuf) /* long_bd's tx_buf ? */ + continue; + + mbuf = rte_pktmbuf_prefree_seg(mbuf); + if (unlikely(!mbuf)) + continue; + + /* EW - no need to unmap DMA memory? */ + + if (likely(mbuf->pool == pool)) { + /* Add mbuf to the bulk free array */ + free[blk++] = mbuf; + } else { + /* Found an mbuf from a different pool. Free + * mbufs accumulated so far to the previous + * pool + */ + if (likely(pool != NULL)) + rte_mempool_put_bulk(pool, + (void *)free, + blk); + + /* Start accumulating mbufs in a new pool */ + free[0] = mbuf; + pool = mbuf->pool; + blk = 1; + } + } } + if (blk) + rte_mempool_put_bulk(pool, (void *)free, blk); txr->tx_cons = cons; }