From patchwork Mon Aug 5 13:03:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 57442 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D8CA01BE01; Mon, 5 Aug 2019 15:04:10 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 6633B1BDF6 for ; Mon, 5 Aug 2019 15:04:09 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Aug 2019 16:04:04 +0300 Received: from pegasus12.mtr.labs.mlnx (pegasus12.mtr.labs.mlnx [10.210.17.40]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x75D44m0006405; Mon, 5 Aug 2019 16:04:04 +0300 Received: from pegasus12.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus12.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id x75D44r9022350; Mon, 5 Aug 2019 13:04:04 GMT Received: (from viacheslavo@localhost) by pegasus12.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id x75D44Yp022349; Mon, 5 Aug 2019 13:04:04 GMT X-Authentication-Warning: pegasus12.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: yskoh@mellanox.com, matan@mellanox.com Date: Mon, 5 Aug 2019 13:03:49 +0000 Message-Id: <1565010234-21769-2-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1565010234-21769-1-git-send-email-viacheslavo@mellanox.com> References: <1565010234-21769-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH v2 1/6] net/mlx5: fix default minimal data inline X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The patch [Fixes] sets the default value of required minimal inline data to 0 bytes. On some configurations (depends on switchdev/legacy settings and FW version/settings) the ConnectX-4LX NIC requires minimal 18 bytes of Tx descriptor inline data to operate correctly. Wrongly set to 0 default value may prevent NIC from operating with out-of-the-box settings, this patch reverts default value for ConnectX-4LX back to 18 bytes (inline L2). Fixes: 9f350504bb32 ("net/mlx5: fix ConnectX-4LX minimal inline data limit") Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 909c22e..a3eacdb 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1325,12 +1325,9 @@ struct mlx5_dev_spawn_data { switch (spawn->pci_dev->id.device_id) { case PCI_DEVICE_ID_MELLANOX_CONNECTX4: case PCI_DEVICE_ID_MELLANOX_CONNECTX4VF: - config->txq_inline_min = MLX5_INLINE_HSIZE_L2; - config->hw_vlan_insert = 0; - break; case PCI_DEVICE_ID_MELLANOX_CONNECTX4LX: case PCI_DEVICE_ID_MELLANOX_CONNECTX4LXVF: - config->txq_inline_min = MLX5_INLINE_HSIZE_NONE; + config->txq_inline_min = MLX5_INLINE_HSIZE_L2; config->hw_vlan_insert = 0; break; case PCI_DEVICE_ID_MELLANOX_CONNECTX5: From patchwork Mon Aug 5 13:03:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 57443 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E87931BE06; Mon, 5 Aug 2019 15:04:15 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id B6DA51BDF6 for ; Mon, 5 Aug 2019 15:04:13 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Aug 2019 16:04:11 +0300 Received: from pegasus12.mtr.labs.mlnx (pegasus12.mtr.labs.mlnx [10.210.17.40]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x75D4B9Y006455; Mon, 5 Aug 2019 16:04:11 +0300 Received: from pegasus12.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus12.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id x75D4BSm022355; Mon, 5 Aug 2019 13:04:11 GMT Received: (from viacheslavo@localhost) by pegasus12.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id x75D4BcN022354; Mon, 5 Aug 2019 13:04:11 GMT X-Authentication-Warning: pegasus12.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: yskoh@mellanox.com, matan@mellanox.com Date: Mon, 5 Aug 2019 13:03:50 +0000 Message-Id: <1565010234-21769-3-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1565010234-21769-1-git-send-email-viacheslavo@mellanox.com> References: <1565010234-21769-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH v2 2/6] net/mlx5: fix inline data len assert condition X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The debug assert wrongly triggers on the inline data 18B, this should be passed successfully. Fixes: 18a1c20044c0 ("net/mlx5: implement Tx burst template") Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_rxtx.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c index a4fc388..1ec3793 100644 --- a/drivers/net/mlx5/mlx5_rxtx.c +++ b/drivers/net/mlx5/mlx5_rxtx.c @@ -2594,7 +2594,7 @@ enum mlx5_txcmp_code { sizeof(struct rte_vlan_hdr) + 2 * RTE_ETHER_ADDR_LEN), "invalid Ethernet Segment data size"); - assert(inlen > MLX5_ESEG_MIN_INLINE_SIZE); + assert(inlen >= MLX5_ESEG_MIN_INLINE_SIZE); es->inline_hdr_sz = rte_cpu_to_be_16(inlen); pdst = (uint8_t *)&es->inline_data; if (MLX5_TXOFF_CONFIG(VLAN) && vlan) { From patchwork Mon Aug 5 13:03:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 57444 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2A9771BE16; Mon, 5 Aug 2019 15:04:26 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 299D01BE0C for ; Mon, 5 Aug 2019 15:04:24 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Aug 2019 16:04:19 +0300 Received: from pegasus12.mtr.labs.mlnx (pegasus12.mtr.labs.mlnx [10.210.17.40]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x75D4JOH006533; Mon, 5 Aug 2019 16:04:19 +0300 Received: from pegasus12.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus12.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id x75D4J9j022369; Mon, 5 Aug 2019 13:04:19 GMT Received: (from viacheslavo@localhost) by pegasus12.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id x75D4JFp022368; Mon, 5 Aug 2019 13:04:19 GMT X-Authentication-Warning: pegasus12.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: yskoh@mellanox.com, matan@mellanox.com Date: Mon, 5 Aug 2019 13:03:51 +0000 Message-Id: <1565010234-21769-4-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1565010234-21769-1-git-send-email-viacheslavo@mellanox.com> References: <1565010234-21769-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH v2 3/6] net/mlx5: fix completion queue drain loop X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The completion loop speed optimizations for error-free operations are done - no CQE field fetch on each loop iteration. Also, code size is oprimized - the flush buffers routine is invoked once. Fixes: 318ea4cfa1b1 ("net/mlx5: fix Tx completion descriptors fetching loop") Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_rxtx.c | 98 +++++++++++++++++++++++++++++--------------- drivers/net/mlx5/mlx5_rxtx.h | 8 ++-- 2 files changed, 68 insertions(+), 38 deletions(-) diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c index 1ec3793..a890f41 100644 --- a/drivers/net/mlx5/mlx5_rxtx.c +++ b/drivers/net/mlx5/mlx5_rxtx.c @@ -654,9 +654,10 @@ enum mlx5_txcmp_code { * Pointer to the error CQE. * * @return - * The last Tx buffer element to free. + * Negative value if queue recovery failed, + * the last Tx buffer element to free otherwise. */ -uint16_t +int mlx5_tx_error_cqe_handle(struct mlx5_txq_data *restrict txq, volatile struct mlx5_err_cqe *err_cqe) { @@ -706,6 +707,7 @@ enum mlx5_txcmp_code { return txq->elts_head; } /* Recovering failed - try again later on the same WQE. */ + return -1; } else { txq->cq_ci++; } @@ -2010,6 +2012,45 @@ enum mlx5_txcmp_code { } /** + * Update completion queue consuming index via doorbell + * and flush the completed data buffers. + * + * @param txq + * Pointer to TX queue structure. + * @param valid CQE pointer + * if not NULL update txq->wqe_pi and flush the buffers + * @param itail + * if not negative - flush the buffers till this index. + * @param olx + * Configured Tx offloads mask. It is fully defined at + * compile time and may be used for optimization. + */ +static __rte_always_inline void +mlx5_tx_comp_flush(struct mlx5_txq_data *restrict txq, + volatile struct mlx5_cqe *last_cqe, + int itail, + unsigned int olx __rte_unused) +{ + uint16_t tail; + + if (likely(last_cqe != NULL)) { + txq->wqe_pi = rte_be_to_cpu_16(last_cqe->wqe_counter); + tail = ((volatile struct mlx5_wqe_cseg *) + (txq->wqes + (txq->wqe_pi & txq->wqe_m)))->misc; + } else if (itail >= 0) { + tail = (uint16_t)itail; + } else { + return; + } + rte_compiler_barrier(); + *txq->cq_db = rte_cpu_to_be_32(txq->cq_ci); + if (likely(tail != txq->elts_tail)) { + mlx5_tx_free_elts(txq, tail, olx); + assert(tail == txq->elts_tail); + } +} + +/** * Manage TX completions. This routine checks the CQ for * arrived CQEs, deduces the last accomplished WQE in SQ, * updates SQ producing index and frees all completed mbufs. @@ -2028,10 +2069,11 @@ enum mlx5_txcmp_code { unsigned int olx __rte_unused) { unsigned int count = MLX5_TX_COMP_MAX_CQE; - bool update = false; - uint16_t tail = txq->elts_tail; + volatile struct mlx5_cqe *last_cqe = NULL; int ret; + static_assert(MLX5_CQE_STATUS_HW_OWN < 0, "Must be negative value"); + static_assert(MLX5_CQE_STATUS_SW_OWN < 0, "Must be negative value"); do { volatile struct mlx5_cqe *cqe; @@ -2043,32 +2085,30 @@ enum mlx5_txcmp_code { assert(ret == MLX5_CQE_STATUS_HW_OWN); break; } - /* Some error occurred, try to restart. */ + /* + * Some error occurred, try to restart. + * We have no barrier after WQE related Doorbell + * written, make sure all writes are completed + * here, before we might perform SQ reset. + */ rte_wmb(); - tail = mlx5_tx_error_cqe_handle + ret = mlx5_tx_error_cqe_handle (txq, (volatile struct mlx5_err_cqe *)cqe); - if (likely(tail != txq->elts_tail)) { - mlx5_tx_free_elts(txq, tail, olx); - assert(tail == txq->elts_tail); - } - /* Allow flushing all CQEs from the queue. */ - count = txq->cqe_s; - } else { - volatile struct mlx5_wqe_cseg *cseg; - - /* Normal transmit completion. */ - ++txq->cq_ci; - rte_cio_rmb(); - txq->wqe_pi = rte_be_to_cpu_16(cqe->wqe_counter); - cseg = (volatile struct mlx5_wqe_cseg *) - (txq->wqes + (txq->wqe_pi & txq->wqe_m)); - tail = cseg->misc; + /* + * Flush buffers, update consuming index + * if recovery succeeded. Otherwise + * just try to recover later. + */ + last_cqe = NULL; + break; } + /* Normal transmit completion. */ + ++txq->cq_ci; + last_cqe = cqe; #ifndef NDEBUG if (txq->cq_pi) --txq->cq_pi; #endif - update = true; /* * We have to restrict the amount of processed CQEs * in one tx_burst routine call. The CQ may be large @@ -2078,17 +2118,7 @@ enum mlx5_txcmp_code { * latency. */ } while (--count); - if (likely(tail != txq->elts_tail)) { - /* Free data buffers from elts. */ - mlx5_tx_free_elts(txq, tail, olx); - assert(tail == txq->elts_tail); - } - if (likely(update)) { - /* Update the consumer index. */ - rte_compiler_barrier(); - *txq->cq_db = - rte_cpu_to_be_32(txq->cq_ci); - } + mlx5_tx_comp_flush(txq, last_cqe, ret, olx); } /** diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index c209d99..aaa02a2 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -400,7 +400,7 @@ struct mlx5_txq_ctrl *mlx5_txq_new(struct rte_eth_dev *dev, uint16_t idx, void mlx5_set_ptype_table(void); void mlx5_set_cksum_table(void); void mlx5_set_swp_types_table(void); -__rte_noinline uint16_t mlx5_tx_error_cqe_handle +__rte_noinline int mlx5_tx_error_cqe_handle (struct mlx5_txq_data *restrict txq, volatile struct mlx5_err_cqe *err_cqe); uint16_t mlx5_rx_burst(void *dpdk_rxq, struct rte_mbuf **pkts, uint16_t pkts_n); @@ -499,9 +499,9 @@ int mlx5_dma_unmap(struct rte_pci_device *pdev, void *addr, uint64_t iova, /* CQE status. */ enum mlx5_cqe_status { - MLX5_CQE_STATUS_SW_OWN, - MLX5_CQE_STATUS_HW_OWN, - MLX5_CQE_STATUS_ERR, + MLX5_CQE_STATUS_SW_OWN = -1, + MLX5_CQE_STATUS_HW_OWN = -2, + MLX5_CQE_STATUS_ERR = -3, }; /** From patchwork Mon Aug 5 13:03:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 57445 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0999E1BE20; Mon, 5 Aug 2019 15:04:30 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 843351BE20 for ; Mon, 5 Aug 2019 15:04:28 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Aug 2019 16:04:25 +0300 Received: from pegasus12.mtr.labs.mlnx (pegasus12.mtr.labs.mlnx [10.210.17.40]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x75D4PA6006591; Mon, 5 Aug 2019 16:04:25 +0300 Received: from pegasus12.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus12.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id x75D4PaM022372; Mon, 5 Aug 2019 13:04:25 GMT Received: (from viacheslavo@localhost) by pegasus12.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id x75D4Pwx022371; Mon, 5 Aug 2019 13:04:25 GMT X-Authentication-Warning: pegasus12.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: yskoh@mellanox.com, matan@mellanox.com Date: Mon, 5 Aug 2019 13:03:52 +0000 Message-Id: <1565010234-21769-5-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1565010234-21769-1-git-send-email-viacheslavo@mellanox.com> References: <1565010234-21769-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH v2 4/6] net/mlx5: fix inline data settings X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" If the minimal inline data are required the data inline feature must be engaged. There were the incorrect settings enabling the entire small packet inline (in size up to 82B) which may result in sending rate declining if there is no enough cores. The same problem was raised if inline was enabled to support VLAN tag insertion by software. Fixes: 38b4b397a57d ("net/mlx5: add Tx configuration and setup") Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_txq.c | 39 ++++++++++++++++++--------------------- 1 file changed, 18 insertions(+), 21 deletions(-) diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c index fe3b4ec..81f3b40 100644 --- a/drivers/net/mlx5/mlx5_txq.c +++ b/drivers/net/mlx5/mlx5_txq.c @@ -784,13 +784,11 @@ struct mlx5_txq_ibv * txq_ctrl->txq.vlan_en = config->hw_vlan_insert; vlan_inline = (dev_txoff & DEV_TX_OFFLOAD_VLAN_INSERT) && !config->hw_vlan_insert; - if (vlan_inline) - inlen_send = RTE_MAX(inlen_send, MLX5_ESEG_MIN_INLINE_SIZE); /* * If there are few Tx queues it is prioritized * to save CPU cycles and disable data inlining at all. */ - if ((inlen_send && priv->txqs_n >= txqs_inline) || vlan_inline) { + if (inlen_send && priv->txqs_n >= txqs_inline) { /* * The data sent with ordinal MLX5_OPCODE_SEND * may be inlined in Ethernet Segment, align the @@ -825,32 +823,31 @@ struct mlx5_txq_ibv * MLX5_WQE_CSEG_SIZE - MLX5_WQE_ESEG_SIZE - MLX5_WQE_DSEG_SIZE * 2); - txq_ctrl->txq.inlen_send = inlen_send; - txq_ctrl->txq.inlen_mode = inlen_mode; - txq_ctrl->txq.inlen_empw = 0; - } else { + } else if (inlen_mode) { /* * If minimal inlining is requested we must * enable inlining in general, despite the - * number of configured queues. + * number of configured queues. Ignore the + * txq_inline_max devarg, this is not + * full-featured inline. */ inlen_send = inlen_mode; - if (inlen_mode) { - /* - * Extend space for inline data to allow - * optional alignment of data buffer - * start address, it may improve PCIe - * performance. - */ - inlen_send = RTE_MIN(inlen_send + MLX5_WQE_SIZE, - MLX5_SEND_MAX_INLINE_LEN); - } - txq_ctrl->txq.inlen_send = inlen_send; - txq_ctrl->txq.inlen_mode = inlen_mode; - txq_ctrl->txq.inlen_empw = 0; + inlen_empw = 0; + } else if (vlan_inline) { + /* + * Hardware does not report offload for + * VLAN insertion, we must enable data inline + * to implement feature by software. + */ + inlen_send = MLX5_ESEG_MIN_INLINE_SIZE; + inlen_empw = 0; + } else { inlen_send = 0; inlen_empw = 0; } + txq_ctrl->txq.inlen_send = inlen_send; + txq_ctrl->txq.inlen_mode = inlen_mode; + txq_ctrl->txq.inlen_empw = 0; if (inlen_send && inlen_empw && priv->txqs_n >= txqs_inline) { /* * The data sent with MLX5_OPCODE_ENHANCED_MPSW From patchwork Mon Aug 5 13:03:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 57446 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id AD9F61BE05; Mon, 5 Aug 2019 15:04:35 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id DAC981BE0F for ; Mon, 5 Aug 2019 15:04:33 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Aug 2019 16:04:32 +0300 Received: from pegasus12.mtr.labs.mlnx (pegasus12.mtr.labs.mlnx [10.210.17.40]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x75D4Wgu006657; Mon, 5 Aug 2019 16:04:32 +0300 Received: from pegasus12.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus12.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id x75D4WZ4022377; Mon, 5 Aug 2019 13:04:32 GMT Received: (from viacheslavo@localhost) by pegasus12.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id x75D4WlX022376; Mon, 5 Aug 2019 13:04:32 GMT X-Authentication-Warning: pegasus12.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: yskoh@mellanox.com, matan@mellanox.com Date: Mon, 5 Aug 2019 13:03:53 +0000 Message-Id: <1565010234-21769-6-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1565010234-21769-1-git-send-email-viacheslavo@mellanox.com> References: <1565010234-21769-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH v2 5/6] net/mlx5: fix packet size inline settings X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch fixes the default settings for packet size to inline with Enhanced Multi-Packet Write feature, allowing 256B packets to be inlined with Out-Of-the-Box settings. Fixes: 50724e1bba76 ("net/mlx5: update Tx definitions") Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- doc/guides/nics/mlx5.rst | 2 +- drivers/net/mlx5/mlx5_prm.h | 5 ++--- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index 46538b8..5102bcd 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -449,7 +449,7 @@ Run-time configuration and CPU resources are scarce), data inline is not performed by the driver. Assigning ``txqs_min_inline`` with zero always enables the data inline. - The default ``txq_inline_mpw`` value is 188. The specified value may be adjusted + The default ``txq_inline_mpw`` value is 268. The specified value may be adjusted by the driver in order not to exceed the limit (930 bytes) and to provide better WQE space filling without gaps, the adjustment is reflected in the debug log. Due to multiple packets may be included to the same WQE with Enhanced Multi diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h index 4ee6d89..d62837e 100644 --- a/drivers/net/mlx5/mlx5_prm.h +++ b/drivers/net/mlx5/mlx5_prm.h @@ -72,9 +72,8 @@ * boundary with accounting the title Control and Ethernet * segments. */ -#define MLX5_EMPW_DEF_INLINE_LEN (3u * MLX5_WQE_SIZE + \ - MLX5_DSEG_MIN_INLINE_SIZE - \ - MLX5_WQE_DSEG_SIZE) +#define MLX5_EMPW_DEF_INLINE_LEN (4u * MLX5_WQE_SIZE + \ + MLX5_DSEG_MIN_INLINE_SIZE) /* * Maximal inline data length sent with enhanced MPW. * Is based on maximal WQE size. From patchwork Mon Aug 5 13:03:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 57447 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 64D7B1BE23; Mon, 5 Aug 2019 15:04:40 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 13B941BE23 for ; Mon, 5 Aug 2019 15:04:39 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Aug 2019 16:04:35 +0300 Received: from pegasus12.mtr.labs.mlnx (pegasus12.mtr.labs.mlnx [10.210.17.40]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x75D4ZJp006697; Mon, 5 Aug 2019 16:04:35 +0300 Received: from pegasus12.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus12.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id x75D4ZAM022381; Mon, 5 Aug 2019 13:04:35 GMT Received: (from viacheslavo@localhost) by pegasus12.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id x75D4Zq3022380; Mon, 5 Aug 2019 13:04:35 GMT X-Authentication-Warning: pegasus12.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: yskoh@mellanox.com, matan@mellanox.com Date: Mon, 5 Aug 2019 13:03:54 +0000 Message-Id: <1565010234-21769-7-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1565010234-21769-1-git-send-email-viacheslavo@mellanox.com> References: <1565010234-21769-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH v2 6/6] net/mlx5: fix completion queue overflow for large bursts X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" There is the limit on completion descriptor fetch to improve latency. If burst size is large there might be not enough resources freed in completion processing. This fix reiterates sending loop and allows multiple completion descriptor fetch and processing. Fixes: 18a1c20044c0 ("net/mlx5: implement Tx burst template") Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_rxtx.c | 36 +++++++++++++++++++++++------------- drivers/net/mlx5/mlx5_rxtx.h | 1 + 2 files changed, 24 insertions(+), 13 deletions(-) diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c index a890f41..39076df 100644 --- a/drivers/net/mlx5/mlx5_rxtx.c +++ b/drivers/net/mlx5/mlx5_rxtx.c @@ -2142,8 +2142,7 @@ enum mlx5_txcmp_code { uint16_t head = txq->elts_head; unsigned int part; - part = MLX5_TXOFF_CONFIG(INLINE) ? 0 : loc->pkts_sent - - (MLX5_TXOFF_CONFIG(MULTI) ? loc->pkts_copy : 0); + part = MLX5_TXOFF_CONFIG(INLINE) ? 0 : loc->pkts_sent - loc->pkts_copy; head += part; if ((uint16_t)(head - txq->elts_comp) >= MLX5_TX_COMP_THRESH || (MLX5_TXOFF_CONFIG(INLINE) && @@ -4520,6 +4519,14 @@ enum mlx5_txcmp_code { assert(txq->elts_s >= (uint16_t)(txq->elts_head - txq->elts_tail)); assert(txq->wqe_s >= (uint16_t)(txq->wqe_ci - txq->wqe_pi)); + if (unlikely(!pkts_n)) + return 0; + loc.pkts_sent = 0; + loc.pkts_copy = 0; + loc.wqe_last = NULL; + +send_loop: + loc.pkts_loop = loc.pkts_sent; /* * Check if there are some CQEs, if any: * - process an encountered errors @@ -4527,9 +4534,7 @@ enum mlx5_txcmp_code { * - free related mbufs * - doorbell the NIC about processed CQEs */ - if (unlikely(!pkts_n)) - return 0; - rte_prefetch0(*pkts); + rte_prefetch0(*(pkts + loc.pkts_sent)); mlx5_tx_handle_completion(txq, olx); /* * Calculate the number of available resources - elts and WQEs. @@ -4546,10 +4551,7 @@ enum mlx5_txcmp_code { loc.wqe_free = txq->wqe_s - (uint16_t)(txq->wqe_ci - txq->wqe_pi); if (unlikely(!loc.elts_free || !loc.wqe_free)) - return 0; - loc.pkts_sent = 0; - loc.pkts_copy = 0; - loc.wqe_last = NULL; + return loc.pkts_sent; for (;;) { /* * Fetch the packet from array. Usually this is @@ -4715,8 +4717,8 @@ enum mlx5_txcmp_code { */ assert(MLX5_TXOFF_CONFIG(INLINE) || loc.pkts_sent >= loc.pkts_copy); /* Take a shortcut if nothing is sent. */ - if (unlikely(loc.pkts_sent == 0)) - return 0; + if (unlikely(loc.pkts_sent == loc.pkts_loop)) + return loc.pkts_sent; /* * Ring QP doorbell immediately after WQE building completion * to improve latencies. The pure software related data treatment @@ -4725,8 +4727,7 @@ enum mlx5_txcmp_code { */ mlx5_tx_dbrec_cond_wmb(txq, loc.wqe_last, 0); /* Not all of the mbufs may be stored into elts yet. */ - part = MLX5_TXOFF_CONFIG(INLINE) ? 0 : loc.pkts_sent - - (MLX5_TXOFF_CONFIG(MULTI) ? loc.pkts_copy : 0); + part = MLX5_TXOFF_CONFIG(INLINE) ? 0 : loc.pkts_sent - loc.pkts_copy; if (!MLX5_TXOFF_CONFIG(INLINE) && part) { /* * There are some single-segment mbufs not stored in elts. @@ -4738,6 +4739,7 @@ enum mlx5_txcmp_code { * inlined mbufs. */ mlx5_tx_copy_elts(txq, pkts + loc.pkts_copy, part, olx); + loc.pkts_copy = loc.pkts_sent; } #ifdef MLX5_PMD_SOFT_COUNTERS /* Increment sent packets counter. */ @@ -4745,6 +4747,14 @@ enum mlx5_txcmp_code { #endif assert(txq->elts_s >= (uint16_t)(txq->elts_head - txq->elts_tail)); assert(txq->wqe_s >= (uint16_t)(txq->wqe_ci - txq->wqe_pi)); + if (pkts_n > loc.pkts_sent) { + /* + * If burst size is large there might be no enough CQE + * fetched from completion queue and no enough resources + * freed to send all the packets. + */ + goto send_loop; + } return loc.pkts_sent; } diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index aaa02a2..bad9e9c 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -239,6 +239,7 @@ struct mlx5_txq_local { struct rte_mbuf *mbuf; /* first mbuf to process. */ uint16_t pkts_copy; /* packets copied to elts. */ uint16_t pkts_sent; /* packets sent. */ + uint16_t pkts_loop; /* packets sent on loop entry. */ uint16_t elts_free; /* available elts remain. */ uint16_t wqe_free; /* available wqe remain. */ uint16_t mbuf_off; /* data offset in current mbuf. */