From patchwork Tue Mar 31 21:52:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Kozyrev X-Patchwork-Id: 67557 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 16003A0562; Tue, 31 Mar 2020 23:52:59 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2407C1C1A9; Tue, 31 Mar 2020 23:52:50 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 6D09F1C0C3 for ; Tue, 31 Mar 2020 23:52:46 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE2 (envelope-from akozyrev@mellanox.com) with ESMTPS (AES256-SHA encrypted); 1 Apr 2020 00:52:45 +0300 Received: from pegasus02.mtr.labs.mlnx. (pegasus02.mtr.labs.mlnx [10.210.16.122]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 02VLqjEJ014122; Wed, 1 Apr 2020 00:52:45 +0300 From: Alexander Kozyrev To: dev@dpdk.org Cc: rasland@mellanox.com, matan@mellanox.com, viacheslavo@mellanox.com, ferruh.yigit@intel.com, thomas@monjalon.net Date: Tue, 31 Mar 2020 21:52:36 +0000 Message-Id: <1585691559-17409-2-git-send-email-akozyrev@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1585691559-17409-1-git-send-email-akozyrev@mellanox.com> References: <1585691559-17409-1-git-send-email-akozyrev@mellanox.com> Subject: [dpdk-dev] [PATCH 1/4] net/mlx5: add a devarg to specify MPRQ stride size X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Define a device parameter to configure log 2 of a stride size for MPRQ - mprq_log_stride_size. User is able to specify a stride size in a range allowed by an underlying hardware. The default stride size is defined as 2048 bytes to encompass most commonly used packet sizes in the Internet (MTU 1518 and less) and will be used in case a maximum configured packet size cannot fit into the largest possible stride size. Otherwise a stride size is set to a large enough value to encompass a whole packet. Signed-off-by: Alexander Kozyrev Acked-by: Viacheslav Ovsiienko --- drivers/net/mlx5/mlx5.c | 34 +++++++++++++++++++++++++++------- drivers/net/mlx5/mlx5.h | 1 + drivers/net/mlx5/mlx5_defs.h | 3 +++ drivers/net/mlx5/mlx5_rxq.c | 22 +++++++++++++--------- 4 files changed, 44 insertions(+), 16 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 6a11b14..a2ba6d3 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -63,6 +63,9 @@ /* Device parameter to configure log 2 of the number of strides for MPRQ. */ #define MLX5_RX_MPRQ_LOG_STRIDE_NUM "mprq_log_stride_num" +/* Device parameter to configure log 2 of the stride size for MPRQ. */ +#define MLX5_RX_MPRQ_LOG_STRIDE_SIZE "mprq_log_stride_size" + /* Device parameter to limit the size of memcpy'd packet for MPRQ. */ #define MLX5_RX_MPRQ_MAX_MEMCPY_LEN "mprq_max_memcpy_len" @@ -1531,6 +1534,8 @@ struct mlx5_flow_id_pool * config->mprq.enabled = !!tmp; } else if (strcmp(MLX5_RX_MPRQ_LOG_STRIDE_NUM, key) == 0) { config->mprq.stride_num_n = tmp; + } else if (strcmp(MLX5_RX_MPRQ_LOG_STRIDE_SIZE, key) == 0) { + config->mprq.stride_size_n = tmp; } else if (strcmp(MLX5_RX_MPRQ_MAX_MEMCPY_LEN, key) == 0) { config->mprq.max_memcpy_len = tmp; } else if (strcmp(MLX5_RXQS_MIN_MPRQ, key) == 0) { @@ -1627,6 +1632,7 @@ struct mlx5_flow_id_pool * MLX5_RXQ_PKT_PAD_EN, MLX5_RX_MPRQ_EN, MLX5_RX_MPRQ_LOG_STRIDE_NUM, + MLX5_RX_MPRQ_LOG_STRIDE_SIZE, MLX5_RX_MPRQ_MAX_MEMCPY_LEN, MLX5_RXQS_MIN_MPRQ, MLX5_TXQ_INLINE, @@ -2302,8 +2308,6 @@ struct mlx5_flow_id_pool * mprq_caps.min_single_wqe_log_num_of_strides; mprq_max_stride_num_n = mprq_caps.max_single_wqe_log_num_of_strides; - config.mprq.stride_num_n = RTE_MAX(MLX5_MPRQ_STRIDE_NUM_N, - mprq_min_stride_num_n); } #endif if (RTE_CACHE_LINE_SIZE == 128 && @@ -2617,17 +2621,32 @@ struct mlx5_flow_id_pool * #endif } if (config.mprq.enabled && mprq) { - if (config.mprq.stride_num_n > mprq_max_stride_num_n || - config.mprq.stride_num_n < mprq_min_stride_num_n) { + if (config.mprq.stride_num_n && + (config.mprq.stride_num_n > mprq_max_stride_num_n || + config.mprq.stride_num_n < mprq_min_stride_num_n)) { config.mprq.stride_num_n = - RTE_MAX(MLX5_MPRQ_STRIDE_NUM_N, - mprq_min_stride_num_n); + RTE_MIN(RTE_MAX(MLX5_MPRQ_STRIDE_NUM_N, + mprq_min_stride_num_n), + mprq_max_stride_num_n); DRV_LOG(WARNING, "the number of strides" " for Multi-Packet RQ is out of range," " setting default value (%u)", 1 << config.mprq.stride_num_n); } + if (config.mprq.stride_size_n && + (config.mprq.stride_size_n > mprq_max_stride_size_n || + config.mprq.stride_size_n < mprq_min_stride_size_n)) { + config.mprq.stride_size_n = + RTE_MIN(RTE_MAX(MLX5_MPRQ_STRIDE_SIZE_N, + mprq_min_stride_size_n), + mprq_max_stride_size_n); + DRV_LOG(WARNING, + "the size of a stride" + " for Multi-Packet RQ is out of range," + " setting default value (%u)", + 1 << config.mprq.stride_size_n); + } config.mprq.min_stride_size_n = mprq_min_stride_size_n; config.mprq.max_stride_size_n = mprq_max_stride_size_n; } else if (config.mprq.enabled && !mprq) { @@ -3361,7 +3380,8 @@ struct mlx5_flow_id_pool * .mr_ext_memseg_en = 1, .mprq = { .enabled = 0, /* Disabled by default. */ - .stride_num_n = MLX5_MPRQ_STRIDE_NUM_N, + .stride_num_n = 0, + .stride_size_n = 0, .max_memcpy_len = MLX5_MPRQ_MEMCPY_DEFAULT_LEN, .min_rxqs_num = MLX5_MPRQ_MIN_RXQS, }, diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 62b0810..c8e2454 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -176,6 +176,7 @@ struct mlx5_dev_config { struct { unsigned int enabled:1; /* Whether MPRQ is enabled. */ unsigned int stride_num_n; /* Number of strides. */ + unsigned int stride_size_n; /* Size of a stride. */ unsigned int min_stride_size_n; /* Min size of a stride. */ unsigned int max_stride_size_n; /* Max size of a stride. */ unsigned int max_memcpy_len; diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h index 19e8253..260f584 100644 --- a/drivers/net/mlx5/mlx5_defs.h +++ b/drivers/net/mlx5/mlx5_defs.h @@ -143,6 +143,9 @@ /* Log 2 of the default number of strides per WQE for Multi-Packet RQ. */ #define MLX5_MPRQ_STRIDE_NUM_N 6U +/* Log 2 of the default size of a stride per WQE for Multi-Packet RQ. */ +#define MLX5_MPRQ_STRIDE_SIZE_N 11U + /* Two-byte shift is disabled for Multi-Packet RQ. */ #define MLX5_MPRQ_TWO_BYTE_SHIFT 0 diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index 0a95e3c..85fcfe6 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -1856,25 +1856,26 @@ struct mlx5_rxq_ctrl * strd_headroom_en = 1; mprq_stride_size = non_scatter_min_mbuf_size; } + if (!config->mprq.stride_num_n) + config->mprq.stride_num_n = MLX5_MPRQ_STRIDE_NUM_N; + if (!config->mprq.stride_size_n) + config->mprq.stride_size_n = (mprq_stride_size <= + (1U << config->mprq.max_stride_size_n)) ? + log2above(mprq_stride_size) : MLX5_MPRQ_STRIDE_SIZE_N; /* * This Rx queue can be configured as a Multi-Packet RQ if all of the * following conditions are met: * - MPRQ is enabled. * - The number of descs is more than the number of strides. - * - max_rx_pkt_len plus overhead is less than the max size of a - * stride. * Otherwise, enable Rx scatter if necessary. */ - if (mprq_en && - desc > (1U << config->mprq.stride_num_n) && - mprq_stride_size <= (1U << config->mprq.max_stride_size_n)) { + if (mprq_en && desc > (1U << config->mprq.stride_num_n)) { /* TODO: Rx scatter isn't supported yet. */ tmpl->rxq.sges_n = 0; /* Trim the number of descs needed. */ desc >>= config->mprq.stride_num_n; tmpl->rxq.strd_num_n = config->mprq.stride_num_n; - tmpl->rxq.strd_sz_n = RTE_MAX(log2above(mprq_stride_size), - config->mprq.min_stride_size_n); + tmpl->rxq.strd_sz_n = config->mprq.stride_size_n; tmpl->rxq.strd_shift_en = MLX5_MPRQ_TWO_BYTE_SHIFT; tmpl->rxq.strd_headroom_en = strd_headroom_en; tmpl->rxq.mprq_max_memcpy_len = RTE_MIN(first_mb_free_size, @@ -1924,9 +1925,12 @@ struct mlx5_rxq_ctrl * DRV_LOG(WARNING, "port %u MPRQ is requested but cannot be enabled" " (requested: desc = %u, stride_sz = %u," - " supported: min_stride_num = %u, max_stride_sz = %u).", - dev->data->port_id, desc, mprq_stride_size, + " supported: min_stride_num = %u, min_stride_sz = %u," + "max_stride_sz = %u).", + dev->data->port_id, desc, + (1 << config->mprq.stride_size_n), (1 << config->mprq.stride_num_n), + (1 << config->mprq.min_stride_size_n), (1 << config->mprq.max_stride_size_n)); DRV_LOG(DEBUG, "port %u maximum number of segments per packet: %u", dev->data->port_id, 1 << tmpl->rxq.sges_n); From patchwork Tue Mar 31 21:52:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Kozyrev X-Patchwork-Id: 67559 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id ADCBAA0562; Tue, 31 Mar 2020 23:53:14 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7967A1C1C4; Tue, 31 Mar 2020 23:52:53 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id C297E1C19E for ; Tue, 31 Mar 2020 23:52:46 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from akozyrev@mellanox.com) with ESMTPS (AES256-SHA encrypted); 1 Apr 2020 00:52:45 +0300 Received: from pegasus02.mtr.labs.mlnx. (pegasus02.mtr.labs.mlnx [10.210.16.122]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 02VLqjEK014122; Wed, 1 Apr 2020 00:52:45 +0300 From: Alexander Kozyrev To: dev@dpdk.org Cc: rasland@mellanox.com, matan@mellanox.com, viacheslavo@mellanox.com, ferruh.yigit@intel.com, thomas@monjalon.net Date: Tue, 31 Mar 2020 21:52:37 +0000 Message-Id: <1585691559-17409-3-git-send-email-akozyrev@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1585691559-17409-1-git-send-email-akozyrev@mellanox.com> References: <1585691559-17409-1-git-send-email-akozyrev@mellanox.com> Subject: [dpdk-dev] [PATCH 2/4] net/mlx5: enable MPRQ multi-stride operations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" MPRQ feature should be updated to allow a packet to be received into multiple strides in order to support the MTU exceeding 8KB. Special care is needed to prevent the headroom corruption in the multi-stride mode since the headroom space is borrowed by the PMD from the tail of the preceding stride. Copy the whole packet into a separate mbuf in this case or just the overlapping data if the Rx scattering is supported by an application. Signed-off-by: Alexander Kozyrev Acked-by: Viacheslav Ovsiienko --- drivers/net/mlx5/mlx5_rxq.c | 25 ++++------------ drivers/net/mlx5/mlx5_rxtx.c | 68 +++++++++++++++++++------------------------- drivers/net/mlx5/mlx5_rxtx.h | 2 +- 3 files changed, 35 insertions(+), 60 deletions(-) diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index 85fcfe6..a64f536 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -1793,9 +1793,7 @@ struct mlx5_rxq_ctrl * struct mlx5_priv *priv = dev->data->dev_private; struct mlx5_rxq_ctrl *tmpl; unsigned int mb_len = rte_pktmbuf_data_room_size(mp); - unsigned int mprq_stride_size; struct mlx5_dev_config *config = &priv->config; - unsigned int strd_headroom_en; /* * Always allocate extra slots, even if eventually * the vector Rx will not be used. @@ -1841,27 +1839,13 @@ struct mlx5_rxq_ctrl * tmpl->socket = socket; if (dev->data->dev_conf.intr_conf.rxq) tmpl->irq = 1; - /* - * LRO packet may consume all the stride memory, hence we cannot - * guaranty head-room near the packet memory in the stride. - * In this case scatter is, for sure, enabled and an empty mbuf may be - * added in the start for the head-room. - */ - if (lro_on_queue && RTE_PKTMBUF_HEADROOM > 0 && - non_scatter_min_mbuf_size > mb_len) { - strd_headroom_en = 0; - mprq_stride_size = RTE_MIN(max_rx_pkt_len, - 1u << config->mprq.max_stride_size_n); - } else { - strd_headroom_en = 1; - mprq_stride_size = non_scatter_min_mbuf_size; - } if (!config->mprq.stride_num_n) config->mprq.stride_num_n = MLX5_MPRQ_STRIDE_NUM_N; if (!config->mprq.stride_size_n) - config->mprq.stride_size_n = (mprq_stride_size <= + config->mprq.stride_size_n = (non_scatter_min_mbuf_size <= (1U << config->mprq.max_stride_size_n)) ? - log2above(mprq_stride_size) : MLX5_MPRQ_STRIDE_SIZE_N; + log2above(non_scatter_min_mbuf_size) : + MLX5_MPRQ_STRIDE_SIZE_N; /* * This Rx queue can be configured as a Multi-Packet RQ if all of the * following conditions are met: @@ -1877,7 +1861,8 @@ struct mlx5_rxq_ctrl * tmpl->rxq.strd_num_n = config->mprq.stride_num_n; tmpl->rxq.strd_sz_n = config->mprq.stride_size_n; tmpl->rxq.strd_shift_en = MLX5_MPRQ_TWO_BYTE_SHIFT; - tmpl->rxq.strd_headroom_en = strd_headroom_en; + tmpl->rxq.strd_scatter_en = + !!(offloads & DEV_RX_OFFLOAD_SCATTER); tmpl->rxq.mprq_max_memcpy_len = RTE_MIN(first_mb_free_size, config->mprq.max_memcpy_len); max_lro_size = RTE_MIN(max_rx_pkt_len, diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c index f3bf763..4c27952 100644 --- a/drivers/net/mlx5/mlx5_rxtx.c +++ b/drivers/net/mlx5/mlx5_rxtx.c @@ -1658,21 +1658,20 @@ enum mlx5_txcmp_code { unsigned int i = 0; uint32_t rq_ci = rxq->rq_ci; uint16_t consumed_strd = rxq->consumed_strd; - uint16_t headroom_sz = rxq->strd_headroom_en * RTE_PKTMBUF_HEADROOM; struct mlx5_mprq_buf *buf = (*rxq->mprq_bufs)[rq_ci & wq_mask]; while (i < pkts_n) { struct rte_mbuf *pkt; void *addr; int ret; - unsigned int len; + uint32_t len; uint16_t strd_cnt; uint16_t strd_idx; uint32_t offset; uint32_t byte_cnt; + int32_t hdrm_overlap; volatile struct mlx5_mini_cqe8 *mcqe = NULL; uint32_t rss_hash_res = 0; - uint8_t lro_num_seg; if (consumed_strd == strd_n) { /* Replace WQE only if the buffer is still in use. */ @@ -1719,18 +1718,6 @@ enum mlx5_txcmp_code { MLX5_ASSERT(strd_idx < strd_n); MLX5_ASSERT(!((rte_be_to_cpu_16(cqe->wqe_id) ^ rq_ci) & wq_mask)); - lro_num_seg = cqe->lro_num_seg; - /* - * Currently configured to receive a packet per a stride. But if - * MTU is adjusted through kernel interface, device could - * consume multiple strides without raising an error. In this - * case, the packet should be dropped because it is bigger than - * the max_rx_pkt_len. - */ - if (unlikely(!lro_num_seg && strd_cnt > 1)) { - ++rxq->stats.idropped; - continue; - } pkt = rte_pktmbuf_alloc(rxq->mp); if (unlikely(pkt == NULL)) { ++rxq->stats.rx_nombuf; @@ -1742,12 +1729,16 @@ enum mlx5_txcmp_code { len -= RTE_ETHER_CRC_LEN; offset = strd_idx * strd_sz + strd_shift; addr = RTE_PTR_ADD(mlx5_mprq_buf_addr(buf, strd_n), offset); + hdrm_overlap = len + RTE_PKTMBUF_HEADROOM - strd_cnt * strd_sz; /* * Memcpy packets to the target mbuf if: * - The size of packet is smaller than mprq_max_memcpy_len. * - Out of buffer in the Mempool for Multi-Packet RQ. + * - There is no space for a headroom and scatter is disabled. */ - if (len <= rxq->mprq_max_memcpy_len || rxq->mprq_repl == NULL) { + if (len <= rxq->mprq_max_memcpy_len || + rxq->mprq_repl == NULL || + (hdrm_overlap > 0 && !rxq->strd_scatter_en)) { /* * When memcpy'ing packet due to out-of-buffer, the * packet must be smaller than the target mbuf. @@ -1769,7 +1760,7 @@ enum mlx5_txcmp_code { rte_atomic16_add_return(&buf->refcnt, 1); MLX5_ASSERT((uint16_t)rte_atomic16_read(&buf->refcnt) <= strd_n + 1); - buf_addr = RTE_PTR_SUB(addr, headroom_sz); + buf_addr = RTE_PTR_SUB(addr, RTE_PKTMBUF_HEADROOM); /* * MLX5 device doesn't use iova but it is necessary in a * case where the Rx packet is transmitted via a @@ -1788,43 +1779,42 @@ enum mlx5_txcmp_code { rte_pktmbuf_attach_extbuf(pkt, buf_addr, buf_iova, buf_len, shinfo); /* Set mbuf head-room. */ - pkt->data_off = headroom_sz; + SET_DATA_OFF(pkt, RTE_PKTMBUF_HEADROOM); MLX5_ASSERT(pkt->ol_flags == EXT_ATTACHED_MBUF); - /* - * Prevent potential overflow due to MTU change through - * kernel interface. - */ - if (unlikely(rte_pktmbuf_tailroom(pkt) < len)) { - rte_pktmbuf_free_seg(pkt); - ++rxq->stats.idropped; - continue; - } + MLX5_ASSERT(rte_pktmbuf_tailroom(pkt) < + len - (hdrm_overlap > 0 ? hdrm_overlap : 0)); DATA_LEN(pkt) = len; /* - * LRO packet may consume all the stride memory, in this - * case packet head-room space is not guaranteed so must - * to add an empty mbuf for the head-room. + * Copy the last fragment of a packet (up to headroom + * size bytes) in case there is a stride overlap with + * a next packet's headroom. Allocate a separate mbuf + * to store this fragment and link it. Scatter is on. */ - if (!rxq->strd_headroom_en) { - struct rte_mbuf *headroom_mbuf = - rte_pktmbuf_alloc(rxq->mp); + if (hdrm_overlap > 0) { + MLX5_ASSERT(rxq->strd_scatter_en); + struct rte_mbuf *seg = + rte_pktmbuf_alloc(rxq->mp); - if (unlikely(headroom_mbuf == NULL)) { + if (unlikely(seg == NULL)) { rte_pktmbuf_free_seg(pkt); ++rxq->stats.rx_nombuf; break; } - PORT(pkt) = rxq->port_id; - NEXT(headroom_mbuf) = pkt; - pkt = headroom_mbuf; + SET_DATA_OFF(seg, 0); + rte_memcpy(rte_pktmbuf_mtod(seg, void *), + RTE_PTR_ADD(addr, len - hdrm_overlap), + hdrm_overlap); + DATA_LEN(seg) = hdrm_overlap; + DATA_LEN(pkt) = len - hdrm_overlap; + NEXT(pkt) = seg; NB_SEGS(pkt) = 2; } } rxq_cq_to_mbuf(rxq, pkt, cqe, rss_hash_res); - if (lro_num_seg > 1) { + if (cqe->lro_num_seg > 1) { mlx5_lro_update_hdr(addr, cqe, len); pkt->ol_flags |= PKT_RX_LRO; - pkt->tso_segsz = strd_sz; + pkt->tso_segsz = len / cqe->lro_num_seg; } PKT_LEN(pkt) = len; PORT(pkt) = rxq->port_id; diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index 939778a..d155c24 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -119,7 +119,7 @@ struct mlx5_rxq_data { unsigned int strd_sz_n:4; /* Log 2 of stride size. */ unsigned int strd_shift_en:1; /* Enable 2bytes shift on a stride. */ unsigned int err_state:2; /* enum mlx5_rxq_err_state. */ - unsigned int strd_headroom_en:1; /* Enable mbuf headroom in MPRQ. */ + unsigned int strd_scatter_en:1; /* Scattered packets from a stride. */ unsigned int lro:1; /* Enable LRO. */ unsigned int :1; /* Remaining bits. */ volatile uint32_t *rq_db; From patchwork Tue Mar 31 21:52:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Kozyrev X-Patchwork-Id: 67556 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6AAE6A0562; Tue, 31 Mar 2020 23:52:49 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 845341C136; Tue, 31 Mar 2020 23:52:48 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 717051C136 for ; Tue, 31 Mar 2020 23:52:46 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE2 (envelope-from akozyrev@mellanox.com) with ESMTPS (AES256-SHA encrypted); 1 Apr 2020 00:52:45 +0300 Received: from pegasus02.mtr.labs.mlnx. (pegasus02.mtr.labs.mlnx [10.210.16.122]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 02VLqjEL014122; Wed, 1 Apr 2020 00:52:45 +0300 From: Alexander Kozyrev To: dev@dpdk.org Cc: rasland@mellanox.com, matan@mellanox.com, viacheslavo@mellanox.com, ferruh.yigit@intel.com, thomas@monjalon.net Date: Tue, 31 Mar 2020 21:52:38 +0000 Message-Id: <1585691559-17409-4-git-send-email-akozyrev@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1585691559-17409-1-git-send-email-akozyrev@mellanox.com> References: <1585691559-17409-1-git-send-email-akozyrev@mellanox.com> Subject: [dpdk-dev] [PATCH 3/4] doc: add a decsription for MPRQ stride size devarg X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Provide a descriptin of a newly added mprq_log_stride_size devarg parameter for specifying a stride size in case of MPRQ Rx is on. Signed-off-by: Alexander Kozyrev --- doc/guides/nics/mlx5.rst | 9 +++++++++ doc/guides/rel_notes/release_20_05.rst | 1 + 2 files changed, 10 insertions(+) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index e13c07d..4e8c130 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -434,6 +434,15 @@ Run-time configuration The size of Rx queue should be bigger than the number of strides. +- ``mprq_log_stride_size`` parameter [int] + + Log 2 of the size of a stride for Multi-Packet Rx queue. Configuring a smaller + stride size can save some memory and reduce probability of a depletion of all + available strides due to unreleased packets by an application. If configured + value is not in the range of device capability, the default value will be set + with a warning message. The default value is 11 which is 2048 bytes per a + stride, valid only if ``mprq_en`` is set. + - ``mprq_max_memcpy_len`` parameter [int] The maximum length of packet to memcpy in case of Multi-Packet Rx queue. Rx diff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst index c960fd2..1459218 100644 --- a/doc/guides/rel_notes/release_20_05.rst +++ b/doc/guides/rel_notes/release_20_05.rst @@ -62,6 +62,7 @@ New Features * Added support for matching on IPv4 Time To Live and IPv6 Hop Limit. * Added support for creating Relaxed Ordering Memory Regions. + * Added support for 9000 MTU in Multi-Packet RQ mode. Removed Items ------------- From patchwork Tue Mar 31 21:52:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Kozyrev X-Patchwork-Id: 67560 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A2F90A0562; Tue, 31 Mar 2020 23:53:24 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0921E1C1DA; Tue, 31 Mar 2020 23:52:55 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id BBBB21C1B7 for ; Tue, 31 Mar 2020 23:52:51 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from akozyrev@mellanox.com) with ESMTPS (AES256-SHA encrypted); 1 Apr 2020 00:52:45 +0300 Received: from pegasus02.mtr.labs.mlnx. (pegasus02.mtr.labs.mlnx [10.210.16.122]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 02VLqjEM014122; Wed, 1 Apr 2020 00:52:45 +0300 From: Alexander Kozyrev To: dev@dpdk.org Cc: rasland@mellanox.com, matan@mellanox.com, viacheslavo@mellanox.com, ferruh.yigit@intel.com, thomas@monjalon.net Date: Tue, 31 Mar 2020 21:52:39 +0000 Message-Id: <1585691559-17409-5-git-send-email-akozyrev@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1585691559-17409-1-git-send-email-akozyrev@mellanox.com> References: <1585691559-17409-1-git-send-email-akozyrev@mellanox.com> Subject: [dpdk-dev] [PATCH 4/4] net/mlx5: add multi-segment packets in MPRQ mode X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The multi-stride operations now allow to reduce a stride size while supporting Jumbo frames. That means that it is possible to have mbufs configured with a size smaller than the whole packet received. It is not an issue during normal MPRQ operations since we attach external buffers instead of copying the data into the mbuf itself. But it is not the case in "emergency mode" when we have to copy every packet because of no more external mbufs are available. Assemble a multi-segment packet to overcome this issue in case scatter mode is enabled, drop a packet if not. Signed-off-by: Alexander Kozyrev Acked-by: Viacheslav Ovsiienko --- drivers/net/mlx5/mlx5_rxtx.c | 47 ++++++++++++++++++++++++++++++++++++-------- 1 file changed, 39 insertions(+), 8 deletions(-) diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c index 4c27952..7ce3732 100644 --- a/drivers/net/mlx5/mlx5_rxtx.c +++ b/drivers/net/mlx5/mlx5_rxtx.c @@ -1734,22 +1734,52 @@ enum mlx5_txcmp_code { * Memcpy packets to the target mbuf if: * - The size of packet is smaller than mprq_max_memcpy_len. * - Out of buffer in the Mempool for Multi-Packet RQ. - * - There is no space for a headroom and scatter is disabled. + * - The packet's stride overlaps a headroom and scatter is off. */ if (len <= rxq->mprq_max_memcpy_len || rxq->mprq_repl == NULL || (hdrm_overlap > 0 && !rxq->strd_scatter_en)) { - /* - * When memcpy'ing packet due to out-of-buffer, the - * packet must be smaller than the target mbuf. - */ - if (unlikely(rte_pktmbuf_tailroom(pkt) < len)) { + if (likely(rte_pktmbuf_tailroom(pkt) >= len)) { + rte_memcpy(rte_pktmbuf_mtod(pkt, void *), + addr, len); + DATA_LEN(pkt) = len; + } else if (rxq->strd_scatter_en) { + struct rte_mbuf *prev = pkt; + uint32_t seg_len = + RTE_MIN(rte_pktmbuf_tailroom(pkt), len); + uint32_t rem_len = len - seg_len; + + rte_memcpy(rte_pktmbuf_mtod(pkt, void *), + addr, seg_len); + DATA_LEN(pkt) = seg_len; + while (rem_len) { + struct rte_mbuf *next = + rte_pktmbuf_alloc(rxq->mp); + + if (unlikely(next == NULL)) { + rte_pktmbuf_free(pkt); + ++rxq->stats.rx_nombuf; + goto out; + } + NEXT(prev) = next; + SET_DATA_OFF(next, 0); + addr = RTE_PTR_ADD(addr, seg_len); + seg_len = RTE_MIN + (rte_pktmbuf_tailroom(next), + rem_len); + rte_memcpy + (rte_pktmbuf_mtod(next, void *), + addr, seg_len); + DATA_LEN(next) = seg_len; + rem_len -= seg_len; + prev = next; + ++NB_SEGS(pkt); + } + } else { rte_pktmbuf_free_seg(pkt); ++rxq->stats.idropped; continue; } - rte_memcpy(rte_pktmbuf_mtod(pkt, void *), addr, len); - DATA_LEN(pkt) = len; } else { rte_iova_t buf_iova; struct rte_mbuf_ext_shared_info *shinfo; @@ -1826,6 +1856,7 @@ enum mlx5_txcmp_code { *(pkts++) = pkt; ++i; } +out: /* Update the consumer indexes. */ rxq->consumed_strd = consumed_strd; rte_cio_wmb();