From patchwork Mon Feb 17 13:31:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bing Zhao X-Patchwork-Id: 65833 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5BD99A0553; Mon, 17 Feb 2020 14:32:02 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 306231DA9F; Mon, 17 Feb 2020 14:32:01 +0100 (CET) Received: from git-send-mailer.rdmz.labs.mlnx (unknown [37.142.13.130]) by dpdk.org (Postfix) with ESMTP id 53A481D6F3; Mon, 17 Feb 2020 14:32:00 +0100 (CET) From: Bing Zhao To: orika@mellanox.com, matan@mellanox.com Cc: viacheslavo@mellanox.com, rasland@mellanox.com, dev@dpdk.org, stable@dpdk.org Date: Mon, 17 Feb 2020 21:31:41 +0800 Message-Id: <1581946301-457865-1-git-send-email-bingz@mellanox.com> X-Mailer: git-send-email 1.8.3.1 Subject: [dpdk-dev] [PATCH] net/mlx5: fix the hairpin queue capacity X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The hairpin TX/RX queue depth and packet size is fixed in the past. When the firmware has some fix or improvement, the PMD will not make full use of it. And also, 32 packets for a single queue will not guarantee a good performance for hairpin flows. The parameter of hairpin queue setup needs to be adjusted. Number of packets of a single queue should be the maximum supported value, and the maximum single packet size should support the standard jumbo frame with 9KB. In the meanwhile, there is no need to support the max capacity of a single packet because the memory consumption should also be taken into consideration. Fixes: e79c9be91515 ("net/mlx5: support Rx hairpin queues") Cc: orika@mellanox.com Cc: stable@dpdk.org Signed-off-by: Bing Zhao Acked-by: Ori Kam --- drivers/net/mlx5/mlx5_defs.h | 4 ++++ drivers/net/mlx5/mlx5_rxq.c | 12 ++++++++---- drivers/net/mlx5/mlx5_txq.c | 12 ++++++++---- 3 files changed, 20 insertions(+), 8 deletions(-) diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h index 9b392ed..19e8253 100644 --- a/drivers/net/mlx5/mlx5_defs.h +++ b/drivers/net/mlx5/mlx5_defs.h @@ -173,6 +173,10 @@ #define MLX5_FLOW_MREG_HNAME "MARK_COPY_TABLE" #define MLX5_DEFAULT_COPY_ID UINT32_MAX +/* Hairpin TX/RX queue configuration parameters. */ +#define MLX5_HAIRPIN_QUEUE_STRIDE 6 +#define MLX5_HAIRPIN_JUMBO_LOG_SIZE (14 + 2) + /* Definition of static_assert found in /usr/include/assert.h */ #ifndef HAVE_STATIC_ASSERT #define static_assert _Static_assert diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index dc0fd82..ac9016e 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -1268,6 +1268,7 @@ struct mlx5_devx_create_rq_attr attr = { 0 }; struct mlx5_rxq_obj *tmpl = NULL; int ret = 0; + uint32_t max_wq_data; MLX5_ASSERT(rxq_data); MLX5_ASSERT(!rxq_ctrl->obj); @@ -1283,11 +1284,14 @@ tmpl->type = MLX5_RXQ_OBJ_TYPE_DEVX_HAIRPIN; tmpl->rxq_ctrl = rxq_ctrl; attr.hairpin = 1; - /* Workaround for hairpin startup */ - attr.wq_attr.log_hairpin_num_packets = log2above(32); - /* Workaround for packets larger than 1KB */ + max_wq_data = priv->config.hca_attr.log_max_hairpin_wq_data_sz; + /* Set the packets number to the maximum value for performance. */ + attr.wq_attr.log_hairpin_num_packets = max_wq_data - + MLX5_HAIRPIN_QUEUE_STRIDE; + /* Jumbo frames > 9KB should be supported. */ attr.wq_attr.log_hairpin_data_sz = - priv->config.hca_attr.log_max_hairpin_wq_data_sz; + (max_wq_data < MLX5_HAIRPIN_JUMBO_LOG_SIZE) ? + max_wq_data : MLX5_HAIRPIN_JUMBO_LOG_SIZE; tmpl->rq = mlx5_devx_cmd_create_rq(priv->sh->ctx, &attr, rxq_ctrl->socket); if (!tmpl->rq) { diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c index bc13abf..6c08bb9 100644 --- a/drivers/net/mlx5/mlx5_txq.c +++ b/drivers/net/mlx5/mlx5_txq.c @@ -493,6 +493,7 @@ struct mlx5_devx_create_sq_attr attr = { 0 }; struct mlx5_txq_obj *tmpl = NULL; int ret = 0; + uint32_t max_wq_data; MLX5_ASSERT(txq_data); MLX5_ASSERT(!txq_ctrl->obj); @@ -509,11 +510,14 @@ tmpl->txq_ctrl = txq_ctrl; attr.hairpin = 1; attr.tis_lst_sz = 1; - /* Workaround for hairpin startup */ - attr.wq_attr.log_hairpin_num_packets = log2above(32); - /* Workaround for packets larger than 1KB */ + max_wq_data = priv->config.hca_attr.log_max_hairpin_wq_data_sz; + /* Set the packets number to the maximum value for performance. */ + attr.wq_attr.log_hairpin_num_packets = max_wq_data - + MLX5_HAIRPIN_QUEUE_STRIDE; + /* Jumbo frames > 9KB should be supported. */ attr.wq_attr.log_hairpin_data_sz = - priv->config.hca_attr.log_max_hairpin_wq_data_sz; + (max_wq_data < MLX5_HAIRPIN_JUMBO_LOG_SIZE) ? + max_wq_data : MLX5_HAIRPIN_JUMBO_LOG_SIZE; attr.tis_num = priv->sh->tis->id; tmpl->sq = mlx5_devx_cmd_create_sq(priv->sh->ctx, &attr); if (!tmpl->sq) {