From patchwork Tue Nov 5 08:01:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 62427 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id ACD98A0352; Tue, 5 Nov 2019 09:03:40 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2E4361BEAA; Tue, 5 Nov 2019 09:02:20 +0100 (CET) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 852D41BE86 for ; Tue, 5 Nov 2019 09:02:13 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Nov 2019 10:02:10 +0200 Received: from pegasus11.mtr.labs.mlnx (pegasus11.mtr.labs.mlnx [10.210.16.104]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id xA582Aai026497; Tue, 5 Nov 2019 10:02:10 +0200 Received: from pegasus11.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id xA582AEa030774; Tue, 5 Nov 2019 08:02:10 GMT Received: (from viacheslavo@localhost) by pegasus11.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id xA582AFt030773; Tue, 5 Nov 2019 08:02:10 GMT X-Authentication-Warning: pegasus11.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net, orika@mellanox.com Date: Tue, 5 Nov 2019 08:01:45 +0000 Message-Id: <1572940915-29416-11-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> References: <1572940915-29416-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH 10/20] net/mlx5: adjust shared register according to mask X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The metadata register reg_c[0] might be used by kernel or firmware for their internal purposes. The actual used mask can be queried from the kernel. The remaining bits can be used by PMD to provide META or MARK feature. The code queries the mask of reg_c[0] and adjust the resource usage dynamically. Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5.c | 95 +++++++++++++++++++++++++++++++++++------ drivers/net/mlx5/mlx5.h | 3 ++ drivers/net/mlx5/mlx5_flow_dv.c | 26 +++++++++-- 3 files changed, 107 insertions(+), 17 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 943d0e8..fb7b94b 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1584,6 +1584,60 @@ struct mlx5_flow_id_pool * } /** + * Configures the metadata mask fields in the shared context. + * + * @param [in] dev + * Pointer to Ethernet device. + */ +static void +mlx5_set_metadata_mask(struct rte_eth_dev *dev) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_ibv_shared *sh = priv->sh; + uint32_t meta, mark, reg_c0; + + reg_c0 = ~priv->vport_meta_mask; + switch (priv->config.dv_xmeta_en) { + case MLX5_XMETA_MODE_LEGACY: + meta = UINT32_MAX; + mark = MLX5_FLOW_MARK_MASK; + break; + case MLX5_XMETA_MODE_META16: + meta = reg_c0 >> rte_bsf32(reg_c0); + mark = MLX5_FLOW_MARK_MASK; + break; + case MLX5_XMETA_MODE_META32: + meta = UINT32_MAX; + mark = (reg_c0 >> rte_bsf32(reg_c0)) & MLX5_FLOW_MARK_MASK; + break; + default: + meta = 0; + mark = 0; + assert(false); + break; + } + if (sh->dv_mark_mask && sh->dv_mark_mask != mark) + DRV_LOG(WARNING, "metadata MARK mask mismatche %08X:%08X", + sh->dv_mark_mask, mark); + else + sh->dv_mark_mask = mark; + if (sh->dv_meta_mask && sh->dv_meta_mask != meta) + DRV_LOG(WARNING, "metadata META mask mismatche %08X:%08X", + sh->dv_meta_mask, meta); + else + sh->dv_meta_mask = meta; + if (sh->dv_regc0_mask && sh->dv_regc0_mask != reg_c0) + DRV_LOG(WARNING, "metadata reg_c0 mask mismatche %08X:%08X", + sh->dv_meta_mask, reg_c0); + else + sh->dv_regc0_mask = reg_c0; + DRV_LOG(DEBUG, "metadata mode %u", priv->config.dv_xmeta_en); + DRV_LOG(DEBUG, "metadata MARK mask %08X", sh->dv_mark_mask); + DRV_LOG(DEBUG, "metadata META mask %08X", sh->dv_meta_mask); + DRV_LOG(DEBUG, "metadata reg_c0 mask %08X", sh->dv_regc0_mask); +} + +/** * Allocate page of door-bells and register it using DevX API. * * @param [in] dev @@ -1803,7 +1857,7 @@ struct mlx5_flow_id_pool * uint16_t port_id; unsigned int i; #ifdef HAVE_MLX5DV_DR_DEVX_PORT - struct mlx5dv_devx_port devx_port; + struct mlx5dv_devx_port devx_port = { .comp_mask = 0 }; #endif /* Determine if this port representor is supposed to be spawned. */ @@ -2035,13 +2089,17 @@ struct mlx5_flow_id_pool * * vport index. The engaged part of metadata register is * defined by mask. */ - devx_port.comp_mask = MLX5DV_DEVX_PORT_VPORT | - MLX5DV_DEVX_PORT_MATCH_REG_C_0; - err = mlx5_glue->devx_port_query(sh->ctx, spawn->ibv_port, &devx_port); - if (err) { - DRV_LOG(WARNING, "can't query devx port %d on device %s", - spawn->ibv_port, spawn->ibv_dev->name); - devx_port.comp_mask = 0; + if (switch_info->representor || switch_info->master) { + devx_port.comp_mask = MLX5DV_DEVX_PORT_VPORT | + MLX5DV_DEVX_PORT_MATCH_REG_C_0; + err = mlx5_glue->devx_port_query(sh->ctx, spawn->ibv_port, + &devx_port); + if (err) { + DRV_LOG(WARNING, + "can't query devx port %d on device %s", + spawn->ibv_port, spawn->ibv_dev->name); + devx_port.comp_mask = 0; + } } if (devx_port.comp_mask & MLX5DV_DEVX_PORT_MATCH_REG_C_0) { priv->vport_meta_tag = devx_port.reg_c_0.value; @@ -2361,18 +2419,27 @@ struct mlx5_flow_id_pool * goto error; } priv->config.flow_prio = err; - /* Query availibility of metadata reg_c's. */ - err = mlx5_flow_discover_mreg_c(eth_dev); - if (err < 0) { - err = -err; - goto error; - } if (!priv->config.dv_esw_en && priv->config.dv_xmeta_en != MLX5_XMETA_MODE_LEGACY) { DRV_LOG(WARNING, "metadata mode %u is not supported " "(no E-Switch)", priv->config.dv_xmeta_en); priv->config.dv_xmeta_en = MLX5_XMETA_MODE_LEGACY; } + mlx5_set_metadata_mask(eth_dev); + if (priv->config.dv_xmeta_en != MLX5_XMETA_MODE_LEGACY && + !priv->sh->dv_regc0_mask) { + DRV_LOG(ERR, "metadata mode %u is not supported " + "(no metadata reg_c[0] is available)", + priv->config.dv_xmeta_en); + err = ENOTSUP; + goto error; + } + /* Query availibility of metadata reg_c's. */ + err = mlx5_flow_discover_mreg_c(eth_dev); + if (err < 0) { + err = -err; + goto error; + } if (!mlx5_flow_ext_mreg_supported(eth_dev)) { DRV_LOG(DEBUG, "port %u extensive metadata register is not supported", diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index e59f8f6..92d445a 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -622,6 +622,9 @@ struct mlx5_ibv_shared { } mr; /* Shared DV/DR flow data section. */ pthread_mutex_t dv_mutex; /* DV context mutex. */ + uint32_t dv_meta_mask; /* flow META metadata supported mask. */ + uint32_t dv_mark_mask; /* flow MARK metadata supported mask. */ + uint32_t dv_regc0_mask; /* available bits of metatada reg_c[0]. */ uint32_t dv_refcnt; /* DV/DR data reference counter. */ void *fdb_domain; /* FDB Direct Rules name space handle. */ struct mlx5_flow_tbl_resource fdb_tbl[MLX5_MAX_TABLES_FDB]; diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index 8b93022..049d6ae 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -899,13 +899,13 @@ struct field_modify_info modify_tcp[] = { * 0 on success, a negative errno value otherwise and rte_errno is set. */ static int -flow_dv_convert_action_copy_mreg(struct rte_eth_dev *dev __rte_unused, +flow_dv_convert_action_copy_mreg(struct rte_eth_dev *dev, struct mlx5_flow_dv_modify_hdr_resource *res, const struct rte_flow_action *action, struct rte_flow_error *error) { const struct mlx5_flow_action_copy_mreg *conf = action->conf; - uint32_t mask = RTE_BE32(UINT32_MAX); + rte_be32_t mask = RTE_BE32(UINT32_MAX); struct rte_flow_item item = { .spec = NULL, .mask = &mask, @@ -915,9 +915,29 @@ struct field_modify_info modify_tcp[] = { {0, 0, 0}, }; struct field_modify_info reg_dst = { - .offset = (uint32_t)-1, /* Same as src. */ + .offset = 0, .id = reg_to_field[conf->dst], }; + /* Adjust reg_c[0] usage according to reported mask. */ + if (conf->dst == REG_C_0 || conf->src == REG_C_0) { + struct mlx5_priv *priv = dev->data->dev_private; + uint32_t reg_c0 = priv->sh->dv_regc0_mask; + + assert(reg_c0); + assert(priv->config.dv_xmeta_en != MLX5_XMETA_MODE_LEGACY); + if (conf->dst == REG_C_0) { + /* Copy to reg_c[0], within mask only. */ + reg_dst.offset = rte_bsf32(reg_c0); + /* + * Mask is ignoring the enianness, because + * there is no conversion in datapath. + */ + mask = reg_c0 >> reg_dst.offset; + } else { + /* Copy from reg_c[0] to destination lower bits. */ + mask = rte_cpu_to_be_32(reg_c0); + } + } return flow_dv_convert_modify_action(&item, reg_src, ®_dst, res, MLX5_MODIFICATION_TYPE_COPY,