From patchwork Sun Oct 29 18:22:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory Etelson X-Patchwork-Id: 133611 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2B25043239; Sun, 29 Oct 2023 19:24:31 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2C9FB410D3; Sun, 29 Oct 2023 19:23:54 +0100 (CET) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2069.outbound.protection.outlook.com [40.107.244.69]) by mails.dpdk.org (Postfix) with ESMTP id EC84740E25 for ; Sun, 29 Oct 2023 19:23:51 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HDA49cPUrvzgGCbBsEqaAG99ZkWFtvnd96QAwwmINbKiElBWW5a59KhW7fPiWulKmPIPNYo+J8gBldPji5DoUYVkzBabeuo3Z4n4ThtFpdmXMcXIpqGqKHdeaCp7A/Isy3pb7KZeEk3P+yAgSWDMwwmqH2Khh0QKhPRoFDUN0VTWyiENF1JSCC06ccUUQ/hQ0xSJFbvc4EGgP8n00ybcCv1A3jgBuGrdjLqPRF9Se3aADYR1eSArrEFy9RN0kDYjAgtcIB8WZmxOiIkKnRvMijMPuWzuz88QmkKu23h0kQTFtK8yaC2c6wNG6rPlLZOH4rQ8lOiG7HRwG0/NlSsnFQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=du5R5pRS4ZrC/V6tF8HLRLJ9fd2QnBF98u6Eljj41Dg=; b=bh1RKFW/TQpWNnyL8TYnwTGflSEjXEFU279OD3ebj9678/OKdm1004L63qVdzkOWs7eI6duAH3HliP82RS2W9TLz9X+4LNdghqGiJ4e1pp24Vm9Q/AmYBasffrtXqrAcauFiU5m82hZJC6MEqRWmRJXuWP7LhifUamZi0yOtdNjm+zhoyViOk+za2XVsrr0sYARZs3bOtl0fWpKpcfImXRWUCoZkZJir+FtpxBn4NbBy+0WxcSYHjy+46tj8yyhOO8O/dB5ufxWAquQUdnXhtLYZ3rL4cxfr+m9iI22A/lSS/p3C/3/UbG4RNT+WxztPlqpVK1MrzUS37wnYj6NSvQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=du5R5pRS4ZrC/V6tF8HLRLJ9fd2QnBF98u6Eljj41Dg=; b=fWWob3BN8a2D2xXWJMFhMMWc4X/3h0VC0TQUc386D0eVRO4Kt4HL32vuLQ9EoMOFabMRmxN65FPAWlH/FDvmWGx8RXqLmdvLr4FD7mjD5wvxf3aGR6rXw6HSUG6w7HDNaAj0XM1ZJ5drsM7AzXdaPualYhm3yzDG1RoLLkBrRNdK2VEs7A26dG1wGUKMYK9qV8UD+xj1d9IWPoXfOfU3LxKjNE3khZv9KmK+mlwoRw9RnLQuLJ2vVqjx35ysTz/dgJrkfuB3fvOUoYEooGHLQ5XDE5U5C/sgY3EzSpsQYiX+DcxOAlWjqz71/myHG2rCors2qPC4DeF+FmCXFEmWsQ== Received: from BYAPR05CA0047.namprd05.prod.outlook.com (2603:10b6:a03:74::24) by MW4PR12MB6952.namprd12.prod.outlook.com (2603:10b6:303:207::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.26; Sun, 29 Oct 2023 18:23:49 +0000 Received: from CO1PEPF000044EF.namprd05.prod.outlook.com (2603:10b6:a03:74:cafe::15) by BYAPR05CA0047.outlook.office365.com (2603:10b6:a03:74::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6954.14 via Frontend Transport; Sun, 29 Oct 2023 18:23:48 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CO1PEPF000044EF.mail.protection.outlook.com (10.167.241.69) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.15 via Frontend Transport; Sun, 29 Oct 2023 18:23:48 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Sun, 29 Oct 2023 11:23:39 -0700 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Sun, 29 Oct 2023 11:23:36 -0700 From: Gregory Etelson To: CC: , , , "Ori Kam" , Matan Azrad , Viacheslav Ovsiienko , Suanming Mou Subject: [PATCH 07/13] net/mlx5: initialize HWS flow tags registers in shared dev context Date: Sun, 29 Oct 2023 20:22:54 +0200 Message-ID: <20231029182300.227879-8-getelson@nvidia.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231029182300.227879-1-getelson@nvidia.com> References: <20231029182300.227879-1-getelson@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044EF:EE_|MW4PR12MB6952:EE_ X-MS-Office365-Filtering-Correlation-Id: 699ccbcd-bdcf-4b5c-eced-08dbd8ac3207 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: JbDsr2T+srlqm5MZj0ruApMVwLV8ohkac9TSr2zXo/cMK9CcKn51e9BzO6V6En9836YjYZ7fpxKaJDOkPHwhvHuz4LTDPgDK0tVZOt0TwKxNv+E9/6+4UofchdJfX0eIESJZWSDqSXRfrw31FsE/Odt5kQoT/TNyQmjbr5JlFRPjYbBLiT+22kl0wyKloIi7JBGC9JkJpSwnTgmF33u7oXed/JyzMlbSpD0v0UtogFrzvGK5HKsJZp2cO9sbl9U8tBtOLMHLi68DqCrZ2hBTh2ODhkTrxsECkvdxt91M2sbIUA3FqmdiqZA90h/ucYYWXQQeizokrXyxObNMZy+zWh9MBuM7IvUYv2PsBgjasJSHt3LIlciFvO7w/M8nfD2noyPyh+h1BEjzfEk+4KVUqWpCtR48QsvJZyrMDYOSy9AT8OqSFeBWxvb/alnL9qfMpW6U4INaRtO3clRyRdMdROt1pAmoxa3wtSCjxmYLBJzcD/6XmYDQ9kkaPNgEO7BSAr+EikAnB0mDN04fUI4DaTN8mAYOs6sPE32r7mTVk/ZsjKRx639FO5ZwPknNEKCKNL64Bvu9skY1jp5Z813rmjMx5IFfwv/13Jcb4IY0zo9OZ5DUoj5nokCpXyVNUVLI35xm4rrA4a2cgjNlVjCBLODqumVIZva4OkQhnfpq8UOuD2rNdS7WlF3geV4aVh5onH0lErO+JiilCp33SCEa+oYPHKhTbsiNWYn9w7SyHUzJjyhwwCWSg9MBjE+eBoWcqCzDXCCrkEsZC1JTak+6uFnFpm7EVpiM8R0dODNdZtY= X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230031)(4636009)(396003)(376002)(346002)(39860400002)(136003)(230922051799003)(230273577357003)(230173577357003)(64100799003)(82310400011)(186009)(1800799009)(451199024)(46966006)(40470700004)(36840700001)(7696005)(6666004)(478600001)(83380400001)(47076005)(6286002)(26005)(16526019)(2616005)(1076003)(107886003)(336012)(426003)(2906002)(5660300002)(54906003)(41300700001)(6916009)(70586007)(316002)(70206006)(8936002)(8676002)(4326008)(36860700001)(86362001)(36756003)(356005)(7636003)(82740400003)(40480700001)(55016003)(40460700003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Oct 2023 18:23:48.6849 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 699ccbcd-bdcf-4b5c-eced-08dbd8ac3207 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044EF.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB6952 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Move HWS flow tags registers initialization to shared dev context. Signed-off-by: Gregory Etelson Acked-by: Ori Kam --- drivers/net/mlx5/linux/mlx5_os.c | 35 ++------------- drivers/net/mlx5/mlx5.c | 75 ++++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5.h | 6 +++ drivers/net/mlx5/mlx5_flow.h | 3 -- drivers/net/mlx5/mlx5_flow_hw.c | 34 --------------- 5 files changed, 84 insertions(+), 69 deletions(-) diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index ed273e14cf..ec067ef52c 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -1304,38 +1304,12 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, struct mlx5_hca_attr *hca_attr = &sh->cdev->config.hca_attr; sh->steering_format_version = hca_attr->steering_format_version; -#if defined(HAVE_MLX5DV_DR) && \ - (defined(HAVE_MLX5_DR_CREATE_ACTION_FLOW_METER) || \ - defined(HAVE_MLX5_DR_CREATE_ACTION_ASO)) +#if defined(HAVE_MLX5_DR_CREATE_ACTION_ASO_EXT) if (hca_attr->qos.sup && hca_attr->qos.flow_meter_old && sh->config.dv_flow_en) { - uint8_t reg_c_mask = hca_attr->qos.flow_meter_reg_c_ids; - /* - * Meter needs two REG_C's for color match and pre-sfx - * flow match. Here get the REG_C for color match. - * REG_C_0 and REG_C_1 is reserved for metadata feature. - */ - reg_c_mask &= 0xfc; - if (rte_popcount32(reg_c_mask) < 1) { - priv->mtr_en = 0; - DRV_LOG(WARNING, "No available register for" - " meter."); - } else { - /* - * The meter color register is used by the - * flow-hit feature as well. - * The flow-hit feature must use REG_C_3 - * Prefer REG_C_3 if it is available. - */ - if (reg_c_mask & (1 << (REG_C_3 - REG_C_0))) - sh->registers.aso_reg = REG_C_3; - else - sh->registers.aso_reg = - ffs(reg_c_mask) - 1 + REG_C_0; + if (sh->registers.aso_reg != REG_NON) { priv->mtr_en = 1; priv->mtr_reg_share = hca_attr->qos.flow_meter; - DRV_LOG(DEBUG, "The REG_C meter uses is %d", - sh->registers.aso_reg); } } if (hca_attr->qos.sup && hca_attr->qos.flow_meter_aso_sup) { @@ -1358,7 +1332,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, sh->tunnel_header_0_1 = 1; if (hca_attr->flow.tunnel_header_2_3) sh->tunnel_header_2_3 = 1; -#endif +#endif /* HAVE_MLX5_DR_CREATE_ACTION_ASO_EXT */ #ifdef HAVE_MLX5_DR_CREATE_ACTION_ASO if (hca_attr->flow_hit_aso && sh->registers.aso_reg == REG_C_3) { sh->flow_hit_aso_en = 1; @@ -1617,9 +1591,6 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, err = ENOTSUP; goto error; } - /* Only HWS requires this information. */ - if (sh->refcnt == 1) - flow_hw_init_tags_set(eth_dev); if (priv->sh->config.dv_esw_en && flow_hw_create_vport_action(eth_dev)) { DRV_LOG(ERR, "port %u failed to create vport action", diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index c13ce2c13c..840c566162 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1599,6 +1599,80 @@ mlx5_rt_timestamp_config(struct mlx5_dev_ctx_shared *sh, } } +static void +mlx5_init_hws_flow_tags_registers(struct mlx5_dev_ctx_shared *sh) +{ + struct mlx5_dev_registers *reg = &sh->registers; + uint32_t meta_mode = sh->config.dv_xmeta_en; + uint8_t masks = (uint8_t)sh->cdev->config.hca_attr.set_reg_c; + uint8_t unset = 0; + uint32_t i, j; + + /* + * The CAPA is global for common device but only used in net. + * It is shared per eswitch domain. + */ + if (reg->aso_reg != REG_NON) + unset |= 1 << mlx5_regc_index(reg->aso_reg); + unset |= 1 << mlx5_regc_index(REG_C_6); + if (sh->config.dv_esw_en) + unset |= 1 << mlx5_regc_index(REG_C_0); + if (meta_mode == MLX5_XMETA_MODE_META32_HWS) + unset |= 1 << mlx5_regc_index(REG_C_1); + masks &= ~unset; + for (i = 0, j = 0; i < MLX5_FLOW_HW_TAGS_MAX; i++) { + if (!!((1 << i) & masks)) + reg->hw_avl_tags[j++] = mlx5_regc_value(i); + } +} + +static void +mlx5_init_aso_register(struct mlx5_dev_ctx_shared *sh) +{ +#if defined(HAVE_MLX5_DR_CREATE_ACTION_ASO_EXT) + const struct mlx5_hca_attr *hca_attr = &sh->cdev->config.hca_attr; + const struct mlx5_hca_qos_attr *qos = &hca_attr->qos; + uint8_t reg_c_mask = qos->flow_meter_reg_c_ids & 0xfc; + + if (!(qos->sup && qos->flow_meter_old && sh->config.dv_flow_en)) + return; + /* + * Meter needs two REG_C's for color match and pre-sfx + * flow match. Here get the REG_C for color match. + * REG_C_0 and REG_C_1 is reserved for metadata feature. + */ + if (__builtin_popcount(reg_c_mask) > 0) { + /* + * The meter color register is used by the + * flow-hit feature as well. + * The flow-hit feature must use REG_C_3 + * Prefer REG_C_3 if it is available. + */ + if (reg_c_mask & (1 << mlx5_regc_index(REG_C_3))) + sh->registers.aso_reg = REG_C_3; + else + sh->registers.aso_reg = + mlx5_regc_value(ffs(reg_c_mask) - 1); + } +#else + RTE_SET_USED(sh); +#endif +} + +static void +mlx5_init_shared_dev_registers(struct mlx5_dev_ctx_shared *sh) +{ + if (sh->cdev->config.devx) + mlx5_init_aso_register(sh); + if (sh->registers.aso_reg != REG_NON) { + DRV_LOG(DEBUG, "ASO register: REG_C%d", + mlx5_regc_index(sh->registers.aso_reg)); + } else { + DRV_LOG(DEBUG, "ASO register: NONE"); + } + mlx5_init_hws_flow_tags_registers(sh); +} + /** * Allocate shared device context. If there is multiport device the * master and representors will share this context, if there is single @@ -1720,6 +1794,7 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn, /* Add context to the global device list. */ LIST_INSERT_HEAD(&mlx5_dev_ctx_list, sh, next); rte_spinlock_init(&sh->geneve_tlv_opt_sl); + mlx5_init_shared_dev_registers(sh); exit: pthread_mutex_unlock(&mlx5_dev_ctx_list_mutex); return sh; diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 99a2ad88ed..a0dcd788b4 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -1380,6 +1380,12 @@ struct mlx5_dev_registers { enum modify_reg hw_avl_tags[MLX5_FLOW_HW_TAGS_MAX]; }; +#if defined(HAVE_MLX5DV_DR) && \ + (defined(HAVE_MLX5_DR_CREATE_ACTION_FLOW_METER) || \ + defined(HAVE_MLX5_DR_CREATE_ACTION_ASO)) +#define HAVE_MLX5_DR_CREATE_ACTION_ASO_EXT +#endif + /* * Shared Infiniband device context for Master/Representors * which belong to same IB device with multiple IB ports. diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index 643f9f4329..81ec8fd7f1 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -1730,9 +1730,6 @@ flow_hw_get_reg_id_from_ctx(void *dr_ctx, void flow_hw_set_port_info(struct rte_eth_dev *dev); void flow_hw_clear_port_info(struct rte_eth_dev *dev); - -void flow_hw_init_tags_set(struct rte_eth_dev *dev); - int flow_hw_create_vport_action(struct rte_eth_dev *dev); void flow_hw_destroy_vport_action(struct rte_eth_dev *dev); diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c index 1c79a92fb3..cccf7de13f 100644 --- a/drivers/net/mlx5/mlx5_flow_hw.c +++ b/drivers/net/mlx5/mlx5_flow_hw.c @@ -8449,40 +8449,6 @@ flow_hw_clear_port_info(struct rte_eth_dev *dev) info->is_wire = 0; } -/* - * Initialize the information of available tag registers and an intersection - * of all the probed devices' REG_C_Xs. - * PS. No port concept in steering part, right now it cannot be per port level. - * - * @param[in] dev - * Pointer to the rte_eth_dev structure. - */ -void flow_hw_init_tags_set(struct rte_eth_dev *dev) -{ - struct mlx5_dev_ctx_shared *sh = MLX5_SH(dev); - struct mlx5_dev_registers *reg = &sh->registers; - uint32_t meta_mode = sh->config.dv_xmeta_en; - uint8_t masks = (uint8_t)sh->cdev->config.hca_attr.set_reg_c; - uint8_t unset = 0; - uint32_t i, j; - - /* - * The CAPA is global for common device but only used in net. - * It is shared per eswitch domain. - */ - unset |= 1 << mlx5_regc_index(reg->aso_reg); - unset |= 1 << mlx5_regc_index(REG_C_6); - if (sh->config.dv_esw_en) - unset |= 1 << mlx5_regc_index(REG_C_0); - if (meta_mode == MLX5_XMETA_MODE_META32_HWS) - unset |= 1 << mlx5_regc_index(REG_C_1); - masks &= ~unset; - for (i = 0, j = 0; i < MLX5_FLOW_HW_TAGS_MAX; i++) { - if (!!((1 << i) & masks)) - reg->hw_avl_tags[j++] = mlx5_regc_value(i); - } -} - static int flow_hw_conntrack_destroy(struct rte_eth_dev *dev __rte_unused, uint32_t idx,