From patchwork Thu Feb 29 11:51:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dariusz Sosnowski X-Patchwork-Id: 137474 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 73EF943C35; Thu, 29 Feb 2024 12:53:21 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AAD2A42D45; Thu, 29 Feb 2024 12:52:50 +0100 (CET) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2087.outbound.protection.outlook.com [40.107.92.87]) by mails.dpdk.org (Postfix) with ESMTP id 8215942D35 for ; Thu, 29 Feb 2024 12:52:48 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bMESkfQwobV2fOc950sopIY2lxxcv6Ziv/gh2ApmBgTIX7jjfLL5eteZgpUA5j8qFg3NC4d+s/QVWB1CWWOs6SnR5Ckij9iPDG5/2RzcR47GlTLuIiGF8qp7MUBi4ul21sKVCOmsK3iv0i7irZkrkpwVRhB/4D0RIMQQuTgpirBxRoHhHeSmhLqLNHbN76OAuw+eReGK3E8pa2Jr+Jc47oguCgcN6a6kN1RarrrXc6Q/L7va1keq4DYtRk7sjS7/LsNNkTS2pcfr0YRBtwBauyyEdKdVBh5qRfzhNGjH89xlruZTMgsr09TRCB6kIAF5kmI6a93+wDIyhLimYT8K2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PznySkH1+SHta9eTtor8K6eGzzaDdNTL651VHjiZExY=; b=d4M2hk1tT0Lp8cLCkpBywL2HVsLpjj/IbE07Hxw6wdihF6BgbBUEIszsxReeUb6xgMc6PAQPEmbkVjaC+V/0j8lhPpKH/FZaV2mc0trTm4Q0ZDhbf9eZGEHUFdSX0JHfdhrxLrP5e9VziCMq8qYY1+9hV2zXheivQfOlZELpxBp07vf9vrmXAA2DPL0rKyEHf8TbZZQ3kRFF7Do/vo4h7G9A6chUzQ+z9TVbOHyqAv+KFAQM9Do3iLCaSiwUGM4okFSOgRcgp2QoTakTvfkw1D6RpKbEVsvYwUnipDD77dG9nMWbxCE/HntiksaPFecfiz+bCm9B+AAw9QlxaVQIwA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PznySkH1+SHta9eTtor8K6eGzzaDdNTL651VHjiZExY=; b=bINpnIXfSGyF4rpNl/iHTu2shxxAYqkHoI3RolFuD/Y9oa/mv7sSoSAgX8buNKF+YFkFX0AR/eS1MtJpNAvznEvUGR3gTvlDirZSvhhzoMMohhi5SD8cT4jWTXEIPe/1yi1NH/WQODsEB3OQmOEEvYckBXPOscAnPa8m3/Ubu9cmMJDMxGKB7DkEv9mr1CWEdsaw2zROi7TWPHWyYbaQbtz+EDOjzhnL39R5XfiMg7qh6SEv2DEo2ACDZjTlrkJy+41hIXL0KjmWRUmF91TxmGPihpzOuJtFq7H7o/i65UZndMHcn1UL667rBtvQtrlbftHrGGqIwq1o0pTPw6arFw== Received: from BYAPR07CA0085.namprd07.prod.outlook.com (2603:10b6:a03:12b::26) by LV2PR12MB6013.namprd12.prod.outlook.com (2603:10b6:408:171::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.39; Thu, 29 Feb 2024 11:52:46 +0000 Received: from CO1PEPF000044EF.namprd05.prod.outlook.com (2603:10b6:a03:12b:cafe::ac) by BYAPR07CA0085.outlook.office365.com (2603:10b6:a03:12b::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.49 via Frontend Transport; Thu, 29 Feb 2024 11:52:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CO1PEPF000044EF.mail.protection.outlook.com (10.167.241.69) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.25 via Frontend Transport; Thu, 29 Feb 2024 11:52:45 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 29 Feb 2024 03:52:26 -0800 Received: from nvidia.com (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.12; Thu, 29 Feb 2024 03:52:24 -0800 From: Dariusz Sosnowski To: Viacheslav Ovsiienko , Ori Kam , Suanming Mou , Matan Azrad CC: , Raslan Darawsheh , Bing Zhao Subject: [PATCH v2 07/11] net/mlx5: remove updated flow from job Date: Thu, 29 Feb 2024 12:51:52 +0100 Message-ID: <20240229115157.201671-8-dsosnowski@nvidia.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240229115157.201671-1-dsosnowski@nvidia.com> References: <20240228170046.176600-1-dsosnowski@nvidia.com> <20240229115157.201671-1-dsosnowski@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.230.35] X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044EF:EE_|LV2PR12MB6013:EE_ X-MS-Office365-Filtering-Correlation-Id: 28cc8df9-72d8-44fa-2225-08dc391cf1be X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: f97eSS64+EtlON5OHmv2naht6aqRE8rmr+hSEPxLueMwn2hoxnt9NcY/ktVFNPm+OjovyXMmBpPyk9zRYEVq/kj08zSm5hEkvJsXcUO08U4mj3SNgJRF8B4iPpdXweHqFIbcSqnhFta/gnqIWkE2De/4lK4it9TlZtVOGtg9Bqh3YoOiwhvftI2EGCQH0HltUQU4acYElD+kWYG6zHf30MBTJ8uHYx8It28961aulnzSZRL7ZbkxXpIIZG1zevCfAXBLVtfRmulUqgQg87EE2LrTcyAwo0GaWKjnbXgoNakje45ll47VvJt6cgXaZCsGklVqQtANZyuEaK8jP0nQyydVXJVvsBpNFEsxc6KiX8JXOC/fHWXjtlVjEvhVKGuSnF6o/93G9ldwxdk36QYmw0qh/f1Jd01kCV5Pwx9xZaroI4o+uX2yOyN6rozmNNAAuoFDn6Dz0M2zcditYqZsg0qNl2/3XsbN30vAJozEegqsL4q+tTaalOFjf3GGG/+bytyMBXT25jObssRpq66uzVBHf/zSK873tqLLiO8xihY7vStMIDMvjao6mLPOFUdgU31CCxzSyv9KknWh1C8MWtLlndCkwo+LE6ZY4s5iDnbJPXXuupiWWEuVCA4cNyuASQ9VspJBvkMlboBFYNWbXLdtv1fMfa5sNmzTccErX0ycsJ+U5iZEdRNEDqnlg7E4SHkJ3DWFuGYUWQngxzQZuUAs7ntAvevK0P/A4GAkrBV6kkPJ7/qlp6swprFA6Lfe X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230031)(82310400014)(36860700004); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Feb 2024 11:52:45.5529 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 28cc8df9-72d8-44fa-2225-08dc391cf1be X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044EF.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR12MB6013 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org mlx5_hw_q_job struct held a reference to a temporary flow rule struct, used during flow rule update operation. It serves as a container for flow actions data calculated during actions construction. After flow rule update operation succeeds, data from temporary flow rule is copied over to original flow rule. Although access to this temporary flow rule struct is required during both operation enqueue step and completion polling step, there can be only one ongoing flow update operation for a given flow rule. As a result there is no need to store it per job. This patch removes all references to temporary flow rule struct stored in mlx5_hw_q_job and removes relevant allocations to reduce job memory footprint. Temporary flow rule struct stored per job is replaced with: - If table is not resizable - An array of rte_flow_hw_aux structs, stored in template table. This array holds one entry per each flow rule, each containing a single mentioned temporary struct. - If table is resizable - Additional rte_flow_hw_aux struct, allocated alongside rte_flow_hw in resizable ipool. Signed-off-by: Dariusz Sosnowski Acked-by: Ori Kam --- drivers/net/mlx5/mlx5.h | 1 - drivers/net/mlx5/mlx5_flow.h | 7 +++ drivers/net/mlx5/mlx5_flow_hw.c | 100 ++++++++++++++++++++++++++------ 3 files changed, 89 insertions(+), 19 deletions(-) diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 1ca6223f95..2e2504f20f 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -407,7 +407,6 @@ struct mlx5_hw_q_job { /* Data extracted from hardware */ void *hw; } query; - struct rte_flow_hw *upd_flow; /* Flow with updated values. */ }; /* HW steering job descriptor LIFO pool. */ diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index 436d1391bc..a204f94624 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -1293,6 +1293,12 @@ struct rte_flow_hw { uint8_t rule[]; /* HWS layer data struct. */ } __rte_packed; +/** Auxiliary data stored per flow which is not required to be stored in main flow structure. */ +struct rte_flow_hw_aux { + /** Placeholder flow struct used during flow rule update operation. */ + struct rte_flow_hw upd_flow; +}; + #ifdef PEDANTIC #pragma GCC diagnostic error "-Wpedantic" #endif @@ -1601,6 +1607,7 @@ struct rte_flow_template_table { /* Action templates bind to the table. */ struct mlx5_hw_action_template ats[MLX5_HW_TBL_MAX_ACTION_TEMPLATE]; struct mlx5_indexed_pool *flow; /* The table's flow ipool. */ + struct rte_flow_hw_aux *flow_aux; /**< Auxiliary data stored per flow. */ struct mlx5_indexed_pool *resource; /* The table's resource ipool. */ struct mlx5_flow_template_table_cfg cfg; uint32_t type; /* Flow table type RX/TX/FDB. */ diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c index ab67dc139e..cbbf87b999 100644 --- a/drivers/net/mlx5/mlx5_flow_hw.c +++ b/drivers/net/mlx5/mlx5_flow_hw.c @@ -79,6 +79,66 @@ struct mlx5_indlst_legacy { #define MLX5_CONST_ENCAP_ITEM(encap_type, ptr) \ (((const struct encap_type *)(ptr))->definition) +/** + * Returns the size of a struct with a following layout: + * + * @code{.c} + * struct rte_flow_hw { + * // rte_flow_hw fields + * uint8_t rule[mlx5dr_rule_get_handle_size()]; + * }; + * @endcode + * + * Such struct is used as a basic container for HW Steering flow rule. + */ +static size_t +mlx5_flow_hw_entry_size(void) +{ + return sizeof(struct rte_flow_hw) + mlx5dr_rule_get_handle_size(); +} + +/** + * Returns the size of "auxed" rte_flow_hw structure which is assumed to be laid out as follows: + * + * @code{.c} + * struct { + * struct rte_flow_hw { + * // rte_flow_hw fields + * uint8_t rule[mlx5dr_rule_get_handle_size()]; + * } flow; + * struct rte_flow_hw_aux aux; + * }; + * @endcode + * + * Such struct is used whenever rte_flow_hw_aux cannot be allocated separately from the rte_flow_hw + * e.g., when table is resizable. + */ +static size_t +mlx5_flow_hw_auxed_entry_size(void) +{ + size_t rule_size = mlx5dr_rule_get_handle_size(); + + return sizeof(struct rte_flow_hw) + rule_size + sizeof(struct rte_flow_hw_aux); +} + +/** + * Returns a valid pointer to rte_flow_hw_aux associated with given rte_flow_hw + * depending on template table configuration. + */ +static __rte_always_inline struct rte_flow_hw_aux * +mlx5_flow_hw_aux(uint16_t port_id, struct rte_flow_hw *flow) +{ + struct rte_flow_template_table *table = flow->table; + + if (rte_flow_template_table_resizable(port_id, &table->cfg.attr)) { + size_t offset = sizeof(struct rte_flow_hw) + mlx5dr_rule_get_handle_size(); + + return RTE_PTR_ADD(flow, offset); + } else { + return &table->flow_aux[flow->idx - 1]; + } +} + static int mlx5_tbl_multi_pattern_process(struct rte_eth_dev *dev, struct rte_flow_template_table *tbl, @@ -3651,6 +3711,7 @@ flow_hw_async_flow_update(struct rte_eth_dev *dev, struct mlx5_flow_hw_action_params ap; struct rte_flow_hw *of = (struct rte_flow_hw *)flow; struct rte_flow_hw *nf; + struct rte_flow_hw_aux *aux; struct rte_flow_template_table *table = of->table; struct mlx5_hw_q_job *job = NULL; uint32_t res_idx = 0; @@ -3661,7 +3722,8 @@ flow_hw_async_flow_update(struct rte_eth_dev *dev, rte_errno = ENOMEM; goto error; } - nf = job->upd_flow; + aux = mlx5_flow_hw_aux(dev->data->port_id, of); + nf = &aux->upd_flow; memset(nf, 0, sizeof(struct rte_flow_hw)); rule_acts = flow_hw_get_dr_action_buffer(priv, table, action_template_index, queue); /* @@ -3708,11 +3770,8 @@ flow_hw_async_flow_update(struct rte_eth_dev *dev, rte_errno = EINVAL; goto error; } - /* - * Switch the old flow and the new flow. - */ + /* Switch to the old flow. New flow will retrieved from the table on completion. */ job->flow = of; - job->upd_flow = nf; ret = mlx5dr_rule_action_update((struct mlx5dr_rule *)of->rule, action_template_index, rule_acts, &rule_attr); if (likely(!ret)) @@ -3985,8 +4044,10 @@ hw_cmpl_flow_update_or_destroy(struct rte_eth_dev *dev, mlx5_ipool_free(table->flow, flow->idx); } } else { - rte_memcpy(flow, job->upd_flow, - offsetof(struct rte_flow_hw, rule)); + struct rte_flow_hw_aux *aux = mlx5_flow_hw_aux(dev->data->port_id, flow); + struct rte_flow_hw *upd_flow = &aux->upd_flow; + + rte_memcpy(flow, upd_flow, offsetof(struct rte_flow_hw, rule)); if (table->resource) mlx5_ipool_free(table->resource, res_idx); } @@ -4475,7 +4536,6 @@ flow_hw_table_create(struct rte_eth_dev *dev, .data = &flow_attr, }; struct mlx5_indexed_pool_config cfg = { - .size = sizeof(struct rte_flow_hw) + mlx5dr_rule_get_handle_size(), .trunk_size = 1 << 12, .per_core_cache = 1 << 13, .need_lock = 1, @@ -4496,6 +4556,9 @@ flow_hw_table_create(struct rte_eth_dev *dev, if (!attr->flow_attr.group) max_tpl = 1; cfg.max_idx = nb_flows; + cfg.size = !rte_flow_template_table_resizable(dev->data->port_id, attr) ? + mlx5_flow_hw_entry_size() : + mlx5_flow_hw_auxed_entry_size(); /* For table has very limited flows, disable cache. */ if (nb_flows < cfg.trunk_size) { cfg.per_core_cache = 0; @@ -4526,6 +4589,11 @@ flow_hw_table_create(struct rte_eth_dev *dev, tbl->flow = mlx5_ipool_create(&cfg); if (!tbl->flow) goto error; + /* Allocate table of auxiliary flow rule structs. */ + tbl->flow_aux = mlx5_malloc(MLX5_MEM_ZERO, sizeof(struct rte_flow_hw_aux) * nb_flows, + RTE_CACHE_LINE_SIZE, rte_dev_numa_node(dev->device)); + if (!tbl->flow_aux) + goto error; /* Register the flow group. */ ge = mlx5_hlist_register(priv->sh->groups, attr->flow_attr.group, &ctx); if (!ge) @@ -4646,6 +4714,8 @@ flow_hw_table_create(struct rte_eth_dev *dev, if (tbl->grp) mlx5_hlist_unregister(priv->sh->groups, &tbl->grp->entry); + if (tbl->flow_aux) + mlx5_free(tbl->flow_aux); if (tbl->flow) mlx5_ipool_destroy(tbl->flow); mlx5_free(tbl); @@ -4884,6 +4954,7 @@ flow_hw_table_destroy(struct rte_eth_dev *dev, mlx5_hlist_unregister(priv->sh->groups, &table->grp->entry); if (table->resource) mlx5_ipool_destroy(table->resource); + mlx5_free(table->flow_aux); mlx5_ipool_destroy(table->flow); mlx5_free(table); return 0; @@ -10122,8 +10193,7 @@ flow_hw_configure(struct rte_eth_dev *dev, goto err; } mem_size += (sizeof(struct mlx5_hw_q_job *) + - sizeof(struct mlx5_hw_q_job) + - sizeof(struct rte_flow_hw)) * _queue_attr[i]->size; + sizeof(struct mlx5_hw_q_job)) * _queue_attr[i]->size; } priv->hw_q = mlx5_malloc(MLX5_MEM_ZERO, mem_size, 64, SOCKET_ID_ANY); @@ -10132,23 +10202,17 @@ flow_hw_configure(struct rte_eth_dev *dev, goto err; } for (i = 0; i < nb_q_updated; i++) { - struct rte_flow_hw *upd_flow = NULL; - priv->hw_q[i].job_idx = _queue_attr[i]->size; priv->hw_q[i].size = _queue_attr[i]->size; if (i == 0) priv->hw_q[i].job = (struct mlx5_hw_q_job **) &priv->hw_q[nb_q_updated]; else - priv->hw_q[i].job = (struct mlx5_hw_q_job **) - &job[_queue_attr[i - 1]->size - 1].upd_flow[1]; + priv->hw_q[i].job = (struct mlx5_hw_q_job **)&job[_queue_attr[i - 1]->size]; job = (struct mlx5_hw_q_job *) &priv->hw_q[i].job[_queue_attr[i]->size]; - upd_flow = (struct rte_flow_hw *)&job[_queue_attr[i]->size]; - for (j = 0; j < _queue_attr[i]->size; j++) { - job[j].upd_flow = &upd_flow[j]; + for (j = 0; j < _queue_attr[i]->size; j++) priv->hw_q[i].job[j] = &job[j]; - } /* Notice ring name length is limited. */ priv->hw_q[i].indir_cq = mlx5_hwq_ring_create (dev->data->port_id, i, _queue_attr[i]->size, "indir_act_cq");