From patchwork Wed Nov 16 09:36:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suanming Mou X-Patchwork-Id: 119885 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7C1F5A0553; Wed, 16 Nov 2022 10:37:22 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1EB2240DFB; Wed, 16 Nov 2022 10:37:22 +0100 (CET) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2052.outbound.protection.outlook.com [40.107.244.52]) by mails.dpdk.org (Postfix) with ESMTP id E9459400EF for ; Wed, 16 Nov 2022 10:37:20 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=J0yTvm33C0Zgl2Jz8+FmyklajZAD3hJA6hFnmpzVahbknXZEM8SvUDuH3lbBmhWFQgyKstxVC2ZnUBS8SZC2LAxqEj4GON1fc/4Sfn0kbR+XOAaXk1nk63l83A3pDWlNB8f9/SgfM8gEP4jOgGPsAFjrM0t3IaVjp1DOFHka0GGPOSvkDcEHCE1bStjKjNGIE7VwL97SPdBJfX4rpG8XIhVNzirHcQIVns9SI/ppUFkdhmLkVfev03S+ruRE/9s74qwO4IzA/kSNH6+qjR/4huZIoi5MZz6y0rmkK0IvefW0dBabeIbr74wHS7yk1fy1RXYj+LhxMdp1MAHgNvNyIA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JaGExycoTdp49bvzXuaZampaI2CIqKIOOqUqCrcn2CU=; b=HScSpnHjmv+yP+OQsYC+p0ssjhNyJX5SCzYjSEASY1jx+bEh3SwCBVwsmaC0NAdgjiEmMyW58FPhV6cVxCGztNuH/3aav2PqneXIxcoKaC0EkMhrWRd00l9aSTYrhN8H7n998m9x38aGaRBFUsi0mlD7Wht6xok8JGUqN6hwufvThIBQHOsEruaMehuEjHKyU+Mx6t34RKDjfjNLLdRwsy6DjGkjnpkDLiBdovzP5GbxDOEpZtPXjRKq+H8APUDFCHQ5hHySlpK5s5UfZKRVT+Y8ul54aFRXbfTTDvtp1R/XKFYP0Ew2oM8LS/hmAhCYcUHfcQX5sGpj1XqUnxs6QA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JaGExycoTdp49bvzXuaZampaI2CIqKIOOqUqCrcn2CU=; b=lSRwML8uUVXHbNccXbsvzm91ZtliVFc04E2Gkr6d0OfG+X4ozb9Hy0NvfcQgRitx7pOnyOGLcIafHkXRfBmJ2mltu6jI+TIF6KOhmxfkZafYZtj1qsQheA7WfzmVr1eIVL4FHGyZTf3LhMII5YF2oPcvnIdmkq+fq0MQCxP8gwZMKgaVPCbpd6GHLPDX/45F6wwl1oS5gIxVJ70STlT6oRrFhxqn08MHlV2bPOkmsfhiFXHtKinrZAyF1GRoy7XjtDQT1QT/PGDD1/mYtWdJ+7IBF5cfJHJqm9SGqiIn/8LKrbqgfUm/roGMgGOGqhVWaNFeRJkYTlGGS85cOsHBhA== Received: from DM6PR04CA0025.namprd04.prod.outlook.com (2603:10b6:5:334::30) by PH8PR12MB6723.namprd12.prod.outlook.com (2603:10b6:510:1ce::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5813.18; Wed, 16 Nov 2022 09:37:19 +0000 Received: from DM6NAM11FT020.eop-nam11.prod.protection.outlook.com (2603:10b6:5:334:cafe::96) by DM6PR04CA0025.outlook.office365.com (2603:10b6:5:334::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5813.18 via Frontend Transport; Wed, 16 Nov 2022 09:37:18 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by DM6NAM11FT020.mail.protection.outlook.com (10.13.172.224) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.8 via Frontend Transport; Wed, 16 Nov 2022 09:37:18 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Wed, 16 Nov 2022 01:37:08 -0800 Received: from nvidia.com (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Wed, 16 Nov 2022 01:37:06 -0800 From: Suanming Mou To: Matan Azrad , Viacheslav Ovsiienko CC: , Subject: [PATCH] net/mlx5: fix GENEVE resource management Date: Wed, 16 Nov 2022 11:36:54 +0200 Message-ID: <20221116093654.19165-1-suanmingm@nvidia.com> X-Mailer: git-send-email 2.18.1 MIME-Version: 1.0 X-Originating-IP: [10.126.230.35] X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT020:EE_|PH8PR12MB6723:EE_ X-MS-Office365-Filtering-Correlation-Id: d3e1b483-9a6b-4c29-82b3-08dac7b6277a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: CmcHPTCn5FdE6OuE/H/lVuslCrLsHAjx3q9zoULpW20eurZ+hdshrGbOs73+YKfuNIyPyO0OkFhHu3IwK1D9nJU13baWf7S1qYACCIrKlBUCHYpl1oRxtukFlP63JhLj8fCNVMeubPmVZvz3ezXWMCv7sJ6a+QXGfM/fJ/OWd+rJo5sW031F//wtatozrQtdEu77p3jPdYsCdlBEXI2o/dDCCuS/XXy27YcJOBU2AU5OveiQTag2HaLtkyPIxC895y+8E3YHSUEvWukoe/tTEODQRYdAmxIWnyIVsJvvAa+de5keL0elkbO/c27AcMLItx7LH7X1iC6tbPQ6aH5coJF9CU+Zn5Hnx/XDAjS19ZlEqfdy9d3lwcycpD3oVpGHV6hFUg5Vw+/JXH+P1+oTD2k4qn2yUeL7AGLbCncEUZTLxqKn6jBbcUmZ1WdxOuhjFx/1BDaD2WNQDHYbDlobbs3t41STRNT9KvWZZb/1WY9DtgVZyDx1mLMCl+CAEIu64aI8gjCuWSVKLgM+MVrF/fA8Ayt3JfFvXMjfffo+Kuu1UPasf5qtZYxtE8wp3Hz1XCQaa+Z3F/u7pcRP8oX0k2d4+Wjxjgem9tHNAMK09QAYhwVw5f4YR+LI1hubyzixwHKmj0V8AvR8Zcy9XpoLhf5OqWTjiADy+pmlowcuJWKorrQON7eiPYWTDx5Hdg1pt3NAXtCFPRc5W+FfGV2ELNVuObvTHRayyFQth5iz4O1nu+lQbKGMb39gfm2oW8eN7DoIZkm2Ra9CSVHb0MDHng== X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(39860400002)(376002)(396003)(346002)(136003)(451199015)(40470700004)(46966006)(36840700001)(26005)(6286002)(86362001)(4326008)(70586007)(8676002)(70206006)(478600001)(1076003)(16526019)(5660300002)(7636003)(186003)(316002)(47076005)(356005)(426003)(41300700001)(2906002)(336012)(82740400003)(2616005)(8936002)(40460700003)(36756003)(83380400001)(6666004)(107886003)(36860700001)(55016003)(6636002)(7696005)(110136005)(54906003)(40480700001)(82310400005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Nov 2022 09:37:18.5099 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d3e1b483-9a6b-4c29-82b3-08dac7b6277a X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT020.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB6723 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The item translation split causes GENEVE TLV option resource register function flow_dev_geneve_tlv_option_resource_register() to be called twice incorrectly both in spec and mask translation. In SWS mode the refcnt will only be decreased by 1 in flow release. The refcnt will never be 0 again, it causes the resource be leaked. In HWS mode the resource is allocated as global, the refcnt should not be increased after the resource be allocated. And the resource should be released during PMD exists. This commit fixes GENEVE resource management. Fixes: 75a00812b18f ("net/mlx5: add hardware steering item translation") Fixes: cd4ab742064a ("net/mlx5: split flow item matcher and value translation") Signed-off-by: Suanming Mou Acked-by: Viacheslav Ovsiienko --- drivers/net/mlx5/mlx5.c | 8 +++++++- drivers/net/mlx5/mlx5_flow.h | 2 ++ drivers/net/mlx5/mlx5_flow_dv.c | 31 ++++++++++++++++++------------- 3 files changed, 27 insertions(+), 14 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index b3efdad293..6a0d66247a 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1757,7 +1757,13 @@ mlx5_free_shared_dev_ctx(struct mlx5_dev_ctx_shared *sh) } while (++i < sh->bond.n_port); if (sh->td) claim_zero(mlx5_devx_cmd_destroy(sh->td)); - MLX5_ASSERT(sh->geneve_tlv_option_resource == NULL); +#ifdef HAVE_MLX5_HWS_SUPPORT + /* HWS manages geneve_tlv_option resource as global. */ + if (sh->config.dv_flow_en == 2) + flow_dev_geneve_tlv_option_resource_release(sh); + else +#endif + MLX5_ASSERT(sh->geneve_tlv_option_resource == NULL); pthread_mutex_destroy(&sh->txpp.mutex); mlx5_lwm_unset(sh); mlx5_free(sh); diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index 94e0ac99b9..1f57ecd6e1 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -2484,6 +2484,8 @@ struct mlx5_aso_age_action *flow_aso_age_get_by_idx(struct rte_eth_dev *dev, int flow_dev_geneve_tlv_option_resource_register(struct rte_eth_dev *dev, const struct rte_flow_item *item, struct rte_flow_error *error); +void flow_dev_geneve_tlv_option_resource_release(struct mlx5_dev_ctx_shared *sh); + void flow_release_workspace(void *data); int mlx5_flow_os_init_workspace_once(void); void *mlx5_flow_os_get_specific_workspace(void); diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index bc9a75f225..a9357096f5 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -9493,9 +9493,13 @@ flow_dev_geneve_tlv_option_resource_register(struct rte_eth_dev *dev, geneve_opt_v->option_type && geneve_opt_resource->length == geneve_opt_v->option_len) { - /* We already have GENEVE TLV option obj allocated. */ - __atomic_fetch_add(&geneve_opt_resource->refcnt, 1, - __ATOMIC_RELAXED); + /* + * We already have GENEVE TLV option obj allocated. + * Increasing refcnt only in SWS. HWS uses it as global. + */ + if (priv->sh->config.dv_flow_en == 1) + __atomic_fetch_add(&geneve_opt_resource->refcnt, 1, + __ATOMIC_RELAXED); } else { ret = rte_flow_error_set(error, ENOMEM, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, @@ -9571,11 +9575,14 @@ flow_dv_translate_item_geneve_opt(struct rte_eth_dev *dev, void *key, return -1; MLX5_ITEM_UPDATE(item, key_type, geneve_opt_v, geneve_opt_m, &rte_flow_item_geneve_opt_mask); - ret = flow_dev_geneve_tlv_option_resource_register(dev, item, - error); - if (ret) { - DRV_LOG(ERR, "Failed to create geneve_tlv_obj"); - return ret; + /* Register resource requires item spec. */ + if (key_type & MLX5_SET_MATCHER_V) { + ret = flow_dev_geneve_tlv_option_resource_register(dev, item, + error); + if (ret) { + DRV_LOG(ERR, "Failed to create geneve_tlv_obj"); + return ret; + } } /* * Set the option length in GENEVE header if not requested. @@ -15226,11 +15233,9 @@ flow_dv_dest_array_resource_release(struct rte_eth_dev *dev, &resource->entry); } -static void -flow_dv_geneve_tlv_option_resource_release(struct rte_eth_dev *dev) +void +flow_dev_geneve_tlv_option_resource_release(struct mlx5_dev_ctx_shared *sh) { - struct mlx5_priv *priv = dev->data->dev_private; - struct mlx5_dev_ctx_shared *sh = priv->sh; struct mlx5_geneve_tlv_option_resource *geneve_opt_resource = sh->geneve_tlv_option_resource; rte_spinlock_lock(&sh->geneve_tlv_opt_sl); @@ -15318,7 +15323,7 @@ flow_dv_destroy(struct rte_eth_dev *dev, struct rte_flow *flow) else if (flow->age) flow_dv_aso_age_release(dev, flow->age); if (flow->geneve_tlv_option) { - flow_dv_geneve_tlv_option_resource_release(dev); + flow_dev_geneve_tlv_option_resource_release(priv->sh); flow->geneve_tlv_option = 0; } while (flow->dev_handles) {