From patchwork Mon Oct 31 16:08:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 119367 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CFAAEA0540; Mon, 31 Oct 2022 17:08:49 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4DE85410F2; Mon, 31 Oct 2022 17:08:44 +0100 (CET) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) by mails.dpdk.org (Postfix) with ESMTP id 3AD2040151; Mon, 31 Oct 2022 17:08:41 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=N014IjoOvLI13btTcWBluZoy0c+Yqr6I2NodAS1XjG8JBQUGpg6YL9D/KtKapLey84kNHQZAIEA+EaJSrx+ob4SAYYzlBa3FSU97IAeXsYIsejfTy9c7tNibuVnsyVr5yvN/sfbaglVuXnODAze0VdFCCOOVdPYW1JrXvXAbTNa/mTviECVcfdgYNjIAwE3LT0062JMQweO/Emz6MaTcTdgXyeibmjf45lhVO5whWB2WZIJLDVxNh16qkyjuRyxJpyRGx409ydi0zfwkgTfaElyMB9nl/GCYLenErdiME8XBI4FInE4ffcdlDJVwzLi36z1FGLesGeLHCVlIupm9+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vCP+cQ5BkgEEk4BT0OfU5sX6nIBRKj5sPRSn5o0DvLA=; b=CY9oCy0nTRHR9VZuEjkdqLk5ud/0aO9rb6hbVEkp3c6APYAz1clw0Ig8ESXo4YHDsE+Hcwx0X9ooiYTWC9vqAeTRFdZ4ESfoI/jkNp/yUdiMndmT3jpUucuJ8zL3DVn5Tc3HEoSADu7iwUSHfEdOd18f6TZCyBane1Y7zEh/dHdHy/FlRfEdxF4BCVRN2cKm0cNf8Hog+AysDytdFVk4m814OPyhsF7xtkZiQ0TEXIEZFlkCIx34lUpBMhom+u9F3YaXEMOhO2jqju6mMh1Ta0YVarufcuBlmlIMBIfr42fYHexLGwY9A1CVnLMa5yaAHbquwY/COml4Q4jW5FN6xw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vCP+cQ5BkgEEk4BT0OfU5sX6nIBRKj5sPRSn5o0DvLA=; b=I7sgxKwjqeGY97tnDOosxAG/35peY1Q7CQmMoBWifrTqtTfJ93/M3eEDG6pr0Z+4dhl9RAp8u3H/rfF2C8NHM9dz8/Z2HEO9STSPkJZacLBsqQmJfbeW/MRM3H+PjEfvanRhLWcUjIUXS+AyDMvy9fi8ktvNhEEF0zjwFrj7v2OAc25F9XofsTZcP0wS8+Mfn3ICVs4YKcnrGY+mQoOPn6QTkw2ybUUxbMfbr0Xw4pvkz2Hra6BChipwZ2anWRlwNENU50tqfXjMBIlbu3UcYge4uTdlZIRVbuUb38GPBbELIkDdqiNyIRZ/zCUCJU+VjD9MWV2VxbjmGpru7xHA8A== Received: from BN9PR03CA0937.namprd03.prod.outlook.com (2603:10b6:408:108::12) by MN2PR12MB4109.namprd12.prod.outlook.com (2603:10b6:208:1d9::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.16; Mon, 31 Oct 2022 16:08:39 +0000 Received: from BN8NAM11FT034.eop-nam11.prod.protection.outlook.com (2603:10b6:408:108:cafe::7d) by BN9PR03CA0937.outlook.office365.com (2603:10b6:408:108::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.19 via Frontend Transport; Mon, 31 Oct 2022 16:08:39 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by BN8NAM11FT034.mail.protection.outlook.com (10.13.176.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.14 via Frontend Transport; Mon, 31 Oct 2022 16:08:38 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Mon, 31 Oct 2022 09:08:31 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail201.nvidia.com (10.126.190.180) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Mon, 31 Oct 2022 09:08:31 -0700 Received: from nvidia.com (10.127.8.13) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29 via Frontend Transport; Mon, 31 Oct 2022 09:08:29 -0700 From: Michael Baum To: CC: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko , , Subject: [PATCH 1/5] net/mlx5: fix race condition in counter pool resizing Date: Mon, 31 Oct 2022 18:08:20 +0200 Message-ID: <20221031160824.330200-2-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221031160824.330200-1-michaelba@nvidia.com> References: <20221031160824.330200-1-michaelba@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT034:EE_|MN2PR12MB4109:EE_ X-MS-Office365-Filtering-Correlation-Id: 6adcbac8-55dc-4b7c-e893-08dabb5a2c46 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: K/tRsyRs8hyR689km1/3e4pmO069ZSkFXiHatKU+IseTLDHSYRT7vioeMCyEmHZubpGEf/A+0mif/BqAh+OKZ4wtTyHm4b1DfgTMKb4w3juTlbFAkagn+oMqXU8rSVbJu+xI8ea0uras3AS32p8tHuFZGpqBrcQFHgaZnZ+8GyHYi+rjFafMOOFrerZ5YxRKACmBu+ymU++dDopjJPM2Nd6sycHcMO9scI6LCSHjvPGP27yemsgBbc6EvlRLUJgxWt7FSxxmnpNL0u7Kkq+qqcbMkfVMmTnWxlYOrw1TdaANETOzaiaMo1md0m5X2TKkY6+JSDdLCsh95n2lxgf0x92aYKAQg7oXKejNcO3S5whIpSOyg0CMwp4AQXOTLkff3wLwLdaRLbMF2rMGHOx/CNyxibZx5aEZXrEHjee4I36dQoaOaIYpZzGsTlG4Lu6dsjGNlf7Ia7k1YKRdjkCFmd2wGOf5rOJZ6gp6FyZZFsBN6S9c5JZGTnrh/A9FY8EJRGY8z+Ww5lXMI+WUJnDdO3cSHDf1fQoXoLTpdtNbCs1t4oXAT+MH91ZQsr8L9HOW/J4n7r+mUlOy+YRVf5+T98SjuNKT2UkZC5zxQkpIG9UA8Qao+ZgKqwxLthuQEft++S61em454+7tUP2xhMUNQT/wKvrY8OH0Lh1IS4+sOMRUCS2WgvRs9Q4jY8lNyi7LuMmJ5GwECD6qZzT74EgO92zDilMnR9qtBsrrggbk0NqDiucHW3/WD2Q0Y69aZwrhDA2dcRpfQi7tTdYdqecqDA== X-Forefront-Antispam-Report: CIP:216.228.118.232; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge1.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(376002)(39860400002)(136003)(346002)(396003)(451199015)(40470700004)(46966006)(36840700001)(6666004)(6286002)(478600001)(26005)(316002)(54906003)(6916009)(7696005)(82310400005)(55016003)(36860700001)(7636003)(47076005)(40460700003)(40480700001)(336012)(356005)(36756003)(2616005)(82740400003)(1076003)(186003)(83380400001)(86362001)(426003)(2906002)(4326008)(70586007)(8676002)(70206006)(41300700001)(30864003)(450100002)(8936002)(5660300002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Oct 2022 16:08:38.8766 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6adcbac8-55dc-4b7c-e893-08dabb5a2c46 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.232]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT034.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4109 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Counter management structure has array of counter pools. This array is invalid in management structure initialization and grows on demand. The resizing include: 1. Allocate memory for the new size. 2. Copy the existing data to the new memory. 3. Move the pointer to the new memory. 4. Free the old memory. The third step can be performed before for this function, and compiler may do that, but another thread might read the pointer before coping and read invalid data or even crash. This patch allocates memory for this array once in management structure initialization and limit the counters number by 16M. Fixes: 3aa279157fa0 ("net/mlx5: synchronize flow counter pool creation") Cc: suanmingm@nvidia.com Cc: stable@dpdk.org Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5.c | 28 +++++++++++++--- drivers/net/mlx5/mlx5.h | 7 ++-- drivers/net/mlx5/mlx5_flow.c | 24 +++++++------- drivers/net/mlx5/mlx5_flow_dv.c | 53 +++++------------------------- drivers/net/mlx5/mlx5_flow_verbs.c | 23 +++---------- 5 files changed, 52 insertions(+), 83 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 78234b116c..b85a56ec24 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -561,18 +561,34 @@ mlx5_flow_counter_mode_config(struct rte_eth_dev *dev __rte_unused) * * @param[in] sh * Pointer to mlx5_dev_ctx_shared object to free + * + * @return + * 0 on success, otherwise negative errno value and rte_errno is set. */ -static void +static int mlx5_flow_counters_mng_init(struct mlx5_dev_ctx_shared *sh) { int i, j; if (sh->config.dv_flow_en < 2) { + void *pools; + + pools = mlx5_malloc(MLX5_MEM_ZERO, + sizeof(struct mlx5_flow_counter_pool *) * + MLX5_COUNTER_POOLS_MAX_NUM, + 0, SOCKET_ID_ANY); + if (!pools) { + DRV_LOG(ERR, + "Counter management allocation was failed."); + rte_errno = ENOMEM; + return -rte_errno; + } memset(&sh->sws_cmng, 0, sizeof(sh->sws_cmng)); TAILQ_INIT(&sh->sws_cmng.flow_counters); sh->sws_cmng.min_id = MLX5_CNT_BATCH_OFFSET; sh->sws_cmng.max_id = -1; sh->sws_cmng.last_pool_idx = POOL_IDX_INVALID; + sh->sws_cmng.pools = pools; rte_spinlock_init(&sh->sws_cmng.pool_update_sl); for (i = 0; i < MLX5_COUNTER_TYPE_MAX; i++) { TAILQ_INIT(&sh->sws_cmng.counters[i]); @@ -598,6 +614,7 @@ mlx5_flow_counters_mng_init(struct mlx5_dev_ctx_shared *sh) sh->hws_max_log_bulk_sz = log_dcs; sh->hws_max_nb_counters = max_nb_cnts; } + return 0; } /** @@ -655,8 +672,7 @@ mlx5_flow_counters_mng_close(struct mlx5_dev_ctx_shared *sh) claim_zero (mlx5_flow_os_destroy_flow_action (cnt->action)); - if (fallback && MLX5_POOL_GET_CNT - (pool, j)->dcs_when_free) + if (fallback && cnt->dcs_when_free) claim_zero(mlx5_devx_cmd_destroy (cnt->dcs_when_free)); } @@ -1572,8 +1588,12 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn, if (err) goto error; } + err = mlx5_flow_counters_mng_init(sh); + if (err) { + DRV_LOG(ERR, "Fail to initialize counters manage."); + goto error; + } mlx5_flow_aging_init(sh); - mlx5_flow_counters_mng_init(sh); mlx5_flow_ipool_create(sh); /* Add context to the global device list. */ LIST_INSERT_HEAD(&mlx5_dev_ctx_list, sh, next); diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index c9fcb71b69..cbe2d88b9e 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -386,11 +386,10 @@ struct mlx5_hw_q { } __rte_cache_aligned; - - +#define MLX5_COUNTER_POOLS_MAX_NUM (1 << 15) #define MLX5_COUNTERS_PER_POOL 512 #define MLX5_MAX_PENDING_QUERIES 4 -#define MLX5_CNT_CONTAINER_RESIZE 64 +#define MLX5_CNT_MR_ALLOC_BULK 64 #define MLX5_CNT_SHARED_OFFSET 0x80000000 #define IS_BATCH_CNT(cnt) (((cnt) & (MLX5_CNT_SHARED_OFFSET - 1)) >= \ MLX5_CNT_BATCH_OFFSET) @@ -549,7 +548,6 @@ TAILQ_HEAD(mlx5_counter_pools, mlx5_flow_counter_pool); /* Counter global management structure. */ struct mlx5_flow_counter_mng { volatile uint16_t n_valid; /* Number of valid pools. */ - uint16_t n; /* Number of pools. */ uint16_t last_pool_idx; /* Last used pool index */ int min_id; /* The minimum counter ID in the pools. */ int max_id; /* The maximum counter ID in the pools. */ @@ -621,6 +619,7 @@ struct mlx5_aso_age_action { }; #define MLX5_ASO_AGE_ACTIONS_PER_POOL 512 +#define MLX5_ASO_AGE_CONTAINER_RESIZE 64 struct mlx5_aso_age_pool { struct mlx5_devx_obj *flow_hit_aso_obj; diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 8e7d649d15..e25154199f 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -9063,7 +9063,7 @@ mlx5_flow_create_counter_stat_mem_mng(struct mlx5_dev_ctx_shared *sh) { struct mlx5_counter_stats_mem_mng *mem_mng; volatile struct flow_counter_stats *raw_data; - int raws_n = MLX5_CNT_CONTAINER_RESIZE + MLX5_MAX_PENDING_QUERIES; + int raws_n = MLX5_CNT_MR_ALLOC_BULK + MLX5_MAX_PENDING_QUERIES; int size = (sizeof(struct flow_counter_stats) * MLX5_COUNTERS_PER_POOL + sizeof(struct mlx5_counter_stats_raw)) * raws_n + @@ -9101,7 +9101,7 @@ mlx5_flow_create_counter_stat_mem_mng(struct mlx5_dev_ctx_shared *sh) } for (i = 0; i < MLX5_MAX_PENDING_QUERIES; ++i) LIST_INSERT_HEAD(&sh->sws_cmng.free_stat_raws, - mem_mng->raws + MLX5_CNT_CONTAINER_RESIZE + i, + mem_mng->raws + MLX5_CNT_MR_ALLOC_BULK + i, next); LIST_INSERT_HEAD(&sh->sws_cmng.mem_mngs, mem_mng, next); sh->sws_cmng.mem_mng = mem_mng; @@ -9125,14 +9125,13 @@ mlx5_flow_set_counter_stat_mem(struct mlx5_dev_ctx_shared *sh, { struct mlx5_flow_counter_mng *cmng = &sh->sws_cmng; /* Resize statistic memory once used out. */ - if (!(pool->index % MLX5_CNT_CONTAINER_RESIZE) && + if (!(pool->index % MLX5_CNT_MR_ALLOC_BULK) && mlx5_flow_create_counter_stat_mem_mng(sh)) { DRV_LOG(ERR, "Cannot resize counter stat mem."); return -1; } rte_spinlock_lock(&pool->sl); - pool->raw = cmng->mem_mng->raws + pool->index % - MLX5_CNT_CONTAINER_RESIZE; + pool->raw = cmng->mem_mng->raws + pool->index % MLX5_CNT_MR_ALLOC_BULK; rte_spinlock_unlock(&pool->sl); pool->raw_hw = NULL; return 0; @@ -9174,13 +9173,13 @@ void mlx5_flow_query_alarm(void *arg) { struct mlx5_dev_ctx_shared *sh = arg; - int ret; - uint16_t pool_index = sh->sws_cmng.pool_index; struct mlx5_flow_counter_mng *cmng = &sh->sws_cmng; + uint16_t pool_index = cmng->pool_index; struct mlx5_flow_counter_pool *pool; uint16_t n_valid; + int ret; - if (sh->sws_cmng.pending_queries >= MLX5_MAX_PENDING_QUERIES) + if (cmng->pending_queries >= MLX5_MAX_PENDING_QUERIES) goto set_alarm; rte_spinlock_lock(&cmng->pool_update_sl); pool = cmng->pools[pool_index]; @@ -9192,8 +9191,7 @@ mlx5_flow_query_alarm(void *arg) if (pool->raw_hw) /* There is a pool query in progress. */ goto set_alarm; - pool->raw_hw = - LIST_FIRST(&sh->sws_cmng.free_stat_raws); + pool->raw_hw = LIST_FIRST(&cmng->free_stat_raws); if (!pool->raw_hw) /* No free counter statistics raw memory. */ goto set_alarm; @@ -9219,12 +9217,12 @@ mlx5_flow_query_alarm(void *arg) goto set_alarm; } LIST_REMOVE(pool->raw_hw, next); - sh->sws_cmng.pending_queries++; + cmng->pending_queries++; pool_index++; if (pool_index >= n_valid) pool_index = 0; set_alarm: - sh->sws_cmng.pool_index = pool_index; + cmng->pool_index = pool_index; mlx5_set_query_alarm(sh); } @@ -9755,7 +9753,7 @@ mlx5_flow_dev_dump_sh_all(struct rte_eth_dev *dev, } /* get counter */ - MLX5_ASSERT(cmng->n_valid <= cmng->n); + MLX5_ASSERT(cmng->n_valid <= MLX5_COUNTER_POOLS_MAX_NUM); max = MLX5_COUNTERS_PER_POOL * cmng->n_valid; for (j = 1; j <= max; j++) { action = NULL; diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index 1e52278191..e77cbb862b 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -6091,7 +6091,7 @@ flow_dv_counter_get_by_idx(struct rte_eth_dev *dev, /* Decrease to original index and clear shared bit. */ idx = (idx - 1) & (MLX5_CNT_SHARED_OFFSET - 1); - MLX5_ASSERT(idx / MLX5_COUNTERS_PER_POOL < cmng->n); + MLX5_ASSERT(idx / MLX5_COUNTERS_PER_POOL < MLX5_COUNTER_POOLS_MAX_NUM); pool = cmng->pools[idx / MLX5_COUNTERS_PER_POOL]; MLX5_ASSERT(pool); if (ppool) @@ -6167,39 +6167,6 @@ flow_dv_find_pool_by_id(struct mlx5_flow_counter_mng *cmng, int id) return pool; } -/** - * Resize a counter container. - * - * @param[in] dev - * Pointer to the Ethernet device structure. - * - * @return - * 0 on success, otherwise negative errno value and rte_errno is set. - */ -static int -flow_dv_container_resize(struct rte_eth_dev *dev) -{ - struct mlx5_priv *priv = dev->data->dev_private; - struct mlx5_flow_counter_mng *cmng = &priv->sh->sws_cmng; - void *old_pools = cmng->pools; - uint32_t resize = cmng->n + MLX5_CNT_CONTAINER_RESIZE; - uint32_t mem_size = sizeof(struct mlx5_flow_counter_pool *) * resize; - void *pools = mlx5_malloc(MLX5_MEM_ZERO, mem_size, 0, SOCKET_ID_ANY); - - if (!pools) { - rte_errno = ENOMEM; - return -ENOMEM; - } - if (old_pools) - memcpy(pools, old_pools, cmng->n * - sizeof(struct mlx5_flow_counter_pool *)); - cmng->n = resize; - cmng->pools = pools; - if (old_pools) - mlx5_free(old_pools); - return 0; -} - /** * Query a devx flow counter. * @@ -6251,8 +6218,6 @@ _flow_dv_query_count(struct rte_eth_dev *dev, uint32_t counter, uint64_t *pkts, * The devX counter handle. * @param[in] age * Whether the pool is for counter that was allocated for aging. - * @param[in/out] cont_cur - * Pointer to the container pointer, it will be update in pool resize. * * @return * The pool container pointer on success, NULL otherwise and rte_errno is set. @@ -6264,9 +6229,14 @@ flow_dv_pool_create(struct rte_eth_dev *dev, struct mlx5_devx_obj *dcs, struct mlx5_priv *priv = dev->data->dev_private; struct mlx5_flow_counter_pool *pool; struct mlx5_flow_counter_mng *cmng = &priv->sh->sws_cmng; - bool fallback = priv->sh->sws_cmng.counter_fallback; + bool fallback = cmng->counter_fallback; uint32_t size = sizeof(*pool); + if (cmng->n_valid == MLX5_COUNTER_POOLS_MAX_NUM) { + DRV_LOG(ERR, "All counter is in used, try again later."); + rte_errno = EAGAIN; + return NULL; + } size += MLX5_COUNTERS_PER_POOL * MLX5_CNT_SIZE; size += (!age ? 0 : MLX5_COUNTERS_PER_POOL * MLX5_AGE_SIZE); pool = mlx5_malloc(MLX5_MEM_ZERO, size, 0, SOCKET_ID_ANY); @@ -6285,11 +6255,6 @@ flow_dv_pool_create(struct rte_eth_dev *dev, struct mlx5_devx_obj *dcs, pool->time_of_last_age_check = MLX5_CURR_TIME_SEC; rte_spinlock_lock(&cmng->pool_update_sl); pool->index = cmng->n_valid; - if (pool->index == cmng->n && flow_dv_container_resize(dev)) { - mlx5_free(pool); - rte_spinlock_unlock(&cmng->pool_update_sl); - return NULL; - } cmng->pools[pool->index] = pool; cmng->n_valid++; if (unlikely(fallback)) { @@ -12511,7 +12476,7 @@ flow_dv_aso_age_release(struct rte_eth_dev *dev, uint32_t age_idx) } /** - * Resize the ASO age pools array by MLX5_CNT_CONTAINER_RESIZE pools. + * Resize the ASO age pools array by MLX5_ASO_AGE_CONTAINER_RESIZE pools. * * @param[in] dev * Pointer to the Ethernet device structure. @@ -12525,7 +12490,7 @@ flow_dv_aso_age_pools_resize(struct rte_eth_dev *dev) struct mlx5_priv *priv = dev->data->dev_private; struct mlx5_aso_age_mng *mng = priv->sh->aso_age_mng; void *old_pools = mng->pools; - uint32_t resize = mng->n + MLX5_CNT_CONTAINER_RESIZE; + uint32_t resize = mng->n + MLX5_ASO_AGE_CONTAINER_RESIZE; uint32_t mem_size = sizeof(struct mlx5_aso_age_pool *) * resize; void *pools = mlx5_malloc(MLX5_MEM_ZERO, mem_size, 0, SOCKET_ID_ANY); diff --git a/drivers/net/mlx5/mlx5_flow_verbs.c b/drivers/net/mlx5/mlx5_flow_verbs.c index 81a33ddf09..4bca685674 100644 --- a/drivers/net/mlx5/mlx5_flow_verbs.c +++ b/drivers/net/mlx5/mlx5_flow_verbs.c @@ -232,27 +232,14 @@ flow_verbs_counter_new(struct rte_eth_dev *dev, uint32_t id __rte_unused) break; } if (!cnt) { - struct mlx5_flow_counter_pool **pools; uint32_t size; - if (n_valid == cmng->n) { - /* Resize the container pool array. */ - size = sizeof(struct mlx5_flow_counter_pool *) * - (n_valid + MLX5_CNT_CONTAINER_RESIZE); - pools = mlx5_malloc(MLX5_MEM_ZERO, size, 0, - SOCKET_ID_ANY); - if (!pools) - return 0; - if (n_valid) { - memcpy(pools, cmng->pools, - sizeof(struct mlx5_flow_counter_pool *) * - n_valid); - mlx5_free(cmng->pools); - } - cmng->pools = pools; - cmng->n += MLX5_CNT_CONTAINER_RESIZE; + if (n_valid == MLX5_COUNTER_POOLS_MAX_NUM) { + DRV_LOG(ERR, "All counter is in used, try again later."); + rte_errno = EAGAIN; + return 0; } - /* Allocate memory for new pool*/ + /* Allocate memory for new pool */ size = sizeof(*pool) + sizeof(*cnt) * MLX5_COUNTERS_PER_POOL; pool = mlx5_malloc(MLX5_MEM_ZERO, size, 0, SOCKET_ID_ANY); if (!pool) From patchwork Mon Oct 31 16:08:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 119368 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DDB90A0540; Mon, 31 Oct 2022 17:08:57 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 44B9240E5A; Mon, 31 Oct 2022 17:08:48 +0100 (CET) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2056.outbound.protection.outlook.com [40.107.243.56]) by mails.dpdk.org (Postfix) with ESMTP id A8B6F40F16 for ; Mon, 31 Oct 2022 17:08:44 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Hb5DoBocvbyJiEKi82PFbQhj+JRm4zXKC74xnhUkvmA99DMSSgkLi/31mnKQ3qIU59QktVPTS1RhbPjzWAwlafaoUUrbHbWUgucPtijf830Pd7S08Knfjx/MlXjTXuOZjuiWQdl5dSrsBQ9+DpSwOnlL/lFpa3+nt2aFHDNtqYlRGUlK7L6KR51Sjx7f2J/N/iRZoX01AQqD9b0MiB/jdB/xmb03CY/psjgfeQphA+z2uBAhLZtxeeOVMqDMZEx+mVYP6NHuYA3bKKL421aPVcF06X0w0Zyn2YfZ189X5wL6kpyzON+V0qLIYU/aHi3ZCLywJWdE/+S6DdiogCLfZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/XWAFrDYc+4/aEhFbxOwax9JVPcyAR6zylNos/NOKsg=; b=OLqww2PTVK/GHiGUHm/Go1/K/kttAHnjIoh8dSAd0pTZBAs5laQP3wHYb+E3ca+knvT477pEXS4+RV+F5Fo/TuOt99kzj/cD6VtUHdypjdV6eDWuh1HZq7nhcNYOFxF1ZKiZdpM5ylq81LpDzxLeLXb9lD8UG29NTzpu47e6zMP0+xSUhMx9rYwel9xE8L7WsQFdBmO4t2sRlTZFe6Q5TAUckIUxbNhxsaP3AxME2hEe4a4yRFuJKW4ng8wK3Z5rxSRp+tB/jFEMRyvbQ59p9weliMIzM3/tj6etyvdFndz2zySGKZebYo5QDQMwSB8MIqeZESB2A/vXaX7LF0iWcw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/XWAFrDYc+4/aEhFbxOwax9JVPcyAR6zylNos/NOKsg=; b=MZAnPFBB8HX/bdh8+62VGIFwlcMorFsVW5M9z1FQNgXcOaRfAxy3yN0fcz8N+P4ReUjXUBxE5tsxgzHtKYg6UF5vjr+zvjR30cKdHcfQ5v5C7Y5t7oDM9Q3b0WY5AkieLlUFpVZEYugTptBpNjL51aaNl8hPVVFsqJAN42gxoPrMF880jYjj+DPGfAElnpTN1vvLV15qLj3/QmCR0TBBhvX5AY9oBFq4QvmDmcg5IGCvINl26eZawATeFg9GAUjY7laR/4+xP/3ji0S0G1224pOHOvt4fWx279w3it4o1WRBIWRejBard2HP2nbbcMy1y0MgPHUOaenuGXIYJB0MVw== Received: from BN9PR03CA0174.namprd03.prod.outlook.com (2603:10b6:408:f4::29) by CH2PR12MB4328.namprd12.prod.outlook.com (2603:10b6:610:a6::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.19; Mon, 31 Oct 2022 16:08:43 +0000 Received: from BN8NAM11FT010.eop-nam11.prod.protection.outlook.com (2603:10b6:408:f4:cafe::ca) by BN9PR03CA0174.outlook.office365.com (2603:10b6:408:f4::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.19 via Frontend Transport; Mon, 31 Oct 2022 16:08:42 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by BN8NAM11FT010.mail.protection.outlook.com (10.13.177.53) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.14 via Frontend Transport; Mon, 31 Oct 2022 16:08:42 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Mon, 31 Oct 2022 09:08:33 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail201.nvidia.com (10.126.190.180) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Mon, 31 Oct 2022 09:08:33 -0700 Received: from nvidia.com (10.127.8.13) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29 via Frontend Transport; Mon, 31 Oct 2022 09:08:31 -0700 From: Michael Baum To: CC: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko , Subject: [PATCH 2/5] net/mlx5: fix accessing the wrong counter Date: Mon, 31 Oct 2022 18:08:21 +0200 Message-ID: <20221031160824.330200-3-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221031160824.330200-1-michaelba@nvidia.com> References: <20221031160824.330200-1-michaelba@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT010:EE_|CH2PR12MB4328:EE_ X-MS-Office365-Filtering-Correlation-Id: 01f14294-2aea-46ed-1904-08dabb5a2e5d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: t9Bu7QX6VbP2+ouREAu40IJh4oWntLyq5zwVEm+SqyT5ILy55B0lpxdVekg9zKmIxJzLVuEP2RJf5qulqP5Go2LngpQsnUrPt/unR36Gcv8wryUEUeifgeAQV7N38AFTv/FL5D1zqZsd8T1bEo/Dnz+vlVhS5ZLLOmNsNQmvKAL9Ax0F+pzS5iYJJooHLdeOyzeFVTPfUjDNGStyVNZrwTPA0sLOsLsjR9HFZH76CDHhI9xgwIZ3QzIaofPBFcxK+qPr6sLFPmCtbG63U8rK9IqrgCWcLqfrQL3irbziN5FDDW/f5QgMVLDMwQNO37gV4AOzWjvootgeQmR4xTu6VifBOp/abky0/YBDieFdJQST+R5p12obP+c/Yk0vkznqZNGhldKFos2MWyG7tGktu9rBjSFRuF5YMswpSc+OIErHczRUR8jPulioN53bT+nqNO0YdMpeEEupcPa1acpoPWAf3kovyOMlteCkCaa1fs4Bos9KnZIWizDjn+1krsl/cdxuqcVmZHjSRwah23M/AQNAN1OYPlXP7JsgWld6IKXYpu0ww6ZA/O2I5NwA1eP8ksvkDWtRJOsQxGk/k+nOd67p7tV7NFOul7apagRCrZ/Z+KONhfC+nY29LsrWEwBwK2/ZavArXgDZyQ9Gp6E80k7Ajg+66ykQ716mFvfVt24kIu22/y6//tZJP9/iYnVGUM3EF4CMuMtwF/QEVWQkGk6Thx7P8hwTAaD4c0VnCJlHBxj56Vdy9BdGsff35VfaazNnP9ytYKD8ZEycNHxqrw== X-Forefront-Antispam-Report: CIP:216.228.118.233; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge2.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(396003)(39860400002)(346002)(136003)(376002)(451199015)(46966006)(36840700001)(40470700004)(36860700001)(2616005)(7696005)(6286002)(478600001)(26005)(83380400001)(47076005)(107886003)(186003)(1076003)(336012)(6666004)(426003)(2906002)(40480700001)(40460700003)(55016003)(54906003)(82310400005)(316002)(8676002)(70206006)(70586007)(4326008)(8936002)(5660300002)(6916009)(41300700001)(86362001)(36756003)(82740400003)(356005)(7636003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Oct 2022 16:08:42.3775 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 01f14294-2aea-46ed-1904-08dabb5a2e5d X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.233]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT010.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4328 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The HWS counter has 2 different identifiers: 1. Type "cnt_id_t" which represents the counter inside caches and in the flow structure. This index cannot be zero and is mostly called "cnt_id". 2. Internal index, the index in counters array with type "uint32_t". mostly it is called "iidx". The second ID is calculated from the first using "mlx5_hws_cnt_iidx()" function. When a direct counter is allocated, if the queue cache is not empty, the counter represented by cnt_id is popped from the cache. This counter may be invalid according to the query_gen field. Thus, the "iidx" is parsed from cnt_id and if it is valid, it is used to update the fields of the counter structure. When this counter is invalid, all the cache is flashed and new counters are fetched into the cache. After fetching, another counter represented by cnt_id is taken from the cache. Unfortunately, for updating fields like "in_used" or "age_idx", the function wrongly may use the old "iidx" coming from an invalid cnt_id. Update the "iidx" in case of an invalid counter popped from the cache. Fixes: 4d368e1da3a4 ("net/mlx5: support flow counter action for HWS") Cc: jackmin@nvidia.com Signed-off-by: Michael Baum Acked-by: Matan Azrad Acked-by: Xiaoyu Min --- drivers/net/mlx5/mlx5_hws_cnt.h | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/net/mlx5/mlx5_hws_cnt.h b/drivers/net/mlx5/mlx5_hws_cnt.h index e311923f71..196604aded 100644 --- a/drivers/net/mlx5/mlx5_hws_cnt.h +++ b/drivers/net/mlx5/mlx5_hws_cnt.h @@ -506,6 +506,7 @@ mlx5_hws_cnt_pool_get(struct mlx5_hws_cnt_pool *cpool, uint32_t *queue, rte_ring_dequeue_zc_burst_elem_start(qcache, sizeof(cnt_id_t), 1, &zcdc, NULL); *cnt_id = *(cnt_id_t *)zcdc.ptr1; + iidx = mlx5_hws_cnt_iidx(cpool, *cnt_id); } __hws_cnt_query_raw(cpool, *cnt_id, &cpool->pool[iidx].reset.hits, &cpool->pool[iidx].reset.bytes); From patchwork Mon Oct 31 16:08:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 119369 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D4313A0540; Mon, 31 Oct 2022 17:09:04 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 12D494161A; Mon, 31 Oct 2022 17:08:51 +0100 (CET) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2089.outbound.protection.outlook.com [40.107.243.89]) by mails.dpdk.org (Postfix) with ESMTP id 7E37340E5A for ; Mon, 31 Oct 2022 17:08:47 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nBAtNmzUEcJrjfUTJPykqkZ8fDtMaU6O+km+ltuzfrpRh3KZspH10TbPu+t6hvoMwfLpV/ZQ5V1JDk2awxVDzICtmkyUQ84w1Fwo1JEHXSj423c2hMahGrF1l+dMFeUlZS73cMYW1kWhn86jJEXqt3f52uLVGdXem9GTt0Fw/A7U1GPcpAkpzTzISDRb0kwk5OcKScBtGZQwRNgnvQfGairtHtGZIt9ss+CdnVMvbF24m33pDWQCpBXqCGIKYXZU49XnAcsm+3EDu52I1GdgKyz/r0a8juvxcJdbD5taRW1yaWnKyH/f4BGlRPJifYg7BFGT7pXa/jr7biiZ80eJYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rlv5wSRGF7mZxCT3q9aqbcze4d0l6xfumaRBI8azqVI=; b=GilJ6Uo+fMGgCYQsstKo2/tl1DwkVHa9R9e0thiT8VJDXyDxN86QzB0owh3cKSkc2ZvU7yotJ3VKyNFMGGMrlTuuqoKhbfI/IFgcQc0QDtkzRlRANmaCqTbIMQafPRliKb/Ps2Sk8IUsmAU+fBTsajl4ypOb3IDeor4PjN8pTi50CRFgW6nazkSHl5U5xzzQsI67waGhhutXin3OKtOkFj6X2lw+kfSEDrHvy+N7JtzSqD/YpGYRE58FMGKxAQIqeQB3KBVn6f+LelGPKLwARKExbPw0X/m2/YUQcCJ5dlK0TMNRMRmqx/DGYcYxDAvqAQH1AVJyOCXos5JU4p9KTA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rlv5wSRGF7mZxCT3q9aqbcze4d0l6xfumaRBI8azqVI=; b=gdGBALTNKcNC43dZ3MOpghazphd5WjYksGrg0xKXxS3Yjt8G1FbKngv+e7yPDZSyZx7OEgZHhi0C+TKmFFe+SQA3KsiFUI75oZSm3yk36x6NqGKOtaov8MziPS5LpyZUOApXVfNgOFABeMG7QrKhhEnXeYshcRpjzzgbq2FNEc+O7ArOjZ+akC11h8itgYU8A0iKh8G1tpdkTEN7GH2BZ4XLx8XSrAzsuGZQuo1R4mEsSU5v7Js6HHk8DqKIj5HRHH90RCg+BzfXZ+IVaQK6igQDBPOLZWG23GckuFYIg3r2SiyhDpydRmpyHMujIFsntl00OEN3c5qIjg61Du/HgA== Received: from BN0PR02CA0049.namprd02.prod.outlook.com (2603:10b6:408:e5::24) by DS7PR12MB6094.namprd12.prod.outlook.com (2603:10b6:8:9d::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.15; Mon, 31 Oct 2022 16:08:45 +0000 Received: from BN8NAM11FT081.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e5:cafe::1) by BN0PR02CA0049.outlook.office365.com (2603:10b6:408:e5::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.19 via Frontend Transport; Mon, 31 Oct 2022 16:08:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by BN8NAM11FT081.mail.protection.outlook.com (10.13.177.233) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.14 via Frontend Transport; Mon, 31 Oct 2022 16:08:45 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Mon, 31 Oct 2022 09:08:35 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Mon, 31 Oct 2022 09:08:35 -0700 Received: from nvidia.com (10.127.8.13) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29 via Frontend Transport; Mon, 31 Oct 2022 09:08:33 -0700 From: Michael Baum To: CC: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko , Subject: [PATCH 3/5] net/mlx5: fix missing counter elements copies in r2r cases Date: Mon, 31 Oct 2022 18:08:22 +0200 Message-ID: <20221031160824.330200-4-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221031160824.330200-1-michaelba@nvidia.com> References: <20221031160824.330200-1-michaelba@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT081:EE_|DS7PR12MB6094:EE_ X-MS-Office365-Filtering-Correlation-Id: a5be50db-9aa6-4a51-6e8f-08dabb5a3038 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ct5blpE+Aa3pTk7YDngb5e7EkK40rxXomzyXWDWaGEWTk3GA3bnIkoUtAqysg+vr9bqRty4NCFMUCLMoSW+BmyO3mr9hB0ra/PGLuBVAlDrcN2IL3sFg0DH1Fe6vyNvQrIQv5AyyJgO0fhFM+1ZL3sV/Xyzf2bYr49wJ/xWnS2MLqnde8y2rLSgVohoYz7tFO8itli/dWIa7OQsSDZg/OzKep3EzIXBaicgWKrbFFxFK7RZjftQPCrihA71ZF79pvPdpvYzXtz5jmrhTKNXkxGjwsfS8UkSI8gCWVZiEDFlQI+LNYKziAYG4qn7+6qTYQonKcxdaubV/eCfJ/5qQpIZYdSaCkE/8iLvEYXGF5a5H5koviObz5EfA2g80/+NqRopei18S9HT+GY0T2NbYpDSBDBqVjsmCYHL7XXrC/zjbvdRlcyJ92+4r5851BbgdE5Wl014dY3tg2vYtwpCcqsm8Ad97ZLEyY7y5KMkoPHCD4hKRp9nIAGJ4R0+ikLyjxK3zmtlSssxvm+ZAXyCiG2qic0LW9KPLLMWSESNAsWdNvqrIGv+kuW017oOxJ17s7WxcRlrYz4oS313aHFiHiUMq8583qbC3Aq5RyGdOAly+AW/dODX2MVRNbItCOH2LAFpc9wC5wYjDw7OHCGF9cIwQqswAfza9d4+keA0Xrq7BHW0+I8h8+mWLvZZOYDup5VbofK9EQcjXacHQY73GJmMqZL+D4SULntwfQEz9BstneVSIujlHnGCMqycL40z4bDrcdkCehAnsWNVS0mA+0A== X-Forefront-Antispam-Report: CIP:216.228.118.232; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge1.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(136003)(39860400002)(396003)(346002)(376002)(451199015)(36840700001)(46966006)(40470700004)(6666004)(426003)(47076005)(7636003)(36860700001)(40460700003)(356005)(82740400003)(86362001)(2906002)(5660300002)(8676002)(70206006)(41300700001)(4326008)(2616005)(6286002)(82310400005)(70586007)(107886003)(7696005)(54906003)(186003)(336012)(1076003)(478600001)(316002)(8936002)(6916009)(26005)(55016003)(40480700001)(83380400001)(36756003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Oct 2022 16:08:45.4927 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a5be50db-9aa6-4a51-6e8f-08dabb5a3038 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.232]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT081.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB6094 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The __hws_cnt_r2rcpy() function copies elements from one zero-copy ring to another zero-copy ring in place. This routine needs to consider the situation that the address was given by source and destination could be both wrapped. It uses 4 different "n" local variables to manage it: - n: Number of elements to copy in total. - n1: Number of elements to copy from ptr1, it is the minimal value from source/dest n1 field. - n2: Number of elements to copy from src->ptr1 to dst->ptr2 or from src->ptr2 to dst->ptr1, this variable is 0 when both source and dest n1 field are equal. - n3: Number of elements to copy from src->ptr2 to dst->ptr2. The function copies the first n1 elements. If n2 isn't zero it copies more elements and check whether n3 is zero. This logic is wrong since n3 may be bigger than zero even when n2 is zero. This scenario is commonly happening in counters when the internal mlx5 service thread copies elements from the reset ring into the reuse ring. This patch changes the function to copy n3 regardless of n2 value. Fixes: 4d368e1da3a4 ("net/mlx5: support flow counter action for HWS") Cc: jackmin@nvidia.com Signed-off-by: Michael Baum Acked-by: Matan Azrad Acked-by: Xiaoyu Min --- drivers/net/mlx5/mlx5_hws_cnt.h | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/drivers/net/mlx5/mlx5_hws_cnt.h b/drivers/net/mlx5/mlx5_hws_cnt.h index 196604aded..6e371f1929 100644 --- a/drivers/net/mlx5/mlx5_hws_cnt.h +++ b/drivers/net/mlx5/mlx5_hws_cnt.h @@ -281,11 +281,10 @@ __hws_cnt_r2rcpy(struct rte_ring_zc_data *zcdd, struct rte_ring_zc_data *zcds, d3 = zcdd->ptr2; } memcpy(d1, s1, n1 * sizeof(cnt_id_t)); - if (n2 != 0) { + if (n2 != 0) memcpy(d2, s2, n2 * sizeof(cnt_id_t)); - if (n3 != 0) - memcpy(d3, s3, n3 * sizeof(cnt_id_t)); - } + if (n3 != 0) + memcpy(d3, s3, n3 * sizeof(cnt_id_t)); } static __rte_always_inline int From patchwork Mon Oct 31 16:08:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 119370 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B3E2EA0540; Mon, 31 Oct 2022 17:09:10 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id EA31D427ED; Mon, 31 Oct 2022 17:08:52 +0100 (CET) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) by mails.dpdk.org (Postfix) with ESMTP id 2D561427EA for ; Mon, 31 Oct 2022 17:08:51 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eviymSqNKw/sS/yT8z0o8e4MG148TS7PEJe+BT27Pk9Hf93JBUO0iRTx9WgDwmUQP0OAX4dox1oijG8WbM2pNDwmG6CMtrUYA5l6ReXrwsljX8r3cf1m0wgNFfWU1rtFRVmAKXLR2kYyp4bPdlpjExbvJPerj5GCA1HHR5fc6b9yAO/m1zcc31nne+0LnMn/8RZB2s6W4lTy4J7EPOnBtTOjnAdBjessH+/mKDr0y1KINW9X0+kpE9cxOHOG0vsg9JdjIcDf5HHCFvhr3hMVgiVcCD7Ubi+mYsGcjzifiDrqQq1AgDoz2HJcrZKO3bGkdNlUBr7KxRJHXMAPLwhdbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ih5ZgqWjqLg8Zmhc5NqbUthU6YqEM/d6Zhyg8BiwLGI=; b=mZmJHtBWpf5XElcwZlTlpY78ilgOGQ2bJjwE4g3VyEE1soILmdpmJk7sH91BF2hrCwIn6uvv2WT8436CFC9gFQDpRC+bEZCwk8uSNrD0WUpzSQa5hQ7Pp3TT7IA9QIKNkJzu3wm9LlnVZ0r85q7qUt26hWYycdlet+8xZzkfBQWrk8/jOSTjTyZj3QRT8eKlJgJizXBtEUwimGlHf62YEAlzAwE04C0MFdHdoCMYrqa6G4CRT/yhVEgnhfHpHX+ytUIMk0vBILCJo8/f2Ptn29k3/F7urx2pkyu36/OachL1USGPhUuwhOvobCi6t+IXtENWgrNssYlZPy7OZYUd+w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ih5ZgqWjqLg8Zmhc5NqbUthU6YqEM/d6Zhyg8BiwLGI=; b=H0scpUsNXBLQaZcpAbmGwfxtDG7AyJDQMtyLz7tOIS2ufubIB7mDIwTC5Xbxl70p2ZRcpkxLvd0JtVUztxaXb1eLMPScKYIMKgJZRpQC+7LujIrCqXOnw0PDcr3KZVUzoEExl5iZb42lxvJq45A8KR2/i9qgJZr6xEGV8PIO0A+HmWHnXEa09IREe+hz/XOD0b1UfGNw+IJcBzssQj+fw+j1VCzY1fnyLvr37erSsLqgDagYBOkHNGuMSeGTSccG4a++8us+TRiTNtpeURh0OqIN6u3sYoKLIkCmQBNDHklUsBHKysTNLX/k/tPgiHRgAa6hBl73fGJ+hr90eUOXBw== Received: from BN0PR04CA0011.namprd04.prod.outlook.com (2603:10b6:408:ee::16) by DM8PR12MB5414.namprd12.prod.outlook.com (2603:10b6:8:3e::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.16; Mon, 31 Oct 2022 16:08:47 +0000 Received: from BN8NAM11FT069.eop-nam11.prod.protection.outlook.com (2603:10b6:408:ee:cafe::ae) by BN0PR04CA0011.outlook.office365.com (2603:10b6:408:ee::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.19 via Frontend Transport; Mon, 31 Oct 2022 16:08:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by BN8NAM11FT069.mail.protection.outlook.com (10.13.176.152) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.14 via Frontend Transport; Mon, 31 Oct 2022 16:08:47 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Mon, 31 Oct 2022 09:08:37 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail201.nvidia.com (10.126.190.180) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Mon, 31 Oct 2022 09:08:36 -0700 Received: from nvidia.com (10.127.8.13) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29 via Frontend Transport; Mon, 31 Oct 2022 09:08:35 -0700 From: Michael Baum To: CC: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko , Xiaoyu Min Subject: [PATCH 4/5] net/mlx5: add assertions in counter get/put Date: Mon, 31 Oct 2022 18:08:23 +0200 Message-ID: <20221031160824.330200-5-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221031160824.330200-1-michaelba@nvidia.com> References: <20221031160824.330200-1-michaelba@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT069:EE_|DM8PR12MB5414:EE_ X-MS-Office365-Filtering-Correlation-Id: 4e62456c-a87b-40a2-47c4-08dabb5a3171 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: oDIHOT98HHnV7zrJvjVk82ujezIqJzODd6mQYo7ZELqhQLFq1gR5GJZaynDhVc5wj6V3+U5dDrv9/ArAhYxiFcVUuZGNIjLKVSBbpu8syC5l3uTq+wU6nJ269+d14XJv5hgnXTu1HgaIEYC8Rc5Sh1cAwhiD2CcYR5t3NJ6BdV6uLTPSFxe6jWEAznLl68NzRP+LyvxH9zTe8AZ3UYkGr0DhvT2BMsUJwiRxC5UXquHD+DHZlAWEKSiUB819+C4ZCiV2pYYofWRsD4UUJqVGmKyMSkDUgOOTtSe9BTew34KEWBfY8pzAIkJyJlgtNu6A5oZ0ITTjYH2oxXrq8FBw8Mc61rK2DRzJ12EPwH/dMRSqPH6B9dfp2VUusszatPd6B3w/3DFJTS5oehbUYjrdFQkHJENd9qcLn2ZsdcLRYrbJZLQT0oKEW8gLJqlJzP5OMwly9P9sKmMcbHjU9mUpp7yCGsTC806EKXe3ZFj4a0ut5lfQB26KRU0HdS87zRS1YF05+ghUNlaJYcbMXmIMhZJrK/mRZEZfqiRUZUAAqhaJ4+QH5Bt4+FkppBJhTClgM5O+koQSF62RVxgm9jNchVJJuMeteq3+XNzOR3j4DnQJ4HQ4fZNyyvtZSvyf2y1f6qnJqx8i0xFTbhLTDvkv1aYEXq7sVA7pwePelBb449DEPPPElMZgoQSoVNfxqUSThHVZP6iFl69WbdStYuPrjugAQEtA+wlv9cJrtPXHiKp5oyYuWHA3swiB3nm3IcnusDkfvtBeh4ixKUqyOzVVAA== X-Forefront-Antispam-Report: CIP:216.228.118.233; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge2.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(39860400002)(396003)(376002)(346002)(136003)(451199015)(40470700004)(46966006)(36840700001)(70586007)(70206006)(356005)(316002)(41300700001)(6916009)(7636003)(82310400005)(54906003)(107886003)(8936002)(7696005)(2616005)(36860700001)(5660300002)(6666004)(55016003)(40480700001)(426003)(478600001)(47076005)(186003)(4326008)(8676002)(82740400003)(1076003)(336012)(26005)(86362001)(6286002)(36756003)(40460700003)(2906002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Oct 2022 16:08:47.5429 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4e62456c-a87b-40a2-47c4-08dabb5a3171 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.233]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT069.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8PR12MB5414 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add assertions to help debug in case of counter double alloc/free. Signed-off-by: Michael Baum Acked-by: Matan Azrad Acked-by: Xiaoyu Min --- drivers/net/mlx5/mlx5_hws_cnt.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/net/mlx5/mlx5_hws_cnt.h b/drivers/net/mlx5/mlx5_hws_cnt.h index 6e371f1929..338ee4d688 100644 --- a/drivers/net/mlx5/mlx5_hws_cnt.h +++ b/drivers/net/mlx5/mlx5_hws_cnt.h @@ -396,6 +396,7 @@ mlx5_hws_cnt_pool_put(struct mlx5_hws_cnt_pool *cpool, uint32_t iidx; iidx = mlx5_hws_cnt_iidx(cpool, *cnt_id); + MLX5_ASSERT(cpool->pool[iidx].in_used); cpool->pool[iidx].in_used = false; cpool->pool[iidx].query_gen_when_free = __atomic_load_n(&cpool->query_gen, __ATOMIC_RELAXED); @@ -475,6 +476,7 @@ mlx5_hws_cnt_pool_get(struct mlx5_hws_cnt_pool *cpool, uint32_t *queue, __hws_cnt_query_raw(cpool, *cnt_id, &cpool->pool[iidx].reset.hits, &cpool->pool[iidx].reset.bytes); + MLX5_ASSERT(!cpool->pool[iidx].in_used); cpool->pool[iidx].in_used = true; cpool->pool[iidx].age_idx = age_idx; return 0; @@ -511,6 +513,7 @@ mlx5_hws_cnt_pool_get(struct mlx5_hws_cnt_pool *cpool, uint32_t *queue, &cpool->pool[iidx].reset.bytes); rte_ring_dequeue_zc_elem_finish(qcache, 1); cpool->pool[iidx].share = 0; + MLX5_ASSERT(!cpool->pool[iidx].in_used); cpool->pool[iidx].in_used = true; cpool->pool[iidx].age_idx = age_idx; return 0; From patchwork Mon Oct 31 16:08:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 119371 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 495B0A0540; Mon, 31 Oct 2022 17:09:19 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 120904281C; Mon, 31 Oct 2022 17:08:55 +0100 (CET) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2085.outbound.protection.outlook.com [40.107.243.85]) by mails.dpdk.org (Postfix) with ESMTP id 58BB0427F3 for ; Mon, 31 Oct 2022 17:08:53 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Uw+ufOm1t9W1ZymjOzFPtr0h1uIproa82G0B8z4Spwp77soLtyCQp8Lhd3kTyHvwiAt3z8uWgRflyFzE5lQ65EZ/y7MRcx8k4UEjKT0CaoaahLHlmkf9W8pdP8abHNN5Zq26nKunVk/v+ZEz3r9e8iXSIHmJ6PIt1EIL+jwEdvjtFTbEQdK+aBx1yzOSDXjZQ7t5wuIailFtc088dt/iIgiH/usRZ+2x86LPFdvQLEKUqBExIzWGqsyvrTHdicJL3hjy2ElJy0y8SKjuaqAzStX0wc1PvSGFkUAd+4zwJBMCzB9OBK4fF7RE9NSqpxiy4nh1bJEOFzpFiyKiUylvYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LNO9/5ja7ILi+ymsQhgBxCMbfWC/te54y+GXqj0pecA=; b=hG7VIBO6LWDUJ9MgJbkWUluFKSZBpq4MshQdnDlcRwEQuAYzCicy/eNYcXhBmZnFaT33F1syfC9qzADP6GOyNFMjrJHV4d2Fbdl267iLMNKQhO4n1hS320sJxmBsv4uPgVSsZNCcNOYLh9fbouOyadZ0rHZ4OeUbfbZeYSn8CkjKwHHaZ++ALiJmHgQ5S1I83UG2OWBKSBsZyJ9JbkpAQhBVYJrSNgbQLJwZ6GoVM0CaH5rfTtY/WeGgHo9a8yFM17kaRFdol+tKYU1/9RJe+CYFWYCRyhhhcL7VNKhGf9hu/1wxnhXW8lowWUHLyIf/cTRLjd02l2xdaeHAqPN9yQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LNO9/5ja7ILi+ymsQhgBxCMbfWC/te54y+GXqj0pecA=; b=jG7jdn+MeNZq6pzS3LZzUH0Fsyx2EiFQIxkh+O94mzZxmQnemsCWENdyhF9honPS1AVqRtc8IPEQyG0ncDFRIt36ZPNtze39YlFitQV8dcGxSD0K8G8wsY81/gz72c8EkUZm+Qa9YIJAL5Wr3udN6a7Jly3DuEDwb8EMZ8TXYuJmoaEHRKf1rD3jGwQrCAl+yHIYSDcZT/2MEgBtRV92RMfoyJ/5WuFoloZV36zLfIJdd8HmAWeFnyRUNIM/N9l+6Sw9WLOWF0ZVkaBCI6iX7FpDNzFDjgahnG2Wu/8kyVjscbds2fpqyLKFX2jbEPqawah+BsXld2M1krTJ57xHzA== Received: from BN9PR03CA0075.namprd03.prod.outlook.com (2603:10b6:408:fc::20) by SA1PR12MB7223.namprd12.prod.outlook.com (2603:10b6:806:2bc::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5746.28; Mon, 31 Oct 2022 16:08:51 +0000 Received: from BN8NAM11FT004.eop-nam11.prod.protection.outlook.com (2603:10b6:408:fc:cafe::f5) by BN9PR03CA0075.outlook.office365.com (2603:10b6:408:fc::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.19 via Frontend Transport; Mon, 31 Oct 2022 16:08:51 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by BN8NAM11FT004.mail.protection.outlook.com (10.13.176.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.14 via Frontend Transport; Mon, 31 Oct 2022 16:08:51 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Mon, 31 Oct 2022 09:08:39 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Mon, 31 Oct 2022 09:08:38 -0700 Received: from nvidia.com (10.127.8.13) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29 via Frontend Transport; Mon, 31 Oct 2022 09:08:37 -0700 From: Michael Baum To: CC: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko , Xiaoyu Min Subject: [PATCH 5/5] net/mlx5: assert for enough space in counter rings Date: Mon, 31 Oct 2022 18:08:24 +0200 Message-ID: <20221031160824.330200-6-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221031160824.330200-1-michaelba@nvidia.com> References: <20221031160824.330200-1-michaelba@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT004:EE_|SA1PR12MB7223:EE_ X-MS-Office365-Filtering-Correlation-Id: 33bf08db-4a27-4b26-b9d0-08dabb5a3396 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5NbTVyDdYorBb5B1lnbfL51ap3wkxNDC9YFHboYqG+BuJQZosyeuGrMm9ouC5wBU8AQiIM6NgxQB+2JL7kM4R96fWaRWLPRaU+RcQxbr1DnaujL03o/1VnMX84YiUVEuKOsFnYGRAlZLBCj7cfGOtcCbhwqfczDZ20Vd8pMAoMOmJCTOXX2z8N4iKMbzFPTsKCy+oVkv2yjrEmzkiifl2mqrZRIyYKBITwHO0yl4VV6ymQ2ftmA5JYUcZEFdbSyNyzIF/RnY4W7LhOYttog2j8zvP7VQvnojjdZCS0l5D37ulUUkbTKoBxBl8OkLRS4l5xk6ruJl4RpEgHkRsNSHnKOsjvosB8lExTCX4P/qOj+P/IG4Yc6i6lIbIKmm/i7070uiEL2UCFMxALekm9bOqYpOsQwNvUsdItTuktwEBquCh4InHjTFp2kHuP7bAbof9HjCl9DzfmQ2U8IQMTf0MfutMR+Z3Ars8OHOmp928fNbRV2wgxj8vZtDAtIM1aPP7w9FR5EGfiIaGXefX4fqgJGsNyrzrn+9PUzDONCw3FHohMyR8BVBfghtUhdmmYHtm1uLHWTk9eywKo1DhABxdR0SlBZnFhCuI709Ki3O4L7F8KkIAjAuWNvRt/pHfEU/SC3ukj4TMIb8WCgKgx5QkEkYfEzkMvAWVVSsTmvYXLTckuTDIFw+rNSsKQ6lgr72RDEhYgUvxRNXUtOG+Luq8g9AfdZFhcFeRRj38XL6cMf5eYz+thEh3IZ7EhKFWO9y9vK0phZHb+qD363E8GmgZg== X-Forefront-Antispam-Report: CIP:216.228.118.233; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge2.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(136003)(346002)(376002)(396003)(39860400002)(451199015)(36840700001)(40470700004)(46966006)(40460700003)(2616005)(82310400005)(5660300002)(478600001)(47076005)(426003)(8936002)(7696005)(36860700001)(40480700001)(82740400003)(1076003)(6916009)(186003)(41300700001)(356005)(36756003)(7636003)(54906003)(4326008)(30864003)(2906002)(83380400001)(70586007)(6286002)(336012)(316002)(86362001)(70206006)(26005)(6666004)(107886003)(55016003)(8676002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Oct 2022 16:08:51.1438 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 33bf08db-4a27-4b26-b9d0-08dabb5a3396 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.233]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT004.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB7223 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org There is a by-design assumption in the code that the global counter rings can contain all the port counters. So, enqueuing to these global rings should always succeed. Add assertions to help for debugging this assumption. In addition, change mlx5_hws_cnt_pool_put() function to return void due to those assumptions. Signed-off-by: Michael Baum Acked-by: Matan Azrad Acked-by: Xiaoyu Min --- drivers/net/mlx5/mlx5_flow_hw.c | 2 +- drivers/net/mlx5/mlx5_hws_cnt.c | 25 ++++---- drivers/net/mlx5/mlx5_hws_cnt.h | 106 +++++++++++++++++--------------- 3 files changed, 72 insertions(+), 61 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c index 2d275ad111..54a0afe45f 100644 --- a/drivers/net/mlx5/mlx5_flow_hw.c +++ b/drivers/net/mlx5/mlx5_flow_hw.c @@ -7874,7 +7874,7 @@ flow_hw_action_handle_destroy(struct rte_eth_dev *dev, uint32_t queue, * time to update the AGE. */ mlx5_hws_age_nb_cnt_decrease(priv, age_idx); - ret = mlx5_hws_cnt_shared_put(priv->hws_cpool, &act_idx); + mlx5_hws_cnt_shared_put(priv->hws_cpool, &act_idx); break; case MLX5_INDIRECT_ACTION_TYPE_CT: ret = flow_hw_conntrack_destroy(dev, act_idx, error); diff --git a/drivers/net/mlx5/mlx5_hws_cnt.c b/drivers/net/mlx5/mlx5_hws_cnt.c index b8ce69af57..24c01eace0 100644 --- a/drivers/net/mlx5/mlx5_hws_cnt.c +++ b/drivers/net/mlx5/mlx5_hws_cnt.c @@ -58,13 +58,14 @@ __hws_cnt_id_load(struct mlx5_hws_cnt_pool *cpool) static void __mlx5_hws_cnt_svc(struct mlx5_dev_ctx_shared *sh, - struct mlx5_hws_cnt_pool *cpool) + struct mlx5_hws_cnt_pool *cpool) { struct rte_ring *reset_list = cpool->wait_reset_list; struct rte_ring *reuse_list = cpool->reuse_list; uint32_t reset_cnt_num; struct rte_ring_zc_data zcdr = {0}; struct rte_ring_zc_data zcdu = {0}; + uint32_t ret __rte_unused; reset_cnt_num = rte_ring_count(reset_list); do { @@ -72,17 +73,19 @@ __mlx5_hws_cnt_svc(struct mlx5_dev_ctx_shared *sh, mlx5_aso_cnt_query(sh, cpool); zcdr.n1 = 0; zcdu.n1 = 0; - rte_ring_enqueue_zc_burst_elem_start(reuse_list, - sizeof(cnt_id_t), reset_cnt_num, &zcdu, - NULL); - rte_ring_dequeue_zc_burst_elem_start(reset_list, - sizeof(cnt_id_t), reset_cnt_num, &zcdr, - NULL); + ret = rte_ring_enqueue_zc_burst_elem_start(reuse_list, + sizeof(cnt_id_t), + reset_cnt_num, &zcdu, + NULL); + MLX5_ASSERT(ret == reset_cnt_num); + ret = rte_ring_dequeue_zc_burst_elem_start(reset_list, + sizeof(cnt_id_t), + reset_cnt_num, &zcdr, + NULL); + MLX5_ASSERT(ret == reset_cnt_num); __hws_cnt_r2rcpy(&zcdu, &zcdr, reset_cnt_num); - rte_ring_dequeue_zc_elem_finish(reset_list, - reset_cnt_num); - rte_ring_enqueue_zc_elem_finish(reuse_list, - reset_cnt_num); + rte_ring_dequeue_zc_elem_finish(reset_list, reset_cnt_num); + rte_ring_enqueue_zc_elem_finish(reuse_list, reset_cnt_num); reset_cnt_num = rte_ring_count(reset_list); } while (reset_cnt_num > 0); } diff --git a/drivers/net/mlx5/mlx5_hws_cnt.h b/drivers/net/mlx5/mlx5_hws_cnt.h index 338ee4d688..030dcead86 100644 --- a/drivers/net/mlx5/mlx5_hws_cnt.h +++ b/drivers/net/mlx5/mlx5_hws_cnt.h @@ -116,7 +116,7 @@ enum { HWS_AGE_CANDIDATE_INSIDE_RING, /* * AGE assigned to flows but it still in ring. It was aged-out but the - * timeout was changed, so it in ring but stiil candidate. + * timeout was changed, so it in ring but still candidate. */ HWS_AGE_AGED_OUT_REPORTED, /* @@ -182,7 +182,7 @@ mlx5_hws_cnt_id_valid(cnt_id_t cnt_id) * * @param cpool * The pointer to counter pool - * @param index + * @param iidx * The internal counter index. * * @return @@ -231,32 +231,32 @@ __hws_cnt_query_raw(struct mlx5_hws_cnt_pool *cpool, cnt_id_t cnt_id, } /** - * Copy elems from one zero-copy ring to zero-copy ring in place. + * Copy elements from one zero-copy ring to zero-copy ring in place. * * The input is a rte ring zero-copy data struct, which has two pointer. * in case of the wrapper happened, the ptr2 will be meaningful. * - * So this rountin needs to consider the situation that the address given by + * So this routine needs to consider the situation that the address given by * source and destination could be both wrapped. * First, calculate the first number of element needs to be copied until wrapped * address, which could be in source or destination. * Second, copy left number of element until second wrapped address. If in first * step the wrapped address is source, then this time it must be in destination. - * and vice-vers. - * Third, copy all left numbe of element. + * and vice-versa. + * Third, copy all left number of element. * * In worst case, we need copy three pieces of continuous memory. * * @param zcdd - * A pointer to zero-copy data of dest ring. + * A pointer to zero-copy data of destination ring. * @param zcds * A pointer to zero-copy data of source ring. * @param n - * Number of elems to copy. + * Number of elements to copy. */ static __rte_always_inline void __hws_cnt_r2rcpy(struct rte_ring_zc_data *zcdd, struct rte_ring_zc_data *zcds, - unsigned int n) + unsigned int n) { unsigned int n1, n2, n3; void *s1, *s2, *s3; @@ -291,22 +291,23 @@ static __rte_always_inline int mlx5_hws_cnt_pool_cache_flush(struct mlx5_hws_cnt_pool *cpool, uint32_t queue_id) { - unsigned int ret; + unsigned int ret __rte_unused; struct rte_ring_zc_data zcdr = {0}; struct rte_ring_zc_data zcdc = {0}; struct rte_ring *reset_list = NULL; struct rte_ring *qcache = cpool->cache->qcache[queue_id]; + uint32_t ring_size = rte_ring_count(qcache); - ret = rte_ring_dequeue_zc_burst_elem_start(qcache, - sizeof(cnt_id_t), rte_ring_count(qcache), &zcdc, - NULL); - MLX5_ASSERT(ret); + ret = rte_ring_dequeue_zc_burst_elem_start(qcache, sizeof(cnt_id_t), + ring_size, &zcdc, NULL); + MLX5_ASSERT(ret == ring_size); reset_list = cpool->wait_reset_list; - rte_ring_enqueue_zc_burst_elem_start(reset_list, - sizeof(cnt_id_t), ret, &zcdr, NULL); - __hws_cnt_r2rcpy(&zcdr, &zcdc, ret); - rte_ring_enqueue_zc_elem_finish(reset_list, ret); - rte_ring_dequeue_zc_elem_finish(qcache, ret); + ret = rte_ring_enqueue_zc_burst_elem_start(reset_list, sizeof(cnt_id_t), + ring_size, &zcdr, NULL); + MLX5_ASSERT(ret == ring_size); + __hws_cnt_r2rcpy(&zcdr, &zcdc, ring_size); + rte_ring_enqueue_zc_elem_finish(reset_list, ring_size); + rte_ring_dequeue_zc_elem_finish(qcache, ring_size); return 0; } @@ -323,7 +324,7 @@ mlx5_hws_cnt_pool_cache_fetch(struct mlx5_hws_cnt_pool *cpool, struct rte_ring_zc_data zcdu = {0}; struct rte_ring_zc_data zcds = {0}; struct mlx5_hws_cnt_pool_caches *cache = cpool->cache; - unsigned int ret; + unsigned int ret, actual_fetch_size __rte_unused; reuse_list = cpool->reuse_list; ret = rte_ring_dequeue_zc_burst_elem_start(reuse_list, @@ -334,7 +335,9 @@ mlx5_hws_cnt_pool_cache_fetch(struct mlx5_hws_cnt_pool *cpool, rte_ring_dequeue_zc_elem_finish(reuse_list, 0); free_list = cpool->free_list; ret = rte_ring_dequeue_zc_burst_elem_start(free_list, - sizeof(cnt_id_t), cache->fetch_sz, &zcdf, NULL); + sizeof(cnt_id_t), + cache->fetch_sz, + &zcdf, NULL); zcds = zcdf; list = free_list; if (unlikely(ret == 0)) { /* no free counter. */ @@ -344,8 +347,10 @@ mlx5_hws_cnt_pool_cache_fetch(struct mlx5_hws_cnt_pool *cpool, return -ENOENT; } } - rte_ring_enqueue_zc_burst_elem_start(qcache, sizeof(cnt_id_t), - ret, &zcdc, NULL); + actual_fetch_size = ret; + ret = rte_ring_enqueue_zc_burst_elem_start(qcache, sizeof(cnt_id_t), + ret, &zcdc, NULL); + MLX5_ASSERT(ret == actual_fetch_size); __hws_cnt_r2rcpy(&zcdc, &zcds, ret); rte_ring_dequeue_zc_elem_finish(list, ret); rte_ring_enqueue_zc_elem_finish(qcache, ret); @@ -378,15 +383,14 @@ __mlx5_hws_cnt_pool_enqueue_revert(struct rte_ring *r, unsigned int n, * * @param cpool * A pointer to the counter pool structure. + * @param queue + * A pointer to HWS queue. If null, it means put into common pool. * @param cnt_id * A counter id to be added. - * @return - * - 0: Success; object taken - * - -ENOENT: not enough entry in pool */ -static __rte_always_inline int -mlx5_hws_cnt_pool_put(struct mlx5_hws_cnt_pool *cpool, - uint32_t *queue, cnt_id_t *cnt_id) +static __rte_always_inline void +mlx5_hws_cnt_pool_put(struct mlx5_hws_cnt_pool *cpool, uint32_t *queue, + cnt_id_t *cnt_id) { unsigned int ret = 0; struct rte_ring_zc_data zcdc = {0}; @@ -404,25 +408,29 @@ mlx5_hws_cnt_pool_put(struct mlx5_hws_cnt_pool *cpool, qcache = cpool->cache->qcache[*queue]; if (unlikely(qcache == NULL)) { ret = rte_ring_enqueue_elem(cpool->wait_reset_list, cnt_id, - sizeof(cnt_id_t)); + sizeof(cnt_id_t)); MLX5_ASSERT(ret == 0); - return ret; + return; } ret = rte_ring_enqueue_burst_elem(qcache, cnt_id, sizeof(cnt_id_t), 1, NULL); if (unlikely(ret == 0)) { /* cache is full. */ + struct rte_ring *reset_list = cpool->wait_reset_list; + wb_num = rte_ring_count(qcache) - cpool->cache->threshold; MLX5_ASSERT(wb_num < rte_ring_count(qcache)); __mlx5_hws_cnt_pool_enqueue_revert(qcache, wb_num, &zcdc); - rte_ring_enqueue_zc_burst_elem_start(cpool->wait_reset_list, - sizeof(cnt_id_t), wb_num, &zcdr, NULL); - __hws_cnt_r2rcpy(&zcdr, &zcdc, wb_num); - rte_ring_enqueue_zc_elem_finish(cpool->wait_reset_list, wb_num); + ret = rte_ring_enqueue_zc_burst_elem_start(reset_list, + sizeof(cnt_id_t), + wb_num, &zcdr, NULL); + MLX5_ASSERT(ret == wb_num); + __hws_cnt_r2rcpy(&zcdr, &zcdc, ret); + rte_ring_enqueue_zc_elem_finish(reset_list, ret); /* write-back THIS counter too */ - ret = rte_ring_enqueue_burst_elem(cpool->wait_reset_list, - cnt_id, sizeof(cnt_id_t), 1, NULL); + ret = rte_ring_enqueue_burst_elem(reset_list, cnt_id, + sizeof(cnt_id_t), 1, NULL); } - return ret == 1 ? 0 : -ENOENT; + MLX5_ASSERT(ret == 1); } /** @@ -482,15 +490,17 @@ mlx5_hws_cnt_pool_get(struct mlx5_hws_cnt_pool *cpool, uint32_t *queue, return 0; } ret = rte_ring_dequeue_zc_burst_elem_start(qcache, sizeof(cnt_id_t), 1, - &zcdc, NULL); + &zcdc, NULL); if (unlikely(ret == 0)) { /* local cache is empty. */ rte_ring_dequeue_zc_elem_finish(qcache, 0); /* let's fetch from global free list. */ ret = mlx5_hws_cnt_pool_cache_fetch(cpool, *queue); if (unlikely(ret != 0)) return ret; - rte_ring_dequeue_zc_burst_elem_start(qcache, sizeof(cnt_id_t), - 1, &zcdc, NULL); + ret = rte_ring_dequeue_zc_burst_elem_start(qcache, + sizeof(cnt_id_t), 1, + &zcdc, NULL); + MLX5_ASSERT(ret == 1); } /* get one from local cache. */ *cnt_id = (*(cnt_id_t *)zcdc.ptr1); @@ -504,8 +514,10 @@ mlx5_hws_cnt_pool_get(struct mlx5_hws_cnt_pool *cpool, uint32_t *queue, ret = mlx5_hws_cnt_pool_cache_fetch(cpool, *queue); if (unlikely(ret != 0)) return ret; - rte_ring_dequeue_zc_burst_elem_start(qcache, sizeof(cnt_id_t), - 1, &zcdc, NULL); + ret = rte_ring_dequeue_zc_burst_elem_start(qcache, + sizeof(cnt_id_t), 1, + &zcdc, NULL); + MLX5_ASSERT(ret == 1); *cnt_id = *(cnt_id_t *)zcdc.ptr1; iidx = mlx5_hws_cnt_iidx(cpool, *cnt_id); } @@ -553,17 +565,13 @@ mlx5_hws_cnt_shared_get(struct mlx5_hws_cnt_pool *cpool, cnt_id_t *cnt_id, return 0; } -static __rte_always_inline int +static __rte_always_inline void mlx5_hws_cnt_shared_put(struct mlx5_hws_cnt_pool *cpool, cnt_id_t *cnt_id) { - int ret; uint32_t iidx = mlx5_hws_cnt_iidx(cpool, *cnt_id); cpool->pool[iidx].share = 0; - ret = mlx5_hws_cnt_pool_put(cpool, NULL, cnt_id); - if (unlikely(ret != 0)) - cpool->pool[iidx].share = 1; /* fail to release, restore. */ - return ret; + mlx5_hws_cnt_pool_put(cpool, NULL, cnt_id); } static __rte_always_inline bool