From patchwork Sun Jul 19 13:39:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 74439 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B032DA0540; Sun, 19 Jul 2020 15:40:05 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 180741C0DC; Sun, 19 Jul 2020 15:40:04 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 43AEB1C0CA for ; Sun, 19 Jul 2020 15:40:02 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with SMTP; 19 Jul 2020 16:39:58 +0300 Received: from pegasus12.mtr.labs.mlnx (pegasus12.mtr.labs.mlnx [10.210.17.40]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06JDdw0T019391; Sun, 19 Jul 2020 16:39:58 +0300 Received: from pegasus12.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus12.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id 06JDdw0w012262; Sun, 19 Jul 2020 13:39:58 GMT Received: (from viacheslavo@localhost) by pegasus12.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id 06JDdw0N012261; Sun, 19 Jul 2020 13:39:58 GMT X-Authentication-Warning: pegasus12.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net, dekelp@mellanox.com, stable@dpdk.org Date: Sun, 19 Jul 2020 13:39:57 +0000 Message-Id: <1595165997-12225-1-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 Subject: [dpdk-dev] [PATCH] common/mlx5: fix receiving queue doorbell register size X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When received queue is being created with DevX the allocated doorbell register size is only uint64_t. That is definitely less than size of CPU cacheline and it might happen the registers attached to different queues handled by different cores are allocated within same cacheline. It might cause contention on doorbell register writing. This patch extends the allocated memory size for doorbell register to cacheline size. Fixes: 21cae8580fd0 ("net/mlx5: allocate door-bells via DevX") Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko --- drivers/common/mlx5/mlx5_common.c | 2 +- drivers/common/mlx5/mlx5_common.h | 10 ++++++---- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/drivers/common/mlx5/mlx5_common.c b/drivers/common/mlx5/mlx5_common.c index 79cd5ba..193e73b 100644 --- a/drivers/common/mlx5/mlx5_common.c +++ b/drivers/common/mlx5/mlx5_common.c @@ -187,7 +187,7 @@ static inline void mlx5_cpu_id(unsigned int level, page->dbr_bitmap[i] |= (UINT64_C(1) << j); page->dbr_count++; *dbr_page = page; - return (((i * 64) + j) * sizeof(uint64_t)); + return (i * CHAR_BIT * sizeof(uint64_t) + j) * MLX5_DBR_SIZE; } /** diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index 5b9b7bd..ef7b61b 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -214,10 +214,12 @@ enum mlx5_class { MLX5_CLASS_VDPA = RTE_BIT64(1), }; -#define MLX5_DBR_PAGE_SIZE 4096 /* Must be >= 512. */ -#define MLX5_DBR_SIZE 8 -#define MLX5_DBR_PER_PAGE (MLX5_DBR_PAGE_SIZE / MLX5_DBR_SIZE) -#define MLX5_DBR_BITMAP_SIZE (MLX5_DBR_PER_PAGE / 64) +#define MLX5_DBR_SIZE RTE_CACHE_LINE_SIZE +#define MLX5_DBR_PER_PAGE 64 +/* Must be >= CHAR_BIT * sizeof(uint64_t) */ +#define MLX5_DBR_PAGE_SIZE (MLX5_DBR_PER_PAGE * MLX5_DBR_SIZE) +/* Page size must be >= 512. */ +#define MLX5_DBR_BITMAP_SIZE (MLX5_DBR_PER_PAGE / (CHAR_BIT * sizeof(uint64_t))) struct mlx5_devx_dbr_page { /* Door-bell records, must be first member in structure. */