From patchwork Wed Oct 6 22:03:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 100665 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EA872A0C4D; Thu, 7 Oct 2021 00:06:04 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DAD264122A; Thu, 7 Oct 2021 00:04:40 +0200 (CEST) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2058.outbound.protection.outlook.com [40.107.94.58]) by mails.dpdk.org (Postfix) with ESMTP id E962841196 for ; Thu, 7 Oct 2021 00:04:35 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CCMjLpdSzu59O1daId4SAm9vmvQh1UkMxAZmo7CehhVQOXGUuE4JNtbHDLxqJy8WNUWXW0708Nb41YxvzMKDzyqD0aehVzQJPs7l5mIgs9EnPdECe2MdkxBykH8/Ycr0jE3iM4BhbEq8fUqYZH6UjeQZt5HiN2u+gw7w33lUiqV3vdE+4aIlUJxUj8G39xRzyNrNVGyyfv8EM1JYNwKR58M1w0QJRQRduXUtRQKMTsF4W3/knrtSaVSUmkuqa2e4ScX27PDAmCF+ICfq5IR3lurbJGSnSYf0sXrtMog4Z9JdKuRVLxyqZPethbqPwnrQBA3iCVyJFE4QIC1OSxXUmA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=L/4bytLFlkZ5sF43LLaF2GobjhLx8K0Hj/vRpOS6tZY=; b=RjAp2yM5ygT0Up/j4WYRgKmqY91N4Px+GsHxu7F5onFKeRO8Va88szu+EMaGTLpCF8kOT1QmmP2XJ8vFSqfKlmv3U+cNBtfUjN1VhgraoVUiFUcj3gWhiSeacRnrkz12FW77z5GXgGyeOLw/nnSBonqUwcaKkkgl3N0YKWrrOyAzgfQly/XuFdh5s7UIZ9tp9+A3iQ3Ar6kd4T3hXoAatj4Q0ju52AcNceBroWm7WM8WwtLVRBf8G3POKC0YAABh0/cCz5RRTDRnSc1f6P9w9zgB24+omFCgj3gxLycim7PUMINSXGVmH99j5By9f1NoAG0aHd2PecUZyIQgIZ/L9w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=L/4bytLFlkZ5sF43LLaF2GobjhLx8K0Hj/vRpOS6tZY=; b=Q7zc8xFjhyFDbBeDjzaVMn9BXoRnSJv07msib3+hEecI1v2r3aSO+4df6733T0ZVNZ6OzL4+i6zLUB8VQBk1uYbeHcYxhQtivTPI4/Bxywow+BhOOu527FIbrPofcUcNYq6y3oKw+lHOF87zNXXfdOmn9CA3gzDqqTqV87xm++454zANkBPOAOhjr65n+S2Iqgh1jHyMXQQLtPZtwVyIvUq6QCSTVnWCkhmXcoON44xO0Ueu/TuRRCwLg+EoGN3wpQXdp31nMImDektk3vd1QfQgUGD4/cdNwBMxMRtRTu98cCkaa8JujNSyTHiSBFNoJAxiIFOAnj+l3A3hY8HHJA== Received: from CO2PR04CA0204.namprd04.prod.outlook.com (2603:10b6:104:5::34) by MW3PR12MB4522.namprd12.prod.outlook.com (2603:10b6:303:5f::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.16; Wed, 6 Oct 2021 22:04:34 +0000 Received: from CO1NAM11FT015.eop-nam11.prod.protection.outlook.com (2603:10b6:104:5:cafe::66) by CO2PR04CA0204.outlook.office365.com (2603:10b6:104:5::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.18 via Frontend Transport; Wed, 6 Oct 2021 22:04:34 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; monjalon.net; dkim=none (message not signed) header.d=none;monjalon.net; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT015.mail.protection.outlook.com (10.13.175.130) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4587.18 via Frontend Transport; Wed, 6 Oct 2021 22:04:33 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Wed, 6 Oct 2021 22:04:32 +0000 From: To: CC: Matan Azrad , Thomas Monjalon , Michael Baum Date: Thu, 7 Oct 2021 01:03:45 +0300 Message-ID: <20211006220350.2357487-14-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211006220350.2357487-1-michaelba@nvidia.com> References: <20210930172822.1949969-1-michaelba@nvidia.com> <20211006220350.2357487-1-michaelba@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 20344347-692e-4ae8-49a0-08d9891547b9 X-MS-TrafficTypeDiagnostic: MW3PR12MB4522: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4714; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: aVKoSDrJwHhL3dgUGx/P9LnCpkAFgedygSxxtwS/e+0sW2Ww7w/GyQfQJUJNgH7QErkpEoeWo0jwC6tnoMXmsRZ/5ZtQOd6tbWzDldwLDLldGONkx2tQI/XRFs00wCTJRSoAzT3YJr/wdD7cFLU4remJQaa4JMRYyB+5haSU+T5eUaGNfiNZMXG+mWXuuRZ0y22CKvv139vP+lr6N6CVfd7/46+dyU04FNA8TjtZ8SkhuXX05FzKgKf0LqB+/GAJ7te4iYP/OI8lZNtRMWHxK01KfGrCItf6wCxpI0Khj+UxKL4dpaAPo7TDl2+y9Q6MZyGf86+m9WI+73puQZp01JtA948imsVqTE+tiXUZNJ+N6Vge0ELB2fxWGGoYZkFLnoynn/0dYusWANmpMtPzFXoaJMVF8qHXTcYbUdn3+kucMqmJpz1+wHhs8Uk2ZOmAaJmEFgMcSh+0L0kka22NBEgmrucV3QMzTmMqrGiAXWdBBgg3EnTs2c4p6tXvHbkUeL4XoFeq3MDKpm22w9x/vDqv9DZi8zL9pdFF8ClPOnfV/r+Z2U/mL0eNV0LJY3Q31bbVvgq/aulm5/BgBHFl4Je8sf5tlFVwf8zbgWYYzBuTRYz6soZdjMqlj0Ex3gxSkjvsywXFAy+jWkHW1zcN5V62BCs1Ldn93SnT/YVvDk6nlWG5/9maZBSn9wO14itZlhvAz5L+WUOWNIb1Nm87kQ== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(36840700001)(46966006)(7636003)(6666004)(70586007)(186003)(55016002)(316002)(356005)(1076003)(2616005)(4326008)(26005)(508600001)(6916009)(16526019)(7696005)(8676002)(83380400001)(2906002)(36860700001)(107886003)(86362001)(336012)(70206006)(5660300002)(2876002)(426003)(6286002)(8936002)(47076005)(82310400003)(36756003)(54906003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Oct 2021 22:04:33.9220 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 20344347-692e-4ae8-49a0-08d9891547b9 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT015.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR12MB4522 Subject: [dpdk-dev] [PATCH v2 13/18] common/mlx5: add MR ctrl init function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Michael Baum Add function for MR control structure initialization. This function include: - btree initialization. - dev_gen_ptr initialization. Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/common/mlx5/mlx5_common_mr.c | 28 +++++++++++++++++++++++++ drivers/common/mlx5/mlx5_common_mr.h | 3 +++ drivers/common/mlx5/version.map | 1 + drivers/compress/mlx5/mlx5_compress.c | 6 ++---- drivers/crypto/mlx5/mlx5_crypto.c | 5 ++--- drivers/net/mlx5/mlx5_rxq.c | 6 ++---- drivers/net/mlx5/mlx5_txq.c | 6 ++---- drivers/regex/mlx5/mlx5_regex_control.c | 6 ++---- 8 files changed, 42 insertions(+), 19 deletions(-) diff --git a/drivers/common/mlx5/mlx5_common_mr.c b/drivers/common/mlx5/mlx5_common_mr.c index 2e039a4e70..8fd65484cf 100644 --- a/drivers/common/mlx5/mlx5_common_mr.c +++ b/drivers/common/mlx5/mlx5_common_mr.c @@ -271,6 +271,34 @@ mlx5_mr_btree_dump(struct mlx5_mr_btree *bt __rte_unused) #endif } +/** + * Initialize per-queue MR control descriptor. + * + * @param mr_ctrl + * Pointer to MR control structure. + * @param dev_gen_ptr + * Pointer to generation number of global cache. + * @param socket + * NUMA socket on which memory must be allocated. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_mr_ctrl_init(struct mlx5_mr_ctrl *mr_ctrl, uint32_t *dev_gen_ptr, + int socket) +{ + if (mr_ctrl == NULL) { + rte_errno = EINVAL; + return -rte_errno; + } + /* Save pointer of global generation number to check memory event. */ + mr_ctrl->dev_gen_ptr = dev_gen_ptr; + /* Initialize B-tree and allocate memory for bottom-half cache table. */ + return mlx5_mr_btree_init(&mr_ctrl->cache_bh, MLX5_MR_BTREE_CACHE_N, + socket); +} + /** * Find virtually contiguous memory chunk in a given MR. * diff --git a/drivers/common/mlx5/mlx5_common_mr.h b/drivers/common/mlx5/mlx5_common_mr.h index 15489cd399..1392d9b55a 100644 --- a/drivers/common/mlx5/mlx5_common_mr.h +++ b/drivers/common/mlx5/mlx5_common_mr.h @@ -124,6 +124,9 @@ mlx5_mr_lookup_lkey(struct mr_cache_entry *lkp_tbl, uint16_t *cached_idx, return UINT32_MAX; } +__rte_internal +int mlx5_mr_ctrl_init(struct mlx5_mr_ctrl *mr_ctrl, uint32_t *dev_gen_ptr, + int socket); __rte_internal int mlx5_mr_btree_init(struct mlx5_mr_btree *bt, int n, int socket); __rte_internal diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index 44c4593888..6200c013fb 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -111,6 +111,7 @@ INTERNAL { mlx5_mr_btree_free; mlx5_mr_btree_init; mlx5_mr_create_primary; + mlx5_mr_ctrl_init; mlx5_mr_dump_cache; mlx5_mr_flush_local_cache; mlx5_mr_free; diff --git a/drivers/compress/mlx5/mlx5_compress.c b/drivers/compress/mlx5/mlx5_compress.c index f1d3416237..90d1684e00 100644 --- a/drivers/compress/mlx5/mlx5_compress.c +++ b/drivers/compress/mlx5/mlx5_compress.c @@ -206,8 +206,8 @@ mlx5_compress_qp_setup(struct rte_compressdev *dev, uint16_t qp_id, return -rte_errno; } dev->data->queue_pairs[qp_id] = qp; - if (mlx5_mr_btree_init(&qp->mr_ctrl.cache_bh, MLX5_MR_BTREE_CACHE_N, - priv->dev_config.socket_id)) { + if (mlx5_mr_ctrl_init(&qp->mr_ctrl, &priv->mr_scache.dev_gen, + priv->dev_config.socket_id)) { DRV_LOG(ERR, "Cannot allocate MR Btree for qp %u.", (uint32_t)qp_id); rte_errno = ENOMEM; @@ -258,8 +258,6 @@ mlx5_compress_qp_setup(struct rte_compressdev *dev, uint16_t qp_id, ret = mlx5_devx_qp2rts(&qp->qp, 0); if (ret) goto err; - /* Save pointer of global generation number to check memory event. */ - qp->mr_ctrl.dev_gen_ptr = &priv->mr_scache.dev_gen; DRV_LOG(INFO, "QP %u: SQN=0x%X CQN=0x%X entries num = %u", (uint32_t)qp_id, qp->qp.qp->id, qp->cq.cq->id, qp->entries_n); return 0; diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c index 30e23e7b00..acfb856ef8 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.c +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -677,14 +677,13 @@ mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, DRV_LOG(ERR, "Failed to create QP."); goto error; } - if (mlx5_mr_btree_init(&qp->mr_ctrl.cache_bh, MLX5_MR_BTREE_CACHE_N, - priv->dev_config.socket_id) != 0) { + if (mlx5_mr_ctrl_init(&qp->mr_ctrl, &priv->mr_scache.dev_gen, + priv->dev_config.socket_id) != 0) { DRV_LOG(ERR, "Cannot allocate MR Btree for qp %u.", (uint32_t)qp_id); rte_errno = ENOMEM; goto error; } - qp->mr_ctrl.dev_gen_ptr = &priv->mr_scache.dev_gen; /* * In Order to configure self loopback, when calling devx qp2rts the * remote QP id that is used is the id of the same QP. diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index f7f71f933e..479ea1324c 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -1452,13 +1452,11 @@ mlx5_rxq_new(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, goto error; } tmpl->type = MLX5_RXQ_TYPE_STANDARD; - if (mlx5_mr_btree_init(&tmpl->rxq.mr_ctrl.cache_bh, - MLX5_MR_BTREE_CACHE_N, socket)) { + if (mlx5_mr_ctrl_init(&tmpl->rxq.mr_ctrl, + &priv->sh->share_cache.dev_gen, socket)) { /* rte_errno is already set. */ goto error; } - /* Rx queues don't use this pointer, but we want a valid structure. */ - tmpl->rxq.mr_ctrl.dev_gen_ptr = &priv->sh->share_cache.dev_gen; tmpl->socket = socket; if (dev->data->dev_conf.intr_conf.rxq) tmpl->irq = 1; diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c index a9c8ce52c9..4b027d6807 100644 --- a/drivers/net/mlx5/mlx5_txq.c +++ b/drivers/net/mlx5/mlx5_txq.c @@ -1124,13 +1124,11 @@ mlx5_txq_new(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, rte_errno = ENOMEM; return NULL; } - if (mlx5_mr_btree_init(&tmpl->txq.mr_ctrl.cache_bh, - MLX5_MR_BTREE_CACHE_N, socket)) { + if (mlx5_mr_ctrl_init(&tmpl->txq.mr_ctrl, + &priv->sh->share_cache.dev_gen, socket)) { /* rte_errno is already set. */ goto error; } - /* Save pointer of global generation number to check memory event. */ - tmpl->txq.mr_ctrl.dev_gen_ptr = &priv->sh->share_cache.dev_gen; MLX5_ASSERT(desc > MLX5_TX_COMP_THRESH); tmpl->txq.offloads = conf->offloads | dev->data->dev_conf.txmode.offloads; diff --git a/drivers/regex/mlx5/mlx5_regex_control.c b/drivers/regex/mlx5/mlx5_regex_control.c index 545bbbcf89..6735e51976 100644 --- a/drivers/regex/mlx5/mlx5_regex_control.c +++ b/drivers/regex/mlx5/mlx5_regex_control.c @@ -242,10 +242,8 @@ mlx5_regex_qp_setup(struct rte_regexdev *dev, uint16_t qp_ind, nb_sq_config++; } - /* Save pointer of global generation number to check memory event. */ - qp->mr_ctrl.dev_gen_ptr = &priv->mr_scache.dev_gen; - ret = mlx5_mr_btree_init(&qp->mr_ctrl.cache_bh, MLX5_MR_BTREE_CACHE_N, - rte_socket_id()); + ret = mlx5_mr_ctrl_init(&qp->mr_ctrl, &priv->mr_scache.dev_gen, + rte_socket_id()); if (ret) { DRV_LOG(ERR, "Error setting up mr btree"); goto err_btree;