From patchwork Thu Sep 30 17:28:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 100182 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6B16FA0C43; Thu, 30 Sep 2021 19:41:15 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DFC9441162; Thu, 30 Sep 2021 19:39:59 +0200 (CEST) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2083.outbound.protection.outlook.com [40.107.220.83]) by mails.dpdk.org (Postfix) with ESMTP id A99374067E for ; Thu, 30 Sep 2021 19:29:10 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LPRt1yeSK7fymHe8hZ6OParECAD1Ba2Tnt+q2eQhk6e8E0k245jAs36tv9j1jR8B6c8K+akm3i3oAyNAy+EEFcQ8sh2rqX922skqEbLzrSbcCq6qqLftCNtR9BNKcrfIvUWZHIqQRg/WHZ2VUIRHhJxJoNoow9QG3r0IABn2fHT+Lubt7jqBr//P8tzx0PXFzw4kk3hBzDuhR4IAUNiw02uGB5aWnIiloUa4hYJGQ+5a0VuWAaAnVXHxJxfWOr+0eKK228WcVNBM/iVGBBZAYeVEejs8xjglwiVvYin8gOwIt3v0J+PUwoDfDl02uUPiUobyaoighCDYkp3BEThGoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=pY17cGc0AslqRFcTcwjFlofemIRZGl3d958w13biLuA=; b=WO0BuvsiWI5E0Q7l2kz7MFAfV82+zK2HUGBKj/TfJ63Eji7cwqMaFGUUBbmi6g3JsxeyiLVmPlM2tCfGiQX63ReKr5c8kX8cEQ4x/CDk1W0elfJdtdYyi6iB7MLZLMcqnTTywFRBKZDO6rWq2E0GUPIcuFrmmYxgAh3k35ZJ0VStKxwTeyCIIDjCZVv19aHt2CiPQ/5V9FHDvCfRyrhh50F75h/H6VMR21JyARLHFY+u2zeCb8EY3lG/goUvUMsQ+80ahwmZbZJu/aHa3JQPG2thgSpYmakWtUsHO+LaJwFcfRyCMxyTT9tD8Vjr2RVbOJ0CE+kBfvzDLlFX3O92dg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pY17cGc0AslqRFcTcwjFlofemIRZGl3d958w13biLuA=; b=l9yk2H+JjNgRS+41tz8ZpOW5NL+GkZillfVHA/bOCucSkNe6gW9ri+Ut0Kl23p8w2uFeWttiKA5S9/wM1C7two7XIMNKD9XPfIlNdeaExYymiKHckwL1MJtau3qfKlT9TZbRIMxpqluJZp/vShQSGKP+ui8Ak1X+xz7RQ/n+HUAao2wjDA319PRBkO0cLQ1ZcLLDCgxgpaOjc4EVnhH+PpWMeaaDGoIeja5SvZtmTUxZDWPo/A40MJS0V13MkCel8/CLR59fzA/cT7NCjYH/fxlLUstTfzjoHURgFgiad9riiwJaAIRkWXZW5iKSSvW0JZcuZ/tpofVOW5a4U1NtZg== Received: from BN0PR02CA0046.namprd02.prod.outlook.com (2603:10b6:408:e5::21) by DM6PR12MB4862.namprd12.prod.outlook.com (2603:10b6:5:1b7::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15; Thu, 30 Sep 2021 17:29:09 +0000 Received: from BN8NAM11FT010.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e5:cafe::79) by BN0PR02CA0046.outlook.office365.com (2603:10b6:408:e5::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.17 via Frontend Transport; Thu, 30 Sep 2021 17:29:09 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; monjalon.net; dkim=none (message not signed) header.d=none;monjalon.net; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT010.mail.protection.outlook.com (10.13.177.53) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4566.14 via Frontend Transport; Thu, 30 Sep 2021 17:29:08 +0000 Received: from nvidia.com (172.20.187.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Thu, 30 Sep 2021 17:29:02 +0000 From: To: CC: Matan Azrad , Thomas Monjalon , Michael Baum Date: Thu, 30 Sep 2021 20:28:17 +0300 Message-ID: <20210930172822.1949969-14-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210930172822.1949969-1-michaelba@nvidia.com> References: <20210930172822.1949969-1-michaelba@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1a109f09-2e38-4f89-019c-08d98437cfb3 X-MS-TrafficTypeDiagnostic: DM6PR12MB4862: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:84; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: sFjqXE/Ltle4UnNliFlF9dxvBoqTDmfV5oaw9WgtrZkFTP43cOAkXo90qRbO0fwlMTSk1gZYLGKKlhTX+kYEeYgsCzT+k089roqohrb3had6g6D0VFHHIvAU/kpR/SMgAi5ac/wMxIQ5Smqn3EQ0t0+rmcbtXFUtvaVgRhFQ2U7eGEXfx5HT378u2yRu34TY0QgCkLZ94BYTsGxTQREodb/PeKONBuN35h9V866/E0dTlwTYopp8/byywtYgrF1854wMLN4XKxFb2D9tU5v8jWFzSc+loXOA+xbnvJtSpjwKQ8g6Rqso/ZW2qD9bnM5+bBp3caSC/FYo3LwudsfFVuVq1H61zVC2/3FtEVVec8+YfGZbObs0O+4cK5lBOA8btqlwfXOXYAJ33WKsyaI7IIMN6yFiPlwih5ETpKjjCwRCZYIvSr8nxG9jLlbqewsCMAcNmATsRJ7dRERJBOMiHpyIzM5DLbgVUviyFzF9Nzzp+ZY+XK73YSm/DIDWwo1gwetDP+4NOlAMcbbgiAbZ4HcfFpeTmV8GYFwiqfSN5oNcClpBfXEn4nI+wj11b+xy2EkwNPjhogXXLHf8pU1RB89PHZL1YePz62KSfMJKzrHls6IB2fnSe37PVViUWQb4kTwjUkwxMSE+9zTpjY3YWn73yFCAYvWCLSA2Xz4oyQoR+zV2Z+hfQbi42LxAs9RmJspX+tcLX+PtYly92KmvQQ== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(86362001)(70206006)(8936002)(26005)(55016002)(82310400003)(70586007)(6286002)(426003)(107886003)(83380400001)(2616005)(16526019)(186003)(7696005)(336012)(1076003)(2876002)(4326008)(8676002)(5660300002)(6666004)(36860700001)(356005)(36756003)(508600001)(6916009)(2906002)(54906003)(36906005)(47076005)(7636003)(316002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 17:29:08.7531 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1a109f09-2e38-4f89-019c-08d98437cfb3 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT010.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4862 X-Mailman-Approved-At: Thu, 30 Sep 2021 19:39:43 +0200 Subject: [dpdk-dev] [PATCH 13/18] common/mlx5: add MR ctrl init function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Michael Baum Add function for MR control structure initialization. This function include: - btree initialization. - dev_gen_ptr initialization. Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/common/mlx5/mlx5_common_mr.c | 28 +++++++++++++++++++++++++ drivers/common/mlx5/mlx5_common_mr.h | 3 +++ drivers/common/mlx5/version.map | 1 + drivers/compress/mlx5/mlx5_compress.c | 6 ++---- drivers/crypto/mlx5/mlx5_crypto.c | 5 ++--- drivers/net/mlx5/mlx5_rxq.c | 6 ++---- drivers/net/mlx5/mlx5_txq.c | 6 ++---- drivers/regex/mlx5/mlx5_regex_control.c | 6 ++---- 8 files changed, 42 insertions(+), 19 deletions(-) diff --git a/drivers/common/mlx5/mlx5_common_mr.c b/drivers/common/mlx5/mlx5_common_mr.c index 2e039a4e70..8fd65484cf 100644 --- a/drivers/common/mlx5/mlx5_common_mr.c +++ b/drivers/common/mlx5/mlx5_common_mr.c @@ -271,6 +271,34 @@ mlx5_mr_btree_dump(struct mlx5_mr_btree *bt __rte_unused) #endif } +/** + * Initialize per-queue MR control descriptor. + * + * @param mr_ctrl + * Pointer to MR control structure. + * @param dev_gen_ptr + * Pointer to generation number of global cache. + * @param socket + * NUMA socket on which memory must be allocated. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_mr_ctrl_init(struct mlx5_mr_ctrl *mr_ctrl, uint32_t *dev_gen_ptr, + int socket) +{ + if (mr_ctrl == NULL) { + rte_errno = EINVAL; + return -rte_errno; + } + /* Save pointer of global generation number to check memory event. */ + mr_ctrl->dev_gen_ptr = dev_gen_ptr; + /* Initialize B-tree and allocate memory for bottom-half cache table. */ + return mlx5_mr_btree_init(&mr_ctrl->cache_bh, MLX5_MR_BTREE_CACHE_N, + socket); +} + /** * Find virtually contiguous memory chunk in a given MR. * diff --git a/drivers/common/mlx5/mlx5_common_mr.h b/drivers/common/mlx5/mlx5_common_mr.h index 15489cd399..1392d9b55a 100644 --- a/drivers/common/mlx5/mlx5_common_mr.h +++ b/drivers/common/mlx5/mlx5_common_mr.h @@ -124,6 +124,9 @@ mlx5_mr_lookup_lkey(struct mr_cache_entry *lkp_tbl, uint16_t *cached_idx, return UINT32_MAX; } +__rte_internal +int mlx5_mr_ctrl_init(struct mlx5_mr_ctrl *mr_ctrl, uint32_t *dev_gen_ptr, + int socket); __rte_internal int mlx5_mr_btree_init(struct mlx5_mr_btree *bt, int n, int socket); __rte_internal diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index 2baf82cad5..f786287986 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -108,6 +108,7 @@ INTERNAL { mlx5_mr_btree_free; mlx5_mr_btree_init; mlx5_mr_create_primary; + mlx5_mr_ctrl_init; mlx5_mr_dump_cache; mlx5_mr_flush_local_cache; mlx5_mr_free; diff --git a/drivers/compress/mlx5/mlx5_compress.c b/drivers/compress/mlx5/mlx5_compress.c index e1c45be259..7e297778ef 100644 --- a/drivers/compress/mlx5/mlx5_compress.c +++ b/drivers/compress/mlx5/mlx5_compress.c @@ -206,8 +206,8 @@ mlx5_compress_qp_setup(struct rte_compressdev *dev, uint16_t qp_id, return -rte_errno; } dev->data->queue_pairs[qp_id] = qp; - if (mlx5_mr_btree_init(&qp->mr_ctrl.cache_bh, MLX5_MR_BTREE_CACHE_N, - priv->dev_config.socket_id)) { + if (mlx5_mr_ctrl_init(&qp->mr_ctrl, &priv->mr_scache.dev_gen, + priv->dev_config.socket_id)) { DRV_LOG(ERR, "Cannot allocate MR Btree for qp %u.", (uint32_t)qp_id); rte_errno = ENOMEM; @@ -254,8 +254,6 @@ mlx5_compress_qp_setup(struct rte_compressdev *dev, uint16_t qp_id, DRV_LOG(ERR, "Can't change SQ state to ready."); goto err; } - /* Save pointer of global generation number to check memory event. */ - qp->mr_ctrl.dev_gen_ptr = &priv->mr_scache.dev_gen; DRV_LOG(INFO, "QP %u: SQN=0x%X CQN=0x%X entries num = %u", (uint32_t)qp_id, qp->sq.sq->id, qp->cq.cq->id, qp->entries_n); return 0; diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c index d553dd3df5..af32e0a2ac 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.c +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -712,14 +712,13 @@ mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, DRV_LOG(ERR, "Failed to register QP umem."); goto error; } - if (mlx5_mr_btree_init(&qp->mr_ctrl.cache_bh, MLX5_MR_BTREE_CACHE_N, - priv->dev_config.socket_id) != 0) { + if (mlx5_mr_ctrl_init(&qp->mr_ctrl, &priv->mr_scache.dev_gen, + priv->dev_config.socket_id) != 0) { DRV_LOG(ERR, "Cannot allocate MR Btree for qp %u.", (uint32_t)qp_id); rte_errno = ENOMEM; goto error; } - qp->mr_ctrl.dev_gen_ptr = &priv->mr_scache.dev_gen; attr.pd = priv->cdev->pdn; attr.uar_index = mlx5_os_get_devx_uar_page_id(priv->uar); attr.cqn = qp->cq_obj.cq->id; diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index f7f71f933e..479ea1324c 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -1452,13 +1452,11 @@ mlx5_rxq_new(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, goto error; } tmpl->type = MLX5_RXQ_TYPE_STANDARD; - if (mlx5_mr_btree_init(&tmpl->rxq.mr_ctrl.cache_bh, - MLX5_MR_BTREE_CACHE_N, socket)) { + if (mlx5_mr_ctrl_init(&tmpl->rxq.mr_ctrl, + &priv->sh->share_cache.dev_gen, socket)) { /* rte_errno is already set. */ goto error; } - /* Rx queues don't use this pointer, but we want a valid structure. */ - tmpl->rxq.mr_ctrl.dev_gen_ptr = &priv->sh->share_cache.dev_gen; tmpl->socket = socket; if (dev->data->dev_conf.intr_conf.rxq) tmpl->irq = 1; diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c index a9c8ce52c9..4b027d6807 100644 --- a/drivers/net/mlx5/mlx5_txq.c +++ b/drivers/net/mlx5/mlx5_txq.c @@ -1124,13 +1124,11 @@ mlx5_txq_new(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, rte_errno = ENOMEM; return NULL; } - if (mlx5_mr_btree_init(&tmpl->txq.mr_ctrl.cache_bh, - MLX5_MR_BTREE_CACHE_N, socket)) { + if (mlx5_mr_ctrl_init(&tmpl->txq.mr_ctrl, + &priv->sh->share_cache.dev_gen, socket)) { /* rte_errno is already set. */ goto error; } - /* Save pointer of global generation number to check memory event. */ - tmpl->txq.mr_ctrl.dev_gen_ptr = &priv->sh->share_cache.dev_gen; MLX5_ASSERT(desc > MLX5_TX_COMP_THRESH); tmpl->txq.offloads = conf->offloads | dev->data->dev_conf.txmode.offloads; diff --git a/drivers/regex/mlx5/mlx5_regex_control.c b/drivers/regex/mlx5/mlx5_regex_control.c index f58365665b..cd5dfa34e4 100644 --- a/drivers/regex/mlx5/mlx5_regex_control.c +++ b/drivers/regex/mlx5/mlx5_regex_control.c @@ -243,10 +243,8 @@ mlx5_regex_qp_setup(struct rte_regexdev *dev, uint16_t qp_ind, nb_sq_config++; } - /* Save pointer of global generation number to check memory event. */ - qp->mr_ctrl.dev_gen_ptr = &priv->mr_scache.dev_gen; - ret = mlx5_mr_btree_init(&qp->mr_ctrl.cache_bh, MLX5_MR_BTREE_CACHE_N, - rte_socket_id()); + ret = mlx5_mr_ctrl_init(&qp->mr_ctrl, &priv->mr_scache.dev_gen, + rte_socket_id()); if (ret) { DRV_LOG(ERR, "Error setting up mr btree"); goto err_btree;