From patchwork Tue Oct 19 20:55:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 102318 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CCD21A0C41; Tue, 19 Oct 2021 22:58:07 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E9C9341100; Tue, 19 Oct 2021 22:57:19 +0200 (CEST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2054.outbound.protection.outlook.com [40.107.236.54]) by mails.dpdk.org (Postfix) with ESMTP id EF61A4113A for ; Tue, 19 Oct 2021 22:56:52 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AmQOpl0/vrBIHupT5BgVn9oDySWaRiUCB488Uks+wBdT+OqUrle5wXnMZYVtaEyTb8QG+aJIBUOkSM4ewbDGCO2GyYKEfhH9N5mRfJl7KhLYFE8LFSq/QLr0m1Pp+zz2c4h9cXY9idSFIfaiQLB3eMrnL82/SkHdagETdLjyF2Nmh44MGgJmx6OdVes5DRPFIIcRsojfH6vrsfnD0DsDl8WOg+LH1c27iGyhGmMhkcRRs0AN5Hu93Gp9HlP2Inkz5+Dr8pz4fNaoYCbtvuO2JnM5p5ztpcW8oZDBWNTSgpIjU7tjr6bdJ0tei4fq8P3vnHrYr3Qw18PVC7SsKKNJQg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4h+8fKLMTnzFIXM4CRj3WhtDonOAqLcWQS5/Qp7wa3Y=; b=LTcs/emiiHXuyZZgMmPNxJU8fiWScGulPPQXY5NnOgvJ26cDjhYOwdXigiDF3aIe/ZS6ERKzom/AYL9pjtiuty4V6wOMimKlLSpOfxI5Neebers+d1Ngt/9SPqXLKqI/wbo13+k27cXjqUyBYlIqogKd0Pie4wrFZSo1tJZL+oxQM92aAqT0FLtybZFCFCvFZ/dpybMukwYGWAvZd+aUaCRhiX4QQnnjqn1cEjSXB42J1ia7lWL7CQUaiDFXR0XrlXDgU6qVuD1KL7ZjLurrg923GSc9FbUpB5lRu3MP1kws9QlLenVUubDG2Ii7FFKD8wwkiPni7QGYvpad8ksbSQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4h+8fKLMTnzFIXM4CRj3WhtDonOAqLcWQS5/Qp7wa3Y=; b=cFUuCL6vZUOaHNsK2PKdAcZaVh7all7KKi4wHYbQjcagxmbxi0W9iwRn2EfEYun2SbixCID3lk69vGND2rZNQOeIss0YVyjIxNjmfUFGWu3Jb4b1t6kmcClQQzWnNry6RlpZJKwKefSQ76SRzvKps4h2I7w8Hue+m8mIVy6BRePv5TeK7edF4HfiUBuOwhxN3p+pDfS3ZPnY8Fl4AE3dZ61Aehbsb3+Zm26RmJGccp6CehMrrPobCIqs4Fs0Xf9R0lbVmbmkerRW+p9QnsUJGfTw7nwQVZcvNWzIQdQyTNEjV2LtNWVY2ivKnBWKw+wJzzkTL41/N3O8rqPwV4Zqfg== Received: from BN9PR03CA0906.namprd03.prod.outlook.com (2603:10b6:408:107::11) by BN7PR12MB2659.namprd12.prod.outlook.com (2603:10b6:408:27::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4608.18; Tue, 19 Oct 2021 20:56:47 +0000 Received: from BN8NAM11FT035.eop-nam11.prod.protection.outlook.com (2603:10b6:408:107:cafe::96) by BN9PR03CA0906.outlook.office365.com (2603:10b6:408:107::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4608.17 via Frontend Transport; Tue, 19 Oct 2021 20:56:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; monjalon.net; dkim=none (message not signed) header.d=none;monjalon.net; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT035.mail.protection.outlook.com (10.13.177.116) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4608.15 via Frontend Transport; Tue, 19 Oct 2021 20:56:46 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 19 Oct 2021 20:56:45 +0000 From: To: CC: Matan Azrad , Thomas Monjalon , Michael Baum Date: Tue, 19 Oct 2021 23:55:57 +0300 Message-ID: <20211019205602.3188203-14-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211019205602.3188203-1-michaelba@nvidia.com> References: <20211006220350.2357487-1-michaelba@nvidia.com> <20211019205602.3188203-1-michaelba@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL101.nvidia.com (172.20.187.10) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1cdc9082-2acc-4aaa-dd8a-08d99342f6ee X-MS-TrafficTypeDiagnostic: BN7PR12MB2659: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4714; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: NNjFwzqGKBXnO/X3scrKlsvypG3KElQStT1ZUm68uk9WIz8KMh+smcFalncuOjo8KFwksmFcoB22UQKHxhY8q4syXiFUCO6ctjm72VTYhkDaMq9OttU4XHbCdtTtCmbw84szBUlK1Nl9uD8HmlyQq+eb4gIdA4mvEbJ4+R/GcS8GUcH5lISnwoMdYs0ydL2X0qAiDdEi6EPc9nlGoDEOH6ujHeX55eQB8azxyp761zLM1817KJPTTWulXAO1RW+g2xHZB1jIdPBUucSOYsYXgAwH6crhT1QbiJuLAEfnBB1JnXYQxtdcjKaXaUbJr6rHUnZNnKNA8527zz5LZaPM1Z28jBqZ6QEQHbHC6pwupC7Ej0dHbXEOZz7T6CJfGYoGxNxhLGn755zfm/CdKWEGD/mjdMMMoiTswEUNofrcj9KFSgMz5IvbUz+1bk0NKE/QnaGHYOp7XKLLvwL5xTWqFUGgCquqXDUv3dH8+7MQOCMz7xwYxWvceYchAGK5DwSOkI+jMoa3dOazwfsccL3m69ORy59ZZ9cY50Xd+K5aIJRE36+vja5f55ma7cW8xKf8jkcf0dDmrfJfqYYsESILaVHmVroKS3FLEa3PfJ0y0Qpb3eUSvBQ5RgL+ccaL+uWw2b9ZHjIFMkuNZmxQXiPcFVUh5iA0DqDUWbGjE9wJACWDQsD2raPQcKMHAAT+z9n3tF+n+cZmHD+Bb/ORTsL5VQ== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(36840700001)(46966006)(36860700001)(86362001)(82310400003)(356005)(107886003)(7636003)(47076005)(54906003)(83380400001)(316002)(426003)(36756003)(508600001)(7696005)(6666004)(55016002)(6286002)(2616005)(4326008)(336012)(8676002)(2876002)(70206006)(8936002)(186003)(16526019)(26005)(70586007)(1076003)(6916009)(5660300002)(2906002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Oct 2021 20:56:46.6621 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1cdc9082-2acc-4aaa-dd8a-08d99342f6ee X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT035.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR12MB2659 Subject: [dpdk-dev] [PATCH v3 13/18] common/mlx5: add MR ctrl init function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Michael Baum Add function for MR control structure initialization. This function include: - btree initialization. - dev_gen_ptr initialization. Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/common/mlx5/mlx5_common_mr.c | 28 +++++++++++++++++++++++++ drivers/common/mlx5/mlx5_common_mr.h | 3 +++ drivers/common/mlx5/version.map | 1 + drivers/compress/mlx5/mlx5_compress.c | 6 ++---- drivers/crypto/mlx5/mlx5_crypto.c | 5 ++--- drivers/net/mlx5/mlx5_rxq.c | 6 ++---- drivers/net/mlx5/mlx5_txq.c | 6 ++---- drivers/regex/mlx5/mlx5_regex_control.c | 6 ++---- 8 files changed, 42 insertions(+), 19 deletions(-) diff --git a/drivers/common/mlx5/mlx5_common_mr.c b/drivers/common/mlx5/mlx5_common_mr.c index 2e039a4e70..8fd65484cf 100644 --- a/drivers/common/mlx5/mlx5_common_mr.c +++ b/drivers/common/mlx5/mlx5_common_mr.c @@ -271,6 +271,34 @@ mlx5_mr_btree_dump(struct mlx5_mr_btree *bt __rte_unused) #endif } +/** + * Initialize per-queue MR control descriptor. + * + * @param mr_ctrl + * Pointer to MR control structure. + * @param dev_gen_ptr + * Pointer to generation number of global cache. + * @param socket + * NUMA socket on which memory must be allocated. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_mr_ctrl_init(struct mlx5_mr_ctrl *mr_ctrl, uint32_t *dev_gen_ptr, + int socket) +{ + if (mr_ctrl == NULL) { + rte_errno = EINVAL; + return -rte_errno; + } + /* Save pointer of global generation number to check memory event. */ + mr_ctrl->dev_gen_ptr = dev_gen_ptr; + /* Initialize B-tree and allocate memory for bottom-half cache table. */ + return mlx5_mr_btree_init(&mr_ctrl->cache_bh, MLX5_MR_BTREE_CACHE_N, + socket); +} + /** * Find virtually contiguous memory chunk in a given MR. * diff --git a/drivers/common/mlx5/mlx5_common_mr.h b/drivers/common/mlx5/mlx5_common_mr.h index 15489cd399..1392d9b55a 100644 --- a/drivers/common/mlx5/mlx5_common_mr.h +++ b/drivers/common/mlx5/mlx5_common_mr.h @@ -124,6 +124,9 @@ mlx5_mr_lookup_lkey(struct mr_cache_entry *lkp_tbl, uint16_t *cached_idx, return UINT32_MAX; } +__rte_internal +int mlx5_mr_ctrl_init(struct mlx5_mr_ctrl *mr_ctrl, uint32_t *dev_gen_ptr, + int socket); __rte_internal int mlx5_mr_btree_init(struct mlx5_mr_btree *bt, int n, int socket); __rte_internal diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index 44c4593888..6200c013fb 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -111,6 +111,7 @@ INTERNAL { mlx5_mr_btree_free; mlx5_mr_btree_init; mlx5_mr_create_primary; + mlx5_mr_ctrl_init; mlx5_mr_dump_cache; mlx5_mr_flush_local_cache; mlx5_mr_free; diff --git a/drivers/compress/mlx5/mlx5_compress.c b/drivers/compress/mlx5/mlx5_compress.c index 8fe65293a6..66b80a46c3 100644 --- a/drivers/compress/mlx5/mlx5_compress.c +++ b/drivers/compress/mlx5/mlx5_compress.c @@ -206,8 +206,8 @@ mlx5_compress_qp_setup(struct rte_compressdev *dev, uint16_t qp_id, return -rte_errno; } dev->data->queue_pairs[qp_id] = qp; - if (mlx5_mr_btree_init(&qp->mr_ctrl.cache_bh, MLX5_MR_BTREE_CACHE_N, - priv->dev_config.socket_id)) { + if (mlx5_mr_ctrl_init(&qp->mr_ctrl, &priv->mr_scache.dev_gen, + priv->dev_config.socket_id)) { DRV_LOG(ERR, "Cannot allocate MR Btree for qp %u.", (uint32_t)qp_id); rte_errno = ENOMEM; @@ -258,8 +258,6 @@ mlx5_compress_qp_setup(struct rte_compressdev *dev, uint16_t qp_id, ret = mlx5_devx_qp2rts(&qp->qp, 0); if (ret) goto err; - /* Save pointer of global generation number to check memory event. */ - qp->mr_ctrl.dev_gen_ptr = &priv->mr_scache.dev_gen; DRV_LOG(INFO, "QP %u: SQN=0x%X CQN=0x%X entries num = %u", (uint32_t)qp_id, qp->qp.qp->id, qp->cq.cq->id, qp->entries_n); return 0; diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c index ff4c67c0a0..77f0688ba0 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.c +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -677,14 +677,13 @@ mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, DRV_LOG(ERR, "Failed to create QP."); goto error; } - if (mlx5_mr_btree_init(&qp->mr_ctrl.cache_bh, MLX5_MR_BTREE_CACHE_N, - priv->dev_config.socket_id) != 0) { + if (mlx5_mr_ctrl_init(&qp->mr_ctrl, &priv->mr_scache.dev_gen, + priv->dev_config.socket_id) != 0) { DRV_LOG(ERR, "Cannot allocate MR Btree for qp %u.", (uint32_t)qp_id); rte_errno = ENOMEM; goto error; } - qp->mr_ctrl.dev_gen_ptr = &priv->mr_scache.dev_gen; /* * In Order to configure self loopback, when calling devx qp2rts the * remote QP id that is used is the id of the same QP. diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index 43ea890d2b..53c8c5439d 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -1449,13 +1449,11 @@ mlx5_rxq_new(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, goto error; } tmpl->type = MLX5_RXQ_TYPE_STANDARD; - if (mlx5_mr_btree_init(&tmpl->rxq.mr_ctrl.cache_bh, - MLX5_MR_BTREE_CACHE_N, socket)) { + if (mlx5_mr_ctrl_init(&tmpl->rxq.mr_ctrl, + &priv->sh->share_cache.dev_gen, socket)) { /* rte_errno is already set. */ goto error; } - /* Rx queues don't use this pointer, but we want a valid structure. */ - tmpl->rxq.mr_ctrl.dev_gen_ptr = &priv->sh->share_cache.dev_gen; tmpl->socket = socket; if (dev->data->dev_conf.intr_conf.rxq) tmpl->irq = 1; diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c index ffb252525d..f12510712a 100644 --- a/drivers/net/mlx5/mlx5_txq.c +++ b/drivers/net/mlx5/mlx5_txq.c @@ -1117,13 +1117,11 @@ mlx5_txq_new(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, rte_errno = ENOMEM; return NULL; } - if (mlx5_mr_btree_init(&tmpl->txq.mr_ctrl.cache_bh, - MLX5_MR_BTREE_CACHE_N, socket)) { + if (mlx5_mr_ctrl_init(&tmpl->txq.mr_ctrl, + &priv->sh->share_cache.dev_gen, socket)) { /* rte_errno is already set. */ goto error; } - /* Save pointer of global generation number to check memory event. */ - tmpl->txq.mr_ctrl.dev_gen_ptr = &priv->sh->share_cache.dev_gen; MLX5_ASSERT(desc > MLX5_TX_COMP_THRESH); tmpl->txq.offloads = conf->offloads | dev->data->dev_conf.txmode.offloads; diff --git a/drivers/regex/mlx5/mlx5_regex_control.c b/drivers/regex/mlx5/mlx5_regex_control.c index 545bbbcf89..6735e51976 100644 --- a/drivers/regex/mlx5/mlx5_regex_control.c +++ b/drivers/regex/mlx5/mlx5_regex_control.c @@ -242,10 +242,8 @@ mlx5_regex_qp_setup(struct rte_regexdev *dev, uint16_t qp_ind, nb_sq_config++; } - /* Save pointer of global generation number to check memory event. */ - qp->mr_ctrl.dev_gen_ptr = &priv->mr_scache.dev_gen; - ret = mlx5_mr_btree_init(&qp->mr_ctrl.cache_bh, MLX5_MR_BTREE_CACHE_N, - rte_socket_id()); + ret = mlx5_mr_ctrl_init(&qp->mr_ctrl, &priv->mr_scache.dev_gen, + rte_socket_id()); if (ret) { DRV_LOG(ERR, "Error setting up mr btree"); goto err_btree;