From patchwork Tue Oct 19 20:55:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 102316 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 69011A0C41; Tue, 19 Oct 2021 22:57:49 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9CD414117D; Tue, 19 Oct 2021 22:57:03 +0200 (CEST) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2075.outbound.protection.outlook.com [40.107.223.75]) by mails.dpdk.org (Postfix) with ESMTP id 5C33941176 for ; Tue, 19 Oct 2021 22:56:47 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RauYW9YZYuDqGTAIBW4QIsmzqzShQG1n+dgCAHGfvxsXBT+wy+3RQ69WYYRJtNr39dEbkZfAfFM4oCTzRSA3VIQMPTCO91PWBrMuCF9vbS1/EiIxCfQDS9H+6Cm5CNmtT9qXrJo36Y17m7c861OC0uTBTSey4qtra/Hamjr9vcg7w1Q0dfwEpPuLszj+mHzvK2UILj08iKvhlkO9Ev6XnxmPXA5dOOMGIqYb+IRTuB7myZuQALkuO8Y1daHFEzEnhb4xHNjLzCV+NUOj1uSYbw9t6dG+WFT2WAsFw6hxqByT3ZOJRuTy9A6i6QsThP6knUM8xu2k92RdMmcT7Htpnw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=i6LzPHiglo5LesLy4q0MxHU2lEosVTpAcqE+YdOjHzU=; b=gWHT9THg4cmJMFo5EgFo9UxYGuwUrovAtm0xRAgByxkWHr1Ib1aMq2CXZoTl/SdFnJTYNScqT5WXUicA1NIv6C72FDJlgN3S2DjVS2XnBsTVB2QiTstQtwnYM/rVvUb0cLnKz/vycCNXcvD9MGBYJLRJu5ApaiCzipMceMCtKiAEb0QzZk5TPClKw+1AWpeVUWBPe3JvL4jeM8I+EQYNmD6tAQTMAhBYjGhcdy88ROT9ccSc10/7ZLt9hUWd7pE8XRIgCjlhz2oJ7fFZ7g3HPlXMw3AndZPUyAPToFdze8BA+W/et00FZJlc4CxzLOlGc2FqZIVFO2xcMLNOCSvxfw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=i6LzPHiglo5LesLy4q0MxHU2lEosVTpAcqE+YdOjHzU=; b=O0zqY8tHC3p71qElt8ZT4jFQ2+LHIqhkXNhRlSOC7k3zcx7YGohEka0DkoPOkNsEdvceUHb2LX25N2cLaw969KQsJ3oH83bHEJG7NScFme6HF+Kep/toPc7Kw6ulkDwe4Kc37Az/p/d+Ol9JXTffrihFitMs9uHgH7DF9lpiEn1VdVOwCErX1NZxwDd/LPhsP8gawy5C/BmLCCFuSa1yNRJmn1YNw6NJnKl2coDOQva2aESF4qFZIXto9x4mtLXUA1+fqDZWv4N6+VWjnax+aEBvu+vOmJ3Yhnzqv2LBOMiKi9MEmazkmqLTbKhix9Fv4g+gsHLrQJwYGtK25y852Q== Received: from BN9PR03CA0930.namprd03.prod.outlook.com (2603:10b6:408:107::35) by CY4PR12MB1575.namprd12.prod.outlook.com (2603:10b6:910:f::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4608.16; Tue, 19 Oct 2021 20:56:45 +0000 Received: from BN8NAM11FT035.eop-nam11.prod.protection.outlook.com (2603:10b6:408:107:cafe::2a) by BN9PR03CA0930.outlook.office365.com (2603:10b6:408:107::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4608.16 via Frontend Transport; Tue, 19 Oct 2021 20:56:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; monjalon.net; dkim=none (message not signed) header.d=none;monjalon.net; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT035.mail.protection.outlook.com (10.13.177.116) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4608.15 via Frontend Transport; Tue, 19 Oct 2021 20:56:45 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 19 Oct 2021 20:56:43 +0000 From: To: CC: Matan Azrad , Thomas Monjalon , Michael Baum Date: Tue, 19 Oct 2021 23:55:56 +0300 Message-ID: <20211019205602.3188203-13-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211019205602.3188203-1-michaelba@nvidia.com> References: <20211006220350.2357487-1-michaelba@nvidia.com> <20211019205602.3188203-1-michaelba@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL101.nvidia.com (172.20.187.10) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 46c4e44b-ad66-4b8d-0d63-08d99342f5f7 X-MS-TrafficTypeDiagnostic: CY4PR12MB1575: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:238; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: TdKSdeEtYzBsoONYSlyw2ktgDtTpqF34cOhffiBLaAOmpNSFuug9oQx/LCTgxxmvpLVmbbIZi3kUFzQLZQgG78JZ67547riPDic8OgrrUz7xxW/Vh0a5y4sMBYfArrgAyp/MZ+JLoHifbQCS1E23F6Yu8kg05cIzwIFpDu1bT8rVKYNenyV8IPBdTY2fTklS69yXR8yJQsCv4AwJg/+gIEg9+cLGAJbwwPYlxLcvuFzQF/t+s0pMwOGpNzcfQANnyGk3uceMWIIo1BcGAtynSioQV3VK+Cfvh40FvJXi+699Py/pnEg6bYdxSodFC+gRk0FuY1x7Gt3bFivF1ZIYVIwZ+phbuJLCP7xG3Hg3jqlG+m/Np4XmP2mTG+xHPiJMxXElAprx8I9SVDdrl/qG7gS/dAzuf86EkUssyak8tPOxAhA8BmjKz5UR8L0gNjfzpTwDd3YgJlzmeBXP+Gs10fcS9c7hm5w1xqFad2L0wt5IkYx4YwZyXs+UQzyjMQzRDU+dJaW9Tlg2+eCT67MmWkoTR8m7JN3IhfAfbPRtB6TXiJUQODb+rjOcgo/wdBU5PTNy/X1BSOWcsjRhChKPONGrT70FLXkP7qRTNfMlubSiLJe4UpLt2xyAAVvpwldLmIpPK9AGr3aMvHcIE1BuJGNMhTdLgAdf7YigC0M8usZRIu8EeT1rbB3VJfiKw6PvHMM+WlTvNXA8uNml4C1tSg== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(8676002)(5660300002)(83380400001)(55016002)(508600001)(86362001)(36860700001)(2906002)(82310400003)(26005)(47076005)(36756003)(6916009)(16526019)(356005)(316002)(2616005)(7696005)(4326008)(70586007)(70206006)(54906003)(2876002)(336012)(107886003)(7636003)(8936002)(186003)(1076003)(426003)(6286002)(6666004); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Oct 2021 20:56:45.1700 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 46c4e44b-ad66-4b8d-0d63-08d99342f5f7 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT035.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR12MB1575 Subject: [dpdk-dev] [PATCH v3 12/18] net/mlx5: remove redundancy in MR file X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Michael Baum This patch remove two redundant things from MR file: 1. mr_find_contig_memsegs_data structure which is moved to common file before. 2. External memory mechanism - mlx5_tx_update_ext_mp function. Since commit [1] which added support for DMA map and unmap, external mem must be configured by the user using rte_mem_map function and no need to handle this in pmd. [1] commit 989e999d9305 ("net/mlx5: support PCI device DMA map and unmap") Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_mr.c | 142 +------------------------------------ drivers/net/mlx5/mlx5_tx.h | 2 - 2 files changed, 1 insertion(+), 143 deletions(-) diff --git a/drivers/net/mlx5/mlx5_mr.c b/drivers/net/mlx5/mlx5_mr.c index 4d884f7295..9ce973d95c 100644 --- a/drivers/net/mlx5/mlx5_mr.c +++ b/drivers/net/mlx5/mlx5_mr.c @@ -17,19 +17,6 @@ #include "mlx5_rx.h" #include "mlx5_tx.h" -struct mr_find_contig_memsegs_data { - uintptr_t addr; - uintptr_t start; - uintptr_t end; - const struct rte_memseg_list *msl; -}; - -struct mr_update_mp_data { - struct rte_eth_dev *dev; - struct mlx5_mr_ctrl *mr_ctrl; - int ret; -}; - /** * Callback for memory event. This can be called from both primary and secondary * process. @@ -134,70 +121,7 @@ mlx5_tx_mb2mr_bh(struct mlx5_txq_data *txq, struct rte_mbuf *mb) } /* Fallback for generic mechanism in corner cases. */ } - lkey = mlx5_tx_addr2mr_bh(txq, addr); - if (lkey == UINT32_MAX && rte_errno == ENXIO) { - /* Mempool may have externally allocated memory. */ - return mlx5_tx_update_ext_mp(txq, addr, mlx5_mb2mp(mb)); - } - return lkey; -} - -/** - * Called during rte_mempool_mem_iter() by mlx5_mr_update_ext_mp(). - * - * Externally allocated chunk is registered and a MR is created for the chunk. - * The MR object is added to the global list. If memseg list of a MR object - * (mr->msl) is null, the MR object can be regarded as externally allocated - * memory. - * - * Once external memory is registered, it should be static. If the memory is - * freed and the virtual address range has different physical memory mapped - * again, it may cause crash on device due to the wrong translation entry. PMD - * can't track the free event of the external memory for now. - */ -static void -mlx5_mr_update_ext_mp_cb(struct rte_mempool *mp, void *opaque, - struct rte_mempool_memhdr *memhdr, - unsigned mem_idx __rte_unused) -{ - struct mr_update_mp_data *data = opaque; - struct rte_eth_dev *dev = data->dev; - struct mlx5_priv *priv = dev->data->dev_private; - struct mlx5_dev_ctx_shared *sh = priv->sh; - struct mlx5_mr_ctrl *mr_ctrl = data->mr_ctrl; - struct mlx5_mr *mr = NULL; - uintptr_t addr = (uintptr_t)memhdr->addr; - size_t len = memhdr->len; - struct mr_cache_entry entry; - uint32_t lkey; - - MLX5_ASSERT(rte_eal_process_type() == RTE_PROC_PRIMARY); - /* If already registered, it should return. */ - rte_rwlock_read_lock(&sh->share_cache.rwlock); - lkey = mlx5_mr_lookup_cache(&sh->share_cache, &entry, addr); - rte_rwlock_read_unlock(&sh->share_cache.rwlock); - if (lkey != UINT32_MAX) - return; - DRV_LOG(DEBUG, "port %u register MR for chunk #%d of mempool (%s)", - dev->data->port_id, mem_idx, mp->name); - mr = mlx5_create_mr_ext(sh->cdev->pd, addr, len, mp->socket_id, - sh->share_cache.reg_mr_cb); - if (!mr) { - DRV_LOG(WARNING, - "port %u unable to allocate a new MR of" - " mempool (%s).", - dev->data->port_id, mp->name); - data->ret = -1; - return; - } - rte_rwlock_write_lock(&sh->share_cache.rwlock); - LIST_INSERT_HEAD(&sh->share_cache.mr_list, mr, mr); - /* Insert to the global cache table. */ - mlx5_mr_insert_cache(&sh->share_cache, mr); - rte_rwlock_write_unlock(&sh->share_cache.rwlock); - /* Insert to the local cache table */ - mlx5_mr_addr2mr_bh(sh->cdev->pd, &priv->mp_id, &sh->share_cache, - mr_ctrl, addr, sh->cdev->config.mr_ext_memseg_en); + return mlx5_tx_addr2mr_bh(txq, addr); } /** @@ -331,67 +255,3 @@ mlx5_net_dma_unmap(struct rte_device *rte_dev, void *addr, rte_rwlock_write_unlock(&sh->share_cache.rwlock); return 0; } - -/** - * Register MR for entire memory chunks in a Mempool having externally allocated - * memory and fill in local cache. - * - * @param dev - * Pointer to Ethernet device. - * @param mr_ctrl - * Pointer to per-queue MR control structure. - * @param mp - * Pointer to registering Mempool. - * - * @return - * 0 on success, -1 on failure. - */ -static uint32_t -mlx5_mr_update_ext_mp(struct rte_eth_dev *dev, struct mlx5_mr_ctrl *mr_ctrl, - struct rte_mempool *mp) -{ - struct mr_update_mp_data data = { - .dev = dev, - .mr_ctrl = mr_ctrl, - .ret = 0, - }; - - rte_mempool_mem_iter(mp, mlx5_mr_update_ext_mp_cb, &data); - return data.ret; -} - -/** - * Register MR entire memory chunks in a Mempool having externally allocated - * memory and search LKey of the address to return. - * - * @param dev - * Pointer to Ethernet device. - * @param addr - * Search key. - * @param mp - * Pointer to registering Mempool where addr belongs. - * - * @return - * LKey for address on success, UINT32_MAX on failure. - */ -uint32_t -mlx5_tx_update_ext_mp(struct mlx5_txq_data *txq, uintptr_t addr, - struct rte_mempool *mp) -{ - struct mlx5_txq_ctrl *txq_ctrl = - container_of(txq, struct mlx5_txq_ctrl, txq); - struct mlx5_mr_ctrl *mr_ctrl = &txq->mr_ctrl; - struct mlx5_priv *priv = txq_ctrl->priv; - - if (rte_eal_process_type() != RTE_PROC_PRIMARY) { - DRV_LOG(WARNING, - "port %u using address (%p) from unregistered mempool" - " having externally allocated memory" - " in secondary process, please create mempool" - " prior to rte_eth_dev_start()", - PORT_ID(priv), (void *)addr); - return UINT32_MAX; - } - mlx5_mr_update_ext_mp(ETH_DEV(priv), mr_ctrl, mp); - return mlx5_tx_addr2mr_bh(txq, addr); -} diff --git a/drivers/net/mlx5/mlx5_tx.h b/drivers/net/mlx5/mlx5_tx.h index e722738682..cdbcf659df 100644 --- a/drivers/net/mlx5/mlx5_tx.h +++ b/drivers/net/mlx5/mlx5_tx.h @@ -239,8 +239,6 @@ int mlx5_tx_burst_mode_get(struct rte_eth_dev *dev, uint16_t tx_queue_id, /* mlx5_mr.c */ uint32_t mlx5_tx_mb2mr_bh(struct mlx5_txq_data *txq, struct rte_mbuf *mb); -uint32_t mlx5_tx_update_ext_mp(struct mlx5_txq_data *txq, uintptr_t addr, - struct rte_mempool *mp); /* mlx5_tx_empw.c */