From patchwork Mon Sep 18 14:36:16 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xueming Li X-Patchwork-Id: 28848 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A6D5D1B1AB; Mon, 18 Sep 2017 16:40:39 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 119551B19D for ; Mon, 18 Sep 2017 16:40:36 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from xuemingl@mellanox.com) with ESMTPS (AES256-SHA encrypted); 18 Sep 2017 17:40:34 +0300 Received: from dev-r630-06.mtbc.labs.mlnx (dev-r630-06.mtbc.labs.mlnx [10.12.205.180]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id v8IEeRir015498; Mon, 18 Sep 2017 17:40:33 +0300 Received: from dev-r630-06.mtbc.labs.mlnx (localhost [127.0.0.1]) by dev-r630-06.mtbc.labs.mlnx (8.14.7/8.14.7) with ESMTP id v8IEaVPh177480; Mon, 18 Sep 2017 22:36:31 +0800 Received: (from xuemingl@localhost) by dev-r630-06.mtbc.labs.mlnx (8.14.7/8.14.7/Submit) id v8IEaV7g177479; Mon, 18 Sep 2017 22:36:31 +0800 From: Xueming Li To: Nelio Laranjeiro , Adrien Mazarguil Cc: Xueming Li , dev@dpdk.org Date: Mon, 18 Sep 2017 22:36:16 +0800 Message-Id: <20170918143619.177422-4-xuemingl@mellanox.com> X-Mailer: git-send-email 2.13.3 In-Reply-To: <20170918143619.177422-1-xuemingl@mellanox.com> References: <20170918143619.177422-1-xuemingl@mellanox.com> In-Reply-To: <20170824140341.95471-1-xuemingl@mellanox.com> References: <20170824140341.95471-1-xuemingl@mellanox.com> Subject: [dpdk-dev] [PATCH v3 3/6] net/mlx5: allocate verbs object into shared memory X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" PMD uses Verbs object which were not available in the shared memory. This patch modify the location where Verbs objects are allocated (from process memory address space to shared memory address space) and thus allow a secondary process to use those object by mapping this shared memory space its own memory space. Signed-off-by: Xueming Li Acked-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index bfa38ba..11490d4 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -132,6 +132,52 @@ struct mlx5_args { } /** + * Verbs callback to allocate a memory. This function should allocate the space + * according to the size provided residing inside a huge page. + * Please note that all allocation must respect the alignment from libmlx5 + * (i.e. currently sysconf(_SC_PAGESIZE)). + * + * @param[in] size + * The size in bytes of the memory to allocate. + * @param[in] data + * A pointer to the callback data. + * + * @return + * a pointer to the allocate space. + */ +static void * +mlx5_alloc_verbs_buf(size_t size, void *data) +{ + struct priv *priv = data; + void *ret; + size_t alignment = sysconf(_SC_PAGESIZE); + + assert(data != NULL); + assert(!mlx5_is_secondary()); + ret = rte_malloc_socket(__func__, size, alignment, + priv->dev->device->numa_node); + DEBUG("Extern alloc size: %lu, align: %lu: %p", size, alignment, ret); + return ret; +} + +/** + * Verbs callback to free a memory. + * + * @param[in] ptr + * A pointer to the memory to free. + * @param[in] data + * A pointer to the callback data. + */ +static void +mlx5_free_verbs_buf(void *ptr, void *data __rte_unused) +{ + assert(data != NULL); + assert(!mlx5_is_secondary()); + DEBUG("Extern free request: %p", ptr); + rte_free(ptr); +} + +/** * DPDK callback to close the device. * * Destroy all queues and objects, free memory. @@ -826,6 +872,15 @@ struct mlx5_args { eth_dev->dev_ops = &mlx5_dev_ops; TAILQ_INIT(&priv->flows); + /* Hint libmlx5 to use PMD allocator for data plane resources */ + struct mlx5dv_ctx_allocators alctr = { + .alloc = &mlx5_alloc_verbs_buf, + .free = &mlx5_free_verbs_buf, + .data = priv, + }; + mlx5dv_set_context_attr(ctx, MLX5DV_CTX_ATTR_BUF_ALLOCATORS, + (void *)((uintptr_t)&alctr)); + /* Bring Ethernet device up. */ DEBUG("forcing Ethernet interface up"); priv_set_flags(priv, ~IFF_UP, IFF_UP);