From patchwork Sun Jan 2 07:06:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 105544 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3946AA0093; Sun, 2 Jan 2022 08:07:08 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4012D41141; Sun, 2 Jan 2022 08:07:04 +0100 (CET) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2062.outbound.protection.outlook.com [40.107.220.62]) by mails.dpdk.org (Postfix) with ESMTP id 4F0E540698 for ; Sun, 2 Jan 2022 08:07:02 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dZ4rt7JeJo5Rz4wsu/aVvAWq8p6hIrtJSMjYh4d02Ry3uqy6JqNEPvLKRUrmbcwFb1clYBc0VRqfofErlYk3J6dg8tZ4YDXdy7AjkWe3llIKGDIMPv4rn/HEuakdL60m/AjFDA+6pkhIXBbVV7mvvsgoZrZkQAZD/ySZVfv5Gc0v1bv9otFWHuSLSRHF0tQJiCMYXZZfkRPrKrl4+W2QntgS5IOgYemLy4yyvcVKwM1o0o1oZSl9zMCBuWhmVkuUAU0sKMFkOYAuyFawD+GaMXFdUXADVecD1aDNw5DOWRJl41sPv1t8z0HkH36JZ0F10v7mZXNtfZbrfR/x1VYgag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/5osMn1o2lpKet5rZC+zD9QiI2GYfRkcQzj9rJFp4oc=; b=WVjPMCn5fQfMeuHK21OJiBhhRjcJ+mj659TFlJG3zWUQcp4DwnqL4LD4ABNZdOv06ejEXhUTz+nvNMaTYlnZRrIoxSQk4frlQEQ+BcSPiBDxrif0R+i0I1nxEJPaueBlzd/1fQQZqDiPLYFZQez8miW9/Rze2ROjcyNP40eNbc0y1NgY8hHmLwXBsvhooF5YgphWTTADwvqTEhJlJEy+WQET5mFl3Ykmr6Sihc3mB7WgxYOKOyZDpK4xgz18wqgVbBNJHMmXpMeh8kmMgGcQcdO0cOrNQs2asZ6AcZZM9tKXt1gYsWGRdlxQMvGsuHWq8zbTw/xFYoJIGKKNZfBwyQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.238) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/5osMn1o2lpKet5rZC+zD9QiI2GYfRkcQzj9rJFp4oc=; b=qQxEioEvYLFlFA4WycMKxIlDCr8ZoKI39raFAJ37/WWXfvuL4fyd5nrZHKX7e2UIhOJ6X62okbWy+H73CytlRZsQfFYwT4nIvYIM9qzw8OtOlEg1WJ1ykzIo20g0sV392aqq2V4FMppMFroM+TNq9bEo08SajdyP6ohzD/UkLUmz7o9d3YICoG2KlVfZDtq3o7cPJfhCu+hrbQe7ODzEBqHfb4zakvm1Dfr+q3nC1A8zEApbQs/LsY8+Gym2rZMhfx2wJWgIaTRA8YMVCLze0dYKkkLg/4qAdOD95D6gHINsuMkYax91zXBIcDBXVyewuVbMZ9JIqB6/SXhVp+tqPA== Received: from MW2PR16CA0023.namprd16.prod.outlook.com (2603:10b6:907::36) by DM6PR12MB3228.namprd12.prod.outlook.com (2603:10b6:5:18c::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4844.15; Sun, 2 Jan 2022 07:07:00 +0000 Received: from CO1NAM11FT033.eop-nam11.prod.protection.outlook.com (2603:10b6:907:0:cafe::1f) by MW2PR16CA0023.outlook.office365.com (2603:10b6:907::36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4844.14 via Frontend Transport; Sun, 2 Jan 2022 07:07:00 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.238) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.238 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.238; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.238) by CO1NAM11FT033.mail.protection.outlook.com (10.13.174.247) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4844.14 via Frontend Transport; Sun, 2 Jan 2022 07:07:00 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL105.nvidia.com (10.27.9.14) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Sun, 2 Jan 2022 07:06:59 +0000 Received: from nvidia.com (172.20.187.6) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.986.9; Sat, 1 Jan 2022 23:06:57 -0800 From: To: CC: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko , Michael Baum Subject: [RFC 1/3] common/mlx5: glue device and PD importation Date: Sun, 2 Jan 2022 09:06:36 +0200 Message-ID: <20220102070638.2211160-2-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220102070638.2211160-1-michaelba@nvidia.com> References: <20220102070638.2211160-1-michaelba@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 443b5278-8dfa-44a4-97fc-08d9cdbe78b8 X-MS-TrafficTypeDiagnostic: DM6PR12MB3228:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4714; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YiiToGXsI1U/n1R6cpGXyCEd+SHAVupPNu+LgG5XnPMOk83lKoFbTLefBbvbrv9FviMzmAj+mhuHNFYoICKXmdFlU5XLkxmzNRFAS/fpoATfaUnEY9RMurPkRqzIFITp1f10nSGMcpd1pJPJE+8IPf/tfM/JFg1TedDaxkNmbUAcrjZD0/Go21SJlnY2iBHU8M0Jeu3c95i+pQZZXyaYet68wJUHUh/UcbNzVQq6UgUE6670D8UBnrvImHkaVmBjDv2RTgvyqC7NVI55sHjHWjNiZllF6GBXFhV5msFoIm1pVCW8PRPbXGeXmsepcZA64NqjZeDKpYm8vjYs5lPyGHarHS9mjirBxizax3MXpnK6OY16kFetFxJ2F81cvQXeW/O5OgJDXLYMvxcZZd2qHYf8rBTIQ3neEzj3Y1+xLLYb7Zvxx0Z5G6QPhGVZGeewz4iNP4Kotj+OWKpXAOqi2Baqy8b4qlhJc7kEbsFT7J1lUtx4i4k4Sq4Edz5xo5J/phhbbZKmuxT/NOva1McVTnlKVEUs40EvgXZjrUS3TFO2nnZsjpKAGnicXaABm1YCk3L/mjkac+Ik0kbRc2FooyTpufZwgWltdAnpgs9bK5a2xeGSWwK+9R8YYEcQBToENR1GvGsB6p1cMiNIy+4HspWUY93/jx4hKcC/77RnL/HHEiOC95VwAoarcyv1Zp0o53qb6UYN+sxEjdLAsvkOyp1jJEblUB54SKf9zmZBIuyhp0Of8epeUqGcznTCTdjYmWvHk4Dpjbup63piehtv6lgfyZKzWaaKO9c0xscc5Og= X-Forefront-Antispam-Report: CIP:12.22.5.238; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(4636009)(40470700002)(46966006)(36840700001)(4326008)(70206006)(426003)(1076003)(70586007)(86362001)(2616005)(107886003)(2906002)(5660300002)(316002)(47076005)(6286002)(508600001)(6666004)(336012)(55016003)(8676002)(8936002)(83380400001)(36860700001)(16526019)(36756003)(40460700001)(7696005)(6916009)(26005)(186003)(356005)(81166007)(54906003)(2876002)(82310400004)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Jan 2022 07:07:00.1385 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 443b5278-8dfa-44a4-97fc-08d9cdbe78b8 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.238]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT033.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB3228 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Michael Baum Add support for rdma-core API to import device. The API gets ibv_context file descriptor and returns an ibv_context pointer that is associated with the given file descriptor. Add also support for rdma-core API to import PD. The API gets ibv_context and PD handle and returns a protection domain (PD) that is associated with the given handle in the given context. Signed-off-by: Michael Baum --- drivers/common/mlx5/linux/mlx5_glue.c | 21 +++++++++++++++++++++ drivers/common/mlx5/linux/mlx5_glue.h | 4 ++++ 2 files changed, 25 insertions(+) diff --git a/drivers/common/mlx5/linux/mlx5_glue.c b/drivers/common/mlx5/linux/mlx5_glue.c index bc6622053f..2f70e7f320 100644 --- a/drivers/common/mlx5/linux/mlx5_glue.c +++ b/drivers/common/mlx5/linux/mlx5_glue.c @@ -34,6 +34,18 @@ mlx5_glue_dealloc_pd(struct ibv_pd *pd) return ibv_dealloc_pd(pd); } +static struct ibv_pd * +mlx5_glue_import_pd(struct ibv_context *context, uint32_t pd_handle) +{ + return ibv_import_pd(context, pd_handle); +} + +static void +mlx5_glue_unimport_pd(struct ibv_pd *pd) +{ + ibv_unimport_pd(pd); +} + static struct ibv_device ** mlx5_glue_get_device_list(int *num_devices) { @@ -52,6 +64,12 @@ mlx5_glue_open_device(struct ibv_device *device) return ibv_open_device(device); } +static struct ibv_context * +mlx5_glue_import_device(int cmd_fd) +{ + return ibv_import_device(cmd_fd); +} + static int mlx5_glue_close_device(struct ibv_context *context) { @@ -1402,9 +1420,12 @@ const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue) { .fork_init = mlx5_glue_fork_init, .alloc_pd = mlx5_glue_alloc_pd, .dealloc_pd = mlx5_glue_dealloc_pd, + .import_pd = mlx5_glue_import_pd, + .unimport_pd = mlx5_glue_unimport_pd, .get_device_list = mlx5_glue_get_device_list, .free_device_list = mlx5_glue_free_device_list, .open_device = mlx5_glue_open_device, + .import_device = mlx5_glue_import_device, .close_device = mlx5_glue_close_device, .query_device = mlx5_glue_query_device, .query_device_ex = mlx5_glue_query_device_ex, diff --git a/drivers/common/mlx5/linux/mlx5_glue.h b/drivers/common/mlx5/linux/mlx5_glue.h index 4e6d31f263..5ed38e4fd9 100644 --- a/drivers/common/mlx5/linux/mlx5_glue.h +++ b/drivers/common/mlx5/linux/mlx5_glue.h @@ -151,9 +151,13 @@ struct mlx5_glue { int (*fork_init)(void); struct ibv_pd *(*alloc_pd)(struct ibv_context *context); int (*dealloc_pd)(struct ibv_pd *pd); + struct ibv_pd *(*import_pd)(struct ibv_context *context, + uint32_t pd_handle); + void (*unimport_pd)(struct ibv_pd *pd); struct ibv_device **(*get_device_list)(int *num_devices); void (*free_device_list)(struct ibv_device **list); struct ibv_context *(*open_device)(struct ibv_device *device); + struct ibv_context *(*import_device)(int cmd_fd); int (*close_device)(struct ibv_context *context); int (*query_device)(struct ibv_context *context, struct ibv_device_attr *device_attr); From patchwork Sun Jan 2 07:06:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 105545 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 57C10A0093; Sun, 2 Jan 2022 08:07:13 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 737BF406FF; Sun, 2 Jan 2022 08:07:06 +0100 (CET) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2049.outbound.protection.outlook.com [40.107.93.49]) by mails.dpdk.org (Postfix) with ESMTP id E972F41145 for ; Sun, 2 Jan 2022 08:07:04 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=d18SCcC9/pa7HeWB7M4X6B4iUphYkM6umEQUex4RzxJfnGH1I+VNycatZRbcYJpto17SHc5fE7h5bQq5CM+fU148zkG04r1VP1FbKhf0fRYwl8soL9FTblV5I4ntopJ2G5KwL017PXEL/ZEpKeqTonA5Cjx2LXIs5R99uIciTwvySJlQPxOlLNpHr8yD+Q4uW1P5rOOUdcjzlgv1MqnvkQGyv4m6KHRJvzw0y0SQC1R2QrWBUleD/4yDVHLbQn9tMjiVi3tsH5DwJ+GDBRzF/X3XXn5hqvEGMvHzBy+dT93mi6edHe30Ob773Yeq91mL0WhJd5OnDqCtCDHF7w/WLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OP0vx9dw8ftr97wXAZs+NJfH+skIMSpJExNrM0ugv7c=; b=QALA6+mlQLXHa7mFuI2u9Y1oinX/WAUfKGDnnhuszab9yAQfMJlZOx3H9CxEHFZMCVeaTcRHbWbVqT28Y3G0s9oinb9ML/DJccdMutnTmGIzY+6FyL3TYTpXYkeeC3CbEsFSjuyu7AapGdIqScjsOXS15O6t4OqM+K7HFtm8gS3I1TZFynAdHDHlfAzbWyRhqlyRmeWhj1WHU0WYr5EaXtuzP+OGLsuYrQm91N31aqvKAIut+KFqItEc2VVfJlbU1isB9c509Fvdz/MLwSBgeWA5jhonW4PpGOxFWTEVwNwxq7XXMqaWQaTwwlXHW8p/md8tR57qYjKvum0GL1gK4Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.234) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OP0vx9dw8ftr97wXAZs+NJfH+skIMSpJExNrM0ugv7c=; b=STaJfBPyP07WsuVCNZqI3n+kSdwRKM6AvEAgGfnzoUxOdu2xDQlHE4Vy0leUKbJJkidkX/0q854dKNBXlr9tRXLU17p+wnwcePa/vOxph3bJnNAHxK/DdD2jQrbrxBak9znRXeNbz4QzhTpA8B8QyOHJtVR8va3MhN601pKB7S4SJy9xpXCMACiQDIV9yOm2Wy7bGYI1nNT2ChdXI6g5hThVpJ31l/TYfwp9qpZXzUKJBqWTkBQGsHEMbjKMMIpclGhAtgJ9mwMOUlGki2ZTKiXbUIsXYNkSgxPyo+MkIbDDiZGUp05HIK0czHi2DABi5Z8sHeVi3HJkVxxxcop8Bw== Received: from DM5PR07CA0127.namprd07.prod.outlook.com (2603:10b6:3:13e::17) by DM5PR12MB2359.namprd12.prod.outlook.com (2603:10b6:4:b4::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4844.15; Sun, 2 Jan 2022 07:07:02 +0000 Received: from DM6NAM11FT054.eop-nam11.prod.protection.outlook.com (2603:10b6:3:13e:cafe::aa) by DM5PR07CA0127.outlook.office365.com (2603:10b6:3:13e::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4844.13 via Frontend Transport; Sun, 2 Jan 2022 07:07:02 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.234) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.234 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.234; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.234) by DM6NAM11FT054.mail.protection.outlook.com (10.13.173.95) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4844.14 via Frontend Transport; Sun, 2 Jan 2022 07:07:02 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL101.nvidia.com (10.27.9.10) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Sun, 2 Jan 2022 07:07:01 +0000 Received: from nvidia.com (172.20.187.6) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.986.9; Sat, 1 Jan 2022 23:06:59 -0800 From: To: CC: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko , Michael Baum Subject: [RFC 2/3] common/mlx5: add shared PD support Date: Sun, 2 Jan 2022 09:06:37 +0200 Message-ID: <20220102070638.2211160-3-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220102070638.2211160-1-michaelba@nvidia.com> References: <20220102070638.2211160-1-michaelba@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: daf375ac-ac4e-4f11-71fe-08d9cdbe79e4 X-MS-TrafficTypeDiagnostic: DM5PR12MB2359:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:17; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: f27QRoZ4Gj5XcSur1b/+1xuW3kXIY1XQvo3TLRKBi7Igq2R/pY66ZGCGMv9gyIgUkpeebDgF5o5gbffJl3OvQHuo4rOxj61kJ3rsnnX/cfA6EFMVIXaUtEJzOkNorIT9QJuB68kLxWdKHXFaYUP8L992CrD+k5O6L+SqvyyQqxAQ+pw2UheNwpx1Gh0PVoyaCPEHSbeKnJs0CUmLBoWAZ39Ah4XO6cD5HtR4qXgPATv0XfzY8oi82jfmn7K/Q8DgzambHNg2XKUsSF8r94BEUA5qLEosx7x/OLNXIXgbe9FJ+fc+3iD14RTyu6kAfNzBG5Nofsa0fxzjy1y4Gkh3Nl4vSdekjnfB4U4LwLjGUimy29ML5C/k79jgFG4F9NzbM53FaCMDEz6feU50jeELZ0I+m6LVFLSHktDzXUN3dc+9tD1XnkkRM0zzB3oq0iQsJryLgjCLVMpt31zAyFdCuMtByKSdRst4aTdiEF3uAopJP08BMTcY1qGZLJSOsyt0olwA0YIgEaSzzBaeTBUP7TVA1ei52vFCEnF2xjf2OM6Ih5CMh0cqRgm9iGCa/sqr75DKqvvpIU9Ttlw0WkmmCt6uSoONnDW2Mn4Mm0BQ9q6iPcUSx55gs1z3qMty1Vc7fZov8ff7zhsG0O78g5fqAb2eBQa16VPwxwpioyj+CcSvfe0lEXYMGCjvsnIv0L4y754rCFCeow+Tau7u/tL4IHW3uPiPmKUzF8DreFuyPdXf43GRjT3zl0AEGcctn4U3f5fQT26O2SNp9gDu51TlJYpeDzS2TFy6cRLlXHTr5S3F+8rlz1Sp4yi3FKO3IVar7IE0e4MjR3/auxgyFHY1sQ== X-Forefront-Antispam-Report: CIP:12.22.5.234; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(4636009)(36840700001)(40470700002)(46966006)(6666004)(6286002)(316002)(2906002)(36756003)(36860700001)(7696005)(81166007)(8676002)(4326008)(107886003)(26005)(70206006)(40460700001)(47076005)(16526019)(8936002)(186003)(70586007)(508600001)(2616005)(1076003)(6916009)(86362001)(336012)(5660300002)(54906003)(356005)(55016003)(82310400004)(83380400001)(2876002)(30864003)(426003)(36900700001)(309714004); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Jan 2022 07:07:02.0412 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: daf375ac-ac4e-4f11-71fe-08d9cdbe79e4 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.234]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT054.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB2359 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Michael Baum Add option to probe common device using import CTX/PD functions instead of create functions. This option requires accepting the context FD and the PD handle as devargs. Signed-off-by: Michael Baum --- drivers/common/mlx5/linux/mlx5_common_os.c | 160 +++++++++++++++++-- drivers/common/mlx5/linux/mlx5_common_os.h | 6 - drivers/common/mlx5/mlx5_common.c | 32 ++-- drivers/common/mlx5/mlx5_common.h | 6 +- drivers/common/mlx5/windows/mlx5_common_os.c | 29 +++- drivers/common/mlx5/windows/mlx5_common_os.h | 1 - 6 files changed, 196 insertions(+), 38 deletions(-) diff --git a/drivers/common/mlx5/linux/mlx5_common_os.c b/drivers/common/mlx5/linux/mlx5_common_os.c index 0d3e24e04e..a8226e987f 100644 --- a/drivers/common/mlx5/linux/mlx5_common_os.c +++ b/drivers/common/mlx5/linux/mlx5_common_os.c @@ -416,7 +416,7 @@ mlx5_glue_constructor(void) * @return * 0 on success, a negative errno value otherwise and rte_errno is set. */ -int +static int mlx5_os_pd_create(struct mlx5_common_device *cdev) { #ifdef HAVE_IBV_FLOW_DV_SUPPORT @@ -450,6 +450,65 @@ mlx5_os_pd_create(struct mlx5_common_device *cdev) #endif /* HAVE_IBV_FLOW_DV_SUPPORT */ } +/** + * Import Protection Domain object according to given pdn. + * + * @param[out] cdev + * Pointer to the mlx5 device. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +mlx5_os_pd_import(struct mlx5_common_device *cdev) +{ + struct mlx5_common_dev_config *config = &cdev->config; + + cdev->pd = mlx5_glue->import_pd(cdev->ctx, config->pd_handle); + if (cdev->pd == NULL) { + DRV_LOG(ERR, "Failed to import PD."); + return errno ? -errno : -ENOMEM; + } + cdev->pdn = config->pd_handle; + return 0; +} + +/** + * Prepare Protection Domain object and extract its pdn using DV API. + * + * @param[out] cdev + * Pointer to the mlx5 device. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_os_pd_prepare(struct mlx5_common_device *cdev) +{ + if (cdev->config.pd_handle == MLX5_ARG_UNSET) + return mlx5_os_pd_create(cdev); + else + return mlx5_os_pd_import(cdev); +} + +/** + * Release Protection Domain object. + * + * @param[out] cdev + * Pointer to the mlx5 device. + * + * @return + * 0 on success, a negative errno value otherwise. + */ +int +mlx5_os_pd_release(struct mlx5_common_device *cdev) +{ + if (cdev->config.pd_handle == MLX5_ARG_UNSET) + return mlx5_glue->dealloc_pd(cdev->pd); + mlx5_glue->unimport_pd(cdev->pd); + return 0; +} + static struct ibv_device * mlx5_os_get_ibv_device(const struct rte_pci_addr *addr) { @@ -648,28 +707,28 @@ mlx5_restore_doorbell_mapping_env(int value) /** * Function API to open IB device. * - * * @param cdev * Pointer to the mlx5 device. * @param classes * Chosen classes come from device arguments. * * @return - * 0 on success, a negative errno value otherwise and rte_errno is set. + * Pointer to ibv_context on success, NULL otherwise and rte_errno is set. */ -int -mlx5_os_open_device(struct mlx5_common_device *cdev, uint32_t classes) +static struct ibv_context * +mlx5_open_device(struct mlx5_common_device *cdev, uint32_t classes) { struct ibv_device *ibv; struct ibv_context *ctx = NULL; int dbmap_env; + MLX5_ASSERT(cdev->config.device_fd == MLX5_ARG_UNSET); if (classes & MLX5_CLASS_VDPA) ibv = mlx5_vdpa_get_ibv_dev(cdev->dev); else ibv = mlx5_os_get_ibv_dev(cdev->dev); if (!ibv) - return -rte_errno; + return NULL; DRV_LOG(INFO, "Dev information matches for device \"%s\".", ibv->name); /* * Configure environment variable "MLX5_BF_SHUT_UP" before the device @@ -682,29 +741,104 @@ mlx5_os_open_device(struct mlx5_common_device *cdev, uint32_t classes) ctx = mlx5_glue->dv_open_device(ibv); if (ctx) { cdev->config.devx = 1; - DRV_LOG(DEBUG, "DevX is supported."); } else if (classes == MLX5_CLASS_ETH) { /* The environment variable is still configured. */ ctx = mlx5_glue->open_device(ibv); if (ctx == NULL) goto error; - DRV_LOG(DEBUG, "DevX is NOT supported."); } else { goto error; } /* The device is created, no need for environment. */ mlx5_restore_doorbell_mapping_env(dbmap_env); - /* Hint libmlx5 to use PMD allocator for data plane resources */ - mlx5_set_context_attr(cdev->dev, ctx); - cdev->ctx = ctx; - return 0; + return ctx; error: rte_errno = errno ? errno : ENODEV; /* The device creation is failed, no need for environment. */ mlx5_restore_doorbell_mapping_env(dbmap_env); DRV_LOG(ERR, "Failed to open IB device \"%s\".", ibv->name); - return -rte_errno; + return NULL; } + +/** + * Function API to import IB device. + * + * @param cdev + * Pointer to the mlx5 device. + * + * @return + * Pointer to ibv_context on success, NULL otherwise and rte_errno is set. + */ +static struct ibv_context * +mlx5_import_device(struct mlx5_common_device *cdev) +{ + struct ibv_context *ctx = NULL; + + MLX5_ASSERT(cdev->config.device_fd != MLX5_ARG_UNSET); + ctx = mlx5_glue->import_device(cdev->config.device_fd); + if (!ctx) { + DRV_LOG(ERR, "Failed to import device for fd=%d.", + cdev->config.device_fd); + rte_errno = errno; + } + return ctx; +} + +/** + * Function API to prepare IB device. + * + * @param cdev + * Pointer to the mlx5 device. + * @param classes + * Chosen classes come from device arguments. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_os_open_device(struct mlx5_common_device *cdev, uint32_t classes) +{ + + struct ibv_context *ctx = NULL; + + if (cdev->config.device_fd == MLX5_ARG_UNSET) + ctx = mlx5_open_device(cdev, classes); + else + ctx = mlx5_import_device(cdev); + if (ctx == NULL) + return -rte_errno; + /* Hint libmlx5 to use PMD allocator for data plane resources */ + mlx5_set_context_attr(cdev->dev, ctx); + cdev->ctx = ctx; + return 0; +} + +/** + * Query HCA attributes. + * For remote context, it is check if DevX is supported. + * + * @param cdev + * Pointer to mlx5 device structure. + * + * @return + * 0 on success, a negative value otherwise. + */ +int +mlx5_os_query_hca_attr(struct mlx5_common_device *cdev) +{ + int ret; + + ret = mlx5_devx_cmd_query_hca_attr(cdev->ctx, &cdev->config.hca_attr); + if (ret) { + if (cdev->config.device_fd == MLX5_ARG_UNSET) { + rte_errno = ENOTSUP; + return -rte_errno; + } + cdev->config.devx = 0; + } + return 0; +} + int mlx5_get_device_guid(const struct rte_pci_addr *dev, uint8_t *guid, size_t len) { diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h index 83066e752d..246e8b2784 100644 --- a/drivers/common/mlx5/linux/mlx5_common_os.h +++ b/drivers/common/mlx5/linux/mlx5_common_os.h @@ -203,12 +203,6 @@ mlx5_os_get_devx_uar_page_id(void *uar) #endif } -static inline int -mlx5_os_dealloc_pd(void *pd) -{ - return mlx5_glue->dealloc_pd(pd); -} - __rte_internal static inline void * mlx5_os_umem_reg(void *ctx, void *addr, size_t size, uint32_t access) diff --git a/drivers/common/mlx5/mlx5_common.c b/drivers/common/mlx5/mlx5_common.c index f1650f94c6..bd7f0021d5 100644 --- a/drivers/common/mlx5/mlx5_common.c +++ b/drivers/common/mlx5/mlx5_common.c @@ -133,6 +133,10 @@ mlx5_common_args_check_handler(const char *key, const char *val, void *opaque) config->mr_mempool_reg_en = !!tmp; } else if (strcmp(key, "sys_mem_en") == 0) { config->sys_mem_en = !!tmp; + } else if (strcmp(key, "device_fd") == 0) { + config->device_fd = !!tmp; + } else if (strcmp(key, "pd_handle") == 0) { + config->pd_handle = !!tmp; } return 0; } @@ -160,6 +164,8 @@ mlx5_common_config_get(struct rte_devargs *devargs, config->mr_mempool_reg_en = 1; config->sys_mem_en = 0; config->dbnc = MLX5_ARG_UNSET; + config->device_fd = MLX5_ARG_UNSET; + config->pd_handle = MLX5_ARG_UNSET; if (devargs == NULL) return 0; kvlist = rte_kvargs_parse(devargs->args, NULL); @@ -492,7 +498,7 @@ static void mlx5_dev_hw_global_release(struct mlx5_common_device *cdev) { if (cdev->pd != NULL) { - claim_zero(mlx5_os_dealloc_pd(cdev->pd)); + claim_zero(mlx5_os_pd_release(cdev)); cdev->pd = NULL; } if (cdev->ctx != NULL) { @@ -522,19 +528,23 @@ mlx5_dev_hw_global_prepare(struct mlx5_common_device *cdev, uint32_t classes) if (ret < 0) return ret; /* Allocate Protection Domain object and extract its pdn. */ - ret = mlx5_os_pd_create(cdev); + ret = mlx5_os_pd_prepare(cdev); if (ret) goto error; - /* All actions taken below are relevant only when DevX is supported */ - if (cdev->config.devx == 0) - return 0; - /* Query HCA attributes. */ - ret = mlx5_devx_cmd_query_hca_attr(cdev->ctx, &cdev->config.hca_attr); - if (ret) { - DRV_LOG(ERR, "Unable to read HCA capabilities."); - rte_errno = ENOTSUP; - goto error; + /* + * TODO: write it again... + * All actions taken below are relevant only when DevX is supported + */ + if (cdev->config.devx || cdev->config.device_fd != MLX5_ARG_UNSET) { + /* Query HCA attributes. */ + ret = mlx5_os_query_hca_attr(cdev); + if (ret) { + DRV_LOG(ERR, "Unable to read HCA capabilities."); + rte_errno = ENOTSUP; + goto error; + } } + DRV_LOG(DEBUG, "DevX is %ssupported.", cdev->config.devx ? "" : "NOT "); return 0; error: mlx5_dev_hw_global_release(cdev); diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index e8809844af..78c8a29740 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -414,6 +414,8 @@ void mlx5_common_init(void); struct mlx5_common_dev_config { struct mlx5_hca_attr hca_attr; /* HCA attributes. */ int dbnc; /* Skip doorbell register write barrier. */ + int device_fd; /* Device file descriptor for importation. */ + int pd_handle; /* Protection Domain handle for importation. */ unsigned int devx:1; /* Whether devx interface is available or not. */ unsigned int sys_mem_en:1; /* The default memory allocator. */ unsigned int mr_mempool_reg_en:1; @@ -507,7 +509,9 @@ mlx5_devx_uar_release(struct mlx5_uar *uar); /* mlx5_common_os.c */ int mlx5_os_open_device(struct mlx5_common_device *cdev, uint32_t classes); -int mlx5_os_pd_create(struct mlx5_common_device *cdev); +int mlx5_os_query_hca_attr(struct mlx5_common_device *cdev); +int mlx5_os_pd_prepare(struct mlx5_common_device *cdev); +int mlx5_os_pd_release(struct mlx5_common_device *cdev); /* mlx5 PMD wrapped MR struct. */ struct mlx5_pmd_wrapped_mr { diff --git a/drivers/common/mlx5/windows/mlx5_common_os.c b/drivers/common/mlx5/windows/mlx5_common_os.c index 162c7476cc..d8410f57c1 100644 --- a/drivers/common/mlx5/windows/mlx5_common_os.c +++ b/drivers/common/mlx5/windows/mlx5_common_os.c @@ -25,21 +25,38 @@ mlx5_glue_constructor(void) { } +/** + * Query HCA attributes. + * + * @param cdev + * Pointer to mlx5 device structure. + * + * @return + * 0 on success, a negative value otherwise. + */ +int +mlx5_os_query_hca_attr(struct mlx5_common_device *cdev) +{ + return mlx5_devx_cmd_query_hca_attr(cdev->ctx, &cdev->config.hca_attr); +} + /** * Release PD. Releases a given mlx5_pd object * - * @param[in] pd - * Pointer to mlx5_pd. + * @param[in] cdev + * Pointer to the mlx5 device. * * @return * Zero if pd is released successfully, negative number otherwise. */ int -mlx5_os_dealloc_pd(void *pd) +mlx5_os_pd_release(struct mlx5_common_device *cdev) { + struct mlx5_pd *pd = cdev->pd; + if (!pd) return -EINVAL; - mlx5_devx_cmd_destroy(((struct mlx5_pd *)pd)->obj); + mlx5_devx_cmd_destroy(pd->obj); mlx5_free(pd); return 0; } @@ -47,14 +64,14 @@ mlx5_os_dealloc_pd(void *pd) /** * Allocate Protection Domain object and extract its pdn using DV API. * - * @param[out] dev + * @param[out] cdev * Pointer to the mlx5 device. * * @return * 0 on success, a negative value otherwise. */ int -mlx5_os_pd_create(struct mlx5_common_device *cdev) +mlx5_os_pd_prepare(struct mlx5_common_device *cdev) { struct mlx5_pd *pd; diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h index 3afce56cd9..a6a1b6890f 100644 --- a/drivers/common/mlx5/windows/mlx5_common_os.h +++ b/drivers/common/mlx5/windows/mlx5_common_os.h @@ -248,7 +248,6 @@ mlx5_os_devx_subscribe_devx_event(void *eventc, return -ENOTSUP; } -int mlx5_os_dealloc_pd(void *pd); __rte_internal void *mlx5_os_umem_reg(void *ctx, void *addr, size_t size, uint32_t access); __rte_internal From patchwork Sun Jan 2 07:06:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 105546 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3BED2A0093; Sun, 2 Jan 2022 08:07:20 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F415741158; Sun, 2 Jan 2022 08:07:07 +0100 (CET) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2081.outbound.protection.outlook.com [40.107.220.81]) by mails.dpdk.org (Postfix) with ESMTP id 1EE3B41148 for ; Sun, 2 Jan 2022 08:07:06 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gzDu+3yES6CBhqp1Zf8q4Pp/YhLb0h3/97JLRwSF4m29XwOaM/ekHi9ptYqALK6PP8cILSZD4/iCunZoYlaBBXaDVJsVImTJddjZlrp6MdbBbVr3mQbUm143dSASocGd9AJdUn+a2pueoTaD4OHQGWJTER9ifJaXLalcMPPtqU3qkGC6tNdXaFk0OOqOzEBBjlxrmgbdxX7pxjeb5Dz26lUeX8Noc0U7NBAUkkuzK2W2Wnu6WiAIb3l2fpK1oljFyEyQBCnhc1Uw8wbKveGUmX08V8jbfAMmORw/D0z01o4aoHc2V4VBvXFn73ZrerrP/hEFGFnVLQTgLeLPAHiF2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PasilrtK3r6slqZK1hFDDo7A2ZZII/8lNi23Gi198Rw=; b=OG0X2/f83wYaGM43gyQqq13JmO1cHSwUI+o9Os8e29WHext5/fcDyVpW46osEpCIDiRPy2yv1lfhulT53OGWRirnpthQMMFQ3D5EHiQ8JBNUMn30hqQDrgbRzcmuaEURR6Ta8iYtH4pJqpfYt3sXMjnw2+rdaH9QVXJSrtvLOZZw75sorcqAibrZYDOuuE+Q+PZG1eLNk/H369y6FLQRphehGrvdw7mgrmPfpjZJ93/Qi2ilEvAEoFk0SKHGJ9gbKTxGwGXo4TpI6oTp+uzSM8LTKWk7FygjlPQpEITF96w2FNUZRPROph7YXJTSwIGaoc4Zk61ur9QTtuyP5XA4+w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PasilrtK3r6slqZK1hFDDo7A2ZZII/8lNi23Gi198Rw=; b=OVVj9az6AWK464BYrFge7J9CJdOFcPkdROWljbVtgqgUDouVkyXSXOPpIuLngzBcSYXnHY7CsQKEvyc4/xvVi3791YuZwUvLojGmY5vENr7EfFhVMtid6Lb9YA4gI5qPI48eFPCZ5Cwww4bRcxTR0WGL2g3NQHmLtIgkPUfYohRNecKXRqfXRCh3rcA/XhSG7LN26UZRkoFWq31IX9VrB/nCTsXd9XAePPUFDBgXEZ4E2e09wx7Z4AR6GxgvuxePwHC7iy9anxaDZSG4sqDeGqKJ7uX/ThuNnxFxcU5g1zRgix8LuNrEC/okQV4G8ItuVpuXMFbC2V7NyYqqg41+pA== Received: from DM6PR14CA0062.namprd14.prod.outlook.com (2603:10b6:5:18f::39) by BYAPR12MB4613.namprd12.prod.outlook.com (2603:10b6:a03:a4::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4844.15; Sun, 2 Jan 2022 07:07:04 +0000 Received: from DM6NAM11FT062.eop-nam11.prod.protection.outlook.com (2603:10b6:5:18f:cafe::fa) by DM6PR14CA0062.outlook.office365.com (2603:10b6:5:18f::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4844.14 via Frontend Transport; Sun, 2 Jan 2022 07:07:03 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.236) by DM6NAM11FT062.mail.protection.outlook.com (10.13.173.40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4844.14 via Frontend Transport; Sun, 2 Jan 2022 07:07:03 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Sun, 2 Jan 2022 07:07:03 +0000 Received: from nvidia.com (172.20.187.6) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.986.9; Sat, 1 Jan 2022 23:07:01 -0800 From: To: CC: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko , Michael Baum Subject: [RFC 3/3] net/mlx5: add external Rx queue Date: Sun, 2 Jan 2022 09:06:38 +0200 Message-ID: <20220102070638.2211160-4-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220102070638.2211160-1-michaelba@nvidia.com> References: <20220102070638.2211160-1-michaelba@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 36e4bca8-9448-4c09-0029-08d9cdbe7ad6 X-MS-TrafficTypeDiagnostic: BYAPR12MB4613:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YXObsifOS52oLlmzPp5j6TTgILyv+K+aS7VtvzGZO4XNJy+/Ap3FD8qatb0q4jTRH2u+wxSySc0x4t9S4K1zNowMxSzYnW+cDDUMmpIE12rOo2+7Qmr6RkWNmC3rKkQlNQrtVfrHNf2BMwkShvI/IKAxRL4CBaVc7/joNURrJkOaO1RfOlHGa6Lk+fbcHQnTUMNHs3X0Lk9WR4/RsYLlAO+X7e6NUvzoKxOjGkR7w5hVlbK7Cv2jltdNzB0rHZWwdAojfjKsO6XpJSUYEwxXOky3KVlj/oFZDmpULTOw3t7GEDPBoL/gKNuugWvT246/O8SQUIF/Y/94Lk6igru0ndlQPLwEvOYeZukZCe13qZ+hRPF4oecaYvPJb0QkGyesvyuCgJ7e7vuIjWV1wPXf3xh2766spt8XmGN08dK0hSfei9V4zpFWDpjFGV7ggH6l5n7CNsU2GDp9OABWjijKf8kWNP3WFv7jtp9uoNwgKbL1ARG2P2JItBm5NOJI8d/DteYaopscwTNWodYRvHew0bhgVtMq7eBc7tha+AB1MVwbyvEGGFFoQ82AKeJP5FmbNjnvsKKUTuy3eHYXpbnb9ZFhKEBbzF6JE3QdOdoJYdlnFZV+4EGEACVSZu8ZMNEWAD5euA4cLK4yoNttY8bXVxbwEMUmKLFPZV/3zDf6w6Do6hhDrK9p8V2Tug/Am/zihNBVrTUqeY5v59drveF9XNuOC8UbrrPsqzDQvS9nwk0WPtxGV+WSN3nxmlNOs2gVn0B8HRUj6/NaOvDW1yiIK+4KbMKWOP+K4FYZtCGWmm8= X-Forefront-Antispam-Report: CIP:12.22.5.236; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(4636009)(40470700002)(46966006)(36840700001)(508600001)(83380400001)(7696005)(6666004)(1076003)(2906002)(81166007)(5660300002)(54906003)(2876002)(356005)(40460700001)(16526019)(316002)(26005)(70206006)(8676002)(426003)(6286002)(82310400004)(86362001)(336012)(186003)(107886003)(70586007)(36860700001)(2616005)(55016003)(8936002)(47076005)(6916009)(36756003)(4326008)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Jan 2022 07:07:03.6711 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 36e4bca8-9448-4c09-0029-08d9cdbe7ad6 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.236]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT062.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB4613 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Michael Baum Add API to map and unmap HW queue id to rte_flow queue id. Signed-off-by: Michael Baum --- drivers/net/mlx5/mlx5_rx.h | 6 +++++ drivers/net/mlx5/rte_pmd_mlx5.h | 45 +++++++++++++++++++++++++++++++++ 2 files changed, 51 insertions(+) diff --git a/drivers/net/mlx5/mlx5_rx.h b/drivers/net/mlx5/mlx5_rx.h index adb759c235..832f66004a 100644 --- a/drivers/net/mlx5/mlx5_rx.h +++ b/drivers/net/mlx5/mlx5_rx.h @@ -182,6 +182,12 @@ struct mlx5_rxq_priv { uint32_t hairpin_status; /* Hairpin binding status. */ }; +/* External RX queue descriptor. */ +struct mlx5_external_rxq { + uint32_t hw_id; /* Queue index in the Hardware. */ + uint32_t refcnt; /* Reference counter. */ +}; + /* mlx5_rxq.c */ extern uint8_t rss_hash_default_key[]; diff --git a/drivers/net/mlx5/rte_pmd_mlx5.h b/drivers/net/mlx5/rte_pmd_mlx5.h index e531e527b6..12e9f8d604 100644 --- a/drivers/net/mlx5/rte_pmd_mlx5.h +++ b/drivers/net/mlx5/rte_pmd_mlx5.h @@ -57,4 +57,49 @@ int rte_pmd_mlx5_get_dyn_flag_names(char *names[], unsigned int n); __rte_experimental int rte_pmd_mlx5_sync_flow(uint16_t port_id, uint32_t domains); +/** + * Minimum of + */ +#define MLX5_EXTERNAL_RX_QUEUE_ID_MIN (UINT16_MAX - 1024) + +/** + * + * + * @param[in] port_id + * The port identifier of the Ethernet device. + * @param[in] dpdk_idx + * Queue index in rte_flow. + * @param[in] hw_idx + * Queue index in hardware. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + * Possible values for rte_errno: + * - ENOMEM - allocation failure. + * - EEXIST - a mapping with the same rte_flow index already exists. + * - EINVAL - invalid rte_flow index, out of range. + * - ENODEV - there is no Ethernet device for this port id. + * - ENOTSUP - the port doesn't support external Rx. + */ +int rte_pmd_mlx5_external_rx_queue_id_map(uint16_t port_id, uint16_t dpdk_idx, + uint32_t hw_idx); + +/** + * + * + * @param[in] port_id + * The port identifier of the Ethernet device. + * @param[in] dpdk_idx + * Queue index in rte_flow. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + * Possible values for rte_errno: + * - EINVAL - invalid index, out of range or doesn't exist. + * - ENODEV - there is no Ethernet device for this port id. + * - ENOTSUP - the port doesn't support external Rx. + * + */ +int rte_pmd_mlx5_external_rx_queue_id_unmap(uint16_t port_id, uint16_t idx); + #endif