From patchwork Wed Jun 7 04:24:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Gupta, Nipun" X-Patchwork-Id: 128267 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B0E3242C46; Wed, 7 Jun 2023 06:24:48 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9B312427F5; Wed, 7 Jun 2023 06:24:48 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2072.outbound.protection.outlook.com [40.107.93.72]) by mails.dpdk.org (Postfix) with ESMTP id A2DE840A84 for ; Wed, 7 Jun 2023 06:24:47 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lLjeNUacFt9k52JmhCa2EtJS5lW7zNPDCrtrGEfCeafpe3w6xhgBPx7fCiBPLYdklujKzXKyhpg1Y5esoVvebJSjnUQCARm6gniYm99eu4VT89Cl7qZ7+4bfi+Uf6alx1GTWOhIRHm9FcOWDjzPx5J3Gzo+IlJEwA7cJxfPPk6UxHGUyoa3mDx4LkEKYay/9BilLk0TTiPALKwJLLbIrKvj01m6dL4lVkVyhzr6TlEXKSY9r5NDnQl2KYURWPQ1vSFVKmTdOOrkefjPF/pbn4QKPco2HC7SVINxvvZq6dIiijB9GEdagTTEtA/uPF3Xp6qEE53U/tiTm54KULDxDiw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+r1cakJoQg81VGMOOKf5U/hIJb6yqlyu6PaLUWp1Cz8=; b=IfHuR64OSNpK04h+IjtA2U5xRJkMDYSSDuvT0aindxv9w8DR/gfpN76SU1oKOVmQq/ANlVrUym7WJ+ekvmvQ/mVSOka2tG43U/rl16CvVNiDLrlqMfLE/NE5wH2vm+SCCuCJK+nHo3VhLrHi+G9v9F0MY7DZpOkpFEW73Gt2SfkyL85ossc7p4rAG+m1Ft+bqZFR1ML+dSXBb4NN23Zclrfm232KaaVlWgexyp/Btn6qNshwSC0jVVfccjYg62sc5Q75paf/IxALNNUT9YC31a3eceDlwj6+kqP4ltwE0/81YMRbD04qg8dhbdpmEk9e+RsYlvycx74xM+REx0RoZg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=dpdk.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+r1cakJoQg81VGMOOKf5U/hIJb6yqlyu6PaLUWp1Cz8=; b=aVfV+wrAkrSJmawgijDUXwKUsgHAXYnYRR9OUsh/bgjqMEQ4KEHSW/fmPDhWS6/EayZdf+kVia7QjIrzHAN6/T5jVJWrzgYTsckbx4PU8H9LMWU0xFJldmd8rXh83/fNcor2w26gCyy8MpBBfbfXENzZFB+ZC67KGIDMrHX56WU= Received: from CYXPR02CA0006.namprd02.prod.outlook.com (2603:10b6:930:cf::10) by DS7PR12MB8323.namprd12.prod.outlook.com (2603:10b6:8:da::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.19; Wed, 7 Jun 2023 04:24:44 +0000 Received: from CY4PEPF0000E9D2.namprd03.prod.outlook.com (2603:10b6:930:cf:cafe::90) by CYXPR02CA0006.outlook.office365.com (2603:10b6:930:cf::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.34 via Frontend Transport; Wed, 7 Jun 2023 04:24:44 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CY4PEPF0000E9D2.mail.protection.outlook.com (10.167.241.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6477.13 via Frontend Transport; Wed, 7 Jun 2023 04:24:44 +0000 Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 6 Jun 2023 23:24:43 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 6 Jun 2023 23:24:43 -0500 Received: from xhdipdslab41.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Tue, 6 Jun 2023 23:24:40 -0500 From: Nipun Gupta To: , , , , , CC: , , , Nipun Gupta Subject: [PATCH v8 1/4] bus/cdx: introduce AMD CDX bus Date: Wed, 7 Jun 2023 09:54:22 +0530 Message-ID: <20230607042425.30714-2-nipun.gupta@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230607042425.30714-1-nipun.gupta@amd.com> References: <20230124140746.594066-1-nipun.gupta@amd.com> <20230607042425.30714-1-nipun.gupta@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9D2:EE_|DS7PR12MB8323:EE_ X-MS-Office365-Filtering-Correlation-Id: 247ebd84-1799-4af6-529b-08db670f1ee3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: cYjbBJYMjhisSFEiclg6DRv4/IlGgCPwpy1S1vXSliFlct7Ri91e/Ip/6J5SxCCa8qWFkSUzHN4tEOo5UntnVpWRJKBcDAujUrTMWyHBo2jSk5/diT2s3nmhH6UPWTXH+cQrul1fUHk+XtElC1fXewMsGeRIlYK92I56WG8vt9PSt6kg3JuMS8ZneSy1rcWqjgQzjWcBDjjrrpLtqZn/RHL98Z30Dsg+gcUiy3iyxmYaar2cf5N6ro77yBuBTamXWxqO0XMWrvbrPMhk2gkuqpvxu36mZmlOktuIKMKp0UZ/9d2Q2j1sEw3zZmRtnBkoQEliyp5XaWEwB+A8cYAwUH9oCh1SnpxQev5umXv/O2zZEYQwuJNA7AgDwR17SIU5iYxyoXA81fLw7LNa1nV23oYeN2WB9ZS302U50PsFX9BH8UIhoJpspicuHOe2mAWibGXCpUalQ9bk7vRkHXDUSvnk4rfmGfRHCfcbjwBQ5KEcdaoa5a83Zqh4wsQr87ZqEG9taVwIuJoeGIZvgULOSOB/AmAV5XezT3Hdux7vDU/qVmbaaMf9bnubr3CtPJ7hXcouwndgKU/ZRVH76s6wnpFGyKVGk7nuY+M9GZTZ1LkMhROm23GgHVvvCrJ+dd56ucL++w7PBu7+4Ww3zhCZmX6hz+F/GD+pcl0nk3mSkS9jaelIWhR2aNtL8gNOc8+/qT7wVO8duWg6rjEFpLLMvhN2sfq4fkgY2h1jPPQigoe5PdsSytk8q4jkEgpvSEElWTMWVWuU4d+zbC62QnbVwA== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230028)(4636009)(376002)(346002)(396003)(39860400002)(136003)(451199021)(46966006)(36840700001)(40470700004)(36756003)(478600001)(5660300002)(110136005)(40460700003)(54906003)(6666004)(8936002)(8676002)(316002)(4326008)(70206006)(70586007)(41300700001)(44832011)(2906002)(30864003)(81166007)(40480700001)(356005)(82740400003)(1076003)(26005)(86362001)(186003)(47076005)(2616005)(36860700001)(83380400001)(336012)(426003)(82310400005)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2023 04:24:44.2348 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 247ebd84-1799-4af6-529b-08db670f1ee3 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000E9D2.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB8323 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org AMD CDX bus supports multiple type of devices, which can be exposed to user-space via vfio-cdx. vfio-cdx provides the MMIO IO_MEMORY regions as well as the DMA interface for the device (IOMMU). This support aims to enable the DPDK to support the cdx devices in user-space using VFIO interface. Signed-off-by: Nipun Gupta Acked-by: Ferruh Yigit --- MAINTAINERS | 5 + doc/guides/rel_notes/release_23_07.rst | 6 + drivers/bus/cdx/bus_cdx_driver.h | 166 ++++++++ drivers/bus/cdx/cdx.c | 503 +++++++++++++++++++++++++ drivers/bus/cdx/cdx_logs.h | 37 ++ drivers/bus/cdx/cdx_vfio.c | 426 +++++++++++++++++++++ drivers/bus/cdx/meson.build | 13 + drivers/bus/cdx/private.h | 53 +++ drivers/bus/cdx/version.map | 10 + drivers/bus/meson.build | 1 + 10 files changed, 1220 insertions(+) create mode 100644 drivers/bus/cdx/bus_cdx_driver.h create mode 100644 drivers/bus/cdx/cdx.c create mode 100644 drivers/bus/cdx/cdx_logs.h create mode 100644 drivers/bus/cdx/cdx_vfio.c create mode 100644 drivers/bus/cdx/meson.build create mode 100644 drivers/bus/cdx/private.h create mode 100644 drivers/bus/cdx/version.map diff --git a/MAINTAINERS b/MAINTAINERS index a5219926ab..c4b2b3565b 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -569,6 +569,11 @@ M: Parav Pandit M: Xueming Li F: drivers/bus/auxiliary/ +AMD CDX bus driver +M: Nipun Gupta +M: Nikhil Agarwal +F: drivers/bus/cdx/ + Intel FPGA bus M: Rosen Xu F: drivers/bus/ifpga/ diff --git a/doc/guides/rel_notes/release_23_07.rst b/doc/guides/rel_notes/release_23_07.rst index a9b1293689..7c6bb2b894 100644 --- a/doc/guides/rel_notes/release_23_07.rst +++ b/doc/guides/rel_notes/release_23_07.rst @@ -55,6 +55,12 @@ New Features Also, make sure to start the actual text at the margin. ======================================================= +* **Added AMD CDX bus support.** + + CDX bus driver has been added to support AMD CDX bus, which operates + on FPGA based CDX devices. The CDX devices are memory mapped on system + bus for embedded CPUs. + Removed Items ------------- diff --git a/drivers/bus/cdx/bus_cdx_driver.h b/drivers/bus/cdx/bus_cdx_driver.h new file mode 100644 index 0000000000..9a2be6481a --- /dev/null +++ b/drivers/bus/cdx/bus_cdx_driver.h @@ -0,0 +1,166 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2022-2023, Advanced Micro Devices, Inc. + */ + +#ifndef BUS_CDX_DRIVER_H +#define BUS_CDX_DRIVER_H + +/** + * @file + * AMD CDX bus interface + */ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include + +#include +#include +#include + +/* Forward declarations */ +struct rte_cdx_device; +struct rte_cdx_driver; +struct rte_cdx_bus; + +#define RTE_CDX_BUS_DEVICES_PATH "/sys/bus/cdx/devices" + +#define RTE_CDX_MAX_RESOURCE 4 + +/** Any CDX device identifier (vendor, device). */ +#define RTE_CDX_ANY_ID (0xffff) + +#define RTE_PMD_REGISTER_CDX_TABLE(name, table) \ +static const char DRV_EXP_TAG(name, cdx_tbl_export)[] __rte_used = \ +RTE_STR(table) + +/** + * A structure describing an ID for a CDX driver. Each driver provides a + * table of these IDs for each device that it supports. + */ +struct rte_cdx_id { + uint16_t vendor_id; /**< Vendor ID. */ + uint16_t device_id; /**< Device ID. */ +}; + +/** + * A structure describing a CDX device. + */ +struct rte_cdx_device { + RTE_TAILQ_ENTRY(rte_cdx_device) next; /**< Next probed CDX device. */ + struct rte_device device; /**< Inherit core device */ + char name[RTE_DEV_NAME_MAX_LEN]; /**< Device name */ + struct rte_cdx_id id; /**< CDX ID. */ + struct rte_mem_resource mem_resource[RTE_CDX_MAX_RESOURCE]; + /**< CDX Memory Resource */ +}; + +/** + * @internal + * Helper macro for drivers that need to convert to struct rte_cdx_device. + */ +#define RTE_DEV_TO_CDX_DEV(ptr) \ + container_of(ptr, struct rte_cdx_device, device) + +#define RTE_DEV_TO_CDX_DEV_CONST(ptr) \ + container_of(ptr, const struct rte_cdx_device, device) + +#define RTE_ETH_DEV_TO_CDX_DEV(eth_dev) RTE_DEV_TO_CDX_DEV((eth_dev)->device) + +#ifdef __cplusplus +/** C++ macro used to help building up tables of device IDs. */ +#define RTE_CDX_DEVICE(vend, dev) \ + (vend), \ + (dev) +#else +/** Macro used to help building up tables of device IDs. */ +#define RTE_CDX_DEVICE(vend, dev) \ + .vendor_id = (vend), \ + .device_id = (dev) +#endif + +/** + * Initialisation function for the driver called during CDX probing. + */ +typedef int (rte_cdx_probe_t)(struct rte_cdx_driver *, struct rte_cdx_device *); + +/** + * Uninitialisation function for the driver called during hotplugging. + */ +typedef int (rte_cdx_remove_t)(struct rte_cdx_device *); + +/** + * A structure describing a CDX driver. + */ +struct rte_cdx_driver { + RTE_TAILQ_ENTRY(rte_cdx_driver) next; /**< Next in list. */ + struct rte_driver driver; /**< Inherit core driver. */ + struct rte_cdx_bus *bus; /**< CDX bus reference. */ + rte_cdx_probe_t *probe; /**< Device probe function. */ + rte_cdx_remove_t *remove; /**< Device remove function. */ + const struct rte_cdx_id *id_table; /**< ID table, NULL terminated. */ + uint32_t drv_flags; /**< Flags RTE_CDX_DRV_*. */ +}; + +/** + * Map the CDX device resources in user space virtual memory address. + * + * @param dev + * A pointer to a rte_cdx_device structure describing the device + * to use. + * + * @return + * 0 on success, <0 on error. + */ +__rte_internal +int rte_cdx_map_device(struct rte_cdx_device *dev); + +/** + * Unmap this device. + * + * @param dev + * A pointer to a rte_cdx_device structure describing the device + * to use. + */ +__rte_internal +void rte_cdx_unmap_device(struct rte_cdx_device *dev); + +/** + * Register a CDX driver. + * + * @param driver + * A pointer to a rte_cdx_driver structure describing the driver + * to be registered. + */ +__rte_internal +void rte_cdx_register(struct rte_cdx_driver *driver); + +/** + * Helper for CDX device registration from driver (eth, crypto, raw) instance. + */ +#define RTE_PMD_REGISTER_CDX(nm, cdx_drv) \ + RTE_INIT(cdxinitfn_ ##nm) \ + {\ + (cdx_drv).driver.name = RTE_STR(nm);\ + rte_cdx_register(&cdx_drv); \ + } \ + RTE_PMD_EXPORT_NAME(nm, __COUNTER__) + +/** + * Unregister a CDX driver. + * + * @param driver + * A pointer to a rte_cdx_driver structure describing the driver + * to be unregistered. + */ +__rte_internal +void rte_cdx_unregister(struct rte_cdx_driver *driver); + +#ifdef __cplusplus +} +#endif + +#endif /* BUS_CDX_DRIVER_H */ diff --git a/drivers/bus/cdx/cdx.c b/drivers/bus/cdx/cdx.c new file mode 100644 index 0000000000..38515e7fda --- /dev/null +++ b/drivers/bus/cdx/cdx.c @@ -0,0 +1,503 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2022-2023, Advanced Micro Devices, Inc. + */ + +/* + * Architecture Overview + * ===================== + * CDX is a Hardware Architecture designed for AMD FPGA devices. It + * consists of sophisticated mechanism for interaction between FPGA, + * Firmware and the APUs (Application CPUs). + * + * Firmware resides on RPU (Realtime CPUs) which interacts with + * the FPGA program manager and the APUs. The RPU provides memory-mapped + * interface (RPU if) which is used to communicate with APUs. + * + * The diagram below shows an overview of the AMD CDX architecture: + * + * +--------------------------------------+ + * | DPDK | + * | DPDK CDX drivers | + * | | | + * | DPDK AMD CDX bus | + * | | | + * +-----------------------------|--------+ + * | + * +-----------------------------|--------+ + * | Application CPUs (APU) | | + * | | | + * | VFIO CDX driver | + * | Linux OS | | + * | Linux AMD CDX bus | + * | | | + * +-----------------------------|--------+ + * | + * | + * +------------------------| RPU if |----+ + * | | | + * | V | + * | Realtime CPUs (RPU) | + * | | + * +--------------------------------------+ + * | + * +---------------------|----------------+ + * | FPGA | | + * | +-----------------------+ | + * | | | | | + * | +-------+ +-------+ +-------+ | + * | | dev 1 | | dev 2 | | dev 3 | | + * | +-------+ +-------+ +-------+ | + * +--------------------------------------+ + * + * The RPU firmware extracts the device information from the loaded FPGA + * image and implements a mechanism that allows the APU drivers to + * enumerate such devices (device personality and resource details) via + * a dedicated communication channel. + * + * VFIO CDX driver provides the CDX device resources like MMIO and interrupts + * to map to user-space. DPDK CDX bus uses sysfs interface and the vfio-cdx + * driver to discover and initialize the CDX devices for user-space + * applications. + */ + +/** + * @file + * CDX probing using Linux sysfs. + */ + +#include +#include + +#include +#include +#include +#include +#include + +#include + +#include "bus_cdx_driver.h" +#include "cdx_logs.h" +#include "private.h" + +#define CDX_BUS_NAME cdx +#define CDX_DEV_PREFIX "cdx-" + +/* CDX Bus iterators */ +#define FOREACH_DEVICE_ON_CDXBUS(p) \ + RTE_TAILQ_FOREACH(p, &rte_cdx_bus.device_list, next) + +#define FOREACH_DRIVER_ON_CDXBUS(p) \ + RTE_TAILQ_FOREACH(p, &rte_cdx_bus.driver_list, next) + +struct rte_cdx_bus rte_cdx_bus; + +/* Add a device to CDX bus */ +static void +cdx_add_device(struct rte_cdx_device *cdx_dev) +{ + TAILQ_INSERT_TAIL(&rte_cdx_bus.device_list, cdx_dev, next); +} + +static int +cdx_get_kernel_driver_by_path(const char *filename, char *driver_name, + size_t len) +{ + int count; + char path[PATH_MAX]; + char *name; + + if (!filename || !driver_name) + return -1; + + count = readlink(filename, path, PATH_MAX); + if (count >= PATH_MAX) + return -1; + + /* For device does not have a driver */ + if (count < 0) + return 1; + + path[count] = '\0'; + + name = strrchr(path, '/'); + if (name) { + strlcpy(driver_name, name + 1, len); + return 0; + } + + return -1; +} + +int rte_cdx_map_device(struct rte_cdx_device *dev) +{ + return cdx_vfio_map_resource(dev); +} + +void rte_cdx_unmap_device(struct rte_cdx_device *dev) +{ + cdx_vfio_unmap_resource(dev); +} + +static struct rte_devargs * +cdx_devargs_lookup(const char *dev_name) +{ + struct rte_devargs *devargs; + + RTE_EAL_DEVARGS_FOREACH("cdx", devargs) { + if (strcmp(devargs->name, dev_name) == 0) + return devargs; + } + return NULL; +} + +static bool +cdx_ignore_device(const char *dev_name) +{ + struct rte_devargs *devargs = cdx_devargs_lookup(dev_name); + + switch (rte_cdx_bus.bus.conf.scan_mode) { + case RTE_BUS_SCAN_ALLOWLIST: + if (devargs && devargs->policy == RTE_DEV_ALLOWED) + return false; + break; + case RTE_BUS_SCAN_UNDEFINED: + case RTE_BUS_SCAN_BLOCKLIST: + if (devargs == NULL || devargs->policy != RTE_DEV_BLOCKED) + return false; + break; + } + return true; +} + +/* + * Scan one cdx sysfs entry, and fill the devices list from it. + * It checks if the CDX device is bound to vfio-cdx driver. In case + * the device is vfio bound, it reads the vendor and device id and + * stores it for device-driver matching. + */ +static int +cdx_scan_one(const char *dirname, const char *dev_name) +{ + char filename[PATH_MAX]; + struct rte_cdx_device *dev = NULL; + char driver[PATH_MAX]; + unsigned long tmp; + int ret; + + dev = calloc(1, sizeof(*dev)); + if (!dev) + return -ENOMEM; + + dev->device.bus = &rte_cdx_bus.bus; + memcpy(dev->name, dev_name, RTE_DEV_NAME_MAX_LEN); + dev->device.name = dev->name; + + /* parse driver */ + snprintf(filename, sizeof(filename), "%s/driver", dirname); + ret = cdx_get_kernel_driver_by_path(filename, driver, sizeof(driver)); + if (ret < 0) { + CDX_BUS_ERR("Fail to get kernel driver"); + ret = -1; + goto err; + } + + /* + * Check if device is bound to 'vfio-cdx' driver, so that user-space + * can gracefully access the device. + */ + if (ret || strcmp(driver, "vfio-cdx")) { + ret = 0; + goto err; + } + + /* get vendor id */ + snprintf(filename, sizeof(filename), "%s/vendor", dirname); + if (eal_parse_sysfs_value(filename, &tmp) < 0) { + ret = -1; + goto err; + } + dev->id.vendor_id = (uint16_t)tmp; + + /* get device id */ + snprintf(filename, sizeof(filename), "%s/device", dirname); + if (eal_parse_sysfs_value(filename, &tmp) < 0) { + free(dev); + return -1; + } + dev->id.device_id = (uint16_t)tmp; + + cdx_add_device(dev); + + return 0; + +err: + free(dev); + return ret; +} + +/* + * Scan the content of the CDX bus, and the devices in the devices + * list. + */ +static int +cdx_scan(void) +{ + struct dirent *e; + DIR *dir; + char dirname[PATH_MAX]; + + dir = opendir(RTE_CDX_BUS_DEVICES_PATH); + if (dir == NULL) { + CDX_BUS_ERR("%s(): opendir failed: %s", __func__, + strerror(errno)); + return -1; + } + + while ((e = readdir(dir)) != NULL) { + if (e->d_name[0] == '.') + continue; + + if (cdx_ignore_device(e->d_name)) + continue; + + snprintf(dirname, sizeof(dirname), "%s/%s", + RTE_CDX_BUS_DEVICES_PATH, e->d_name); + + if (cdx_scan_one(dirname, e->d_name) < 0) + goto error; + } + closedir(dir); + return 0; + +error: + closedir(dir); + return -1; +} + +/* map a particular resource from a file */ +void * +cdx_map_resource(void *requested_addr, int fd, uint64_t offset, size_t size, + int additional_flags) +{ + void *mapaddr; + + /* Map the cdx MMIO memory resource of device */ + mapaddr = rte_mem_map(requested_addr, size, + RTE_PROT_READ | RTE_PROT_WRITE, + RTE_MAP_SHARED | additional_flags, fd, offset); + if (mapaddr == NULL) { + CDX_BUS_ERR("%s(): cannot map resource(%d, %p, 0x%zx, 0x%"PRIx64"): %s (%p)", + __func__, fd, requested_addr, size, offset, + rte_strerror(rte_errno), mapaddr); + } + CDX_BUS_DEBUG("CDX MMIO memory mapped at %p", mapaddr); + + return mapaddr; +} + +/* unmap a particular resource */ +void +cdx_unmap_resource(void *requested_addr, size_t size) +{ + if (requested_addr == NULL) + return; + + /* Unmap the CDX memory resource of device */ + if (rte_mem_unmap(requested_addr, size)) { + CDX_BUS_ERR("%s(): cannot mem unmap(%p, %#zx): %s", __func__, + requested_addr, size, rte_strerror(rte_errno)); + } + CDX_BUS_DEBUG("CDX memory unmapped at %p", requested_addr); +} +/* + * Match the CDX Driver and Device using device id and vendor id. + */ +static bool +cdx_match(const struct rte_cdx_driver *cdx_drv, + const struct rte_cdx_device *cdx_dev) +{ + const struct rte_cdx_id *id_table; + + for (id_table = cdx_drv->id_table; id_table->vendor_id != 0; + id_table++) { + /* check if device's identifiers match the driver's ones */ + if (id_table->vendor_id != cdx_dev->id.vendor_id && + id_table->vendor_id != RTE_CDX_ANY_ID) + continue; + if (id_table->device_id != cdx_dev->id.device_id && + id_table->device_id != RTE_CDX_ANY_ID) + continue; + + return 1; + } + + return 0; +} + +/* + * If vendor id and device id match, call the probe() function of the + * driver. + */ +static int +cdx_probe_one_driver(struct rte_cdx_driver *dr, + struct rte_cdx_device *dev) +{ + const char *dev_name = dev->name; + bool already_probed; + int ret; + + /* The device is not blocked; Check if driver supports it */ + if (!cdx_match(dr, dev)) + /* Match of device and driver failed */ + return 1; + + already_probed = rte_dev_is_probed(&dev->device); + if (already_probed) { + CDX_BUS_INFO("Device %s is already probed", dev_name); + return -EEXIST; + } + + CDX_BUS_DEBUG(" probe device %s using driver: %s", dev_name, + dr->driver.name); + + ret = cdx_vfio_map_resource(dev); + if (ret != 0) { + CDX_BUS_ERR("CDX map device failed: %d", ret); + goto error_map_device; + } + + /* call the driver probe() function */ + ret = dr->probe(dr, dev); + if (ret) { + CDX_BUS_ERR("Probe CDX driver: %s device: %s failed: %d", + dr->driver.name, dev_name, ret); + goto error_probe; + } else { + dev->device.driver = &dr->driver; + } + + return ret; + +error_probe: + cdx_vfio_unmap_resource(dev); +error_map_device: + return ret; +} + +/* + * If vendor/device ID match, call the probe() function of all + * registered driver for the given device. Return < 0 if initialization + * failed, return 1 if no driver is found for this device. + */ +static int +cdx_probe_all_drivers(struct rte_cdx_device *dev) +{ + struct rte_cdx_driver *dr = NULL; + int rc = 0; + + FOREACH_DRIVER_ON_CDXBUS(dr) { + rc = cdx_probe_one_driver(dr, dev); + if (rc < 0) + /* negative value is an error */ + return rc; + if (rc > 0) + /* positive value means driver doesn't support it */ + continue; + return 0; + } + return 1; +} + +/* + * Scan the content of the CDX bus, and call the probe() function for + * all registered drivers that have a matching entry in its id_table + * for discovered devices. + */ +static int +cdx_probe(void) +{ + struct rte_cdx_device *dev = NULL; + size_t probed = 0, failed = 0; + int ret = 0; + + FOREACH_DEVICE_ON_CDXBUS(dev) { + probed++; + + ret = cdx_probe_all_drivers(dev); + if (ret < 0) { + CDX_BUS_ERR("Requested device %s cannot be used", + dev->name); + rte_errno = errno; + failed++; + ret = 0; + } + } + + return (probed && probed == failed) ? -1 : 0; +} + +static int +cdx_parse(const char *name, void *addr) +{ + const char **out = addr; + int ret; + + ret = strncmp(name, CDX_DEV_PREFIX, strlen(CDX_DEV_PREFIX)); + + if (ret == 0 && addr) + *out = name; + + return ret; +} + +/* register a driver */ +void +rte_cdx_register(struct rte_cdx_driver *driver) +{ + TAILQ_INSERT_TAIL(&rte_cdx_bus.driver_list, driver, next); + driver->bus = &rte_cdx_bus; +} + +/* unregister a driver */ +void +rte_cdx_unregister(struct rte_cdx_driver *driver) +{ + TAILQ_REMOVE(&rte_cdx_bus.driver_list, driver, next); + driver->bus = NULL; +} + +static struct rte_device * +cdx_find_device(const struct rte_device *start, rte_dev_cmp_t cmp, + const void *data) +{ + const struct rte_cdx_device *cdx_start; + struct rte_cdx_device *cdx_dev; + + if (start != NULL) { + cdx_start = RTE_DEV_TO_CDX_DEV_CONST(start); + cdx_dev = TAILQ_NEXT(cdx_start, next); + } else { + cdx_dev = TAILQ_FIRST(&rte_cdx_bus.device_list); + } + while (cdx_dev != NULL) { + if (cmp(&cdx_dev->device, data) == 0) + return &cdx_dev->device; + cdx_dev = TAILQ_NEXT(cdx_dev, next); + } + return NULL; +} + +struct rte_cdx_bus rte_cdx_bus = { + .bus = { + .scan = cdx_scan, + .probe = cdx_probe, + .find_device = cdx_find_device, + .parse = cdx_parse, + }, + .device_list = TAILQ_HEAD_INITIALIZER(rte_cdx_bus.device_list), + .driver_list = TAILQ_HEAD_INITIALIZER(rte_cdx_bus.driver_list), +}; + +RTE_REGISTER_BUS(cdx, rte_cdx_bus.bus); +RTE_LOG_REGISTER_DEFAULT(cdx_logtype_bus, NOTICE); diff --git a/drivers/bus/cdx/cdx_logs.h b/drivers/bus/cdx/cdx_logs.h new file mode 100644 index 0000000000..a1046ce544 --- /dev/null +++ b/drivers/bus/cdx/cdx_logs.h @@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2022-2023, Advanced Micro Devices, Inc. + */ + +#ifndef CDX_LOGS_H +#define CDX_LOGS_H + +extern int cdx_logtype_bus; + +#define CDX_BUS_LOG(level, fmt, args...) \ + rte_log(RTE_LOG_ ## level, cdx_logtype_bus, "cdx: " fmt "\n", \ + ##args) + +/* Debug logs with Function names */ +#define CDX_BUS_DEBUG(fmt, args...) \ + rte_log(RTE_LOG_DEBUG, cdx_logtype_bus, "cdx: %s(): " fmt "\n", \ + __func__, ##args) + +#define CDX_BUS_INFO(fmt, args...) \ + CDX_BUS_LOG(INFO, fmt, ## args) +#define CDX_BUS_ERR(fmt, args...) \ + CDX_BUS_LOG(ERR, fmt, ## args) +#define CDX_BUS_WARN(fmt, args...) \ + CDX_BUS_LOG(WARNING, fmt, ## args) + +/* DP Logs, toggled out at compile time if level lower than current level */ +#define CDX_BUS_DP_LOG(level, fmt, args...) \ + RTE_LOG_DP(level, PMD, fmt, ## args) + +#define CDX_BUS_DP_DEBUG(fmt, args...) \ + CDX_BUS_DP_LOG(DEBUG, fmt, ## args) +#define CDX_BUS_DP_INFO(fmt, args...) \ + CDX_BUS_DP_LOG(INFO, fmt, ## args) +#define CDX_BUS_DP_WARN(fmt, args...) \ + CDX_BUS_DP_LOG(WARNING, fmt, ## args) + +#endif /* CDX_LOGS_H */ diff --git a/drivers/bus/cdx/cdx_vfio.c b/drivers/bus/cdx/cdx_vfio.c new file mode 100644 index 0000000000..86290f0450 --- /dev/null +++ b/drivers/bus/cdx/cdx_vfio.c @@ -0,0 +1,426 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2022-2023, Advanced Micro Devices, Inc. + */ + +/** + * @file + * CDX probing using Linux VFIO. + * + * This code tries to determine if the CDX device is bound to VFIO driver, + * and initialize it (map MMIO regions, set up interrupts) if that's the case. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "bus_cdx_driver.h" +#include "cdx_logs.h" +#include "private.h" + +/** + * A structure describing a CDX mapping. + */ +struct cdx_map { + void *addr; + char *path; + uint64_t offset; + uint64_t size; +}; + +/** + * A structure describing a mapped CDX resource. + * For multi-process we need to reproduce all CDX mappings in secondary + * processes, so save them in a tailq. + */ +struct mapped_cdx_resource { + TAILQ_ENTRY(mapped_cdx_resource) next; + char name[RTE_DEV_NAME_MAX_LEN]; /**< CDX device name */ + char path[PATH_MAX]; + int nb_maps; + struct cdx_map maps[RTE_CDX_MAX_RESOURCE]; +}; + +/** mapped cdx device list */ +TAILQ_HEAD(mapped_cdx_res_list, mapped_cdx_resource); + +static struct rte_tailq_elem cdx_vfio_tailq = { + .name = "VFIO_CDX_RESOURCE_LIST", +}; +EAL_REGISTER_TAILQ(cdx_vfio_tailq) + +static struct mapped_cdx_resource * +cdx_vfio_find_and_unmap_resource(struct mapped_cdx_res_list *vfio_res_list, + struct rte_cdx_device *dev) +{ + struct mapped_cdx_resource *vfio_res = NULL; + const char *dev_name = dev->device.name; + struct cdx_map *maps; + int i; + + /* Get vfio_res */ + TAILQ_FOREACH(vfio_res, vfio_res_list, next) { + if (strcmp(vfio_res->name, dev_name)) + continue; + break; + } + + if (vfio_res == NULL) + return vfio_res; + + CDX_BUS_INFO("Releasing CDX mapped resource for %s", dev_name); + + maps = vfio_res->maps; + for (i = 0; i < vfio_res->nb_maps; i++) { + if (maps[i].addr) { + CDX_BUS_DEBUG("Calling cdx_unmap_resource for %s at %p", + dev_name, maps[i].addr); + cdx_unmap_resource(maps[i].addr, maps[i].size); + } + } + + return vfio_res; +} + +static int +cdx_vfio_unmap_resource_primary(struct rte_cdx_device *dev) +{ + char cdx_addr[PATH_MAX] = {0}; + struct mapped_cdx_resource *vfio_res = NULL; + struct mapped_cdx_res_list *vfio_res_list; + + vfio_res_list = + RTE_TAILQ_CAST(cdx_vfio_tailq.head, mapped_cdx_res_list); + vfio_res = cdx_vfio_find_and_unmap_resource(vfio_res_list, dev); + + /* if we haven't found our tailq entry, something's wrong */ + if (vfio_res == NULL) { + CDX_BUS_ERR("%s cannot find TAILQ entry for cdx device!", + cdx_addr); + return -1; + } + + TAILQ_REMOVE(vfio_res_list, vfio_res, next); + rte_free(vfio_res); + return 0; +} + +static int +cdx_vfio_unmap_resource_secondary(struct rte_cdx_device *dev) +{ + struct mapped_cdx_resource *vfio_res = NULL; + struct mapped_cdx_res_list *vfio_res_list; + + vfio_res_list = + RTE_TAILQ_CAST(cdx_vfio_tailq.head, mapped_cdx_res_list); + vfio_res = cdx_vfio_find_and_unmap_resource(vfio_res_list, dev); + + /* if we haven't found our tailq entry, something's wrong */ + if (vfio_res == NULL) { + CDX_BUS_ERR("%s cannot find TAILQ entry for CDX device!", + dev->device.name); + return -1; + } + + return 0; +} + +int +cdx_vfio_unmap_resource(struct rte_cdx_device *dev) +{ + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + return cdx_vfio_unmap_resource_primary(dev); + else + return cdx_vfio_unmap_resource_secondary(dev); +} + +static int +cdx_vfio_setup_device(int vfio_dev_fd) +{ + /* + * Reset the device. If the device is not capable of resetting, + * then it updates errno as EINVAL. + */ + if (ioctl(vfio_dev_fd, VFIO_DEVICE_RESET) && errno != EINVAL) { + CDX_BUS_ERR("Unable to reset device! Error: %d (%s)", errno, + strerror(errno)); + return -1; + } + + return 0; +} + +static int +cdx_vfio_mmap_resource(int vfio_dev_fd, struct mapped_cdx_resource *vfio_res, + int index, int additional_flags) +{ + struct cdx_map *map = &vfio_res->maps[index]; + void *vaddr; + + if (map->size == 0) { + CDX_BUS_DEBUG("map size is 0, skip region %d", index); + return 0; + } + + /* reserve the address using an inaccessible mapping */ + vaddr = mmap(map->addr, map->size, 0, MAP_PRIVATE | + MAP_ANONYMOUS | additional_flags, -1, 0); + if (vaddr != MAP_FAILED) { + void *map_addr = NULL; + + if (map->size) { + /* actual map of first part */ + map_addr = cdx_map_resource(vaddr, vfio_dev_fd, + map->offset, map->size, + RTE_MAP_FORCE_ADDRESS); + } + + if (map_addr == NULL) { + munmap(vaddr, map->size); + vaddr = MAP_FAILED; + CDX_BUS_ERR("Failed to map cdx MMIO region %d", index); + return -1; + } + } else { + CDX_BUS_ERR("Failed to create inaccessible mapping for MMIO region %d", + index); + return -1; + } + + map->addr = vaddr; + return 0; +} + +/* + * region info may contain capability headers, so we need to keep reallocating + * the memory until we match allocated memory size with argsz. + */ +static int +cdx_vfio_get_region_info(int vfio_dev_fd, struct vfio_region_info **info, + int region) +{ + struct vfio_region_info *ri; + size_t argsz = sizeof(*ri); + int ret; + + ri = malloc(sizeof(*ri)); + if (ri == NULL) { + CDX_BUS_ERR("Cannot allocate memory for VFIO region info"); + return -1; + } +again: + memset(ri, 0, argsz); + ri->argsz = argsz; + ri->index = region; + + ret = ioctl(vfio_dev_fd, VFIO_DEVICE_GET_REGION_INFO, ri); + if (ret < 0) { + free(ri); + return ret; + } + if (ri->argsz != argsz) { + struct vfio_region_info *tmp; + + argsz = ri->argsz; + tmp = realloc(ri, argsz); + + if (tmp == NULL) { + /* realloc failed but the ri is still there */ + free(ri); + CDX_BUS_ERR("Cannot reallocate memory for VFIO region info"); + return -1; + } + ri = tmp; + goto again; + } + *info = ri; + + return 0; +} + +static int +find_max_end_va(const struct rte_memseg_list *msl, void *arg) +{ + size_t sz = msl->len; + void *end_va = RTE_PTR_ADD(msl->base_va, sz); + void **max_va = arg; + + if (*max_va < end_va) + *max_va = end_va; + return 0; +} + +static void * +cdx_find_max_end_va(void) +{ + void *va = NULL; + + rte_memseg_list_walk(find_max_end_va, &va); + return va; +} + +static int +cdx_vfio_map_resource_primary(struct rte_cdx_device *dev) +{ + struct vfio_device_info device_info = { .argsz = sizeof(device_info) }; + char cdx_addr[PATH_MAX] = {0}; + static void *cdx_map_addr; + struct mapped_cdx_resource *vfio_res = NULL; + struct mapped_cdx_res_list *vfio_res_list = + RTE_TAILQ_CAST(cdx_vfio_tailq.head, mapped_cdx_res_list); + const char *dev_name = dev->device.name; + struct cdx_map *maps; + int vfio_dev_fd, i, ret; + + ret = rte_vfio_setup_device(RTE_CDX_BUS_DEVICES_PATH, dev_name, + &vfio_dev_fd, &device_info); + if (ret) + return ret; + + /* allocate vfio_res and get region info */ + vfio_res = rte_zmalloc("VFIO_RES", sizeof(*vfio_res), 0); + if (vfio_res == NULL) { + CDX_BUS_ERR("Cannot store VFIO mmap details"); + goto err_vfio_dev_fd; + } + memcpy(vfio_res->name, dev_name, RTE_DEV_NAME_MAX_LEN); + + /* get number of registers */ + vfio_res->nb_maps = device_info.num_regions; + + /* map memory regions */ + maps = vfio_res->maps; + + for (i = 0; i < vfio_res->nb_maps; i++) { + struct vfio_region_info *reg = NULL; + void *vaddr; + + ret = cdx_vfio_get_region_info(vfio_dev_fd, ®, i); + if (ret < 0) { + CDX_BUS_ERR("%s cannot get device region info error %i (%s)", + dev_name, errno, strerror(errno)); + goto err_vfio_res; + } + + /* skip non-mmappable regions */ + if ((reg->flags & VFIO_REGION_INFO_FLAG_MMAP) == 0) { + free(reg); + continue; + } + + /* try mapping somewhere close to the end of hugepages */ + if (cdx_map_addr == NULL) + cdx_map_addr = cdx_find_max_end_va(); + + vaddr = cdx_map_addr; + cdx_map_addr = RTE_PTR_ADD(vaddr, (size_t)reg->size); + + cdx_map_addr = RTE_PTR_ALIGN(cdx_map_addr, + sysconf(_SC_PAGE_SIZE)); + + maps[i].addr = vaddr; + maps[i].offset = reg->offset; + maps[i].size = reg->size; + maps[i].path = NULL; /* vfio doesn't have per-resource paths */ + + ret = cdx_vfio_mmap_resource(vfio_dev_fd, vfio_res, i, 0); + if (ret < 0) { + CDX_BUS_ERR("%s mapping region %i failed: %s", + cdx_addr, i, strerror(errno)); + free(reg); + goto err_vfio_res; + } + + dev->mem_resource[i].addr = maps[i].addr; + dev->mem_resource[i].len = maps[i].size; + + free(reg); + } + + if (cdx_vfio_setup_device(vfio_dev_fd) < 0) { + CDX_BUS_ERR("%s setup device failed", dev_name); + goto err_vfio_res; + } + + TAILQ_INSERT_TAIL(vfio_res_list, vfio_res, next); + + return 0; +err_vfio_res: + cdx_vfio_find_and_unmap_resource(vfio_res_list, dev); + rte_free(vfio_res); +err_vfio_dev_fd: + rte_vfio_release_device(RTE_CDX_BUS_DEVICES_PATH, dev_name, vfio_dev_fd); + return -1; +} + +static int +cdx_vfio_map_resource_secondary(struct rte_cdx_device *dev) +{ + struct vfio_device_info device_info = { .argsz = sizeof(device_info) }; + char cdx_addr[PATH_MAX] = {0}; + int vfio_dev_fd; + int i, ret; + struct mapped_cdx_resource *vfio_res = NULL; + struct mapped_cdx_res_list *vfio_res_list = + RTE_TAILQ_CAST(cdx_vfio_tailq.head, mapped_cdx_res_list); + const char *dev_name = dev->device.name; + struct cdx_map *maps; + + /* if we're in a secondary process, just find our tailq entry */ + TAILQ_FOREACH(vfio_res, vfio_res_list, next) { + if (strcmp(vfio_res->name, dev_name)) + continue; + break; + } + /* if we haven't found our tailq entry, something's wrong */ + if (vfio_res == NULL) { + CDX_BUS_ERR("%s cannot find TAILQ entry for cdx device!", + dev_name); + return -1; + } + + ret = rte_vfio_setup_device(RTE_CDX_BUS_DEVICES_PATH, dev_name, + &vfio_dev_fd, &device_info); + if (ret) + return ret; + + /* map MMIO regions */ + maps = vfio_res->maps; + + for (i = 0; i < vfio_res->nb_maps; i++) { + ret = cdx_vfio_mmap_resource(vfio_dev_fd, vfio_res, i, MAP_FIXED); + if (ret < 0) { + CDX_BUS_ERR("%s mapping MMIO region %i failed: %s", + dev_name, i, strerror(errno)); + goto err_vfio_dev_fd; + } + + dev->mem_resource[i].addr = maps[i].addr; + dev->mem_resource[i].len = maps[i].size; + } + + return 0; +err_vfio_dev_fd: + rte_vfio_release_device(RTE_CDX_BUS_DEVICES_PATH, cdx_addr, vfio_dev_fd); + return -1; +} + +/* + * map the CDX resources of a CDX device in virtual memory (VFIO version). + * primary and secondary processes follow almost exactly the same path + */ +int +cdx_vfio_map_resource(struct rte_cdx_device *dev) +{ + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + return cdx_vfio_map_resource_primary(dev); + else + return cdx_vfio_map_resource_secondary(dev); +} diff --git a/drivers/bus/cdx/meson.build b/drivers/bus/cdx/meson.build new file mode 100644 index 0000000000..f2ca104d34 --- /dev/null +++ b/drivers/bus/cdx/meson.build @@ -0,0 +1,13 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (C) 2022-2023, Advanced Micro Devices, Inc. + +if not is_linux + build = false + reason = 'only supported on Linux' +endif + +driver_sdk_headers = files('bus_cdx_driver.h') +sources = files( + 'cdx.c', + 'cdx_vfio.c', +) diff --git a/drivers/bus/cdx/private.h b/drivers/bus/cdx/private.h new file mode 100644 index 0000000000..81987d0cfe --- /dev/null +++ b/drivers/bus/cdx/private.h @@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2022-2023, Advanced Micro Devices, Inc. + */ + +#ifndef CDX_PRIVATE_H +#define CDX_PRIVATE_H + +#include "bus_cdx_driver.h" + +/** + * Structure describing the CDX bus. + */ +struct rte_cdx_bus { + struct rte_bus bus; /**< Inherit the generic class */ + RTE_TAILQ_HEAD(, rte_cdx_device) device_list; /**< List of CDX devices */ + RTE_TAILQ_HEAD(, rte_cdx_driver) driver_list; /**< List of CDX drivers */ +}; + +/** + * Map a particular resource from a file. + * + * @param requested_addr + * The starting address for the new mapping range. + * @param fd + * The file descriptor. + * @param offset + * The offset for the mapping range. + * @param size + * The size for the mapping range. + * @param additional_flags + * The additional rte_mem_map() flags for the mapping range. + * @return + * - On success, the function returns a pointer to the mapped area. + * - On error, NULL is returned. + */ +void *cdx_map_resource(void *requested_addr, int fd, uint64_t offset, + size_t size, int additional_flags); + +/** + * Unmap a particular resource. + * + * @param requested_addr + * The address for the unmapping range. + * @param size + * The size for the unmapping range. + */ +void cdx_unmap_resource(void *requested_addr, size_t size); + +/* map/unmap VFIO resource */ +int cdx_vfio_map_resource(struct rte_cdx_device *dev); +int cdx_vfio_unmap_resource(struct rte_cdx_device *dev); + +#endif /* CDX_PRIVATE_H */ diff --git a/drivers/bus/cdx/version.map b/drivers/bus/cdx/version.map new file mode 100644 index 0000000000..360460da18 --- /dev/null +++ b/drivers/bus/cdx/version.map @@ -0,0 +1,10 @@ +INTERNAL { + global: + + rte_cdx_map_device; + rte_cdx_register; + rte_cdx_unmap_device; + rte_cdx_unregister; + + local: *; +}; diff --git a/drivers/bus/meson.build b/drivers/bus/meson.build index 6d2520c543..a78b4283bf 100644 --- a/drivers/bus/meson.build +++ b/drivers/bus/meson.build @@ -3,6 +3,7 @@ drivers = [ 'auxiliary', + 'cdx', 'dpaa', 'fslmc', 'ifpga', From patchwork Wed Jun 7 04:24:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Gupta, Nipun" X-Patchwork-Id: 128268 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 492EB42C46; Wed, 7 Jun 2023 06:24:57 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 27E2742D17; Wed, 7 Jun 2023 06:24:54 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2075.outbound.protection.outlook.com [40.107.93.75]) by mails.dpdk.org (Postfix) with ESMTP id 1D04940A84 for ; Wed, 7 Jun 2023 06:24:53 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jTQ0pbnE+QqgA//mkEOUi3exE9fcY2/1AgQDi5QxJ97Eq1DgovHyvozXPuLu/9B3qEnvBCrP6l8dKEj6sDvl9cF0q3i6p+TNWRB+LwUG2b+61D6imbfMa6umB1RH6MnO1sH5hN8xdR1s4L9RjoZ29K8hhlqEZZPRZ6Z7/Shcd8+U1YRO2CQCDIrglrL5KIAWNUJo8uvX3vNTQ1xYk7cvzKdjT1u2PveP0Il6EMMpJBTf9wN6Szo/HoznXgJqe81K/Ki+9D2mUnSCDbCdqHGEpczJjvV/sKa05RDnYkKlVRhi/yLbY+xGuCAF1c1R7NuaCfSVBeLRam+AQnOpunGd9g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=prYhMKF16BYiz/RWDcLgUEsFA2ITZByZjDF3CWWlDgQ=; b=Rh0pdy58Sc6zbaV1CaZ85BKDuzKQh+kNkkEnXNIYWWZFpmifvrVxwYC5jbUAcYOTK6LNDA3lNgZ8fYqx+SDdXd1S087Rzr0++YvvdhQVqNFwGaPJEAQhY2kqqW2ZJxnvh0TFO/W98AEgrbgdq8ez3cU1I0ALpqTDIM/8mr+7oqm8pbo1WTHH9LW58WFPTVb0vYh3NQrQR0EFWEpt9K24G+0RMky/S8rDELskMPs3AP58QQmS3MOzJpViGqIP6Yr4eEq94x/CoDH+2wWrAju3on9zw6Q+EinDX+CxN8VUAKuRmFnhsYZCi9gGW64oUI/2bvrOGoKcxX1BnqcHXUbvnA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=dpdk.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=prYhMKF16BYiz/RWDcLgUEsFA2ITZByZjDF3CWWlDgQ=; b=BGCdQBeIqxIEjXECi7+gEY9nkflAQUk+ugOi4RDO6YqMwXM/h2dh0dd7pXlMOPSilrFn8hPImxqHbN83nHfnmZVQsxcgyDSxC04fFmn3Qj7A0xBhzPjbI3nAp0YYASxL3WW2R7qhkaMA0ggjblaB/xZwFrx6i5rf42zjVjstmrs= Received: from CYZPR11CA0011.namprd11.prod.outlook.com (2603:10b6:930:8d::20) by LV8PR12MB9153.namprd12.prod.outlook.com (2603:10b6:408:185::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.33; Wed, 7 Jun 2023 04:24:49 +0000 Received: from CY4PEPF0000EE39.namprd03.prod.outlook.com (2603:10b6:930:8d:cafe::96) by CYZPR11CA0011.outlook.office365.com (2603:10b6:930:8d::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.19 via Frontend Transport; Wed, 7 Jun 2023 04:24:49 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C Received: from SATLEXMB03.amd.com (165.204.84.17) by CY4PEPF0000EE39.mail.protection.outlook.com (10.167.242.13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6477.13 via Frontend Transport; Wed, 7 Jun 2023 04:24:48 +0000 Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 6 Jun 2023 23:24:48 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 6 Jun 2023 23:24:47 -0500 Received: from xhdipdslab41.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Tue, 6 Jun 2023 23:24:45 -0500 From: Nipun Gupta To: , , , , , CC: , , , Nipun Gupta Subject: [PATCH v8 2/4] bus/cdx: add DMA map and unmap support Date: Wed, 7 Jun 2023 09:54:23 +0530 Message-ID: <20230607042425.30714-3-nipun.gupta@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230607042425.30714-1-nipun.gupta@amd.com> References: <20230124140746.594066-1-nipun.gupta@amd.com> <20230607042425.30714-1-nipun.gupta@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EE39:EE_|LV8PR12MB9153:EE_ X-MS-Office365-Filtering-Correlation-Id: a387093f-f5c5-4675-52dc-08db670f21a5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: PhLA9tUnJ4jRsP+7WMhnihTYK9EmGFSuKMW8l7biSoCoEuNXii+pZ0iT3Rih7JgytiGMrTPCc/KYmRYNr1zljzTDzwhkUf8VFJrT85Aq1n3ZyrWZ1026KGdFFmJhSsmqLj33VwXSk5Q1XW9DAjtr1BizEXQcce1tLztb2EzgpR/YwGOAy6I+l0eCDnaDhIhjswif2qlvxE5QZVxyncZVKqAsxMVt4mWe94We7RZKrbZRz5Lnm7V+879Xt7EuI3OO1z+/U7A4erY0SbNPY0oPdgW/Ay7qpFI5WGCQlBNAnE8rZUSc6q2zoVJVaGyj4NvDwHEm/Uaq5BKyX9hw6EE802x9Vu1G7KmR8K3b1EPBuElRNiM3/ZWysUT6lk5lIb8/yTnC+TPzL1MYErRVNMFmns/NF+rKVEWfbvS0ls8Ys7WP8g3H9RuCSZRpnq4AdP8J52yFQ+b0fUOkdp8PS3KjlqhZLl4iQyPZvNFcIc34YIOW5MySMl97pKaCKnCW7/kW1MvC4T0F/7YO3jSfhiKmSZI4UEfRqCKif5JTpbB8hkvmoJbU+MFdMh3uc0t7ohU74mn+SSINvmrODZ7KhO/OtRQvV6DE6Ji/oNZsngjbtJgMrmFvOZ04pSHXaZb68Ht1gq9WubVVL5utIVMZUZlWf22ezq+aECps+nEymAqaymItmqUcyLc/o+WLM0PgqF5w6ioPfHUgK1jrxXvngeKO1YyGc3Q+pgReTDarSmaBo1BMu4Ks6iEk40Kt0kUoht7LUJ+QVBEhp4m8UmqQSQ5h6A== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB03.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230028)(4636009)(136003)(396003)(39860400002)(376002)(346002)(451199021)(46966006)(40470700004)(36840700001)(54906003)(110136005)(40460700003)(478600001)(40480700001)(44832011)(8936002)(8676002)(2906002)(36756003)(5660300002)(86362001)(82310400005)(4326008)(81166007)(70206006)(70586007)(356005)(316002)(82740400003)(2616005)(1076003)(41300700001)(26005)(47076005)(36860700001)(186003)(336012)(6666004)(426003)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2023 04:24:48.8772 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a387093f-f5c5-4675-52dc-08db670f21a5 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EE39.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV8PR12MB9153 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org AMD CDX bus can use VFIO interface for mapping and unmapping of DMA addresses in the IOMMU. This change adds the callback support for map and unmap APIs as well as fetching the IOMMU class. Signed-off-by: Nipun Gupta Acked-by: Ferruh Yigit --- drivers/bus/cdx/cdx.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/drivers/bus/cdx/cdx.c b/drivers/bus/cdx/cdx.c index 38515e7fda..0c6bac3a44 100644 --- a/drivers/bus/cdx/cdx.c +++ b/drivers/bus/cdx/cdx.c @@ -488,12 +488,42 @@ cdx_find_device(const struct rte_device *start, rte_dev_cmp_t cmp, return NULL; } +static int +cdx_dma_map(struct rte_device *dev, void *addr, uint64_t iova, size_t len) +{ + RTE_SET_USED(dev); + + return rte_vfio_container_dma_map(RTE_VFIO_DEFAULT_CONTAINER_FD, + (uintptr_t)addr, iova, len); +} + +static int +cdx_dma_unmap(struct rte_device *dev, void *addr, uint64_t iova, size_t len) +{ + RTE_SET_USED(dev); + + return rte_vfio_container_dma_unmap(RTE_VFIO_DEFAULT_CONTAINER_FD, + (uintptr_t)addr, iova, len); +} + +static enum rte_iova_mode +cdx_get_iommu_class(void) +{ + if (TAILQ_EMPTY(&rte_cdx_bus.device_list)) + return RTE_IOVA_DC; + + return RTE_IOVA_VA; +} + struct rte_cdx_bus rte_cdx_bus = { .bus = { .scan = cdx_scan, .probe = cdx_probe, .find_device = cdx_find_device, .parse = cdx_parse, + .dma_map = cdx_dma_map, + .dma_unmap = cdx_dma_unmap, + .get_iommu_class = cdx_get_iommu_class, }, .device_list = TAILQ_HEAD_INITIALIZER(rte_cdx_bus.device_list), .driver_list = TAILQ_HEAD_INITIALIZER(rte_cdx_bus.driver_list), From patchwork Wed Jun 7 04:24:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Gupta, Nipun" X-Patchwork-Id: 128269 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 54DEE42C46; Wed, 7 Jun 2023 06:25:03 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 59DB542D12; Wed, 7 Jun 2023 06:24:57 +0200 (CEST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) by mails.dpdk.org (Postfix) with ESMTP id F09BE42D30 for ; Wed, 7 Jun 2023 06:24:55 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nMpsmC9BylUuIk0/bwp7xkfM8pZBXXPGh/T9Pw4FAidYdIj2mNtHZGljWNexggUFgoBlfNHwvjysjdL9fY3NgB9bf1qN6emuofnD6AGvWeinxQHxE/9Mj0gZeWse6IPBe9rmTxBk72qHz4bsPiz9d3nspRItRzJEk20M3vxG9rocA5E5DUSXGrsxSZYVBXAbPEvrhrvmXCSlZydsN25KvZ8FyIy0yVaZpKX52QFoDjAP2MsAHSiUnVHDcXUNBFOOnaQkxK5nkHVR+vsTwZdJ+T//XxC8g4kmsILArlvNQ36y2M7jYpn3n5dx3T/AiMpWJZ3U53l7WM02pAdGeMLtog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Cs3pbPJnxU1UZrM0chSUfegkCbAvBm4e5KtdqO6CMIA=; b=PcdLLxkyFJEjGF0hZ0P3wtmrj73nV5M1w0vmG+qH2DC84CHSFExX3dBH+gLEflQa1oDQXunv3JHcBQ+5dOTVdkKfADl+nAjh4ZFmFwg0K7y1nz42FcpW1cSuGxJEe9cnCm8Eg8kPaB5zmEGKZyjsDTRyR19fYqYhL6DGncQqriT1Rknny1DtkyBlWPyYmcqVn4HhF59YHZmw9y74DUgpBaI/+6ntuKgjsldVCCR3gEdk07I/p8aw8s7w8cXq/wE7cZUb1jvDfbsotCpBWcqnoqa8oCXAofBt3e1y8QTl8VksLZUrzysJ2VH9OB958DVES1WC3mQAxRnxo6+C25jG9A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=dpdk.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Cs3pbPJnxU1UZrM0chSUfegkCbAvBm4e5KtdqO6CMIA=; b=VQBRBpBKWHPsEafI+hSyiFtbVQVsymxq/0iKe0SY/+Pur71ijjd0X56Jpvy/pp6NSh8dIeUKUM34wYWOnIWPm0wteRL/FbxSbtxQc5MIM2Qo/vSY6yJvmXWEg9hnHnra/DEuTm4Ms8e+gowt7icV4rA5+IfVU2vj8pk4QCJBOeA= Received: from CY5PR22CA0054.namprd22.prod.outlook.com (2603:10b6:930:1d::18) by CO6PR12MB5396.namprd12.prod.outlook.com (2603:10b6:303:139::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.33; Wed, 7 Jun 2023 04:24:53 +0000 Received: from CY4PEPF0000E9D3.namprd03.prod.outlook.com (2603:10b6:930:1d:cafe::3d) by CY5PR22CA0054.outlook.office365.com (2603:10b6:930:1d::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.33 via Frontend Transport; Wed, 7 Jun 2023 04:24:53 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CY4PEPF0000E9D3.mail.protection.outlook.com (10.167.241.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6477.13 via Frontend Transport; Wed, 7 Jun 2023 04:24:53 +0000 Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 6 Jun 2023 23:24:52 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 6 Jun 2023 23:24:52 -0500 Received: from xhdipdslab41.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Tue, 6 Jun 2023 23:24:49 -0500 From: Nipun Gupta To: , , , , , CC: , , , Nipun Gupta Subject: [PATCH v8 3/4] bus/cdx: add support for MSI Date: Wed, 7 Jun 2023 09:54:24 +0530 Message-ID: <20230607042425.30714-4-nipun.gupta@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230607042425.30714-1-nipun.gupta@amd.com> References: <20230124140746.594066-1-nipun.gupta@amd.com> <20230607042425.30714-1-nipun.gupta@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9D3:EE_|CO6PR12MB5396:EE_ X-MS-Office365-Filtering-Correlation-Id: d3bb1eb9-b800-46c4-db7c-08db670f2445 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ggmnesm0EZRCtabRGoSHZe8trbXWcWZlWGwpKMTkQ+nSQvxbg7oM+ZTddWfmGXt/rHPuDDVwEeVRLvrHVZavCOOr3Whb8v493JlaaRWwuRlenw5YZPU++OdQ1SeXZadgG+qGqL12ES4/M1jox7Ahdy3m+K3WsZAifyIe0h21P/EUGSoCR/GLdodR+lyv1YG3T/pGF3EjxGufiXpQyokXpg+NtU69X8Giz2AX8tSDfTrn7Nia8l/mgdpQqVeQIdTo7SIveeZKb8YH+NihF4nY6c09SmW+aJBOJMYs1mBvBE4OrRQgdAT9bYrm5lzkUxAQ4gRzlfj8w2rZUu4+ZBlVrcGkQTgiVx5EfzWDPTKjefOi2NZxim30QHD4SEn/4cfadCbAOZI0rNOEfwRNBFEIVXmS59GtceWqMdBNgoHiz65VTUkUIbcJuiitA/bwcTLSnZ0YRSWQzJPrXCpIK6uEXiurry9QkOONr2cPXqbEzBoDN8TD9wN7j9hA5lj8zSSj7r8Vn2ZEYdJnBkZPiJu67jCsiII5BKSgKjIhCBqEC7GKl6OyFFjXSLFakqm/hSBG3Sw9EMlqcUhg6LIppCmrKKPyhlBVC5cYtJrI6U/IRVrLc0/2GxT0Im2fAZxLlKesHqSuJPd1roN9oxUzNRTTWOSKMT/zNp7+YgNc4BSRG4Y/TbOccdmdlmDDCNPj2xGwSZHaWSLQQOhbE4TpOFXVTFX8Z2ltaKF7mgzBh6hvG1AghT3WJsAJuJ/4td4sfj5i9y7fajcf/QSZSVyOKbiWZg== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230028)(4636009)(346002)(396003)(376002)(136003)(39860400002)(451199021)(46966006)(36840700001)(40470700004)(40480700001)(54906003)(70586007)(110136005)(82740400003)(478600001)(81166007)(316002)(356005)(47076005)(8676002)(8936002)(41300700001)(4326008)(70206006)(2616005)(186003)(426003)(336012)(36860700001)(6666004)(83380400001)(1076003)(40460700003)(26005)(86362001)(44832011)(5660300002)(2906002)(82310400005)(36756003)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2023 04:24:53.2847 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d3bb1eb9-b800-46c4-db7c-08db670f2445 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000E9D3.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR12MB5396 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org MSI's are exposed to the devices using VFIO (vfio-cdx). This patch uses the same to add support for MSI for the devices on the cdx bus. Signed-off-by: Nipun Gupta Acked-by: Ferruh Yigit --- drivers/bus/cdx/bus_cdx_driver.h | 25 +++++ drivers/bus/cdx/cdx.c | 11 ++ drivers/bus/cdx/cdx_vfio.c | 176 ++++++++++++++++++++++++++++++- drivers/bus/cdx/version.map | 2 + 4 files changed, 212 insertions(+), 2 deletions(-) diff --git a/drivers/bus/cdx/bus_cdx_driver.h b/drivers/bus/cdx/bus_cdx_driver.h index 9a2be6481a..95c266bccb 100644 --- a/drivers/bus/cdx/bus_cdx_driver.h +++ b/drivers/bus/cdx/bus_cdx_driver.h @@ -56,6 +56,7 @@ struct rte_cdx_device { struct rte_cdx_id id; /**< CDX ID. */ struct rte_mem_resource mem_resource[RTE_CDX_MAX_RESOURCE]; /**< CDX Memory Resource */ + struct rte_intr_handle *intr_handle; /**< Interrupt handle */ }; /** @@ -149,6 +150,30 @@ void rte_cdx_register(struct rte_cdx_driver *driver); } \ RTE_PMD_EXPORT_NAME(nm, __COUNTER__) +/** + * Enables VFIO Interrupts for CDX bus devices. + * + * @param intr_handle + * Pointer to the interrupt handle. + * + * @return + * 0 on success, -1 on error. + */ +__rte_internal +int rte_cdx_vfio_intr_enable(const struct rte_intr_handle *intr_handle); + +/** + * Disable VFIO Interrupts for CDX bus devices. + * + * @param intr_handle + * Pointer to the interrupt handle. + * + * @return + * 0 on success, -1 on error. + */ +__rte_internal +int rte_cdx_vfio_intr_disable(const struct rte_intr_handle *intr_handle); + /** * Unregister a CDX driver. * diff --git a/drivers/bus/cdx/cdx.c b/drivers/bus/cdx/cdx.c index 0c6bac3a44..88505cfbfb 100644 --- a/drivers/bus/cdx/cdx.c +++ b/drivers/bus/cdx/cdx.c @@ -202,6 +202,15 @@ cdx_scan_one(const char *dirname, const char *dev_name) goto err; } + /* Allocate interrupt instance for cdx device */ + dev->intr_handle = + rte_intr_instance_alloc(RTE_INTR_INSTANCE_F_PRIVATE); + if (dev->intr_handle == NULL) { + CDX_BUS_ERR("Failed to create interrupt instance for %s", + dev->device.name); + return -ENOMEM; + } + /* * Check if device is bound to 'vfio-cdx' driver, so that user-space * can gracefully access the device. @@ -380,6 +389,8 @@ cdx_probe_one_driver(struct rte_cdx_driver *dr, return ret; error_probe: + rte_intr_instance_free(dev->intr_handle); + dev->intr_handle = NULL; cdx_vfio_unmap_resource(dev); error_map_device: return ret; diff --git a/drivers/bus/cdx/cdx_vfio.c b/drivers/bus/cdx/cdx_vfio.c index 86290f0450..8a3ac0b995 100644 --- a/drivers/bus/cdx/cdx_vfio.c +++ b/drivers/bus/cdx/cdx_vfio.c @@ -51,6 +51,10 @@ struct mapped_cdx_resource { /** mapped cdx device list */ TAILQ_HEAD(mapped_cdx_res_list, mapped_cdx_resource); +/* IRQ set buffer length for MSI interrupts */ +#define MSI_IRQ_SET_BUF_LEN (sizeof(struct vfio_irq_set) + \ + sizeof(int) * (RTE_MAX_RXTX_INTR_VEC_ID + 1)) + static struct rte_tailq_elem cdx_vfio_tailq = { .name = "VFIO_CDX_RESOURCE_LIST", }; @@ -95,6 +99,27 @@ cdx_vfio_unmap_resource_primary(struct rte_cdx_device *dev) char cdx_addr[PATH_MAX] = {0}; struct mapped_cdx_resource *vfio_res = NULL; struct mapped_cdx_res_list *vfio_res_list; + int ret, vfio_dev_fd; + + if (rte_intr_fd_get(dev->intr_handle) < 0) + return -1; + + if (close(rte_intr_fd_get(dev->intr_handle)) < 0) { + CDX_BUS_ERR("Error when closing eventfd file descriptor for %s", + dev->device.name); + return -1; + } + + vfio_dev_fd = rte_intr_dev_fd_get(dev->intr_handle); + if (vfio_dev_fd < 0) + return -1; + + ret = rte_vfio_release_device(RTE_CDX_BUS_DEVICES_PATH, dev->device.name, + vfio_dev_fd); + if (ret < 0) { + CDX_BUS_ERR("Cannot release VFIO device"); + return ret; + } vfio_res_list = RTE_TAILQ_CAST(cdx_vfio_tailq.head, mapped_cdx_res_list); @@ -117,6 +142,18 @@ cdx_vfio_unmap_resource_secondary(struct rte_cdx_device *dev) { struct mapped_cdx_resource *vfio_res = NULL; struct mapped_cdx_res_list *vfio_res_list; + int ret, vfio_dev_fd; + + vfio_dev_fd = rte_intr_dev_fd_get(dev->intr_handle); + if (vfio_dev_fd < 0) + return -1; + + ret = rte_vfio_release_device(RTE_CDX_BUS_DEVICES_PATH, dev->device.name, + vfio_dev_fd); + if (ret < 0) { + CDX_BUS_ERR("Cannot release VFIO device"); + return ret; + } vfio_res_list = RTE_TAILQ_CAST(cdx_vfio_tailq.head, mapped_cdx_res_list); @@ -141,9 +178,74 @@ cdx_vfio_unmap_resource(struct rte_cdx_device *dev) return cdx_vfio_unmap_resource_secondary(dev); } +/* set up interrupt support (but not enable interrupts) */ static int -cdx_vfio_setup_device(int vfio_dev_fd) +cdx_vfio_setup_interrupts(struct rte_cdx_device *dev, int vfio_dev_fd, + int num_irqs) { + int i, ret; + + if (num_irqs == 0) + return 0; + + /* start from MSI interrupt type */ + for (i = 0; i < num_irqs; i++) { + struct vfio_irq_info irq = { .argsz = sizeof(irq) }; + int fd = -1; + + irq.index = i; + + ret = ioctl(vfio_dev_fd, VFIO_DEVICE_GET_IRQ_INFO, &irq); + if (ret < 0) { + CDX_BUS_ERR("Cannot get VFIO IRQ info, error %i (%s)", + errno, strerror(errno)); + return -1; + } + + /* if this vector cannot be used with eventfd, fail if we explicitly + * specified interrupt type, otherwise continue + */ + if ((irq.flags & VFIO_IRQ_INFO_EVENTFD) == 0) + continue; + + /* Set nb_intr to the total number of interrupts */ + if (rte_intr_event_list_update(dev->intr_handle, irq.count)) + return -1; + + /* set up an eventfd for interrupts */ + fd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); + if (fd < 0) { + CDX_BUS_ERR("Cannot set up eventfd, error %i (%s)", + errno, strerror(errno)); + return -1; + } + + if (rte_intr_fd_set(dev->intr_handle, fd)) + return -1; + + /* DPDK CDX bus currently supports only MSI-X */ + if (rte_intr_type_set(dev->intr_handle, RTE_INTR_HANDLE_VFIO_MSIX)) + return -1; + + if (rte_intr_dev_fd_set(dev->intr_handle, vfio_dev_fd)) + return -1; + + return 0; + } + + /* if we're here, we haven't found a suitable interrupt vector */ + return -1; +} + +static int +cdx_vfio_setup_device(struct rte_cdx_device *dev, int vfio_dev_fd, + int num_irqs) +{ + if (cdx_vfio_setup_interrupts(dev, vfio_dev_fd, num_irqs) != 0) { + CDX_BUS_ERR("Error setting up interrupts!"); + return -1; + } + /* * Reset the device. If the device is not capable of resetting, * then it updates errno as EINVAL. @@ -279,6 +381,9 @@ cdx_vfio_map_resource_primary(struct rte_cdx_device *dev) struct cdx_map *maps; int vfio_dev_fd, i, ret; + if (rte_intr_fd_set(dev->intr_handle, -1)) + return -1; + ret = rte_vfio_setup_device(RTE_CDX_BUS_DEVICES_PATH, dev_name, &vfio_dev_fd, &device_info); if (ret) @@ -344,7 +449,7 @@ cdx_vfio_map_resource_primary(struct rte_cdx_device *dev) free(reg); } - if (cdx_vfio_setup_device(vfio_dev_fd) < 0) { + if (cdx_vfio_setup_device(dev, vfio_dev_fd, device_info.num_irqs) < 0) { CDX_BUS_ERR("%s setup device failed", dev_name); goto err_vfio_res; } @@ -373,6 +478,9 @@ cdx_vfio_map_resource_secondary(struct rte_cdx_device *dev) const char *dev_name = dev->device.name; struct cdx_map *maps; + if (rte_intr_fd_set(dev->intr_handle, -1)) + return -1; + /* if we're in a secondary process, just find our tailq entry */ TAILQ_FOREACH(vfio_res, vfio_res_list, next) { if (strcmp(vfio_res->name, dev_name)) @@ -406,6 +514,10 @@ cdx_vfio_map_resource_secondary(struct rte_cdx_device *dev) dev->mem_resource[i].len = maps[i].size; } + /* we need save vfio_dev_fd, so it can be used during release */ + if (rte_intr_dev_fd_set(dev->intr_handle, vfio_dev_fd)) + goto err_vfio_dev_fd; + return 0; err_vfio_dev_fd: rte_vfio_release_device(RTE_CDX_BUS_DEVICES_PATH, cdx_addr, vfio_dev_fd); @@ -424,3 +536,63 @@ cdx_vfio_map_resource(struct rte_cdx_device *dev) else return cdx_vfio_map_resource_secondary(dev); } + +int +rte_cdx_vfio_intr_enable(const struct rte_intr_handle *intr_handle) +{ + char irq_set_buf[MSI_IRQ_SET_BUF_LEN]; + struct vfio_irq_set *irq_set; + int *fd_ptr, vfio_dev_fd, i; + int ret; + + irq_set = (struct vfio_irq_set *) irq_set_buf; + irq_set->count = rte_intr_nb_intr_get(intr_handle); + irq_set->argsz = sizeof(struct vfio_irq_set) + + (sizeof(int) * irq_set->count); + + irq_set->flags = VFIO_IRQ_SET_DATA_EVENTFD | VFIO_IRQ_SET_ACTION_TRIGGER; + irq_set->index = 0; + irq_set->start = 0; + fd_ptr = (int *) &irq_set->data; + + for (i = 0; i < rte_intr_nb_efd_get(intr_handle); i++) + fd_ptr[i] = rte_intr_efds_index_get(intr_handle, i); + + vfio_dev_fd = rte_intr_dev_fd_get(intr_handle); + ret = ioctl(vfio_dev_fd, VFIO_DEVICE_SET_IRQS, irq_set); + + if (ret) { + CDX_BUS_ERR("Error enabling MSI interrupts for fd %d", + rte_intr_fd_get(intr_handle)); + return -1; + } + + return 0; +} + +/* disable MSI interrupts */ +int +rte_cdx_vfio_intr_disable(const struct rte_intr_handle *intr_handle) +{ + struct vfio_irq_set *irq_set; + char irq_set_buf[MSI_IRQ_SET_BUF_LEN]; + int len, ret, vfio_dev_fd; + + len = sizeof(struct vfio_irq_set); + + irq_set = (struct vfio_irq_set *) irq_set_buf; + irq_set->argsz = len; + irq_set->count = 0; + irq_set->flags = VFIO_IRQ_SET_DATA_NONE | VFIO_IRQ_SET_ACTION_TRIGGER; + irq_set->index = 0; + irq_set->start = 0; + + vfio_dev_fd = rte_intr_dev_fd_get(intr_handle); + ret = ioctl(vfio_dev_fd, VFIO_DEVICE_SET_IRQS, irq_set); + + if (ret) + CDX_BUS_ERR("Error disabling MSI interrupts for fd %d", + rte_intr_fd_get(intr_handle)); + + return ret; +} diff --git a/drivers/bus/cdx/version.map b/drivers/bus/cdx/version.map index 360460da18..0a15d39ae8 100644 --- a/drivers/bus/cdx/version.map +++ b/drivers/bus/cdx/version.map @@ -5,6 +5,8 @@ INTERNAL { rte_cdx_register; rte_cdx_unmap_device; rte_cdx_unregister; + rte_cdx_vfio_intr_disable; + rte_cdx_vfio_intr_enable; local: *; }; From patchwork Wed Jun 7 04:24:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Gupta, Nipun" X-Patchwork-Id: 128270 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8774642C46; Wed, 7 Jun 2023 06:25:09 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 61F1F42D35; Wed, 7 Jun 2023 06:25:05 +0200 (CEST) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2076.outbound.protection.outlook.com [40.107.92.76]) by mails.dpdk.org (Postfix) with ESMTP id B391142D35 for ; Wed, 7 Jun 2023 06:25:03 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Xav19O5cpAqsQbjld5eo2A7JzJv9jkcHPGsHI3OdtPOd9ktUfVWKy3eG5ufFfQzC46yJIN4h4xlK6lrRL5ODVe366dZyTAqln01SAEJZpB6coXZs8lcM6Gci9HWVh8tAfe79dxe7Lhlt/PCMkXT1b9o6v00EQxZx/aLmpYYLkmtNVcJornVLTRZFCfj71NJ9w+WGfBOOX9CmA2UWumrPYyRsHHbeDY2X8U6qu7CNMv1IbmgyVwN0d0agnHK3ZKgQ6vBmZ2SlAPQe4xneT6zNyfLKxXKtEi5dmxbYtpA9Ypd/IZOt1CvYtfIl3AW7dScQe7bf4RQ/16N7ji5scizMXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mXnTV8xwi+SuTvBFmXodYcInIgl27fzkwB1Py/HrfmA=; b=FDrF3nLWx7f5x7SLoExhJc58X19dq/rw89YNPFU3bQhRnDwkueB36cFc+5D2FjNaGPLdo63jCgHAJ6DxNz2ptbjipJR0inho/9JFFRyHppjyOucBvYlGEWrPNgqXNc5CeMx8NMMDc3Pr6Dptk9nMNMIfW9JdPo+J4+3SBud1a4/B6I8jqB4HdOBYiUzGMNlatQ3cf4L8bzUNpsUv8/fvgnQ72PAsEOv6W4bJcSQVaiQ5OGJT8Oyel84Z0NrzAddCW/hHmJbOlZL0wubOl2Phz0fAkWPGuvzuKHAakfScdIVC4ZhxUOx6qMMStLFZqJd+P8PuxQ+etkGDBXqhmbX9EQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=dpdk.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mXnTV8xwi+SuTvBFmXodYcInIgl27fzkwB1Py/HrfmA=; b=Ua5VBBViA4Jexxr5WSWxARJyE+1+xXHYvCb1GXjJohk9M3s/dteHG+gGYyNaRpFQXg6y9KCn8JnIMY+2YOukBJvaVPbe+gTL6F2gRd6LQU1ISPPW6rUdCwjr+KN32CvNamypSGB8K1p6OJGkkK4yFaJbfkJKcX5lF2KR/Qot9s8= Received: from DM5PR07CA0084.namprd07.prod.outlook.com (2603:10b6:4:ad::49) by BY5PR12MB4856.namprd12.prod.outlook.com (2603:10b6:a03:1d5::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.19; Wed, 7 Jun 2023 04:24:58 +0000 Received: from CY4PEPF0000EE3B.namprd03.prod.outlook.com (2603:10b6:4:ad:cafe::bb) by DM5PR07CA0084.outlook.office365.com (2603:10b6:4:ad::49) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.33 via Frontend Transport; Wed, 7 Jun 2023 04:24:57 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C Received: from SATLEXMB03.amd.com (165.204.84.17) by CY4PEPF0000EE3B.mail.protection.outlook.com (10.167.242.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6477.13 via Frontend Transport; Wed, 7 Jun 2023 04:24:57 +0000 Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 6 Jun 2023 23:24:57 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 6 Jun 2023 23:24:56 -0500 Received: from xhdipdslab41.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Tue, 6 Jun 2023 23:24:54 -0500 From: Nipun Gupta To: , , , , , CC: , , , Nipun Gupta Subject: [PATCH v8 4/4] bus/cdx: support plug unplug and dev iterator Date: Wed, 7 Jun 2023 09:54:25 +0530 Message-ID: <20230607042425.30714-5-nipun.gupta@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230607042425.30714-1-nipun.gupta@amd.com> References: <20230124140746.594066-1-nipun.gupta@amd.com> <20230607042425.30714-1-nipun.gupta@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EE3B:EE_|BY5PR12MB4856:EE_ X-MS-Office365-Filtering-Correlation-Id: 76f50a24-886c-48e5-73b2-08db670f26ed X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: c4lB4doJTptRnvkInxb6XADO+qBMJwuVk3uIa0AKfOHtjNPYMqbNh3QQCus3NONxAjwlJze1A6ZVrPs+6RT85i5ztXug1bGGTv0cyoJi8J8AH7+4a6ZcAJCUezeR5RHxPsF4Iltu2m6OOM3uq6Eb7vs6Py0NvHZJjHcRwPMN3R2XiUCmtSomKZRzuJQVJ2vi3SXB3/EBDBfl/MGxANjb51u4RnldKhf5WconzGJGePAL6KBaikKOJn6zkteI5Z94N4kLYs8JrCv+Sc8tESlv52agnRI8OSKkO306MGvvEc1FUF+S7C8yF2rcRQGrpL4NuufZb3a109QJ6YaBdf+/Sh3iiauY4Ko11mFL88jdcU2mReZNCkmvHcFHqgoNXN2X9OTLBOjlh6ussbAQnZltlnPuZvz/21FZyTxNaFP6ZIN9gI7SljXX9W5l29CKuTEqDmSCGyeEFjhofDcvqZXobWJXz/YLF7xuVFIR1rS6jhxsFcQf3KLg2tQimOfeb/rQPM5/iVnoVOCdGPOlf73klEhvApcaIZBvdXOCR56gGeDA5DnVh/L+askRoP8Xu/EJIz1zmoo5AHA5Dh8LrHcLHNLZwxoVKmqGDMAgRkn37l/332df//zTevfnwW2fLSN0sdh1XL1UoK9bH5HyQ9s2zTEuyaoy6URbjhyBNEAU9AJP7oWWxhYvpnRlMlKzF4Y4vwizKdPL5ORjE1a7KD6y92336eNiv2F5hgYz7XdQKlFRZUUAzIbdVYCdK90LfgH1ign/585fg6t8CHRBpxcMSQ== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB03.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230028)(4636009)(39860400002)(376002)(136003)(346002)(396003)(451199021)(40470700004)(46966006)(36840700001)(86362001)(40460700003)(2906002)(110136005)(54906003)(41300700001)(8936002)(82310400005)(8676002)(5660300002)(82740400003)(356005)(36756003)(44832011)(316002)(4326008)(81166007)(70206006)(70586007)(40480700001)(6666004)(478600001)(36860700001)(186003)(47076005)(336012)(2616005)(426003)(26005)(1076003)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2023 04:24:57.7385 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 76f50a24-886c-48e5-73b2-08db670f26ed X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EE3B.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4856 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This change adds support for plugging and unplugging CDX devices on AMD CDX bus. Also, CDX dev iterator support has been added for the CDX bus. Signed-off-by: Nipun Gupta Acked-by: Ferruh Yigit --- drivers/bus/cdx/bus_cdx_driver.h | 1 + drivers/bus/cdx/cdx.c | 122 +++++++++++++++++++++++++++++++ 2 files changed, 123 insertions(+) diff --git a/drivers/bus/cdx/bus_cdx_driver.h b/drivers/bus/cdx/bus_cdx_driver.h index 95c266bccb..fcacdb5896 100644 --- a/drivers/bus/cdx/bus_cdx_driver.h +++ b/drivers/bus/cdx/bus_cdx_driver.h @@ -52,6 +52,7 @@ struct rte_cdx_id { struct rte_cdx_device { RTE_TAILQ_ENTRY(rte_cdx_device) next; /**< Next probed CDX device. */ struct rte_device device; /**< Inherit core device */ + struct rte_cdx_driver *driver; /**< CDX driver used in probing */ char name[RTE_DEV_NAME_MAX_LEN]; /**< Device name */ struct rte_cdx_id id; /**< CDX ID. */ struct rte_mem_resource mem_resource[RTE_CDX_MAX_RESOURCE]; diff --git a/drivers/bus/cdx/cdx.c b/drivers/bus/cdx/cdx.c index 88505cfbfb..9607096d18 100644 --- a/drivers/bus/cdx/cdx.c +++ b/drivers/bus/cdx/cdx.c @@ -71,6 +71,7 @@ #include #include #include +#include #include #include @@ -92,6 +93,15 @@ struct rte_cdx_bus rte_cdx_bus; +enum cdx_params { + RTE_CDX_PARAM_NAME, +}; + +static const char * const cdx_params_keys[] = { + [RTE_CDX_PARAM_NAME] = "name", + NULL, +}; + /* Add a device to CDX bus */ static void cdx_add_device(struct rte_cdx_device *cdx_dev) @@ -385,6 +395,7 @@ cdx_probe_one_driver(struct rte_cdx_driver *dr, } else { dev->device.driver = &dr->driver; } + dev->driver = dr; return ret; @@ -499,6 +510,71 @@ cdx_find_device(const struct rte_device *start, rte_dev_cmp_t cmp, return NULL; } +/* Remove a device from CDX bus */ +static void +cdx_remove_device(struct rte_cdx_device *cdx_dev) +{ + TAILQ_REMOVE(&rte_cdx_bus.device_list, cdx_dev, next); +} + +/* + * If vendor/device ID match, call the remove() function of the + * driver. + */ +static int +cdx_detach_dev(struct rte_cdx_device *dev) +{ + struct rte_cdx_driver *dr; + int ret = 0; + + if (dev == NULL) + return -EINVAL; + + dr = dev->driver; + + CDX_BUS_DEBUG("detach device %s using driver: %s", + dev->device.name, dr->driver.name); + + if (dr->remove) { + ret = dr->remove(dev); + if (ret < 0) + return ret; + } + + /* clear driver structure */ + dev->driver = NULL; + dev->device.driver = NULL; + + rte_cdx_unmap_device(dev); + + rte_intr_instance_free(dev->intr_handle); + dev->intr_handle = NULL; + + return 0; +} + +static int +cdx_plug(struct rte_device *dev) +{ + return cdx_probe_all_drivers(RTE_DEV_TO_CDX_DEV(dev)); +} + +static int +cdx_unplug(struct rte_device *dev) +{ + struct rte_cdx_device *cdx_dev; + int ret; + + cdx_dev = RTE_DEV_TO_CDX_DEV(dev); + ret = cdx_detach_dev(cdx_dev); + if (ret == 0) { + cdx_remove_device(cdx_dev); + rte_devargs_remove(dev->devargs); + free(cdx_dev); + } + return ret; +} + static int cdx_dma_map(struct rte_device *dev, void *addr, uint64_t iova, size_t len) { @@ -526,15 +602,61 @@ cdx_get_iommu_class(void) return RTE_IOVA_VA; } +static int +cdx_dev_match(const struct rte_device *dev, + const void *_kvlist) +{ + const struct rte_kvargs *kvlist = _kvlist; + const char *key = cdx_params_keys[RTE_CDX_PARAM_NAME]; + const char *name; + + /* no kvlist arg, all devices match */ + if (kvlist == NULL) + return 0; + + /* if key is present in kvlist and does not match, filter device */ + name = rte_kvargs_get(kvlist, key); + if (name != NULL && strcmp(name, dev->name)) + return -1; + + return 0; +} + +static void * +cdx_dev_iterate(const void *start, + const char *str, + const struct rte_dev_iterator *it __rte_unused) +{ + rte_bus_find_device_t find_device; + struct rte_kvargs *kvargs = NULL; + struct rte_device *dev; + + if (str != NULL) { + kvargs = rte_kvargs_parse(str, cdx_params_keys); + if (kvargs == NULL) { + CDX_BUS_ERR("cannot parse argument list %s", str); + rte_errno = EINVAL; + return NULL; + } + } + find_device = rte_cdx_bus.bus.find_device; + dev = find_device(start, cdx_dev_match, kvargs); + rte_kvargs_free(kvargs); + return dev; +} + struct rte_cdx_bus rte_cdx_bus = { .bus = { .scan = cdx_scan, .probe = cdx_probe, .find_device = cdx_find_device, + .plug = cdx_plug, + .unplug = cdx_unplug, .parse = cdx_parse, .dma_map = cdx_dma_map, .dma_unmap = cdx_dma_unmap, .get_iommu_class = cdx_get_iommu_class, + .dev_iterate = cdx_dev_iterate, }, .device_list = TAILQ_HEAD_INITIALIZER(rte_cdx_bus.device_list), .driver_list = TAILQ_HEAD_INITIALIZER(rte_cdx_bus.driver_list),