From patchwork Mon May 8 11:18:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Gupta, Nipun" X-Patchwork-Id: 126759 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D473B42A94; Mon, 8 May 2023 13:18:44 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C6295427E9; Mon, 8 May 2023 13:18:44 +0200 (CEST) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2086.outbound.protection.outlook.com [40.107.92.86]) by mails.dpdk.org (Postfix) with ESMTP id 2842D40685 for ; Mon, 8 May 2023 13:18:43 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SyooAEIZO/y0if7kpt1cMeuHKos0IHEQnpX8VRFIraSZqE29U3vCWHSGvPzRm0DiLFlSn8Am7631PnyHGOR4ilEImCnL7xR7lgYvRL2hH8WZ74WhBQ/Mw/UZ+uP6KXGkskiN8TOc/DCCmn/+w+cY0OrRImgUjPEaUKLlhPXQ52Vef8/WCkUh2hs5rtPU7NpJLaXEfgMCyyRlROFjku56S5vvkp7EGCLUNt/HGoFq4JtTj823Z988ncSXYkLQo/gOXI2pLrilNJ0PSmiIzcX5RNCW2ytl7p9/1aFJgzDrWTocTE0ggKaiheg8ijnjdSDsNzjxVl8gPGAKrQjUMHkHfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=L5+/Dw9JmGl2N71BWsPV7+xNWeRZsLKNkII51JIrTrg=; b=RP+jfnivHduH4Ld6rzS1K/bgLpdnTRfW+zSX4bq1i2RecKHyJwzU04PlRsl1ZOoQH191A4Glkky15RZdgnY6gLbu5QtJzy/8rPjLBEr0bTEjI+TYVJa2wPfGOiOFyDKuoVr+gHM+NKOYlDdPhrahOMCjI72u70M9Pszt3d73NhJ5E2B3fhEU6hnWIAqryX8XTy9qpg4L+W+zqFICnHCs78G6a9AdBC2ZDO+xMQYfqFYLMAO1CcNdChwEoSIewC3UDlOV/0brgvM05PBUPAQsUbrQ59GLsQKyDJDYGlA8hsKE1Y8ApMid2bT9D02N/bfbVz1Xe2hZxYQietiMly8X0Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=dpdk.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=L5+/Dw9JmGl2N71BWsPV7+xNWeRZsLKNkII51JIrTrg=; b=PDAqiYVQC+dMsN5lWgo4fuyb9acH243MUKr9X1GLcgO1QKw9Q/AIvstgmY8f9Nx5ybZX4UTRW/bFB9AweXUf2W8FW55WOCssF/DD4aQRH5bauI6nv619iSQhI/ULAMdQQagHejDt1NfpWeuPtz5YwiTHN+KKoQUw79Rd395Dip4= Received: from BN9PR03CA0493.namprd03.prod.outlook.com (2603:10b6:408:130::18) by DM4PR12MB5200.namprd12.prod.outlook.com (2603:10b6:5:397::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6363.32; Mon, 8 May 2023 11:18:40 +0000 Received: from BN8NAM11FT111.eop-nam11.prod.protection.outlook.com (2603:10b6:408:130:cafe::cc) by BN9PR03CA0493.outlook.office365.com (2603:10b6:408:130::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6363.32 via Frontend Transport; Mon, 8 May 2023 11:18:40 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C Received: from SATLEXMB03.amd.com (165.204.84.17) by BN8NAM11FT111.mail.protection.outlook.com (10.13.177.54) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6363.32 via Frontend Transport; Mon, 8 May 2023 11:18:39 +0000 Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 8 May 2023 06:18:38 -0500 Received: from xhdipdslab41.xilinx.com (10.180.168.240) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Mon, 8 May 2023 06:18:36 -0500 From: Nipun Gupta To: , , CC: , , , Nipun Gupta Subject: [PATCH v4 1/4] bus/cdx: introduce cdx bus Date: Mon, 8 May 2023 16:48:09 +0530 Message-ID: <20230508111812.2655-2-nipun.gupta@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230508111812.2655-1-nipun.gupta@amd.com> References: <20230124140746.594066-1-nipun.gupta@amd.com> <20230508111812.2655-1-nipun.gupta@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT111:EE_|DM4PR12MB5200:EE_ X-MS-Office365-Filtering-Correlation-Id: b9a9b8ed-fdd5-470d-845c-08db4fb5f9ab X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: hnJ8foZ4i8prvmgUFfZIalHMPwV/Q6WRVub2q69mp2kpDVxMr9zI38C9qo8DKpfMYPxI3uU9nf7nN5BEqtwbJDPl0Fgm6Vb8JBWS7BLLm++JKUeuNVpj5NVoQ3hJH8Xb3I8qfmyBCIeonMEWAztkkvH3jGy8bNUQxJdz7Fnt4X6AY7FcBUlReJwF8w0KHEh/MGcDyKEDoTgf7KMgEuaR0SaOQhvPjfaW7ggYqZyK7BZeqTr54V7xgqyj4pJHgGFGt7XNoSlmkCHIjPDCMo3Jo80dMPGdq3+WQXo6TyRU90zeDnVNWtM7fVZ3iQAG6fgJG6UxSG1Rpwza8rwvXRIGfLd2oE9RelEOXGEYk8Lc/s4ypWCna7prfTcz2lwaWGQkJ/eAXBTX9xtnRO6t0y6cQ+NCLC7BDRggo6hzn66iwZQQM4ZndX1xtP3vC7DAyJtWEHL5w3egoR22RtBnpyam7MtipTGAFPAIjZ/Wx/GYYmUgH1SnjwS/lZ6FTBL6fKmxJChMyZ/nyorD/e7fgBXfae7DTHKh/R5Mb1wragXmMj5lAh3YjguZ3Mlt2vZszNJ0lh1x+3Vi7whzG8NWMsQmFGZk1Ki24bQw9d0+qw9FvlzX5Hsx2pV2/+E0WJRwfT6M7QYZDxxEugyv7zBH7bKrUFEmHh1oVivCgbhCIOC1Nug/wEpPKpkE/WvZJWT656ZHkbuBAaw1XxMXWyYcA8z2hawNcXf3iRAdYyrPMbnOrJc= X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB03.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230028)(4636009)(346002)(376002)(396003)(136003)(39860400002)(451199021)(36840700001)(46966006)(40470700004)(36860700001)(426003)(336012)(83380400001)(47076005)(478600001)(6666004)(110136005)(54906003)(2616005)(1076003)(26005)(186003)(44832011)(30864003)(2906002)(36756003)(40460700003)(356005)(70206006)(82310400005)(4326008)(82740400003)(41300700001)(81166007)(70586007)(8936002)(40480700001)(8676002)(86362001)(316002)(5660300002)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2023 11:18:39.9186 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b9a9b8ed-fdd5-470d-845c-08db4fb5f9ab X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT111.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5200 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org CDX bus supports multiple type of devices, which can be exposed to user-space via vfio-cdx. vfio-cdx provides the MMIO IO_MEMORY regions as well as the DMA interface for the device (IOMMU). This support aims to enable the DPDK to support the cdx devices in user-space using VFIO interface. Signed-off-by: Nipun Gupta --- MAINTAINERS | 5 + doc/guides/rel_notes/release_23_07.rst | 6 + drivers/bus/cdx/bus_cdx_driver.h | 201 ++++++++++ drivers/bus/cdx/cdx.c | 520 +++++++++++++++++++++++++ drivers/bus/cdx/cdx_logs.h | 37 ++ drivers/bus/cdx/cdx_vfio.c | 437 +++++++++++++++++++++ drivers/bus/cdx/meson.build | 13 + drivers/bus/cdx/private.h | 49 +++ drivers/bus/cdx/version.map | 11 + drivers/bus/meson.build | 1 + 10 files changed, 1280 insertions(+) create mode 100644 drivers/bus/cdx/bus_cdx_driver.h create mode 100644 drivers/bus/cdx/cdx.c create mode 100644 drivers/bus/cdx/cdx_logs.h create mode 100644 drivers/bus/cdx/cdx_vfio.c create mode 100644 drivers/bus/cdx/meson.build create mode 100644 drivers/bus/cdx/private.h create mode 100644 drivers/bus/cdx/version.map diff --git a/MAINTAINERS b/MAINTAINERS index 8df23e5099..1f9b6af9b9 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -569,6 +569,11 @@ M: Parav Pandit M: Xueming Li F: drivers/bus/auxiliary/ +CDX bus driver +M: Nipun Gupta +M: Nikhil Agarwal +F: drivers/bus/cdx/ + Intel FPGA bus M: Rosen Xu F: drivers/bus/ifpga/ diff --git a/doc/guides/rel_notes/release_23_07.rst b/doc/guides/rel_notes/release_23_07.rst index a9b1293689..a5f581dc3d 100644 --- a/doc/guides/rel_notes/release_23_07.rst +++ b/doc/guides/rel_notes/release_23_07.rst @@ -55,6 +55,12 @@ New Features Also, make sure to start the actual text at the margin. ======================================================= +* **Added CDX bus support.** + + CDX bus driver has been added to support AMD CDX bus, which operates + on FPGA based CDX devices. The CDX devices are memory mapped on system + bus for embedded CPUs. + Removed Items ------------- diff --git a/drivers/bus/cdx/bus_cdx_driver.h b/drivers/bus/cdx/bus_cdx_driver.h new file mode 100644 index 0000000000..7edcb019eb --- /dev/null +++ b/drivers/bus/cdx/bus_cdx_driver.h @@ -0,0 +1,201 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2022-2023, Advanced Micro Devices, Inc. + */ + +#ifndef _BUS_CDX_DRIVER_H_ +#define _BUS_CDX_DRIVER_H_ + +/** + * @file + * + * CDX bus interface + */ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include + +/* Forward declarations */ +struct rte_cdx_device; +struct rte_cdx_driver; + +#define CDX_MAX_RESOURCE 4 + +/** List of CDX devices */ +RTE_TAILQ_HEAD(rte_cdx_device_list, rte_cdx_device); +/** List of CDX drivers */ +RTE_TAILQ_HEAD(rte_cdx_driver_list, rte_cdx_driver); + +/* CDX Bus iterators */ +#define FOREACH_DEVICE_ON_CDXBUS(p) \ + RTE_TAILQ_FOREACH(p, &rte_cdx_bus.device_list, next) + +#define FOREACH_DRIVER_ON_CDXBUS(p) \ + RTE_TAILQ_FOREACH(p, &rte_cdx_bus.driver_list, next) + +/** Any CDX device identifier (vendor, device) */ +#define RTE_CDX_ANY_ID (0xffff) + +#define RTE_PMD_REGISTER_CDX_TABLE(name, table) \ +static const char DRV_EXP_TAG(name, cdx_tbl_export)[] __rte_used = \ +RTE_STR(table) + +/** + * A structure describing an ID for a CDX driver. Each driver provides a + * table of these IDs for each device that it supports. + */ +struct rte_cdx_id { + uint16_t vendor_id; /**< Vendor ID. */ + uint16_t device_id; /**< Device ID. */ +}; + +/** + * A structure describing a CDX device. + */ +struct rte_cdx_device { + RTE_TAILQ_ENTRY(rte_cdx_device) next; /**< Next probed CDX device. */ + struct rte_device device; /**< Inherit core device */ + struct rte_cdx_id id; /**< CDX ID. */ + struct rte_mem_resource mem_resource[CDX_MAX_RESOURCE]; + /**< CDX Memory Resource */ +}; + +/** + * @internal + * Helper macro for drivers that need to convert to struct rte_cdx_device. + */ +#define RTE_DEV_TO_CDX_DEV(ptr) \ + container_of(ptr, struct rte_cdx_device, device) + +#define RTE_DEV_TO_CDX_DEV_CONST(ptr) \ + container_of(ptr, const struct rte_cdx_device, device) + +#define RTE_ETH_DEV_TO_CDX_DEV(eth_dev) RTE_DEV_TO_CDX_DEV((eth_dev)->device) + +#ifdef __cplusplus +/** C++ macro used to help building up tables of device IDs */ +#define RTE_CDX_DEVICE(vend, dev) \ + (vend), \ + (dev) +#else +/** Macro used to help building up tables of device IDs */ +#define RTE_CDX_DEVICE(vend, dev) \ + .vendor_id = (vend), \ + .device_id = (dev) +#endif + +/** + * Initialisation function for the driver called during CDX probing. + */ +typedef int (rte_cdx_probe_t)(struct rte_cdx_driver *, struct rte_cdx_device *); + +/** + * Uninitialisation function for the driver called during hotplugging. + */ +typedef int (rte_cdx_remove_t)(struct rte_cdx_device *); + +/** + * A structure describing a CDX driver. + */ +struct rte_cdx_driver { + RTE_TAILQ_ENTRY(rte_cdx_driver) next; /**< Next in list. */ + struct rte_driver driver; /**< Inherit core driver. */ + struct rte_cdx_bus *bus; /**< CDX bus reference. */ + rte_cdx_probe_t *probe; /**< Device probe function. */ + rte_cdx_remove_t *remove; /**< Device remove function. */ + const struct rte_cdx_id *id_table; /**< ID table, NULL terminated. */ + uint32_t drv_flags; /**< Flags RTE_CDX_DRV_*. */ +}; + +/** + * Structure describing the CDX bus + */ +struct rte_cdx_bus { + struct rte_bus bus; /**< Inherit the generic class */ + struct rte_cdx_device_list device_list; /**< List of CDX devices */ + struct rte_cdx_driver_list driver_list; /**< List of CDX drivers */ +}; + +/** + * Get Pathname of CDX devices directory. + * + * @return + * sysfs path for CDX devices. + */ +__rte_internal +const char *rte_cdx_get_sysfs_path(void); + +/** + * Map the CDX device resources in user space virtual memory address + * + * @param dev + * A pointer to a rte_cdx_device structure describing the device + * to use + * + * @return + * 0 on success, negative on error and positive if no driver + * is found for the device. + */ +__rte_internal +int rte_cdx_map_device(struct rte_cdx_device *dev); + +/** + * Unmap this device + * + * @param dev + * A pointer to a rte_cdx_device structure describing the device + * to use + */ +__rte_internal +void rte_cdx_unmap_device(struct rte_cdx_device *dev); + +/** + * Register a CDX driver. + * + * @param driver + * A pointer to a rte_cdx_driver structure describing the driver + * to be registered. + */ +__rte_internal +void rte_cdx_register(struct rte_cdx_driver *driver); + +/** + * Helper for CDX device registration from driver (eth, crypto, raw) instance + */ +#define RTE_PMD_REGISTER_CDX(nm, cdx_drv) \ + RTE_INIT(cdxinitfn_ ##nm) \ + {\ + (cdx_drv).driver.name = RTE_STR(nm);\ + rte_cdx_register(&cdx_drv); \ + } \ + RTE_PMD_EXPORT_NAME(nm, __COUNTER__) + +/** + * Unregister a CDX driver. + * + * @param driver + * A pointer to a rte_cdx_driver structure describing the driver + * to be unregistered. + */ +__rte_internal +void rte_cdx_unregister(struct rte_cdx_driver *driver); + +#ifdef __cplusplus +} +#endif + +#endif /* _BUS_CDX_DRIVER_H_ */ diff --git a/drivers/bus/cdx/cdx.c b/drivers/bus/cdx/cdx.c new file mode 100644 index 0000000000..d479daa315 --- /dev/null +++ b/drivers/bus/cdx/cdx.c @@ -0,0 +1,520 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2022-2023, Advanced Micro Devices, Inc. + */ + +/* + * Architecture Overview + * ===================== + * CDX is a Hardware Architecture designed for AMD FPGA devices. It + * consists of sophisticated mechanism for interaction between FPGA, + * Firmware and the APUs (Application CPUs). + * + * Firmware resides on RPU (Realtime CPUs) which interacts with + * the FPGA program manager and the APUs. The RPU provides memory-mapped + * interface (RPU if) which is used to communicate with APUs. + * + * The diagram below shows an overview of the CDX architecture: + * + * +--------------------------------------+ + * | DPDK | + * | DPDK CDX drivers | + * | | | + * | DPDK CDX bus | + * | | | + * +-----------------------------|--------+ + * | + * +-----------------------------|--------+ + * | Application CPUs (APU) | | + * | | | + * | VFIO CDX driver | + * | Linux OS | | + * | Linux CDX bus | + * | | | + * +-----------------------------|--------+ + * | + * | + * +------------------------| RPU if |----+ + * | | | + * | V | + * | Realtime CPUs (RPU) | + * | | + * +--------------------------------------+ + * | + * +---------------------|----------------+ + * | FPGA | | + * | +-----------------------+ | + * | | | | | + * | +-------+ +-------+ +-------+ | + * | | dev 1 | | dev 2 | | dev 3 | | + * | +-------+ +-------+ +-------+ | + * +--------------------------------------+ + * + * The RPU firmware extracts the device information from the loaded FPGA + * image and implements a mechanism that allows the APU drivers to + * enumerate such devices (device personality and resource details) via + * a dedicated communication channel. + * + * VFIO CDX driver provides the CDX device resources like MMIO and interrupts + * to map to user-space. DPDK CDX bus uses sysfs interface and the vfio-cdx + * driver to discover and initialize the CDX devices for user-space + * applications. + */ + +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "bus_cdx_driver.h" +#include "cdx_logs.h" +#include "private.h" + +#define SYSFS_CDX_DEVICES "/sys/bus/cdx/devices" +#define CDX_BUS_NAME cdx +#define CDX_DEV_PREFIX "cdx-" + +/** + * @file + * CDX probing using Linux sysfs. + */ + +/* Add a device to CDX bus */ +static void +cdx_add_device(struct rte_cdx_device *cdx_dev) +{ + TAILQ_INSERT_TAIL(&rte_cdx_bus.device_list, cdx_dev, next); +} + +static int +cdx_get_kernel_driver_by_path(const char *filename, char *driver_name, + size_t len) +{ + int count; + char path[PATH_MAX]; + char *name; + + if (!filename || !driver_name) + return -1; + + count = readlink(filename, path, PATH_MAX); + if (count >= PATH_MAX) + return -1; + + /* For device does not have a driver */ + if (count < 0) + return 1; + + path[count] = '\0'; + + name = strrchr(path, '/'); + if (name) { + strlcpy(driver_name, name + 1, len); + return 0; + } + + return -1; +} + +int rte_cdx_map_device(struct rte_cdx_device *dev) +{ + return cdx_vfio_map_resource(dev); +} + +void rte_cdx_unmap_device(struct rte_cdx_device *dev) +{ + cdx_vfio_unmap_resource(dev); +} + +static struct rte_devargs * +cdx_devargs_lookup(const char *dev_name) +{ + struct rte_devargs *devargs; + + RTE_EAL_DEVARGS_FOREACH("cdx", devargs) { + if (strcmp(devargs->name, dev_name) == 0) + return devargs; + } + return NULL; +} + +static bool +cdx_ignore_device(const char *dev_name) +{ + struct rte_devargs *devargs = cdx_devargs_lookup(dev_name); + + switch (rte_cdx_bus.bus.conf.scan_mode) { + case RTE_BUS_SCAN_ALLOWLIST: + if (devargs && devargs->policy == RTE_DEV_ALLOWED) + return false; + break; + case RTE_BUS_SCAN_UNDEFINED: + case RTE_BUS_SCAN_BLOCKLIST: + if (devargs == NULL || devargs->policy != RTE_DEV_BLOCKED) + return false; + break; + } + return true; +} + +/* + * Scan one cdx sysfs entry, and fill the devices list from it. + * It checks if the CDX device is bound to vfio-cdx driver. In case + * the device is vfio bound, it reads the vendor and device id and + * stores it for device-driver matching. + */ +static int +cdx_scan_one(const char *dirname, const char *dev_name) +{ + char filename[PATH_MAX]; + struct rte_cdx_device *dev = NULL; + char driver[PATH_MAX]; + unsigned long tmp; + char *name = NULL; + int ret; + + dev = calloc(1, sizeof(*dev)); + if (!dev) + return -ENOMEM; + + name = calloc(1, RTE_DEV_NAME_MAX_LEN); + if (!name) { + ret = -ENOMEM; + goto err; + } + + dev->device.bus = &rte_cdx_bus.bus; + memcpy(name, dev_name, RTE_DEV_NAME_MAX_LEN); + dev->device.name = name; + + /* parse driver */ + snprintf(filename, sizeof(filename), "%s/driver", dirname); + ret = cdx_get_kernel_driver_by_path(filename, driver, sizeof(driver)); + if (ret < 0) { + CDX_BUS_ERR("Fail to get kernel driver"); + ret = -1; + goto err; + } + + /* + * Check if device is bound to 'vfio-cdx' driver, so that user-space + * can gracefully access the device. + */ + if (ret || strcmp(driver, "vfio-cdx")) { + ret = 0; + goto err; + } + + /* get vendor id */ + snprintf(filename, sizeof(filename), "%s/vendor", dirname); + if (eal_parse_sysfs_value(filename, &tmp) < 0) { + ret = -1; + goto err; + } + dev->id.vendor_id = (uint16_t)tmp; + + /* get device id */ + snprintf(filename, sizeof(filename), "%s/device", dirname); + if (eal_parse_sysfs_value(filename, &tmp) < 0) { + free(dev); + return -1; + } + dev->id.device_id = (uint16_t)tmp; + + cdx_add_device(dev); + + return 0; + +err: + if (name) + free(name); + if (dev) + free(dev); + return ret; +} + +/* + * Scan the content of the CDX bus, and the devices in the devices + * list. + */ +static int +cdx_scan(void) +{ + struct dirent *e; + DIR *dir; + char dirname[PATH_MAX]; + + dir = opendir(rte_cdx_get_sysfs_path()); + if (dir == NULL) { + CDX_BUS_ERR("%s(): opendir failed: %s", __func__, + strerror(errno)); + return -1; + } + + while ((e = readdir(dir)) != NULL) { + if (e->d_name[0] == '.') + continue; + + if (cdx_ignore_device(e->d_name)) + continue; + + snprintf(dirname, sizeof(dirname), "%s/%s", + rte_cdx_get_sysfs_path(), e->d_name); + + if (cdx_scan_one(dirname, e->d_name) < 0) + goto error; + } + closedir(dir); + return 0; + +error: + closedir(dir); + return -1; +} + +const char * +rte_cdx_get_sysfs_path(void) +{ + return SYSFS_CDX_DEVICES; +} + +/* map a particular resource from a file */ +void * +cdx_map_resource(void *requested_addr, int fd, uint64_t offset, size_t size, + int additional_flags) +{ + void *mapaddr; + + /* Map the cdx MMIO memory resource of device */ + mapaddr = rte_mem_map(requested_addr, size, + RTE_PROT_READ | RTE_PROT_WRITE, + RTE_MAP_SHARED | additional_flags, fd, offset); + if (mapaddr == NULL) { + CDX_BUS_ERR("%s(): cannot map resource(%d, %p, 0x%zx, 0x%"PRIx64"): %s (%p)", + __func__, fd, requested_addr, size, offset, + rte_strerror(rte_errno), mapaddr); + } + CDX_BUS_DEBUG("CDX MMIO memory mapped at %p", mapaddr); + + return mapaddr; +} + +/* unmap a particular resource */ +void +cdx_unmap_resource(void *requested_addr, size_t size) +{ + if (requested_addr == NULL) + return; + + /* Unmap the CDX memory resource of device */ + if (rte_mem_unmap(requested_addr, size)) { + CDX_BUS_ERR("%s(): cannot mem unmap(%p, %#zx): %s", __func__, + requested_addr, size, rte_strerror(rte_errno)); + } + CDX_BUS_DEBUG("CDX memory unmapped at %p", requested_addr); +} +/* + * Match the CDX Driver and Device using device id and vendor id. + */ +static int +cdx_match(const struct rte_cdx_driver *cdx_drv, + const struct rte_cdx_device *cdx_dev) +{ + const struct rte_cdx_id *id_table; + + for (id_table = cdx_drv->id_table; id_table->vendor_id != 0; + id_table++) { + /* check if device's identifiers match the driver's ones */ + if (id_table->vendor_id != cdx_dev->id.vendor_id && + id_table->vendor_id != RTE_CDX_ANY_ID) + continue; + if (id_table->device_id != cdx_dev->id.device_id && + id_table->device_id != RTE_CDX_ANY_ID) + continue; + + return 1; + } + + return 0; +} + +/* + * If vendor id and device id match, call the probe() function of the + * driver. + */ +static int +cdx_probe_one_driver(struct rte_cdx_driver *dr, + struct rte_cdx_device *dev) +{ + const char *dev_name = dev->device.name; + bool already_probed; + int ret; + + if ((dr == NULL) || (dev == NULL)) + return -EINVAL; + + /* The device is not blocked; Check if driver supports it */ + if (!cdx_match(dr, dev)) + /* Match of device and driver failed */ + return 1; + + already_probed = rte_dev_is_probed(&dev->device); + if (already_probed) { + CDX_BUS_INFO("Device %s is already probed", dev->device.name); + return -EEXIST; + } + + CDX_BUS_DEBUG(" probe device %s using driver: %s", dev_name, + dr->driver.name); + + ret = cdx_vfio_map_resource(dev); + if (ret != 0) { + CDX_BUS_ERR("CDX map device failed: %d", ret); + goto error_map_device; + } + + /* call the driver probe() function */ + ret = dr->probe(dr, dev); + if (ret) { + CDX_BUS_ERR("Probe CDX driver: %s device: %s failed: %d", + dr->driver.name, dev_name, ret); + goto error_probe; + } else { + dev->device.driver = &dr->driver; + } + + return ret; + +error_probe: + cdx_vfio_unmap_resource(dev); +error_map_device: + return ret; +} + +/* + * If vendor/device ID match, call the probe() function of all + * registered driver for the given device. Return < 0 if initialization + * failed, return 1 if no driver is found for this device. + */ +static int +cdx_probe_all_drivers(struct rte_cdx_device *dev) +{ + struct rte_cdx_driver *dr = NULL; + int rc = 0; + + if (dev == NULL) + return -EINVAL; + + FOREACH_DRIVER_ON_CDXBUS(dr) { + rc = cdx_probe_one_driver(dr, dev); + if (rc < 0) + /* negative value is an error */ + return rc; + if (rc > 0) + /* positive value means driver doesn't support it */ + continue; + return 0; + } + return 1; +} + +/* + * Scan the content of the CDX bus, and call the probe() function for + * all registered drivers that have a matching entry in its id_table + * for discovered devices. + */ +static int +cdx_probe(void) +{ + struct rte_cdx_device *dev = NULL; + size_t probed = 0, failed = 0; + int ret = 0; + + FOREACH_DEVICE_ON_CDXBUS(dev) { + probed++; + + ret = cdx_probe_all_drivers(dev); + if (ret < 0) { + CDX_BUS_ERR("Requested device %s cannot be used", + dev->device.name); + rte_errno = errno; + failed++; + ret = 0; + } + } + + return (probed && probed == failed) ? -1 : 0; +} + +static int +cdx_parse(const char *name, void *addr) +{ + const char **out = addr; + int ret; + + ret = strncmp(name, CDX_DEV_PREFIX, strlen(CDX_DEV_PREFIX)); + + if (ret == 0 && addr) + *out = name; + + return ret; +} + +/* register a driver */ +void +rte_cdx_register(struct rte_cdx_driver *driver) +{ + TAILQ_INSERT_TAIL(&rte_cdx_bus.driver_list, driver, next); + driver->bus = &rte_cdx_bus; +} + +/* unregister a driver */ +void +rte_cdx_unregister(struct rte_cdx_driver *driver) +{ + TAILQ_REMOVE(&rte_cdx_bus.driver_list, driver, next); + driver->bus = NULL; +} + +static struct rte_device * +cdx_find_device(const struct rte_device *start, rte_dev_cmp_t cmp, + const void *data) +{ + const struct rte_cdx_device *cdx_start; + struct rte_cdx_device *cdx_dev; + + if (start != NULL) { + cdx_start = RTE_DEV_TO_CDX_DEV_CONST(start); + cdx_dev = TAILQ_NEXT(cdx_start, next); + } else { + cdx_dev = TAILQ_FIRST(&rte_cdx_bus.device_list); + } + while (cdx_dev != NULL) { + if (cmp(&cdx_dev->device, data) == 0) + return &cdx_dev->device; + cdx_dev = TAILQ_NEXT(cdx_dev, next); + } + return NULL; +} + +struct rte_cdx_bus rte_cdx_bus = { + .bus = { + .scan = cdx_scan, + .probe = cdx_probe, + .find_device = cdx_find_device, + .parse = cdx_parse, + }, + .device_list = TAILQ_HEAD_INITIALIZER(rte_cdx_bus.device_list), + .driver_list = TAILQ_HEAD_INITIALIZER(rte_cdx_bus.driver_list), +}; + +RTE_REGISTER_BUS(cdx, rte_cdx_bus.bus); +RTE_LOG_REGISTER_DEFAULT(cdx_logtype_bus, NOTICE); diff --git a/drivers/bus/cdx/cdx_logs.h b/drivers/bus/cdx/cdx_logs.h new file mode 100644 index 0000000000..e5e23d8b94 --- /dev/null +++ b/drivers/bus/cdx/cdx_logs.h @@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2022-2023, Advanced Micro Devices, Inc. + */ + +#ifndef _CDX_LOGS_H_ +#define _CDX_LOGS_H_ + +extern int cdx_logtype_bus; + +#define CDX_BUS_LOG(level, fmt, args...) \ + rte_log(RTE_LOG_ ## level, cdx_logtype_bus, "cdx: " fmt "\n", \ + ##args) + +/* Debug logs are with Function names */ +#define CDX_BUS_DEBUG(fmt, args...) \ + rte_log(RTE_LOG_DEBUG, cdx_logtype_bus, "cdx: %s(): " fmt "\n", \ + __func__, ##args) + +#define CDX_BUS_INFO(fmt, args...) \ + CDX_BUS_LOG(INFO, fmt, ## args) +#define CDX_BUS_ERR(fmt, args...) \ + CDX_BUS_LOG(ERR, fmt, ## args) +#define CDX_BUS_WARN(fmt, args...) \ + CDX_BUS_LOG(WARNING, fmt, ## args) + +/* DP Logs, toggled out at compile time if level lower than current level */ +#define CDX_BUS_DP_LOG(level, fmt, args...) \ + RTE_LOG_DP(level, PMD, fmt, ## args) + +#define CDX_BUS_DP_DEBUG(fmt, args...) \ + CDX_BUS_DP_LOG(DEBUG, fmt, ## args) +#define CDX_BUS_DP_INFO(fmt, args...) \ + CDX_BUS_DP_LOG(INFO, fmt, ## args) +#define CDX_BUS_DP_WARN(fmt, args...) \ + CDX_BUS_DP_LOG(WARNING, fmt, ## args) + +#endif /* _CDX_LOGS_H_ */ diff --git a/drivers/bus/cdx/cdx_vfio.c b/drivers/bus/cdx/cdx_vfio.c new file mode 100644 index 0000000000..ae11f589b3 --- /dev/null +++ b/drivers/bus/cdx/cdx_vfio.c @@ -0,0 +1,437 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2022-2023, Advanced Micro Devices, Inc. + */ + +/** + * @file + * CDX probing using Linux VFIO. + * + * This code tries to determine if the CDX device is bound to VFIO driver, + * and initialize it (map MMIO regions, set up interrupts) if that's the case. + * + */ + +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "eal_filesystem.h" + +#include "bus_cdx_driver.h" +#include "cdx_logs.h" +#include "private.h" + +/** + * A structure describing a CDX mapping. + */ +struct cdx_map { + void *addr; + char *path; + uint64_t offset; + uint64_t size; +}; + +/** + * A structure describing a mapped CDX resource. + * For multi-process we need to reproduce all CDX mappings in secondary + * processes, so save them in a tailq. + */ +struct mapped_cdx_resource { + TAILQ_ENTRY(mapped_cdx_resource) next; + char name[RTE_DEV_NAME_MAX_LEN]; /**< CDX device name */ + char path[PATH_MAX]; + int nb_maps; + struct cdx_map maps[CDX_MAX_RESOURCE]; +}; + +/** mapped cdx device list */ +TAILQ_HEAD(mapped_cdx_res_list, mapped_cdx_resource); + +static struct rte_tailq_elem cdx_vfio_tailq = { + .name = "VFIO_CDX_RESOURCE_LIST", +}; +EAL_REGISTER_TAILQ(cdx_vfio_tailq) + +static struct mapped_cdx_resource * +cdx_vfio_find_and_unmap_resource(struct mapped_cdx_res_list *vfio_res_list, + struct rte_cdx_device *dev) +{ + struct mapped_cdx_resource *vfio_res = NULL; + const char *dev_name = dev->device.name; + struct cdx_map *maps; + int i; + + /* Get vfio_res */ + TAILQ_FOREACH(vfio_res, vfio_res_list, next) { + if (strcmp(vfio_res->name, dev_name)) + continue; + break; + } + + if (vfio_res == NULL) + return vfio_res; + + CDX_BUS_INFO("Releasing CDX mapped resource for %s", dev_name); + + maps = vfio_res->maps; + for (i = 0; i < vfio_res->nb_maps; i++) { + if (maps[i].addr) { + CDX_BUS_DEBUG("Calling cdx_unmap_resource for %s at %p", + dev_name, maps[i].addr); + cdx_unmap_resource(maps[i].addr, maps[i].size); + } + } + + return vfio_res; +} + +static int +cdx_vfio_unmap_resource_primary(struct rte_cdx_device *dev) +{ + char cdx_addr[PATH_MAX] = {0}; + struct mapped_cdx_resource *vfio_res = NULL; + struct mapped_cdx_res_list *vfio_res_list; + + vfio_res_list = + RTE_TAILQ_CAST(cdx_vfio_tailq.head, mapped_cdx_res_list); + vfio_res = cdx_vfio_find_and_unmap_resource(vfio_res_list, dev); + + /* if we haven't found our tailq entry, something's wrong */ + if (vfio_res == NULL) { + CDX_BUS_ERR("%s cannot find TAILQ entry for cdx device!", + cdx_addr); + return -1; + } + + TAILQ_REMOVE(vfio_res_list, vfio_res, next); + rte_free(vfio_res); + return 0; +} + +static int +cdx_vfio_unmap_resource_secondary(struct rte_cdx_device *dev) +{ + struct mapped_cdx_resource *vfio_res = NULL; + struct mapped_cdx_res_list *vfio_res_list; + + vfio_res_list = + RTE_TAILQ_CAST(cdx_vfio_tailq.head, mapped_cdx_res_list); + vfio_res = cdx_vfio_find_and_unmap_resource(vfio_res_list, dev); + + /* if we haven't found our tailq entry, something's wrong */ + if (vfio_res == NULL) { + CDX_BUS_ERR("%s cannot find TAILQ entry for CDX device!", + dev->device.name); + return -1; + } + + return 0; +} + +int +cdx_vfio_unmap_resource(struct rte_cdx_device *dev) +{ + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + return cdx_vfio_unmap_resource_primary(dev); + else + return cdx_vfio_unmap_resource_secondary(dev); +} + +static int +cdx_rte_vfio_setup_device(int vfio_dev_fd) +{ + /* + * Reset the device. If the device is not capable of resetting, + * then it updates errno as EINVAL. + */ + if (ioctl(vfio_dev_fd, VFIO_DEVICE_RESET) && errno != EINVAL) { + CDX_BUS_ERR("Unable to reset device! Error: %d (%s)", errno, + strerror(errno)); + return -1; + } + + return 0; +} + +static int +cdx_vfio_mmap_resource(int vfio_dev_fd, struct mapped_cdx_resource *vfio_res, + int index, int additional_flags) +{ + struct cdx_map *map = &vfio_res->maps[index]; + void *vaddr; + + if (map->size == 0) { + CDX_BUS_DEBUG("map size is 0, skip region %d", index); + return 0; + } + + /* reserve the address using an inaccessible mapping */ + vaddr = mmap(map->addr, map->size, 0, MAP_PRIVATE | + MAP_ANONYMOUS | additional_flags, -1, 0); + if (vaddr != MAP_FAILED) { + void *map_addr = NULL; + + if (map->size) { + /* actual map of first part */ + map_addr = cdx_map_resource(vaddr, vfio_dev_fd, + map->offset, map->size, + RTE_MAP_FORCE_ADDRESS); + } + + if (map_addr == NULL) { + munmap(vaddr, map->size); + vaddr = MAP_FAILED; + CDX_BUS_ERR("Failed to map cdx MMIO region %d", index); + return -1; + } + } else { + CDX_BUS_ERR("Failed to create inaccessible mapping for MMIO region %d", + index); + return -1; + } + + map->addr = vaddr; + return 0; +} + +/* + * region info may contain capability headers, so we need to keep reallocating + * the memory until we match allocated memory size with argsz. + */ +static int +cdx_vfio_get_region_info(int vfio_dev_fd, struct vfio_region_info **info, + int region) +{ + struct vfio_region_info *ri; + size_t argsz = sizeof(*ri); + int ret; + + ri = malloc(sizeof(*ri)); + if (ri == NULL) { + CDX_BUS_ERR("Cannot allocate memory for VFIO region info"); + return -1; + } +again: + memset(ri, 0, argsz); + ri->argsz = argsz; + ri->index = region; + + ret = ioctl(vfio_dev_fd, VFIO_DEVICE_GET_REGION_INFO, ri); + if (ret < 0) { + free(ri); + return ret; + } + if (ri->argsz != argsz) { + struct vfio_region_info *tmp; + + argsz = ri->argsz; + tmp = realloc(ri, argsz); + + if (tmp == NULL) { + /* realloc failed but the ri is still there */ + free(ri); + CDX_BUS_ERR("Cannot reallocate memory for VFIO region info"); + return -1; + } + ri = tmp; + goto again; + } + *info = ri; + + return 0; +} + +static int +find_max_end_va(const struct rte_memseg_list *msl, void *arg) +{ + size_t sz = msl->len; + void *end_va = RTE_PTR_ADD(msl->base_va, sz); + void **max_va = arg; + + if (*max_va < end_va) + *max_va = end_va; + return 0; +} + +static void * +cdx_find_max_end_va(void) +{ + void *va = NULL; + + rte_memseg_list_walk(find_max_end_va, &va); + return va; +} + +static int +cdx_vfio_map_resource_primary(struct rte_cdx_device *dev) +{ + struct vfio_device_info device_info = { .argsz = sizeof(device_info) }; + char cdx_addr[PATH_MAX] = {0}; + static void *cdx_map_addr; + struct mapped_cdx_resource *vfio_res = NULL; + struct mapped_cdx_res_list *vfio_res_list = + RTE_TAILQ_CAST(cdx_vfio_tailq.head, mapped_cdx_res_list); + const char *dev_name = dev->device.name; + struct cdx_map *maps; + int vfio_dev_fd, i, ret; + + ret = rte_vfio_setup_device(rte_cdx_get_sysfs_path(), dev_name, + &vfio_dev_fd, &device_info); + if (ret) + return ret; + + /* allocate vfio_res and get region info */ + vfio_res = rte_zmalloc("VFIO_RES", sizeof(*vfio_res), 0); + if (vfio_res == NULL) { + CDX_BUS_ERR("Cannot store VFIO mmap details"); + goto err_vfio_dev_fd; + } + memcpy(vfio_res->name, dev_name, RTE_DEV_NAME_MAX_LEN); + + /* get number of registers */ + vfio_res->nb_maps = device_info.num_regions; + + /* map memory regions */ + maps = vfio_res->maps; + + for (i = 0; i < vfio_res->nb_maps; i++) { + struct vfio_region_info *reg = NULL; + void *vaddr; + + ret = cdx_vfio_get_region_info(vfio_dev_fd, ®, i); + if (ret < 0) { + CDX_BUS_ERR("%s cannot get device region info error %i (%s)", + dev_name, errno, strerror(errno)); + goto err_vfio_res; + } + + /* skip non-mmappable regions */ + if ((reg->flags & VFIO_REGION_INFO_FLAG_MMAP) == 0) { + free(reg); + continue; + } + + /* try mapping somewhere close to the end of hugepages */ + if (cdx_map_addr == NULL) + cdx_map_addr = cdx_find_max_end_va(); + + vaddr = cdx_map_addr; + cdx_map_addr = RTE_PTR_ADD(vaddr, (size_t)reg->size); + + cdx_map_addr = RTE_PTR_ALIGN(cdx_map_addr, + sysconf(_SC_PAGE_SIZE)); + + maps[i].addr = vaddr; + maps[i].offset = reg->offset; + maps[i].size = reg->size; + maps[i].path = NULL; /* vfio doesn't have per-resource paths */ + + ret = cdx_vfio_mmap_resource(vfio_dev_fd, vfio_res, i, 0); + if (ret < 0) { + CDX_BUS_ERR("%s mapping region %i failed: %s", + cdx_addr, i, strerror(errno)); + free(reg); + goto err_vfio_res; + } + + dev->mem_resource[i].addr = maps[i].addr; + dev->mem_resource[i].len = maps[i].size; + + free(reg); + } + + if (cdx_rte_vfio_setup_device(vfio_dev_fd) < 0) { + CDX_BUS_ERR("%s setup device failed", dev_name); + goto err_vfio_res; + } + + TAILQ_INSERT_TAIL(vfio_res_list, vfio_res, next); + + return 0; +err_vfio_res: + cdx_vfio_find_and_unmap_resource(vfio_res_list, dev); + rte_free(vfio_res); +err_vfio_dev_fd: + rte_vfio_release_device(rte_cdx_get_sysfs_path(), + dev_name, vfio_dev_fd); + return -1; +} + +static int +cdx_vfio_map_resource_secondary(struct rte_cdx_device *dev) +{ + struct vfio_device_info device_info = { .argsz = sizeof(device_info) }; + char cdx_addr[PATH_MAX] = {0}; + int vfio_dev_fd; + int i, ret; + struct mapped_cdx_resource *vfio_res = NULL; + struct mapped_cdx_res_list *vfio_res_list = + RTE_TAILQ_CAST(cdx_vfio_tailq.head, mapped_cdx_res_list); + const char *dev_name = dev->device.name; + struct cdx_map *maps; + + /* if we're in a secondary process, just find our tailq entry */ + TAILQ_FOREACH(vfio_res, vfio_res_list, next) { + if (strcmp(vfio_res->name, dev_name)) + continue; + break; + } + /* if we haven't found our tailq entry, something's wrong */ + if (vfio_res == NULL) { + CDX_BUS_ERR("%s cannot find TAILQ entry for cdx device!", + dev_name); + return -1; + } + + ret = rte_vfio_setup_device(rte_cdx_get_sysfs_path(), dev_name, + &vfio_dev_fd, &device_info); + if (ret) + return ret; + + /* map MMIO regions */ + maps = vfio_res->maps; + + for (i = 0; i < vfio_res->nb_maps; i++) { + ret = cdx_vfio_mmap_resource(vfio_dev_fd, vfio_res, i, MAP_FIXED); + if (ret < 0) { + CDX_BUS_ERR("%s mapping MMIO region %i failed: %s", + dev_name, i, strerror(errno)); + goto err_vfio_dev_fd; + } + + dev->mem_resource[i].addr = maps[i].addr; + dev->mem_resource[i].len = maps[i].size; + } + + return 0; +err_vfio_dev_fd: + rte_vfio_release_device(rte_cdx_get_sysfs_path(), + cdx_addr, vfio_dev_fd); + return -1; +} + +/* + * map the CDX resources of a CDX device in virtual memory (VFIO version). + * primary and secondary processes follow almost exactly the same path + */ +int +cdx_vfio_map_resource(struct rte_cdx_device *dev) +{ + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + return cdx_vfio_map_resource_primary(dev); + else + return cdx_vfio_map_resource_secondary(dev); +} diff --git a/drivers/bus/cdx/meson.build b/drivers/bus/cdx/meson.build new file mode 100644 index 0000000000..f2ca104d34 --- /dev/null +++ b/drivers/bus/cdx/meson.build @@ -0,0 +1,13 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (C) 2022-2023, Advanced Micro Devices, Inc. + +if not is_linux + build = false + reason = 'only supported on Linux' +endif + +driver_sdk_headers = files('bus_cdx_driver.h') +sources = files( + 'cdx.c', + 'cdx_vfio.c', +) diff --git a/drivers/bus/cdx/private.h b/drivers/bus/cdx/private.h new file mode 100644 index 0000000000..c4ca76dc1b --- /dev/null +++ b/drivers/bus/cdx/private.h @@ -0,0 +1,49 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (C) 2022-2023, Advanced Micro Devices, Inc. + */ + +#ifndef _CDX_PRIVATE_H_ +#define _CDX_PRIVATE_H_ + +#include +#include + +#include "bus_cdx_driver.h" + +extern struct rte_cdx_bus rte_cdx_bus; + +/** + * Map a particular resource from a file. + * + * @param requested_addr + * The starting address for the new mapping range. + * @param fd + * The file descriptor. + * @param offset + * The offset for the mapping range. + * @param size + * The size for the mapping range. + * @param additional_flags + * The additional rte_mem_map() flags for the mapping range. + * @return + * - On success, the function returns a pointer to the mapped area. + * - On error, NULL is returned. + */ +void *cdx_map_resource(void *requested_addr, int fd, uint64_t offset, + size_t size, int additional_flags); + +/** + * Unmap a particular resource. + * + * @param requested_addr + * The address for the unmapping range. + * @param size + * The size for the unmapping range. + */ +void cdx_unmap_resource(void *requested_addr, size_t size); + +/* map/unmap VFIO resource */ +int cdx_vfio_map_resource(struct rte_cdx_device *dev); +int cdx_vfio_unmap_resource(struct rte_cdx_device *dev); + +#endif /* _CDX_PRIVATE_H_ */ diff --git a/drivers/bus/cdx/version.map b/drivers/bus/cdx/version.map new file mode 100644 index 0000000000..957fcab978 --- /dev/null +++ b/drivers/bus/cdx/version.map @@ -0,0 +1,11 @@ +INTERNAL { + global: + + rte_cdx_get_sysfs_path; + rte_cdx_map_device; + rte_cdx_register; + rte_cdx_unmap_device; + rte_cdx_unregister; + + local: *; +}; diff --git a/drivers/bus/meson.build b/drivers/bus/meson.build index 6d2520c543..a78b4283bf 100644 --- a/drivers/bus/meson.build +++ b/drivers/bus/meson.build @@ -3,6 +3,7 @@ drivers = [ 'auxiliary', + 'cdx', 'dpaa', 'fslmc', 'ifpga', From patchwork Mon May 8 11:18:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Gupta, Nipun" X-Patchwork-Id: 126760 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9DE7142A94; Mon, 8 May 2023 13:18:52 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 638F542D1A; Mon, 8 May 2023 13:18:47 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2046.outbound.protection.outlook.com [40.107.244.46]) by mails.dpdk.org (Postfix) with ESMTP id D8A6842D1A for ; Mon, 8 May 2023 13:18:45 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RJDYL5rU2Tufw9ROf5ISljVuIiWUX1fqP3LJcr7//lj9hn525QBpuprjk5BvsJKjIJO1CwYh/wtbu846wZOECflDHy4sduLq+4kNcsCEAWku4j4UWn6Smi/e6w+M/jD19lu8J+ZD1n0auFGIcgYomtYAM68LIyQtq17sVC5cWPRdRQKsqkz1sosYjjeBrO2HLVmZS+mL5dz2zuZiHvefLOxMp8T5ssaNJccdVjaFxkhXcwyNWhrcRubyqUlew7Pyved6oqXu4/dQb7Bm+XIaUk3FiwB4PrVxveRoN9hzG3/PRw7CCdnWH9Pdpd3xq+eJfWEQSbhP8s5n0HKAyxZTSg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HSSroEHH/p4mEPGHjbBg/tJA7WlY3xcGMa3jL2YuG8M=; b=MrvYsRAoFe+0FpsHX3tCNifcoFIpbQsbghO/RtBhVVGp00al1iQ15VDFMz+DnEKrwQthc3dTnzelcqCoBXrCDVmr0S9TcvfZXvee5gSe4AMJvimhpn1Nxdz6AGbFDl9sFEJkQiVMkAF+o2xtv4Nm/JXkfeLmfk4jsuIVEQ6voqBXH4kBdszdOCWplyC541YLqXzwJZ6JxhzY/k3uqHIR/4Ov4CbFgzNJY/tK9qCmzTKVtLxV273rs24wE8qIOGEt57nO59hjPaF7VXah5gnJrFmO5Pldd3dFPCmkDv8Z3N1UpbcOAsb1i34I8y3hT73WLFqFCrPyz8vqlCLbtSg7mw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=dpdk.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HSSroEHH/p4mEPGHjbBg/tJA7WlY3xcGMa3jL2YuG8M=; b=tN4QxynlI2M8oidwGIuMpscAlgJOqQOiwwlqIx7sQr1IdLrkTESGAMF+VzzgNHP8OAY0uO6Pb54JzitHxl9SDRs7ISZ+bXg5jhEFGqRmpsIsDQaiM1hwq3+LbLtcefSnwMcCietI6H28rZjwhfih09OdjILiyvSPQqNTsdjE5/A= Received: from BN9PR03CA0947.namprd03.prod.outlook.com (2603:10b6:408:108::22) by BL3PR12MB6476.namprd12.prod.outlook.com (2603:10b6:208:3bc::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6363.29; Mon, 8 May 2023 11:18:43 +0000 Received: from BN8NAM11FT014.eop-nam11.prod.protection.outlook.com (2603:10b6:408:108:cafe::e2) by BN9PR03CA0947.outlook.office365.com (2603:10b6:408:108::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6363.32 via Frontend Transport; Mon, 8 May 2023 11:18:43 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN8NAM11FT014.mail.protection.outlook.com (10.13.177.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6363.32 via Frontend Transport; Mon, 8 May 2023 11:18:43 +0000 Received: from SATLEXMB08.amd.com (10.181.40.132) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 8 May 2023 06:18:43 -0500 Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB08.amd.com (10.181.40.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 8 May 2023 04:18:42 -0700 Received: from xhdipdslab41.xilinx.com (10.180.168.240) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Mon, 8 May 2023 06:18:40 -0500 From: Nipun Gupta To: , , CC: , , , Nipun Gupta Subject: [PATCH v4 2/4] bus/cdx: add DMA map and unmap support Date: Mon, 8 May 2023 16:48:10 +0530 Message-ID: <20230508111812.2655-3-nipun.gupta@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230508111812.2655-1-nipun.gupta@amd.com> References: <20230124140746.594066-1-nipun.gupta@amd.com> <20230508111812.2655-1-nipun.gupta@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT014:EE_|BL3PR12MB6476:EE_ X-MS-Office365-Filtering-Correlation-Id: 717332e0-9859-4e78-7494-08db4fb5fbcc X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ayeQq8v9S4sY1rvlHWSaNBv4Ujqdp29jDtVrNyZfpc3qe5/0VAxR5u9Mq4XZCe2ngesXblxZVl5bk4g1tGMWPxbceP3xhhlYQgVoburGbdlikJvSZ9sWBJV8PMp9AekBQPcsWBhm0MWkt4kKUGOF2/ftIzglpHy6tfDhVUGYxCuHHhfZ7rq6jXAloG4Mvu5i3x/AULa8+bJw6DjWlxdf/3RLWbPvedSs7795/ZiEWc2U/690w3rXTjo85IEpp/KZTpaiAIgS0APlTDDHfrL3J5H2d30Rk/5hneWlhpdM3GExQrmPXAKyzdAD6Et0JYsdtbSJO3hOh3n1GJyVPRBrgOWE9ny4l0uovgDKbdaHfF0FQiQaUfmZxhJCPIq/ElgLTCF09YwtWHidoQ8+XnUMf4ZH7N7ThxQW80GVZAwkFxEa49V6TDDZavyCae7Lpf5TE9APFYrB4uJ5/AipvIA6e68ZMSiOkoiostGLDUKsJRxwG5sqfgJHywmhTKDJMbKKWtSQBo4iHJKR2XgXXCskjJk5DXK5ysJ6EblDLQuBCEuGqr4bq6qWg8TUps88nQgU6EMziUaniD6ZeCEUYHI1FDT1bbPb9jfuQ/QXh1MJjeYEe7b2BFAU3wwO2Tq5FPhkw6hy0q9UHtJrWzLDZOA8OXjXrFWw9nhkjsbBqguZB1E8CcY42CvPRo0K5aMLXTS46gD6bzQIH3SYzbOxopeiMTI1IMOiBWBf8wzpnUeXMWk= X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230028)(4636009)(346002)(39860400002)(376002)(136003)(396003)(451199021)(40470700004)(46966006)(36840700001)(86362001)(40480700001)(8676002)(8936002)(70586007)(70206006)(4326008)(82740400003)(356005)(5660300002)(40460700003)(36756003)(82310400005)(81166007)(478600001)(47076005)(36860700001)(336012)(426003)(41300700001)(2906002)(54906003)(110136005)(2616005)(1076003)(316002)(26005)(6666004)(44832011)(186003)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2023 11:18:43.4899 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 717332e0-9859-4e78-7494-08db4fb5fbcc X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT014.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR12MB6476 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org CDX bus can use VFIO interface for mapping and unmapping of DMA addresses in the IOMMU. This change adds the callback support for map and unmap APIs as well as fetching the IOMMU class. Signed-off-by: Nipun Gupta --- drivers/bus/cdx/cdx.c | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/drivers/bus/cdx/cdx.c b/drivers/bus/cdx/cdx.c index d479daa315..8cc273336e 100644 --- a/drivers/bus/cdx/cdx.c +++ b/drivers/bus/cdx/cdx.c @@ -505,12 +505,52 @@ cdx_find_device(const struct rte_device *start, rte_dev_cmp_t cmp, return NULL; } +static int +cdx_dma_map(struct rte_device *dev, void *addr, uint64_t iova, size_t len) +{ + struct rte_cdx_device *cdx_dev = RTE_DEV_TO_CDX_DEV(dev); + + if (!cdx_dev) { + rte_errno = EINVAL; + return -1; + } + + return rte_vfio_container_dma_map(RTE_VFIO_DEFAULT_CONTAINER_FD, + (uintptr_t)addr, iova, len); +} + +static int +cdx_dma_unmap(struct rte_device *dev, void *addr, uint64_t iova, size_t len) +{ + struct rte_cdx_device *cdx_dev = RTE_DEV_TO_CDX_DEV(dev); + + if (!cdx_dev) { + rte_errno = EINVAL; + return -1; + } + + return rte_vfio_container_dma_unmap(RTE_VFIO_DEFAULT_CONTAINER_FD, + (uintptr_t)addr, iova, len); +} + +static enum rte_iova_mode +cdx_get_iommu_class(void) +{ + if (TAILQ_EMPTY(&rte_cdx_bus.device_list)) + return RTE_IOVA_DC; + + return RTE_IOVA_VA; +} + struct rte_cdx_bus rte_cdx_bus = { .bus = { .scan = cdx_scan, .probe = cdx_probe, .find_device = cdx_find_device, .parse = cdx_parse, + .dma_map = cdx_dma_map, + .dma_unmap = cdx_dma_unmap, + .get_iommu_class = cdx_get_iommu_class, }, .device_list = TAILQ_HEAD_INITIALIZER(rte_cdx_bus.device_list), .driver_list = TAILQ_HEAD_INITIALIZER(rte_cdx_bus.driver_list), From patchwork Mon May 8 11:18:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Gupta, Nipun" X-Patchwork-Id: 126761 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EA72E42A94; Mon, 8 May 2023 13:18:58 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 631DE42D29; Mon, 8 May 2023 13:18:52 +0200 (CEST) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2045.outbound.protection.outlook.com [40.107.94.45]) by mails.dpdk.org (Postfix) with ESMTP id 5358042D1D for ; Mon, 8 May 2023 13:18:50 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SeAmN3lKkcm+BLTpnGDhU9ULQyn2mL92YFSnUKp4Te+H/YDhHOmreyvmTsmlY19Eq4AOhmnSLclR9+z06QabffOfpc/7oehuWwBwpqkpZwtrzFFnE1kzcolZpm06wmP2G+1/YXmdcZ8aMmbCO3eTy//JdHx/ZYpMK8yAFgiVSlRiTd4JD2ncj+ZT4I+1UtyM+BKvoW/zTFsP5rK0nrsAf9yMCHFZ4wkiIhCkk5ZvSspGGvXmg5eUcotRFLPoBFlSdaBhxfuJsS6zIO9re2s0z0OvCDfdAECSfMysKMoS9Yq+5LnXySHCuk/RbsqgCOUGibXsXHj3ov404Hz0WT5B2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=AFF+ocYjjjSnSaoknsFP51vJasx6rKF0GwKcp63UACw=; b=c+r61Vo0sVeIhTYOCj1/iJbMysWq0xdUzycLvdOfysp8EshVjSCHsjMTnl7scvefKz4d9jG3Jhepxp+RtYMJ8Fp76Tdf7ZdmmJVMubTKNp3H2RKXIAJf+6FQzNCdjYGd6Qrv2BqJEwVZPaIjxpb/9wFzhhVEe/xxg+wcJwARaAWgB/9Bvn9bF6KyYcTBmF+mXAgjWwmUstIVmTiknFXoRiKjL7vgIW1AjMZKZiH4JAks9npKXlabvUvn4QAK8DOk0x2x+HQnuXrEHd3hPWZePZD0UPz1fGoQvzNUcutlOKD2v4LJL1qFFNiTrg1+0XWTUJD0w80W7fdOWO18/rOisQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=dpdk.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AFF+ocYjjjSnSaoknsFP51vJasx6rKF0GwKcp63UACw=; b=MattyYbsWAR5j4nOJiDVtHrIEEG85exz37LL1Dt0XlAQzdFnQPeqXpGhylglIHwb16uT3Uba6aGDspNdHp6aPqlmoduzaUtnctB8Z7Q4SzRrImA2LBRCUJ+NeUk450AJrf4hT61kgQSaA9L+WTn9VnSCsYBF5u0RZzhVn4eVGow= Received: from BN9PR03CA0546.namprd03.prod.outlook.com (2603:10b6:408:138::11) by MW3PR12MB4409.namprd12.prod.outlook.com (2603:10b6:303:2d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6363.32; Mon, 8 May 2023 11:18:47 +0000 Received: from BN8NAM11FT099.eop-nam11.prod.protection.outlook.com (2603:10b6:408:138:cafe::8a) by BN9PR03CA0546.outlook.office365.com (2603:10b6:408:138::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6363.32 via Frontend Transport; Mon, 8 May 2023 11:18:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN8NAM11FT099.mail.protection.outlook.com (10.13.177.197) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6363.33 via Frontend Transport; Mon, 8 May 2023 11:18:47 +0000 Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 8 May 2023 06:18:46 -0500 Received: from xhdipdslab41.xilinx.com (10.180.168.240) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Mon, 8 May 2023 06:18:44 -0500 From: Nipun Gupta To: , , CC: , , , Nipun Gupta Subject: [PATCH v4 3/4] bus/cdx: add support for MSI Date: Mon, 8 May 2023 16:48:11 +0530 Message-ID: <20230508111812.2655-4-nipun.gupta@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230508111812.2655-1-nipun.gupta@amd.com> References: <20230124140746.594066-1-nipun.gupta@amd.com> <20230508111812.2655-1-nipun.gupta@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT099:EE_|MW3PR12MB4409:EE_ X-MS-Office365-Filtering-Correlation-Id: 3777ee1d-f8b2-4672-381f-08db4fb5fde9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: BZCbika9woPk3Rv6+ct2hZt+LheLrzfs958+vBS6x+72CMxLqy767ZwtdHav3o5Jkz6krpUTUvB2d4bh7dVjKmgjqpRTF8TARpO5bQdLtyafYg97qjK+7G1RPs5CBFZ+MzgAoePNiQacbptkL174SPrrrV56q7bDXSoMIdOi1AWcy7ReBYatum/9j4V0sRy/xbHm/x/BxUF2AbsBYzqY9NbaQ0a6XN1cUuLZgqDobP1B5XvSssrhL3n4fa6+yACZnRw3bIJfIzwEGEC9jSPp0uWWqJFzSdjkv8GEnnjbEiMF7pGNtrVWv0O55Fnjbwl9UgAmbt+GjIjViIRbHfw8IivoPzlg7OJYVvi+5/BHhn2GQBY2B+6DY34Bjntqu3bcAG44aUwXzUvzWhSHbyKQ532ds/lrFOzL7jMkIf3b5mPyCwojqf/G2z7GsHc83UT2M1E2/lZYEgZZpWmbx3pPV6l6XLIiSF4EOILMF9qKaVPkNzSG8f4jQwcGvxoifqS1xTx4ptPfOol0EOYVJ43NiTGobTGAf+BZav8T2prHyINq1/p4BhcDOcFMmF5h22a1cDgWLcuXDZMGlu0xvvnmelpCEZGqFdOeNyAJY+doUEDlpNJC1+iLPyYum9LafMJtWRIr7HexDjcpnpQhOHmXl+JCknEr8wA56OuzPSNt5MZNvWyiogkOwbPToA5tFEvYi4vBUFXWqCW6uGJJxB2kXgUltHJVf2hXBRFnH4M31dg= X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230028)(4636009)(39860400002)(396003)(376002)(346002)(136003)(451199021)(40470700004)(46966006)(36840700001)(1076003)(26005)(83380400001)(336012)(40480700001)(426003)(36860700001)(36756003)(2616005)(81166007)(86362001)(82310400005)(356005)(82740400003)(186003)(40460700003)(47076005)(110136005)(41300700001)(2906002)(54906003)(30864003)(5660300002)(4326008)(316002)(8936002)(8676002)(478600001)(44832011)(70586007)(70206006)(6666004)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2023 11:18:47.0183 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3777ee1d-f8b2-4672-381f-08db4fb5fde9 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT099.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR12MB4409 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org MSI's are exposed to the devices using VFIO (vfio-cdx). This patch uses the same to add support for MSI for the devices on the cdx bus. A couple of API's have been introduced in the EAL interrupt framework: - rte_intr_irq_count_set: This API is used to set the total interrupts on the interrupt handle. This would be provided by VFIO (irq.count) for VFIO enabled devices. - rte_intr_irq_count_get: This API returns the total number interrupts which were set. Signed-off-by: Nipun Gupta --- drivers/bus/cdx/bus_cdx_driver.h | 25 ++++ drivers/bus/cdx/cdx.c | 11 ++ drivers/bus/cdx/cdx_vfio.c | 182 ++++++++++++++++++++++++- drivers/bus/cdx/version.map | 2 + lib/eal/common/eal_common_interrupts.c | 21 +++ lib/eal/common/eal_interrupts.h | 1 + lib/eal/include/rte_interrupts.h | 32 +++++ lib/eal/version.map | 2 + 8 files changed, 274 insertions(+), 2 deletions(-) diff --git a/drivers/bus/cdx/bus_cdx_driver.h b/drivers/bus/cdx/bus_cdx_driver.h index 7edcb019eb..fdeaf46664 100644 --- a/drivers/bus/cdx/bus_cdx_driver.h +++ b/drivers/bus/cdx/bus_cdx_driver.h @@ -72,6 +72,7 @@ struct rte_cdx_device { struct rte_cdx_id id; /**< CDX ID. */ struct rte_mem_resource mem_resource[CDX_MAX_RESOURCE]; /**< CDX Memory Resource */ + struct rte_intr_handle *intr_handle; /**< Interrupt handle */ }; /** @@ -173,6 +174,30 @@ void rte_cdx_unmap_device(struct rte_cdx_device *dev); __rte_internal void rte_cdx_register(struct rte_cdx_driver *driver); +/** + * Enables VFIO Interrupts for CDX bus devices. + * + * @param intr_handle + * Pointer to the interrupt handle. + * + * @return + * 0 on success, -1 on error. + */ +__rte_internal +int rte_cdx_vfio_intr_enable(const struct rte_intr_handle *intr_handle); + +/** + * Disable VFIO Interrupts for CDX bus devices. + * + * @param intr_handle + * Pointer to the interrupt handle. + * + * @return + * 0 on success, -1 on error. + */ +__rte_internal +int rte_cdx_vfio_intr_disable(const struct rte_intr_handle *intr_handle); + /** * Helper for CDX device registration from driver (eth, crypto, raw) instance */ diff --git a/drivers/bus/cdx/cdx.c b/drivers/bus/cdx/cdx.c index 8cc273336e..6c9ceaaf7f 100644 --- a/drivers/bus/cdx/cdx.c +++ b/drivers/bus/cdx/cdx.c @@ -204,6 +204,15 @@ cdx_scan_one(const char *dirname, const char *dev_name) goto err; } + /* Allocate interrupt instance for cdx device */ + dev->intr_handle = + rte_intr_instance_alloc(RTE_INTR_INSTANCE_F_PRIVATE); + if (dev->intr_handle == NULL) { + CDX_BUS_ERR("Failed to create interrupt instance for %s\n", + dev->device.name); + return -ENOMEM; + } + /* * Check if device is bound to 'vfio-cdx' driver, so that user-space * can gracefully access the device. @@ -394,6 +403,8 @@ cdx_probe_one_driver(struct rte_cdx_driver *dr, return ret; error_probe: + rte_intr_instance_free(dev->intr_handle); + dev->intr_handle = NULL; cdx_vfio_unmap_resource(dev); error_map_device: return ret; diff --git a/drivers/bus/cdx/cdx_vfio.c b/drivers/bus/cdx/cdx_vfio.c index ae11f589b3..1422b98503 100644 --- a/drivers/bus/cdx/cdx_vfio.c +++ b/drivers/bus/cdx/cdx_vfio.c @@ -60,6 +60,10 @@ struct mapped_cdx_resource { /** mapped cdx device list */ TAILQ_HEAD(mapped_cdx_res_list, mapped_cdx_resource); +/* irq set buffer length for MSI interrupts */ +#define MSI_IRQ_SET_BUF_LEN (sizeof(struct vfio_irq_set) + \ + sizeof(int) * (RTE_MAX_RXTX_INTR_VEC_ID + 1)) + static struct rte_tailq_elem cdx_vfio_tailq = { .name = "VFIO_CDX_RESOURCE_LIST", }; @@ -104,6 +108,27 @@ cdx_vfio_unmap_resource_primary(struct rte_cdx_device *dev) char cdx_addr[PATH_MAX] = {0}; struct mapped_cdx_resource *vfio_res = NULL; struct mapped_cdx_res_list *vfio_res_list; + int ret, vfio_dev_fd; + + if (rte_intr_fd_get(dev->intr_handle) < 0) + return -1; + + if (close(rte_intr_fd_get(dev->intr_handle)) < 0) { + CDX_BUS_ERR("Error when closing eventfd file descriptor for %s", + dev->device.name); + return -1; + } + + vfio_dev_fd = rte_intr_dev_fd_get(dev->intr_handle); + if (vfio_dev_fd < 0) + return -1; + + ret = rte_vfio_release_device(rte_cdx_get_sysfs_path(), dev->device.name, + vfio_dev_fd); + if (ret < 0) { + CDX_BUS_ERR("Cannot release VFIO device"); + return ret; + } vfio_res_list = RTE_TAILQ_CAST(cdx_vfio_tailq.head, mapped_cdx_res_list); @@ -126,6 +151,18 @@ cdx_vfio_unmap_resource_secondary(struct rte_cdx_device *dev) { struct mapped_cdx_resource *vfio_res = NULL; struct mapped_cdx_res_list *vfio_res_list; + int ret, vfio_dev_fd; + + vfio_dev_fd = rte_intr_dev_fd_get(dev->intr_handle); + if (vfio_dev_fd < 0) + return -1; + + ret = rte_vfio_release_device(rte_cdx_get_sysfs_path(), dev->device.name, + vfio_dev_fd); + if (ret < 0) { + CDX_BUS_ERR("Cannot release VFIO device"); + return ret; + } vfio_res_list = RTE_TAILQ_CAST(cdx_vfio_tailq.head, mapped_cdx_res_list); @@ -150,9 +187,80 @@ cdx_vfio_unmap_resource(struct rte_cdx_device *dev) return cdx_vfio_unmap_resource_secondary(dev); } +/* set up interrupt support (but not enable interrupts) */ static int -cdx_rte_vfio_setup_device(int vfio_dev_fd) +cdx_vfio_setup_interrupts(struct rte_cdx_device *dev, int vfio_dev_fd, + int num_irqs) { + int i, ret; + + if (num_irqs == 0) + return 0; + + /* start from MSI interrupt type */ + for (i = 0; i < num_irqs; i++) { + struct vfio_irq_info irq = { .argsz = sizeof(irq) }; + int fd = -1; + + irq.index = i; + + ret = ioctl(vfio_dev_fd, VFIO_DEVICE_GET_IRQ_INFO, &irq); + if (ret < 0) { + CDX_BUS_ERR("Cannot get VFIO IRQ info, error %i (%s)", + errno, strerror(errno)); + return -1; + } + + /* if this vector cannot be used with eventfd, fail if we explicitly + * specified interrupt type, otherwise continue + */ + if ((irq.flags & VFIO_IRQ_INFO_EVENTFD) == 0) + continue; + + if (rte_intr_irq_count_set(dev->intr_handle, irq.count)) + return -1; + + /* Reallocate the efds and elist fields of intr_handle based + * on CDX device MSI size. + */ + if ((uint32_t)rte_intr_nb_intr_get(dev->intr_handle) < irq.count && + rte_intr_event_list_update(dev->intr_handle, irq.count)) + return -1; + + /* set up an eventfd for interrupts */ + fd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); + if (fd < 0) { + CDX_BUS_ERR("Cannot set up eventfd, error %i (%s)", + errno, strerror(errno)); + return -1; + } + + if (rte_intr_fd_set(dev->intr_handle, fd)) + return -1; + + /* DPDK CDX bus currently supports only MSI-X */ + if (rte_intr_type_set(dev->intr_handle, RTE_INTR_HANDLE_VFIO_MSIX)) + return -1; + + if (rte_intr_dev_fd_set(dev->intr_handle, vfio_dev_fd)) + return -1; + + return 0; + } + + /* if we're here, we haven't found a suitable interrupt vector */ + return -1; +} + +static int +cdx_vfio_setup_device(struct rte_cdx_device *dev, int vfio_dev_fd, + int num_irqs) +{ + if (cdx_vfio_setup_interrupts(dev, vfio_dev_fd, num_irqs) != 0) { + CDX_BUS_ERR("Error setting up interrupts!"); + return -1; + } + /* * Reset the device. If the device is not capable of resetting, * then it updates errno as EINVAL. @@ -288,6 +396,9 @@ cdx_vfio_map_resource_primary(struct rte_cdx_device *dev) struct cdx_map *maps; int vfio_dev_fd, i, ret; + if (rte_intr_fd_set(dev->intr_handle, -1)) + return -1; + ret = rte_vfio_setup_device(rte_cdx_get_sysfs_path(), dev_name, &vfio_dev_fd, &device_info); if (ret) @@ -353,7 +464,7 @@ cdx_vfio_map_resource_primary(struct rte_cdx_device *dev) free(reg); } - if (cdx_rte_vfio_setup_device(vfio_dev_fd) < 0) { + if (cdx_vfio_setup_device(dev, vfio_dev_fd, device_info.num_irqs) < 0) { CDX_BUS_ERR("%s setup device failed", dev_name); goto err_vfio_res; } @@ -383,6 +494,9 @@ cdx_vfio_map_resource_secondary(struct rte_cdx_device *dev) const char *dev_name = dev->device.name; struct cdx_map *maps; + if (rte_intr_fd_set(dev->intr_handle, -1)) + return -1; + /* if we're in a secondary process, just find our tailq entry */ TAILQ_FOREACH(vfio_res, vfio_res_list, next) { if (strcmp(vfio_res->name, dev_name)) @@ -416,6 +530,10 @@ cdx_vfio_map_resource_secondary(struct rte_cdx_device *dev) dev->mem_resource[i].len = maps[i].size; } + /* we need save vfio_dev_fd, so it can be used during release */ + if (rte_intr_dev_fd_set(dev->intr_handle, vfio_dev_fd)) + goto err_vfio_dev_fd; + return 0; err_vfio_dev_fd: rte_vfio_release_device(rte_cdx_get_sysfs_path(), @@ -435,3 +553,63 @@ cdx_vfio_map_resource(struct rte_cdx_device *dev) else return cdx_vfio_map_resource_secondary(dev); } + +int +rte_cdx_vfio_intr_enable(const struct rte_intr_handle *intr_handle) +{ + char irq_set_buf[MSI_IRQ_SET_BUF_LEN]; + struct vfio_irq_set *irq_set; + int *fd_ptr, vfio_dev_fd, i; + int ret; + + irq_set = (struct vfio_irq_set *) irq_set_buf; + irq_set->count = rte_intr_irq_count_get(intr_handle); + irq_set->argsz = sizeof(struct vfio_irq_set) + + (sizeof(int) * irq_set->count); + + irq_set->flags = VFIO_IRQ_SET_DATA_EVENTFD | VFIO_IRQ_SET_ACTION_TRIGGER; + irq_set->index = 0; + irq_set->start = 0; + fd_ptr = (int *) &irq_set->data; + + for (i = 0; i < rte_intr_nb_efd_get(intr_handle); i++) + fd_ptr[i] = rte_intr_efds_index_get(intr_handle, i); + + vfio_dev_fd = rte_intr_dev_fd_get(intr_handle); + ret = ioctl(vfio_dev_fd, VFIO_DEVICE_SET_IRQS, irq_set); + + if (ret) { + CDX_BUS_ERR("Error enabling MSI interrupts for fd %d", + rte_intr_fd_get(intr_handle)); + return -1; + } + + return 0; +} + +/* disable MSI interrupts */ +int +rte_cdx_vfio_intr_disable(const struct rte_intr_handle *intr_handle) +{ + struct vfio_irq_set *irq_set; + char irq_set_buf[MSI_IRQ_SET_BUF_LEN]; + int len, ret, vfio_dev_fd; + + len = sizeof(struct vfio_irq_set); + + irq_set = (struct vfio_irq_set *) irq_set_buf; + irq_set->argsz = len; + irq_set->count = 0; + irq_set->flags = VFIO_IRQ_SET_DATA_NONE | VFIO_IRQ_SET_ACTION_TRIGGER; + irq_set->index = 0; + irq_set->start = 0; + + vfio_dev_fd = rte_intr_dev_fd_get(intr_handle); + ret = ioctl(vfio_dev_fd, VFIO_DEVICE_SET_IRQS, irq_set); + + if (ret) + CDX_BUS_ERR("Error disabling MSI interrupts for fd %d", + rte_intr_fd_get(intr_handle)); + + return ret; +} diff --git a/drivers/bus/cdx/version.map b/drivers/bus/cdx/version.map index 957fcab978..2f3d484ebd 100644 --- a/drivers/bus/cdx/version.map +++ b/drivers/bus/cdx/version.map @@ -6,6 +6,8 @@ INTERNAL { rte_cdx_register; rte_cdx_unmap_device; rte_cdx_unregister; + rte_cdx_vfio_intr_disable; + rte_cdx_vfio_intr_enable; local: *; }; diff --git a/lib/eal/common/eal_common_interrupts.c b/lib/eal/common/eal_common_interrupts.c index 97b64fed58..a0167d9ad4 100644 --- a/lib/eal/common/eal_common_interrupts.c +++ b/lib/eal/common/eal_common_interrupts.c @@ -398,6 +398,27 @@ int rte_intr_elist_index_set(struct rte_intr_handle *intr_handle, return -rte_errno; } +int rte_intr_irq_count_set(struct rte_intr_handle *intr_handle, + int irq_count) +{ + CHECK_VALID_INTR_HANDLE(intr_handle); + + intr_handle->irq_count = irq_count; + + return 0; +fail: + return -rte_errno; +} + +int rte_intr_irq_count_get(const struct rte_intr_handle *intr_handle) +{ + CHECK_VALID_INTR_HANDLE(intr_handle); + + return intr_handle->irq_count; +fail: + return -rte_errno; +} + int rte_intr_vec_list_alloc(struct rte_intr_handle *intr_handle, const char *name, int size) { diff --git a/lib/eal/common/eal_interrupts.h b/lib/eal/common/eal_interrupts.h index 482781b862..237f471a76 100644 --- a/lib/eal/common/eal_interrupts.h +++ b/lib/eal/common/eal_interrupts.h @@ -16,6 +16,7 @@ struct rte_intr_handle { }; uint32_t alloc_flags; /**< flags passed at allocation */ enum rte_intr_handle_type type; /**< handle type */ + uint32_t irq_count; /**< IRQ count provided via VFIO */ uint32_t max_intr; /**< max interrupt requested */ uint32_t nb_efd; /**< number of available efd(event fd) */ uint8_t efd_counter_size; /**< size of efd counter, used for vdev */ diff --git a/lib/eal/include/rte_interrupts.h b/lib/eal/include/rte_interrupts.h index 487e3c8875..bc477a483f 100644 --- a/lib/eal/include/rte_interrupts.h +++ b/lib/eal/include/rte_interrupts.h @@ -506,6 +506,38 @@ __rte_internal int rte_intr_max_intr_get(const struct rte_intr_handle *intr_handle); +/** + * @internal + * Set the irq count field of interrupt handle with user + * provided irq count value. + * + * @param intr_handle + * pointer to the interrupt handle. + * @param irq_count + * IRQ count + * + * @return + * - On success, zero. + * - On failure, a negative value and rte_errno is set. + */ +__rte_internal +int +rte_intr_irq_count_set(struct rte_intr_handle *intr_handle, int irq_count); + +/** + * @internal + * Returns the irq count field of the given interrupt handle instance. + * + * @param intr_handle + * pointer to the interrupt handle. + * + * @return + * - On success, ir count. + * - On failure, a negative value and rte_errno is set. + */ +__rte_internal +int rte_intr_irq_count_get(const struct rte_intr_handle *intr_handle); + /** * @internal * Set the number of event fd field of interrupt handle diff --git a/lib/eal/version.map b/lib/eal/version.map index 6d6978f108..14bf7ade77 100644 --- a/lib/eal/version.map +++ b/lib/eal/version.map @@ -458,6 +458,8 @@ INTERNAL { rte_intr_instance_dup; rte_intr_instance_windows_handle_get; rte_intr_instance_windows_handle_set; + rte_intr_irq_count_get; + rte_intr_irq_count_set; rte_intr_max_intr_get; rte_intr_max_intr_set; rte_intr_nb_efd_get; From patchwork Mon May 8 11:18:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Gupta, Nipun" X-Patchwork-Id: 126762 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 426B542A94; Mon, 8 May 2023 13:19:05 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 661F242D35; Mon, 8 May 2023 13:18:56 +0200 (CEST) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2042.outbound.protection.outlook.com [40.107.92.42]) by mails.dpdk.org (Postfix) with ESMTP id 22EC142D38 for ; Mon, 8 May 2023 13:18:54 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=d3P5+0Dbi3W2+Sj8ZmyYOoY3q7vzzomCjDDRpmNnA4VQZH0epOdaQK1eyDk2YUhKU4OtmuwEl7V9iZuFwyiLY/CKW8kMiGOSVbMoZ1RB5nKQ0AB/RcJ27oRYgCdo+erMbpwhnj9CJwO6Hhle6iQ0o7B/d2keNasDVla/tCZ7ZlG2a3e+11iDm+qoiX641BOQfDYdVwadii7v5fP3y9X70t0XdytY4yMunQD/0bVxwRLSDK/qGzs0RasNCw/whGWVJA5sgprmDmm+Z67Ynj5PTFq+l0PvOzUezXKvXOBd/KPiM6EVJcEVF3zVBkCKqBux2rRqMtdJBITMB0V16qmA/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=GBqkQop4jNZxBaHULpWywUcKNeajmq0i5o2wOvDRkNc=; b=TOfniiZ6AXAbPJcBtL3307csTk/ZlGgD2MoEXo3K1ucXpz+pkVc7k8ibb15KCAGUCNenHHQSetsJLweCIDGBL2QSaI/Z6Stj4OraBZS2ZVbf4ih2iAkbMGT5ZQgQLr9WvtMsqdJiegc+fkXfZh2StRFJXCFl9Kl0jz1MuAk5L8TWnC/fMfWOvrQegACEgXljpX6pOv3TOmSv8MG0hDY28/Ybvw17Uo3E5Tn02Hv70EzL/bEHwUgZVdtVMb0XzXTUpnoFCX1kWo3Mgy+qtnunIdkMlzaYJTWcuWh7k81nFEp3aQLa6ueG7pd1/QSIQn1mleg0YZrY2eChFJHhE2xxPQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=dpdk.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GBqkQop4jNZxBaHULpWywUcKNeajmq0i5o2wOvDRkNc=; b=AsBB8diy8sOwO6AKcFArWvTFCTrqS/bKlInlSl6eFMKsCPkUtCNE3rAy+VhQ8Pd09PFqfwCWMgN0sSbB/FQ7VFhQw2PMsJGySsa+Jg+319f4FVazzbwv/XA/M7sNyVNNGOIM6IutEp+qo2wm3w0mSwcjUXkqHPAQLnwK09DRCqo= Received: from BN7PR02CA0015.namprd02.prod.outlook.com (2603:10b6:408:20::28) by CYYPR12MB8654.namprd12.prod.outlook.com (2603:10b6:930:c9::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6363.32; Mon, 8 May 2023 11:18:51 +0000 Received: from BN8NAM11FT060.eop-nam11.prod.protection.outlook.com (2603:10b6:408:20:cafe::58) by BN7PR02CA0015.outlook.office365.com (2603:10b6:408:20::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6363.32 via Frontend Transport; Mon, 8 May 2023 11:18:50 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C Received: from SATLEXMB03.amd.com (165.204.84.17) by BN8NAM11FT060.mail.protection.outlook.com (10.13.177.211) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6363.33 via Frontend Transport; Mon, 8 May 2023 11:18:50 +0000 Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 8 May 2023 06:18:50 -0500 Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB05.amd.com (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 8 May 2023 06:18:50 -0500 Received: from xhdipdslab41.xilinx.com (10.180.168.240) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server id 15.1.2375.34 via Frontend Transport; Mon, 8 May 2023 06:18:48 -0500 From: Nipun Gupta To: , , CC: , , , Nipun Gupta Subject: [PATCH v4 4/4] bus/cdx: support plug unplug and dev iterator Date: Mon, 8 May 2023 16:48:12 +0530 Message-ID: <20230508111812.2655-5-nipun.gupta@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230508111812.2655-1-nipun.gupta@amd.com> References: <20230124140746.594066-1-nipun.gupta@amd.com> <20230508111812.2655-1-nipun.gupta@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT060:EE_|CYYPR12MB8654:EE_ X-MS-Office365-Filtering-Correlation-Id: 335aa2b8-f0dc-4b01-9591-08db4fb6002e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7jvcVrOPicoORR+BtjynHTjphMga7fEs50zQ2l26PxTbLXitohDEApzEGMu5q9gugZJAQuW4t5mmQIQK/S1CVlwgOZnaIGJ2uds1XZXMiAiqRQpFeV3X89pLKPTEMPknD3bNNZyFOVAHtdS1z2szETLPvo53MksBS/LFe1fZt4EUHEHQUI3RfuyZYFqwCecIyoViFzZWA9T5Hr/ND2JxdEqDjcH9Aiu2wwwOlTlPui542U7C5VYkPvZTJ1R6XvUscdbDtNl7jDDOdIlLEwYlUCfIY38cKNK8MDq9+xw8ol7tSrWA4V60Ma3as93YFaNJ0QJ4EcMieZ+8/Suuhr6z2HmuFLgySSUZ1ushKHXCEXj4Pyb4U5Fwsq/lRS7/GpVw6jK2axOuscvQrefU6fZCYohb4A4YcwPhB9oQkm2YT+veqqoa2hnjIMVkCp/nbogFRiRuOnL9MvkTU3j+pmycaj6digBBcRKfq22Ex+Hd1kPKGcTHoedeOFPozSjhzd29tiV+rVG0D3r3WFES+tz43Ek5UGDjYte9a+d5KAJXHAHOKH7vuEzniaXpWGwMWkwOc9bUcqdiMBFxYWMAxUrR5QFRSocw22Qtpv4hZwSoOOrSAGP0lopQcH678v+cQC5vZK0WK2fNc7GU0YOG0LRkS5NYjTi6TDj2xFIrB/iHm+ACUhoGAfQxnxZh2JFSxZuV79LrzB3M6Vejw6yiuvqzz+rbpN3bxn/Xghgc+4+oFUo= X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB03.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230028)(4636009)(376002)(346002)(39860400002)(136003)(396003)(451199021)(46966006)(40470700004)(36840700001)(8676002)(8936002)(5660300002)(44832011)(336012)(426003)(82310400005)(86362001)(2616005)(36860700001)(47076005)(186003)(1076003)(26005)(82740400003)(356005)(81166007)(40460700003)(478600001)(54906003)(110136005)(6666004)(41300700001)(40480700001)(316002)(70586007)(4326008)(36756003)(70206006)(2906002)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2023 11:18:50.8283 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 335aa2b8-f0dc-4b01-9591-08db4fb6002e X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT060.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8654 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This change adds support for plugging and unplugging CDX devices on the CDX bus. Also, CDX dev iterator support has been added for the CDX bus. Signed-off-by: Nipun Gupta --- drivers/bus/cdx/bus_cdx_driver.h | 1 + drivers/bus/cdx/cdx.c | 122 +++++++++++++++++++++++++++++++ 2 files changed, 123 insertions(+) diff --git a/drivers/bus/cdx/bus_cdx_driver.h b/drivers/bus/cdx/bus_cdx_driver.h index fdeaf46664..3d89e7c054 100644 --- a/drivers/bus/cdx/bus_cdx_driver.h +++ b/drivers/bus/cdx/bus_cdx_driver.h @@ -69,6 +69,7 @@ struct rte_cdx_id { struct rte_cdx_device { RTE_TAILQ_ENTRY(rte_cdx_device) next; /**< Next probed CDX device. */ struct rte_device device; /**< Inherit core device */ + struct rte_cdx_driver *driver; /**< CDX driver used in probing */ struct rte_cdx_id id; /**< CDX ID. */ struct rte_mem_resource mem_resource[CDX_MAX_RESOURCE]; /**< CDX Memory Resource */ diff --git a/drivers/bus/cdx/cdx.c b/drivers/bus/cdx/cdx.c index 6c9ceaaf7f..0a30b8648a 100644 --- a/drivers/bus/cdx/cdx.c +++ b/drivers/bus/cdx/cdx.c @@ -68,6 +68,7 @@ #include #include #include +#include #include #include #include @@ -82,6 +83,15 @@ #define CDX_BUS_NAME cdx #define CDX_DEV_PREFIX "cdx-" +enum cdx_params { + RTE_CDX_PARAM_NAME, +}; + +static const char * const cdx_params_keys[] = { + [RTE_CDX_PARAM_NAME] = "name", + NULL, +}; + /** * @file * CDX probing using Linux sysfs. @@ -399,6 +409,7 @@ cdx_probe_one_driver(struct rte_cdx_driver *dr, } else { dev->device.driver = &dr->driver; } + dev->driver = dr; return ret; @@ -516,6 +527,71 @@ cdx_find_device(const struct rte_device *start, rte_dev_cmp_t cmp, return NULL; } +/* Remove a device from CDX bus */ +static void +cdx_remove_device(struct rte_cdx_device *cdx_dev) +{ + TAILQ_REMOVE(&rte_cdx_bus.device_list, cdx_dev, next); +} + +/* + * If vendor/device ID match, call the remove() function of the + * driver. + */ +static int +cdx_detach_dev(struct rte_cdx_device *dev) +{ + struct rte_cdx_driver *dr; + int ret = 0; + + if (dev == NULL) + return -EINVAL; + + dr = dev->driver; + + CDX_BUS_DEBUG("detach device %s using driver: %s", + dev->device.name, dr->driver.name); + + if (dr->remove) { + ret = dr->remove(dev); + if (ret < 0) + return ret; + } + + /* clear driver structure */ + dev->driver = NULL; + dev->device.driver = NULL; + + rte_cdx_unmap_device(dev); + + rte_intr_instance_free(dev->intr_handle); + dev->intr_handle = NULL; + + return 0; +} + +static int +cdx_plug(struct rte_device *dev) +{ + return cdx_probe_all_drivers(RTE_DEV_TO_CDX_DEV(dev)); +} + +static int +cdx_unplug(struct rte_device *dev) +{ + struct rte_cdx_device *cdx_dev; + int ret; + + cdx_dev = RTE_DEV_TO_CDX_DEV(dev); + ret = cdx_detach_dev(cdx_dev); + if (ret == 0) { + cdx_remove_device(cdx_dev); + rte_devargs_remove(dev->devargs); + free(cdx_dev); + } + return ret; +} + static int cdx_dma_map(struct rte_device *dev, void *addr, uint64_t iova, size_t len) { @@ -553,15 +629,61 @@ cdx_get_iommu_class(void) return RTE_IOVA_VA; } +static int +cdx_dev_match(const struct rte_device *dev, + const void *_kvlist) +{ + const struct rte_kvargs *kvlist = _kvlist; + const char *key = cdx_params_keys[RTE_CDX_PARAM_NAME]; + const char *name; + + /* no kvlist arg, all devices match */ + if (kvlist == NULL) + return 0; + + /* if key is present in kvlist and does not match, filter device */ + name = rte_kvargs_get(kvlist, key); + if (name != NULL && strcmp(name, dev->name)) + return -1; + + return 0; +} + +static void * +cdx_dev_iterate(const void *start, + const char *str, + const struct rte_dev_iterator *it __rte_unused) +{ + rte_bus_find_device_t find_device; + struct rte_kvargs *kvargs = NULL; + struct rte_device *dev; + + if (str != NULL) { + kvargs = rte_kvargs_parse(str, cdx_params_keys); + if (kvargs == NULL) { + CDX_BUS_ERR("cannot parse argument list %s", str); + rte_errno = EINVAL; + return NULL; + } + } + find_device = rte_cdx_bus.bus.find_device; + dev = find_device(start, cdx_dev_match, kvargs); + rte_kvargs_free(kvargs); + return dev; +} + struct rte_cdx_bus rte_cdx_bus = { .bus = { .scan = cdx_scan, .probe = cdx_probe, .find_device = cdx_find_device, + .plug = cdx_plug, + .unplug = cdx_unplug, .parse = cdx_parse, .dma_map = cdx_dma_map, .dma_unmap = cdx_dma_unmap, .get_iommu_class = cdx_get_iommu_class, + .dev_iterate = cdx_dev_iterate, }, .device_list = TAILQ_HEAD_INITIALIZER(rte_cdx_bus.device_list), .driver_list = TAILQ_HEAD_INITIALIZER(rte_cdx_bus.driver_list),