From patchwork Fri Sep 17 15:24:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99212 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4B978A0C43; Fri, 17 Sep 2021 17:24:56 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2CB4E41147; Fri, 17 Sep 2021 17:24:52 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id A0CDB406B4 for ; Fri, 17 Sep 2021 17:24:49 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222475986" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222475986" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:24:49 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="651909739" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by orsmga005.jf.intel.com with ESMTP; 17 Sep 2021 08:24:47 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 17 Sep 2021 15:24:22 +0000 Message-Id: <20210917152437.3270330-2-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210917152437.3270330-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210917152437.3270330-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 01/16] raw/ioat: only build if dmadev not present X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Bruce Richardson Only build the rawdev IDXD/IOAT drivers if the dmadev drivers are not present. A not is also added to the documentation to inform users of this change. Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh --- v4: - Fix build issue - Add note in raw documentation to outline this change --- doc/guides/rawdevs/ioat.rst | 7 +++++++ drivers/meson.build | 2 +- drivers/raw/ioat/meson.build | 23 ++++++++++++++++++++--- 3 files changed, 28 insertions(+), 4 deletions(-) diff --git a/doc/guides/rawdevs/ioat.rst b/doc/guides/rawdevs/ioat.rst index a28e909935..4fc327f1a4 100644 --- a/doc/guides/rawdevs/ioat.rst +++ b/doc/guides/rawdevs/ioat.rst @@ -34,6 +34,13 @@ Compilation For builds using ``meson`` and ``ninja``, the driver will be built when the target platform is x86-based. No additional compilation steps are necessary. +.. note:: + Since the addition of the DMAdev library, the ``ioat`` and ``idxd`` parts of this driver + will only be built if their ``DMAdev`` counterparts are not built. The following can be used + to disable the ``DMAdev`` drivers, if the raw drivers are to be used instead:: + + $ meson -Ddisable_drivers=dma/* + Device Setup ------------- diff --git a/drivers/meson.build b/drivers/meson.build index b7d680868a..27ff10a9fc 100644 --- a/drivers/meson.build +++ b/drivers/meson.build @@ -10,6 +10,7 @@ subdirs = [ 'common/qat', # depends on bus. 'common/sfc_efx', # depends on bus. 'mempool', # depends on common and bus. + 'dma', # depends on common and bus. 'net', # depends on common, bus, mempool 'raw', # depends on common, bus and net. 'crypto', # depends on common, bus and mempool (net in future). @@ -18,7 +19,6 @@ subdirs = [ 'vdpa', # depends on common, bus and mempool. 'event', # depends on common, bus, mempool and net. 'baseband', # depends on common and bus. - 'dma', # depends on common and bus. ] if meson.is_cross_build() diff --git a/drivers/raw/ioat/meson.build b/drivers/raw/ioat/meson.build index 0e81cb5951..9be9d8cc65 100644 --- a/drivers/raw/ioat/meson.build +++ b/drivers/raw/ioat/meson.build @@ -2,14 +2,31 @@ # Copyright 2019 Intel Corporation build = dpdk_conf.has('RTE_ARCH_X86') +# only use ioat rawdev driver if we don't have the equivalent dmadev ones +if dpdk_conf.has('RTE_DMA_IDXD') and dpdk_conf.has('RTE_DMA_IOAT') + build = false + subdir_done() +endif + reason = 'only supported on x86' sources = files( - 'idxd_bus.c', - 'idxd_pci.c', 'ioat_common.c', - 'ioat_rawdev.c', 'ioat_rawdev_test.c', ) + +if not dpdk_conf.has('RTE_DMA_IDXD') + sources += files( + 'idxd_bus.c', + 'idxd_pci.c', + ) +endif + +if not dpdk_conf.has('RTE_DMA_IOAT') + sources += files ( + 'ioat_rawdev.c', + ) +endif + deps += ['bus_pci', 'mbuf', 'rawdev'] headers = files( 'rte_ioat_rawdev.h', From patchwork Fri Sep 17 15:24:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99213 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 992D3A0C43; Fri, 17 Sep 2021 17:25:01 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4966641159; Fri, 17 Sep 2021 17:24:53 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id ED9124113A for ; Fri, 17 Sep 2021 17:24:51 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222475992" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222475992" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:24:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="651909750" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by orsmga005.jf.intel.com with ESMTP; 17 Sep 2021 08:24:49 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 17 Sep 2021 15:24:23 +0000 Message-Id: <20210917152437.3270330-3-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210917152437.3270330-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210917152437.3270330-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 02/16] dma/idxd: add skeleton for VFIO based DSA device X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the basic device probe/remove skeleton code for DSA device bound to the vfio pci driver. Relevant documentation and MAINTAINERS update also included. Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh --- v5: add missing toctree entry for idxd driver --- MAINTAINERS | 10 +++++ doc/guides/dmadevs/idxd.rst | 58 ++++++++++++++++++++++++++ doc/guides/dmadevs/index.rst | 2 + doc/guides/rel_notes/release_21_11.rst | 5 +++ drivers/dma/idxd/idxd_internal.h | 27 ++++++++++++ drivers/dma/idxd/idxd_pci.c | 55 ++++++++++++++++++++++++ drivers/dma/idxd/meson.build | 11 +++++ drivers/dma/idxd/version.map | 3 ++ drivers/dma/meson.build | 2 + 9 files changed, 173 insertions(+) create mode 100644 doc/guides/dmadevs/idxd.rst create mode 100644 drivers/dma/idxd/idxd_internal.h create mode 100644 drivers/dma/idxd/idxd_pci.c create mode 100644 drivers/dma/idxd/meson.build create mode 100644 drivers/dma/idxd/version.map diff --git a/MAINTAINERS b/MAINTAINERS index 3258da194d..9cb59b831d 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1200,6 +1200,16 @@ F: doc/guides/compressdevs/zlib.rst F: doc/guides/compressdevs/features/zlib.ini +DMAdev Drivers +-------------- + +Intel IDXD - EXPERIMENTAL +M: Bruce Richardson +M: Kevin Laatz +F: drivers/dma/idxd/ +F: doc/guides/dmadevs/idxd.rst + + RegEx Drivers ------------- diff --git a/doc/guides/dmadevs/idxd.rst b/doc/guides/dmadevs/idxd.rst new file mode 100644 index 0000000000..924700d17e --- /dev/null +++ b/doc/guides/dmadevs/idxd.rst @@ -0,0 +1,58 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2021 Intel Corporation. + +.. include:: + +IDXD DMA Device Driver +====================== + +The ``idxd`` dmadev driver provides a poll-mode driver (PMD) for Intel\ |reg| +Data Streaming Accelerator `(Intel DSA) +`_. +This PMD can be used in conjunction with Intel\ |reg| DSA devices to offload +data operations, such as data copies, to hardware, freeing up CPU cycles for +other tasks. + +Hardware Requirements +---------------------- + +The ``dpdk-devbind.py`` script, included with DPDK, can be used to show the +presence of supported hardware. Running ``dpdk-devbind.py --status-dev dma`` +will show all the DMA devices on the system, including IDXD supported devices. +Intel\ |reg| DSA devices, are currently (at time of writing) appearing +as devices with type “0b25”, due to the absence of pci-id database entries for +them at this point. + +Compilation +------------ + +For builds using ``meson`` and ``ninja``, the driver will be built when the +target platform is x86-based. No additional compilation steps are necessary. + +Device Setup +------------- + +Devices using VFIO/UIO drivers +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The HW devices to be used will need to be bound to a user-space IO driver for use. +The ``dpdk-devbind.py`` script can be used to view the state of the devices +and to bind them to a suitable DPDK-supported driver, such as ``vfio-pci``. +For example:: + + $ dpdk-devbind.py -b vfio-pci 6a:01.0 + +Device Probing and Initialization +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For devices bound to a suitable DPDK-supported VFIO/UIO driver, the HW devices will +be found as part of the device scan done at application initialization time without +the need to pass parameters to the application. + +For Intel\ |reg| DSA devices, DPDK will automatically configure the device with the +maximum number of workqueues available on it, partitioning all resources equally +among the queues. +If fewer workqueues are required, then the ``max_queues`` parameter may be passed to +the device driver on the EAL commandline, via the ``allowlist`` or ``-a`` flag e.g.:: + + $ dpdk-test -a ,max_queues=4 diff --git a/doc/guides/dmadevs/index.rst b/doc/guides/dmadevs/index.rst index 0bce29d766..5d4abf880e 100644 --- a/doc/guides/dmadevs/index.rst +++ b/doc/guides/dmadevs/index.rst @@ -10,3 +10,5 @@ an application through DMA API. .. toctree:: :maxdepth: 2 :numbered: + + idxd diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst index a71853b9c3..c0bfd9c1ba 100644 --- a/doc/guides/rel_notes/release_21_11.rst +++ b/doc/guides/rel_notes/release_21_11.rst @@ -92,6 +92,11 @@ New Features * Device allocation and it's multi-process support. * Control and data plane functions. +* **Added IDXD dmadev driver implementation.** + + The IDXD dmadev driver provide device drivers for the Intel DSA devices. + This device driver can be used through the generic dmadev API. + Removed Items ------------- diff --git a/drivers/dma/idxd/idxd_internal.h b/drivers/dma/idxd/idxd_internal.h new file mode 100644 index 0000000000..c6a7dcd72f --- /dev/null +++ b/drivers/dma/idxd/idxd_internal.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2021 Intel Corporation + */ + +#ifndef _IDXD_INTERNAL_H_ +#define _IDXD_INTERNAL_H_ + +/** + * @file idxd_internal.h + * + * Internal data structures for the idxd/DSA driver for dmadev + * + * @warning + * @b EXPERIMENTAL: these structures and APIs may change without prior notice + */ + +extern int idxd_pmd_logtype; + +#define IDXD_PMD_LOG(level, fmt, args...) rte_log(RTE_LOG_ ## level, \ + idxd_pmd_logtype, "IDXD: %s(): " fmt "\n", __func__, ##args) + +#define IDXD_PMD_DEBUG(fmt, args...) IDXD_PMD_LOG(DEBUG, fmt, ## args) +#define IDXD_PMD_INFO(fmt, args...) IDXD_PMD_LOG(INFO, fmt, ## args) +#define IDXD_PMD_ERR(fmt, args...) IDXD_PMD_LOG(ERR, fmt, ## args) +#define IDXD_PMD_WARN(fmt, args...) IDXD_PMD_LOG(WARNING, fmt, ## args) + +#endif /* _IDXD_INTERNAL_H_ */ diff --git a/drivers/dma/idxd/idxd_pci.c b/drivers/dma/idxd/idxd_pci.c new file mode 100644 index 0000000000..79e4aadcab --- /dev/null +++ b/drivers/dma/idxd/idxd_pci.c @@ -0,0 +1,55 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2021 Intel Corporation + */ + +#include + +#include "idxd_internal.h" + +#define IDXD_VENDOR_ID 0x8086 +#define IDXD_DEVICE_ID_SPR 0x0B25 + +#define IDXD_PMD_DMADEV_NAME_PCI dmadev_idxd_pci + +const struct rte_pci_id pci_id_idxd_map[] = { + { RTE_PCI_DEVICE(IDXD_VENDOR_ID, IDXD_DEVICE_ID_SPR) }, + { .vendor_id = 0, /* sentinel */ }, +}; + +static int +idxd_dmadev_probe_pci(struct rte_pci_driver *drv, struct rte_pci_device *dev) +{ + int ret = 0; + char name[PCI_PRI_STR_SIZE]; + + rte_pci_device_name(&dev->addr, name, sizeof(name)); + IDXD_PMD_INFO("Init %s on NUMA node %d", name, dev->device.numa_node); + dev->device.driver = &drv->driver; + + return ret; +} + +static int +idxd_dmadev_remove_pci(struct rte_pci_device *dev) +{ + char name[PCI_PRI_STR_SIZE]; + + rte_pci_device_name(&dev->addr, name, sizeof(name)); + + IDXD_PMD_INFO("Closing %s on NUMA node %d", + name, dev->device.numa_node); + + return 0; +} + +struct rte_pci_driver idxd_pmd_drv_pci = { + .id_table = pci_id_idxd_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING, + .probe = idxd_dmadev_probe_pci, + .remove = idxd_dmadev_remove_pci, +}; + +RTE_PMD_REGISTER_PCI(IDXD_PMD_DMADEV_NAME_PCI, idxd_pmd_drv_pci); +RTE_PMD_REGISTER_PCI_TABLE(IDXD_PMD_DMADEV_NAME_PCI, pci_id_idxd_map); +RTE_PMD_REGISTER_KMOD_DEP(IDXD_PMD_DMADEV_NAME_PCI, "vfio-pci"); +RTE_PMD_REGISTER_PARAM_STRING(dmadev_idxd_pci, "max_queues=0"); diff --git a/drivers/dma/idxd/meson.build b/drivers/dma/idxd/meson.build new file mode 100644 index 0000000000..9a64d75005 --- /dev/null +++ b/drivers/dma/idxd/meson.build @@ -0,0 +1,11 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2021 Intel Corporation + +if is_windows + subdir_done() +endif + +deps += ['bus_pci'] +sources = files( + 'idxd_pci.c' +) \ No newline at end of file diff --git a/drivers/dma/idxd/version.map b/drivers/dma/idxd/version.map new file mode 100644 index 0000000000..4a76d1d52d --- /dev/null +++ b/drivers/dma/idxd/version.map @@ -0,0 +1,3 @@ +DPDK_21 { + local: *; +}; diff --git a/drivers/dma/meson.build b/drivers/dma/meson.build index d9c7ede32f..411be7a240 100644 --- a/drivers/dma/meson.build +++ b/drivers/dma/meson.build @@ -2,5 +2,7 @@ # Copyright 2021 HiSilicon Limited drivers = [ + 'idxd', 'skeleton', ] +std_deps = ['dmadev'] From patchwork Fri Sep 17 15:24:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99214 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9217BA0C43; Fri, 17 Sep 2021 17:25:09 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C02244116A; Fri, 17 Sep 2021 17:24:54 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id 2E30E41157 for ; Fri, 17 Sep 2021 17:24:53 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222475995" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222475995" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:24:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="651909757" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by orsmga005.jf.intel.com with ESMTP; 17 Sep 2021 08:24:50 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 17 Sep 2021 15:24:24 +0000 Message-Id: <20210917152437.3270330-4-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210917152437.3270330-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210917152437.3270330-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 03/16] dma/idxd: add bus device probing X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the basic device probing for DSA devices bound to the IDXD kernel driver. These devices can be configured via sysfs and made available to DPDK if they are found during bus scan. Relevant documentation is included. Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh --- v4: fix 'vdev' naming, changed to 'bus' --- doc/guides/dmadevs/idxd.rst | 64 +++++++ drivers/dma/idxd/idxd_bus.c | 351 +++++++++++++++++++++++++++++++++++ drivers/dma/idxd/meson.build | 1 + 3 files changed, 416 insertions(+) create mode 100644 drivers/dma/idxd/idxd_bus.c diff --git a/doc/guides/dmadevs/idxd.rst b/doc/guides/dmadevs/idxd.rst index 924700d17e..ce33e2857a 100644 --- a/doc/guides/dmadevs/idxd.rst +++ b/doc/guides/dmadevs/idxd.rst @@ -32,6 +32,56 @@ target platform is x86-based. No additional compilation steps are necessary. Device Setup ------------- +Intel\ |reg| DSA devices can use the IDXD kernel driver or DPDK-supported drivers, +such as ``vfio-pci``. Both are supported by the IDXD PMD. + +Intel\ |reg| DSA devices using IDXD kernel driver +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To use an Intel\ |reg| DSA device bound to the IDXD kernel driver, the device must first be configured. +The `accel-config `_ utility library can be used for configuration. + +.. note:: + The device configuration can also be done by directly interacting with the sysfs nodes. + An example of how this may be done can be seen in the script ``dpdk_idxd_cfg.py`` + included in the driver source directory. + +There are some mandatory configuration steps before being able to use a device with an application. +The internal engines, which do the copies or other operations, +and the work-queues, which are used by applications to assign work to the device, +need to be assigned to groups, and the various other configuration options, +such as priority or queue depth, need to be set for each queue. + +To assign an engine to a group:: + + $ accel-config config-engine dsa0/engine0.0 --group-id=0 + $ accel-config config-engine dsa0/engine0.1 --group-id=1 + +To assign work queues to groups for passing descriptors to the engines a similar accel-config command can be used. +However, the work queues also need to be configured depending on the use case. +Some configuration options include: + +* mode (Dedicated/Shared): Indicates whether a WQ may accept jobs from multiple queues simultaneously. +* priority: WQ priority between 1 and 15. Larger value means higher priority. +* wq-size: the size of the WQ. Sum of all WQ sizes must be less that the total-size defined by the device. +* type: WQ type (kernel/mdev/user). Determines how the device is presented. +* name: identifier given to the WQ. + +Example configuration for a work queue:: + + $ accel-config config-wq dsa0/wq0.0 --group-id=0 \ + --mode=dedicated --priority=10 --wq-size=8 \ + --type=user --name=dpdk_app1 + +Once the devices have been configured, they need to be enabled:: + + $ accel-config enable-device dsa0 + $ accel-config enable-wq dsa0/wq0.0 + +Check the device configuration:: + + $ accel-config list + Devices using VFIO/UIO drivers ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -56,3 +106,17 @@ If fewer workqueues are required, then the ``max_queues`` parameter may be passe the device driver on the EAL commandline, via the ``allowlist`` or ``-a`` flag e.g.:: $ dpdk-test -a ,max_queues=4 + +For devices bound to the IDXD kernel driver, +the DPDK IDXD driver will automatically perform a scan for available workqueues +to use. Any workqueues found listed in ``/dev/dsa`` on the system will be checked +in ``/sys``, and any which have ``dpdk_`` prefix in their name will be automatically +probed by the driver to make them available to the application. +Alternatively, to support use by multiple DPDK processes simultaneously, +the value used as the DPDK ``--file-prefix`` parameter may be used as a workqueue +name prefix, instead of ``dpdk_``, allowing each DPDK application instance to only +use a subset of configured queues. + +Once probed successfully, irrespective of kernel driver, the device will appear as a ``dmadev``, +that is a "DMA device type" inside DPDK, and can be accessed using APIs from the +``rte_dmadev`` library. diff --git a/drivers/dma/idxd/idxd_bus.c b/drivers/dma/idxd/idxd_bus.c new file mode 100644 index 0000000000..ef589af30e --- /dev/null +++ b/drivers/dma/idxd/idxd_bus.c @@ -0,0 +1,351 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2021 Intel Corporation + */ + +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +#include "idxd_internal.h" + +/* default value for DSA paths, but allow override in environment for testing */ +#define DSA_DEV_PATH "/dev/dsa" +#define DSA_SYSFS_PATH "/sys/bus/dsa/devices" + +static unsigned int devcount; + +/** unique identifier for a DSA device/WQ instance */ +struct dsa_wq_addr { + uint16_t device_id; + uint16_t wq_id; +}; + +/** a DSA device instance */ +struct rte_dsa_device { + struct rte_device device; /**< Inherit core device */ + TAILQ_ENTRY(rte_dsa_device) next; /**< next dev in list */ + + char wq_name[32]; /**< the workqueue name/number e.g. wq0.1 */ + struct dsa_wq_addr addr; /**< Identifies the specific WQ */ +}; + +/* forward prototypes */ +struct dsa_bus; +static int dsa_scan(void); +static int dsa_probe(void); +static struct rte_device *dsa_find_device(const struct rte_device *start, + rte_dev_cmp_t cmp, const void *data); +static enum rte_iova_mode dsa_get_iommu_class(void); +static int dsa_addr_parse(const char *name, void *addr); + +/** List of devices */ +TAILQ_HEAD(dsa_device_list, rte_dsa_device); + +/** + * Structure describing the DSA bus + */ +struct dsa_bus { + struct rte_bus bus; /**< Inherit the generic class */ + struct rte_driver driver; /**< Driver struct for devices to point to */ + struct dsa_device_list device_list; /**< List of PCI devices */ +}; + +struct dsa_bus dsa_bus = { + .bus = { + .scan = dsa_scan, + .probe = dsa_probe, + .find_device = dsa_find_device, + .get_iommu_class = dsa_get_iommu_class, + .parse = dsa_addr_parse, + }, + .driver = { + .name = "dmadev_idxd" + }, + .device_list = TAILQ_HEAD_INITIALIZER(dsa_bus.device_list), +}; + +static inline const char * +dsa_get_dev_path(void) +{ + const char *path = getenv("DSA_DEV_PATH"); + return path ? path : DSA_DEV_PATH; +} + +static inline const char * +dsa_get_sysfs_path(void) +{ + const char *path = getenv("DSA_SYSFS_PATH"); + return path ? path : DSA_SYSFS_PATH; +} + +static void * +idxd_bus_mmap_wq(struct rte_dsa_device *dev) +{ + void *addr; + char path[PATH_MAX]; + int fd; + + snprintf(path, sizeof(path), "%s/%s", dsa_get_dev_path(), dev->wq_name); + fd = open(path, O_RDWR); + if (fd < 0) { + IDXD_PMD_ERR("Failed to open device path: %s", path); + return NULL; + } + + addr = mmap(NULL, 0x1000, PROT_WRITE, MAP_SHARED, fd, 0); + close(fd); + if (addr == MAP_FAILED) { + IDXD_PMD_ERR("Failed to mmap device %s", path); + return NULL; + } + + return addr; +} + +static int +read_wq_string(struct rte_dsa_device *dev, const char *filename, + char *value, size_t valuelen) +{ + char sysfs_node[PATH_MAX]; + int len; + int fd; + + snprintf(sysfs_node, sizeof(sysfs_node), "%s/%s/%s", + dsa_get_sysfs_path(), dev->wq_name, filename); + fd = open(sysfs_node, O_RDONLY); + if (fd < 0) { + IDXD_PMD_ERR("%s(): opening file '%s' failed: %s", + __func__, sysfs_node, strerror(errno)); + return -1; + } + + len = read(fd, value, valuelen - 1); + close(fd); + if (len < 0) { + IDXD_PMD_ERR("%s(): error reading file '%s': %s", + __func__, sysfs_node, strerror(errno)); + return -1; + } + value[len] = '\0'; + return 0; +} + +static int +read_wq_int(struct rte_dsa_device *dev, const char *filename, + int *value) +{ + char sysfs_node[PATH_MAX]; + FILE *f; + int ret = 0; + + snprintf(sysfs_node, sizeof(sysfs_node), "%s/%s/%s", + dsa_get_sysfs_path(), dev->wq_name, filename); + f = fopen(sysfs_node, "r"); + if (f == NULL) { + IDXD_PMD_ERR("%s(): opening file '%s' failed: %s", + __func__, sysfs_node, strerror(errno)); + return -1; + } + + if (fscanf(f, "%d", value) != 1) { + IDXD_PMD_ERR("%s(): error reading file '%s': %s", + __func__, sysfs_node, strerror(errno)); + ret = -1; + } + + fclose(f); + return ret; +} + +static int +read_device_int(struct rte_dsa_device *dev, const char *filename, + int *value) +{ + char sysfs_node[PATH_MAX]; + FILE *f; + int ret = 0; + + snprintf(sysfs_node, sizeof(sysfs_node), "%s/dsa%d/%s", + dsa_get_sysfs_path(), dev->addr.device_id, filename); + f = fopen(sysfs_node, "r"); + if (f == NULL) { + IDXD_PMD_ERR("%s(): opening file '%s' failed: %s", + __func__, sysfs_node, strerror(errno)); + return -1; + } + + if (fscanf(f, "%d", value) != 1) { + IDXD_PMD_ERR("%s(): error reading file '%s': %s", + __func__, sysfs_node, strerror(errno)); + ret = -1; + } + + fclose(f); + return ret; +} + +static int +idxd_probe_dsa(struct rte_dsa_device *dev) +{ + struct idxd_dmadev idxd = {0}; + int ret = 0; + + IDXD_PMD_INFO("Probing device %s on numa node %d", + dev->wq_name, dev->device.numa_node); + if (read_wq_int(dev, "size", &ret) < 0) + return -1; + idxd.max_batches = ret; + if (read_wq_int(dev, "max_batch_size", &ret) < 0) + return -1; + idxd.max_batch_size = ret; + idxd.qid = dev->addr.wq_id; + idxd.sva_support = 1; + + idxd.portal = idxd_bus_mmap_wq(dev); + if (idxd.portal == NULL) { + IDXD_PMD_ERR("WQ mmap failed"); + return -ENOENT; + } + + return 0; +} + +static int +is_for_this_process_use(const char *name) +{ + char *runtime_dir = strdup(rte_eal_get_runtime_dir()); + char *prefix = basename(runtime_dir); + int prefixlen = strlen(prefix); + int retval = 0; + + if (strncmp(name, "dpdk_", 5) == 0) + retval = 1; + if (strncmp(name, prefix, prefixlen) == 0 && name[prefixlen] == '_') + retval = 1; + + free(runtime_dir); + return retval; +} + +static int +dsa_probe(void) +{ + struct rte_dsa_device *dev; + + TAILQ_FOREACH(dev, &dsa_bus.device_list, next) { + char type[64], name[64]; + + if (read_wq_string(dev, "type", type, sizeof(type)) < 0 || + read_wq_string(dev, "name", name, sizeof(name)) < 0) + continue; + + if (strncmp(type, "user", 4) == 0 && is_for_this_process_use(name)) { + dev->device.driver = &dsa_bus.driver; + idxd_probe_dsa(dev); + continue; + } + IDXD_PMD_DEBUG("WQ '%s', not allocated to DPDK", dev->wq_name); + } + + return 0; +} + +static int +dsa_scan(void) +{ + const char *path = dsa_get_dev_path(); + struct dirent *wq; + DIR *dev_dir; + + dev_dir = opendir(path); + if (dev_dir == NULL) { + if (errno == ENOENT) + return 0; /* no bus, return without error */ + IDXD_PMD_ERR("%s(): opendir '%s' failed: %s", + __func__, path, strerror(errno)); + return -1; + } + + while ((wq = readdir(dev_dir)) != NULL) { + struct rte_dsa_device *dev; + int numa_node = -1; + + if (strncmp(wq->d_name, "wq", 2) != 0) + continue; + if (strnlen(wq->d_name, sizeof(dev->wq_name)) == sizeof(dev->wq_name)) { + IDXD_PMD_ERR("%s(): wq name too long: '%s', skipping", + __func__, wq->d_name); + continue; + } + IDXD_PMD_DEBUG("%s(): found %s/%s", __func__, path, wq->d_name); + + dev = malloc(sizeof(*dev)); + if (dsa_addr_parse(wq->d_name, &dev->addr) < 0) { + IDXD_PMD_ERR("Error parsing WQ name: %s", wq->d_name); + free(dev); + continue; + } + dev->device.bus = &dsa_bus.bus; + strlcpy(dev->wq_name, wq->d_name, sizeof(dev->wq_name)); + TAILQ_INSERT_TAIL(&dsa_bus.device_list, dev, next); + devcount++; + + read_device_int(dev, "numa_node", &numa_node); + dev->device.numa_node = numa_node; + dev->device.name = dev->wq_name; + } + + closedir(dev_dir); + return 0; +} + +static struct rte_device * +dsa_find_device(const struct rte_device *start, rte_dev_cmp_t cmp, + const void *data) +{ + struct rte_dsa_device *dev = TAILQ_FIRST(&dsa_bus.device_list); + + /* the rte_device struct must be at start of dsa structure */ + RTE_BUILD_BUG_ON(offsetof(struct rte_dsa_device, device) != 0); + + if (start != NULL) /* jump to start point if given */ + dev = TAILQ_NEXT((const struct rte_dsa_device *)start, next); + while (dev != NULL) { + if (cmp(&dev->device, data) == 0) + return &dev->device; + dev = TAILQ_NEXT(dev, next); + } + return NULL; +} + +static enum rte_iova_mode +dsa_get_iommu_class(void) +{ + /* if there are no devices, report don't care, otherwise VA mode */ + return devcount > 0 ? RTE_IOVA_VA : RTE_IOVA_DC; +} + +static int +dsa_addr_parse(const char *name, void *addr) +{ + struct dsa_wq_addr *wq = addr; + unsigned int device_id, wq_id; + + if (sscanf(name, "wq%u.%u", &device_id, &wq_id) != 2) { + IDXD_PMD_DEBUG("Parsing WQ name failed: %s", name); + return -1; + } + + wq->device_id = device_id; + wq->wq_id = wq_id; + return 0; +} + +RTE_REGISTER_BUS(dsa, dsa_bus.bus); diff --git a/drivers/dma/idxd/meson.build b/drivers/dma/idxd/meson.build index 9a64d75005..c864fce3b3 100644 --- a/drivers/dma/idxd/meson.build +++ b/drivers/dma/idxd/meson.build @@ -7,5 +7,6 @@ endif deps += ['bus_pci'] sources = files( + 'idxd_bus.c', 'idxd_pci.c' ) \ No newline at end of file From patchwork Fri Sep 17 15:24:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99215 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 65C00A0C43; Fri, 17 Sep 2021 17:25:16 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CF64D41161; Fri, 17 Sep 2021 17:24:57 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id 3FAA441143 for ; Fri, 17 Sep 2021 17:24:55 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222475999" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222475999" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:24:54 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="651909764" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by orsmga005.jf.intel.com with ESMTP; 17 Sep 2021 08:24:52 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 17 Sep 2021 15:24:25 +0000 Message-Id: <20210917152437.3270330-5-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210917152437.3270330-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210917152437.3270330-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 04/16] dma/idxd: create dmadev instances on bus probe X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When a suitable device is found during the bus scan/probe, create a dmadev instance for each HW queue. Internal structures required for device creation are also added. Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh --- v4: - fix 'vdev' naming, changed to 'bus' - rebase changes --- drivers/dma/idxd/idxd_bus.c | 19 ++++++++ drivers/dma/idxd/idxd_common.c | 76 ++++++++++++++++++++++++++++++++ drivers/dma/idxd/idxd_internal.h | 40 +++++++++++++++++ drivers/dma/idxd/meson.build | 1 + 4 files changed, 136 insertions(+) create mode 100644 drivers/dma/idxd/idxd_common.c diff --git a/drivers/dma/idxd/idxd_bus.c b/drivers/dma/idxd/idxd_bus.c index ef589af30e..b48fa954ed 100644 --- a/drivers/dma/idxd/idxd_bus.c +++ b/drivers/dma/idxd/idxd_bus.c @@ -85,6 +85,18 @@ dsa_get_sysfs_path(void) return path ? path : DSA_SYSFS_PATH; } +static int +idxd_dev_close(struct rte_dma_dev *dev) +{ + struct idxd_dmadev *idxd = dev->data->dev_private; + munmap(idxd->portal, 0x1000); + return 0; +} + +static const struct rte_dma_dev_ops idxd_bus_ops = { + .dev_close = idxd_dev_close, +}; + static void * idxd_bus_mmap_wq(struct rte_dsa_device *dev) { @@ -206,6 +218,7 @@ idxd_probe_dsa(struct rte_dsa_device *dev) return -1; idxd.max_batch_size = ret; idxd.qid = dev->addr.wq_id; + idxd.u.bus.dsa_id = dev->addr.device_id; idxd.sva_support = 1; idxd.portal = idxd_bus_mmap_wq(dev); @@ -214,6 +227,12 @@ idxd_probe_dsa(struct rte_dsa_device *dev) return -ENOENT; } + ret = idxd_dmadev_create(dev->wq_name, &dev->device, &idxd, &idxd_bus_ops); + if (ret) { + IDXD_PMD_ERR("Failed to create rawdev %s", dev->wq_name); + return ret; + } + return 0; } diff --git a/drivers/dma/idxd/idxd_common.c b/drivers/dma/idxd/idxd_common.c new file mode 100644 index 0000000000..8afad637fc --- /dev/null +++ b/drivers/dma/idxd/idxd_common.c @@ -0,0 +1,76 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2021 Intel Corporation + */ + +#include +#include +#include + +#include "idxd_internal.h" + +#define IDXD_PMD_NAME_STR "dmadev_idxd" + +int +idxd_dmadev_create(const char *name, struct rte_device *dev, + const struct idxd_dmadev *base_idxd, + const struct rte_dma_dev_ops *ops) +{ + struct idxd_dmadev *idxd; + struct rte_dma_dev *dmadev = NULL; + int ret = 0; + + if (!name) { + IDXD_PMD_ERR("Invalid name of the device!"); + ret = -EINVAL; + goto cleanup; + } + + /* Allocate device structure */ + dmadev = rte_dma_pmd_allocate(name, dev->numa_node, + sizeof(dmadev->dev_private)); + if (dmadev == NULL) { + IDXD_PMD_ERR("Unable to allocate raw device"); + ret = -ENOMEM; + goto cleanup; + } + dmadev->dev_ops = ops; + dmadev->device = dev; + + idxd = rte_malloc_socket(NULL, sizeof(struct idxd_dmadev), 0, dev->numa_node); + if (idxd == NULL) { + IDXD_PMD_ERR("Unable to allocate memory for device"); + ret = -ENOMEM; + goto cleanup; + } + dmadev->data->dev_private = idxd; + dmadev->dev_private = idxd; + *idxd = *base_idxd; /* copy over the main fields already passed in */ + idxd->dmadev = dmadev; + + /* allocate batch index ring and completion ring. + * The +1 is because we can never fully use + * the ring, otherwise read == write means both full and empty. + */ + idxd->batch_comp_ring = rte_zmalloc(NULL, (sizeof(idxd->batch_idx_ring[0]) + + sizeof(idxd->batch_comp_ring[0])) * (idxd->max_batches + 1), + sizeof(idxd->batch_comp_ring[0])); + if (idxd->batch_comp_ring == NULL) { + IDXD_PMD_ERR("Unable to reserve memory for batch data\n"); + ret = -ENOMEM; + goto cleanup; + } + idxd->batch_idx_ring = (void *)&idxd->batch_comp_ring[idxd->max_batches+1]; + idxd->batch_iova = rte_mem_virt2iova(idxd->batch_comp_ring); + + return 0; + +cleanup: + if (dmadev) + rte_dma_pmd_release(name); + + return ret; +} + +int idxd_pmd_logtype; + +RTE_LOG_REGISTER_DEFAULT(idxd_pmd_logtype, WARNING); diff --git a/drivers/dma/idxd/idxd_internal.h b/drivers/dma/idxd/idxd_internal.h index c6a7dcd72f..fa6f053f72 100644 --- a/drivers/dma/idxd/idxd_internal.h +++ b/drivers/dma/idxd/idxd_internal.h @@ -24,4 +24,44 @@ extern int idxd_pmd_logtype; #define IDXD_PMD_ERR(fmt, args...) IDXD_PMD_LOG(ERR, fmt, ## args) #define IDXD_PMD_WARN(fmt, args...) IDXD_PMD_LOG(WARNING, fmt, ## args) +struct idxd_dmadev { + /* counters to track the batches */ + unsigned short max_batches; + unsigned short batch_idx_read; + unsigned short batch_idx_write; + + /* track descriptors and handles */ + unsigned short desc_ring_mask; + unsigned short ids_avail; /* handles for ops completed */ + unsigned short ids_returned; /* the read pointer for hdls/desc rings */ + unsigned short batch_start; /* start+size == write pointer for hdls/desc */ + unsigned short batch_size; + + void *portal; /* address to write the batch descriptor */ + + struct idxd_completion *batch_comp_ring; + unsigned short *batch_idx_ring; /* store where each batch ends */ + + struct rte_dma_stats stats; + + rte_iova_t batch_iova; /* base address of the batch comp ring */ + rte_iova_t desc_iova; /* base address of desc ring, needed for completions */ + + unsigned short max_batch_size; + + struct rte_dma_dev *dmadev; + struct rte_dma_vchan_conf qcfg; + uint8_t sva_support; + uint8_t qid; + + union { + struct { + unsigned int dsa_id; + } bus; + } u; +}; + +int idxd_dmadev_create(const char *name, struct rte_device *dev, + const struct idxd_dmadev *base_idxd, const struct rte_dma_dev_ops *ops); + #endif /* _IDXD_INTERNAL_H_ */ diff --git a/drivers/dma/idxd/meson.build b/drivers/dma/idxd/meson.build index c864fce3b3..36dbd3e518 100644 --- a/drivers/dma/idxd/meson.build +++ b/drivers/dma/idxd/meson.build @@ -8,5 +8,6 @@ endif deps += ['bus_pci'] sources = files( 'idxd_bus.c', + 'idxd_common.c', 'idxd_pci.c' ) \ No newline at end of file From patchwork Fri Sep 17 15:24:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99216 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 44FD0A0C43; Fri, 17 Sep 2021 17:25:22 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E07B741173; Fri, 17 Sep 2021 17:24:58 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id 59B64406B4 for ; Fri, 17 Sep 2021 17:24:57 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222476001" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222476001" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:24:56 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="651909774" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by orsmga005.jf.intel.com with ESMTP; 17 Sep 2021 08:24:54 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 17 Sep 2021 15:24:26 +0000 Message-Id: <20210917152437.3270330-6-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210917152437.3270330-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210917152437.3270330-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 05/16] dma/idxd: create dmadev instances on pci probe X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When a suitable device is found during the PCI probe, create a dmadev instance for each HW queue. HW definitions required are also included. Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh --- v4: rebase changes --- drivers/dma/idxd/idxd_hw_defs.h | 71 ++++++++ drivers/dma/idxd/idxd_internal.h | 16 ++ drivers/dma/idxd/idxd_pci.c | 278 ++++++++++++++++++++++++++++++- 3 files changed, 362 insertions(+), 3 deletions(-) create mode 100644 drivers/dma/idxd/idxd_hw_defs.h diff --git a/drivers/dma/idxd/idxd_hw_defs.h b/drivers/dma/idxd/idxd_hw_defs.h new file mode 100644 index 0000000000..ea627cba6d --- /dev/null +++ b/drivers/dma/idxd/idxd_hw_defs.h @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2021 Intel Corporation + */ + +#ifndef _IDXD_HW_DEFS_H_ +#define _IDXD_HW_DEFS_H_ + +/*** Definitions for Intel(R) Data Streaming Accelerator ***/ + +#define IDXD_CMD_SHIFT 20 +enum rte_idxd_cmds { + idxd_enable_dev = 1, + idxd_disable_dev, + idxd_drain_all, + idxd_abort_all, + idxd_reset_device, + idxd_enable_wq, + idxd_disable_wq, + idxd_drain_wq, + idxd_abort_wq, + idxd_reset_wq, +}; + +/* General bar0 registers */ +struct rte_idxd_bar0 { + uint32_t __rte_cache_aligned version; /* offset 0x00 */ + uint64_t __rte_aligned(0x10) gencap; /* offset 0x10 */ + uint64_t __rte_aligned(0x10) wqcap; /* offset 0x20 */ + uint64_t __rte_aligned(0x10) grpcap; /* offset 0x30 */ + uint64_t __rte_aligned(0x08) engcap; /* offset 0x38 */ + uint64_t __rte_aligned(0x10) opcap; /* offset 0x40 */ + uint64_t __rte_aligned(0x20) offsets[2]; /* offset 0x60 */ + uint32_t __rte_aligned(0x20) gencfg; /* offset 0x80 */ + uint32_t __rte_aligned(0x08) genctrl; /* offset 0x88 */ + uint32_t __rte_aligned(0x10) gensts; /* offset 0x90 */ + uint32_t __rte_aligned(0x08) intcause; /* offset 0x98 */ + uint32_t __rte_aligned(0x10) cmd; /* offset 0xA0 */ + uint32_t __rte_aligned(0x08) cmdstatus; /* offset 0xA8 */ + uint64_t __rte_aligned(0x20) swerror[4]; /* offset 0xC0 */ +}; + +/* workqueue config is provided by array of uint32_t. */ +enum rte_idxd_wqcfg { + wq_size_idx, /* size is in first 32-bit value */ + wq_threshold_idx, /* WQ threshold second 32-bits */ + wq_mode_idx, /* WQ mode and other flags */ + wq_sizes_idx, /* WQ transfer and batch sizes */ + wq_occ_int_idx, /* WQ occupancy interrupt handle */ + wq_occ_limit_idx, /* WQ occupancy limit */ + wq_state_idx, /* WQ state and occupancy state */ +}; + +#define WQ_MODE_SHARED 0 +#define WQ_MODE_DEDICATED 1 +#define WQ_PRIORITY_SHIFT 4 +#define WQ_BATCH_SZ_SHIFT 5 +#define WQ_STATE_SHIFT 30 +#define WQ_STATE_MASK 0x3 + +struct rte_idxd_grpcfg { + uint64_t grpwqcfg[4] __rte_cache_aligned; /* 64-byte register set */ + uint64_t grpengcfg; /* offset 32 */ + uint32_t grpflags; /* offset 40 */ +}; + +#define GENSTS_DEV_STATE_MASK 0x03 +#define CMDSTATUS_ACTIVE_SHIFT 31 +#define CMDSTATUS_ACTIVE_MASK (1 << 31) +#define CMDSTATUS_ERR_MASK 0xFF + +#endif diff --git a/drivers/dma/idxd/idxd_internal.h b/drivers/dma/idxd/idxd_internal.h index fa6f053f72..cb3a68c69b 100644 --- a/drivers/dma/idxd/idxd_internal.h +++ b/drivers/dma/idxd/idxd_internal.h @@ -5,6 +5,10 @@ #ifndef _IDXD_INTERNAL_H_ #define _IDXD_INTERNAL_H_ +#include + +#include "idxd_hw_defs.h" + /** * @file idxd_internal.h * @@ -24,6 +28,16 @@ extern int idxd_pmd_logtype; #define IDXD_PMD_ERR(fmt, args...) IDXD_PMD_LOG(ERR, fmt, ## args) #define IDXD_PMD_WARN(fmt, args...) IDXD_PMD_LOG(WARNING, fmt, ## args) +struct idxd_pci_common { + rte_spinlock_t lk; + + uint8_t wq_cfg_sz; + volatile struct rte_idxd_bar0 *regs; + volatile uint32_t *wq_regs_base; + volatile struct rte_idxd_grpcfg *grp_regs; + volatile void *portals; +}; + struct idxd_dmadev { /* counters to track the batches */ unsigned short max_batches; @@ -58,6 +72,8 @@ struct idxd_dmadev { struct { unsigned int dsa_id; } bus; + + struct idxd_pci_common *pci; } u; }; diff --git a/drivers/dma/idxd/idxd_pci.c b/drivers/dma/idxd/idxd_pci.c index 79e4aadcab..171e5ffc07 100644 --- a/drivers/dma/idxd/idxd_pci.c +++ b/drivers/dma/idxd/idxd_pci.c @@ -3,6 +3,9 @@ */ #include +#include +#include +#include #include "idxd_internal.h" @@ -16,17 +19,286 @@ const struct rte_pci_id pci_id_idxd_map[] = { { .vendor_id = 0, /* sentinel */ }, }; +static inline int +idxd_pci_dev_command(struct idxd_dmadev *idxd, enum rte_idxd_cmds command) +{ + uint8_t err_code; + uint16_t qid = idxd->qid; + int i = 0; + + if (command >= idxd_disable_wq && command <= idxd_reset_wq) + qid = (1 << qid); + rte_spinlock_lock(&idxd->u.pci->lk); + idxd->u.pci->regs->cmd = (command << IDXD_CMD_SHIFT) | qid; + + do { + rte_pause(); + err_code = idxd->u.pci->regs->cmdstatus; + if (++i >= 1000) { + IDXD_PMD_ERR("Timeout waiting for command response from HW"); + rte_spinlock_unlock(&idxd->u.pci->lk); + return err_code; + } + } while (idxd->u.pci->regs->cmdstatus & CMDSTATUS_ACTIVE_MASK); + rte_spinlock_unlock(&idxd->u.pci->lk); + + return err_code & CMDSTATUS_ERR_MASK; +} + +static uint32_t * +idxd_get_wq_cfg(struct idxd_pci_common *pci, uint8_t wq_idx) +{ + return RTE_PTR_ADD(pci->wq_regs_base, + (uintptr_t)wq_idx << (5 + pci->wq_cfg_sz)); +} + +static int +idxd_is_wq_enabled(struct idxd_dmadev *idxd) +{ + uint32_t state = idxd_get_wq_cfg(idxd->u.pci, idxd->qid)[wq_state_idx]; + return ((state >> WQ_STATE_SHIFT) & WQ_STATE_MASK) == 0x1; +} + +static const struct rte_dma_dev_ops idxd_pci_ops = { + +}; + +/* each portal uses 4 x 4k pages */ +#define IDXD_PORTAL_SIZE (4096 * 4) + +static int +init_pci_device(struct rte_pci_device *dev, struct idxd_dmadev *idxd, + unsigned int max_queues) +{ + struct idxd_pci_common *pci; + uint8_t nb_groups, nb_engines, nb_wqs; + uint16_t grp_offset, wq_offset; /* how far into bar0 the regs are */ + uint16_t wq_size, total_wq_size; + uint8_t lg2_max_batch, lg2_max_copy_size; + unsigned int i, err_code; + + pci = malloc(sizeof(*pci)); + if (pci == NULL) { + IDXD_PMD_ERR("%s: Can't allocate memory", __func__); + goto err; + } + rte_spinlock_init(&pci->lk); + + /* assign the bar registers, and then configure device */ + pci->regs = dev->mem_resource[0].addr; + grp_offset = (uint16_t)pci->regs->offsets[0]; + pci->grp_regs = RTE_PTR_ADD(pci->regs, grp_offset * 0x100); + wq_offset = (uint16_t)(pci->regs->offsets[0] >> 16); + pci->wq_regs_base = RTE_PTR_ADD(pci->regs, wq_offset * 0x100); + pci->portals = dev->mem_resource[2].addr; + pci->wq_cfg_sz = (pci->regs->wqcap >> 24) & 0x0F; + + /* sanity check device status */ + if (pci->regs->gensts & GENSTS_DEV_STATE_MASK) { + /* need function-level-reset (FLR) or is enabled */ + IDXD_PMD_ERR("Device status is not disabled, cannot init"); + goto err; + } + if (pci->regs->cmdstatus & CMDSTATUS_ACTIVE_MASK) { + /* command in progress */ + IDXD_PMD_ERR("Device has a command in progress, cannot init"); + goto err; + } + + /* read basic info about the hardware for use when configuring */ + nb_groups = (uint8_t)pci->regs->grpcap; + nb_engines = (uint8_t)pci->regs->engcap; + nb_wqs = (uint8_t)(pci->regs->wqcap >> 16); + total_wq_size = (uint16_t)pci->regs->wqcap; + lg2_max_copy_size = (uint8_t)(pci->regs->gencap >> 16) & 0x1F; + lg2_max_batch = (uint8_t)(pci->regs->gencap >> 21) & 0x0F; + + IDXD_PMD_DEBUG("nb_groups = %u, nb_engines = %u, nb_wqs = %u", + nb_groups, nb_engines, nb_wqs); + + /* zero out any old config */ + for (i = 0; i < nb_groups; i++) { + pci->grp_regs[i].grpengcfg = 0; + pci->grp_regs[i].grpwqcfg[0] = 0; + } + for (i = 0; i < nb_wqs; i++) + idxd_get_wq_cfg(pci, i)[0] = 0; + + /* limit queues if necessary */ + if (max_queues != 0 && nb_wqs > max_queues) { + nb_wqs = max_queues; + if (nb_engines > max_queues) + nb_engines = max_queues; + if (nb_groups > max_queues) + nb_engines = max_queues; + IDXD_PMD_DEBUG("Limiting queues to %u", nb_wqs); + } + + /* put each engine into a separate group to avoid reordering */ + if (nb_groups > nb_engines) + nb_groups = nb_engines; + if (nb_groups < nb_engines) + nb_engines = nb_groups; + + /* assign engines to groups, round-robin style */ + for (i = 0; i < nb_engines; i++) { + IDXD_PMD_DEBUG("Assigning engine %u to group %u", + i, i % nb_groups); + pci->grp_regs[i % nb_groups].grpengcfg |= (1ULL << i); + } + + /* now do the same for queues and give work slots to each queue */ + wq_size = total_wq_size / nb_wqs; + IDXD_PMD_DEBUG("Work queue size = %u, max batch = 2^%u, max copy = 2^%u", + wq_size, lg2_max_batch, lg2_max_copy_size); + for (i = 0; i < nb_wqs; i++) { + /* add engine "i" to a group */ + IDXD_PMD_DEBUG("Assigning work queue %u to group %u", + i, i % nb_groups); + pci->grp_regs[i % nb_groups].grpwqcfg[0] |= (1ULL << i); + /* now configure it, in terms of size, max batch, mode */ + idxd_get_wq_cfg(pci, i)[wq_size_idx] = wq_size; + idxd_get_wq_cfg(pci, i)[wq_mode_idx] = (1 << WQ_PRIORITY_SHIFT) | + WQ_MODE_DEDICATED; + idxd_get_wq_cfg(pci, i)[wq_sizes_idx] = lg2_max_copy_size | + (lg2_max_batch << WQ_BATCH_SZ_SHIFT); + } + + /* dump the group configuration to output */ + for (i = 0; i < nb_groups; i++) { + IDXD_PMD_DEBUG("## Group %d", i); + IDXD_PMD_DEBUG(" GRPWQCFG: %"PRIx64, pci->grp_regs[i].grpwqcfg[0]); + IDXD_PMD_DEBUG(" GRPENGCFG: %"PRIx64, pci->grp_regs[i].grpengcfg); + IDXD_PMD_DEBUG(" GRPFLAGS: %"PRIx32, pci->grp_regs[i].grpflags); + } + + idxd->u.pci = pci; + idxd->max_batches = wq_size; + + /* enable the device itself */ + err_code = idxd_pci_dev_command(idxd, idxd_enable_dev); + if (err_code) { + IDXD_PMD_ERR("Error enabling device: code %#x", err_code); + return err_code; + } + IDXD_PMD_DEBUG("IDXD Device enabled OK"); + + return nb_wqs; + +err: + free(pci); + return -1; +} + static int idxd_dmadev_probe_pci(struct rte_pci_driver *drv, struct rte_pci_device *dev) { - int ret = 0; + struct idxd_dmadev idxd = {0}; + uint8_t nb_wqs; + int qid, ret = 0; char name[PCI_PRI_STR_SIZE]; + unsigned int max_queues = 0; rte_pci_device_name(&dev->addr, name, sizeof(name)); IDXD_PMD_INFO("Init %s on NUMA node %d", name, dev->device.numa_node); dev->device.driver = &drv->driver; - return ret; + if (dev->device.devargs && dev->device.devargs->args[0] != '\0') { + /* if the number of devargs grows beyond just 1, use rte_kvargs */ + if (sscanf(dev->device.devargs->args, + "max_queues=%u", &max_queues) != 1) { + IDXD_PMD_ERR("Invalid device parameter: '%s'", + dev->device.devargs->args); + return -1; + } + } + + ret = init_pci_device(dev, &idxd, max_queues); + if (ret < 0) { + IDXD_PMD_ERR("Error initializing PCI hardware"); + return ret; + } + if (idxd.u.pci->portals == NULL) { + IDXD_PMD_ERR("Error, invalid portal assigned during initialization\n"); + return -EINVAL; + } + nb_wqs = (uint8_t)ret; + + /* set up one device for each queue */ + for (qid = 0; qid < nb_wqs; qid++) { + char qname[32]; + + /* add the queue number to each device name */ + snprintf(qname, sizeof(qname), "%s-q%d", name, qid); + idxd.qid = qid; + idxd.portal = RTE_PTR_ADD(idxd.u.pci->portals, + qid * IDXD_PORTAL_SIZE); + if (idxd_is_wq_enabled(&idxd)) + IDXD_PMD_ERR("Error, WQ %u seems enabled", qid); + ret = idxd_dmadev_create(qname, &dev->device, + &idxd, &idxd_pci_ops); + if (ret != 0) { + IDXD_PMD_ERR("Failed to create dmadev %s", name); + if (qid == 0) /* if no devices using this, free pci */ + free(idxd.u.pci); + return ret; + } + } + + return 0; +} + +static int +idxd_dmadev_destroy(const char *name) +{ + int ret; + uint8_t err_code; + struct rte_dma_dev *dmadev; + struct idxd_dmadev *idxd; + int dev_id = rte_dma_get_dev_id(name); + + if (!name) { + IDXD_PMD_ERR("Invalid device name"); + return -EINVAL; + } + + if (dev_id < 0) { + IDXD_PMD_ERR("Invalid device ID"); + return -EINVAL; + } + + dmadev = &rte_dma_devices[dev_id]; + if (!dmadev) { + IDXD_PMD_ERR("Invalid device name (%s)", name); + return -EINVAL; + } + + idxd = dmadev->dev_private; + if (!idxd) { + IDXD_PMD_ERR("Error getting dev_private"); + return -EINVAL; + } + + /* disable the device */ + err_code = idxd_pci_dev_command(idxd, idxd_disable_dev); + if (err_code) { + IDXD_PMD_ERR("Error disabling device: code %#x", err_code); + return err_code; + } + IDXD_PMD_DEBUG("IDXD Device disabled OK"); + + /* free device memory */ + IDXD_PMD_DEBUG("Freeing device driver memory"); + dmadev->dev_private = NULL; + rte_free(idxd->batch_idx_ring); + rte_free(idxd->desc_ring); + + /* rte_dma_close is called by pmd_release */ + ret = rte_dma_pmd_release(name); + if (ret) + IDXD_PMD_DEBUG("Device cleanup failed"); + + return 0; } static int @@ -39,7 +311,7 @@ idxd_dmadev_remove_pci(struct rte_pci_device *dev) IDXD_PMD_INFO("Closing %s on NUMA node %d", name, dev->device.numa_node); - return 0; + return idxd_dmadev_destroy(name); } struct rte_pci_driver idxd_pmd_drv_pci = { From patchwork Fri Sep 17 15:24:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99217 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 22446A0C43; Fri, 17 Sep 2021 17:25:30 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4A43E41180; Fri, 17 Sep 2021 17:25:01 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id 3FC6041176 for ; Fri, 17 Sep 2021 17:24:59 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222476005" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222476005" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:24:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="651909782" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by orsmga005.jf.intel.com with ESMTP; 17 Sep 2021 08:24:57 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 17 Sep 2021 15:24:27 +0000 Message-Id: <20210917152437.3270330-7-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210917152437.3270330-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210917152437.3270330-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 06/16] dma/idxd: add datapath structures X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add data structures required for the data path for IDXD devices. Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh --- v2: add completion status for invalid opcode --- drivers/dma/idxd/idxd_bus.c | 1 + drivers/dma/idxd/idxd_common.c | 33 ++++++++++++++++++ drivers/dma/idxd/idxd_hw_defs.h | 60 ++++++++++++++++++++++++++++++++ drivers/dma/idxd/idxd_internal.h | 3 ++ drivers/dma/idxd/idxd_pci.c | 2 +- 5 files changed, 98 insertions(+), 1 deletion(-) diff --git a/drivers/dma/idxd/idxd_bus.c b/drivers/dma/idxd/idxd_bus.c index b48fa954ed..3c0837ec52 100644 --- a/drivers/dma/idxd/idxd_bus.c +++ b/drivers/dma/idxd/idxd_bus.c @@ -95,6 +95,7 @@ idxd_dev_close(struct rte_dma_dev *dev) static const struct rte_dma_dev_ops idxd_bus_ops = { .dev_close = idxd_dev_close, + .dev_dump = idxd_dump, }; static void * diff --git a/drivers/dma/idxd/idxd_common.c b/drivers/dma/idxd/idxd_common.c index 8afad637fc..45cde78e88 100644 --- a/drivers/dma/idxd/idxd_common.c +++ b/drivers/dma/idxd/idxd_common.c @@ -10,6 +10,35 @@ #define IDXD_PMD_NAME_STR "dmadev_idxd" +int +idxd_dump(const struct rte_dma_dev *dev, FILE *f) +{ + struct idxd_dmadev *idxd = dev->dev_private; + unsigned int i; + + fprintf(f, "== IDXD Private Data ==\n"); + fprintf(f, " Portal: %p\n", idxd->portal); + fprintf(f, " Config: { ring_size: %u }\n", + idxd->qcfg.nb_desc); + fprintf(f, " Batch ring (sz = %u, max_batches = %u):\n\t", + idxd->max_batches + 1, idxd->max_batches); + for (i = 0; i <= idxd->max_batches; i++) { + fprintf(f, " %u ", idxd->batch_idx_ring[i]); + if (i == idxd->batch_idx_read && i == idxd->batch_idx_write) + fprintf(f, "[rd ptr, wr ptr] "); + else if (i == idxd->batch_idx_read) + fprintf(f, "[rd ptr] "); + else if (i == idxd->batch_idx_write) + fprintf(f, "[wr ptr] "); + if (i == idxd->max_batches) + fprintf(f, "\n"); + } + + fprintf(f, " Curr batch: start = %u, size = %u\n", idxd->batch_start, idxd->batch_size); + fprintf(f, " IDS: avail = %u, returned: %u\n", idxd->ids_avail, idxd->ids_returned); + return 0; +} + int idxd_dmadev_create(const char *name, struct rte_device *dev, const struct idxd_dmadev *base_idxd, @@ -19,6 +48,10 @@ idxd_dmadev_create(const char *name, struct rte_device *dev, struct rte_dma_dev *dmadev = NULL; int ret = 0; + RTE_BUILD_BUG_ON(sizeof(struct idxd_hw_desc) != 64); + RTE_BUILD_BUG_ON(offsetof(struct idxd_hw_desc, size) != 32); + RTE_BUILD_BUG_ON(sizeof(struct idxd_completion) != 32); + if (!name) { IDXD_PMD_ERR("Invalid name of the device!"); ret = -EINVAL; diff --git a/drivers/dma/idxd/idxd_hw_defs.h b/drivers/dma/idxd/idxd_hw_defs.h index ea627cba6d..55ca9f7f52 100644 --- a/drivers/dma/idxd/idxd_hw_defs.h +++ b/drivers/dma/idxd/idxd_hw_defs.h @@ -5,6 +5,66 @@ #ifndef _IDXD_HW_DEFS_H_ #define _IDXD_HW_DEFS_H_ +/* + * Defines used in the data path for interacting with IDXD hardware. + */ +#define IDXD_CMD_OP_SHIFT 24 +enum rte_idxd_ops { + idxd_op_nop = 0, + idxd_op_batch, + idxd_op_drain, + idxd_op_memmove, + idxd_op_fill +}; + +#define IDXD_FLAG_FENCE (1 << 0) +#define IDXD_FLAG_COMPLETION_ADDR_VALID (1 << 2) +#define IDXD_FLAG_REQUEST_COMPLETION (1 << 3) +#define IDXD_FLAG_CACHE_CONTROL (1 << 8) + +/** + * Hardware descriptor used by DSA hardware, for both bursts and + * for individual operations. + */ +struct idxd_hw_desc { + uint32_t pasid; + uint32_t op_flags; + rte_iova_t completion; + + RTE_STD_C11 + union { + rte_iova_t src; /* source address for copy ops etc. */ + rte_iova_t desc_addr; /* descriptor pointer for batch */ + }; + rte_iova_t dst; + + uint32_t size; /* length of data for op, or batch size */ + + uint16_t intr_handle; /* completion interrupt handle */ + + /* remaining 26 bytes are reserved */ + uint16_t __reserved[13]; +} __rte_aligned(64); + +#define IDXD_COMP_STATUS_INCOMPLETE 0 +#define IDXD_COMP_STATUS_SUCCESS 1 +#define IDXD_COMP_STATUS_INVALID_OPCODE 0x10 +#define IDXD_COMP_STATUS_INVALID_SIZE 0x13 +#define IDXD_COMP_STATUS_SKIPPED 0xFF /* not official IDXD error, needed as placeholder */ + +/** + * Completion record structure written back by DSA + */ +struct idxd_completion { + uint8_t status; + uint8_t result; + /* 16-bits pad here */ + uint32_t completed_size; /* data length, or descriptors for batch */ + + rte_iova_t fault_address; + uint32_t invalid_flags; +} __rte_aligned(32); + /*** Definitions for Intel(R) Data Streaming Accelerator ***/ #define IDXD_CMD_SHIFT 20 diff --git a/drivers/dma/idxd/idxd_internal.h b/drivers/dma/idxd/idxd_internal.h index cb3a68c69b..99c8e04302 100644 --- a/drivers/dma/idxd/idxd_internal.h +++ b/drivers/dma/idxd/idxd_internal.h @@ -39,6 +39,8 @@ struct idxd_pci_common { }; struct idxd_dmadev { + struct idxd_hw_desc *desc_ring; + /* counters to track the batches */ unsigned short max_batches; unsigned short batch_idx_read; @@ -79,5 +81,6 @@ struct idxd_dmadev { int idxd_dmadev_create(const char *name, struct rte_device *dev, const struct idxd_dmadev *base_idxd, const struct rte_dma_dev_ops *ops); +int idxd_dump(const struct rte_dma_dev *dev, FILE *f); #endif /* _IDXD_INTERNAL_H_ */ diff --git a/drivers/dma/idxd/idxd_pci.c b/drivers/dma/idxd/idxd_pci.c index 171e5ffc07..33cf76adfb 100644 --- a/drivers/dma/idxd/idxd_pci.c +++ b/drivers/dma/idxd/idxd_pci.c @@ -60,7 +60,7 @@ idxd_is_wq_enabled(struct idxd_dmadev *idxd) } static const struct rte_dma_dev_ops idxd_pci_ops = { - + .dev_dump = idxd_dump, }; /* each portal uses 4 x 4k pages */ From patchwork Fri Sep 17 15:24:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99218 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7189CA0C43; Fri, 17 Sep 2021 17:25:36 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5642041169; Fri, 17 Sep 2021 17:25:04 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id 1836541179 for ; Fri, 17 Sep 2021 17:25:00 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222476010" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222476010" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:25:00 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="651909791" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by orsmga005.jf.intel.com with ESMTP; 17 Sep 2021 08:24:58 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 17 Sep 2021 15:24:28 +0000 Message-Id: <20210917152437.3270330-8-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210917152437.3270330-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210917152437.3270330-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 07/16] dma/idxd: add configure and info_get functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add functions for device configuration. The info_get function is included here since it can be useful for checking successful configuration. Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh --- v2: - fix reconfigure bug in idxd_vchan_setup() - add literal include comment for the docs to pick up v3: - fixes needed after changes from rebasing --- app/test/test_dmadev.c | 2 + doc/guides/dmadevs/idxd.rst | 30 +++++++++++++ drivers/dma/idxd/idxd_bus.c | 3 ++ drivers/dma/idxd/idxd_common.c | 72 ++++++++++++++++++++++++++++++++ drivers/dma/idxd/idxd_internal.h | 6 +++ drivers/dma/idxd/idxd_pci.c | 3 ++ 6 files changed, 116 insertions(+) diff --git a/app/test/test_dmadev.c b/app/test/test_dmadev.c index 98fcab67f3..5bbe4250e0 100644 --- a/app/test/test_dmadev.c +++ b/app/test/test_dmadev.c @@ -739,6 +739,7 @@ test_dmadev_instance(uint16_t dev_id) { #define TEST_RINGSIZE 512 #define CHECK_ERRS true + /* Setup of the dmadev device. 8< */ struct rte_dma_stats stats; struct rte_dma_info info; const struct rte_dma_conf conf = { .nb_vchans = 1}; @@ -759,6 +760,7 @@ test_dmadev_instance(uint16_t dev_id) if (rte_dma_vchan_setup(dev_id, vchan, &qconf) < 0) ERR_RETURN("Error with queue configuration\n"); + /* >8 End of setup of the dmadev device. */ rte_dma_info_get(dev_id, &info); if (info.nb_vchans != 1) diff --git a/doc/guides/dmadevs/idxd.rst b/doc/guides/dmadevs/idxd.rst index ce33e2857a..abfa5be9ea 100644 --- a/doc/guides/dmadevs/idxd.rst +++ b/doc/guides/dmadevs/idxd.rst @@ -120,3 +120,33 @@ use a subset of configured queues. Once probed successfully, irrespective of kernel driver, the device will appear as a ``dmadev``, that is a "DMA device type" inside DPDK, and can be accessed using APIs from the ``rte_dmadev`` library. + +Using IDXD DMAdev Devices +-------------------------- + +To use the devices from an application, the dmadev API can be used. + +Getting Device Information +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Basic information about each dmadev device can be queried using the +``rte_dma_info_get()`` API. This will return basic device information such as +the ``rte_device`` structure, device capabilities and other device specific values. + +Device Configuration +~~~~~~~~~~~~~~~~~~~~~ + +Configuring an IDXD dmadev device is done using the ``rte_dma_configure()`` and +``rte_dma_vchan_setup`` APIs. The configurations are passed to these APIs using +the ``rte_dma_conf`` and ``rte_dma_vchan_conf`` structures, respectively. For +example, these can be used to configure the number of ``vchans`` per device, the +ring size, etc. The ring size must be a power of two, between 64 and 4096. + +The following code shows how the device is configured in +``test_dmadev.c``: + +.. literalinclude:: ../../../app/test/test_dmadev.c + :language: c + :start-after: Setup of the dmadev device. 8< + :end-before: >8 End of setup of the dmadev device. + :dedent: 1 diff --git a/drivers/dma/idxd/idxd_bus.c b/drivers/dma/idxd/idxd_bus.c index 3c0837ec52..b2acdac4f9 100644 --- a/drivers/dma/idxd/idxd_bus.c +++ b/drivers/dma/idxd/idxd_bus.c @@ -96,6 +96,9 @@ idxd_dev_close(struct rte_dma_dev *dev) static const struct rte_dma_dev_ops idxd_bus_ops = { .dev_close = idxd_dev_close, .dev_dump = idxd_dump, + .dev_configure = idxd_configure, + .vchan_setup = idxd_vchan_setup, + .dev_info_get = idxd_info_get, }; static void * diff --git a/drivers/dma/idxd/idxd_common.c b/drivers/dma/idxd/idxd_common.c index 45cde78e88..2c222708cf 100644 --- a/drivers/dma/idxd/idxd_common.c +++ b/drivers/dma/idxd/idxd_common.c @@ -39,6 +39,78 @@ idxd_dump(const struct rte_dma_dev *dev, FILE *f) return 0; } +int +idxd_info_get(const struct rte_dma_dev *dev, struct rte_dma_info *info, uint32_t size) +{ + struct idxd_dmadev *idxd = dev->dev_private; + + if (size < sizeof(*info)) + return -EINVAL; + + *info = (struct rte_dma_info) { + .dev_capa = RTE_DMA_CAPA_MEM_TO_MEM | + RTE_DMA_CAPA_OPS_COPY | RTE_DMA_CAPA_OPS_FILL, + .max_vchans = 1, + .max_desc = 4096, + .min_desc = 64, + .nb_vchans = (idxd->desc_ring != NULL), /* returns 1 or 0 */ + }; + if (idxd->sva_support) + info->dev_capa |= RTE_DMA_CAPA_SVA; + return 0; +} + +int +idxd_configure(struct rte_dma_dev *dev __rte_unused, const struct rte_dma_conf *dev_conf, + uint32_t conf_sz) +{ + if (sizeof(struct rte_dma_conf) != conf_sz) + return -EINVAL; + + if (dev_conf->nb_vchans != 1) + return -EINVAL; + return 0; +} + +int +idxd_vchan_setup(struct rte_dma_dev *dev, uint16_t vchan __rte_unused, + const struct rte_dma_vchan_conf *qconf, uint32_t qconf_sz) +{ + struct idxd_dmadev *idxd = dev->dev_private; + uint16_t max_desc = qconf->nb_desc; + + if (sizeof(struct rte_dma_vchan_conf) != qconf_sz) + return -EINVAL; + + idxd->qcfg = *qconf; + + if (!rte_is_power_of_2(max_desc)) + max_desc = rte_align32pow2(max_desc); + IDXD_PMD_DEBUG("DMA dev %u using %u descriptors", dev->data->dev_id, max_desc); + idxd->desc_ring_mask = max_desc - 1; + idxd->qcfg.nb_desc = max_desc; + + /* in case we are reconfiguring a device, free any existing memory */ + rte_free(idxd->desc_ring); + + /* allocate the descriptor ring at 2x size as batches can't wrap */ + idxd->desc_ring = rte_zmalloc(NULL, sizeof(*idxd->desc_ring) * max_desc * 2, 0); + if (idxd->desc_ring == NULL) + return -ENOMEM; + idxd->desc_iova = rte_mem_virt2iova(idxd->desc_ring); + + idxd->batch_idx_read = 0; + idxd->batch_idx_write = 0; + idxd->batch_start = 0; + idxd->batch_size = 0; + idxd->ids_returned = 0; + idxd->ids_avail = 0; + + memset(idxd->batch_comp_ring, 0, sizeof(*idxd->batch_comp_ring) * + (idxd->max_batches + 1)); + return 0; +} + int idxd_dmadev_create(const char *name, struct rte_device *dev, const struct idxd_dmadev *base_idxd, diff --git a/drivers/dma/idxd/idxd_internal.h b/drivers/dma/idxd/idxd_internal.h index 99c8e04302..fdd018ca35 100644 --- a/drivers/dma/idxd/idxd_internal.h +++ b/drivers/dma/idxd/idxd_internal.h @@ -82,5 +82,11 @@ struct idxd_dmadev { int idxd_dmadev_create(const char *name, struct rte_device *dev, const struct idxd_dmadev *base_idxd, const struct rte_dma_dev_ops *ops); int idxd_dump(const struct rte_dma_dev *dev, FILE *f); +int idxd_configure(struct rte_dma_dev *dev, const struct rte_dma_conf *dev_conf, + uint32_t conf_sz); +int idxd_vchan_setup(struct rte_dma_dev *dev, uint16_t vchan, + const struct rte_dma_vchan_conf *qconf, uint32_t qconf_sz); +int idxd_info_get(const struct rte_dma_dev *dev, struct rte_dma_info *dev_info, + uint32_t size); #endif /* _IDXD_INTERNAL_H_ */ diff --git a/drivers/dma/idxd/idxd_pci.c b/drivers/dma/idxd/idxd_pci.c index 33cf76adfb..0216ab80d9 100644 --- a/drivers/dma/idxd/idxd_pci.c +++ b/drivers/dma/idxd/idxd_pci.c @@ -61,6 +61,9 @@ idxd_is_wq_enabled(struct idxd_dmadev *idxd) static const struct rte_dma_dev_ops idxd_pci_ops = { .dev_dump = idxd_dump, + .dev_configure = idxd_configure, + .vchan_setup = idxd_vchan_setup, + .dev_info_get = idxd_info_get, }; /* each portal uses 4 x 4k pages */ From patchwork Fri Sep 17 15:24:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99219 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D8E38A0C43; Fri, 17 Sep 2021 17:25:41 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5B99A41189; Fri, 17 Sep 2021 17:25:05 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id CB40A4114A for ; Fri, 17 Sep 2021 17:25:02 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222476016" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222476016" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:25:02 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="651909819" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by orsmga005.jf.intel.com with ESMTP; 17 Sep 2021 08:25:00 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 17 Sep 2021 15:24:29 +0000 Message-Id: <20210917152437.3270330-9-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210917152437.3270330-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210917152437.3270330-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 08/16] dma/idxd: add start and stop functions for pci devices X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add device start/stop functions for DSA devices bound to vfio. For devices bound to the IDXD kernel driver, these are not required since the IDXD kernel driver takes care of this. Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh --- doc/guides/dmadevs/idxd.rst | 3 +++ drivers/dma/idxd/idxd_pci.c | 52 +++++++++++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+) diff --git a/doc/guides/dmadevs/idxd.rst b/doc/guides/dmadevs/idxd.rst index abfa5be9ea..a603c5dd22 100644 --- a/doc/guides/dmadevs/idxd.rst +++ b/doc/guides/dmadevs/idxd.rst @@ -150,3 +150,6 @@ The following code shows how the device is configured in :start-after: Setup of the dmadev device. 8< :end-before: >8 End of setup of the dmadev device. :dedent: 1 + +Once configured, the device can then be made ready for use by calling the +``rte_dma_start()`` API. diff --git a/drivers/dma/idxd/idxd_pci.c b/drivers/dma/idxd/idxd_pci.c index 0216ab80d9..cfb64ce220 100644 --- a/drivers/dma/idxd/idxd_pci.c +++ b/drivers/dma/idxd/idxd_pci.c @@ -59,11 +59,63 @@ idxd_is_wq_enabled(struct idxd_dmadev *idxd) return ((state >> WQ_STATE_SHIFT) & WQ_STATE_MASK) == 0x1; } +static int +idxd_pci_dev_stop(struct rte_dma_dev *dev) +{ + struct idxd_dmadev *idxd = dev->dev_private; + uint8_t err_code; + + if (!idxd_is_wq_enabled(idxd)) { + IDXD_PMD_ERR("Work queue %d already disabled", idxd->qid); + return -EALREADY; + } + + err_code = idxd_pci_dev_command(idxd, idxd_disable_wq); + if (err_code || idxd_is_wq_enabled(idxd)) { + IDXD_PMD_ERR("Failed disabling work queue %d, error code: %#x", + idxd->qid, err_code); + return -err_code; + } + IDXD_PMD_DEBUG("Work queue %d disabled OK", idxd->qid); + + return 0; +} + +static int +idxd_pci_dev_start(struct rte_dma_dev *dev) +{ + struct idxd_dmadev *idxd = dev->dev_private; + uint8_t err_code; + + if (idxd_is_wq_enabled(idxd)) { + IDXD_PMD_WARN("WQ %d already enabled", idxd->qid); + return 0; + } + + if (idxd->desc_ring == NULL) { + IDXD_PMD_ERR("WQ %d has not been fully configured", idxd->qid); + return -EINVAL; + } + + err_code = idxd_pci_dev_command(idxd, idxd_enable_wq); + if (err_code || !idxd_is_wq_enabled(idxd)) { + IDXD_PMD_ERR("Failed enabling work queue %d, error code: %#x", + idxd->qid, err_code); + return err_code == 0 ? -1 : err_code; + } + + IDXD_PMD_DEBUG("Work queue %d enabled OK", idxd->qid); + + return 0; +} + static const struct rte_dma_dev_ops idxd_pci_ops = { .dev_dump = idxd_dump, .dev_configure = idxd_configure, .vchan_setup = idxd_vchan_setup, .dev_info_get = idxd_info_get, + .dev_start = idxd_pci_dev_start, + .dev_stop = idxd_pci_dev_stop, }; /* each portal uses 4 x 4k pages */ From patchwork Fri Sep 17 15:24:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99220 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 87384A0C43; Fri, 17 Sep 2021 17:25:47 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5F3154118E; Fri, 17 Sep 2021 17:25:07 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id AFF9741143 for ; Fri, 17 Sep 2021 17:25:04 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222476020" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222476020" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:25:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="651909828" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by orsmga005.jf.intel.com with ESMTP; 17 Sep 2021 08:25:02 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 17 Sep 2021 15:24:30 +0000 Message-Id: <20210917152437.3270330-10-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210917152437.3270330-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210917152437.3270330-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 09/16] dma/idxd: add data-path job submission functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add data path functions for enqueuing and submitting operations to DSA devices. Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh --- doc/guides/dmadevs/idxd.rst | 64 +++++++++++++++ drivers/dma/idxd/idxd_common.c | 136 +++++++++++++++++++++++++++++++ drivers/dma/idxd/idxd_internal.h | 5 ++ drivers/dma/idxd/meson.build | 1 + 4 files changed, 206 insertions(+) diff --git a/doc/guides/dmadevs/idxd.rst b/doc/guides/dmadevs/idxd.rst index a603c5dd22..7835461a22 100644 --- a/doc/guides/dmadevs/idxd.rst +++ b/doc/guides/dmadevs/idxd.rst @@ -153,3 +153,67 @@ The following code shows how the device is configured in Once configured, the device can then be made ready for use by calling the ``rte_dma_start()`` API. + +Performing Data Copies +~~~~~~~~~~~~~~~~~~~~~~~ + +To perform data copies using IDXD dmadev devices, descriptors should be enqueued +using the ``rte_dma_copy()`` API. The HW can be triggered to perform the copy +in two ways, either via a ``RTE_DMA_OP_FLAG_SUBMIT`` flag or by calling +``rte_dma_submit()``. Once copies have been completed, the completion will +be reported back when the application calls ``rte_dma_completed()`` or +``rte_dma_completed_status()``. The latter will also report the status of each +completed operation. + +The ``rte_dma_copy()`` function enqueues a single copy to the device ring for +copying at a later point. The parameters to that function include the IOVA addresses +of both the source and destination buffers, as well as the length of the copy. + +The ``rte_dma_copy()`` function enqueues a copy operation on the device ring. +If the ``RTE_DMA_OP_FLAG_SUBMIT`` flag is set when calling ``rte_dma_copy()``, +the device hardware will be informed of the elements. Alternatively, if the flag +is not set, the application needs to call the ``rte_dma_submit()`` function to +notify the device hardware. Once the device hardware is informed of the elements +enqueued on the ring, the device will begin to process them. It is expected +that, for efficiency reasons, a burst of operations will be enqueued to the +device via multiple enqueue calls between calls to the ``rte_dma_submit()`` +function. + +The following code demonstrates how to enqueue a burst of copies to the +device and start the hardware processing of them: + +.. code-block:: C + + struct rte_mbuf *srcs[COMP_BURST_SZ], *dsts[COMP_BURST_SZ]; + unsigned int i; + + for (i = 0; i < RTE_DIM(srcs); i++) { + uint64_t *src_data; + + srcs[i] = rte_pktmbuf_alloc(pool); + dsts[i] = rte_pktmbuf_alloc(pool); + src_data = rte_pktmbuf_mtod(srcs[i], uint64_t *); + if (srcs[i] == NULL || dsts[i] == NULL) { + PRINT_ERR("Error allocating buffers\n"); + return -1; + } + + for (j = 0; j < COPY_LEN/sizeof(uint64_t); j++) + src_data[j] = rte_rand(); + + if (rte_dma_copy(dev_id, vchan, srcs[i]->buf_iova + srcs[i]->data_off, + dsts[i]->buf_iova + dsts[i]->data_off, COPY_LEN, 0) < 0) { + PRINT_ERR("Error with rte_dma_copy for buffer %u\n", i); + return -1; + } + } + rte_dma_submit(dev_id, vchan); + +Filling an Area of Memory +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The IDXD driver also has support for the ``fill`` operation, where an area +of memory is overwritten, or filled, with a short pattern of data. +Fill operations can be performed in much the same was as copy operations +described above, just using the ``rte_dma_fill()`` function rather than the +``rte_dma_copy()`` function. diff --git a/drivers/dma/idxd/idxd_common.c b/drivers/dma/idxd/idxd_common.c index 2c222708cf..b01edeab07 100644 --- a/drivers/dma/idxd/idxd_common.c +++ b/drivers/dma/idxd/idxd_common.c @@ -2,14 +2,144 @@ * Copyright 2021 Intel Corporation */ +#include + #include #include #include +#include #include "idxd_internal.h" #define IDXD_PMD_NAME_STR "dmadev_idxd" +static __rte_always_inline rte_iova_t +__desc_idx_to_iova(struct idxd_dmadev *idxd, uint16_t n) +{ + return idxd->desc_iova + (n * sizeof(struct idxd_hw_desc)); +} + +static __rte_always_inline void +__idxd_movdir64b(volatile void *dst, const struct idxd_hw_desc *src) +{ + asm volatile (".byte 0x66, 0x0f, 0x38, 0xf8, 0x02" + : + : "a" (dst), "d" (src) + : "memory"); +} + +static __rte_always_inline void +__submit(struct idxd_dmadev *idxd) +{ + rte_prefetch1(&idxd->batch_comp_ring[idxd->batch_idx_read]); + + if (idxd->batch_size == 0) + return; + + /* write completion to batch comp ring */ + rte_iova_t comp_addr = idxd->batch_iova + + (idxd->batch_idx_write * sizeof(struct idxd_completion)); + + if (idxd->batch_size == 1) { + /* submit batch directly */ + struct idxd_hw_desc desc = + idxd->desc_ring[idxd->batch_start & idxd->desc_ring_mask]; + desc.completion = comp_addr; + desc.op_flags |= IDXD_FLAG_REQUEST_COMPLETION; + _mm_sfence(); /* fence before writing desc to device */ + __idxd_movdir64b(idxd->portal, &desc); + } else { + const struct idxd_hw_desc batch_desc = { + .op_flags = (idxd_op_batch << IDXD_CMD_OP_SHIFT) | + IDXD_FLAG_COMPLETION_ADDR_VALID | + IDXD_FLAG_REQUEST_COMPLETION, + .desc_addr = __desc_idx_to_iova(idxd, + idxd->batch_start & idxd->desc_ring_mask), + .completion = comp_addr, + .size = idxd->batch_size, + }; + _mm_sfence(); /* fence before writing desc to device */ + __idxd_movdir64b(idxd->portal, &batch_desc); + } + + if (++idxd->batch_idx_write > idxd->max_batches) + idxd->batch_idx_write = 0; + + idxd->batch_start += idxd->batch_size; + idxd->batch_size = 0; + idxd->batch_idx_ring[idxd->batch_idx_write] = idxd->batch_start; + _mm256_store_si256((void *)&idxd->batch_comp_ring[idxd->batch_idx_write], + _mm256_setzero_si256()); +} + +static __rte_always_inline int +__idxd_write_desc(struct rte_dma_dev *dev, + const uint32_t op_flags, + const rte_iova_t src, + const rte_iova_t dst, + const uint32_t size, + const uint32_t flags) +{ + struct idxd_dmadev *idxd = dev->dev_private; + uint16_t mask = idxd->desc_ring_mask; + uint16_t job_id = idxd->batch_start + idxd->batch_size; + /* we never wrap batches, so we only mask the start and allow start+size to overflow */ + uint16_t write_idx = (idxd->batch_start & mask) + idxd->batch_size; + + /* first check batch ring space then desc ring space */ + if ((idxd->batch_idx_read == 0 && idxd->batch_idx_write == idxd->max_batches) || + idxd->batch_idx_write + 1 == idxd->batch_idx_read) + return -1; + if (((write_idx + 1) & mask) == (idxd->ids_returned & mask)) + return -1; + + /* write desc. Note: descriptors don't wrap, but the completion address does */ + const uint64_t op_flags64 = (uint64_t)(op_flags | IDXD_FLAG_COMPLETION_ADDR_VALID) << 32; + const uint64_t comp_addr = __desc_idx_to_iova(idxd, write_idx & mask); + _mm256_store_si256((void *)&idxd->desc_ring[write_idx], + _mm256_set_epi64x(dst, src, comp_addr, op_flags64)); + _mm256_store_si256((void *)&idxd->desc_ring[write_idx].size, + _mm256_set_epi64x(0, 0, 0, size)); + + idxd->batch_size++; + + rte_prefetch0_write(&idxd->desc_ring[write_idx + 1]); + + if (flags & RTE_DMA_OP_FLAG_SUBMIT) + __submit(idxd); + + return job_id; +} + +int +idxd_enqueue_copy(struct rte_dma_dev *dev, uint16_t qid __rte_unused, rte_iova_t src, + rte_iova_t dst, unsigned int length, uint64_t flags) +{ + /* we can take advantage of the fact that the fence flag in dmadev and DSA are the same, + * but check it at compile time to be sure. + */ + RTE_BUILD_BUG_ON(RTE_DMA_OP_FLAG_FENCE != IDXD_FLAG_FENCE); + uint32_t memmove = (idxd_op_memmove << IDXD_CMD_OP_SHIFT) | + IDXD_FLAG_CACHE_CONTROL | (flags & IDXD_FLAG_FENCE); + return __idxd_write_desc(dev, memmove, src, dst, length, flags); +} + +int +idxd_enqueue_fill(struct rte_dma_dev *dev, uint16_t qid __rte_unused, uint64_t pattern, + rte_iova_t dst, unsigned int length, uint64_t flags) +{ + uint32_t fill = (idxd_op_fill << IDXD_CMD_OP_SHIFT) | + IDXD_FLAG_CACHE_CONTROL | (flags & IDXD_FLAG_FENCE); + return __idxd_write_desc(dev, fill, pattern, dst, length, flags); +} + +int +idxd_submit(struct rte_dma_dev *dev, uint16_t qid __rte_unused) +{ + __submit(dev->dev_private); + return 0; +} + int idxd_dump(const struct rte_dma_dev *dev, FILE *f) { @@ -141,6 +271,12 @@ idxd_dmadev_create(const char *name, struct rte_device *dev, dmadev->dev_ops = ops; dmadev->device = dev; + dmadev->copy = idxd_enqueue_copy; + dmadev->fill = idxd_enqueue_fill; + dmadev->submit = idxd_submit; + dmadev->completed = idxd_completed; + dmadev->completed_status = idxd_completed_status; + idxd = rte_malloc_socket(NULL, sizeof(struct idxd_dmadev), 0, dev->numa_node); if (idxd == NULL) { IDXD_PMD_ERR("Unable to allocate memory for device"); diff --git a/drivers/dma/idxd/idxd_internal.h b/drivers/dma/idxd/idxd_internal.h index fdd018ca35..b66c2d0182 100644 --- a/drivers/dma/idxd/idxd_internal.h +++ b/drivers/dma/idxd/idxd_internal.h @@ -88,5 +88,10 @@ int idxd_vchan_setup(struct rte_dma_dev *dev, uint16_t vchan, const struct rte_dma_vchan_conf *qconf, uint32_t qconf_sz); int idxd_info_get(const struct rte_dma_dev *dev, struct rte_dma_info *dev_info, uint32_t size); +int idxd_enqueue_copy(struct rte_dma_dev *dev, uint16_t qid, rte_iova_t src, + rte_iova_t dst, unsigned int length, uint64_t flags); +int idxd_enqueue_fill(struct rte_dma_dev *dev, uint16_t qid, uint64_t pattern, + rte_iova_t dst, unsigned int length, uint64_t flags); +int idxd_submit(struct rte_dma_dev *dev, uint16_t qid); #endif /* _IDXD_INTERNAL_H_ */ diff --git a/drivers/dma/idxd/meson.build b/drivers/dma/idxd/meson.build index 36dbd3e518..acb1b10618 100644 --- a/drivers/dma/idxd/meson.build +++ b/drivers/dma/idxd/meson.build @@ -6,6 +6,7 @@ if is_windows endif deps += ['bus_pci'] +cflags += '-mavx2' # all platforms with idxd HW support AVX sources = files( 'idxd_bus.c', 'idxd_common.c', From patchwork Fri Sep 17 15:24:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99221 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A14E2A0C46; Fri, 17 Sep 2021 17:25:54 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B489741192; Fri, 17 Sep 2021 17:25:09 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id 9EF5F4118D for ; Fri, 17 Sep 2021 17:25:06 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222476026" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222476026" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:25:06 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="651909836" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by orsmga005.jf.intel.com with ESMTP; 17 Sep 2021 08:25:04 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 17 Sep 2021 15:24:31 +0000 Message-Id: <20210917152437.3270330-11-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210917152437.3270330-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210917152437.3270330-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 10/16] dma/idxd: add data-path job completion functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the data path functions for gathering completed operations. Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh --- v2: - fixed typo in docs - add completion status for invalid opcode --- doc/guides/dmadevs/idxd.rst | 32 +++++ drivers/dma/idxd/idxd_common.c | 235 +++++++++++++++++++++++++++++++ drivers/dma/idxd/idxd_internal.h | 5 + 3 files changed, 272 insertions(+) diff --git a/doc/guides/dmadevs/idxd.rst b/doc/guides/dmadevs/idxd.rst index 7835461a22..f942a8aa44 100644 --- a/doc/guides/dmadevs/idxd.rst +++ b/doc/guides/dmadevs/idxd.rst @@ -209,6 +209,38 @@ device and start the hardware processing of them: } rte_dma_submit(dev_id, vchan); +To retrieve information about completed copies, ``rte_dma_completed()`` and +``rte_dma_completed_status()`` APIs should be used. ``rte_dma_completed()`` +will return the number of completed operations, along with the index of the last +successful completed operation and whether or not an error was encountered. If an +error was encountered, ``rte_dma_completed_status()`` must be used to kick the +device off to continue processing operations and also to gather the status of each +individual operations which is filled in to the ``status`` array provided as +parameter by the application. + +The following status codes are supported by IDXD: +* ``RTE_DMA_STATUS_SUCCESSFUL``: The operation was successful. +* ``RTE_DMA_STATUS_INVALID_OPCODE``: The operation failed due to an invalid operation code. +* ``RTE_DMA_STATUS_INVALID_LENGTH``: The operation failed due to an invalid data length. +* ``RTE_DMA_STATUS_NOT_ATTEMPTED``: The operation was not attempted. +* ``RTE_DMA_STATUS_ERROR_UNKNOWN``: The operation failed due to an unspecified error. + +The following code shows how to retrieve the number of successfully completed +copies within a burst and then using ``rte_dma_completed_status()`` to check +which operation failed and kick off the device to continue processing operations: + +.. code-block:: C + + enum rte_dma_status_code status[COMP_BURST_SZ]; + uint16_t count, idx, status_count; + bool error = 0; + + count = rte_dma_completed(dev_id, vchan, COMP_BURST_SZ, &idx, &error); + + if (error){ + status_count = rte_dma_completed_status(dev_id, vchan, COMP_BURST_SZ, &idx, status); + } + Filling an Area of Memory ~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/drivers/dma/idxd/idxd_common.c b/drivers/dma/idxd/idxd_common.c index b01edeab07..a061a956c2 100644 --- a/drivers/dma/idxd/idxd_common.c +++ b/drivers/dma/idxd/idxd_common.c @@ -140,6 +140,241 @@ idxd_submit(struct rte_dma_dev *dev, uint16_t qid __rte_unused) return 0; } +static enum rte_dma_status_code +get_comp_status(struct idxd_completion *c) +{ + uint8_t st = c->status; + switch (st) { + /* successful descriptors are not written back normally */ + case IDXD_COMP_STATUS_INCOMPLETE: + case IDXD_COMP_STATUS_SUCCESS: + return RTE_DMA_STATUS_SUCCESSFUL; + case IDXD_COMP_STATUS_INVALID_OPCODE: + return RTE_DMA_STATUS_INVALID_OPCODE; + case IDXD_COMP_STATUS_INVALID_SIZE: + return RTE_DMA_STATUS_INVALID_LENGTH; + case IDXD_COMP_STATUS_SKIPPED: + return RTE_DMA_STATUS_NOT_ATTEMPTED; + default: + return RTE_DMA_STATUS_ERROR_UNKNOWN; + } +} + +static __rte_always_inline int +batch_ok(struct idxd_dmadev *idxd, uint8_t max_ops, enum rte_dma_status_code *status) +{ + uint16_t ret; + uint8_t bstatus; + + if (max_ops == 0) + return 0; + + /* first check if there are any unreturned handles from last time */ + if (idxd->ids_avail != idxd->ids_returned) { + ret = RTE_MIN((uint16_t)(idxd->ids_avail - idxd->ids_returned), max_ops); + idxd->ids_returned += ret; + if (status) + memset(status, RTE_DMA_STATUS_SUCCESSFUL, ret * sizeof(*status)); + return ret; + } + + if (idxd->batch_idx_read == idxd->batch_idx_write) + return 0; + + bstatus = idxd->batch_comp_ring[idxd->batch_idx_read].status; + /* now check if next batch is complete and successful */ + if (bstatus == IDXD_COMP_STATUS_SUCCESS) { + /* since the batch idx ring stores the start of each batch, pre-increment to lookup + * start of next batch. + */ + if (++idxd->batch_idx_read > idxd->max_batches) + idxd->batch_idx_read = 0; + idxd->ids_avail = idxd->batch_idx_ring[idxd->batch_idx_read]; + + ret = RTE_MIN((uint16_t)(idxd->ids_avail - idxd->ids_returned), max_ops); + idxd->ids_returned += ret; + if (status) + memset(status, RTE_DMA_STATUS_SUCCESSFUL, ret * sizeof(*status)); + return ret; + } + /* check if batch is incomplete */ + else if (bstatus == IDXD_COMP_STATUS_INCOMPLETE) + return 0; + + return -1; /* error case */ +} + +static inline uint16_t +batch_completed(struct idxd_dmadev *idxd, uint8_t max_ops, bool *has_error) +{ + uint16_t i; + uint16_t b_start, b_end, next_batch; + + int ret = batch_ok(idxd, max_ops, NULL); + if (ret >= 0) + return ret; + + /* ERROR case, not successful, not incomplete */ + /* Get the batch size, and special case size 1. + * once we identify the actual failure job, return other jobs, then update + * the batch ring indexes to make it look like the first job of the batch has failed. + * Subsequent calls here will always return zero packets, and the error must be cleared by + * calling the completed_status() function. + */ + next_batch = (idxd->batch_idx_read + 1); + if (next_batch > idxd->max_batches) + next_batch = 0; + b_start = idxd->batch_idx_ring[idxd->batch_idx_read]; + b_end = idxd->batch_idx_ring[next_batch]; + + if (b_end - b_start == 1) { /* not a batch */ + *has_error = true; + return 0; + } + + for (i = b_start; i < b_end; i++) { + struct idxd_completion *c = (void *)&idxd->desc_ring[i & idxd->desc_ring_mask]; + if (c->status > IDXD_COMP_STATUS_SUCCESS) /* ignore incomplete(0) and success(1) */ + break; + } + ret = RTE_MIN((uint16_t)(i - idxd->ids_returned), max_ops); + if (ret < max_ops) + *has_error = true; /* we got up to the point of error */ + idxd->ids_avail = idxd->ids_returned += ret; + + /* to ensure we can call twice and just return 0, set start of batch to where we finished */ + idxd->batch_comp_ring[idxd->batch_idx_read].completed_size -= ret; + idxd->batch_idx_ring[idxd->batch_idx_read] += ret; + if (idxd->batch_idx_ring[next_batch] - idxd->batch_idx_ring[idxd->batch_idx_read] == 1) { + /* copy over the descriptor status to the batch ring as if no batch */ + uint16_t d_idx = idxd->batch_idx_ring[idxd->batch_idx_read] & idxd->desc_ring_mask; + struct idxd_completion *desc_comp = (void *)&idxd->desc_ring[d_idx]; + idxd->batch_comp_ring[idxd->batch_idx_read].status = desc_comp->status; + } + + return ret; +} + +static uint16_t +batch_completed_status(struct idxd_dmadev *idxd, uint16_t max_ops, enum rte_dma_status_code *status) +{ + uint16_t next_batch; + + int ret = batch_ok(idxd, max_ops, status); + if (ret >= 0) + return ret; + + /* ERROR case, not successful, not incomplete */ + /* Get the batch size, and special case size 1. + */ + next_batch = (idxd->batch_idx_read + 1); + if (next_batch > idxd->max_batches) + next_batch = 0; + const uint16_t b_start = idxd->batch_idx_ring[idxd->batch_idx_read]; + const uint16_t b_end = idxd->batch_idx_ring[next_batch]; + const uint16_t b_len = b_end - b_start; + if (b_len == 1) {/* not a batch */ + *status = get_comp_status(&idxd->batch_comp_ring[idxd->batch_idx_read]); + idxd->ids_avail++; + idxd->ids_returned++; + idxd->batch_idx_read = next_batch; + return 1; + } + + /* not a single-element batch, need to process more. + * Scenarios: + * 1. max_ops >= batch_size - can fit everything, simple case + * - loop through completed ops and then add on any not-attempted ones + * 2. max_ops < batch_size - can't fit everything, more complex case + * - loop through completed/incomplete and stop when hit max_ops + * - adjust the batch descriptor to update where we stopped, with appropriate bcount + * - if bcount is to be exactly 1, update the batch descriptor as it will be treated as + * non-batch next time. + */ + const uint16_t bcount = idxd->batch_comp_ring[idxd->batch_idx_read].completed_size; + for (ret = 0; ret < b_len && ret < max_ops; ret++) { + struct idxd_completion *c = (void *) + &idxd->desc_ring[(b_start + ret) & idxd->desc_ring_mask]; + status[ret] = (ret < bcount) ? get_comp_status(c) : RTE_DMA_STATUS_NOT_ATTEMPTED; + } + idxd->ids_avail = idxd->ids_returned += ret; + + /* everything fit */ + if (ret == b_len) { + idxd->batch_idx_read = next_batch; + return ret; + } + + /* set up for next time, update existing batch descriptor & start idx at batch_idx_read */ + idxd->batch_idx_ring[idxd->batch_idx_read] += ret; + if (ret > bcount) { + /* we have only incomplete ones - set batch completed size to 0 */ + struct idxd_completion *comp = &idxd->batch_comp_ring[idxd->batch_idx_read]; + comp->completed_size = 0; + /* if there is only one descriptor left, job skipped so set flag appropriately */ + if (b_len - ret == 1) + comp->status = IDXD_COMP_STATUS_SKIPPED; + } else { + struct idxd_completion *comp = &idxd->batch_comp_ring[idxd->batch_idx_read]; + comp->completed_size -= ret; + /* if there is only one descriptor left, copy status info straight to desc */ + if (comp->completed_size == 1) { + struct idxd_completion *c = (void *) + &idxd->desc_ring[(b_start + ret) & idxd->desc_ring_mask]; + comp->status = c->status; + /* individual descs can be ok without writeback, but not batches */ + if (comp->status == IDXD_COMP_STATUS_INCOMPLETE) + comp->status = IDXD_COMP_STATUS_SUCCESS; + } else if (bcount == b_len) { + /* check if we still have an error, and clear flag if not */ + uint16_t i; + for (i = b_start + ret; i < b_end; i++) { + struct idxd_completion *c = (void *) + &idxd->desc_ring[i & idxd->desc_ring_mask]; + if (c->status > IDXD_COMP_STATUS_SUCCESS) + break; + } + if (i == b_end) /* no errors */ + comp->status = IDXD_COMP_STATUS_SUCCESS; + } + } + + return ret; +} + +uint16_t +idxd_completed(struct rte_dma_dev *dev, uint16_t qid __rte_unused, uint16_t max_ops, + uint16_t *last_idx, bool *has_error) +{ + struct idxd_dmadev *idxd = dev->dev_private; + uint16_t batch, ret = 0; + + do { + batch = batch_completed(idxd, max_ops - ret, has_error); + ret += batch; + } while (batch > 0 && *has_error == false); + + *last_idx = idxd->ids_returned - 1; + return ret; +} + +uint16_t +idxd_completed_status(struct rte_dma_dev *dev, uint16_t qid __rte_unused, uint16_t max_ops, + uint16_t *last_idx, enum rte_dma_status_code *status) +{ + struct idxd_dmadev *idxd = dev->dev_private; + + uint16_t batch, ret = 0; + + do { + batch = batch_completed_status(idxd, max_ops - ret, &status[ret]); + ret += batch; + } while (batch > 0); + + *last_idx = idxd->ids_returned - 1; + return ret; +} + int idxd_dump(const struct rte_dma_dev *dev, FILE *f) { diff --git a/drivers/dma/idxd/idxd_internal.h b/drivers/dma/idxd/idxd_internal.h index b66c2d0182..15115a0966 100644 --- a/drivers/dma/idxd/idxd_internal.h +++ b/drivers/dma/idxd/idxd_internal.h @@ -93,5 +93,10 @@ int idxd_enqueue_copy(struct rte_dma_dev *dev, uint16_t qid, rte_iova_t src, int idxd_enqueue_fill(struct rte_dma_dev *dev, uint16_t qid, uint64_t pattern, rte_iova_t dst, unsigned int length, uint64_t flags); int idxd_submit(struct rte_dma_dev *dev, uint16_t qid); +uint16_t idxd_completed(struct rte_dma_dev *dev, uint16_t qid, uint16_t max_ops, + uint16_t *last_idx, bool *has_error); +uint16_t idxd_completed_status(struct rte_dma_dev *dev, uint16_t qid __rte_unused, + uint16_t max_ops, uint16_t *last_idx, + enum rte_dma_status_code *status); #endif /* _IDXD_INTERNAL_H_ */ From patchwork Fri Sep 17 15:24:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99222 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1D3B1A0C43; Fri, 17 Sep 2021 17:26:00 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B6624411A2; Fri, 17 Sep 2021 17:25:10 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id 5BF8741195 for ; Fri, 17 Sep 2021 17:25:08 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222476034" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222476034" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:25:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="651909842" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by orsmga005.jf.intel.com with ESMTP; 17 Sep 2021 08:25:06 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 17 Sep 2021 15:24:32 +0000 Message-Id: <20210917152437.3270330-12-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210917152437.3270330-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210917152437.3270330-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 11/16] dma/idxd: add operation statistic tracking X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add statistic tracking for DSA devices. Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh Reviewed-by: Chengwen Feng --- doc/guides/dmadevs/idxd.rst | 11 +++++++++++ drivers/dma/idxd/idxd_bus.c | 2 ++ drivers/dma/idxd/idxd_common.c | 27 +++++++++++++++++++++++++++ drivers/dma/idxd/idxd_internal.h | 3 +++ drivers/dma/idxd/idxd_pci.c | 2 ++ 5 files changed, 45 insertions(+) diff --git a/doc/guides/dmadevs/idxd.rst b/doc/guides/dmadevs/idxd.rst index f942a8aa44..c81f1d15cc 100644 --- a/doc/guides/dmadevs/idxd.rst +++ b/doc/guides/dmadevs/idxd.rst @@ -249,3 +249,14 @@ of memory is overwritten, or filled, with a short pattern of data. Fill operations can be performed in much the same was as copy operations described above, just using the ``rte_dma_fill()`` function rather than the ``rte_dma_copy()`` function. + +Querying Device Statistics +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The statistics from the IDXD dmadev device can be got via the stats functions in +the ``rte_dmadev`` library, i.e. ``rte_dma_stats_get()``. The statistics +returned for each device instance are: + +* ``submitted``: The number of operations submitted to the device. +* ``completed``: The number of operations which have completed (successful and failed). +* ``errors``: The number of operations that completed with error. diff --git a/drivers/dma/idxd/idxd_bus.c b/drivers/dma/idxd/idxd_bus.c index b2acdac4f9..b52ea02854 100644 --- a/drivers/dma/idxd/idxd_bus.c +++ b/drivers/dma/idxd/idxd_bus.c @@ -99,6 +99,8 @@ static const struct rte_dma_dev_ops idxd_bus_ops = { .dev_configure = idxd_configure, .vchan_setup = idxd_vchan_setup, .dev_info_get = idxd_info_get, + .stats_get = idxd_stats_get, + .stats_reset = idxd_stats_reset, }; static void * diff --git a/drivers/dma/idxd/idxd_common.c b/drivers/dma/idxd/idxd_common.c index a061a956c2..d86c58c12a 100644 --- a/drivers/dma/idxd/idxd_common.c +++ b/drivers/dma/idxd/idxd_common.c @@ -65,6 +65,8 @@ __submit(struct idxd_dmadev *idxd) if (++idxd->batch_idx_write > idxd->max_batches) idxd->batch_idx_write = 0; + idxd->stats.submitted += idxd->batch_size; + idxd->batch_start += idxd->batch_size; idxd->batch_size = 0; idxd->batch_idx_ring[idxd->batch_idx_write] = idxd->batch_start; @@ -275,6 +277,8 @@ batch_completed_status(struct idxd_dmadev *idxd, uint16_t max_ops, enum rte_dma_ const uint16_t b_len = b_end - b_start; if (b_len == 1) {/* not a batch */ *status = get_comp_status(&idxd->batch_comp_ring[idxd->batch_idx_read]); + if (status != RTE_DMA_STATUS_SUCCESSFUL) + idxd->stats.errors++; idxd->ids_avail++; idxd->ids_returned++; idxd->batch_idx_read = next_batch; @@ -296,6 +300,8 @@ batch_completed_status(struct idxd_dmadev *idxd, uint16_t max_ops, enum rte_dma_ struct idxd_completion *c = (void *) &idxd->desc_ring[(b_start + ret) & idxd->desc_ring_mask]; status[ret] = (ret < bcount) ? get_comp_status(c) : RTE_DMA_STATUS_NOT_ATTEMPTED; + if (status[ret] != RTE_DMA_STATUS_SUCCESSFUL) + idxd->stats.errors++; } idxd->ids_avail = idxd->ids_returned += ret; @@ -354,6 +360,7 @@ idxd_completed(struct rte_dma_dev *dev, uint16_t qid __rte_unused, uint16_t max_ ret += batch; } while (batch > 0 && *has_error == false); + idxd->stats.completed += ret; *last_idx = idxd->ids_returned - 1; return ret; } @@ -371,6 +378,7 @@ idxd_completed_status(struct rte_dma_dev *dev, uint16_t qid __rte_unused, uint16 ret += batch; } while (batch > 0); + idxd->stats.completed += ret; *last_idx = idxd->ids_returned - 1; return ret; } @@ -404,6 +412,25 @@ idxd_dump(const struct rte_dma_dev *dev, FILE *f) return 0; } +int +idxd_stats_get(const struct rte_dma_dev *dev, uint16_t vchan __rte_unused, + struct rte_dma_stats *stats, uint32_t stats_sz) +{ + struct idxd_dmadev *idxd = dev->dev_private; + if (stats_sz < sizeof(*stats)) + return -EINVAL; + *stats = idxd->stats; + return 0; +} + +int +idxd_stats_reset(struct rte_dma_dev *dev, uint16_t vchan __rte_unused) +{ + struct idxd_dmadev *idxd = dev->dev_private; + idxd->stats = (struct rte_dma_stats){0}; + return 0; +} + int idxd_info_get(const struct rte_dma_dev *dev, struct rte_dma_info *info, uint32_t size) { diff --git a/drivers/dma/idxd/idxd_internal.h b/drivers/dma/idxd/idxd_internal.h index 15115a0966..e2a1119ef7 100644 --- a/drivers/dma/idxd/idxd_internal.h +++ b/drivers/dma/idxd/idxd_internal.h @@ -98,5 +98,8 @@ uint16_t idxd_completed(struct rte_dma_dev *dev, uint16_t qid, uint16_t max_ops, uint16_t idxd_completed_status(struct rte_dma_dev *dev, uint16_t qid __rte_unused, uint16_t max_ops, uint16_t *last_idx, enum rte_dma_status_code *status); +int idxd_stats_get(const struct rte_dma_dev *dev, uint16_t vchan, + struct rte_dma_stats *stats, uint32_t stats_sz); +int idxd_stats_reset(struct rte_dma_dev *dev, uint16_t vchan); #endif /* _IDXD_INTERNAL_H_ */ diff --git a/drivers/dma/idxd/idxd_pci.c b/drivers/dma/idxd/idxd_pci.c index cfb64ce220..d73845aa3d 100644 --- a/drivers/dma/idxd/idxd_pci.c +++ b/drivers/dma/idxd/idxd_pci.c @@ -114,6 +114,8 @@ static const struct rte_dma_dev_ops idxd_pci_ops = { .dev_configure = idxd_configure, .vchan_setup = idxd_vchan_setup, .dev_info_get = idxd_info_get, + .stats_get = idxd_stats_get, + .stats_reset = idxd_stats_reset, .dev_start = idxd_pci_dev_start, .dev_stop = idxd_pci_dev_stop, }; From patchwork Fri Sep 17 15:24:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99223 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E46FCA0C43; Fri, 17 Sep 2021 17:26:05 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C4F37411AC; Fri, 17 Sep 2021 17:25:13 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id F22C6411A0 for ; Fri, 17 Sep 2021 17:25:09 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222476043" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222476043" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:25:09 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="651909848" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by orsmga005.jf.intel.com with ESMTP; 17 Sep 2021 08:25:08 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 17 Sep 2021 15:24:33 +0000 Message-Id: <20210917152437.3270330-13-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210917152437.3270330-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210917152437.3270330-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 12/16] dma/idxd: add vchan status function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When testing dmadev drivers, it is useful to have the HW device in a known state. This patch adds the implementation of the function which will wait for the device to be idle (all jobs completed) before proceeding. Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh --- v3: update API name to vchan_status --- drivers/dma/idxd/idxd_bus.c | 1 + drivers/dma/idxd/idxd_common.c | 17 +++++++++++++++++ drivers/dma/idxd/idxd_internal.h | 2 ++ drivers/dma/idxd/idxd_pci.c | 1 + 4 files changed, 21 insertions(+) diff --git a/drivers/dma/idxd/idxd_bus.c b/drivers/dma/idxd/idxd_bus.c index b52ea02854..e6caa048a9 100644 --- a/drivers/dma/idxd/idxd_bus.c +++ b/drivers/dma/idxd/idxd_bus.c @@ -101,6 +101,7 @@ static const struct rte_dma_dev_ops idxd_bus_ops = { .dev_info_get = idxd_info_get, .stats_get = idxd_stats_get, .stats_reset = idxd_stats_reset, + .vchan_status = idxd_vchan_status, }; static void * diff --git a/drivers/dma/idxd/idxd_common.c b/drivers/dma/idxd/idxd_common.c index d86c58c12a..87d84c081e 100644 --- a/drivers/dma/idxd/idxd_common.c +++ b/drivers/dma/idxd/idxd_common.c @@ -162,6 +162,23 @@ get_comp_status(struct idxd_completion *c) } } +int +idxd_vchan_status(const struct rte_dma_dev *dev, uint16_t vchan __rte_unused, + enum rte_dma_vchan_status *status) +{ + struct idxd_dmadev *idxd = dev->dev_private; + uint16_t last_batch_write = idxd->batch_idx_write == 0 ? idxd->max_batches : + idxd->batch_idx_write - 1; + uint8_t bstatus = (idxd->batch_comp_ring[last_batch_write].status != 0); + + /* An IDXD device will always be either active or idle. + * RTE_DMA_VCHAN_HALTED_ERROR is therefore not supported by IDXD. + */ + *status = bstatus ? RTE_DMA_VCHAN_IDLE : RTE_DMA_VCHAN_ACTIVE; + + return 0; +} + static __rte_always_inline int batch_ok(struct idxd_dmadev *idxd, uint8_t max_ops, enum rte_dma_status_code *status) { diff --git a/drivers/dma/idxd/idxd_internal.h b/drivers/dma/idxd/idxd_internal.h index e2a1119ef7..a291ad26d9 100644 --- a/drivers/dma/idxd/idxd_internal.h +++ b/drivers/dma/idxd/idxd_internal.h @@ -101,5 +101,7 @@ uint16_t idxd_completed_status(struct rte_dma_dev *dev, uint16_t qid __rte_unuse int idxd_stats_get(const struct rte_dma_dev *dev, uint16_t vchan, struct rte_dma_stats *stats, uint32_t stats_sz); int idxd_stats_reset(struct rte_dma_dev *dev, uint16_t vchan); +int idxd_vchan_status(const struct rte_dma_dev *dev, uint16_t vchan, + enum rte_dma_vchan_status *status); #endif /* _IDXD_INTERNAL_H_ */ diff --git a/drivers/dma/idxd/idxd_pci.c b/drivers/dma/idxd/idxd_pci.c index d73845aa3d..2464d4a06c 100644 --- a/drivers/dma/idxd/idxd_pci.c +++ b/drivers/dma/idxd/idxd_pci.c @@ -118,6 +118,7 @@ static const struct rte_dma_dev_ops idxd_pci_ops = { .stats_reset = idxd_stats_reset, .dev_start = idxd_pci_dev_start, .dev_stop = idxd_pci_dev_stop, + .vchan_status = idxd_vchan_status, }; /* each portal uses 4 x 4k pages */ From patchwork Fri Sep 17 15:24:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99224 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 31E07A0C43; Fri, 17 Sep 2021 17:26:11 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D4BFB411B1; Fri, 17 Sep 2021 17:25:14 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id B3D89411A9 for ; Fri, 17 Sep 2021 17:25:11 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222476050" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222476050" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:25:11 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="651909858" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by orsmga005.jf.intel.com with ESMTP; 17 Sep 2021 08:25:09 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 17 Sep 2021 15:24:34 +0000 Message-Id: <20210917152437.3270330-14-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210917152437.3270330-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210917152437.3270330-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 13/16] dma/idxd: add burst capacity API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add support for the burst capacity API. This API will provide the calling application with the remaining capacity of the current burst (limited by max HW batch size). Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh Reviewed-by: Bruce Richardson --- drivers/dma/idxd/idxd_bus.c | 1 + drivers/dma/idxd/idxd_common.c | 20 ++++++++++++++++++++ drivers/dma/idxd/idxd_internal.h | 1 + drivers/dma/idxd/idxd_pci.c | 2 ++ 4 files changed, 24 insertions(+) diff --git a/drivers/dma/idxd/idxd_bus.c b/drivers/dma/idxd/idxd_bus.c index e6caa048a9..54129e5083 100644 --- a/drivers/dma/idxd/idxd_bus.c +++ b/drivers/dma/idxd/idxd_bus.c @@ -102,6 +102,7 @@ static const struct rte_dma_dev_ops idxd_bus_ops = { .stats_get = idxd_stats_get, .stats_reset = idxd_stats_reset, .vchan_status = idxd_vchan_status, + .burst_capacity = idxd_burst_capacity, }; static void * diff --git a/drivers/dma/idxd/idxd_common.c b/drivers/dma/idxd/idxd_common.c index 87d84c081e..b31611c8a4 100644 --- a/drivers/dma/idxd/idxd_common.c +++ b/drivers/dma/idxd/idxd_common.c @@ -469,6 +469,26 @@ idxd_info_get(const struct rte_dma_dev *dev, struct rte_dma_info *info, uint32_t return 0; } +uint16_t +idxd_burst_capacity(const struct rte_dma_dev *dev, uint16_t vchan __rte_unused) +{ + struct idxd_dmadev *idxd = dev->dev_private; + uint16_t write_idx = idxd->batch_start + idxd->batch_size; + uint16_t used_space; + + /* Check for space in the batch ring */ + if ((idxd->batch_idx_read == 0 && idxd->batch_idx_write == idxd->max_batches) || + idxd->batch_idx_write + 1 == idxd->batch_idx_read) + return 0; + + /* For descriptors, check for wrap-around on write but not read */ + if (idxd->ids_returned > write_idx) + write_idx += idxd->desc_ring_mask + 1; + used_space = write_idx - idxd->ids_returned; + + return RTE_MIN((idxd->desc_ring_mask - used_space), idxd->max_batch_size); +} + int idxd_configure(struct rte_dma_dev *dev __rte_unused, const struct rte_dma_conf *dev_conf, uint32_t conf_sz) diff --git a/drivers/dma/idxd/idxd_internal.h b/drivers/dma/idxd/idxd_internal.h index a291ad26d9..3ef2f729a8 100644 --- a/drivers/dma/idxd/idxd_internal.h +++ b/drivers/dma/idxd/idxd_internal.h @@ -103,5 +103,6 @@ int idxd_stats_get(const struct rte_dma_dev *dev, uint16_t vchan, int idxd_stats_reset(struct rte_dma_dev *dev, uint16_t vchan); int idxd_vchan_status(const struct rte_dma_dev *dev, uint16_t vchan, enum rte_dma_vchan_status *status); +uint16_t idxd_burst_capacity(const struct rte_dma_dev *dev, uint16_t vchan); #endif /* _IDXD_INTERNAL_H_ */ diff --git a/drivers/dma/idxd/idxd_pci.c b/drivers/dma/idxd/idxd_pci.c index 2464d4a06c..03ddd63f38 100644 --- a/drivers/dma/idxd/idxd_pci.c +++ b/drivers/dma/idxd/idxd_pci.c @@ -119,6 +119,7 @@ static const struct rte_dma_dev_ops idxd_pci_ops = { .dev_start = idxd_pci_dev_start, .dev_stop = idxd_pci_dev_stop, .vchan_status = idxd_vchan_status, + .burst_capacity = idxd_burst_capacity, }; /* each portal uses 4 x 4k pages */ @@ -232,6 +233,7 @@ init_pci_device(struct rte_pci_device *dev, struct idxd_dmadev *idxd, idxd->u.pci = pci; idxd->max_batches = wq_size; + idxd->max_batch_size = 1 << lg2_max_batch; /* enable the device itself */ err_code = idxd_pci_dev_command(idxd, idxd_enable_dev); From patchwork Fri Sep 17 15:24:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99225 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 51000A0C43; Fri, 17 Sep 2021 17:26:17 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 58197411BB; Fri, 17 Sep 2021 17:25:17 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id ABCB0411A9 for ; Fri, 17 Sep 2021 17:25:13 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222476056" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222476056" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:25:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="651909863" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by orsmga005.jf.intel.com with ESMTP; 17 Sep 2021 08:25:11 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 17 Sep 2021 15:24:35 +0000 Message-Id: <20210917152437.3270330-15-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210917152437.3270330-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210917152437.3270330-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 14/16] dma/idxd: move dpdk_idxd_cfg.py from raw to dma X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Conor Walsh Move the example script for configuring IDXD devices bound to the IDXD kernel driver from raw to dma, and create a symlink to still allow use from raw. Signed-off-by: Conor Walsh Signed-off-by: Kevin Laatz Acked-by: Bruce Richardson --- drivers/dma/idxd/dpdk_idxd_cfg.py | 117 +++++++++++++++++++++++++++++ drivers/raw/ioat/dpdk_idxd_cfg.py | 118 +----------------------------- 2 files changed, 118 insertions(+), 117 deletions(-) create mode 100755 drivers/dma/idxd/dpdk_idxd_cfg.py mode change 100755 => 120000 drivers/raw/ioat/dpdk_idxd_cfg.py diff --git a/drivers/dma/idxd/dpdk_idxd_cfg.py b/drivers/dma/idxd/dpdk_idxd_cfg.py new file mode 100755 index 0000000000..fcc27822ef --- /dev/null +++ b/drivers/dma/idxd/dpdk_idxd_cfg.py @@ -0,0 +1,117 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +""" +Configure an entire Intel DSA instance, using idxd kernel driver, for DPDK use +""" + +import sys +import argparse +import os +import os.path + + +class SysfsDir: + "Used to read/write paths in a sysfs directory" + def __init__(self, path): + self.path = path + + def read_int(self, filename): + "Return a value from sysfs file" + with open(os.path.join(self.path, filename)) as f: + return int(f.readline()) + + def write_values(self, values): + "write dictionary, where key is filename and value is value to write" + for filename, contents in values.items(): + with open(os.path.join(self.path, filename), "w") as f: + f.write(str(contents)) + + +def reset_device(dsa_id): + "Reset the DSA device and all its queues" + drv_dir = SysfsDir("/sys/bus/dsa/drivers/dsa") + drv_dir.write_values({"unbind": f"dsa{dsa_id}"}) + + +def get_pci_dir(pci): + "Search for the sysfs directory of the PCI device" + base_dir = '/sys/bus/pci/devices/' + for path, dirs, files in os.walk(base_dir): + for dir in dirs: + if pci in dir: + return os.path.join(base_dir, dir) + sys.exit(f"Could not find sysfs directory for device {pci}") + + +def get_dsa_id(pci): + "Get the DSA instance ID using the PCI address of the device" + pci_dir = get_pci_dir(pci) + for path, dirs, files in os.walk(pci_dir): + for dir in dirs: + if dir.startswith('dsa') and 'wq' not in dir: + return int(dir[3:]) + sys.exit(f"Could not get device ID for device {pci}") + + +def configure_dsa(dsa_id, queues, prefix): + "Configure the DSA instance with appropriate number of queues" + dsa_dir = SysfsDir(f"/sys/bus/dsa/devices/dsa{dsa_id}") + drv_dir = SysfsDir("/sys/bus/dsa/drivers/dsa") + + max_groups = dsa_dir.read_int("max_groups") + max_engines = dsa_dir.read_int("max_engines") + max_queues = dsa_dir.read_int("max_work_queues") + max_work_queues_size = dsa_dir.read_int("max_work_queues_size") + + nb_queues = min(queues, max_queues) + if queues > nb_queues: + print(f"Setting number of queues to max supported value: {max_queues}") + + # we want one engine per group, and no more engines than queues + nb_groups = min(max_engines, max_groups, nb_queues) + for grp in range(nb_groups): + dsa_dir.write_values({f"engine{dsa_id}.{grp}/group_id": grp}) + + # configure each queue + for q in range(nb_queues): + wq_dir = SysfsDir(os.path.join(dsa_dir.path, f"wq{dsa_id}.{q}")) + wq_dir.write_values({"group_id": q % nb_groups, + "type": "user", + "mode": "dedicated", + "name": f"{prefix}_wq{dsa_id}.{q}", + "priority": 1, + "size": int(max_work_queues_size / nb_queues)}) + + # enable device and then queues + drv_dir.write_values({"bind": f"dsa{dsa_id}"}) + for q in range(nb_queues): + drv_dir.write_values({"bind": f"wq{dsa_id}.{q}"}) + + +def main(args): + "Main function, does arg parsing and calls config function" + arg_p = argparse.ArgumentParser( + description="Configure whole DSA device instance for DPDK use") + arg_p.add_argument('dsa_id', + help="Specify DSA instance either via DSA instance number or PCI address") + arg_p.add_argument('-q', metavar='queues', type=int, default=255, + help="Number of queues to set up") + arg_p.add_argument('--name-prefix', metavar='prefix', dest='prefix', + default="dpdk", + help="Prefix for workqueue name to mark for DPDK use [default: 'dpdk']") + arg_p.add_argument('--reset', action='store_true', + help="Reset DSA device and its queues") + parsed_args = arg_p.parse_args(args[1:]) + + dsa_id = parsed_args.dsa_id + dsa_id = get_dsa_id(dsa_id) if ':' in dsa_id else dsa_id + if parsed_args.reset: + reset_device(dsa_id) + else: + configure_dsa(dsa_id, parsed_args.q, parsed_args.prefix) + + +if __name__ == "__main__": + main(sys.argv) diff --git a/drivers/raw/ioat/dpdk_idxd_cfg.py b/drivers/raw/ioat/dpdk_idxd_cfg.py deleted file mode 100755 index fcc27822ef..0000000000 --- a/drivers/raw/ioat/dpdk_idxd_cfg.py +++ /dev/null @@ -1,117 +0,0 @@ -#!/usr/bin/env python3 -# SPDX-License-Identifier: BSD-3-Clause -# Copyright(c) 2020 Intel Corporation - -""" -Configure an entire Intel DSA instance, using idxd kernel driver, for DPDK use -""" - -import sys -import argparse -import os -import os.path - - -class SysfsDir: - "Used to read/write paths in a sysfs directory" - def __init__(self, path): - self.path = path - - def read_int(self, filename): - "Return a value from sysfs file" - with open(os.path.join(self.path, filename)) as f: - return int(f.readline()) - - def write_values(self, values): - "write dictionary, where key is filename and value is value to write" - for filename, contents in values.items(): - with open(os.path.join(self.path, filename), "w") as f: - f.write(str(contents)) - - -def reset_device(dsa_id): - "Reset the DSA device and all its queues" - drv_dir = SysfsDir("/sys/bus/dsa/drivers/dsa") - drv_dir.write_values({"unbind": f"dsa{dsa_id}"}) - - -def get_pci_dir(pci): - "Search for the sysfs directory of the PCI device" - base_dir = '/sys/bus/pci/devices/' - for path, dirs, files in os.walk(base_dir): - for dir in dirs: - if pci in dir: - return os.path.join(base_dir, dir) - sys.exit(f"Could not find sysfs directory for device {pci}") - - -def get_dsa_id(pci): - "Get the DSA instance ID using the PCI address of the device" - pci_dir = get_pci_dir(pci) - for path, dirs, files in os.walk(pci_dir): - for dir in dirs: - if dir.startswith('dsa') and 'wq' not in dir: - return int(dir[3:]) - sys.exit(f"Could not get device ID for device {pci}") - - -def configure_dsa(dsa_id, queues, prefix): - "Configure the DSA instance with appropriate number of queues" - dsa_dir = SysfsDir(f"/sys/bus/dsa/devices/dsa{dsa_id}") - drv_dir = SysfsDir("/sys/bus/dsa/drivers/dsa") - - max_groups = dsa_dir.read_int("max_groups") - max_engines = dsa_dir.read_int("max_engines") - max_queues = dsa_dir.read_int("max_work_queues") - max_work_queues_size = dsa_dir.read_int("max_work_queues_size") - - nb_queues = min(queues, max_queues) - if queues > nb_queues: - print(f"Setting number of queues to max supported value: {max_queues}") - - # we want one engine per group, and no more engines than queues - nb_groups = min(max_engines, max_groups, nb_queues) - for grp in range(nb_groups): - dsa_dir.write_values({f"engine{dsa_id}.{grp}/group_id": grp}) - - # configure each queue - for q in range(nb_queues): - wq_dir = SysfsDir(os.path.join(dsa_dir.path, f"wq{dsa_id}.{q}")) - wq_dir.write_values({"group_id": q % nb_groups, - "type": "user", - "mode": "dedicated", - "name": f"{prefix}_wq{dsa_id}.{q}", - "priority": 1, - "size": int(max_work_queues_size / nb_queues)}) - - # enable device and then queues - drv_dir.write_values({"bind": f"dsa{dsa_id}"}) - for q in range(nb_queues): - drv_dir.write_values({"bind": f"wq{dsa_id}.{q}"}) - - -def main(args): - "Main function, does arg parsing and calls config function" - arg_p = argparse.ArgumentParser( - description="Configure whole DSA device instance for DPDK use") - arg_p.add_argument('dsa_id', - help="Specify DSA instance either via DSA instance number or PCI address") - arg_p.add_argument('-q', metavar='queues', type=int, default=255, - help="Number of queues to set up") - arg_p.add_argument('--name-prefix', metavar='prefix', dest='prefix', - default="dpdk", - help="Prefix for workqueue name to mark for DPDK use [default: 'dpdk']") - arg_p.add_argument('--reset', action='store_true', - help="Reset DSA device and its queues") - parsed_args = arg_p.parse_args(args[1:]) - - dsa_id = parsed_args.dsa_id - dsa_id = get_dsa_id(dsa_id) if ':' in dsa_id else dsa_id - if parsed_args.reset: - reset_device(dsa_id) - else: - configure_dsa(dsa_id, parsed_args.q, parsed_args.prefix) - - -if __name__ == "__main__": - main(sys.argv) diff --git a/drivers/raw/ioat/dpdk_idxd_cfg.py b/drivers/raw/ioat/dpdk_idxd_cfg.py new file mode 120000 index 0000000000..85545548d1 --- /dev/null +++ b/drivers/raw/ioat/dpdk_idxd_cfg.py @@ -0,0 +1 @@ +../../dma/idxd/dpdk_idxd_cfg.py \ No newline at end of file From patchwork Fri Sep 17 15:24:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99226 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D84F7A0C43; Fri, 17 Sep 2021 17:26:21 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7EFED41152; Fri, 17 Sep 2021 17:25:18 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id 6F35B411B4 for ; Fri, 17 Sep 2021 17:25:15 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222476061" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222476061" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:25:15 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="651909868" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by orsmga005.jf.intel.com with ESMTP; 17 Sep 2021 08:25:13 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 17 Sep 2021 15:24:36 +0000 Message-Id: <20210917152437.3270330-16-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210917152437.3270330-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210917152437.3270330-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 15/16] devbind: add dma device class X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add a new class for DMA devices. Devices listed under the DMA class are to be used with the dmadev library. Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh Reviewed-by: Bruce Richardson Reviewed-by: Chengwen Feng --- usertools/dpdk-devbind.py | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py index 74d16e4c4b..8bb573f4b0 100755 --- a/usertools/dpdk-devbind.py +++ b/usertools/dpdk-devbind.py @@ -69,12 +69,13 @@ network_devices = [network_class, cavium_pkx, avp_vnic, ifpga_class] baseband_devices = [acceleration_class] crypto_devices = [encryption_class, intel_processor_class] +dma_devices = [] eventdev_devices = [cavium_sso, cavium_tim, intel_dlb, octeontx2_sso] mempool_devices = [cavium_fpa, octeontx2_npa] compress_devices = [cavium_zip] regex_devices = [octeontx2_ree] -misc_devices = [cnxk_bphy, cnxk_bphy_cgx, intel_ioat_bdw, intel_ioat_skx, intel_ioat_icx, intel_idxd_spr, - intel_ntb_skx, intel_ntb_icx, +misc_devices = [cnxk_bphy, cnxk_bphy_cgx, intel_ioat_bdw, intel_ioat_skx, + intel_ioat_icx, intel_idxd_spr, intel_ntb_skx, intel_ntb_icx, octeontx2_dma] # global dict ethernet devices present. Dictionary indexed by PCI address. @@ -583,6 +584,9 @@ def show_status(): if status_dev in ["crypto", "all"]: show_device_status(crypto_devices, "Crypto") + if status_dev in ["dma", "all"]: + show_device_status(dma_devices, "DMA") + if status_dev in ["event", "all"]: show_device_status(eventdev_devices, "Eventdev") @@ -651,7 +655,7 @@ def parse_args(): parser.add_argument( '--status-dev', help="Print the status of given device group.", - choices=['baseband', 'compress', 'crypto', 'event', + choices=['baseband', 'compress', 'crypto', 'dma', 'event', 'mempool', 'misc', 'net', 'regex']) bind_group = parser.add_mutually_exclusive_group() bind_group.add_argument( @@ -732,6 +736,7 @@ def do_arg_actions(): get_device_details(network_devices) get_device_details(baseband_devices) get_device_details(crypto_devices) + get_device_details(dma_devices) get_device_details(eventdev_devices) get_device_details(mempool_devices) get_device_details(compress_devices) @@ -754,6 +759,7 @@ def main(): get_device_details(network_devices) get_device_details(baseband_devices) get_device_details(crypto_devices) + get_device_details(dma_devices) get_device_details(eventdev_devices) get_device_details(mempool_devices) get_device_details(compress_devices) From patchwork Fri Sep 17 15:24:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99227 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3EFF7A0C43; Fri, 17 Sep 2021 17:26:26 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 878EC411B7; Fri, 17 Sep 2021 17:25:19 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id E8F5A41163 for ; Fri, 17 Sep 2021 17:25:16 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10110"; a="222476069" X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="222476069" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2021 08:25:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,301,1624345200"; d="scan'208";a="651909875" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by orsmga005.jf.intel.com with ESMTP; 17 Sep 2021 08:25:15 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 17 Sep 2021 15:24:37 +0000 Message-Id: <20210917152437.3270330-17-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210917152437.3270330-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210917152437.3270330-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 16/16] devbind: move idxd device ID to dmadev class X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The dmadev library is the preferred abstraction for using IDXD devices and will replace the rawdev implementation in future. This patch moves the IDXD device ID to the dmadev class. Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh Acked-by: Bruce Richardson --- usertools/dpdk-devbind.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py index 8bb573f4b0..98b698ccc0 100755 --- a/usertools/dpdk-devbind.py +++ b/usertools/dpdk-devbind.py @@ -69,13 +69,13 @@ network_devices = [network_class, cavium_pkx, avp_vnic, ifpga_class] baseband_devices = [acceleration_class] crypto_devices = [encryption_class, intel_processor_class] -dma_devices = [] +dma_devices = [intel_idxd_spr] eventdev_devices = [cavium_sso, cavium_tim, intel_dlb, octeontx2_sso] mempool_devices = [cavium_fpa, octeontx2_npa] compress_devices = [cavium_zip] regex_devices = [octeontx2_ree] misc_devices = [cnxk_bphy, cnxk_bphy_cgx, intel_ioat_bdw, intel_ioat_skx, - intel_ioat_icx, intel_idxd_spr, intel_ntb_skx, intel_ntb_icx, + intel_ioat_icx, intel_ntb_skx, intel_ntb_icx, octeontx2_dma] # global dict ethernet devices present. Dictionary indexed by PCI address.