From patchwork Fri Sep 24 13:39:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99593 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 051EEA0548; Fri, 24 Sep 2021 15:39:32 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 12AAD41305; Fri, 24 Sep 2021 15:39:29 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 4BA36411F2 for ; Fri, 24 Sep 2021 15:39:25 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10116"; a="285084245" X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="285084245" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2021 06:39:24 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="515527577" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by fmsmga008.fm.intel.com with ESMTP; 24 Sep 2021 06:39:22 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 24 Sep 2021 13:39:01 +0000 Message-Id: <20210924133916.4042773-2-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210924133916.4042773-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210924133916.4042773-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v6 01/16] raw/ioat: only build if dmadev not present X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Bruce Richardson Only build the rawdev IDXD/IOAT drivers if the dmadev drivers are not present. This change requires the dependencies to be reordered in drivers/meson.build so that rawdev can use the "RTE_DMA_* build macros to check for the presence of the equivalent dmadev driver. A note is also added to the documentation to inform users of this change. Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh --- v4: - Fix build issue - Add note in raw documentation to outline this change v5: - Provide more detail in commit message - Minor doc changes --- doc/guides/rawdevs/ioat.rst | 8 ++++++++ drivers/meson.build | 4 ++-- drivers/raw/ioat/meson.build | 23 ++++++++++++++++++++--- 3 files changed, 30 insertions(+), 5 deletions(-) diff --git a/doc/guides/rawdevs/ioat.rst b/doc/guides/rawdevs/ioat.rst index a28e909935..a65530bd30 100644 --- a/doc/guides/rawdevs/ioat.rst +++ b/doc/guides/rawdevs/ioat.rst @@ -34,6 +34,14 @@ Compilation For builds using ``meson`` and ``ninja``, the driver will be built when the target platform is x86-based. No additional compilation steps are necessary. +.. note:: + Since the addition of the dmadev library, the ``ioat`` and ``idxd`` parts of this driver + will only be built if their ``dmadev`` counterparts are not built. + The following can be used to disable the ``dmadev`` drivers, + if the raw drivers are to be used instead:: + + $ meson -Ddisable_drivers=dma/* + Device Setup ------------- diff --git a/drivers/meson.build b/drivers/meson.build index b7d680868a..34c0276487 100644 --- a/drivers/meson.build +++ b/drivers/meson.build @@ -10,15 +10,15 @@ subdirs = [ 'common/qat', # depends on bus. 'common/sfc_efx', # depends on bus. 'mempool', # depends on common and bus. + 'dma', # depends on common and bus. 'net', # depends on common, bus, mempool - 'raw', # depends on common, bus and net. + 'raw', # depends on common, bus, dma and net. 'crypto', # depends on common, bus and mempool (net in future). 'compress', # depends on common, bus, mempool. 'regex', # depends on common, bus, regexdev. 'vdpa', # depends on common, bus and mempool. 'event', # depends on common, bus, mempool and net. 'baseband', # depends on common and bus. - 'dma', # depends on common and bus. ] if meson.is_cross_build() diff --git a/drivers/raw/ioat/meson.build b/drivers/raw/ioat/meson.build index 0e81cb5951..9be9d8cc65 100644 --- a/drivers/raw/ioat/meson.build +++ b/drivers/raw/ioat/meson.build @@ -2,14 +2,31 @@ # Copyright 2019 Intel Corporation build = dpdk_conf.has('RTE_ARCH_X86') +# only use ioat rawdev driver if we don't have the equivalent dmadev ones +if dpdk_conf.has('RTE_DMA_IDXD') and dpdk_conf.has('RTE_DMA_IOAT') + build = false + subdir_done() +endif + reason = 'only supported on x86' sources = files( - 'idxd_bus.c', - 'idxd_pci.c', 'ioat_common.c', - 'ioat_rawdev.c', 'ioat_rawdev_test.c', ) + +if not dpdk_conf.has('RTE_DMA_IDXD') + sources += files( + 'idxd_bus.c', + 'idxd_pci.c', + ) +endif + +if not dpdk_conf.has('RTE_DMA_IOAT') + sources += files ( + 'ioat_rawdev.c', + ) +endif + deps += ['bus_pci', 'mbuf', 'rawdev'] headers = files( 'rte_ioat_rawdev.h', From patchwork Fri Sep 24 13:39:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99594 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 43A68A0548; Fri, 24 Sep 2021 15:39:38 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3457641317; Fri, 24 Sep 2021 15:39:30 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 64C9D41303 for ; Fri, 24 Sep 2021 15:39:27 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10116"; a="285084252" X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="285084252" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2021 06:39:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="515527586" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by fmsmga008.fm.intel.com with ESMTP; 24 Sep 2021 06:39:24 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 24 Sep 2021 13:39:02 +0000 Message-Id: <20210924133916.4042773-3-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210924133916.4042773-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210924133916.4042773-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v6 02/16] dma/idxd: add skeleton for VFIO based DSA device X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the basic device probe/remove skeleton code for DSA device bound to the vfio pci driver. Relevant documentation and MAINTAINERS update also included. Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh --- v5: add missing toctree entry for idxd driver v6: add missing new line at end of meson file --- MAINTAINERS | 10 +++++ doc/guides/dmadevs/idxd.rst | 58 ++++++++++++++++++++++++++ doc/guides/dmadevs/index.rst | 2 + doc/guides/rel_notes/release_21_11.rst | 5 +++ drivers/dma/idxd/idxd_internal.h | 27 ++++++++++++ drivers/dma/idxd/idxd_pci.c | 55 ++++++++++++++++++++++++ drivers/dma/idxd/meson.build | 11 +++++ drivers/dma/idxd/version.map | 3 ++ drivers/dma/meson.build | 2 + 9 files changed, 173 insertions(+) create mode 100644 doc/guides/dmadevs/idxd.rst create mode 100644 drivers/dma/idxd/idxd_internal.h create mode 100644 drivers/dma/idxd/idxd_pci.c create mode 100644 drivers/dma/idxd/meson.build create mode 100644 drivers/dma/idxd/version.map diff --git a/MAINTAINERS b/MAINTAINERS index 371d80c42c..497219e948 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1200,6 +1200,16 @@ F: doc/guides/compressdevs/zlib.rst F: doc/guides/compressdevs/features/zlib.ini +DMAdev Drivers +-------------- + +Intel IDXD - EXPERIMENTAL +M: Bruce Richardson +M: Kevin Laatz +F: drivers/dma/idxd/ +F: doc/guides/dmadevs/idxd.rst + + RegEx Drivers ------------- diff --git a/doc/guides/dmadevs/idxd.rst b/doc/guides/dmadevs/idxd.rst new file mode 100644 index 0000000000..924700d17e --- /dev/null +++ b/doc/guides/dmadevs/idxd.rst @@ -0,0 +1,58 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2021 Intel Corporation. + +.. include:: + +IDXD DMA Device Driver +====================== + +The ``idxd`` dmadev driver provides a poll-mode driver (PMD) for Intel\ |reg| +Data Streaming Accelerator `(Intel DSA) +`_. +This PMD can be used in conjunction with Intel\ |reg| DSA devices to offload +data operations, such as data copies, to hardware, freeing up CPU cycles for +other tasks. + +Hardware Requirements +---------------------- + +The ``dpdk-devbind.py`` script, included with DPDK, can be used to show the +presence of supported hardware. Running ``dpdk-devbind.py --status-dev dma`` +will show all the DMA devices on the system, including IDXD supported devices. +Intel\ |reg| DSA devices, are currently (at time of writing) appearing +as devices with type “0b25”, due to the absence of pci-id database entries for +them at this point. + +Compilation +------------ + +For builds using ``meson`` and ``ninja``, the driver will be built when the +target platform is x86-based. No additional compilation steps are necessary. + +Device Setup +------------- + +Devices using VFIO/UIO drivers +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The HW devices to be used will need to be bound to a user-space IO driver for use. +The ``dpdk-devbind.py`` script can be used to view the state of the devices +and to bind them to a suitable DPDK-supported driver, such as ``vfio-pci``. +For example:: + + $ dpdk-devbind.py -b vfio-pci 6a:01.0 + +Device Probing and Initialization +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For devices bound to a suitable DPDK-supported VFIO/UIO driver, the HW devices will +be found as part of the device scan done at application initialization time without +the need to pass parameters to the application. + +For Intel\ |reg| DSA devices, DPDK will automatically configure the device with the +maximum number of workqueues available on it, partitioning all resources equally +among the queues. +If fewer workqueues are required, then the ``max_queues`` parameter may be passed to +the device driver on the EAL commandline, via the ``allowlist`` or ``-a`` flag e.g.:: + + $ dpdk-test -a ,max_queues=4 diff --git a/doc/guides/dmadevs/index.rst b/doc/guides/dmadevs/index.rst index 0bce29d766..5d4abf880e 100644 --- a/doc/guides/dmadevs/index.rst +++ b/doc/guides/dmadevs/index.rst @@ -10,3 +10,5 @@ an application through DMA API. .. toctree:: :maxdepth: 2 :numbered: + + idxd diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst index 7ef5c3c7b0..c980e729f8 100644 --- a/doc/guides/rel_notes/release_21_11.rst +++ b/doc/guides/rel_notes/release_21_11.rst @@ -98,6 +98,11 @@ New Features * Data plane APIs. * Multi-process support. +* **Added IDXD dmadev driver implementation.** + + The IDXD dmadev driver provide device drivers for the Intel DSA devices. + This device driver can be used through the generic dmadev API. + Removed Items ------------- diff --git a/drivers/dma/idxd/idxd_internal.h b/drivers/dma/idxd/idxd_internal.h new file mode 100644 index 0000000000..c6a7dcd72f --- /dev/null +++ b/drivers/dma/idxd/idxd_internal.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2021 Intel Corporation + */ + +#ifndef _IDXD_INTERNAL_H_ +#define _IDXD_INTERNAL_H_ + +/** + * @file idxd_internal.h + * + * Internal data structures for the idxd/DSA driver for dmadev + * + * @warning + * @b EXPERIMENTAL: these structures and APIs may change without prior notice + */ + +extern int idxd_pmd_logtype; + +#define IDXD_PMD_LOG(level, fmt, args...) rte_log(RTE_LOG_ ## level, \ + idxd_pmd_logtype, "IDXD: %s(): " fmt "\n", __func__, ##args) + +#define IDXD_PMD_DEBUG(fmt, args...) IDXD_PMD_LOG(DEBUG, fmt, ## args) +#define IDXD_PMD_INFO(fmt, args...) IDXD_PMD_LOG(INFO, fmt, ## args) +#define IDXD_PMD_ERR(fmt, args...) IDXD_PMD_LOG(ERR, fmt, ## args) +#define IDXD_PMD_WARN(fmt, args...) IDXD_PMD_LOG(WARNING, fmt, ## args) + +#endif /* _IDXD_INTERNAL_H_ */ diff --git a/drivers/dma/idxd/idxd_pci.c b/drivers/dma/idxd/idxd_pci.c new file mode 100644 index 0000000000..79e4aadcab --- /dev/null +++ b/drivers/dma/idxd/idxd_pci.c @@ -0,0 +1,55 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2021 Intel Corporation + */ + +#include + +#include "idxd_internal.h" + +#define IDXD_VENDOR_ID 0x8086 +#define IDXD_DEVICE_ID_SPR 0x0B25 + +#define IDXD_PMD_DMADEV_NAME_PCI dmadev_idxd_pci + +const struct rte_pci_id pci_id_idxd_map[] = { + { RTE_PCI_DEVICE(IDXD_VENDOR_ID, IDXD_DEVICE_ID_SPR) }, + { .vendor_id = 0, /* sentinel */ }, +}; + +static int +idxd_dmadev_probe_pci(struct rte_pci_driver *drv, struct rte_pci_device *dev) +{ + int ret = 0; + char name[PCI_PRI_STR_SIZE]; + + rte_pci_device_name(&dev->addr, name, sizeof(name)); + IDXD_PMD_INFO("Init %s on NUMA node %d", name, dev->device.numa_node); + dev->device.driver = &drv->driver; + + return ret; +} + +static int +idxd_dmadev_remove_pci(struct rte_pci_device *dev) +{ + char name[PCI_PRI_STR_SIZE]; + + rte_pci_device_name(&dev->addr, name, sizeof(name)); + + IDXD_PMD_INFO("Closing %s on NUMA node %d", + name, dev->device.numa_node); + + return 0; +} + +struct rte_pci_driver idxd_pmd_drv_pci = { + .id_table = pci_id_idxd_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING, + .probe = idxd_dmadev_probe_pci, + .remove = idxd_dmadev_remove_pci, +}; + +RTE_PMD_REGISTER_PCI(IDXD_PMD_DMADEV_NAME_PCI, idxd_pmd_drv_pci); +RTE_PMD_REGISTER_PCI_TABLE(IDXD_PMD_DMADEV_NAME_PCI, pci_id_idxd_map); +RTE_PMD_REGISTER_KMOD_DEP(IDXD_PMD_DMADEV_NAME_PCI, "vfio-pci"); +RTE_PMD_REGISTER_PARAM_STRING(dmadev_idxd_pci, "max_queues=0"); diff --git a/drivers/dma/idxd/meson.build b/drivers/dma/idxd/meson.build new file mode 100644 index 0000000000..4426a9f65c --- /dev/null +++ b/drivers/dma/idxd/meson.build @@ -0,0 +1,11 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2021 Intel Corporation + +if is_windows + subdir_done() +endif + +deps += ['bus_pci'] +sources = files( + 'idxd_pci.c' +) diff --git a/drivers/dma/idxd/version.map b/drivers/dma/idxd/version.map new file mode 100644 index 0000000000..4a76d1d52d --- /dev/null +++ b/drivers/dma/idxd/version.map @@ -0,0 +1,3 @@ +DPDK_21 { + local: *; +}; diff --git a/drivers/dma/meson.build b/drivers/dma/meson.build index d9c7ede32f..411be7a240 100644 --- a/drivers/dma/meson.build +++ b/drivers/dma/meson.build @@ -2,5 +2,7 @@ # Copyright 2021 HiSilicon Limited drivers = [ + 'idxd', 'skeleton', ] +std_deps = ['dmadev'] From patchwork Fri Sep 24 13:39:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99595 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5A8BEA0548; Fri, 24 Sep 2021 15:39:44 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 57BCA4131D; Fri, 24 Sep 2021 15:39:32 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 8B0DD4131A for ; Fri, 24 Sep 2021 15:39:30 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10116"; a="285084261" X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="285084261" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2021 06:39:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="515527600" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by fmsmga008.fm.intel.com with ESMTP; 24 Sep 2021 06:39:27 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 24 Sep 2021 13:39:03 +0000 Message-Id: <20210924133916.4042773-4-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210924133916.4042773-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210924133916.4042773-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v6 03/16] dma/idxd: add bus device probing X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the basic device probing for DSA devices bound to the IDXD kernel driver. These devices can be configured via sysfs and made available to DPDK if they are found during bus scan. Relevant documentation is included. Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh --- v4: fix 'vdev' naming, changed to 'bus' --- doc/guides/dmadevs/idxd.rst | 64 +++++++ drivers/dma/idxd/idxd_bus.c | 351 +++++++++++++++++++++++++++++++++++ drivers/dma/idxd/meson.build | 1 + 3 files changed, 416 insertions(+) create mode 100644 drivers/dma/idxd/idxd_bus.c diff --git a/doc/guides/dmadevs/idxd.rst b/doc/guides/dmadevs/idxd.rst index 924700d17e..ce33e2857a 100644 --- a/doc/guides/dmadevs/idxd.rst +++ b/doc/guides/dmadevs/idxd.rst @@ -32,6 +32,56 @@ target platform is x86-based. No additional compilation steps are necessary. Device Setup ------------- +Intel\ |reg| DSA devices can use the IDXD kernel driver or DPDK-supported drivers, +such as ``vfio-pci``. Both are supported by the IDXD PMD. + +Intel\ |reg| DSA devices using IDXD kernel driver +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To use an Intel\ |reg| DSA device bound to the IDXD kernel driver, the device must first be configured. +The `accel-config `_ utility library can be used for configuration. + +.. note:: + The device configuration can also be done by directly interacting with the sysfs nodes. + An example of how this may be done can be seen in the script ``dpdk_idxd_cfg.py`` + included in the driver source directory. + +There are some mandatory configuration steps before being able to use a device with an application. +The internal engines, which do the copies or other operations, +and the work-queues, which are used by applications to assign work to the device, +need to be assigned to groups, and the various other configuration options, +such as priority or queue depth, need to be set for each queue. + +To assign an engine to a group:: + + $ accel-config config-engine dsa0/engine0.0 --group-id=0 + $ accel-config config-engine dsa0/engine0.1 --group-id=1 + +To assign work queues to groups for passing descriptors to the engines a similar accel-config command can be used. +However, the work queues also need to be configured depending on the use case. +Some configuration options include: + +* mode (Dedicated/Shared): Indicates whether a WQ may accept jobs from multiple queues simultaneously. +* priority: WQ priority between 1 and 15. Larger value means higher priority. +* wq-size: the size of the WQ. Sum of all WQ sizes must be less that the total-size defined by the device. +* type: WQ type (kernel/mdev/user). Determines how the device is presented. +* name: identifier given to the WQ. + +Example configuration for a work queue:: + + $ accel-config config-wq dsa0/wq0.0 --group-id=0 \ + --mode=dedicated --priority=10 --wq-size=8 \ + --type=user --name=dpdk_app1 + +Once the devices have been configured, they need to be enabled:: + + $ accel-config enable-device dsa0 + $ accel-config enable-wq dsa0/wq0.0 + +Check the device configuration:: + + $ accel-config list + Devices using VFIO/UIO drivers ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -56,3 +106,17 @@ If fewer workqueues are required, then the ``max_queues`` parameter may be passe the device driver on the EAL commandline, via the ``allowlist`` or ``-a`` flag e.g.:: $ dpdk-test -a ,max_queues=4 + +For devices bound to the IDXD kernel driver, +the DPDK IDXD driver will automatically perform a scan for available workqueues +to use. Any workqueues found listed in ``/dev/dsa`` on the system will be checked +in ``/sys``, and any which have ``dpdk_`` prefix in their name will be automatically +probed by the driver to make them available to the application. +Alternatively, to support use by multiple DPDK processes simultaneously, +the value used as the DPDK ``--file-prefix`` parameter may be used as a workqueue +name prefix, instead of ``dpdk_``, allowing each DPDK application instance to only +use a subset of configured queues. + +Once probed successfully, irrespective of kernel driver, the device will appear as a ``dmadev``, +that is a "DMA device type" inside DPDK, and can be accessed using APIs from the +``rte_dmadev`` library. diff --git a/drivers/dma/idxd/idxd_bus.c b/drivers/dma/idxd/idxd_bus.c new file mode 100644 index 0000000000..ef589af30e --- /dev/null +++ b/drivers/dma/idxd/idxd_bus.c @@ -0,0 +1,351 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2021 Intel Corporation + */ + +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +#include "idxd_internal.h" + +/* default value for DSA paths, but allow override in environment for testing */ +#define DSA_DEV_PATH "/dev/dsa" +#define DSA_SYSFS_PATH "/sys/bus/dsa/devices" + +static unsigned int devcount; + +/** unique identifier for a DSA device/WQ instance */ +struct dsa_wq_addr { + uint16_t device_id; + uint16_t wq_id; +}; + +/** a DSA device instance */ +struct rte_dsa_device { + struct rte_device device; /**< Inherit core device */ + TAILQ_ENTRY(rte_dsa_device) next; /**< next dev in list */ + + char wq_name[32]; /**< the workqueue name/number e.g. wq0.1 */ + struct dsa_wq_addr addr; /**< Identifies the specific WQ */ +}; + +/* forward prototypes */ +struct dsa_bus; +static int dsa_scan(void); +static int dsa_probe(void); +static struct rte_device *dsa_find_device(const struct rte_device *start, + rte_dev_cmp_t cmp, const void *data); +static enum rte_iova_mode dsa_get_iommu_class(void); +static int dsa_addr_parse(const char *name, void *addr); + +/** List of devices */ +TAILQ_HEAD(dsa_device_list, rte_dsa_device); + +/** + * Structure describing the DSA bus + */ +struct dsa_bus { + struct rte_bus bus; /**< Inherit the generic class */ + struct rte_driver driver; /**< Driver struct for devices to point to */ + struct dsa_device_list device_list; /**< List of PCI devices */ +}; + +struct dsa_bus dsa_bus = { + .bus = { + .scan = dsa_scan, + .probe = dsa_probe, + .find_device = dsa_find_device, + .get_iommu_class = dsa_get_iommu_class, + .parse = dsa_addr_parse, + }, + .driver = { + .name = "dmadev_idxd" + }, + .device_list = TAILQ_HEAD_INITIALIZER(dsa_bus.device_list), +}; + +static inline const char * +dsa_get_dev_path(void) +{ + const char *path = getenv("DSA_DEV_PATH"); + return path ? path : DSA_DEV_PATH; +} + +static inline const char * +dsa_get_sysfs_path(void) +{ + const char *path = getenv("DSA_SYSFS_PATH"); + return path ? path : DSA_SYSFS_PATH; +} + +static void * +idxd_bus_mmap_wq(struct rte_dsa_device *dev) +{ + void *addr; + char path[PATH_MAX]; + int fd; + + snprintf(path, sizeof(path), "%s/%s", dsa_get_dev_path(), dev->wq_name); + fd = open(path, O_RDWR); + if (fd < 0) { + IDXD_PMD_ERR("Failed to open device path: %s", path); + return NULL; + } + + addr = mmap(NULL, 0x1000, PROT_WRITE, MAP_SHARED, fd, 0); + close(fd); + if (addr == MAP_FAILED) { + IDXD_PMD_ERR("Failed to mmap device %s", path); + return NULL; + } + + return addr; +} + +static int +read_wq_string(struct rte_dsa_device *dev, const char *filename, + char *value, size_t valuelen) +{ + char sysfs_node[PATH_MAX]; + int len; + int fd; + + snprintf(sysfs_node, sizeof(sysfs_node), "%s/%s/%s", + dsa_get_sysfs_path(), dev->wq_name, filename); + fd = open(sysfs_node, O_RDONLY); + if (fd < 0) { + IDXD_PMD_ERR("%s(): opening file '%s' failed: %s", + __func__, sysfs_node, strerror(errno)); + return -1; + } + + len = read(fd, value, valuelen - 1); + close(fd); + if (len < 0) { + IDXD_PMD_ERR("%s(): error reading file '%s': %s", + __func__, sysfs_node, strerror(errno)); + return -1; + } + value[len] = '\0'; + return 0; +} + +static int +read_wq_int(struct rte_dsa_device *dev, const char *filename, + int *value) +{ + char sysfs_node[PATH_MAX]; + FILE *f; + int ret = 0; + + snprintf(sysfs_node, sizeof(sysfs_node), "%s/%s/%s", + dsa_get_sysfs_path(), dev->wq_name, filename); + f = fopen(sysfs_node, "r"); + if (f == NULL) { + IDXD_PMD_ERR("%s(): opening file '%s' failed: %s", + __func__, sysfs_node, strerror(errno)); + return -1; + } + + if (fscanf(f, "%d", value) != 1) { + IDXD_PMD_ERR("%s(): error reading file '%s': %s", + __func__, sysfs_node, strerror(errno)); + ret = -1; + } + + fclose(f); + return ret; +} + +static int +read_device_int(struct rte_dsa_device *dev, const char *filename, + int *value) +{ + char sysfs_node[PATH_MAX]; + FILE *f; + int ret = 0; + + snprintf(sysfs_node, sizeof(sysfs_node), "%s/dsa%d/%s", + dsa_get_sysfs_path(), dev->addr.device_id, filename); + f = fopen(sysfs_node, "r"); + if (f == NULL) { + IDXD_PMD_ERR("%s(): opening file '%s' failed: %s", + __func__, sysfs_node, strerror(errno)); + return -1; + } + + if (fscanf(f, "%d", value) != 1) { + IDXD_PMD_ERR("%s(): error reading file '%s': %s", + __func__, sysfs_node, strerror(errno)); + ret = -1; + } + + fclose(f); + return ret; +} + +static int +idxd_probe_dsa(struct rte_dsa_device *dev) +{ + struct idxd_dmadev idxd = {0}; + int ret = 0; + + IDXD_PMD_INFO("Probing device %s on numa node %d", + dev->wq_name, dev->device.numa_node); + if (read_wq_int(dev, "size", &ret) < 0) + return -1; + idxd.max_batches = ret; + if (read_wq_int(dev, "max_batch_size", &ret) < 0) + return -1; + idxd.max_batch_size = ret; + idxd.qid = dev->addr.wq_id; + idxd.sva_support = 1; + + idxd.portal = idxd_bus_mmap_wq(dev); + if (idxd.portal == NULL) { + IDXD_PMD_ERR("WQ mmap failed"); + return -ENOENT; + } + + return 0; +} + +static int +is_for_this_process_use(const char *name) +{ + char *runtime_dir = strdup(rte_eal_get_runtime_dir()); + char *prefix = basename(runtime_dir); + int prefixlen = strlen(prefix); + int retval = 0; + + if (strncmp(name, "dpdk_", 5) == 0) + retval = 1; + if (strncmp(name, prefix, prefixlen) == 0 && name[prefixlen] == '_') + retval = 1; + + free(runtime_dir); + return retval; +} + +static int +dsa_probe(void) +{ + struct rte_dsa_device *dev; + + TAILQ_FOREACH(dev, &dsa_bus.device_list, next) { + char type[64], name[64]; + + if (read_wq_string(dev, "type", type, sizeof(type)) < 0 || + read_wq_string(dev, "name", name, sizeof(name)) < 0) + continue; + + if (strncmp(type, "user", 4) == 0 && is_for_this_process_use(name)) { + dev->device.driver = &dsa_bus.driver; + idxd_probe_dsa(dev); + continue; + } + IDXD_PMD_DEBUG("WQ '%s', not allocated to DPDK", dev->wq_name); + } + + return 0; +} + +static int +dsa_scan(void) +{ + const char *path = dsa_get_dev_path(); + struct dirent *wq; + DIR *dev_dir; + + dev_dir = opendir(path); + if (dev_dir == NULL) { + if (errno == ENOENT) + return 0; /* no bus, return without error */ + IDXD_PMD_ERR("%s(): opendir '%s' failed: %s", + __func__, path, strerror(errno)); + return -1; + } + + while ((wq = readdir(dev_dir)) != NULL) { + struct rte_dsa_device *dev; + int numa_node = -1; + + if (strncmp(wq->d_name, "wq", 2) != 0) + continue; + if (strnlen(wq->d_name, sizeof(dev->wq_name)) == sizeof(dev->wq_name)) { + IDXD_PMD_ERR("%s(): wq name too long: '%s', skipping", + __func__, wq->d_name); + continue; + } + IDXD_PMD_DEBUG("%s(): found %s/%s", __func__, path, wq->d_name); + + dev = malloc(sizeof(*dev)); + if (dsa_addr_parse(wq->d_name, &dev->addr) < 0) { + IDXD_PMD_ERR("Error parsing WQ name: %s", wq->d_name); + free(dev); + continue; + } + dev->device.bus = &dsa_bus.bus; + strlcpy(dev->wq_name, wq->d_name, sizeof(dev->wq_name)); + TAILQ_INSERT_TAIL(&dsa_bus.device_list, dev, next); + devcount++; + + read_device_int(dev, "numa_node", &numa_node); + dev->device.numa_node = numa_node; + dev->device.name = dev->wq_name; + } + + closedir(dev_dir); + return 0; +} + +static struct rte_device * +dsa_find_device(const struct rte_device *start, rte_dev_cmp_t cmp, + const void *data) +{ + struct rte_dsa_device *dev = TAILQ_FIRST(&dsa_bus.device_list); + + /* the rte_device struct must be at start of dsa structure */ + RTE_BUILD_BUG_ON(offsetof(struct rte_dsa_device, device) != 0); + + if (start != NULL) /* jump to start point if given */ + dev = TAILQ_NEXT((const struct rte_dsa_device *)start, next); + while (dev != NULL) { + if (cmp(&dev->device, data) == 0) + return &dev->device; + dev = TAILQ_NEXT(dev, next); + } + return NULL; +} + +static enum rte_iova_mode +dsa_get_iommu_class(void) +{ + /* if there are no devices, report don't care, otherwise VA mode */ + return devcount > 0 ? RTE_IOVA_VA : RTE_IOVA_DC; +} + +static int +dsa_addr_parse(const char *name, void *addr) +{ + struct dsa_wq_addr *wq = addr; + unsigned int device_id, wq_id; + + if (sscanf(name, "wq%u.%u", &device_id, &wq_id) != 2) { + IDXD_PMD_DEBUG("Parsing WQ name failed: %s", name); + return -1; + } + + wq->device_id = device_id; + wq->wq_id = wq_id; + return 0; +} + +RTE_REGISTER_BUS(dsa, dsa_bus.bus); diff --git a/drivers/dma/idxd/meson.build b/drivers/dma/idxd/meson.build index 4426a9f65c..45418077f4 100644 --- a/drivers/dma/idxd/meson.build +++ b/drivers/dma/idxd/meson.build @@ -7,5 +7,6 @@ endif deps += ['bus_pci'] sources = files( + 'idxd_bus.c', 'idxd_pci.c' ) From patchwork Fri Sep 24 13:39:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99596 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 90AFFA0548; Fri, 24 Sep 2021 15:39:52 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C61134132A; Fri, 24 Sep 2021 15:39:34 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 3838F4131A for ; Fri, 24 Sep 2021 15:39:32 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10116"; a="285084265" X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="285084265" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2021 06:39:31 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="515527614" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by fmsmga008.fm.intel.com with ESMTP; 24 Sep 2021 06:39:30 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 24 Sep 2021 13:39:04 +0000 Message-Id: <20210924133916.4042773-5-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210924133916.4042773-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210924133916.4042773-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v6 04/16] dma/idxd: create dmadev instances on bus probe X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When a suitable device is found during the bus scan/probe, create a dmadev instance for each HW queue. Internal structures required for device creation are also added. Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh --- v4: - fix 'vdev' naming, changed to 'bus' - rebase changes v6: - remove redundant struct initialization in create - set device state to ready at end of create --- drivers/dma/idxd/idxd_bus.c | 19 +++++++++ drivers/dma/idxd/idxd_common.c | 70 ++++++++++++++++++++++++++++++++ drivers/dma/idxd/idxd_internal.h | 40 ++++++++++++++++++ drivers/dma/idxd/meson.build | 1 + 4 files changed, 130 insertions(+) create mode 100644 drivers/dma/idxd/idxd_common.c diff --git a/drivers/dma/idxd/idxd_bus.c b/drivers/dma/idxd/idxd_bus.c index ef589af30e..b48fa954ed 100644 --- a/drivers/dma/idxd/idxd_bus.c +++ b/drivers/dma/idxd/idxd_bus.c @@ -85,6 +85,18 @@ dsa_get_sysfs_path(void) return path ? path : DSA_SYSFS_PATH; } +static int +idxd_dev_close(struct rte_dma_dev *dev) +{ + struct idxd_dmadev *idxd = dev->data->dev_private; + munmap(idxd->portal, 0x1000); + return 0; +} + +static const struct rte_dma_dev_ops idxd_bus_ops = { + .dev_close = idxd_dev_close, +}; + static void * idxd_bus_mmap_wq(struct rte_dsa_device *dev) { @@ -206,6 +218,7 @@ idxd_probe_dsa(struct rte_dsa_device *dev) return -1; idxd.max_batch_size = ret; idxd.qid = dev->addr.wq_id; + idxd.u.bus.dsa_id = dev->addr.device_id; idxd.sva_support = 1; idxd.portal = idxd_bus_mmap_wq(dev); @@ -214,6 +227,12 @@ idxd_probe_dsa(struct rte_dsa_device *dev) return -ENOENT; } + ret = idxd_dmadev_create(dev->wq_name, &dev->device, &idxd, &idxd_bus_ops); + if (ret) { + IDXD_PMD_ERR("Failed to create rawdev %s", dev->wq_name); + return ret; + } + return 0; } diff --git a/drivers/dma/idxd/idxd_common.c b/drivers/dma/idxd/idxd_common.c new file mode 100644 index 0000000000..130153f7d2 --- /dev/null +++ b/drivers/dma/idxd/idxd_common.c @@ -0,0 +1,70 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2021 Intel Corporation + */ + +#include +#include +#include + +#include "idxd_internal.h" + +#define IDXD_PMD_NAME_STR "dmadev_idxd" + +int +idxd_dmadev_create(const char *name, struct rte_device *dev, + const struct idxd_dmadev *base_idxd, + const struct rte_dma_dev_ops *ops) +{ + struct idxd_dmadev *idxd = NULL; + struct rte_dma_dev *dmadev = NULL; + int ret = 0; + + if (!name) { + IDXD_PMD_ERR("Invalid name of the device!"); + ret = -EINVAL; + goto cleanup; + } + + /* Allocate device structure */ + dmadev = rte_dma_pmd_allocate(name, dev->numa_node, sizeof(struct idxd_dmadev)); + if (dmadev == NULL) { + IDXD_PMD_ERR("Unable to allocate raw device"); + ret = -ENOMEM; + goto cleanup; + } + dmadev->dev_ops = ops; + dmadev->device = dev; + + idxd = dmadev->dev_private; + *idxd = *base_idxd; /* copy over the main fields already passed in */ + idxd->dmadev = dmadev; + + /* allocate batch index ring and completion ring. + * The +1 is because we can never fully use + * the ring, otherwise read == write means both full and empty. + */ + idxd->batch_comp_ring = rte_zmalloc(NULL, (sizeof(idxd->batch_idx_ring[0]) + + sizeof(idxd->batch_comp_ring[0])) * (idxd->max_batches + 1), + sizeof(idxd->batch_comp_ring[0])); + if (idxd->batch_comp_ring == NULL) { + IDXD_PMD_ERR("Unable to reserve memory for batch data\n"); + ret = -ENOMEM; + goto cleanup; + } + idxd->batch_idx_ring = (void *)&idxd->batch_comp_ring[idxd->max_batches+1]; + idxd->batch_iova = rte_mem_virt2iova(idxd->batch_comp_ring); + + idxd->dmadev->state = RTE_DMA_DEV_READY; + + return 0; + +cleanup: + if (dmadev) + rte_dma_pmd_release(name); + + return ret; +} + +int idxd_pmd_logtype; + +RTE_LOG_REGISTER_DEFAULT(idxd_pmd_logtype, WARNING); diff --git a/drivers/dma/idxd/idxd_internal.h b/drivers/dma/idxd/idxd_internal.h index c6a7dcd72f..fa6f053f72 100644 --- a/drivers/dma/idxd/idxd_internal.h +++ b/drivers/dma/idxd/idxd_internal.h @@ -24,4 +24,44 @@ extern int idxd_pmd_logtype; #define IDXD_PMD_ERR(fmt, args...) IDXD_PMD_LOG(ERR, fmt, ## args) #define IDXD_PMD_WARN(fmt, args...) IDXD_PMD_LOG(WARNING, fmt, ## args) +struct idxd_dmadev { + /* counters to track the batches */ + unsigned short max_batches; + unsigned short batch_idx_read; + unsigned short batch_idx_write; + + /* track descriptors and handles */ + unsigned short desc_ring_mask; + unsigned short ids_avail; /* handles for ops completed */ + unsigned short ids_returned; /* the read pointer for hdls/desc rings */ + unsigned short batch_start; /* start+size == write pointer for hdls/desc */ + unsigned short batch_size; + + void *portal; /* address to write the batch descriptor */ + + struct idxd_completion *batch_comp_ring; + unsigned short *batch_idx_ring; /* store where each batch ends */ + + struct rte_dma_stats stats; + + rte_iova_t batch_iova; /* base address of the batch comp ring */ + rte_iova_t desc_iova; /* base address of desc ring, needed for completions */ + + unsigned short max_batch_size; + + struct rte_dma_dev *dmadev; + struct rte_dma_vchan_conf qcfg; + uint8_t sva_support; + uint8_t qid; + + union { + struct { + unsigned int dsa_id; + } bus; + } u; +}; + +int idxd_dmadev_create(const char *name, struct rte_device *dev, + const struct idxd_dmadev *base_idxd, const struct rte_dma_dev_ops *ops); + #endif /* _IDXD_INTERNAL_H_ */ diff --git a/drivers/dma/idxd/meson.build b/drivers/dma/idxd/meson.build index 45418077f4..da5dc2b019 100644 --- a/drivers/dma/idxd/meson.build +++ b/drivers/dma/idxd/meson.build @@ -8,5 +8,6 @@ endif deps += ['bus_pci'] sources = files( 'idxd_bus.c', + 'idxd_common.c', 'idxd_pci.c' ) From patchwork Fri Sep 24 13:39:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99597 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 860E4A0548; Fri, 24 Sep 2021 15:39:58 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id EB6DC41330; Fri, 24 Sep 2021 15:39:36 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 499F54132D for ; Fri, 24 Sep 2021 15:39:35 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10116"; a="285084274" X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="285084274" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2021 06:39:34 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="515527631" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by fmsmga008.fm.intel.com with ESMTP; 24 Sep 2021 06:39:31 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 24 Sep 2021 13:39:05 +0000 Message-Id: <20210924133916.4042773-6-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210924133916.4042773-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210924133916.4042773-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v6 05/16] dma/idxd: create dmadev instances on pci probe X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When a suitable device is found during the PCI probe, create a dmadev instance for each HW queue. HW definitions required are also included. Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh --- v4: rebase changes v6: add close function for device destroy and cleanup --- drivers/dma/idxd/idxd_hw_defs.h | 71 ++++++++ drivers/dma/idxd/idxd_internal.h | 16 ++ drivers/dma/idxd/idxd_pci.c | 285 ++++++++++++++++++++++++++++++- 3 files changed, 369 insertions(+), 3 deletions(-) create mode 100644 drivers/dma/idxd/idxd_hw_defs.h diff --git a/drivers/dma/idxd/idxd_hw_defs.h b/drivers/dma/idxd/idxd_hw_defs.h new file mode 100644 index 0000000000..ea627cba6d --- /dev/null +++ b/drivers/dma/idxd/idxd_hw_defs.h @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2021 Intel Corporation + */ + +#ifndef _IDXD_HW_DEFS_H_ +#define _IDXD_HW_DEFS_H_ + +/*** Definitions for Intel(R) Data Streaming Accelerator ***/ + +#define IDXD_CMD_SHIFT 20 +enum rte_idxd_cmds { + idxd_enable_dev = 1, + idxd_disable_dev, + idxd_drain_all, + idxd_abort_all, + idxd_reset_device, + idxd_enable_wq, + idxd_disable_wq, + idxd_drain_wq, + idxd_abort_wq, + idxd_reset_wq, +}; + +/* General bar0 registers */ +struct rte_idxd_bar0 { + uint32_t __rte_cache_aligned version; /* offset 0x00 */ + uint64_t __rte_aligned(0x10) gencap; /* offset 0x10 */ + uint64_t __rte_aligned(0x10) wqcap; /* offset 0x20 */ + uint64_t __rte_aligned(0x10) grpcap; /* offset 0x30 */ + uint64_t __rte_aligned(0x08) engcap; /* offset 0x38 */ + uint64_t __rte_aligned(0x10) opcap; /* offset 0x40 */ + uint64_t __rte_aligned(0x20) offsets[2]; /* offset 0x60 */ + uint32_t __rte_aligned(0x20) gencfg; /* offset 0x80 */ + uint32_t __rte_aligned(0x08) genctrl; /* offset 0x88 */ + uint32_t __rte_aligned(0x10) gensts; /* offset 0x90 */ + uint32_t __rte_aligned(0x08) intcause; /* offset 0x98 */ + uint32_t __rte_aligned(0x10) cmd; /* offset 0xA0 */ + uint32_t __rte_aligned(0x08) cmdstatus; /* offset 0xA8 */ + uint64_t __rte_aligned(0x20) swerror[4]; /* offset 0xC0 */ +}; + +/* workqueue config is provided by array of uint32_t. */ +enum rte_idxd_wqcfg { + wq_size_idx, /* size is in first 32-bit value */ + wq_threshold_idx, /* WQ threshold second 32-bits */ + wq_mode_idx, /* WQ mode and other flags */ + wq_sizes_idx, /* WQ transfer and batch sizes */ + wq_occ_int_idx, /* WQ occupancy interrupt handle */ + wq_occ_limit_idx, /* WQ occupancy limit */ + wq_state_idx, /* WQ state and occupancy state */ +}; + +#define WQ_MODE_SHARED 0 +#define WQ_MODE_DEDICATED 1 +#define WQ_PRIORITY_SHIFT 4 +#define WQ_BATCH_SZ_SHIFT 5 +#define WQ_STATE_SHIFT 30 +#define WQ_STATE_MASK 0x3 + +struct rte_idxd_grpcfg { + uint64_t grpwqcfg[4] __rte_cache_aligned; /* 64-byte register set */ + uint64_t grpengcfg; /* offset 32 */ + uint32_t grpflags; /* offset 40 */ +}; + +#define GENSTS_DEV_STATE_MASK 0x03 +#define CMDSTATUS_ACTIVE_SHIFT 31 +#define CMDSTATUS_ACTIVE_MASK (1 << 31) +#define CMDSTATUS_ERR_MASK 0xFF + +#endif diff --git a/drivers/dma/idxd/idxd_internal.h b/drivers/dma/idxd/idxd_internal.h index fa6f053f72..cb3a68c69b 100644 --- a/drivers/dma/idxd/idxd_internal.h +++ b/drivers/dma/idxd/idxd_internal.h @@ -5,6 +5,10 @@ #ifndef _IDXD_INTERNAL_H_ #define _IDXD_INTERNAL_H_ +#include + +#include "idxd_hw_defs.h" + /** * @file idxd_internal.h * @@ -24,6 +28,16 @@ extern int idxd_pmd_logtype; #define IDXD_PMD_ERR(fmt, args...) IDXD_PMD_LOG(ERR, fmt, ## args) #define IDXD_PMD_WARN(fmt, args...) IDXD_PMD_LOG(WARNING, fmt, ## args) +struct idxd_pci_common { + rte_spinlock_t lk; + + uint8_t wq_cfg_sz; + volatile struct rte_idxd_bar0 *regs; + volatile uint32_t *wq_regs_base; + volatile struct rte_idxd_grpcfg *grp_regs; + volatile void *portals; +}; + struct idxd_dmadev { /* counters to track the batches */ unsigned short max_batches; @@ -58,6 +72,8 @@ struct idxd_dmadev { struct { unsigned int dsa_id; } bus; + + struct idxd_pci_common *pci; } u; }; diff --git a/drivers/dma/idxd/idxd_pci.c b/drivers/dma/idxd/idxd_pci.c index 79e4aadcab..0c03a51449 100644 --- a/drivers/dma/idxd/idxd_pci.c +++ b/drivers/dma/idxd/idxd_pci.c @@ -3,6 +3,9 @@ */ #include +#include +#include +#include #include "idxd_internal.h" @@ -16,17 +19,293 @@ const struct rte_pci_id pci_id_idxd_map[] = { { .vendor_id = 0, /* sentinel */ }, }; +static inline int +idxd_pci_dev_command(struct idxd_dmadev *idxd, enum rte_idxd_cmds command) +{ + uint8_t err_code; + uint16_t qid = idxd->qid; + int i = 0; + + if (command >= idxd_disable_wq && command <= idxd_reset_wq) + qid = (1 << qid); + rte_spinlock_lock(&idxd->u.pci->lk); + idxd->u.pci->regs->cmd = (command << IDXD_CMD_SHIFT) | qid; + + do { + rte_pause(); + err_code = idxd->u.pci->regs->cmdstatus; + if (++i >= 1000) { + IDXD_PMD_ERR("Timeout waiting for command response from HW"); + rte_spinlock_unlock(&idxd->u.pci->lk); + return err_code; + } + } while (idxd->u.pci->regs->cmdstatus & CMDSTATUS_ACTIVE_MASK); + rte_spinlock_unlock(&idxd->u.pci->lk); + + return err_code & CMDSTATUS_ERR_MASK; +} + +static uint32_t * +idxd_get_wq_cfg(struct idxd_pci_common *pci, uint8_t wq_idx) +{ + return RTE_PTR_ADD(pci->wq_regs_base, + (uintptr_t)wq_idx << (5 + pci->wq_cfg_sz)); +} + +static int +idxd_is_wq_enabled(struct idxd_dmadev *idxd) +{ + uint32_t state = idxd_get_wq_cfg(idxd->u.pci, idxd->qid)[wq_state_idx]; + return ((state >> WQ_STATE_SHIFT) & WQ_STATE_MASK) == 0x1; +} + +static int +idxd_pci_dev_close(struct rte_dma_dev *dev) +{ + struct idxd_dmadev *idxd = dev->dev_private; + uint8_t err_code; + + /* disable the device */ + err_code = idxd_pci_dev_command(idxd, idxd_disable_dev); + if (err_code) { + IDXD_PMD_ERR("Error disabling device: code %#x", err_code); + return err_code; + } + IDXD_PMD_DEBUG("IDXD Device disabled OK"); + + /* free device memory */ + IDXD_PMD_DEBUG("Freeing device driver memory"); + rte_free(idxd->batch_idx_ring); + rte_free(idxd->desc_ring); + + return 0; +} + +static const struct rte_dma_dev_ops idxd_pci_ops = { + .dev_close = idxd_pci_dev_close, +}; + +/* each portal uses 4 x 4k pages */ +#define IDXD_PORTAL_SIZE (4096 * 4) + +static int +init_pci_device(struct rte_pci_device *dev, struct idxd_dmadev *idxd, + unsigned int max_queues) +{ + struct idxd_pci_common *pci; + uint8_t nb_groups, nb_engines, nb_wqs; + uint16_t grp_offset, wq_offset; /* how far into bar0 the regs are */ + uint16_t wq_size, total_wq_size; + uint8_t lg2_max_batch, lg2_max_copy_size; + unsigned int i, err_code; + + pci = malloc(sizeof(*pci)); + if (pci == NULL) { + IDXD_PMD_ERR("%s: Can't allocate memory", __func__); + goto err; + } + rte_spinlock_init(&pci->lk); + + /* assign the bar registers, and then configure device */ + pci->regs = dev->mem_resource[0].addr; + grp_offset = (uint16_t)pci->regs->offsets[0]; + pci->grp_regs = RTE_PTR_ADD(pci->regs, grp_offset * 0x100); + wq_offset = (uint16_t)(pci->regs->offsets[0] >> 16); + pci->wq_regs_base = RTE_PTR_ADD(pci->regs, wq_offset * 0x100); + pci->portals = dev->mem_resource[2].addr; + pci->wq_cfg_sz = (pci->regs->wqcap >> 24) & 0x0F; + + /* sanity check device status */ + if (pci->regs->gensts & GENSTS_DEV_STATE_MASK) { + /* need function-level-reset (FLR) or is enabled */ + IDXD_PMD_ERR("Device status is not disabled, cannot init"); + goto err; + } + if (pci->regs->cmdstatus & CMDSTATUS_ACTIVE_MASK) { + /* command in progress */ + IDXD_PMD_ERR("Device has a command in progress, cannot init"); + goto err; + } + + /* read basic info about the hardware for use when configuring */ + nb_groups = (uint8_t)pci->regs->grpcap; + nb_engines = (uint8_t)pci->regs->engcap; + nb_wqs = (uint8_t)(pci->regs->wqcap >> 16); + total_wq_size = (uint16_t)pci->regs->wqcap; + lg2_max_copy_size = (uint8_t)(pci->regs->gencap >> 16) & 0x1F; + lg2_max_batch = (uint8_t)(pci->regs->gencap >> 21) & 0x0F; + + IDXD_PMD_DEBUG("nb_groups = %u, nb_engines = %u, nb_wqs = %u", + nb_groups, nb_engines, nb_wqs); + + /* zero out any old config */ + for (i = 0; i < nb_groups; i++) { + pci->grp_regs[i].grpengcfg = 0; + pci->grp_regs[i].grpwqcfg[0] = 0; + } + for (i = 0; i < nb_wqs; i++) + idxd_get_wq_cfg(pci, i)[0] = 0; + + /* limit queues if necessary */ + if (max_queues != 0 && nb_wqs > max_queues) { + nb_wqs = max_queues; + if (nb_engines > max_queues) + nb_engines = max_queues; + if (nb_groups > max_queues) + nb_engines = max_queues; + IDXD_PMD_DEBUG("Limiting queues to %u", nb_wqs); + } + + /* put each engine into a separate group to avoid reordering */ + if (nb_groups > nb_engines) + nb_groups = nb_engines; + if (nb_groups < nb_engines) + nb_engines = nb_groups; + + /* assign engines to groups, round-robin style */ + for (i = 0; i < nb_engines; i++) { + IDXD_PMD_DEBUG("Assigning engine %u to group %u", + i, i % nb_groups); + pci->grp_regs[i % nb_groups].grpengcfg |= (1ULL << i); + } + + /* now do the same for queues and give work slots to each queue */ + wq_size = total_wq_size / nb_wqs; + IDXD_PMD_DEBUG("Work queue size = %u, max batch = 2^%u, max copy = 2^%u", + wq_size, lg2_max_batch, lg2_max_copy_size); + for (i = 0; i < nb_wqs; i++) { + /* add engine "i" to a group */ + IDXD_PMD_DEBUG("Assigning work queue %u to group %u", + i, i % nb_groups); + pci->grp_regs[i % nb_groups].grpwqcfg[0] |= (1ULL << i); + /* now configure it, in terms of size, max batch, mode */ + idxd_get_wq_cfg(pci, i)[wq_size_idx] = wq_size; + idxd_get_wq_cfg(pci, i)[wq_mode_idx] = (1 << WQ_PRIORITY_SHIFT) | + WQ_MODE_DEDICATED; + idxd_get_wq_cfg(pci, i)[wq_sizes_idx] = lg2_max_copy_size | + (lg2_max_batch << WQ_BATCH_SZ_SHIFT); + } + + /* dump the group configuration to output */ + for (i = 0; i < nb_groups; i++) { + IDXD_PMD_DEBUG("## Group %d", i); + IDXD_PMD_DEBUG(" GRPWQCFG: %"PRIx64, pci->grp_regs[i].grpwqcfg[0]); + IDXD_PMD_DEBUG(" GRPENGCFG: %"PRIx64, pci->grp_regs[i].grpengcfg); + IDXD_PMD_DEBUG(" GRPFLAGS: %"PRIx32, pci->grp_regs[i].grpflags); + } + + idxd->u.pci = pci; + idxd->max_batches = wq_size; + + /* enable the device itself */ + err_code = idxd_pci_dev_command(idxd, idxd_enable_dev); + if (err_code) { + IDXD_PMD_ERR("Error enabling device: code %#x", err_code); + return err_code; + } + IDXD_PMD_DEBUG("IDXD Device enabled OK"); + + return nb_wqs; + +err: + free(pci); + return -1; +} + static int idxd_dmadev_probe_pci(struct rte_pci_driver *drv, struct rte_pci_device *dev) { - int ret = 0; + struct idxd_dmadev idxd = {0}; + uint8_t nb_wqs; + int qid, ret = 0; char name[PCI_PRI_STR_SIZE]; + unsigned int max_queues = 0; rte_pci_device_name(&dev->addr, name, sizeof(name)); IDXD_PMD_INFO("Init %s on NUMA node %d", name, dev->device.numa_node); dev->device.driver = &drv->driver; - return ret; + if (dev->device.devargs && dev->device.devargs->args[0] != '\0') { + /* if the number of devargs grows beyond just 1, use rte_kvargs */ + if (sscanf(dev->device.devargs->args, + "max_queues=%u", &max_queues) != 1) { + IDXD_PMD_ERR("Invalid device parameter: '%s'", + dev->device.devargs->args); + return -1; + } + } + + ret = init_pci_device(dev, &idxd, max_queues); + if (ret < 0) { + IDXD_PMD_ERR("Error initializing PCI hardware"); + return ret; + } + if (idxd.u.pci->portals == NULL) { + IDXD_PMD_ERR("Error, invalid portal assigned during initialization\n"); + return -EINVAL; + } + nb_wqs = (uint8_t)ret; + + /* set up one device for each queue */ + for (qid = 0; qid < nb_wqs; qid++) { + char qname[32]; + + /* add the queue number to each device name */ + snprintf(qname, sizeof(qname), "%s-q%d", name, qid); + idxd.qid = qid; + idxd.portal = RTE_PTR_ADD(idxd.u.pci->portals, + qid * IDXD_PORTAL_SIZE); + if (idxd_is_wq_enabled(&idxd)) + IDXD_PMD_ERR("Error, WQ %u seems enabled", qid); + ret = idxd_dmadev_create(qname, &dev->device, + &idxd, &idxd_pci_ops); + if (ret != 0) { + IDXD_PMD_ERR("Failed to create dmadev %s", name); + if (qid == 0) /* if no devices using this, free pci */ + free(idxd.u.pci); + return ret; + } + } + + return 0; +} + +static int +idxd_dmadev_destroy(const char *name) +{ + int ret; + struct rte_dma_dev *dmadev; + struct idxd_dmadev *idxd; + int dev_id = rte_dma_get_dev_id(name); + + if (!name) { + IDXD_PMD_ERR("Invalid device name"); + return -EINVAL; + } + + if (dev_id < 0) { + IDXD_PMD_ERR("Invalid device ID"); + return -EINVAL; + } + + dmadev = &rte_dma_devices[dev_id]; + if (!dmadev) { + IDXD_PMD_ERR("Invalid device name (%s)", name); + return -EINVAL; + } + + idxd = dmadev->dev_private; + if (!idxd) { + IDXD_PMD_ERR("Error getting dev_private"); + return -EINVAL; + } + + /* rte_dma_close is called by pmd_release */ + ret = rte_dma_pmd_release(name); + if (ret) + IDXD_PMD_DEBUG("Device cleanup failed"); + + return 0; } static int @@ -39,7 +318,7 @@ idxd_dmadev_remove_pci(struct rte_pci_device *dev) IDXD_PMD_INFO("Closing %s on NUMA node %d", name, dev->device.numa_node); - return 0; + return idxd_dmadev_destroy(name); } struct rte_pci_driver idxd_pmd_drv_pci = { From patchwork Fri Sep 24 13:39:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99598 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8447DA0548; Fri, 24 Sep 2021 15:40:05 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1570341333; Fri, 24 Sep 2021 15:39:40 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 300BE41322 for ; Fri, 24 Sep 2021 15:39:38 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10116"; a="285084284" X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="285084284" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2021 06:39:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="515527644" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by fmsmga008.fm.intel.com with ESMTP; 24 Sep 2021 06:39:34 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 24 Sep 2021 13:39:06 +0000 Message-Id: <20210924133916.4042773-7-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210924133916.4042773-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210924133916.4042773-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v6 06/16] dma/idxd: add datapath structures X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add data structures required for the data path for IDXD devices. Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh --- v2: add completion status for invalid opcode --- drivers/dma/idxd/idxd_bus.c | 1 + drivers/dma/idxd/idxd_common.c | 33 ++++++++++++++++++ drivers/dma/idxd/idxd_hw_defs.h | 60 ++++++++++++++++++++++++++++++++ drivers/dma/idxd/idxd_internal.h | 3 ++ drivers/dma/idxd/idxd_pci.c | 1 + 5 files changed, 98 insertions(+) diff --git a/drivers/dma/idxd/idxd_bus.c b/drivers/dma/idxd/idxd_bus.c index b48fa954ed..3c0837ec52 100644 --- a/drivers/dma/idxd/idxd_bus.c +++ b/drivers/dma/idxd/idxd_bus.c @@ -95,6 +95,7 @@ idxd_dev_close(struct rte_dma_dev *dev) static const struct rte_dma_dev_ops idxd_bus_ops = { .dev_close = idxd_dev_close, + .dev_dump = idxd_dump, }; static void * diff --git a/drivers/dma/idxd/idxd_common.c b/drivers/dma/idxd/idxd_common.c index 130153f7d2..b285fda65b 100644 --- a/drivers/dma/idxd/idxd_common.c +++ b/drivers/dma/idxd/idxd_common.c @@ -10,6 +10,35 @@ #define IDXD_PMD_NAME_STR "dmadev_idxd" +int +idxd_dump(const struct rte_dma_dev *dev, FILE *f) +{ + struct idxd_dmadev *idxd = dev->dev_private; + unsigned int i; + + fprintf(f, "== IDXD Private Data ==\n"); + fprintf(f, " Portal: %p\n", idxd->portal); + fprintf(f, " Config: { ring_size: %u }\n", + idxd->qcfg.nb_desc); + fprintf(f, " Batch ring (sz = %u, max_batches = %u):\n\t", + idxd->max_batches + 1, idxd->max_batches); + for (i = 0; i <= idxd->max_batches; i++) { + fprintf(f, " %u ", idxd->batch_idx_ring[i]); + if (i == idxd->batch_idx_read && i == idxd->batch_idx_write) + fprintf(f, "[rd ptr, wr ptr] "); + else if (i == idxd->batch_idx_read) + fprintf(f, "[rd ptr] "); + else if (i == idxd->batch_idx_write) + fprintf(f, "[wr ptr] "); + if (i == idxd->max_batches) + fprintf(f, "\n"); + } + + fprintf(f, " Curr batch: start = %u, size = %u\n", idxd->batch_start, idxd->batch_size); + fprintf(f, " IDS: avail = %u, returned: %u\n", idxd->ids_avail, idxd->ids_returned); + return 0; +} + int idxd_dmadev_create(const char *name, struct rte_device *dev, const struct idxd_dmadev *base_idxd, @@ -19,6 +48,10 @@ idxd_dmadev_create(const char *name, struct rte_device *dev, struct rte_dma_dev *dmadev = NULL; int ret = 0; + RTE_BUILD_BUG_ON(sizeof(struct idxd_hw_desc) != 64); + RTE_BUILD_BUG_ON(offsetof(struct idxd_hw_desc, size) != 32); + RTE_BUILD_BUG_ON(sizeof(struct idxd_completion) != 32); + if (!name) { IDXD_PMD_ERR("Invalid name of the device!"); ret = -EINVAL; diff --git a/drivers/dma/idxd/idxd_hw_defs.h b/drivers/dma/idxd/idxd_hw_defs.h index ea627cba6d..55ca9f7f52 100644 --- a/drivers/dma/idxd/idxd_hw_defs.h +++ b/drivers/dma/idxd/idxd_hw_defs.h @@ -5,6 +5,66 @@ #ifndef _IDXD_HW_DEFS_H_ #define _IDXD_HW_DEFS_H_ +/* + * Defines used in the data path for interacting with IDXD hardware. + */ +#define IDXD_CMD_OP_SHIFT 24 +enum rte_idxd_ops { + idxd_op_nop = 0, + idxd_op_batch, + idxd_op_drain, + idxd_op_memmove, + idxd_op_fill +}; + +#define IDXD_FLAG_FENCE (1 << 0) +#define IDXD_FLAG_COMPLETION_ADDR_VALID (1 << 2) +#define IDXD_FLAG_REQUEST_COMPLETION (1 << 3) +#define IDXD_FLAG_CACHE_CONTROL (1 << 8) + +/** + * Hardware descriptor used by DSA hardware, for both bursts and + * for individual operations. + */ +struct idxd_hw_desc { + uint32_t pasid; + uint32_t op_flags; + rte_iova_t completion; + + RTE_STD_C11 + union { + rte_iova_t src; /* source address for copy ops etc. */ + rte_iova_t desc_addr; /* descriptor pointer for batch */ + }; + rte_iova_t dst; + + uint32_t size; /* length of data for op, or batch size */ + + uint16_t intr_handle; /* completion interrupt handle */ + + /* remaining 26 bytes are reserved */ + uint16_t __reserved[13]; +} __rte_aligned(64); + +#define IDXD_COMP_STATUS_INCOMPLETE 0 +#define IDXD_COMP_STATUS_SUCCESS 1 +#define IDXD_COMP_STATUS_INVALID_OPCODE 0x10 +#define IDXD_COMP_STATUS_INVALID_SIZE 0x13 +#define IDXD_COMP_STATUS_SKIPPED 0xFF /* not official IDXD error, needed as placeholder */ + +/** + * Completion record structure written back by DSA + */ +struct idxd_completion { + uint8_t status; + uint8_t result; + /* 16-bits pad here */ + uint32_t completed_size; /* data length, or descriptors for batch */ + + rte_iova_t fault_address; + uint32_t invalid_flags; +} __rte_aligned(32); + /*** Definitions for Intel(R) Data Streaming Accelerator ***/ #define IDXD_CMD_SHIFT 20 diff --git a/drivers/dma/idxd/idxd_internal.h b/drivers/dma/idxd/idxd_internal.h index cb3a68c69b..99c8e04302 100644 --- a/drivers/dma/idxd/idxd_internal.h +++ b/drivers/dma/idxd/idxd_internal.h @@ -39,6 +39,8 @@ struct idxd_pci_common { }; struct idxd_dmadev { + struct idxd_hw_desc *desc_ring; + /* counters to track the batches */ unsigned short max_batches; unsigned short batch_idx_read; @@ -79,5 +81,6 @@ struct idxd_dmadev { int idxd_dmadev_create(const char *name, struct rte_device *dev, const struct idxd_dmadev *base_idxd, const struct rte_dma_dev_ops *ops); +int idxd_dump(const struct rte_dma_dev *dev, FILE *f); #endif /* _IDXD_INTERNAL_H_ */ diff --git a/drivers/dma/idxd/idxd_pci.c b/drivers/dma/idxd/idxd_pci.c index 0c03a51449..add241d172 100644 --- a/drivers/dma/idxd/idxd_pci.c +++ b/drivers/dma/idxd/idxd_pci.c @@ -83,6 +83,7 @@ idxd_pci_dev_close(struct rte_dma_dev *dev) static const struct rte_dma_dev_ops idxd_pci_ops = { .dev_close = idxd_pci_dev_close, + .dev_dump = idxd_dump, }; /* each portal uses 4 x 4k pages */ From patchwork Fri Sep 24 13:39:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99599 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B8938A0548; Fri, 24 Sep 2021 15:40:11 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 212F441339; Fri, 24 Sep 2021 15:39:42 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 1C8BF41334 for ; Fri, 24 Sep 2021 15:39:39 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10116"; a="285084288" X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="285084288" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2021 06:39:39 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="515527660" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by fmsmga008.fm.intel.com with ESMTP; 24 Sep 2021 06:39:37 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 24 Sep 2021 13:39:07 +0000 Message-Id: <20210924133916.4042773-8-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210924133916.4042773-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210924133916.4042773-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v6 07/16] dma/idxd: add configure and info_get functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add functions for device configuration. The info_get function is included here since it can be useful for checking successful configuration. Documentation is also updated to add device configuration usage info. Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh --- v2: - fix reconfigure bug in idxd_vchan_setup() - add literal include comment for the docs to pick up v3: - fixes needed after changes from rebasing v6: - update doc to reference library documentation to remove duplication - remove nb_vchans from info_get() since the lib fills it - add error handling capability flag to info_get --- doc/guides/dmadevs/idxd.rst | 19 +++++++++ doc/guides/prog_guide/dmadev.rst | 4 ++ drivers/dma/idxd/idxd_bus.c | 3 ++ drivers/dma/idxd/idxd_common.c | 71 ++++++++++++++++++++++++++++++++ drivers/dma/idxd/idxd_internal.h | 6 +++ drivers/dma/idxd/idxd_pci.c | 3 ++ 6 files changed, 106 insertions(+) diff --git a/doc/guides/dmadevs/idxd.rst b/doc/guides/dmadevs/idxd.rst index ce33e2857a..42efd59594 100644 --- a/doc/guides/dmadevs/idxd.rst +++ b/doc/guides/dmadevs/idxd.rst @@ -120,3 +120,22 @@ use a subset of configured queues. Once probed successfully, irrespective of kernel driver, the device will appear as a ``dmadev``, that is a "DMA device type" inside DPDK, and can be accessed using APIs from the ``rte_dmadev`` library. + +Using IDXD DMAdev Devices +-------------------------- + +To use the devices from an application, the dmadev API can be used. + +Device Configuration +~~~~~~~~~~~~~~~~~~~~~ + +Refer to the :ref:`Device Configuration ` and +:ref:`Configuration of Virtual DMA Channels ` sections +of the dmadev library documentation for details on device configuration API usage. + +IDXD configuration requirements: + +* ``ring_size`` must be a power of two, between 64 and 4096. +* Only one ``vchan`` is supported per device (work queue). +* IDXD devices do not support silent mode. +* The transfer direction must be set to ``RTE_DMA_DIR_MEM_TO_MEM`` to copy from memory to memory. diff --git a/doc/guides/prog_guide/dmadev.rst b/doc/guides/prog_guide/dmadev.rst index de8b599d96..3612315325 100644 --- a/doc/guides/prog_guide/dmadev.rst +++ b/doc/guides/prog_guide/dmadev.rst @@ -63,6 +63,8 @@ identifiers: - A device name used to designate the DMA device in console messages, for administration or debugging purposes. +.. _dmadev_device_configuration: + Device Configuration ~~~~~~~~~~~~~~~~~~~~ @@ -79,6 +81,8 @@ for the DMA device for example the number of virtual DMA channels to set up, indication of whether to enable silent mode. +.. _dmadev_vchan_configuration: + Configuration of Virtual DMA Channels ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/drivers/dma/idxd/idxd_bus.c b/drivers/dma/idxd/idxd_bus.c index 3c0837ec52..b2acdac4f9 100644 --- a/drivers/dma/idxd/idxd_bus.c +++ b/drivers/dma/idxd/idxd_bus.c @@ -96,6 +96,9 @@ idxd_dev_close(struct rte_dma_dev *dev) static const struct rte_dma_dev_ops idxd_bus_ops = { .dev_close = idxd_dev_close, .dev_dump = idxd_dump, + .dev_configure = idxd_configure, + .vchan_setup = idxd_vchan_setup, + .dev_info_get = idxd_info_get, }; static void * diff --git a/drivers/dma/idxd/idxd_common.c b/drivers/dma/idxd/idxd_common.c index b285fda65b..32ddb5f7f8 100644 --- a/drivers/dma/idxd/idxd_common.c +++ b/drivers/dma/idxd/idxd_common.c @@ -39,6 +39,77 @@ idxd_dump(const struct rte_dma_dev *dev, FILE *f) return 0; } +int +idxd_info_get(const struct rte_dma_dev *dev, struct rte_dma_info *info, uint32_t size) +{ + struct idxd_dmadev *idxd = dev->dev_private; + + if (size < sizeof(*info)) + return -EINVAL; + + *info = (struct rte_dma_info) { + .dev_capa = RTE_DMA_CAPA_MEM_TO_MEM | RTE_DMA_CAPA_HANDLES_ERRORS | + RTE_DMA_CAPA_OPS_COPY | RTE_DMA_CAPA_OPS_FILL, + .max_vchans = 1, + .max_desc = 4096, + .min_desc = 64, + }; + if (idxd->sva_support) + info->dev_capa |= RTE_DMA_CAPA_SVA; + return 0; +} + +int +idxd_configure(struct rte_dma_dev *dev __rte_unused, const struct rte_dma_conf *dev_conf, + uint32_t conf_sz) +{ + if (sizeof(struct rte_dma_conf) != conf_sz) + return -EINVAL; + + if (dev_conf->nb_vchans != 1) + return -EINVAL; + return 0; +} + +int +idxd_vchan_setup(struct rte_dma_dev *dev, uint16_t vchan __rte_unused, + const struct rte_dma_vchan_conf *qconf, uint32_t qconf_sz) +{ + struct idxd_dmadev *idxd = dev->dev_private; + uint16_t max_desc = qconf->nb_desc; + + if (sizeof(struct rte_dma_vchan_conf) != qconf_sz) + return -EINVAL; + + idxd->qcfg = *qconf; + + if (!rte_is_power_of_2(max_desc)) + max_desc = rte_align32pow2(max_desc); + IDXD_PMD_DEBUG("DMA dev %u using %u descriptors", dev->data->dev_id, max_desc); + idxd->desc_ring_mask = max_desc - 1; + idxd->qcfg.nb_desc = max_desc; + + /* in case we are reconfiguring a device, free any existing memory */ + rte_free(idxd->desc_ring); + + /* allocate the descriptor ring at 2x size as batches can't wrap */ + idxd->desc_ring = rte_zmalloc(NULL, sizeof(*idxd->desc_ring) * max_desc * 2, 0); + if (idxd->desc_ring == NULL) + return -ENOMEM; + idxd->desc_iova = rte_mem_virt2iova(idxd->desc_ring); + + idxd->batch_idx_read = 0; + idxd->batch_idx_write = 0; + idxd->batch_start = 0; + idxd->batch_size = 0; + idxd->ids_returned = 0; + idxd->ids_avail = 0; + + memset(idxd->batch_comp_ring, 0, sizeof(*idxd->batch_comp_ring) * + (idxd->max_batches + 1)); + return 0; +} + int idxd_dmadev_create(const char *name, struct rte_device *dev, const struct idxd_dmadev *base_idxd, diff --git a/drivers/dma/idxd/idxd_internal.h b/drivers/dma/idxd/idxd_internal.h index 99c8e04302..fdd018ca35 100644 --- a/drivers/dma/idxd/idxd_internal.h +++ b/drivers/dma/idxd/idxd_internal.h @@ -82,5 +82,11 @@ struct idxd_dmadev { int idxd_dmadev_create(const char *name, struct rte_device *dev, const struct idxd_dmadev *base_idxd, const struct rte_dma_dev_ops *ops); int idxd_dump(const struct rte_dma_dev *dev, FILE *f); +int idxd_configure(struct rte_dma_dev *dev, const struct rte_dma_conf *dev_conf, + uint32_t conf_sz); +int idxd_vchan_setup(struct rte_dma_dev *dev, uint16_t vchan, + const struct rte_dma_vchan_conf *qconf, uint32_t qconf_sz); +int idxd_info_get(const struct rte_dma_dev *dev, struct rte_dma_info *dev_info, + uint32_t size); #endif /* _IDXD_INTERNAL_H_ */ diff --git a/drivers/dma/idxd/idxd_pci.c b/drivers/dma/idxd/idxd_pci.c index add241d172..0ac5e5f30a 100644 --- a/drivers/dma/idxd/idxd_pci.c +++ b/drivers/dma/idxd/idxd_pci.c @@ -84,6 +84,9 @@ idxd_pci_dev_close(struct rte_dma_dev *dev) static const struct rte_dma_dev_ops idxd_pci_ops = { .dev_close = idxd_pci_dev_close, .dev_dump = idxd_dump, + .dev_configure = idxd_configure, + .vchan_setup = idxd_vchan_setup, + .dev_info_get = idxd_info_get, }; /* each portal uses 4 x 4k pages */ From patchwork Fri Sep 24 13:39:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99600 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 56CE9A0548; Fri, 24 Sep 2021 15:40:17 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 329F641327; Fri, 24 Sep 2021 15:39:44 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id BD20441323 for ; Fri, 24 Sep 2021 15:39:42 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10116"; a="285084296" X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="285084296" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2021 06:39:42 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="515527679" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by fmsmga008.fm.intel.com with ESMTP; 24 Sep 2021 06:39:39 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 24 Sep 2021 13:39:08 +0000 Message-Id: <20210924133916.4042773-9-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210924133916.4042773-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210924133916.4042773-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v6 08/16] dma/idxd: add start and stop functions for pci devices X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add device start/stop functions for DSA devices bound to vfio. For devices bound to the IDXD kernel driver, these are not required since the IDXD kernel driver takes care of this. Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh --- v6: fix return values of start and stop functions --- doc/guides/dmadevs/idxd.rst | 3 +++ drivers/dma/idxd/idxd_pci.c | 51 +++++++++++++++++++++++++++++++++++++ 2 files changed, 54 insertions(+) diff --git a/doc/guides/dmadevs/idxd.rst b/doc/guides/dmadevs/idxd.rst index 42efd59594..da5e51bfa7 100644 --- a/doc/guides/dmadevs/idxd.rst +++ b/doc/guides/dmadevs/idxd.rst @@ -139,3 +139,6 @@ IDXD configuration requirements: * Only one ``vchan`` is supported per device (work queue). * IDXD devices do not support silent mode. * The transfer direction must be set to ``RTE_DMA_DIR_MEM_TO_MEM`` to copy from memory to memory. + +Once configured, the device can then be made ready for use by calling the +``rte_dma_start()`` API. diff --git a/drivers/dma/idxd/idxd_pci.c b/drivers/dma/idxd/idxd_pci.c index 0ac5e5f30a..86a033862b 100644 --- a/drivers/dma/idxd/idxd_pci.c +++ b/drivers/dma/idxd/idxd_pci.c @@ -59,6 +59,55 @@ idxd_is_wq_enabled(struct idxd_dmadev *idxd) return ((state >> WQ_STATE_SHIFT) & WQ_STATE_MASK) == 0x1; } +static int +idxd_pci_dev_stop(struct rte_dma_dev *dev) +{ + struct idxd_dmadev *idxd = dev->dev_private; + uint8_t err_code; + + if (!idxd_is_wq_enabled(idxd)) { + IDXD_PMD_ERR("Work queue %d already disabled", idxd->qid); + return -EALREADY; + } + + err_code = idxd_pci_dev_command(idxd, idxd_disable_wq); + if (err_code || idxd_is_wq_enabled(idxd)) { + IDXD_PMD_ERR("Failed disabling work queue %d, error code: %#x", + idxd->qid, err_code); + return err_code == 0 ? -1 : -err_code; + } + IDXD_PMD_DEBUG("Work queue %d disabled OK", idxd->qid); + + return 0; +} + +static int +idxd_pci_dev_start(struct rte_dma_dev *dev) +{ + struct idxd_dmadev *idxd = dev->dev_private; + uint8_t err_code; + + if (idxd_is_wq_enabled(idxd)) { + IDXD_PMD_WARN("WQ %d already enabled", idxd->qid); + return 0; + } + + if (idxd->desc_ring == NULL) { + IDXD_PMD_ERR("WQ %d has not been fully configured", idxd->qid); + return -EINVAL; + } + + err_code = idxd_pci_dev_command(idxd, idxd_enable_wq); + if (err_code || !idxd_is_wq_enabled(idxd)) { + IDXD_PMD_ERR("Failed enabling work queue %d, error code: %#x", + idxd->qid, err_code); + return err_code == 0 ? -1 : -err_code; + } + IDXD_PMD_DEBUG("Work queue %d enabled OK", idxd->qid); + + return 0; +} + static int idxd_pci_dev_close(struct rte_dma_dev *dev) { @@ -87,6 +136,8 @@ static const struct rte_dma_dev_ops idxd_pci_ops = { .dev_configure = idxd_configure, .vchan_setup = idxd_vchan_setup, .dev_info_get = idxd_info_get, + .dev_start = idxd_pci_dev_start, + .dev_stop = idxd_pci_dev_stop, }; /* each portal uses 4 x 4k pages */ From patchwork Fri Sep 24 13:39:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99601 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E9061A0548; Fri, 24 Sep 2021 15:40:24 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 91A4741341; Fri, 24 Sep 2021 15:39:47 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id D476641348 for ; Fri, 24 Sep 2021 15:39:45 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10116"; a="285084307" X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="285084307" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2021 06:39:45 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="515527695" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by fmsmga008.fm.intel.com with ESMTP; 24 Sep 2021 06:39:42 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 24 Sep 2021 13:39:09 +0000 Message-Id: <20210924133916.4042773-10-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210924133916.4042773-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210924133916.4042773-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v6 09/16] dma/idxd: add data-path job submission functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add data path functions for enqueuing and submitting operations to DSA devices. Documentation updates are included for dmadev library and IDXD driver docs as appropriate. Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh --- v6: - add references to dmadev lib docs for generic info - fix return values in "__idxd_write_desc()" --- doc/guides/dmadevs/idxd.rst | 9 ++ doc/guides/prog_guide/dmadev.rst | 19 +++++ drivers/dma/idxd/idxd_common.c | 136 +++++++++++++++++++++++++++++++ drivers/dma/idxd/idxd_internal.h | 5 ++ drivers/dma/idxd/meson.build | 1 + 5 files changed, 170 insertions(+) diff --git a/doc/guides/dmadevs/idxd.rst b/doc/guides/dmadevs/idxd.rst index da5e51bfa7..b3d78482be 100644 --- a/doc/guides/dmadevs/idxd.rst +++ b/doc/guides/dmadevs/idxd.rst @@ -142,3 +142,12 @@ IDXD configuration requirements: Once configured, the device can then be made ready for use by calling the ``rte_dma_start()`` API. + +Performing Data Copies +~~~~~~~~~~~~~~~~~~~~~~~ + +Refer to the :ref:`Enqueue / Dequeue APIs ` section of the dmadev library +documentation for details on operation enqueue and submission API usage. + +It is expected that, for efficiency reasons, a burst of operations will be enqueued to the +device via multiple enqueue calls between calls to the ``rte_dma_submit()`` function. diff --git a/doc/guides/prog_guide/dmadev.rst b/doc/guides/prog_guide/dmadev.rst index 3612315325..4908e33762 100644 --- a/doc/guides/prog_guide/dmadev.rst +++ b/doc/guides/prog_guide/dmadev.rst @@ -108,6 +108,8 @@ can be used to get the device info and supported features. Silent mode is a special device capability which does not require the application to invoke dequeue APIs. +.. _dmadev_enqueue_dequeue: + Enqueue / Dequeue APIs ~~~~~~~~~~~~~~~~~~~~~~ @@ -121,6 +123,23 @@ The ``rte_dma_submit`` API is used to issue doorbell to hardware. Alternatively the ``RTE_DMA_OP_FLAG_SUBMIT`` flag can be passed to the enqueue APIs to also issue the doorbell to hardware. +The following code demonstrates how to enqueue a burst of copies to the +device and start the hardware processing of them: + +.. code-block:: C + + struct rte_mbuf *srcs[DMA_BURST_SZ], *dsts[DMA_BURST_SZ]; + unsigned int i; + + for (i = 0; i < RTE_DIM(srcs); i++) { + if (rte_dma_copy(dev_id, vchan, rte_pktmbuf_iova(srcs), + rte_pktmbuf_iova(dsts), COPY_LEN, 0) < 0) { + PRINT_ERR("Error with rte_dma_copy for buffer %u\n", i); + return -1; + } + } + rte_dma_submit(dev_id, vchan); + There are two dequeue APIs ``rte_dma_completed`` and ``rte_dma_completed_status``, these are used to obtain the results of the enqueue requests. ``rte_dma_completed`` will return the number of successfully diff --git a/drivers/dma/idxd/idxd_common.c b/drivers/dma/idxd/idxd_common.c index 32ddb5f7f8..1580f5029c 100644 --- a/drivers/dma/idxd/idxd_common.c +++ b/drivers/dma/idxd/idxd_common.c @@ -2,14 +2,144 @@ * Copyright 2021 Intel Corporation */ +#include + #include #include #include +#include #include "idxd_internal.h" #define IDXD_PMD_NAME_STR "dmadev_idxd" +static __rte_always_inline rte_iova_t +__desc_idx_to_iova(struct idxd_dmadev *idxd, uint16_t n) +{ + return idxd->desc_iova + (n * sizeof(struct idxd_hw_desc)); +} + +static __rte_always_inline void +__idxd_movdir64b(volatile void *dst, const struct idxd_hw_desc *src) +{ + asm volatile (".byte 0x66, 0x0f, 0x38, 0xf8, 0x02" + : + : "a" (dst), "d" (src) + : "memory"); +} + +static __rte_always_inline void +__submit(struct idxd_dmadev *idxd) +{ + rte_prefetch1(&idxd->batch_comp_ring[idxd->batch_idx_read]); + + if (idxd->batch_size == 0) + return; + + /* write completion to batch comp ring */ + rte_iova_t comp_addr = idxd->batch_iova + + (idxd->batch_idx_write * sizeof(struct idxd_completion)); + + if (idxd->batch_size == 1) { + /* submit batch directly */ + struct idxd_hw_desc desc = + idxd->desc_ring[idxd->batch_start & idxd->desc_ring_mask]; + desc.completion = comp_addr; + desc.op_flags |= IDXD_FLAG_REQUEST_COMPLETION; + _mm_sfence(); /* fence before writing desc to device */ + __idxd_movdir64b(idxd->portal, &desc); + } else { + const struct idxd_hw_desc batch_desc = { + .op_flags = (idxd_op_batch << IDXD_CMD_OP_SHIFT) | + IDXD_FLAG_COMPLETION_ADDR_VALID | + IDXD_FLAG_REQUEST_COMPLETION, + .desc_addr = __desc_idx_to_iova(idxd, + idxd->batch_start & idxd->desc_ring_mask), + .completion = comp_addr, + .size = idxd->batch_size, + }; + _mm_sfence(); /* fence before writing desc to device */ + __idxd_movdir64b(idxd->portal, &batch_desc); + } + + if (++idxd->batch_idx_write > idxd->max_batches) + idxd->batch_idx_write = 0; + + idxd->batch_start += idxd->batch_size; + idxd->batch_size = 0; + idxd->batch_idx_ring[idxd->batch_idx_write] = idxd->batch_start; + _mm256_store_si256((void *)&idxd->batch_comp_ring[idxd->batch_idx_write], + _mm256_setzero_si256()); +} + +static __rte_always_inline int +__idxd_write_desc(struct rte_dma_dev *dev, + const uint32_t op_flags, + const rte_iova_t src, + const rte_iova_t dst, + const uint32_t size, + const uint32_t flags) +{ + struct idxd_dmadev *idxd = dev->dev_private; + uint16_t mask = idxd->desc_ring_mask; + uint16_t job_id = idxd->batch_start + idxd->batch_size; + /* we never wrap batches, so we only mask the start and allow start+size to overflow */ + uint16_t write_idx = (idxd->batch_start & mask) + idxd->batch_size; + + /* first check batch ring space then desc ring space */ + if ((idxd->batch_idx_read == 0 && idxd->batch_idx_write == idxd->max_batches) || + idxd->batch_idx_write + 1 == idxd->batch_idx_read) + return -ENOSPC; + if (((write_idx + 1) & mask) == (idxd->ids_returned & mask)) + return -ENOSPC; + + /* write desc. Note: descriptors don't wrap, but the completion address does */ + const uint64_t op_flags64 = (uint64_t)(op_flags | IDXD_FLAG_COMPLETION_ADDR_VALID) << 32; + const uint64_t comp_addr = __desc_idx_to_iova(idxd, write_idx & mask); + _mm256_store_si256((void *)&idxd->desc_ring[write_idx], + _mm256_set_epi64x(dst, src, comp_addr, op_flags64)); + _mm256_store_si256((void *)&idxd->desc_ring[write_idx].size, + _mm256_set_epi64x(0, 0, 0, size)); + + idxd->batch_size++; + + rte_prefetch0_write(&idxd->desc_ring[write_idx + 1]); + + if (flags & RTE_DMA_OP_FLAG_SUBMIT) + __submit(idxd); + + return job_id; +} + +int +idxd_enqueue_copy(struct rte_dma_dev *dev, uint16_t qid __rte_unused, rte_iova_t src, + rte_iova_t dst, unsigned int length, uint64_t flags) +{ + /* we can take advantage of the fact that the fence flag in dmadev and DSA are the same, + * but check it at compile time to be sure. + */ + RTE_BUILD_BUG_ON(RTE_DMA_OP_FLAG_FENCE != IDXD_FLAG_FENCE); + uint32_t memmove = (idxd_op_memmove << IDXD_CMD_OP_SHIFT) | + IDXD_FLAG_CACHE_CONTROL | (flags & IDXD_FLAG_FENCE); + return __idxd_write_desc(dev, memmove, src, dst, length, flags); +} + +int +idxd_enqueue_fill(struct rte_dma_dev *dev, uint16_t qid __rte_unused, uint64_t pattern, + rte_iova_t dst, unsigned int length, uint64_t flags) +{ + uint32_t fill = (idxd_op_fill << IDXD_CMD_OP_SHIFT) | + IDXD_FLAG_CACHE_CONTROL | (flags & IDXD_FLAG_FENCE); + return __idxd_write_desc(dev, fill, pattern, dst, length, flags); +} + +int +idxd_submit(struct rte_dma_dev *dev, uint16_t qid __rte_unused) +{ + __submit(dev->dev_private); + return 0; +} + int idxd_dump(const struct rte_dma_dev *dev, FILE *f) { @@ -139,6 +269,12 @@ idxd_dmadev_create(const char *name, struct rte_device *dev, dmadev->dev_ops = ops; dmadev->device = dev; + dmadev->copy = idxd_enqueue_copy; + dmadev->fill = idxd_enqueue_fill; + dmadev->submit = idxd_submit; + dmadev->completed = idxd_completed; + dmadev->completed_status = idxd_completed_status; + idxd = dmadev->dev_private; *idxd = *base_idxd; /* copy over the main fields already passed in */ idxd->dmadev = dmadev; diff --git a/drivers/dma/idxd/idxd_internal.h b/drivers/dma/idxd/idxd_internal.h index fdd018ca35..b66c2d0182 100644 --- a/drivers/dma/idxd/idxd_internal.h +++ b/drivers/dma/idxd/idxd_internal.h @@ -88,5 +88,10 @@ int idxd_vchan_setup(struct rte_dma_dev *dev, uint16_t vchan, const struct rte_dma_vchan_conf *qconf, uint32_t qconf_sz); int idxd_info_get(const struct rte_dma_dev *dev, struct rte_dma_info *dev_info, uint32_t size); +int idxd_enqueue_copy(struct rte_dma_dev *dev, uint16_t qid, rte_iova_t src, + rte_iova_t dst, unsigned int length, uint64_t flags); +int idxd_enqueue_fill(struct rte_dma_dev *dev, uint16_t qid, uint64_t pattern, + rte_iova_t dst, unsigned int length, uint64_t flags); +int idxd_submit(struct rte_dma_dev *dev, uint16_t qid); #endif /* _IDXD_INTERNAL_H_ */ diff --git a/drivers/dma/idxd/meson.build b/drivers/dma/idxd/meson.build index da5dc2b019..3b5133c578 100644 --- a/drivers/dma/idxd/meson.build +++ b/drivers/dma/idxd/meson.build @@ -6,6 +6,7 @@ if is_windows endif deps += ['bus_pci'] +cflags += '-mavx2' # all platforms with idxd HW support AVX sources = files( 'idxd_bus.c', 'idxd_common.c', From patchwork Fri Sep 24 13:39:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99602 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A4820A0548; Fri, 24 Sep 2021 15:40:30 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AA4784134A; Fri, 24 Sep 2021 15:39:49 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id AFB1041349 for ; Fri, 24 Sep 2021 15:39:47 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10116"; a="285084313" X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="285084313" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2021 06:39:47 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="515527701" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by fmsmga008.fm.intel.com with ESMTP; 24 Sep 2021 06:39:45 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 24 Sep 2021 13:39:10 +0000 Message-Id: <20210924133916.4042773-11-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210924133916.4042773-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210924133916.4042773-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v6 10/16] dma/idxd: add data-path job completion functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the data path functions for gathering completed operations. Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh --- v2: - fixed typo in docs - add completion status for invalid opcode v6: - update documentation to reduce duplication --- doc/guides/dmadevs/idxd.rst | 32 ++++- drivers/dma/idxd/idxd_common.c | 235 +++++++++++++++++++++++++++++++ drivers/dma/idxd/idxd_internal.h | 5 + 3 files changed, 271 insertions(+), 1 deletion(-) diff --git a/doc/guides/dmadevs/idxd.rst b/doc/guides/dmadevs/idxd.rst index b3d78482be..2220e454bc 100644 --- a/doc/guides/dmadevs/idxd.rst +++ b/doc/guides/dmadevs/idxd.rst @@ -147,7 +147,37 @@ Performing Data Copies ~~~~~~~~~~~~~~~~~~~~~~~ Refer to the :ref:`Enqueue / Dequeue APIs ` section of the dmadev library -documentation for details on operation enqueue and submission API usage. +documentation for details on operation enqueue, submission and completion API usage. It is expected that, for efficiency reasons, a burst of operations will be enqueued to the device via multiple enqueue calls between calls to the ``rte_dma_submit()`` function. + +When gathering completions, ``rte_dma_completed()`` should be used, up until the point an error +occurs in an operation. If an error was encountered, ``rte_dma_completed_status()`` must be used +to kick the device off to continue processing operations and also to gather the status of each +individual operations which is filled in to the ``status`` array provided as parameter by the +application. + +The following status codes are supported by IDXD: + +* ``RTE_DMA_STATUS_SUCCESSFUL``: The operation was successful. +* ``RTE_DMA_STATUS_INVALID_OPCODE``: The operation failed due to an invalid operation code. +* ``RTE_DMA_STATUS_INVALID_LENGTH``: The operation failed due to an invalid data length. +* ``RTE_DMA_STATUS_NOT_ATTEMPTED``: The operation was not attempted. +* ``RTE_DMA_STATUS_ERROR_UNKNOWN``: The operation failed due to an unspecified error. + +The following code shows how to retrieve the number of successfully completed +copies within a burst and then using ``rte_dma_completed_status()`` to check +which operation failed and kick off the device to continue processing operations: + +.. code-block:: C + + enum rte_dma_status_code status[COMP_BURST_SZ]; + uint16_t count, idx, status_count; + bool error = 0; + + count = rte_dma_completed(dev_id, vchan, COMP_BURST_SZ, &idx, &error); + + if (error){ + status_count = rte_dma_completed_status(dev_id, vchan, COMP_BURST_SZ, &idx, status); + } diff --git a/drivers/dma/idxd/idxd_common.c b/drivers/dma/idxd/idxd_common.c index 1580f5029c..76ef7d0378 100644 --- a/drivers/dma/idxd/idxd_common.c +++ b/drivers/dma/idxd/idxd_common.c @@ -140,6 +140,241 @@ idxd_submit(struct rte_dma_dev *dev, uint16_t qid __rte_unused) return 0; } +static enum rte_dma_status_code +get_comp_status(struct idxd_completion *c) +{ + uint8_t st = c->status; + switch (st) { + /* successful descriptors are not written back normally */ + case IDXD_COMP_STATUS_INCOMPLETE: + case IDXD_COMP_STATUS_SUCCESS: + return RTE_DMA_STATUS_SUCCESSFUL; + case IDXD_COMP_STATUS_INVALID_OPCODE: + return RTE_DMA_STATUS_INVALID_OPCODE; + case IDXD_COMP_STATUS_INVALID_SIZE: + return RTE_DMA_STATUS_INVALID_LENGTH; + case IDXD_COMP_STATUS_SKIPPED: + return RTE_DMA_STATUS_NOT_ATTEMPTED; + default: + return RTE_DMA_STATUS_ERROR_UNKNOWN; + } +} + +static __rte_always_inline int +batch_ok(struct idxd_dmadev *idxd, uint16_t max_ops, enum rte_dma_status_code *status) +{ + uint16_t ret; + uint8_t bstatus; + + if (max_ops == 0) + return 0; + + /* first check if there are any unreturned handles from last time */ + if (idxd->ids_avail != idxd->ids_returned) { + ret = RTE_MIN((uint16_t)(idxd->ids_avail - idxd->ids_returned), max_ops); + idxd->ids_returned += ret; + if (status) + memset(status, RTE_DMA_STATUS_SUCCESSFUL, ret * sizeof(*status)); + return ret; + } + + if (idxd->batch_idx_read == idxd->batch_idx_write) + return 0; + + bstatus = idxd->batch_comp_ring[idxd->batch_idx_read].status; + /* now check if next batch is complete and successful */ + if (bstatus == IDXD_COMP_STATUS_SUCCESS) { + /* since the batch idx ring stores the start of each batch, pre-increment to lookup + * start of next batch. + */ + if (++idxd->batch_idx_read > idxd->max_batches) + idxd->batch_idx_read = 0; + idxd->ids_avail = idxd->batch_idx_ring[idxd->batch_idx_read]; + + ret = RTE_MIN((uint16_t)(idxd->ids_avail - idxd->ids_returned), max_ops); + idxd->ids_returned += ret; + if (status) + memset(status, RTE_DMA_STATUS_SUCCESSFUL, ret * sizeof(*status)); + return ret; + } + /* check if batch is incomplete */ + else if (bstatus == IDXD_COMP_STATUS_INCOMPLETE) + return 0; + + return -1; /* error case */ +} + +static inline uint16_t +batch_completed(struct idxd_dmadev *idxd, uint16_t max_ops, bool *has_error) +{ + uint16_t i; + uint16_t b_start, b_end, next_batch; + + int ret = batch_ok(idxd, max_ops, NULL); + if (ret >= 0) + return ret; + + /* ERROR case, not successful, not incomplete */ + /* Get the batch size, and special case size 1. + * once we identify the actual failure job, return other jobs, then update + * the batch ring indexes to make it look like the first job of the batch has failed. + * Subsequent calls here will always return zero packets, and the error must be cleared by + * calling the completed_status() function. + */ + next_batch = (idxd->batch_idx_read + 1); + if (next_batch > idxd->max_batches) + next_batch = 0; + b_start = idxd->batch_idx_ring[idxd->batch_idx_read]; + b_end = idxd->batch_idx_ring[next_batch]; + + if (b_end - b_start == 1) { /* not a batch */ + *has_error = true; + return 0; + } + + for (i = b_start; i < b_end; i++) { + struct idxd_completion *c = (void *)&idxd->desc_ring[i & idxd->desc_ring_mask]; + if (c->status > IDXD_COMP_STATUS_SUCCESS) /* ignore incomplete(0) and success(1) */ + break; + } + ret = RTE_MIN((uint16_t)(i - idxd->ids_returned), max_ops); + if (ret < max_ops) + *has_error = true; /* we got up to the point of error */ + idxd->ids_avail = idxd->ids_returned += ret; + + /* to ensure we can call twice and just return 0, set start of batch to where we finished */ + idxd->batch_comp_ring[idxd->batch_idx_read].completed_size -= ret; + idxd->batch_idx_ring[idxd->batch_idx_read] += ret; + if (idxd->batch_idx_ring[next_batch] - idxd->batch_idx_ring[idxd->batch_idx_read] == 1) { + /* copy over the descriptor status to the batch ring as if no batch */ + uint16_t d_idx = idxd->batch_idx_ring[idxd->batch_idx_read] & idxd->desc_ring_mask; + struct idxd_completion *desc_comp = (void *)&idxd->desc_ring[d_idx]; + idxd->batch_comp_ring[idxd->batch_idx_read].status = desc_comp->status; + } + + return ret; +} + +static uint16_t +batch_completed_status(struct idxd_dmadev *idxd, uint16_t max_ops, enum rte_dma_status_code *status) +{ + uint16_t next_batch; + + int ret = batch_ok(idxd, max_ops, status); + if (ret >= 0) + return ret; + + /* ERROR case, not successful, not incomplete */ + /* Get the batch size, and special case size 1. + */ + next_batch = (idxd->batch_idx_read + 1); + if (next_batch > idxd->max_batches) + next_batch = 0; + const uint16_t b_start = idxd->batch_idx_ring[idxd->batch_idx_read]; + const uint16_t b_end = idxd->batch_idx_ring[next_batch]; + const uint16_t b_len = b_end - b_start; + if (b_len == 1) {/* not a batch */ + *status = get_comp_status(&idxd->batch_comp_ring[idxd->batch_idx_read]); + idxd->ids_avail++; + idxd->ids_returned++; + idxd->batch_idx_read = next_batch; + return 1; + } + + /* not a single-element batch, need to process more. + * Scenarios: + * 1. max_ops >= batch_size - can fit everything, simple case + * - loop through completed ops and then add on any not-attempted ones + * 2. max_ops < batch_size - can't fit everything, more complex case + * - loop through completed/incomplete and stop when hit max_ops + * - adjust the batch descriptor to update where we stopped, with appropriate bcount + * - if bcount is to be exactly 1, update the batch descriptor as it will be treated as + * non-batch next time. + */ + const uint16_t bcount = idxd->batch_comp_ring[idxd->batch_idx_read].completed_size; + for (ret = 0; ret < b_len && ret < max_ops; ret++) { + struct idxd_completion *c = (void *) + &idxd->desc_ring[(b_start + ret) & idxd->desc_ring_mask]; + status[ret] = (ret < bcount) ? get_comp_status(c) : RTE_DMA_STATUS_NOT_ATTEMPTED; + } + idxd->ids_avail = idxd->ids_returned += ret; + + /* everything fit */ + if (ret == b_len) { + idxd->batch_idx_read = next_batch; + return ret; + } + + /* set up for next time, update existing batch descriptor & start idx at batch_idx_read */ + idxd->batch_idx_ring[idxd->batch_idx_read] += ret; + if (ret > bcount) { + /* we have only incomplete ones - set batch completed size to 0 */ + struct idxd_completion *comp = &idxd->batch_comp_ring[idxd->batch_idx_read]; + comp->completed_size = 0; + /* if there is only one descriptor left, job skipped so set flag appropriately */ + if (b_len - ret == 1) + comp->status = IDXD_COMP_STATUS_SKIPPED; + } else { + struct idxd_completion *comp = &idxd->batch_comp_ring[idxd->batch_idx_read]; + comp->completed_size -= ret; + /* if there is only one descriptor left, copy status info straight to desc */ + if (comp->completed_size == 1) { + struct idxd_completion *c = (void *) + &idxd->desc_ring[(b_start + ret) & idxd->desc_ring_mask]; + comp->status = c->status; + /* individual descs can be ok without writeback, but not batches */ + if (comp->status == IDXD_COMP_STATUS_INCOMPLETE) + comp->status = IDXD_COMP_STATUS_SUCCESS; + } else if (bcount == b_len) { + /* check if we still have an error, and clear flag if not */ + uint16_t i; + for (i = b_start + ret; i < b_end; i++) { + struct idxd_completion *c = (void *) + &idxd->desc_ring[i & idxd->desc_ring_mask]; + if (c->status > IDXD_COMP_STATUS_SUCCESS) + break; + } + if (i == b_end) /* no errors */ + comp->status = IDXD_COMP_STATUS_SUCCESS; + } + } + + return ret; +} + +uint16_t +idxd_completed(struct rte_dma_dev *dev, uint16_t qid __rte_unused, uint16_t max_ops, + uint16_t *last_idx, bool *has_error) +{ + struct idxd_dmadev *idxd = dev->dev_private; + uint16_t batch, ret = 0; + + do { + batch = batch_completed(idxd, max_ops - ret, has_error); + ret += batch; + } while (batch > 0 && *has_error == false); + + *last_idx = idxd->ids_returned - 1; + return ret; +} + +uint16_t +idxd_completed_status(struct rte_dma_dev *dev, uint16_t qid __rte_unused, uint16_t max_ops, + uint16_t *last_idx, enum rte_dma_status_code *status) +{ + struct idxd_dmadev *idxd = dev->dev_private; + + uint16_t batch, ret = 0; + + do { + batch = batch_completed_status(idxd, max_ops - ret, &status[ret]); + ret += batch; + } while (batch > 0); + + *last_idx = idxd->ids_returned - 1; + return ret; +} + int idxd_dump(const struct rte_dma_dev *dev, FILE *f) { diff --git a/drivers/dma/idxd/idxd_internal.h b/drivers/dma/idxd/idxd_internal.h index b66c2d0182..15115a0966 100644 --- a/drivers/dma/idxd/idxd_internal.h +++ b/drivers/dma/idxd/idxd_internal.h @@ -93,5 +93,10 @@ int idxd_enqueue_copy(struct rte_dma_dev *dev, uint16_t qid, rte_iova_t src, int idxd_enqueue_fill(struct rte_dma_dev *dev, uint16_t qid, uint64_t pattern, rte_iova_t dst, unsigned int length, uint64_t flags); int idxd_submit(struct rte_dma_dev *dev, uint16_t qid); +uint16_t idxd_completed(struct rte_dma_dev *dev, uint16_t qid, uint16_t max_ops, + uint16_t *last_idx, bool *has_error); +uint16_t idxd_completed_status(struct rte_dma_dev *dev, uint16_t qid __rte_unused, + uint16_t max_ops, uint16_t *last_idx, + enum rte_dma_status_code *status); #endif /* _IDXD_INTERNAL_H_ */ From patchwork Fri Sep 24 13:39:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99603 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 696CFA0548; Fri, 24 Sep 2021 15:40:36 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BAB914134F; Fri, 24 Sep 2021 15:39:51 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 4F2044134D for ; Fri, 24 Sep 2021 15:39:50 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10116"; a="285084324" X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="285084324" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2021 06:39:49 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="515527710" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by fmsmga008.fm.intel.com with ESMTP; 24 Sep 2021 06:39:47 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 24 Sep 2021 13:39:11 +0000 Message-Id: <20210924133916.4042773-12-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210924133916.4042773-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210924133916.4042773-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v6 11/16] dma/idxd: add operation statistic tracking X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add statistic tracking for DSA devices. The dmadev library documentation is also updated to add a generic section for using the library's statistics APIs. Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh Reviewed-by: Chengwen Feng --- v6: move doc update to dmadev library doC --- doc/guides/prog_guide/dmadev.rst | 11 +++++++++++ drivers/dma/idxd/idxd_bus.c | 2 ++ drivers/dma/idxd/idxd_common.c | 27 +++++++++++++++++++++++++++ drivers/dma/idxd/idxd_internal.h | 3 +++ drivers/dma/idxd/idxd_pci.c | 2 ++ 5 files changed, 45 insertions(+) diff --git a/doc/guides/prog_guide/dmadev.rst b/doc/guides/prog_guide/dmadev.rst index 4908e33762..b268dc8d46 100644 --- a/doc/guides/prog_guide/dmadev.rst +++ b/doc/guides/prog_guide/dmadev.rst @@ -148,3 +148,14 @@ completed operations along with the status of each operation (filled into the ``status`` array passed by user). These two APIs can also return the last completed operation's ``ring_idx`` which could help user track operations within their own application-defined rings. + + +Querying Device Statistics +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The statistics from a dmadev device can be got via the statistics functions, +i.e. ``rte_dma_stats_get()``. The statistics returned for each device instance are: + +* ``submitted``: The number of operations submitted to the device. +* ``completed``: The number of operations which have completed (successful and failed). +* ``errors``: The number of operations that completed with error. diff --git a/drivers/dma/idxd/idxd_bus.c b/drivers/dma/idxd/idxd_bus.c index b2acdac4f9..b52ea02854 100644 --- a/drivers/dma/idxd/idxd_bus.c +++ b/drivers/dma/idxd/idxd_bus.c @@ -99,6 +99,8 @@ static const struct rte_dma_dev_ops idxd_bus_ops = { .dev_configure = idxd_configure, .vchan_setup = idxd_vchan_setup, .dev_info_get = idxd_info_get, + .stats_get = idxd_stats_get, + .stats_reset = idxd_stats_reset, }; static void * diff --git a/drivers/dma/idxd/idxd_common.c b/drivers/dma/idxd/idxd_common.c index 76ef7d0378..7a3eb0a4c1 100644 --- a/drivers/dma/idxd/idxd_common.c +++ b/drivers/dma/idxd/idxd_common.c @@ -65,6 +65,8 @@ __submit(struct idxd_dmadev *idxd) if (++idxd->batch_idx_write > idxd->max_batches) idxd->batch_idx_write = 0; + idxd->stats.submitted += idxd->batch_size; + idxd->batch_start += idxd->batch_size; idxd->batch_size = 0; idxd->batch_idx_ring[idxd->batch_idx_write] = idxd->batch_start; @@ -275,6 +277,8 @@ batch_completed_status(struct idxd_dmadev *idxd, uint16_t max_ops, enum rte_dma_ const uint16_t b_len = b_end - b_start; if (b_len == 1) {/* not a batch */ *status = get_comp_status(&idxd->batch_comp_ring[idxd->batch_idx_read]); + if (status != RTE_DMA_STATUS_SUCCESSFUL) + idxd->stats.errors++; idxd->ids_avail++; idxd->ids_returned++; idxd->batch_idx_read = next_batch; @@ -296,6 +300,8 @@ batch_completed_status(struct idxd_dmadev *idxd, uint16_t max_ops, enum rte_dma_ struct idxd_completion *c = (void *) &idxd->desc_ring[(b_start + ret) & idxd->desc_ring_mask]; status[ret] = (ret < bcount) ? get_comp_status(c) : RTE_DMA_STATUS_NOT_ATTEMPTED; + if (status[ret] != RTE_DMA_STATUS_SUCCESSFUL) + idxd->stats.errors++; } idxd->ids_avail = idxd->ids_returned += ret; @@ -354,6 +360,7 @@ idxd_completed(struct rte_dma_dev *dev, uint16_t qid __rte_unused, uint16_t max_ ret += batch; } while (batch > 0 && *has_error == false); + idxd->stats.completed += ret; *last_idx = idxd->ids_returned - 1; return ret; } @@ -371,6 +378,7 @@ idxd_completed_status(struct rte_dma_dev *dev, uint16_t qid __rte_unused, uint16 ret += batch; } while (batch > 0); + idxd->stats.completed += ret; *last_idx = idxd->ids_returned - 1; return ret; } @@ -404,6 +412,25 @@ idxd_dump(const struct rte_dma_dev *dev, FILE *f) return 0; } +int +idxd_stats_get(const struct rte_dma_dev *dev, uint16_t vchan __rte_unused, + struct rte_dma_stats *stats, uint32_t stats_sz) +{ + struct idxd_dmadev *idxd = dev->dev_private; + if (stats_sz < sizeof(*stats)) + return -EINVAL; + *stats = idxd->stats; + return 0; +} + +int +idxd_stats_reset(struct rte_dma_dev *dev, uint16_t vchan __rte_unused) +{ + struct idxd_dmadev *idxd = dev->dev_private; + idxd->stats = (struct rte_dma_stats){0}; + return 0; +} + int idxd_info_get(const struct rte_dma_dev *dev, struct rte_dma_info *info, uint32_t size) { diff --git a/drivers/dma/idxd/idxd_internal.h b/drivers/dma/idxd/idxd_internal.h index 15115a0966..e2a1119ef7 100644 --- a/drivers/dma/idxd/idxd_internal.h +++ b/drivers/dma/idxd/idxd_internal.h @@ -98,5 +98,8 @@ uint16_t idxd_completed(struct rte_dma_dev *dev, uint16_t qid, uint16_t max_ops, uint16_t idxd_completed_status(struct rte_dma_dev *dev, uint16_t qid __rte_unused, uint16_t max_ops, uint16_t *last_idx, enum rte_dma_status_code *status); +int idxd_stats_get(const struct rte_dma_dev *dev, uint16_t vchan, + struct rte_dma_stats *stats, uint32_t stats_sz); +int idxd_stats_reset(struct rte_dma_dev *dev, uint16_t vchan); #endif /* _IDXD_INTERNAL_H_ */ diff --git a/drivers/dma/idxd/idxd_pci.c b/drivers/dma/idxd/idxd_pci.c index 86a033862b..cf91eb9c5e 100644 --- a/drivers/dma/idxd/idxd_pci.c +++ b/drivers/dma/idxd/idxd_pci.c @@ -136,6 +136,8 @@ static const struct rte_dma_dev_ops idxd_pci_ops = { .dev_configure = idxd_configure, .vchan_setup = idxd_vchan_setup, .dev_info_get = idxd_info_get, + .stats_get = idxd_stats_get, + .stats_reset = idxd_stats_reset, .dev_start = idxd_pci_dev_start, .dev_stop = idxd_pci_dev_stop, }; From patchwork Fri Sep 24 13:39:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99604 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 64D83A0548; Fri, 24 Sep 2021 15:40:43 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2B3F941300; Fri, 24 Sep 2021 15:39:54 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 12BE541352 for ; Fri, 24 Sep 2021 15:39:51 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10116"; a="285084329" X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="285084329" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2021 06:39:51 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="515527715" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by fmsmga008.fm.intel.com with ESMTP; 24 Sep 2021 06:39:50 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 24 Sep 2021 13:39:12 +0000 Message-Id: <20210924133916.4042773-13-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210924133916.4042773-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210924133916.4042773-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v6 12/16] dma/idxd: add vchan status function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When testing dmadev drivers, it is useful to have the HW device in a known state. This patch adds the implementation of the function which will wait for the device to be idle (all jobs completed) before proceeding. Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh --- v3: update API name to vchan_status --- drivers/dma/idxd/idxd_bus.c | 1 + drivers/dma/idxd/idxd_common.c | 17 +++++++++++++++++ drivers/dma/idxd/idxd_internal.h | 2 ++ drivers/dma/idxd/idxd_pci.c | 1 + 4 files changed, 21 insertions(+) diff --git a/drivers/dma/idxd/idxd_bus.c b/drivers/dma/idxd/idxd_bus.c index b52ea02854..e6caa048a9 100644 --- a/drivers/dma/idxd/idxd_bus.c +++ b/drivers/dma/idxd/idxd_bus.c @@ -101,6 +101,7 @@ static const struct rte_dma_dev_ops idxd_bus_ops = { .dev_info_get = idxd_info_get, .stats_get = idxd_stats_get, .stats_reset = idxd_stats_reset, + .vchan_status = idxd_vchan_status, }; static void * diff --git a/drivers/dma/idxd/idxd_common.c b/drivers/dma/idxd/idxd_common.c index 7a3eb0a4c1..12c113a93b 100644 --- a/drivers/dma/idxd/idxd_common.c +++ b/drivers/dma/idxd/idxd_common.c @@ -162,6 +162,23 @@ get_comp_status(struct idxd_completion *c) } } +int +idxd_vchan_status(const struct rte_dma_dev *dev, uint16_t vchan __rte_unused, + enum rte_dma_vchan_status *status) +{ + struct idxd_dmadev *idxd = dev->dev_private; + uint16_t last_batch_write = idxd->batch_idx_write == 0 ? idxd->max_batches : + idxd->batch_idx_write - 1; + uint8_t bstatus = (idxd->batch_comp_ring[last_batch_write].status != 0); + + /* An IDXD device will always be either active or idle. + * RTE_DMA_VCHAN_HALTED_ERROR is therefore not supported by IDXD. + */ + *status = bstatus ? RTE_DMA_VCHAN_IDLE : RTE_DMA_VCHAN_ACTIVE; + + return 0; +} + static __rte_always_inline int batch_ok(struct idxd_dmadev *idxd, uint16_t max_ops, enum rte_dma_status_code *status) { diff --git a/drivers/dma/idxd/idxd_internal.h b/drivers/dma/idxd/idxd_internal.h index e2a1119ef7..a291ad26d9 100644 --- a/drivers/dma/idxd/idxd_internal.h +++ b/drivers/dma/idxd/idxd_internal.h @@ -101,5 +101,7 @@ uint16_t idxd_completed_status(struct rte_dma_dev *dev, uint16_t qid __rte_unuse int idxd_stats_get(const struct rte_dma_dev *dev, uint16_t vchan, struct rte_dma_stats *stats, uint32_t stats_sz); int idxd_stats_reset(struct rte_dma_dev *dev, uint16_t vchan); +int idxd_vchan_status(const struct rte_dma_dev *dev, uint16_t vchan, + enum rte_dma_vchan_status *status); #endif /* _IDXD_INTERNAL_H_ */ diff --git a/drivers/dma/idxd/idxd_pci.c b/drivers/dma/idxd/idxd_pci.c index cf91eb9c5e..3152ec1289 100644 --- a/drivers/dma/idxd/idxd_pci.c +++ b/drivers/dma/idxd/idxd_pci.c @@ -140,6 +140,7 @@ static const struct rte_dma_dev_ops idxd_pci_ops = { .stats_reset = idxd_stats_reset, .dev_start = idxd_pci_dev_start, .dev_stop = idxd_pci_dev_stop, + .vchan_status = idxd_vchan_status, }; /* each portal uses 4 x 4k pages */ From patchwork Fri Sep 24 13:39:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99605 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 688DCA0548; Fri, 24 Sep 2021 15:40:48 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4B6E341343; Fri, 24 Sep 2021 15:39:56 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id D15A441355 for ; Fri, 24 Sep 2021 15:39:53 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10116"; a="285084331" X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="285084331" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2021 06:39:53 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="515527727" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by fmsmga008.fm.intel.com with ESMTP; 24 Sep 2021 06:39:51 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 24 Sep 2021 13:39:13 +0000 Message-Id: <20210924133916.4042773-14-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210924133916.4042773-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210924133916.4042773-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v6 13/16] dma/idxd: add burst capacity API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add support for the burst capacity API. This API will provide the calling application with the remaining capacity of the current burst (limited by max HW batch size). Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh Reviewed-by: Bruce Richardson --- v6: updates for burst capacity api moving to fastpath --- drivers/dma/idxd/idxd_common.c | 21 +++++++++++++++++++++ drivers/dma/idxd/idxd_internal.h | 1 + drivers/dma/idxd/idxd_pci.c | 1 + 3 files changed, 23 insertions(+) diff --git a/drivers/dma/idxd/idxd_common.c b/drivers/dma/idxd/idxd_common.c index 12c113a93b..a00fadc431 100644 --- a/drivers/dma/idxd/idxd_common.c +++ b/drivers/dma/idxd/idxd_common.c @@ -468,6 +468,26 @@ idxd_info_get(const struct rte_dma_dev *dev, struct rte_dma_info *info, uint32_t return 0; } +uint16_t +idxd_burst_capacity(const struct rte_dma_dev *dev, uint16_t vchan __rte_unused) +{ + struct idxd_dmadev *idxd = dev->dev_private; + uint16_t write_idx = idxd->batch_start + idxd->batch_size; + uint16_t used_space; + + /* Check for space in the batch ring */ + if ((idxd->batch_idx_read == 0 && idxd->batch_idx_write == idxd->max_batches) || + idxd->batch_idx_write + 1 == idxd->batch_idx_read) + return 0; + + /* For descriptors, check for wrap-around on write but not read */ + if (idxd->ids_returned > write_idx) + write_idx += idxd->desc_ring_mask + 1; + used_space = write_idx - idxd->ids_returned; + + return RTE_MIN((idxd->desc_ring_mask - used_space), idxd->max_batch_size); +} + int idxd_configure(struct rte_dma_dev *dev __rte_unused, const struct rte_dma_conf *dev_conf, uint32_t conf_sz) @@ -553,6 +573,7 @@ idxd_dmadev_create(const char *name, struct rte_device *dev, dmadev->submit = idxd_submit; dmadev->completed = idxd_completed; dmadev->completed_status = idxd_completed_status; + dmadev->burst_capacity = idxd_burst_capacity; idxd = dmadev->dev_private; *idxd = *base_idxd; /* copy over the main fields already passed in */ diff --git a/drivers/dma/idxd/idxd_internal.h b/drivers/dma/idxd/idxd_internal.h index a291ad26d9..3ef2f729a8 100644 --- a/drivers/dma/idxd/idxd_internal.h +++ b/drivers/dma/idxd/idxd_internal.h @@ -103,5 +103,6 @@ int idxd_stats_get(const struct rte_dma_dev *dev, uint16_t vchan, int idxd_stats_reset(struct rte_dma_dev *dev, uint16_t vchan); int idxd_vchan_status(const struct rte_dma_dev *dev, uint16_t vchan, enum rte_dma_vchan_status *status); +uint16_t idxd_burst_capacity(const struct rte_dma_dev *dev, uint16_t vchan); #endif /* _IDXD_INTERNAL_H_ */ diff --git a/drivers/dma/idxd/idxd_pci.c b/drivers/dma/idxd/idxd_pci.c index 3152ec1289..f76383710c 100644 --- a/drivers/dma/idxd/idxd_pci.c +++ b/drivers/dma/idxd/idxd_pci.c @@ -254,6 +254,7 @@ init_pci_device(struct rte_pci_device *dev, struct idxd_dmadev *idxd, idxd->u.pci = pci; idxd->max_batches = wq_size; + idxd->max_batch_size = 1 << lg2_max_batch; /* enable the device itself */ err_code = idxd_pci_dev_command(idxd, idxd_enable_dev); From patchwork Fri Sep 24 13:39:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99606 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A777CA0548; Fri, 24 Sep 2021 15:40:53 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6EB6B41360; Fri, 24 Sep 2021 15:40:01 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id 670D04135F for ; Fri, 24 Sep 2021 15:40:00 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10116"; a="246541380" X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="246541380" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2021 06:39:59 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="515527738" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by fmsmga008.fm.intel.com with ESMTP; 24 Sep 2021 06:39:53 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 24 Sep 2021 13:39:14 +0000 Message-Id: <20210924133916.4042773-15-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210924133916.4042773-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210924133916.4042773-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v6 14/16] dma/idxd: move dpdk_idxd_cfg.py from raw to dma X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Conor Walsh Move the example script for configuring IDXD devices bound to the IDXD kernel driver from raw to dma, and create a symlink to still allow use from raw. Signed-off-by: Conor Walsh Signed-off-by: Kevin Laatz Acked-by: Bruce Richardson --- drivers/dma/idxd/dpdk_idxd_cfg.py | 117 +++++++++++++++++++++++++++++ drivers/raw/ioat/dpdk_idxd_cfg.py | 118 +----------------------------- 2 files changed, 118 insertions(+), 117 deletions(-) create mode 100755 drivers/dma/idxd/dpdk_idxd_cfg.py mode change 100755 => 120000 drivers/raw/ioat/dpdk_idxd_cfg.py diff --git a/drivers/dma/idxd/dpdk_idxd_cfg.py b/drivers/dma/idxd/dpdk_idxd_cfg.py new file mode 100755 index 0000000000..fcc27822ef --- /dev/null +++ b/drivers/dma/idxd/dpdk_idxd_cfg.py @@ -0,0 +1,117 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +""" +Configure an entire Intel DSA instance, using idxd kernel driver, for DPDK use +""" + +import sys +import argparse +import os +import os.path + + +class SysfsDir: + "Used to read/write paths in a sysfs directory" + def __init__(self, path): + self.path = path + + def read_int(self, filename): + "Return a value from sysfs file" + with open(os.path.join(self.path, filename)) as f: + return int(f.readline()) + + def write_values(self, values): + "write dictionary, where key is filename and value is value to write" + for filename, contents in values.items(): + with open(os.path.join(self.path, filename), "w") as f: + f.write(str(contents)) + + +def reset_device(dsa_id): + "Reset the DSA device and all its queues" + drv_dir = SysfsDir("/sys/bus/dsa/drivers/dsa") + drv_dir.write_values({"unbind": f"dsa{dsa_id}"}) + + +def get_pci_dir(pci): + "Search for the sysfs directory of the PCI device" + base_dir = '/sys/bus/pci/devices/' + for path, dirs, files in os.walk(base_dir): + for dir in dirs: + if pci in dir: + return os.path.join(base_dir, dir) + sys.exit(f"Could not find sysfs directory for device {pci}") + + +def get_dsa_id(pci): + "Get the DSA instance ID using the PCI address of the device" + pci_dir = get_pci_dir(pci) + for path, dirs, files in os.walk(pci_dir): + for dir in dirs: + if dir.startswith('dsa') and 'wq' not in dir: + return int(dir[3:]) + sys.exit(f"Could not get device ID for device {pci}") + + +def configure_dsa(dsa_id, queues, prefix): + "Configure the DSA instance with appropriate number of queues" + dsa_dir = SysfsDir(f"/sys/bus/dsa/devices/dsa{dsa_id}") + drv_dir = SysfsDir("/sys/bus/dsa/drivers/dsa") + + max_groups = dsa_dir.read_int("max_groups") + max_engines = dsa_dir.read_int("max_engines") + max_queues = dsa_dir.read_int("max_work_queues") + max_work_queues_size = dsa_dir.read_int("max_work_queues_size") + + nb_queues = min(queues, max_queues) + if queues > nb_queues: + print(f"Setting number of queues to max supported value: {max_queues}") + + # we want one engine per group, and no more engines than queues + nb_groups = min(max_engines, max_groups, nb_queues) + for grp in range(nb_groups): + dsa_dir.write_values({f"engine{dsa_id}.{grp}/group_id": grp}) + + # configure each queue + for q in range(nb_queues): + wq_dir = SysfsDir(os.path.join(dsa_dir.path, f"wq{dsa_id}.{q}")) + wq_dir.write_values({"group_id": q % nb_groups, + "type": "user", + "mode": "dedicated", + "name": f"{prefix}_wq{dsa_id}.{q}", + "priority": 1, + "size": int(max_work_queues_size / nb_queues)}) + + # enable device and then queues + drv_dir.write_values({"bind": f"dsa{dsa_id}"}) + for q in range(nb_queues): + drv_dir.write_values({"bind": f"wq{dsa_id}.{q}"}) + + +def main(args): + "Main function, does arg parsing and calls config function" + arg_p = argparse.ArgumentParser( + description="Configure whole DSA device instance for DPDK use") + arg_p.add_argument('dsa_id', + help="Specify DSA instance either via DSA instance number or PCI address") + arg_p.add_argument('-q', metavar='queues', type=int, default=255, + help="Number of queues to set up") + arg_p.add_argument('--name-prefix', metavar='prefix', dest='prefix', + default="dpdk", + help="Prefix for workqueue name to mark for DPDK use [default: 'dpdk']") + arg_p.add_argument('--reset', action='store_true', + help="Reset DSA device and its queues") + parsed_args = arg_p.parse_args(args[1:]) + + dsa_id = parsed_args.dsa_id + dsa_id = get_dsa_id(dsa_id) if ':' in dsa_id else dsa_id + if parsed_args.reset: + reset_device(dsa_id) + else: + configure_dsa(dsa_id, parsed_args.q, parsed_args.prefix) + + +if __name__ == "__main__": + main(sys.argv) diff --git a/drivers/raw/ioat/dpdk_idxd_cfg.py b/drivers/raw/ioat/dpdk_idxd_cfg.py deleted file mode 100755 index fcc27822ef..0000000000 --- a/drivers/raw/ioat/dpdk_idxd_cfg.py +++ /dev/null @@ -1,117 +0,0 @@ -#!/usr/bin/env python3 -# SPDX-License-Identifier: BSD-3-Clause -# Copyright(c) 2020 Intel Corporation - -""" -Configure an entire Intel DSA instance, using idxd kernel driver, for DPDK use -""" - -import sys -import argparse -import os -import os.path - - -class SysfsDir: - "Used to read/write paths in a sysfs directory" - def __init__(self, path): - self.path = path - - def read_int(self, filename): - "Return a value from sysfs file" - with open(os.path.join(self.path, filename)) as f: - return int(f.readline()) - - def write_values(self, values): - "write dictionary, where key is filename and value is value to write" - for filename, contents in values.items(): - with open(os.path.join(self.path, filename), "w") as f: - f.write(str(contents)) - - -def reset_device(dsa_id): - "Reset the DSA device and all its queues" - drv_dir = SysfsDir("/sys/bus/dsa/drivers/dsa") - drv_dir.write_values({"unbind": f"dsa{dsa_id}"}) - - -def get_pci_dir(pci): - "Search for the sysfs directory of the PCI device" - base_dir = '/sys/bus/pci/devices/' - for path, dirs, files in os.walk(base_dir): - for dir in dirs: - if pci in dir: - return os.path.join(base_dir, dir) - sys.exit(f"Could not find sysfs directory for device {pci}") - - -def get_dsa_id(pci): - "Get the DSA instance ID using the PCI address of the device" - pci_dir = get_pci_dir(pci) - for path, dirs, files in os.walk(pci_dir): - for dir in dirs: - if dir.startswith('dsa') and 'wq' not in dir: - return int(dir[3:]) - sys.exit(f"Could not get device ID for device {pci}") - - -def configure_dsa(dsa_id, queues, prefix): - "Configure the DSA instance with appropriate number of queues" - dsa_dir = SysfsDir(f"/sys/bus/dsa/devices/dsa{dsa_id}") - drv_dir = SysfsDir("/sys/bus/dsa/drivers/dsa") - - max_groups = dsa_dir.read_int("max_groups") - max_engines = dsa_dir.read_int("max_engines") - max_queues = dsa_dir.read_int("max_work_queues") - max_work_queues_size = dsa_dir.read_int("max_work_queues_size") - - nb_queues = min(queues, max_queues) - if queues > nb_queues: - print(f"Setting number of queues to max supported value: {max_queues}") - - # we want one engine per group, and no more engines than queues - nb_groups = min(max_engines, max_groups, nb_queues) - for grp in range(nb_groups): - dsa_dir.write_values({f"engine{dsa_id}.{grp}/group_id": grp}) - - # configure each queue - for q in range(nb_queues): - wq_dir = SysfsDir(os.path.join(dsa_dir.path, f"wq{dsa_id}.{q}")) - wq_dir.write_values({"group_id": q % nb_groups, - "type": "user", - "mode": "dedicated", - "name": f"{prefix}_wq{dsa_id}.{q}", - "priority": 1, - "size": int(max_work_queues_size / nb_queues)}) - - # enable device and then queues - drv_dir.write_values({"bind": f"dsa{dsa_id}"}) - for q in range(nb_queues): - drv_dir.write_values({"bind": f"wq{dsa_id}.{q}"}) - - -def main(args): - "Main function, does arg parsing and calls config function" - arg_p = argparse.ArgumentParser( - description="Configure whole DSA device instance for DPDK use") - arg_p.add_argument('dsa_id', - help="Specify DSA instance either via DSA instance number or PCI address") - arg_p.add_argument('-q', metavar='queues', type=int, default=255, - help="Number of queues to set up") - arg_p.add_argument('--name-prefix', metavar='prefix', dest='prefix', - default="dpdk", - help="Prefix for workqueue name to mark for DPDK use [default: 'dpdk']") - arg_p.add_argument('--reset', action='store_true', - help="Reset DSA device and its queues") - parsed_args = arg_p.parse_args(args[1:]) - - dsa_id = parsed_args.dsa_id - dsa_id = get_dsa_id(dsa_id) if ':' in dsa_id else dsa_id - if parsed_args.reset: - reset_device(dsa_id) - else: - configure_dsa(dsa_id, parsed_args.q, parsed_args.prefix) - - -if __name__ == "__main__": - main(sys.argv) diff --git a/drivers/raw/ioat/dpdk_idxd_cfg.py b/drivers/raw/ioat/dpdk_idxd_cfg.py new file mode 120000 index 0000000000..85545548d1 --- /dev/null +++ b/drivers/raw/ioat/dpdk_idxd_cfg.py @@ -0,0 +1 @@ +../../dma/idxd/dpdk_idxd_cfg.py \ No newline at end of file From patchwork Fri Sep 24 13:39:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99607 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 484F8A0548; Fri, 24 Sep 2021 15:40:59 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7ACE741329; Fri, 24 Sep 2021 15:40:04 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id 50E814135F for ; Fri, 24 Sep 2021 15:40:01 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10116"; a="246541383" X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="246541383" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2021 06:40:00 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="515527750" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by fmsmga008.fm.intel.com with ESMTP; 24 Sep 2021 06:39:58 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 24 Sep 2021 13:39:15 +0000 Message-Id: <20210924133916.4042773-16-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210924133916.4042773-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210924133916.4042773-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v6 15/16] devbind: add dma device class X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add a new class for DMA devices. Devices listed under the DMA class are to be used with the dmadev library. Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh Reviewed-by: Bruce Richardson Reviewed-by: Chengwen Feng --- v6: remove purely cosmetic change from patch --- usertools/dpdk-devbind.py | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py index 74d16e4c4b..fb43e3c0b1 100755 --- a/usertools/dpdk-devbind.py +++ b/usertools/dpdk-devbind.py @@ -69,6 +69,7 @@ network_devices = [network_class, cavium_pkx, avp_vnic, ifpga_class] baseband_devices = [acceleration_class] crypto_devices = [encryption_class, intel_processor_class] +dma_devices = [] eventdev_devices = [cavium_sso, cavium_tim, intel_dlb, octeontx2_sso] mempool_devices = [cavium_fpa, octeontx2_npa] compress_devices = [cavium_zip] @@ -583,6 +584,9 @@ def show_status(): if status_dev in ["crypto", "all"]: show_device_status(crypto_devices, "Crypto") + if status_dev in ["dma", "all"]: + show_device_status(dma_devices, "DMA") + if status_dev in ["event", "all"]: show_device_status(eventdev_devices, "Eventdev") @@ -651,7 +655,7 @@ def parse_args(): parser.add_argument( '--status-dev', help="Print the status of given device group.", - choices=['baseband', 'compress', 'crypto', 'event', + choices=['baseband', 'compress', 'crypto', 'dma', 'event', 'mempool', 'misc', 'net', 'regex']) bind_group = parser.add_mutually_exclusive_group() bind_group.add_argument( @@ -732,6 +736,7 @@ def do_arg_actions(): get_device_details(network_devices) get_device_details(baseband_devices) get_device_details(crypto_devices) + get_device_details(dma_devices) get_device_details(eventdev_devices) get_device_details(mempool_devices) get_device_details(compress_devices) @@ -754,6 +759,7 @@ def main(): get_device_details(network_devices) get_device_details(baseband_devices) get_device_details(crypto_devices) + get_device_details(dma_devices) get_device_details(eventdev_devices) get_device_details(mempool_devices) get_device_details(compress_devices) From patchwork Fri Sep 24 13:39:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Laatz X-Patchwork-Id: 99608 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4B04FA0548; Fri, 24 Sep 2021 15:41:04 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 906B441319; Fri, 24 Sep 2021 15:40:08 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id 7BC3141346 for ; Fri, 24 Sep 2021 15:40:07 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10116"; a="246541390" X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="246541390" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Sep 2021 06:40:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="515527798" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by fmsmga008.fm.intel.com with ESMTP; 24 Sep 2021 06:40:00 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 24 Sep 2021 13:39:16 +0000 Message-Id: <20210924133916.4042773-17-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210924133916.4042773-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210924133916.4042773-1-kevin.laatz@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v6 16/16] devbind: move idxd device ID to dmadev class X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The dmadev library is the preferred abstraction for using IDXD devices and will replace the rawdev implementation in future. This patch moves the IDXD device ID to the dmadev class. Signed-off-by: Kevin Laatz Reviewed-by: Conor Walsh Acked-by: Bruce Richardson --- usertools/dpdk-devbind.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py index fb43e3c0b1..15d438715f 100755 --- a/usertools/dpdk-devbind.py +++ b/usertools/dpdk-devbind.py @@ -69,12 +69,12 @@ network_devices = [network_class, cavium_pkx, avp_vnic, ifpga_class] baseband_devices = [acceleration_class] crypto_devices = [encryption_class, intel_processor_class] -dma_devices = [] +dma_devices = [intel_idxd_spr] eventdev_devices = [cavium_sso, cavium_tim, intel_dlb, octeontx2_sso] mempool_devices = [cavium_fpa, octeontx2_npa] compress_devices = [cavium_zip] regex_devices = [octeontx2_ree] -misc_devices = [cnxk_bphy, cnxk_bphy_cgx, intel_ioat_bdw, intel_ioat_skx, intel_ioat_icx, intel_idxd_spr, +misc_devices = [cnxk_bphy, cnxk_bphy_cgx, intel_ioat_bdw, intel_ioat_skx, intel_ioat_icx, intel_ntb_skx, intel_ntb_icx, octeontx2_dma]