From patchwork Mon Apr 4 21:13:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chautru, Nicolas" X-Patchwork-Id: 109131 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 520B5A0508; Mon, 4 Apr 2022 23:16:31 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8AF4742833; Mon, 4 Apr 2022 23:16:26 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mails.dpdk.org (Postfix) with ESMTP id 3A6CF4068C for ; Mon, 4 Apr 2022 23:16:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1649106984; x=1680642984; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=7jHzKRkN00heOz0ziVZmU79Hsdocd8szTzwUIPIDi4w=; b=NOfncVr0oul0T/Vrg1H9f5Q5i11qO0beStUP20Grh3Cy0/OhgdCL8Voe h+mlSgwCwUHH+beOn4XUUXpQEAk9Qhi9i5LzdUVh1MlZ78l0A+wBAIrRu 3k5GQC9taNoTpd/1YWT06E0Xz6GTviN8YC5sGZOONTZEWl6cdrA7dKTOh w62TsqHDXHD35xxJhdCjA+pi8ooPzFw1wC8/KMfnYBq5Er7T97VibfX6C jQJMknsfPFJE8e4DKgF+t0xT6S9AnaE2UoBoD+B8iN5LEPGEeIAgkWfzB 48qZuBTLHi6JceAiB8zB0qQhAbJDtpRBdy64dxlBs3tP+ScD1o4mPIcE7 A==; X-IronPort-AV: E=McAfee;i="6200,9189,10307"; a="258194692" X-IronPort-AV: E=Sophos;i="5.90,235,1643702400"; d="scan'208";a="258194692" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2022 14:16:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,235,1643702400"; d="scan'208";a="569552035" Received: from skx-5gnr-sc12-4.sc.intel.com ([172.25.69.210]) by orsmga008.jf.intel.com with ESMTP; 04 Apr 2022 14:16:22 -0700 From: Nicolas Chautru To: dev@dpdk.org, gakhil@marvell.com Cc: trix@redhat.com, thomas@monjalon.net, ray.kinsella@intel.com, bruce.richardson@intel.com, hemant.agrawal@nxp.com, mingshan.zhang@intel.com, david.marchand@redhat.com, Nicolas Chautru Subject: [PATCH v1 1/9] baseband/acc101: introduce PMD for ACC101 Date: Mon, 4 Apr 2022 14:13:40 -0700 Message-Id: <1649106828-116338-2-git-send-email-nicolas.chautru@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1649106828-116338-1-git-send-email-nicolas.chautru@intel.com> References: <1649106828-116338-1-git-send-email-nicolas.chautru@intel.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Skeleton code and documentation for the ACC101 bbdev PMD. Signed-off-by: Nicolas Chautru --- MAINTAINERS | 3 + doc/guides/bbdevs/acc101.rst | 193 +++++++++++++++++++++++++++++++ doc/guides/bbdevs/features/acc101.ini | 13 +++ doc/guides/bbdevs/index.rst | 1 + drivers/baseband/acc101/meson.build | 6 + drivers/baseband/acc101/rte_acc101_pmd.c | 178 ++++++++++++++++++++++++++++ drivers/baseband/acc101/rte_acc101_pmd.h | 53 +++++++++ drivers/baseband/acc101/version.map | 3 + drivers/baseband/meson.build | 1 + 9 files changed, 451 insertions(+) create mode 100644 doc/guides/bbdevs/acc101.rst create mode 100644 doc/guides/bbdevs/features/acc101.ini create mode 100644 drivers/baseband/acc101/meson.build create mode 100644 drivers/baseband/acc101/rte_acc101_pmd.c create mode 100644 drivers/baseband/acc101/rte_acc101_pmd.h create mode 100644 drivers/baseband/acc101/version.map diff --git a/MAINTAINERS b/MAINTAINERS index 15008c0..c4a8047 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1313,6 +1313,9 @@ F: doc/guides/bbdevs/features/fpga_5gnr_fec.ini F: drivers/baseband/acc100/ F: doc/guides/bbdevs/acc100.rst F: doc/guides/bbdevs/features/acc100.ini +F: drivers/baseband/acc101/ +F: doc/guides/bbdevs/acc101.rst +F: doc/guides/bbdevs/features/acc101.ini Null baseband M: Nicolas Chautru diff --git a/doc/guides/bbdevs/acc101.rst b/doc/guides/bbdevs/acc101.rst new file mode 100644 index 0000000..fb5f9f1 --- /dev/null +++ b/doc/guides/bbdevs/acc101.rst @@ -0,0 +1,193 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2020 Intel Corporation + +Intel(R) ACC101 5G/4G FEC Poll Mode Driver +========================================== + +The BBDEV ACC101 5G/4G FEC poll mode driver (PMD) supports an +implementation of a VRAN FEC wireless acceleration function. +This device is also known as Mount Cirrus. +This is a follow-up to Mount Bryce (ACC100) and includes fixes, improved +feature set for error scenarios and performance increase. + +Features +-------- + +ACC101 5G/4G FEC PMD supports the following features: + +- 16 VFs per PF (physical device) +- Maximum of 128 queues per VF +- PCIe Gen-3 x16 Interface +- MSI +- SR-IOV + +Installation +------------ + +Section 3 of the DPDK manual provides instructions on installing and compiling DPDK. + +DPDK requires hugepages to be configured as detailed in section 2 of the DPDK manual. +The bbdev test application has been tested with a configuration 40 x 1GB hugepages. The +hugepage configuration of a server may be examined using: + +.. code-block:: console + + grep Huge* /proc/meminfo + + +Initialization +-------------- + +When the device first powers up, its PCI Physical Functions (PF) can be listed through this command: + +.. code-block:: console + + sudo lspci -vd8086:57c4 + +The physical and virtual functions are compatible with Linux UIO drivers: +``vfio`` and ``igb_uio``. However, in order to work the ACC101 5G/4G +FEC device first needs to be bound to one of these linux drivers through DPDK. + + +Bind PF UIO driver(s) +~~~~~~~~~~~~~~~~~~~~~ + +Install the DPDK igb_uio driver, bind it with the PF PCI device ID and use +``lspci`` to confirm the PF device is under use by ``igb_uio`` DPDK UIO driver. + +The igb_uio driver may be bound to the PF PCI device using one of two methods: + + +1. PCI functions (physical or virtual, depending on the use case) can be bound to +the UIO driver by repeating this command for every function. + +.. code-block:: console + + cd + insmod ./build/kmod/igb_uio.ko + echo "8086 57c4" > /sys/bus/pci/drivers/igb_uio/new_id + lspci -vd8086:57c4 + + +2. Another way to bind PF with DPDK UIO driver is by using the ``dpdk-devbind.py`` tool + +.. code-block:: console + + cd + ./usertools/dpdk-devbind.py -b igb_uio 0000:06:00.0 + +where the PCI device ID (example: 0000:06:00.0) is obtained using lspci -vd8086:57c4 + + +In a similar way the ACC101 5G/4G FEC PF may be bound with vfio-pci as any PCIe device. + + +Enable Virtual Functions +~~~~~~~~~~~~~~~~~~~~~~~~ + +Now, it should be visible in the printouts that PCI PF is under igb_uio control +"``Kernel driver in use: igb_uio``" + +To show the number of available VFs on the device, read ``sriov_totalvfs`` file.. + +.. code-block:: console + + cat /sys/bus/pci/devices/0000\:\:./sriov_totalvfs + + where 0000\:\:. is the PCI device ID + + +To enable VFs via igb_uio, echo the number of virtual functions intended to +enable to ``max_vfs`` file.. + +.. code-block:: console + + echo > /sys/bus/pci/devices/0000\:\:./max_vfs + + +Afterwards, all VFs must be bound to appropriate UIO drivers as required, same +way it was done with the physical function previously. + +Enabling SR-IOV via vfio driver is pretty much the same, except that the file +name is different: + +.. code-block:: console + + echo > /sys/bus/pci/devices/0000\:\:./sriov_numvfs + + +Configure the VFs through PF +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The PCI virtual functions must be configured before working or getting assigned +to VMs/Containers. The configuration involves allocating the number of hardware +queues, priorities, load balance, bandwidth and other settings necessary for the +device to perform FEC functions. + +This configuration needs to be executed at least once after reboot or PCI FLR and can +be achieved by using the function ``acc101_configure()``, which sets up the +parameters defined in ``acc101_conf`` structure. + +Test Application +---------------- + +BBDEV provides a test application, ``test-bbdev.py`` and range of test data for testing +the functionality of ACC101 5G/4G FEC encode and decode, depending on the device's +capabilities. The test application is located under app->test-bbdev folder and has the +following options: + +.. code-block:: console + + "-p", "--testapp-path": specifies path to the bbdev test app. + "-e", "--eal-params" : EAL arguments which are passed to the test app. + "-t", "--timeout" : Timeout in seconds (default=300). + "-c", "--test-cases" : Defines test cases to run. Run all if not specified. + "-v", "--test-vector" : Test vector path (default=dpdk_path+/app/test-bbdev/test_vectors/bbdev_null.data). + "-n", "--num-ops" : Number of operations to process on device (default=32). + "-b", "--burst-size" : Operations enqueue/dequeue burst size (default=32). + "-s", "--snr" : SNR in dB used when generating LLRs for bler tests. + "-s", "--iter_max" : Number of iterations for LDPC decoder. + "-l", "--num-lcores" : Number of lcores to run (default=16). + "-i", "--init-device" : Initialise PF device with default values. + + +To execute the test application tool using simple decode or encode data, +type one of the following: + +.. code-block:: console + + ./test-bbdev.py -c validation -n 64 -b 1 -v ./ldpc_dec_default.data + ./test-bbdev.py -c validation -n 64 -b 1 -v ./ldpc_enc_default.data + + +The test application ``test-bbdev.py``, supports the ability to configure the PF device with +a default set of values, if the "-i" or "- -init-device" option is included. The default values +are defined in test_bbdev_perf.c. + + +Test Vectors +~~~~~~~~~~~~ + +In addition to the simple LDPC decoder and LDPC encoder tests, bbdev also provides +a range of additional tests under the test_vectors folder, which may be useful. The results +of these tests will depend on the ACC101 5G/4G FEC capabilities which may cause some +testcases to be skipped, but no failure should be reported. + + +Alternate Baseband Device configuration tool +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +On top of the embedded configuration feature supported in test-bbdev using "- -init-device" +option mentioned above, there is also a tool available to perform that device configuration +using a companion application. +The ``pf_bb_config`` application notably enables then to run bbdev-test from the VF +and not only limited to the PF as captured above. + +See for more details: https://github.com/intel/pf-bb-config + +Specifically for the BBDEV ACC101 PMD, the command below can be used: + +.. code-block:: console + + ./pf_bb_config ACC101 -c acc101/acc101_config_4vf_4g5g.cfg + ./test-bbdev.py -e="-c 0xff0 -a${VF_PCI_ADDR}" -c validation -l 1 -v ./ldpc_dec_default.data diff --git a/doc/guides/bbdevs/features/acc101.ini b/doc/guides/bbdevs/features/acc101.ini new file mode 100644 index 0000000..1a62d13 --- /dev/null +++ b/doc/guides/bbdevs/features/acc101.ini @@ -0,0 +1,13 @@ +; +; Supported features of the 'acc101' bbdev driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Turbo Decoder (4G) = N +Turbo Encoder (4G) = N +LDPC Decoder (5G) = N +LDPC Encoder (5G) = N +LLR/HARQ Compression = N +External DDR Access = N +HW Accelerated = Y diff --git a/doc/guides/bbdevs/index.rst b/doc/guides/bbdevs/index.rst index cedd706..e76883c 100644 --- a/doc/guides/bbdevs/index.rst +++ b/doc/guides/bbdevs/index.rst @@ -14,4 +14,5 @@ Baseband Device Drivers fpga_lte_fec fpga_5gnr_fec acc100 + acc101 la12xx diff --git a/drivers/baseband/acc101/meson.build b/drivers/baseband/acc101/meson.build new file mode 100644 index 0000000..e94eb2b --- /dev/null +++ b/drivers/baseband/acc101/meson.build @@ -0,0 +1,6 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +deps += ['bbdev', 'bus_vdev', 'ring', 'pci', 'bus_pci'] + +sources = files('rte_acc101_pmd.c') diff --git a/drivers/baseband/acc101/rte_acc101_pmd.c b/drivers/baseband/acc101/rte_acc101_pmd.c new file mode 100644 index 0000000..dff3834 --- /dev/null +++ b/drivers/baseband/acc101/rte_acc101_pmd.c @@ -0,0 +1,178 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#ifdef RTE_BBDEV_OFFLOAD_COST +#include +#endif + +#include +#include +#include "rte_acc101_pmd.h" + +#ifdef RTE_LIBRTE_BBDEV_DEBUG +RTE_LOG_REGISTER_DEFAULT(acc101_logtype, DEBUG); +#else +RTE_LOG_REGISTER_DEFAULT(acc101_logtype, NOTICE); +#endif + +/* Free memory used for software rings */ +static int +acc101_dev_close(struct rte_bbdev *dev) +{ + RTE_SET_USED(dev); + return 0; +} + +static const struct rte_bbdev_ops acc101_bbdev_ops = { + .close = acc101_dev_close, +}; + +/* ACC101 PCI PF address map */ +static struct rte_pci_id pci_id_acc101_pf_map[] = { + { + RTE_PCI_DEVICE(RTE_ACC101_VENDOR_ID, RTE_ACC101_PF_DEVICE_ID) + }, + {.device_id = 0}, +}; + +/* ACC101 PCI VF address map */ +static struct rte_pci_id pci_id_acc101_vf_map[] = { + { + RTE_PCI_DEVICE(RTE_ACC101_VENDOR_ID, RTE_ACC101_VF_DEVICE_ID) + }, + {.device_id = 0}, +}; + +/* Initialization Function */ +static void +acc101_bbdev_init(struct rte_bbdev *dev, struct rte_pci_driver *drv) +{ + struct rte_pci_device *pci_dev = RTE_DEV_TO_PCI(dev->device); + + dev->dev_ops = &acc101_bbdev_ops; + + ((struct acc101_device *) dev->data->dev_private)->pf_device = + !strcmp(drv->driver.name, + RTE_STR(ACC101PF_DRIVER_NAME)); + ((struct acc101_device *) dev->data->dev_private)->mmio_base = + pci_dev->mem_resource[0].addr; + + rte_bbdev_log_debug("Init device %s [%s] @ vaddr %p paddr %#"PRIx64"", + drv->driver.name, dev->data->name, + (void *)pci_dev->mem_resource[0].addr, + pci_dev->mem_resource[0].phys_addr); +} + +static int acc101_pci_probe(struct rte_pci_driver *pci_drv, + struct rte_pci_device *pci_dev) +{ + struct rte_bbdev *bbdev = NULL; + char dev_name[RTE_BBDEV_NAME_MAX_LEN]; + + if (pci_dev == NULL) { + rte_bbdev_log(ERR, "NULL PCI device"); + return -EINVAL; + } + + rte_pci_device_name(&pci_dev->addr, dev_name, sizeof(dev_name)); + + /* Allocate memory to be used privately by drivers */ + bbdev = rte_bbdev_allocate(pci_dev->device.name); + if (bbdev == NULL) + return -ENODEV; + + /* allocate device private memory */ + bbdev->data->dev_private = rte_zmalloc_socket(dev_name, + sizeof(struct acc101_device), RTE_CACHE_LINE_SIZE, + pci_dev->device.numa_node); + + if (bbdev->data->dev_private == NULL) { + rte_bbdev_log(CRIT, + "Allocate of %zu bytes for device \"%s\" failed", + sizeof(struct acc101_device), dev_name); + rte_bbdev_release(bbdev); + return -ENOMEM; + } + + /* Fill HW specific part of device structure */ + bbdev->device = &pci_dev->device; + bbdev->intr_handle = pci_dev->intr_handle; + bbdev->data->socket_id = pci_dev->device.numa_node; + + /* Invoke ACC101 device initialization function */ + acc101_bbdev_init(bbdev, pci_drv); + + rte_bbdev_log_debug("Initialised bbdev %s (id = %u)", + dev_name, bbdev->data->dev_id); + return 0; +} + +static int acc101_pci_remove(struct rte_pci_device *pci_dev) +{ + struct rte_bbdev *bbdev; + int ret; + uint8_t dev_id; + + if (pci_dev == NULL) + return -EINVAL; + + /* Find device */ + bbdev = rte_bbdev_get_named_dev(pci_dev->device.name); + if (bbdev == NULL) { + rte_bbdev_log(CRIT, + "Couldn't find HW dev \"%s\" to uninitialise it", + pci_dev->device.name); + return -ENODEV; + } + dev_id = bbdev->data->dev_id; + + /* free device private memory before close */ + rte_free(bbdev->data->dev_private); + + /* Close device */ + ret = rte_bbdev_close(dev_id); + if (ret < 0) + rte_bbdev_log(ERR, + "Device %i failed to close during uninit: %i", + dev_id, ret); + + /* release bbdev from library */ + rte_bbdev_release(bbdev); + + rte_bbdev_log_debug("Destroyed bbdev = %u", dev_id); + + return 0; +} + +static struct rte_pci_driver acc101_pci_pf_driver = { + .probe = acc101_pci_probe, + .remove = acc101_pci_remove, + .id_table = pci_id_acc101_pf_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING +}; + +static struct rte_pci_driver acc101_pci_vf_driver = { + .probe = acc101_pci_probe, + .remove = acc101_pci_remove, + .id_table = pci_id_acc101_vf_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING +}; + +RTE_PMD_REGISTER_PCI(ACC101PF_DRIVER_NAME, acc101_pci_pf_driver); +RTE_PMD_REGISTER_PCI_TABLE(ACC101PF_DRIVER_NAME, pci_id_acc101_pf_map); +RTE_PMD_REGISTER_PCI(ACC101VF_DRIVER_NAME, acc101_pci_vf_driver); +RTE_PMD_REGISTER_PCI_TABLE(ACC101VF_DRIVER_NAME, pci_id_acc101_vf_map); diff --git a/drivers/baseband/acc101/rte_acc101_pmd.h b/drivers/baseband/acc101/rte_acc101_pmd.h new file mode 100644 index 0000000..499c341 --- /dev/null +++ b/drivers/baseband/acc101/rte_acc101_pmd.h @@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#ifndef _RTE_ACC101_PMD_H_ +#define _RTE_ACC101_PMD_H_ + +/* Helper macro for logging */ +#define rte_bbdev_log(level, fmt, ...) \ + rte_log(RTE_LOG_ ## level, acc101_logtype, fmt "\n", \ + ##__VA_ARGS__) + +#ifdef RTE_LIBRTE_BBDEV_DEBUG +#define rte_bbdev_log_debug(fmt, ...) \ + rte_bbdev_log(DEBUG, "acc101_pmd: " fmt, \ + ##__VA_ARGS__) +#else +#define rte_bbdev_log_debug(fmt, ...) +#endif + +/* ACC101 PF and VF driver names */ +#define ACC101PF_DRIVER_NAME intel_acc101_pf +#define ACC101VF_DRIVER_NAME intel_acc101_vf + +/* ACC101 PCI vendor & device IDs */ +#define RTE_ACC101_VENDOR_ID (0x8086) +#define RTE_ACC101_PF_DEVICE_ID (0x57c4) +#define RTE_ACC101_VF_DEVICE_ID (0x57c5) + +/* Private data structure for each ACC101 device */ +struct acc101_device { + void *mmio_base; /**< Base address of MMIO registers (BAR0) */ + void *sw_rings_base; /* Base addr of un-aligned memory for sw rings */ + void *sw_rings; /* 64MBs of 64MB aligned memory for sw rings */ + rte_iova_t sw_rings_iova; /* IOVA address of sw_rings */ + + union acc101_harq_layout_data *harq_layout; + /* Number of bytes available for each queue in device, depending on + * how many queues are enabled with configure() + */ + uint32_t sw_ring_size; + uint32_t ddr_size; /* Size in kB */ + uint32_t *tail_ptrs; /* Base address of response tail pointer buffer */ + rte_iova_t tail_ptr_iova; /* IOVA address of tail pointers */ + /* Max number of entries available for each queue in device, depending + * on how many queues are enabled with configure() + */ + uint32_t sw_ring_max_depth; + bool pf_device; /**< True if this is a PF ACC101 device */ + bool configured; /**< True if this ACC101 device is configured */ +}; + +#endif /* _RTE_ACC101_PMD_H_ */ diff --git a/drivers/baseband/acc101/version.map b/drivers/baseband/acc101/version.map new file mode 100644 index 0000000..c2e0723 --- /dev/null +++ b/drivers/baseband/acc101/version.map @@ -0,0 +1,3 @@ +DPDK_22 { + local: *; +}; diff --git a/drivers/baseband/meson.build b/drivers/baseband/meson.build index 686e98b..67c7d22 100644 --- a/drivers/baseband/meson.build +++ b/drivers/baseband/meson.build @@ -7,6 +7,7 @@ endif drivers = [ 'acc100', + 'acc101', 'fpga_5gnr_fec', 'fpga_lte_fec', 'la12xx',