[v6,2/2] bus/auxiliary: introduce auxiliary bus
Checks
Commit Message
Auxiliary bus [1] provides a way to split function into child-devices
representing sub-domains of functionality. Each auxiliary device
represents a part of its parent functionality.
Auxiliary device is identified by unique device name, sysfs path:
/sys/bus/auxiliary/devices/<name>
Devargs legacy syntax ofauxiliary device:
-a auxiliary:<name>[,args...]
Devargs generic syntax of auxiliary device:
-a bus=auxiliary,name=<name>,,/class=<classs>,,/driver=<driver>,,
[1] kernel auxiliary bus document:
https://www.kernel.org/doc/html/latest/driver-api/auxiliary_bus.html
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Cc: Wang Haiyue <haiyue.wang@intel.com>
Cc: Thomas Monjalon <thomas@monjalon.net>
Cc: Kinsella Ray <mdr@ashroe.eu>
---
MAINTAINERS | 5 +
doc/guides/rel_notes/release_21_08.rst | 6 +
drivers/bus/auxiliary/auxiliary_common.c | 411 ++++++++++++++++++++++
drivers/bus/auxiliary/auxiliary_params.c | 59 ++++
drivers/bus/auxiliary/linux/auxiliary.c | 141 ++++++++
drivers/bus/auxiliary/meson.build | 16 +
drivers/bus/auxiliary/private.h | 74 ++++
drivers/bus/auxiliary/rte_bus_auxiliary.h | 201 +++++++++++
drivers/bus/auxiliary/version.map | 7 +
drivers/bus/meson.build | 1 +
10 files changed, 921 insertions(+)
create mode 100644 drivers/bus/auxiliary/auxiliary_common.c
create mode 100644 drivers/bus/auxiliary/auxiliary_params.c
create mode 100644 drivers/bus/auxiliary/linux/auxiliary.c
create mode 100644 drivers/bus/auxiliary/meson.build
create mode 100644 drivers/bus/auxiliary/private.h
create mode 100644 drivers/bus/auxiliary/rte_bus_auxiliary.h
create mode 100644 drivers/bus/auxiliary/version.map
Comments
On 6/25/21 2:47 PM, Xueming Li wrote:
> Auxiliary bus [1] provides a way to split function into child-devices
> representing sub-domains of functionality. Each auxiliary device
> represents a part of its parent functionality.
>
> Auxiliary device is identified by unique device name, sysfs path:
> /sys/bus/auxiliary/devices/<name>
>
> Devargs legacy syntax ofauxiliary device:
Missing space after 'of'
> -a auxiliary:<name>[,args...]
> Devargs generic syntax of auxiliary device:
> -a bus=auxiliary,name=<name>,,/class=<classs>,,/driver=<driver>,,
Are two commas above intentionall? What for?
>
> [1] kernel auxiliary bus document:
> https://www.kernel.org/doc/html/latest/driver-api/auxiliary_bus.html
>
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
With my below notes fixed:
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Cc: Wang Haiyue <haiyue.wang@intel.com>
> Cc: Thomas Monjalon <thomas@monjalon.net>
> Cc: Kinsella Ray <mdr@ashroe.eu>
> ---
> MAINTAINERS | 5 +
> doc/guides/rel_notes/release_21_08.rst | 6 +
> drivers/bus/auxiliary/auxiliary_common.c | 411 ++++++++++++++++++++++
> drivers/bus/auxiliary/auxiliary_params.c | 59 ++++
> drivers/bus/auxiliary/linux/auxiliary.c | 141 ++++++++
> drivers/bus/auxiliary/meson.build | 16 +
> drivers/bus/auxiliary/private.h | 74 ++++
> drivers/bus/auxiliary/rte_bus_auxiliary.h | 201 +++++++++++
> drivers/bus/auxiliary/version.map | 7 +
> drivers/bus/meson.build | 1 +
> 10 files changed, 921 insertions(+)
> create mode 100644 drivers/bus/auxiliary/auxiliary_common.c
> create mode 100644 drivers/bus/auxiliary/auxiliary_params.c
> create mode 100644 drivers/bus/auxiliary/linux/auxiliary.c
> create mode 100644 drivers/bus/auxiliary/meson.build
> create mode 100644 drivers/bus/auxiliary/private.h
> create mode 100644 drivers/bus/auxiliary/rte_bus_auxiliary.h
> create mode 100644 drivers/bus/auxiliary/version.map
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 5877a16971..eaf691ca6a 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -525,6 +525,11 @@ F: doc/guides/mempool/octeontx2.rst
> Bus Drivers
> -----------
>
> +Auxiliary bus driver
Shouldn't it be EXPERIMENTAL?
> +M: Parav Pandit <parav@nvidia.com>
> +M: Xueming Li <xuemingl@nvidia.com>
> +F: drivers/bus/auxiliary/
> +
> Intel FPGA bus
> M: Rosen Xu <rosen.xu@intel.com>
> F: drivers/bus/ifpga/
> diff --git a/doc/guides/rel_notes/release_21_08.rst b/doc/guides/rel_notes/release_21_08.rst
> index a6ecfdf3ce..e7ef4c8a05 100644
> --- a/doc/guides/rel_notes/release_21_08.rst
> +++ b/doc/guides/rel_notes/release_21_08.rst
> @@ -55,6 +55,12 @@ New Features
> Also, make sure to start the actual text at the margin.
> =======================================================
>
> +* **Added auxiliary bus support.**
> +
> + Auxiliary bus provides a way to split function into child-devices
> + representing sub-domains of functionality. Each auxiliary device
> + represents a part of its parent functionality.
> +
>
> Removed Items
> -------------
> diff --git a/drivers/bus/auxiliary/auxiliary_common.c b/drivers/bus/auxiliary/auxiliary_common.c
> new file mode 100644
> index 0000000000..8a75306da5
> --- /dev/null
> +++ b/drivers/bus/auxiliary/auxiliary_common.c
> @@ -0,0 +1,411 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright (c) 2021 NVIDIA Corporation & Affiliates
> + */
> +
> +#include <string.h>
> +#include <inttypes.h>
> +#include <stdint.h>
> +#include <stdbool.h>
> +#include <stdlib.h>
> +#include <stdio.h>
> +#include <sys/queue.h>
> +#include <rte_errno.h>
> +#include <rte_interrupts.h>
> +#include <rte_log.h>
> +#include <rte_bus.h>
> +#include <rte_per_lcore.h>
> +#include <rte_memory.h>
> +#include <rte_eal.h>
> +#include <rte_eal_paging.h>
> +#include <rte_string_fns.h>
> +#include <rte_common.h>
> +#include <rte_devargs.h>
> +
> +#include "private.h"
> +#include "rte_bus_auxiliary.h"
> +
> +static struct rte_devargs *
> +auxiliary_devargs_lookup(const char *name)
> +{
> + struct rte_devargs *devargs;
> +
> + RTE_EAL_DEVARGS_FOREACH(RTE_BUS_AUXILIARY_NAME, devargs) {
> + if (strcmp(devargs->name, name) == 0)
> + return devargs;
> + }
> + return NULL;
> +}
> +
> +/*
> + * Test whether the auxiliary device exist
Missing full stop above.
> + *
> + * Stub for OS not supporting auxiliary bus.
> + */
> +__rte_weak bool
> +auxiliary_dev_exists(const char *name)
> +{
> + RTE_SET_USED(name);
> + return false;
> +}
> +
> +/*
> + * Scan the devices in the auxiliary bus.
> + *
> + * Stub for OS not supporting auxiliary bus.
> + */
> +__rte_weak int
> +auxiliary_scan(void)
> +{
> + return 0;
> +}
> +
> +/*
> + * Update a device's devargs being scanned.
> + *
> + * @param aux_dev
> + * AUXILIARY device.
> + */
> +void
> +auxiliary_on_scan(struct rte_auxiliary_device *aux_dev)
> +{
> + aux_dev->device.devargs = auxiliary_devargs_lookup(aux_dev->name);
> +}
> +
> +/*
> + * Match the auxiliary driver and device using driver function.
> + */
> +bool
> +auxiliary_match(const struct rte_auxiliary_driver *aux_drv,
> + const struct rte_auxiliary_device *aux_dev)
> +{
> + if (aux_drv->match == NULL)
> + return false;
> + return aux_drv->match(aux_dev->name);
> +}
> +
> +/*
> + * Call the probe() function of the driver.
> + */
> +static int
> +rte_auxiliary_probe_one_driver(struct rte_auxiliary_driver *drv,
> + struct rte_auxiliary_device *dev)
> +{
> + enum rte_iova_mode iova_mode;
> + int ret;
> +
> + if ((drv == NULL) || (dev == NULL))
Unnecessary internal parenthesis.
> + return -EINVAL;
> +
> + /* Check if driver supports it. */
> + if (!auxiliary_match(drv, dev))
> + /* Match of device and driver failed */
> + return 1;
> +
> + /* No initialization when marked as blocked, return without error. */
> + if (dev->device.devargs != NULL &&
> + dev->device.devargs->policy == RTE_DEV_BLOCKED) {
> + AUXILIARY_LOG(INFO, "Device is blocked, not initializing");
> + return -1;
> + }
> +
> + if (dev->device.numa_node < 0) {
> + AUXILIARY_LOG(INFO, "Device is not NUMA-aware, defaulting socket to 0");
socket -> NUMA node
> + dev->device.numa_node = 0;
> + }
> +
> + iova_mode = rte_eal_iova_mode();
> + if ((drv->drv_flags & RTE_AUXILIARY_DRV_NEED_IOVA_AS_VA) > 0 &&
> + iova_mode != RTE_IOVA_VA) {
> + AUXILIARY_LOG(ERR, "Driver %s expecting VA IOVA mode but current mode is PA, not initializing",
> + drv->driver.name);
> + return -EINVAL;
> + }
> +
> + dev->driver = drv;
> +
> + AUXILIARY_LOG(INFO, "Probe auxiliary driver: %s device: %s (socket %i)",
socket -> NUMA node
> + drv->driver.name, dev->name, dev->device.numa_node);
> + ret = drv->probe(drv, dev);
> + if (ret != 0)
> + dev->driver = NULL;
> + else
> + dev->device.driver = &drv->driver;
> +
> + return ret;
> +}
> +
> +/*
> + * Call the remove() function of the driver.
> + */
> +static int
> +rte_auxiliary_driver_remove_dev(struct rte_auxiliary_device *dev)
> +{
> + struct rte_auxiliary_driver *drv;
> + int ret = 0;
> +
> + if (dev == NULL)
> + return -EINVAL;
> +
> + drv = dev->driver;
> +
> + AUXILIARY_LOG(DEBUG, "Driver %s remove auxiliary device %s on NUMA socket %i",
socket -> node
> + drv->driver.name, dev->name, dev->device.numa_node);
> +
> + if (drv->remove != NULL) {
> + ret = drv->remove(dev);
> + if (ret < 0)
> + return ret;
> + }
> +
> + /* clear driver structure */
> + dev->driver = NULL;
> + dev->device.driver = NULL;
> +
> + return 0;
> +}
> +
> +/*
> + * Call the probe() function of all registered driver for the given device.
> + * Return < 0 if initialization failed.
> + * Return 1 if no driver is found for this device.
> + */
> +static int
> +auxiliary_probe_all_drivers(struct rte_auxiliary_device *dev)
> +{
> + struct rte_auxiliary_driver *drv;
> + int rc;
> +
> + if (dev == NULL)
> + return -EINVAL;
> +
> + FOREACH_DRIVER_ON_AUXILIARY_BUS(drv) {
> + if (!drv->match(dev->name))
> + continue;
> +
> + rc = rte_auxiliary_probe_one_driver(drv, dev);
> + if (rc < 0)
> + /* negative value is an error */
> + return rc;
> + if (rc > 0)
> + /* positive value means driver doesn't support it */
> + continue;
> + return 0;
> + }
> + return 1;
> +}
> +
> +/*
> + * Scan the content of the auxiliary bus, and call the probe function for
> + * all registered drivers to try to probe discovered devices.
> + */
> +static int
> +auxiliary_probe(void)
> +{
> + struct rte_auxiliary_device *dev = NULL;
> + size_t probed = 0, failed = 0;
> + int ret = 0;
> +
> + FOREACH_DEVICE_ON_AUXILIARY_BUS(dev) {
> + probed++;
> +
> + ret = auxiliary_probe_all_drivers(dev);
> + if (ret < 0) {
> + if (ret != -EEXIST) {
> + AUXILIARY_LOG(ERR, "Requested device %s cannot be used",
> + dev->name);
> + rte_errno = errno;
> + failed++;
> + }
> + ret = 0;
> + }
> + }
> +
> + return (probed && probed == failed) ? -1 : 0;
> +}
> +
> +static int
> +auxiliary_parse(const char *name, void *addr)
> +{
> + struct rte_auxiliary_driver *drv = NULL;
> + const char **out = addr;
> +
> + /* Allow empty device name "auxiliary:" to bypass entire bus scan. */
> + if (strlen(name) == 0)
> + return 0;
> +
> + FOREACH_DRIVER_ON_AUXILIARY_BUS(drv) {
> + if (drv->match(name))
> + break;
> + }
> + if (drv != NULL && addr != NULL)
> + *out = name;
> + return drv != NULL ? 0 : -1;
> +}
> +
> +/* Register a driver */
> +void
> +rte_auxiliary_register(struct rte_auxiliary_driver *driver)
> +{
> + TAILQ_INSERT_TAIL(&auxiliary_bus.driver_list, driver, next);
> + driver->bus = &auxiliary_bus;
> +}
> +
> +/* Unregister a driver */
> +void
> +rte_auxiliary_unregister(struct rte_auxiliary_driver *driver)
> +{
> + TAILQ_REMOVE(&auxiliary_bus.driver_list, driver, next);
> + driver->bus = NULL;
> +}
> +
> +/* Add a device to auxiliary bus */
> +void
> +auxiliary_add_device(struct rte_auxiliary_device *aux_dev)
> +{
> + TAILQ_INSERT_TAIL(&auxiliary_bus.device_list, aux_dev, next);
> +}
> +
> +/* Insert a device into a predefined position in auxiliary bus */
> +void
> +auxiliary_insert_device(struct rte_auxiliary_device *exist_aux_dev,
> + struct rte_auxiliary_device *new_aux_dev)
> +{
> + TAILQ_INSERT_BEFORE(exist_aux_dev, new_aux_dev, next);
> +}
> +
> +/* Remove a device from auxiliary bus */
> +static void
> +rte_auxiliary_remove_device(struct rte_auxiliary_device *auxiliary_dev)
> +{
> + TAILQ_REMOVE(&auxiliary_bus.device_list, auxiliary_dev, next);
> +}
> +
> +static struct rte_device *
> +auxiliary_find_device(const struct rte_device *start, rte_dev_cmp_t cmp,
> + const void *data)
> +{
> + const struct rte_auxiliary_device *pstart;
> + struct rte_auxiliary_device *adev;
> +
> + if (start != NULL) {
> + pstart = RTE_DEV_TO_AUXILIARY_CONST(start);
> + adev = TAILQ_NEXT(pstart, next);
> + } else {
> + adev = TAILQ_FIRST(&auxiliary_bus.device_list);
> + }
> + while (adev != NULL) {
> + if (cmp(&adev->device, data) == 0)
> + return &adev->device;
> + adev = TAILQ_NEXT(adev, next);
> + }
> + return NULL;
> +}
> +
> +static int
> +auxiliary_plug(struct rte_device *dev)
> +{
> + if (!auxiliary_dev_exists(dev->name))
> + return -ENOENT;
> + return auxiliary_probe_all_drivers(RTE_DEV_TO_AUXILIARY(dev));
> +}
> +
> +static int
> +auxiliary_unplug(struct rte_device *dev)
> +{
> + struct rte_auxiliary_device *adev;
> + int ret;
> +
> + adev = RTE_DEV_TO_AUXILIARY(dev);
> + ret = rte_auxiliary_driver_remove_dev(adev);
> + if (ret == 0) {
> + rte_auxiliary_remove_device(adev);
> + rte_devargs_remove(dev->devargs);
> + free(adev);
> + }
> + return ret;
> +}
> +
> +static int
> +auxiliary_dma_map(struct rte_device *dev, void *addr, uint64_t iova, size_t len)
> +{
> + struct rte_auxiliary_device *aux_dev = RTE_DEV_TO_AUXILIARY(dev);
> +
> + if (dev == NULL || aux_dev->driver == NULL) {
> + rte_errno = EINVAL;
> + return -1;
> + }
> + if (aux_dev->driver->dma_map == NULL) {
> + rte_errno = ENOTSUP;
> + return -1;
> + }
> + return aux_dev->driver->dma_map(aux_dev, addr, iova, len);
> +}
> +
> +static int
> +auxiliary_dma_unmap(struct rte_device *dev, void *addr, uint64_t iova,
> + size_t len)
> +{
> + struct rte_auxiliary_device *aux_dev = RTE_DEV_TO_AUXILIARY(dev);
> +
> + if (dev == NULL || aux_dev->driver == NULL) {
> + rte_errno = EINVAL;
> + return -1;
> + }
> + if (aux_dev->driver->dma_unmap == NULL) {
> + rte_errno = ENOTSUP;
> + return -1;
> + }
> + return aux_dev->driver->dma_unmap(aux_dev, addr, iova, len);
> +}
> +
> +bool
> +auxiliary_is_ignored_device(const char *name)
> +{
> + struct rte_devargs *devargs = auxiliary_devargs_lookup(name);
> +
> + switch (auxiliary_bus.bus.conf.scan_mode) {
> + case RTE_BUS_SCAN_ALLOWLIST:
> + if (devargs && devargs->policy == RTE_DEV_ALLOWED)
> + return false;
> + break;
> + case RTE_BUS_SCAN_UNDEFINED:
> + case RTE_BUS_SCAN_BLOCKLIST:
> + if (devargs == NULL || devargs->policy != RTE_DEV_BLOCKED)
> + return false;
> + break;
> + }
> + return true;
> +}
> +
> +static enum rte_iova_mode
> +auxiliary_get_iommu_class(void)
> +{
> + const struct rte_auxiliary_driver *drv;
> +
> + FOREACH_DRIVER_ON_AUXILIARY_BUS(drv) {
> + if ((drv->drv_flags & RTE_AUXILIARY_DRV_NEED_IOVA_AS_VA) > 0)
> + return RTE_IOVA_VA;
> + }
> +
> + return RTE_IOVA_DC;
> +}
> +
> +struct rte_auxiliary_bus auxiliary_bus = {
> + .bus = {
> + .scan = auxiliary_scan,
> + .probe = auxiliary_probe,
> + .find_device = auxiliary_find_device,
> + .plug = auxiliary_plug,
> + .unplug = auxiliary_unplug,
> + .parse = auxiliary_parse,
> + .dma_map = auxiliary_dma_map,
> + .dma_unmap = auxiliary_dma_unmap,
> + .get_iommu_class = auxiliary_get_iommu_class,
> + .dev_iterate = auxiliary_dev_iterate,
> + },
> + .device_list = TAILQ_HEAD_INITIALIZER(auxiliary_bus.device_list),
> + .driver_list = TAILQ_HEAD_INITIALIZER(auxiliary_bus.driver_list),
> +};
> +
> +RTE_REGISTER_BUS(auxiliary, auxiliary_bus.bus);
> +RTE_LOG_REGISTER_DEFAULT(auxiliary_bus_logtype, NOTICE);
> diff --git a/drivers/bus/auxiliary/auxiliary_params.c b/drivers/bus/auxiliary/auxiliary_params.c
> new file mode 100644
> index 0000000000..cd3fa56cb4
> --- /dev/null
> +++ b/drivers/bus/auxiliary/auxiliary_params.c
> @@ -0,0 +1,59 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright (c) 2021 NVIDIA Corporation & Affiliates
> + */
> +
> +#include <string.h>
> +
> +#include <rte_bus.h>
> +#include <rte_dev.h>
> +#include <rte_errno.h>
> +#include <rte_kvargs.h>
> +
> +#include "private.h"
> +#include "rte_bus_auxiliary.h"
> +
> +enum auxiliary_params {
> + RTE_AUXILIARY_PARAM_NAME,
> +};
> +
> +static const char * const auxiliary_params_keys[] = {
> + [RTE_AUXILIARY_PARAM_NAME] = "name",
> +};
> +
> +static int
> +auxiliary_dev_match(const struct rte_device *dev,
> + const void *_kvlist)
> +{
> + const struct rte_kvargs *kvlist = _kvlist;
> + int ret;
> +
> + ret = rte_kvargs_process(kvlist,
> + auxiliary_params_keys[RTE_AUXILIARY_PARAM_NAME],
> + rte_kvargs_strcmp, (void *)(uintptr_t)dev->name);
> +
> + return ret != 0 ? -1 : 0;
> +}
> +
> +void *
> +auxiliary_dev_iterate(const void *start,
> + const char *str,
> + const struct rte_dev_iterator *it __rte_unused)
> +{
> + rte_bus_find_device_t find_device;
> + struct rte_kvargs *kvargs = NULL;
> + struct rte_device *dev;
> +
> + if (str != NULL) {
> + kvargs = rte_kvargs_parse(str, auxiliary_params_keys);
> + if (kvargs == NULL) {
> + AUXILIARY_LOG(ERR, "cannot parse argument list %s",
> + str);
> + rte_errno = EINVAL;
> + return NULL;
> + }
> + }
> + find_device = auxiliary_bus.bus.find_device;
> + dev = find_device(start, auxiliary_dev_match, kvargs);
> + rte_kvargs_free(kvargs);
> + return dev;
> +}
> diff --git a/drivers/bus/auxiliary/linux/auxiliary.c b/drivers/bus/auxiliary/linux/auxiliary.c
> new file mode 100644
> index 0000000000..8464487971
> --- /dev/null
> +++ b/drivers/bus/auxiliary/linux/auxiliary.c
> @@ -0,0 +1,141 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright (c) 2021 NVIDIA Corporation & Affiliates
> + */
> +
> +#include <string.h>
> +#include <dirent.h>
> +
> +#include <rte_log.h>
> +#include <rte_bus.h>
> +#include <rte_malloc.h>
> +#include <rte_devargs.h>
> +#include <rte_memcpy.h>
> +#include <eal_filesystem.h>
> +
> +#include "../rte_bus_auxiliary.h"
> +#include "../private.h"
> +
> +#define AUXILIARY_SYSFS_PATH "/sys/bus/auxiliary/devices"
> +
> +/* Scan one auxiliary sysfs entry, and fill the devices list from it. */
> +static int
> +auxiliary_scan_one(const char *dirname, const char *name)
> +{
> + struct rte_auxiliary_device *dev;
> + struct rte_auxiliary_device *dev2;
> + char filename[PATH_MAX];
> + unsigned long tmp;
> + int ret;
> +
> + dev = malloc(sizeof(*dev));
> + if (dev == NULL)
> + return -1;
> +
> + memset(dev, 0, sizeof(*dev));
> + if (rte_strscpy(dev->name, name, sizeof(dev->name)) < 0) {
> + free(dev);
> + return -1;
> + }
> + dev->device.name = dev->name;
> + dev->device.bus = &auxiliary_bus.bus;
> +
> + /* Get NUMA node, default to 0 if not present */
> + snprintf(filename, sizeof(filename), "%s/%s/numa_node",
> + dirname, name);
> + if (access(filename, F_OK) != -1) {
> + if (eal_parse_sysfs_value(filename, &tmp) == 0)
> + dev->device.numa_node = tmp;
> + else
> + dev->device.numa_node = -1;
> + } else {
> + dev->device.numa_node = 0;
> + }
> +
> + auxiliary_on_scan(dev);
> +
> + /* Device is valid, add in list (sorted) */
> + TAILQ_FOREACH(dev2, &auxiliary_bus.device_list, next) {
> + ret = strcmp(dev->name, dev2->name);
> + if (ret > 0)
> + continue;
> + if (ret < 0) {
> + auxiliary_insert_device(dev2, dev);
> + } else { /* already registered */
> + if (rte_dev_is_probed(&dev2->device) &&
> + dev2->device.devargs != dev->device.devargs) {
> + /* To probe device with new devargs. */
> + rte_devargs_remove(dev2->device.devargs);
> + auxiliary_on_scan(dev2);
> + }
> + free(dev);
> + }
> + return 0;
> + }
> + auxiliary_add_device(dev);
> + return 0;
> +}
> +
> +/*
> + * Test whether the auxiliary device exist
Missing full stop above.
> + */
> +bool
> +auxiliary_dev_exists(const char *name)
> +{
> + DIR *dir;
> + char dirname[PATH_MAX];
> +
> + snprintf(dirname, sizeof(dirname), "%s/%s",
> + AUXILIARY_SYSFS_PATH, name);
> + dir = opendir(dirname);
> + if (dir == NULL)
> + return false;
> + closedir(dir);
> + return true;
> +}
> +
> +/*
> + * Scan the devices in the auxiliary bus
Missing full stop above.
> + */
> +int
> +auxiliary_scan(void)
> +{
> + struct dirent *e;
> + DIR *dir;
> + char dirname[PATH_MAX];
> + struct rte_auxiliary_driver *drv;
> +
> + dir = opendir(AUXILIARY_SYSFS_PATH);
> + if (dir == NULL) {
> + AUXILIARY_LOG(INFO, "%s not found, is auxiliary module loaded?",
> + AUXILIARY_SYSFS_PATH);
> + return 0;
> + }
> +
> + while ((e = readdir(dir)) != NULL) {
> + if (e->d_name[0] == '.')
> + continue;
> +
> + if (auxiliary_is_ignored_device(e->d_name))
> + continue;
> +
> + snprintf(dirname, sizeof(dirname), "%s/%s",
> + AUXILIARY_SYSFS_PATH, e->d_name);
> +
> + /* Ignore if no driver can handle. */
> + FOREACH_DRIVER_ON_AUXILIARY_BUS(drv) {
> + if (drv->match(e->d_name))
> + break;
> + }
> + if (drv == NULL)
> + continue;
> +
> + if (auxiliary_scan_one(dirname, e->d_name) < 0)
> + goto error;
> + }
> + closedir(dir);
> + return 0;
> +
> +error:
> + closedir(dir);
> + return -1;
> +}
> diff --git a/drivers/bus/auxiliary/meson.build b/drivers/bus/auxiliary/meson.build
> new file mode 100644
> index 0000000000..357550eff7
> --- /dev/null
> +++ b/drivers/bus/auxiliary/meson.build
> @@ -0,0 +1,16 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright (c) 2021 NVIDIA Corporation & Affiliates
> +
> +headers = files(
> + 'rte_bus_auxiliary.h',
> +)
> +sources = files(
> + 'auxiliary_common.c',
> + 'auxiliary_params.c',
> +)
> +if is_linux
> + sources += files(
> + 'linux/auxiliary.c',
> + )
> +endif
> +deps += ['kvargs']
> diff --git a/drivers/bus/auxiliary/private.h b/drivers/bus/auxiliary/private.h
> new file mode 100644
> index 0000000000..cb3e849993
> --- /dev/null
> +++ b/drivers/bus/auxiliary/private.h
> @@ -0,0 +1,74 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright (c) 2021 NVIDIA Corporation & Affiliates
> + */
> +
> +#ifndef AUXILIARY_PRIVATE_H
May be add BUS_ prefix at leaat?
> +#define AUXILIARY_PRIVATE_H
> +
> +#include <stdbool.h>
> +#include <stdio.h>
> +
> +#include "rte_bus_auxiliary.h"
> +
> +extern struct rte_auxiliary_bus auxiliary_bus;
> +extern int auxiliary_bus_logtype;
> +
> +#define AUXILIARY_LOG(level, ...) \
> + rte_log(RTE_LOG_ ## level, auxiliary_bus_logtype, \
> + RTE_FMT("auxiliary bus: " RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
> + RTE_FMT_TAIL(__VA_ARGS__,)))
> +
> +/* Auxiliary bus iterators */
> +#define FOREACH_DEVICE_ON_AUXILIARY_BUS(p) \
> + TAILQ_FOREACH(p, &(auxiliary_bus.device_list), next)
> +
> +#define FOREACH_DRIVER_ON_AUXILIARY_BUS(p) \
> + TAILQ_FOREACH(p, &(auxiliary_bus.driver_list), next)
> +
> +bool auxiliary_dev_exists(const char *name);
> +
> +/*
> + * Scan the content of the auxiliary bus, and the devices in the devices
> + * list.
> + */
> +int auxiliary_scan(void);
> +
> +/*
> + * Update a device being scanned.
> + */
> +void auxiliary_on_scan(struct rte_auxiliary_device *aux_dev);
> +
> +/*
> + * Validate whether a device with given auxiliary device should be ignored
> + * or not.
> + */
> +bool auxiliary_is_ignored_device(const char *name);
> +
> +/*
> + * Add an auxiliary device to the auxiliary bus (append to auxiliary device
> + * list). This function also updates the bus references of the auxiliary
> + * device and the generic device object embedded within.
> + */
> +void auxiliary_add_device(struct rte_auxiliary_device *aux_dev);
> +
> +/*
> + * Insert an auxiliary device in the auxiliary bus at a particular location
> + * in the device list. It also updates the auxiliary bus reference of the
> + * new devices to be inserted.
> + */
> +void auxiliary_insert_device(struct rte_auxiliary_device *exist_aux_dev,
> + struct rte_auxiliary_device *new_aux_dev);
> +
> +/*
> + * Match the auxiliary driver and device by driver function
Missing full stop.
> + */
> +bool auxiliary_match(const struct rte_auxiliary_driver *aux_drv,
> + const struct rte_auxiliary_device *aux_dev);
> +
> +/*
> + * Iterate over devices, matching any device against the provided string
Missing full stop.
> + */
> +void *auxiliary_dev_iterate(const void *start, const char *str,
> + const struct rte_dev_iterator *it);
> +
> +#endif /* AUXILIARY_PRIVATE_H */
> diff --git a/drivers/bus/auxiliary/rte_bus_auxiliary.h b/drivers/bus/auxiliary/rte_bus_auxiliary.h
> new file mode 100644
> index 0000000000..16b147e387
> --- /dev/null
> +++ b/drivers/bus/auxiliary/rte_bus_auxiliary.h
> @@ -0,0 +1,201 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright (c) 2021 NVIDIA Corporation & Affiliates
> + */
> +
> +#ifndef RTE_BUS_AUXILIARY_H
> +#define RTE_BUS_AUXILIARY_H
> +
> +/**
> + * @file
> + *
> + * Auxiliary Bus Interface.
> + */
> +
> +#ifdef __cplusplus
> +extern "C" {
> +#endif
> +
> +#include <stdio.h>
> +#include <stdlib.h>
> +#include <limits.h>
> +#include <errno.h>
> +#include <sys/queue.h>
> +#include <stdint.h>
> +#include <inttypes.h>
> +
> +#include <rte_debug.h>
> +#include <rte_interrupts.h>
> +#include <rte_dev.h>
> +#include <rte_bus.h>
> +#include <rte_kvargs.h>
> +
> +#define RTE_BUS_AUXILIARY_NAME "auxiliary"
> +
> +/* Forward declarations */
> +struct rte_auxiliary_driver;
> +struct rte_auxiliary_bus;
> +struct rte_auxiliary_device;
> +
> +/**
> + * Match function for the driver to decide if device can be handled.
> + *
> + * @param name
> + * Pointer to the auxiliary device name.
> + * @return
> + * Whether the driver can handle the auxiliary device.
> + */
> +typedef bool(rte_auxiliary_match_t)(const char *name);
> +
> +/**
> + * Initialization function for the driver called during auxiliary probing.
> + *
> + * @param drv
> + * Pointer to the auxiliary driver.
> + * @param dev
> + * Pointer to the auxiliary device.
> + * @return
> + * - 0 On success.
> + * - Negative value and rte_errno is set otherwise.
> + */
> +typedef int(rte_auxiliary_probe_t)(struct rte_auxiliary_driver *drv,
> + struct rte_auxiliary_device *dev);
> +
> +/**
> + * Uninitialization function for the driver called during hotplugging.
> + *
> + * @param dev
> + * Pointer to the auxiliary device.
> + * @return
> + * - 0 On success.
> + * - Negative value and rte_errno is set otherwise.
> + */
> +typedef int (rte_auxiliary_remove_t)(struct rte_auxiliary_device *dev);
> +
> +/**
> + * Driver-specific DMA mapping. After a successful call the device
> + * will be able to read/write from/to this segment.
> + *
> + * @param dev
> + * Pointer to the auxiliary device.
> + * @param addr
> + * Starting virtual address of memory to be mapped.
> + * @param iova
> + * Starting IOVA address of memory to be mapped.
> + * @param len
> + * Length of memory segment being mapped.
> + * @return
> + * - 0 On success.
> + * - Negative value and rte_errno is set otherwise.
> + */
> +typedef int (rte_auxiliary_dma_map_t)(struct rte_auxiliary_device *dev,
> + void *addr, uint64_t iova, size_t len);
> +
> +/**
> + * Driver-specific DMA un-mapping. After a successful call the device
> + * will not be able to read/write from/to this segment.
> + *
> + * @param dev
> + * Pointer to the auxiliary device.
> + * @param addr
> + * Starting virtual address of memory to be unmapped.
> + * @param iova
> + * Starting IOVA address of memory to be unmapped.
> + * @param len
> + * Length of memory segment being unmapped.
> + * @return
> + * - 0 On success.
> + * - Negative value and rte_errno is set otherwise.
> + */
> +typedef int (rte_auxiliary_dma_unmap_t)(struct rte_auxiliary_device *dev,
> + void *addr, uint64_t iova, size_t len);
> +
> +/**
> + * A structure describing an auxiliary device.
> + */
> +struct rte_auxiliary_device {
> + TAILQ_ENTRY(rte_auxiliary_device) next; /**< Next probed device. */
> + struct rte_device device; /**< Inherit core device */
> + char name[RTE_DEV_NAME_MAX_LEN + 1]; /**< ASCII device name */
> + struct rte_intr_handle intr_handle; /**< Interrupt handle */
> + struct rte_auxiliary_driver *driver; /**< Device driver */
> +};
> +
> +/** List of auxiliary devices */
> +TAILQ_HEAD(rte_auxiliary_device_list, rte_auxiliary_device);
> +/** List of auxiliary drivers */
> +TAILQ_HEAD(rte_auxiliary_driver_list, rte_auxiliary_driver);
Shouldn't we hide rte_auxiliary_device inside the library take
API/ABI stability into account? Or will be it DPDK internal anyway? If
so, it should be done INTERNAL from the very
beginning.
> +
> +/**
> + * Structure describing the auxiliary bus
> + */
> +struct rte_auxiliary_bus {
> + struct rte_bus bus; /**< Inherit the generic class */
> + struct rte_auxiliary_device_list device_list; /**< List of devices */
> + struct rte_auxiliary_driver_list driver_list; /**< List of drivers */
> +};
It looks internal. The following forward declaration should be
sufficient to build.
struct rte_auxiliary_bus;
> +
> +/**
> + * A structure describing an auxiliary driver.
> + */
> +struct rte_auxiliary_driver {
> + TAILQ_ENTRY(rte_auxiliary_driver) next; /**< Next in list. */
> + struct rte_driver driver; /**< Inherit core driver. */
> + struct rte_auxiliary_bus *bus; /**< Auxiliary bus reference. */
> + rte_auxiliary_match_t *match; /**< Device match function. */
> + rte_auxiliary_probe_t *probe; /**< Device probe function. */
> + rte_auxiliary_remove_t *remove; /**< Device remove function. */
> + rte_auxiliary_dma_map_t *dma_map; /**< Device DMA map function. */
> + rte_auxiliary_dma_unmap_t *dma_unmap; /**< Device DMA unmap function. */
> + uint32_t drv_flags; /**< Flags RTE_AUXILIARY_DRV_*. */
> +};
> +
> +/**
> + * @internal
> + * Helper macro for drivers that need to convert to struct rte_auxiliary_device.
> + */
> +#define RTE_DEV_TO_AUXILIARY(ptr) \
> + container_of(ptr, struct rte_auxiliary_device, device)
> +
> +#define RTE_DEV_TO_AUXILIARY_CONST(ptr) \
> + container_of(ptr, const struct rte_auxiliary_device, device)
> +
> +#define RTE_ETH_DEV_TO_AUXILIARY(eth_dev) \
> + RTE_DEV_TO_AUXILIARY((eth_dev)->device)
> +
> +/** Device driver needs IOVA as VA and cannot work with IOVA as PA */
> +#define RTE_AUXILIARY_DRV_NEED_IOVA_AS_VA 0x002
> +
> +/**
Don't we need EXPERIMENTAL notice here?
> + * Register an auxiliary driver.
> + *
> + * @param driver
> + * A pointer to a rte_auxiliary_driver structure describing the driver
> + * to be registered.
> + */
> +__rte_experimental
> +void rte_auxiliary_register(struct rte_auxiliary_driver *driver);
> +
> +/** Helper for auxiliary device registration from driver instance */
> +#define RTE_PMD_REGISTER_AUXILIARY(nm, auxiliary_drv) \
> + RTE_INIT(auxiliaryinitfn_ ##nm) \
> + { \
> + (auxiliary_drv).driver.name = RTE_STR(nm); \
> + rte_auxiliary_register(&(auxiliary_drv)); \
> + } \
> + RTE_PMD_EXPORT_NAME(nm, __COUNTER__)
> +
> +/**
Don't we need EXPERIMENTAL notice here?
> + * Unregister an auxiliary driver.
> + *
> + * @param driver
> + * A pointer to a rte_auxiliary_driver structure describing the driver
> + * to be unregistered.
> + */
> +__rte_experimental
> +void rte_auxiliary_unregister(struct rte_auxiliary_driver *driver);
> +
> +#ifdef __cplusplus
> +}
> +#endif
> +
> +#endif /* RTE_BUS_AUXILIARY_H */
> diff --git a/drivers/bus/auxiliary/version.map b/drivers/bus/auxiliary/version.map
> new file mode 100644
> index 0000000000..a52260657c
> --- /dev/null
> +++ b/drivers/bus/auxiliary/version.map
> @@ -0,0 +1,7 @@
> +EXPERIMENTAL {
> + global:
> +
> + # added in 21.08
> + rte_auxiliary_register;
> + rte_auxiliary_unregister;
> +};
> diff --git a/drivers/bus/meson.build b/drivers/bus/meson.build
> index 410058de3a..45eab5233d 100644
> --- a/drivers/bus/meson.build
> +++ b/drivers/bus/meson.build
> @@ -2,6 +2,7 @@
> # Copyright(c) 2017 Intel Corporation
>
> drivers = [
> + 'auxiliary',
> 'dpaa',
> 'fslmc',
> 'ifpga',
>
Hi Andrew,
Thanks very much all the good suggestions, v7 posted.
Best Regards,
Xueming
> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Monday, July 5, 2021 12:13 AM
> To: Xueming(Steven) Li <xuemingl@nvidia.com>
> Cc: dev@dpdk.org; Wang Haiyue <haiyue.wang@intel.com>; NBU-Contact-Thomas Monjalon <thomas@monjalon.net>; Kinsella Ray
> <mdr@ashroe.eu>; Neil Horman <nhorman@tuxdriver.com>
> Subject: Re: [dpdk-dev] [PATCH v6 2/2] bus/auxiliary: introduce auxiliary bus
>
> On 6/25/21 2:47 PM, Xueming Li wrote:
> > Auxiliary bus [1] provides a way to split function into child-devices
> > representing sub-domains of functionality. Each auxiliary device
> > represents a part of its parent functionality.
> >
> > Auxiliary device is identified by unique device name, sysfs path:
> > /sys/bus/auxiliary/devices/<name>
> >
> > Devargs legacy syntax ofauxiliary device:
>
> Missing space after 'of'
>
> > -a auxiliary:<name>[,args...]
> > Devargs generic syntax of auxiliary device:
> > -a bus=auxiliary,name=<name>,,/class=<classs>,,/driver=<driver>,,
>
> Are two commas above intentionall? What for?
>
> >
> > [1] kernel auxiliary bus document:
> > https://www.kernel.org/doc/html/latest/driver-api/auxiliary_bus.html
> >
> > Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>
> With my below notes fixed:
>
> Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>
> > Cc: Wang Haiyue <haiyue.wang@intel.com>
> > Cc: Thomas Monjalon <thomas@monjalon.net>
> > Cc: Kinsella Ray <mdr@ashroe.eu>
> > ---
> > MAINTAINERS | 5 +
> > doc/guides/rel_notes/release_21_08.rst | 6 +
> > drivers/bus/auxiliary/auxiliary_common.c | 411
> > ++++++++++++++++++++++ drivers/bus/auxiliary/auxiliary_params.c | 59 ++++
> > drivers/bus/auxiliary/linux/auxiliary.c | 141 ++++++++
> > drivers/bus/auxiliary/meson.build | 16 +
> > drivers/bus/auxiliary/private.h | 74 ++++
> > drivers/bus/auxiliary/rte_bus_auxiliary.h | 201 +++++++++++
> > drivers/bus/auxiliary/version.map | 7 +
> > drivers/bus/meson.build | 1 +
> > 10 files changed, 921 insertions(+)
> > create mode 100644 drivers/bus/auxiliary/auxiliary_common.c
> > create mode 100644 drivers/bus/auxiliary/auxiliary_params.c
> > create mode 100644 drivers/bus/auxiliary/linux/auxiliary.c
> > create mode 100644 drivers/bus/auxiliary/meson.build create mode
> > 100644 drivers/bus/auxiliary/private.h create mode 100644
> > drivers/bus/auxiliary/rte_bus_auxiliary.h
> > create mode 100644 drivers/bus/auxiliary/version.map
> >
> > diff --git a/MAINTAINERS b/MAINTAINERS index 5877a16971..eaf691ca6a
> > 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -525,6 +525,11 @@ F: doc/guides/mempool/octeontx2.rst Bus Drivers
> > -----------
> >
> > +Auxiliary bus driver
>
> Shouldn't it be EXPERIMENTAL?
>
> > +M: Parav Pandit <parav@nvidia.com>
> > +M: Xueming Li <xuemingl@nvidia.com>
> > +F: drivers/bus/auxiliary/
> > +
> > Intel FPGA bus
> > M: Rosen Xu <rosen.xu@intel.com>
> > F: drivers/bus/ifpga/
> > diff --git a/doc/guides/rel_notes/release_21_08.rst
> > b/doc/guides/rel_notes/release_21_08.rst
> > index a6ecfdf3ce..e7ef4c8a05 100644
> > --- a/doc/guides/rel_notes/release_21_08.rst
> > +++ b/doc/guides/rel_notes/release_21_08.rst
> > @@ -55,6 +55,12 @@ New Features
> > Also, make sure to start the actual text at the margin.
> > =======================================================
> >
> > +* **Added auxiliary bus support.**
> > +
> > + Auxiliary bus provides a way to split function into child-devices
> > + representing sub-domains of functionality. Each auxiliary device
> > + represents a part of its parent functionality.
> > +
> >
> > Removed Items
> > -------------
> > diff --git a/drivers/bus/auxiliary/auxiliary_common.c
> > b/drivers/bus/auxiliary/auxiliary_common.c
> > new file mode 100644
> > index 0000000000..8a75306da5
> > --- /dev/null
> > +++ b/drivers/bus/auxiliary/auxiliary_common.c
> > @@ -0,0 +1,411 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright (c) 2021 NVIDIA Corporation & Affiliates */
> > +
> > +#include <string.h>
> > +#include <inttypes.h>
> > +#include <stdint.h>
> > +#include <stdbool.h>
> > +#include <stdlib.h>
> > +#include <stdio.h>
> > +#include <sys/queue.h>
> > +#include <rte_errno.h>
> > +#include <rte_interrupts.h>
> > +#include <rte_log.h>
> > +#include <rte_bus.h>
> > +#include <rte_per_lcore.h>
> > +#include <rte_memory.h>
> > +#include <rte_eal.h>
> > +#include <rte_eal_paging.h>
> > +#include <rte_string_fns.h>
> > +#include <rte_common.h>
> > +#include <rte_devargs.h>
> > +
> > +#include "private.h"
> > +#include "rte_bus_auxiliary.h"
> > +
> > +static struct rte_devargs *
> > +auxiliary_devargs_lookup(const char *name) {
> > + struct rte_devargs *devargs;
> > +
> > + RTE_EAL_DEVARGS_FOREACH(RTE_BUS_AUXILIARY_NAME, devargs) {
> > + if (strcmp(devargs->name, name) == 0)
> > + return devargs;
> > + }
> > + return NULL;
> > +}
> > +
> > +/*
> > + * Test whether the auxiliary device exist
>
> Missing full stop above.
>
> > + *
> > + * Stub for OS not supporting auxiliary bus.
> > + */
> > +__rte_weak bool
> > +auxiliary_dev_exists(const char *name) {
> > + RTE_SET_USED(name);
> > + return false;
> > +}
> > +
> > +/*
> > + * Scan the devices in the auxiliary bus.
> > + *
> > + * Stub for OS not supporting auxiliary bus.
> > + */
> > +__rte_weak int
> > +auxiliary_scan(void)
> > +{
> > + return 0;
> > +}
> > +
> > +/*
> > + * Update a device's devargs being scanned.
> > + *
> > + * @param aux_dev
> > + * AUXILIARY device.
> > + */
> > +void
> > +auxiliary_on_scan(struct rte_auxiliary_device *aux_dev) {
> > + aux_dev->device.devargs = auxiliary_devargs_lookup(aux_dev->name);
> > +}
> > +
> > +/*
> > + * Match the auxiliary driver and device using driver function.
> > + */
> > +bool
> > +auxiliary_match(const struct rte_auxiliary_driver *aux_drv,
> > + const struct rte_auxiliary_device *aux_dev) {
> > + if (aux_drv->match == NULL)
> > + return false;
> > + return aux_drv->match(aux_dev->name); }
> > +
> > +/*
> > + * Call the probe() function of the driver.
> > + */
> > +static int
> > +rte_auxiliary_probe_one_driver(struct rte_auxiliary_driver *drv,
> > + struct rte_auxiliary_device *dev)
> > +{
> > + enum rte_iova_mode iova_mode;
> > + int ret;
> > +
> > + if ((drv == NULL) || (dev == NULL))
>
> Unnecessary internal parenthesis.
>
> > + return -EINVAL;
> > +
> > + /* Check if driver supports it. */
> > + if (!auxiliary_match(drv, dev))
> > + /* Match of device and driver failed */
> > + return 1;
> > +
> > + /* No initialization when marked as blocked, return without error. */
> > + if (dev->device.devargs != NULL &&
> > + dev->device.devargs->policy == RTE_DEV_BLOCKED) {
> > + AUXILIARY_LOG(INFO, "Device is blocked, not initializing");
> > + return -1;
> > + }
> > +
> > + if (dev->device.numa_node < 0) {
> > + AUXILIARY_LOG(INFO, "Device is not NUMA-aware, defaulting socket to 0");
>
> socket -> NUMA node
>
> > + dev->device.numa_node = 0;
> > + }
> > +
> > + iova_mode = rte_eal_iova_mode();
> > + if ((drv->drv_flags & RTE_AUXILIARY_DRV_NEED_IOVA_AS_VA) > 0 &&
> > + iova_mode != RTE_IOVA_VA) {
> > + AUXILIARY_LOG(ERR, "Driver %s expecting VA IOVA mode but current mode is PA, not initializing",
> > + drv->driver.name);
> > + return -EINVAL;
> > + }
> > +
> > + dev->driver = drv;
> > +
> > + AUXILIARY_LOG(INFO, "Probe auxiliary driver: %s device: %s (socket %i)",
>
> socket -> NUMA node
>
> > + drv->driver.name, dev->name, dev->device.numa_node);
> > + ret = drv->probe(drv, dev);
> > + if (ret != 0)
> > + dev->driver = NULL;
> > + else
> > + dev->device.driver = &drv->driver;
> > +
> > + return ret;
> > +}
> > +
> > +/*
> > + * Call the remove() function of the driver.
> > + */
> > +static int
> > +rte_auxiliary_driver_remove_dev(struct rte_auxiliary_device *dev)
> > +{
> > + struct rte_auxiliary_driver *drv;
> > + int ret = 0;
> > +
> > + if (dev == NULL)
> > + return -EINVAL;
> > +
> > + drv = dev->driver;
> > +
> > + AUXILIARY_LOG(DEBUG, "Driver %s remove auxiliary device %s on NUMA socket %i",
>
> socket -> node
>
> > + drv->driver.name, dev->name, dev->device.numa_node);
> > +
> > + if (drv->remove != NULL) {
> > + ret = drv->remove(dev);
> > + if (ret < 0)
> > + return ret;
> > + }
> > +
> > + /* clear driver structure */
> > + dev->driver = NULL;
> > + dev->device.driver = NULL;
> > +
> > + return 0;
> > +}
> > +
> > +/*
> > + * Call the probe() function of all registered driver for the given device.
> > + * Return < 0 if initialization failed.
> > + * Return 1 if no driver is found for this device.
> > + */
> > +static int
> > +auxiliary_probe_all_drivers(struct rte_auxiliary_device *dev)
> > +{
> > + struct rte_auxiliary_driver *drv;
> > + int rc;
> > +
> > + if (dev == NULL)
> > + return -EINVAL;
> > +
> > + FOREACH_DRIVER_ON_AUXILIARY_BUS(drv) {
> > + if (!drv->match(dev->name))
> > + continue;
> > +
> > + rc = rte_auxiliary_probe_one_driver(drv, dev);
> > + if (rc < 0)
> > + /* negative value is an error */
> > + return rc;
> > + if (rc > 0)
> > + /* positive value means driver doesn't support it */
> > + continue;
> > + return 0;
> > + }
> > + return 1;
> > +}
> > +
> > +/*
> > + * Scan the content of the auxiliary bus, and call the probe function for
> > + * all registered drivers to try to probe discovered devices.
> > + */
> > +static int
> > +auxiliary_probe(void)
> > +{
> > + struct rte_auxiliary_device *dev = NULL;
> > + size_t probed = 0, failed = 0;
> > + int ret = 0;
> > +
> > + FOREACH_DEVICE_ON_AUXILIARY_BUS(dev) {
> > + probed++;
> > +
> > + ret = auxiliary_probe_all_drivers(dev);
> > + if (ret < 0) {
> > + if (ret != -EEXIST) {
> > + AUXILIARY_LOG(ERR, "Requested device %s cannot be used",
> > + dev->name);
> > + rte_errno = errno;
> > + failed++;
> > + }
> > + ret = 0;
> > + }
> > + }
> > +
> > + return (probed && probed == failed) ? -1 : 0;
> > +}
> > +
> > +static int
> > +auxiliary_parse(const char *name, void *addr)
> > +{
> > + struct rte_auxiliary_driver *drv = NULL;
> > + const char **out = addr;
> > +
> > + /* Allow empty device name "auxiliary:" to bypass entire bus scan. */
> > + if (strlen(name) == 0)
> > + return 0;
> > +
> > + FOREACH_DRIVER_ON_AUXILIARY_BUS(drv) {
> > + if (drv->match(name))
> > + break;
> > + }
> > + if (drv != NULL && addr != NULL)
> > + *out = name;
> > + return drv != NULL ? 0 : -1;
> > +}
> > +
> > +/* Register a driver */
> > +void
> > +rte_auxiliary_register(struct rte_auxiliary_driver *driver)
> > +{
> > + TAILQ_INSERT_TAIL(&auxiliary_bus.driver_list, driver, next);
> > + driver->bus = &auxiliary_bus;
> > +}
> > +
> > +/* Unregister a driver */
> > +void
> > +rte_auxiliary_unregister(struct rte_auxiliary_driver *driver)
> > +{
> > + TAILQ_REMOVE(&auxiliary_bus.driver_list, driver, next);
> > + driver->bus = NULL;
> > +}
> > +
> > +/* Add a device to auxiliary bus */
> > +void
> > +auxiliary_add_device(struct rte_auxiliary_device *aux_dev)
> > +{
> > + TAILQ_INSERT_TAIL(&auxiliary_bus.device_list, aux_dev, next);
> > +}
> > +
> > +/* Insert a device into a predefined position in auxiliary bus */
> > +void
> > +auxiliary_insert_device(struct rte_auxiliary_device *exist_aux_dev,
> > + struct rte_auxiliary_device *new_aux_dev)
> > +{
> > + TAILQ_INSERT_BEFORE(exist_aux_dev, new_aux_dev, next);
> > +}
> > +
> > +/* Remove a device from auxiliary bus */
> > +static void
> > +rte_auxiliary_remove_device(struct rte_auxiliary_device *auxiliary_dev)
> > +{
> > + TAILQ_REMOVE(&auxiliary_bus.device_list, auxiliary_dev, next);
> > +}
> > +
> > +static struct rte_device *
> > +auxiliary_find_device(const struct rte_device *start, rte_dev_cmp_t cmp,
> > + const void *data)
> > +{
> > + const struct rte_auxiliary_device *pstart;
> > + struct rte_auxiliary_device *adev;
> > +
> > + if (start != NULL) {
> > + pstart = RTE_DEV_TO_AUXILIARY_CONST(start);
> > + adev = TAILQ_NEXT(pstart, next);
> > + } else {
> > + adev = TAILQ_FIRST(&auxiliary_bus.device_list);
> > + }
> > + while (adev != NULL) {
> > + if (cmp(&adev->device, data) == 0)
> > + return &adev->device;
> > + adev = TAILQ_NEXT(adev, next);
> > + }
> > + return NULL;
> > +}
> > +
> > +static int
> > +auxiliary_plug(struct rte_device *dev)
> > +{
> > + if (!auxiliary_dev_exists(dev->name))
> > + return -ENOENT;
> > + return auxiliary_probe_all_drivers(RTE_DEV_TO_AUXILIARY(dev));
> > +}
> > +
> > +static int
> > +auxiliary_unplug(struct rte_device *dev)
> > +{
> > + struct rte_auxiliary_device *adev;
> > + int ret;
> > +
> > + adev = RTE_DEV_TO_AUXILIARY(dev);
> > + ret = rte_auxiliary_driver_remove_dev(adev);
> > + if (ret == 0) {
> > + rte_auxiliary_remove_device(adev);
> > + rte_devargs_remove(dev->devargs);
> > + free(adev);
> > + }
> > + return ret;
> > +}
> > +
> > +static int
> > +auxiliary_dma_map(struct rte_device *dev, void *addr, uint64_t iova, size_t len)
> > +{
> > + struct rte_auxiliary_device *aux_dev = RTE_DEV_TO_AUXILIARY(dev);
> > +
> > + if (dev == NULL || aux_dev->driver == NULL) {
> > + rte_errno = EINVAL;
> > + return -1;
> > + }
> > + if (aux_dev->driver->dma_map == NULL) {
> > + rte_errno = ENOTSUP;
> > + return -1;
> > + }
> > + return aux_dev->driver->dma_map(aux_dev, addr, iova, len);
> > +}
> > +
> > +static int
> > +auxiliary_dma_unmap(struct rte_device *dev, void *addr, uint64_t iova,
> > + size_t len)
> > +{
> > + struct rte_auxiliary_device *aux_dev = RTE_DEV_TO_AUXILIARY(dev);
> > +
> > + if (dev == NULL || aux_dev->driver == NULL) {
> > + rte_errno = EINVAL;
> > + return -1;
> > + }
> > + if (aux_dev->driver->dma_unmap == NULL) {
> > + rte_errno = ENOTSUP;
> > + return -1;
> > + }
> > + return aux_dev->driver->dma_unmap(aux_dev, addr, iova, len);
> > +}
> > +
> > +bool
> > +auxiliary_is_ignored_device(const char *name)
> > +{
> > + struct rte_devargs *devargs = auxiliary_devargs_lookup(name);
> > +
> > + switch (auxiliary_bus.bus.conf.scan_mode) {
> > + case RTE_BUS_SCAN_ALLOWLIST:
> > + if (devargs && devargs->policy == RTE_DEV_ALLOWED)
> > + return false;
> > + break;
> > + case RTE_BUS_SCAN_UNDEFINED:
> > + case RTE_BUS_SCAN_BLOCKLIST:
> > + if (devargs == NULL || devargs->policy != RTE_DEV_BLOCKED)
> > + return false;
> > + break;
> > + }
> > + return true;
> > +}
> > +
> > +static enum rte_iova_mode
> > +auxiliary_get_iommu_class(void)
> > +{
> > + const struct rte_auxiliary_driver *drv;
> > +
> > + FOREACH_DRIVER_ON_AUXILIARY_BUS(drv) {
> > + if ((drv->drv_flags & RTE_AUXILIARY_DRV_NEED_IOVA_AS_VA) > 0)
> > + return RTE_IOVA_VA;
> > + }
> > +
> > + return RTE_IOVA_DC;
> > +}
> > +
> > +struct rte_auxiliary_bus auxiliary_bus = {
> > + .bus = {
> > + .scan = auxiliary_scan,
> > + .probe = auxiliary_probe,
> > + .find_device = auxiliary_find_device,
> > + .plug = auxiliary_plug,
> > + .unplug = auxiliary_unplug,
> > + .parse = auxiliary_parse,
> > + .dma_map = auxiliary_dma_map,
> > + .dma_unmap = auxiliary_dma_unmap,
> > + .get_iommu_class = auxiliary_get_iommu_class,
> > + .dev_iterate = auxiliary_dev_iterate,
> > + },
> > + .device_list = TAILQ_HEAD_INITIALIZER(auxiliary_bus.device_list),
> > + .driver_list = TAILQ_HEAD_INITIALIZER(auxiliary_bus.driver_list),
> > +};
> > +
> > +RTE_REGISTER_BUS(auxiliary, auxiliary_bus.bus);
> > +RTE_LOG_REGISTER_DEFAULT(auxiliary_bus_logtype, NOTICE);
> > diff --git a/drivers/bus/auxiliary/auxiliary_params.c b/drivers/bus/auxiliary/auxiliary_params.c
> > new file mode 100644
> > index 0000000000..cd3fa56cb4
> > --- /dev/null
> > +++ b/drivers/bus/auxiliary/auxiliary_params.c
> > @@ -0,0 +1,59 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright (c) 2021 NVIDIA Corporation & Affiliates
> > + */
> > +
> > +#include <string.h>
> > +
> > +#include <rte_bus.h>
> > +#include <rte_dev.h>
> > +#include <rte_errno.h>
> > +#include <rte_kvargs.h>
> > +
> > +#include "private.h"
> > +#include "rte_bus_auxiliary.h"
> > +
> > +enum auxiliary_params {
> > + RTE_AUXILIARY_PARAM_NAME,
> > +};
> > +
> > +static const char * const auxiliary_params_keys[] = {
> > + [RTE_AUXILIARY_PARAM_NAME] = "name",
> > +};
> > +
> > +static int
> > +auxiliary_dev_match(const struct rte_device *dev,
> > + const void *_kvlist)
> > +{
> > + const struct rte_kvargs *kvlist = _kvlist;
> > + int ret;
> > +
> > + ret = rte_kvargs_process(kvlist,
> > + auxiliary_params_keys[RTE_AUXILIARY_PARAM_NAME],
> > + rte_kvargs_strcmp, (void *)(uintptr_t)dev->name);
> > +
> > + return ret != 0 ? -1 : 0;
> > +}
> > +
> > +void *
> > +auxiliary_dev_iterate(const void *start,
> > + const char *str,
> > + const struct rte_dev_iterator *it __rte_unused)
> > +{
> > + rte_bus_find_device_t find_device;
> > + struct rte_kvargs *kvargs = NULL;
> > + struct rte_device *dev;
> > +
> > + if (str != NULL) {
> > + kvargs = rte_kvargs_parse(str, auxiliary_params_keys);
> > + if (kvargs == NULL) {
> > + AUXILIARY_LOG(ERR, "cannot parse argument list %s",
> > + str);
> > + rte_errno = EINVAL;
> > + return NULL;
> > + }
> > + }
> > + find_device = auxiliary_bus.bus.find_device;
> > + dev = find_device(start, auxiliary_dev_match, kvargs);
> > + rte_kvargs_free(kvargs);
> > + return dev;
> > +}
> > diff --git a/drivers/bus/auxiliary/linux/auxiliary.c b/drivers/bus/auxiliary/linux/auxiliary.c
> > new file mode 100644
> > index 0000000000..8464487971
> > --- /dev/null
> > +++ b/drivers/bus/auxiliary/linux/auxiliary.c
> > @@ -0,0 +1,141 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright (c) 2021 NVIDIA Corporation & Affiliates
> > + */
> > +
> > +#include <string.h>
> > +#include <dirent.h>
> > +
> > +#include <rte_log.h>
> > +#include <rte_bus.h>
> > +#include <rte_malloc.h>
> > +#include <rte_devargs.h>
> > +#include <rte_memcpy.h>
> > +#include <eal_filesystem.h>
> > +
> > +#include "../rte_bus_auxiliary.h"
> > +#include "../private.h"
> > +
> > +#define AUXILIARY_SYSFS_PATH "/sys/bus/auxiliary/devices"
> > +
> > +/* Scan one auxiliary sysfs entry, and fill the devices list from it. */
> > +static int
> > +auxiliary_scan_one(const char *dirname, const char *name)
> > +{
> > + struct rte_auxiliary_device *dev;
> > + struct rte_auxiliary_device *dev2;
> > + char filename[PATH_MAX];
> > + unsigned long tmp;
> > + int ret;
> > +
> > + dev = malloc(sizeof(*dev));
> > + if (dev == NULL)
> > + return -1;
> > +
> > + memset(dev, 0, sizeof(*dev));
> > + if (rte_strscpy(dev->name, name, sizeof(dev->name)) < 0) {
> > + free(dev);
> > + return -1;
> > + }
> > + dev->device.name = dev->name;
> > + dev->device.bus = &auxiliary_bus.bus;
> > +
> > + /* Get NUMA node, default to 0 if not present */
> > + snprintf(filename, sizeof(filename), "%s/%s/numa_node",
> > + dirname, name);
> > + if (access(filename, F_OK) != -1) {
> > + if (eal_parse_sysfs_value(filename, &tmp) == 0)
> > + dev->device.numa_node = tmp;
> > + else
> > + dev->device.numa_node = -1;
> > + } else {
> > + dev->device.numa_node = 0;
> > + }
> > +
> > + auxiliary_on_scan(dev);
> > +
> > + /* Device is valid, add in list (sorted) */
> > + TAILQ_FOREACH(dev2, &auxiliary_bus.device_list, next) {
> > + ret = strcmp(dev->name, dev2->name);
> > + if (ret > 0)
> > + continue;
> > + if (ret < 0) {
> > + auxiliary_insert_device(dev2, dev);
> > + } else { /* already registered */
> > + if (rte_dev_is_probed(&dev2->device) &&
> > + dev2->device.devargs != dev->device.devargs) {
> > + /* To probe device with new devargs. */
> > + rte_devargs_remove(dev2->device.devargs);
> > + auxiliary_on_scan(dev2);
> > + }
> > + free(dev);
> > + }
> > + return 0;
> > + }
> > + auxiliary_add_device(dev);
> > + return 0;
> > +}
> > +
> > +/*
> > + * Test whether the auxiliary device exist
>
> Missing full stop above.
>
> > + */
> > +bool
> > +auxiliary_dev_exists(const char *name)
> > +{
> > + DIR *dir;
> > + char dirname[PATH_MAX];
> > +
> > + snprintf(dirname, sizeof(dirname), "%s/%s",
> > + AUXILIARY_SYSFS_PATH, name);
> > + dir = opendir(dirname);
> > + if (dir == NULL)
> > + return false;
> > + closedir(dir);
> > + return true;
> > +}
> > +
> > +/*
> > + * Scan the devices in the auxiliary bus
>
> Missing full stop above.
>
> > + */
> > +int
> > +auxiliary_scan(void)
> > +{
> > + struct dirent *e;
> > + DIR *dir;
> > + char dirname[PATH_MAX];
> > + struct rte_auxiliary_driver *drv;
> > +
> > + dir = opendir(AUXILIARY_SYSFS_PATH);
> > + if (dir == NULL) {
> > + AUXILIARY_LOG(INFO, "%s not found, is auxiliary module loaded?",
> > + AUXILIARY_SYSFS_PATH);
> > + return 0;
> > + }
> > +
> > + while ((e = readdir(dir)) != NULL) {
> > + if (e->d_name[0] == '.')
> > + continue;
> > +
> > + if (auxiliary_is_ignored_device(e->d_name))
> > + continue;
> > +
> > + snprintf(dirname, sizeof(dirname), "%s/%s",
> > + AUXILIARY_SYSFS_PATH, e->d_name);
> > +
> > + /* Ignore if no driver can handle. */
> > + FOREACH_DRIVER_ON_AUXILIARY_BUS(drv) {
> > + if (drv->match(e->d_name))
> > + break;
> > + }
> > + if (drv == NULL)
> > + continue;
> > +
> > + if (auxiliary_scan_one(dirname, e->d_name) < 0)
> > + goto error;
> > + }
> > + closedir(dir);
> > + return 0;
> > +
> > +error:
> > + closedir(dir);
> > + return -1;
> > +}
> > diff --git a/drivers/bus/auxiliary/meson.build b/drivers/bus/auxiliary/meson.build
> > new file mode 100644
> > index 0000000000..357550eff7
> > --- /dev/null
> > +++ b/drivers/bus/auxiliary/meson.build
> > @@ -0,0 +1,16 @@
> > +# SPDX-License-Identifier: BSD-3-Clause
> > +# Copyright (c) 2021 NVIDIA Corporation & Affiliates
> > +
> > +headers = files(
> > + 'rte_bus_auxiliary.h',
> > +)
> > +sources = files(
> > + 'auxiliary_common.c',
> > + 'auxiliary_params.c',
> > +)
> > +if is_linux
> > + sources += files(
> > + 'linux/auxiliary.c',
> > + )
> > +endif
> > +deps += ['kvargs']
> > diff --git a/drivers/bus/auxiliary/private.h b/drivers/bus/auxiliary/private.h
> > new file mode 100644
> > index 0000000000..cb3e849993
> > --- /dev/null
> > +++ b/drivers/bus/auxiliary/private.h
> > @@ -0,0 +1,74 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright (c) 2021 NVIDIA Corporation & Affiliates
> > + */
> > +
> > +#ifndef AUXILIARY_PRIVATE_H
>
> May be add BUS_ prefix at leaat?
>
> > +#define AUXILIARY_PRIVATE_H
> > +
> > +#include <stdbool.h>
> > +#include <stdio.h>
> > +
> > +#include "rte_bus_auxiliary.h"
> > +
> > +extern struct rte_auxiliary_bus auxiliary_bus;
> > +extern int auxiliary_bus_logtype;
> > +
> > +#define AUXILIARY_LOG(level, ...) \
> > + rte_log(RTE_LOG_ ## level, auxiliary_bus_logtype, \
> > + RTE_FMT("auxiliary bus: " RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
> > + RTE_FMT_TAIL(__VA_ARGS__,)))
> > +
> > +/* Auxiliary bus iterators */
> > +#define FOREACH_DEVICE_ON_AUXILIARY_BUS(p) \
> > + TAILQ_FOREACH(p, &(auxiliary_bus.device_list), next)
> > +
> > +#define FOREACH_DRIVER_ON_AUXILIARY_BUS(p) \
> > + TAILQ_FOREACH(p, &(auxiliary_bus.driver_list), next)
> > +
> > +bool auxiliary_dev_exists(const char *name);
> > +
> > +/*
> > + * Scan the content of the auxiliary bus, and the devices in the devices
> > + * list.
> > + */
> > +int auxiliary_scan(void);
> > +
> > +/*
> > + * Update a device being scanned.
> > + */
> > +void auxiliary_on_scan(struct rte_auxiliary_device *aux_dev);
> > +
> > +/*
> > + * Validate whether a device with given auxiliary device should be ignored
> > + * or not.
> > + */
> > +bool auxiliary_is_ignored_device(const char *name);
> > +
> > +/*
> > + * Add an auxiliary device to the auxiliary bus (append to auxiliary device
> > + * list). This function also updates the bus references of the auxiliary
> > + * device and the generic device object embedded within.
> > + */
> > +void auxiliary_add_device(struct rte_auxiliary_device *aux_dev);
> > +
> > +/*
> > + * Insert an auxiliary device in the auxiliary bus at a particular location
> > + * in the device list. It also updates the auxiliary bus reference of the
> > + * new devices to be inserted.
> > + */
> > +void auxiliary_insert_device(struct rte_auxiliary_device *exist_aux_dev,
> > + struct rte_auxiliary_device *new_aux_dev);
> > +
> > +/*
> > + * Match the auxiliary driver and device by driver function
>
> Missing full stop.
>
> > + */
> > +bool auxiliary_match(const struct rte_auxiliary_driver *aux_drv,
> > + const struct rte_auxiliary_device *aux_dev);
> > +
> > +/*
> > + * Iterate over devices, matching any device against the provided string
>
> Missing full stop.
>
> > + */
> > +void *auxiliary_dev_iterate(const void *start, const char *str,
> > + const struct rte_dev_iterator *it);
> > +
> > +#endif /* AUXILIARY_PRIVATE_H */
> > diff --git a/drivers/bus/auxiliary/rte_bus_auxiliary.h b/drivers/bus/auxiliary/rte_bus_auxiliary.h
> > new file mode 100644
> > index 0000000000..16b147e387
> > --- /dev/null
> > +++ b/drivers/bus/auxiliary/rte_bus_auxiliary.h
> > @@ -0,0 +1,201 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright (c) 2021 NVIDIA Corporation & Affiliates
> > + */
> > +
> > +#ifndef RTE_BUS_AUXILIARY_H
> > +#define RTE_BUS_AUXILIARY_H
> > +
> > +/**
> > + * @file
> > + *
> > + * Auxiliary Bus Interface.
> > + */
> > +
> > +#ifdef __cplusplus
> > +extern "C" {
> > +#endif
> > +
> > +#include <stdio.h>
> > +#include <stdlib.h>
> > +#include <limits.h>
> > +#include <errno.h>
> > +#include <sys/queue.h>
> > +#include <stdint.h>
> > +#include <inttypes.h>
> > +
> > +#include <rte_debug.h>
> > +#include <rte_interrupts.h>
> > +#include <rte_dev.h>
> > +#include <rte_bus.h>
> > +#include <rte_kvargs.h>
> > +
> > +#define RTE_BUS_AUXILIARY_NAME "auxiliary"
> > +
> > +/* Forward declarations */
> > +struct rte_auxiliary_driver;
> > +struct rte_auxiliary_bus;
> > +struct rte_auxiliary_device;
> > +
> > +/**
> > + * Match function for the driver to decide if device can be handled.
> > + *
> > + * @param name
> > + * Pointer to the auxiliary device name.
> > + * @return
> > + * Whether the driver can handle the auxiliary device.
> > + */
> > +typedef bool(rte_auxiliary_match_t)(const char *name);
> > +
> > +/**
> > + * Initialization function for the driver called during auxiliary probing.
> > + *
> > + * @param drv
> > + * Pointer to the auxiliary driver.
> > + * @param dev
> > + * Pointer to the auxiliary device.
> > + * @return
> > + * - 0 On success.
> > + * - Negative value and rte_errno is set otherwise.
> > + */
> > +typedef int(rte_auxiliary_probe_t)(struct rte_auxiliary_driver *drv,
> > + struct rte_auxiliary_device *dev);
> > +
> > +/**
> > + * Uninitialization function for the driver called during hotplugging.
> > + *
> > + * @param dev
> > + * Pointer to the auxiliary device.
> > + * @return
> > + * - 0 On success.
> > + * - Negative value and rte_errno is set otherwise.
> > + */
> > +typedef int (rte_auxiliary_remove_t)(struct rte_auxiliary_device *dev);
> > +
> > +/**
> > + * Driver-specific DMA mapping. After a successful call the device
> > + * will be able to read/write from/to this segment.
> > + *
> > + * @param dev
> > + * Pointer to the auxiliary device.
> > + * @param addr
> > + * Starting virtual address of memory to be mapped.
> > + * @param iova
> > + * Starting IOVA address of memory to be mapped.
> > + * @param len
> > + * Length of memory segment being mapped.
> > + * @return
> > + * - 0 On success.
> > + * - Negative value and rte_errno is set otherwise.
> > + */
> > +typedef int (rte_auxiliary_dma_map_t)(struct rte_auxiliary_device *dev,
> > + void *addr, uint64_t iova, size_t len);
> > +
> > +/**
> > + * Driver-specific DMA un-mapping. After a successful call the device
> > + * will not be able to read/write from/to this segment.
> > + *
> > + * @param dev
> > + * Pointer to the auxiliary device.
> > + * @param addr
> > + * Starting virtual address of memory to be unmapped.
> > + * @param iova
> > + * Starting IOVA address of memory to be unmapped.
> > + * @param len
> > + * Length of memory segment being unmapped.
> > + * @return
> > + * - 0 On success.
> > + * - Negative value and rte_errno is set otherwise.
> > + */
> > +typedef int (rte_auxiliary_dma_unmap_t)(struct rte_auxiliary_device *dev,
> > + void *addr, uint64_t iova, size_t len);
> > +
> > +/**
> > + * A structure describing an auxiliary device.
> > + */
> > +struct rte_auxiliary_device {
> > + TAILQ_ENTRY(rte_auxiliary_device) next; /**< Next probed device. */
> > + struct rte_device device; /**< Inherit core device */
> > + char name[RTE_DEV_NAME_MAX_LEN + 1]; /**< ASCII device name */
> > + struct rte_intr_handle intr_handle; /**< Interrupt handle */
> > + struct rte_auxiliary_driver *driver; /**< Device driver */
> > +};
> > +
> > +/** List of auxiliary devices */
> > +TAILQ_HEAD(rte_auxiliary_device_list, rte_auxiliary_device);
> > +/** List of auxiliary drivers */
> > +TAILQ_HEAD(rte_auxiliary_driver_list, rte_auxiliary_driver);
>
> Shouldn't we hide rte_auxiliary_device inside the library take
> API/ABI stability into account? Or will be it DPDK internal anyway? If
> so, it should be done INTERNAL from the very
> beginning.
>
> > +
> > +/**
> > + * Structure describing the auxiliary bus
> > + */
> > +struct rte_auxiliary_bus {
> > + struct rte_bus bus; /**< Inherit the generic class */
> > + struct rte_auxiliary_device_list device_list; /**< List of devices */
> > + struct rte_auxiliary_driver_list driver_list; /**< List of drivers */
> > +};
>
> It looks internal. The following forward declaration should be
> sufficient to build.
>
> struct rte_auxiliary_bus;
>
>
> > +
> > +/**
> > + * A structure describing an auxiliary driver.
> > + */
> > +struct rte_auxiliary_driver {
> > + TAILQ_ENTRY(rte_auxiliary_driver) next; /**< Next in list. */
> > + struct rte_driver driver; /**< Inherit core driver. */
> > + struct rte_auxiliary_bus *bus; /**< Auxiliary bus reference. */
> > + rte_auxiliary_match_t *match; /**< Device match function. */
> > + rte_auxiliary_probe_t *probe; /**< Device probe function. */
> > + rte_auxiliary_remove_t *remove; /**< Device remove function. */
> > + rte_auxiliary_dma_map_t *dma_map; /**< Device DMA map function. */
> > + rte_auxiliary_dma_unmap_t *dma_unmap; /**< Device DMA unmap function. */
> > + uint32_t drv_flags; /**< Flags RTE_AUXILIARY_DRV_*. */
> > +};
> > +
> > +/**
> > + * @internal
> > + * Helper macro for drivers that need to convert to struct rte_auxiliary_device.
> > + */
> > +#define RTE_DEV_TO_AUXILIARY(ptr) \
> > + container_of(ptr, struct rte_auxiliary_device, device)
> > +
> > +#define RTE_DEV_TO_AUXILIARY_CONST(ptr) \
> > + container_of(ptr, const struct rte_auxiliary_device, device)
> > +
> > +#define RTE_ETH_DEV_TO_AUXILIARY(eth_dev) \
> > + RTE_DEV_TO_AUXILIARY((eth_dev)->device)
> > +
> > +/** Device driver needs IOVA as VA and cannot work with IOVA as PA */
> > +#define RTE_AUXILIARY_DRV_NEED_IOVA_AS_VA 0x002
> > +
> > +/**
>
> Don't we need EXPERIMENTAL notice here?
>
> > + * Register an auxiliary driver.
> > + *
> > + * @param driver
> > + * A pointer to a rte_auxiliary_driver structure describing the driver
> > + * to be registered.
> > + */
> > +__rte_experimental
> > +void rte_auxiliary_register(struct rte_auxiliary_driver *driver);
> > +
> > +/** Helper for auxiliary device registration from driver instance */
> > +#define RTE_PMD_REGISTER_AUXILIARY(nm, auxiliary_drv) \
> > + RTE_INIT(auxiliaryinitfn_ ##nm) \
> > + { \
> > + (auxiliary_drv).driver.name = RTE_STR(nm); \
> > + rte_auxiliary_register(&(auxiliary_drv)); \
> > + } \
> > + RTE_PMD_EXPORT_NAME(nm, __COUNTER__)
> > +
> > +/**
>
> Don't we need EXPERIMENTAL notice here?
>
> > + * Unregister an auxiliary driver.
> > + *
> > + * @param driver
> > + * A pointer to a rte_auxiliary_driver structure describing the driver
> > + * to be unregistered.
> > + */
> > +__rte_experimental
> > +void rte_auxiliary_unregister(struct rte_auxiliary_driver *driver);
> > +
> > +#ifdef __cplusplus
> > +}
> > +#endif
> > +
> > +#endif /* RTE_BUS_AUXILIARY_H */
> > diff --git a/drivers/bus/auxiliary/version.map b/drivers/bus/auxiliary/version.map
> > new file mode 100644
> > index 0000000000..a52260657c
> > --- /dev/null
> > +++ b/drivers/bus/auxiliary/version.map
> > @@ -0,0 +1,7 @@
> > +EXPERIMENTAL {
> > + global:
> > +
> > + # added in 21.08
> > + rte_auxiliary_register;
> > + rte_auxiliary_unregister;
> > +};
> > diff --git a/drivers/bus/meson.build b/drivers/bus/meson.build
> > index 410058de3a..45eab5233d 100644
> > --- a/drivers/bus/meson.build
> > +++ b/drivers/bus/meson.build
> > @@ -2,6 +2,7 @@
> > # Copyright(c) 2017 Intel Corporation
> >
> > drivers = [
> > + 'auxiliary',
> > 'dpaa',
> > 'fslmc',
> > 'ifpga',
> >
On 25/06/2021 12:47, Xueming Li wrote:
> Auxiliary bus [1] provides a way to split function into child-devices
> representing sub-domains of functionality. Each auxiliary device
> represents a part of its parent functionality.
>
> Auxiliary device is identified by unique device name, sysfs path:
> /sys/bus/auxiliary/devices/<name>
>
> Devargs legacy syntax ofauxiliary device:
> -a auxiliary:<name>[,args...]
> Devargs generic syntax of auxiliary device:
> -a bus=auxiliary,name=<name>,,/class=<classs>,,/driver=<driver>,,
>
> [1] kernel auxiliary bus document:
> https://www.kernel.org/doc/html/latest/driver-api/auxiliary_bus.html
>
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Cc: Wang Haiyue <haiyue.wang@intel.com>
> Cc: Thomas Monjalon <thomas@monjalon.net>
> Cc: Kinsella Ray <mdr@ashroe.eu>
> ---
> MAINTAINERS | 5 +
> doc/guides/rel_notes/release_21_08.rst | 6 +
> drivers/bus/auxiliary/auxiliary_common.c | 411 ++++++++++++++++++++++
> drivers/bus/auxiliary/auxiliary_params.c | 59 ++++
> drivers/bus/auxiliary/linux/auxiliary.c | 141 ++++++++
> drivers/bus/auxiliary/meson.build | 16 +
> drivers/bus/auxiliary/private.h | 74 ++++
> drivers/bus/auxiliary/rte_bus_auxiliary.h | 201 +++++++++++
> drivers/bus/auxiliary/version.map | 7 +
> drivers/bus/meson.build | 1 +
> 10 files changed, 921 insertions(+)
> create mode 100644 drivers/bus/auxiliary/auxiliary_common.c
> create mode 100644 drivers/bus/auxiliary/auxiliary_params.c
> create mode 100644 drivers/bus/auxiliary/linux/auxiliary.c
> create mode 100644 drivers/bus/auxiliary/meson.build
> create mode 100644 drivers/bus/auxiliary/private.h
> create mode 100644 drivers/bus/auxiliary/rte_bus_auxiliary.h
> create mode 100644 drivers/bus/auxiliary/version.map
>
[SNIP]
> diff --git a/drivers/bus/auxiliary/version.map b/drivers/bus/auxiliary/version.map
> new file mode 100644
> index 0000000000..a52260657c
> --- /dev/null
> +++ b/drivers/bus/auxiliary/version.map
> @@ -0,0 +1,7 @@
> +EXPERIMENTAL {
> + global:
> +
> + # added in 21.08
> + rte_auxiliary_register;
> + rte_auxiliary_unregister;
> +};
Aren't these really internal?
> diff --git a/drivers/bus/meson.build b/drivers/bus/meson.build
> index 410058de3a..45eab5233d 100644
> --- a/drivers/bus/meson.build
> +++ b/drivers/bus/meson.build
> @@ -2,6 +2,7 @@
> # Copyright(c) 2017 Intel Corporation
>
> drivers = [
> + 'auxiliary',
> 'dpaa',
> 'fslmc',
> 'ifpga',
>
> -----Original Message-----
> From: Kinsella, Ray <mdr@ashroe.eu>
> Sent: Wednesday, August 4, 2021 5:50 PM
> To: Xueming(Steven) Li <xuemingl@nvidia.com>
> Cc: dev@dpdk.org; Wang Haiyue <haiyue.wang@intel.com>; NBU-Contact-Thomas Monjalon <thomas@monjalon.net>; Neil Horman
> <nhorman@tuxdriver.com>
> Subject: Re: [PATCH v6 2/2] bus/auxiliary: introduce auxiliary bus
>
>
>
> On 25/06/2021 12:47, Xueming Li wrote:
> > Auxiliary bus [1] provides a way to split function into child-devices
> > representing sub-domains of functionality. Each auxiliary device
> > represents a part of its parent functionality.
> >
> > Auxiliary device is identified by unique device name, sysfs path:
> > /sys/bus/auxiliary/devices/<name>
> >
> > Devargs legacy syntax ofauxiliary device:
> > -a auxiliary:<name>[,args...]
> > Devargs generic syntax of auxiliary device:
> > -a bus=auxiliary,name=<name>,,/class=<classs>,,/driver=<driver>,,
> >
> > [1] kernel auxiliary bus document:
> > https://www.kernel.org/doc/html/latest/driver-api/auxiliary_bus.html
> >
> > Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> > Cc: Wang Haiyue <haiyue.wang@intel.com>
> > Cc: Thomas Monjalon <thomas@monjalon.net>
> > Cc: Kinsella Ray <mdr@ashroe.eu>
> > ---
> > MAINTAINERS | 5 +
> > doc/guides/rel_notes/release_21_08.rst | 6 +
> > drivers/bus/auxiliary/auxiliary_common.c | 411
> > ++++++++++++++++++++++ drivers/bus/auxiliary/auxiliary_params.c | 59 ++++
> > drivers/bus/auxiliary/linux/auxiliary.c | 141 ++++++++
> > drivers/bus/auxiliary/meson.build | 16 +
> > drivers/bus/auxiliary/private.h | 74 ++++
> > drivers/bus/auxiliary/rte_bus_auxiliary.h | 201 +++++++++++
> > drivers/bus/auxiliary/version.map | 7 +
> > drivers/bus/meson.build | 1 +
> > 10 files changed, 921 insertions(+)
> > create mode 100644 drivers/bus/auxiliary/auxiliary_common.c
> > create mode 100644 drivers/bus/auxiliary/auxiliary_params.c
> > create mode 100644 drivers/bus/auxiliary/linux/auxiliary.c
> > create mode 100644 drivers/bus/auxiliary/meson.build create mode
> > 100644 drivers/bus/auxiliary/private.h create mode 100644
> > drivers/bus/auxiliary/rte_bus_auxiliary.h
> > create mode 100644 drivers/bus/auxiliary/version.map
> >
>
> [SNIP]
>
> > diff --git a/drivers/bus/auxiliary/version.map
> > b/drivers/bus/auxiliary/version.map
> > new file mode 100644
> > index 0000000000..a52260657c
> > --- /dev/null
> > +++ b/drivers/bus/auxiliary/version.map
> > @@ -0,0 +1,7 @@
> > +EXPERIMENTAL {
> > + global:
> > +
> > + # added in 21.08
> > + rte_auxiliary_register;
> > + rte_auxiliary_unregister;
> > +};
>
> Aren't these really internal?
Hi Kinsella,
Application might implement driver, so keep it public, same as rte_pci_register.
>
> > diff --git a/drivers/bus/meson.build b/drivers/bus/meson.build index
> > 410058de3a..45eab5233d 100644
> > --- a/drivers/bus/meson.build
> > +++ b/drivers/bus/meson.build
> > @@ -2,6 +2,7 @@
> > # Copyright(c) 2017 Intel Corporation
> >
> > drivers = [
> > + 'auxiliary',
> > 'dpaa',
> > 'fslmc',
> > 'ifpga',
> >
On 25/06/2021 12:47, Xueming Li wrote:
> Auxiliary bus [1] provides a way to split function into child-devices
> representing sub-domains of functionality. Each auxiliary device
> represents a part of its parent functionality.
>
> Auxiliary device is identified by unique device name, sysfs path:
> /sys/bus/auxiliary/devices/<name>
>
> Devargs legacy syntax ofauxiliary device:
> -a auxiliary:<name>[,args...]
> Devargs generic syntax of auxiliary device:
> -a bus=auxiliary,name=<name>,,/class=<classs>,,/driver=<driver>,,
>
> [1] kernel auxiliary bus document:
> https://www.kernel.org/doc/html/latest/driver-api/auxiliary_bus.html
>
> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> Cc: Wang Haiyue <haiyue.wang@intel.com>
> Cc: Thomas Monjalon <thomas@monjalon.net>
> Cc: Kinsella Ray <mdr@ashroe.eu>
> ---
> MAINTAINERS | 5 +
> doc/guides/rel_notes/release_21_08.rst | 6 +
> drivers/bus/auxiliary/auxiliary_common.c | 411 ++++++++++++++++++++++
> drivers/bus/auxiliary/auxiliary_params.c | 59 ++++
> drivers/bus/auxiliary/linux/auxiliary.c | 141 ++++++++
> drivers/bus/auxiliary/meson.build | 16 +
> drivers/bus/auxiliary/private.h | 74 ++++
> drivers/bus/auxiliary/rte_bus_auxiliary.h | 201 +++++++++++
> drivers/bus/auxiliary/version.map | 7 +
> drivers/bus/meson.build | 1 +
> 10 files changed, 921 insertions(+)
> create mode 100644 drivers/bus/auxiliary/auxiliary_common.c
> create mode 100644 drivers/bus/auxiliary/auxiliary_params.c
> create mode 100644 drivers/bus/auxiliary/linux/auxiliary.c
> create mode 100644 drivers/bus/auxiliary/meson.build
> create mode 100644 drivers/bus/auxiliary/private.h
> create mode 100644 drivers/bus/auxiliary/rte_bus_auxiliary.h
> create mode 100644 drivers/bus/auxiliary/version.map
>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
On 04/08/2021 13:11, Xueming(Steven) Li wrote:
>
>
>> -----Original Message-----
>> From: Kinsella, Ray <mdr@ashroe.eu>
>> Sent: Wednesday, August 4, 2021 7:46 PM
>> To: Xueming(Steven) Li <xuemingl@nvidia.com>
>> Subject: Re: [PATCH v6 2/2] bus/auxiliary: introduce auxiliary bus
>>
>>
>>
>> On 04/08/2021 12:21, Xueming(Steven) Li wrote:
>>>
>>>
>>>> -----Original Message-----
>>>> From: Kinsella, Ray <mdr@ashroe.eu>
>>>> Sent: Wednesday, August 4, 2021 6:00 PM
>>>> To: Xueming(Steven) Li <xuemingl@nvidia.com>
>>>> Cc: dev@dpdk.org; Wang Haiyue <haiyue.wang@intel.com>;
>>>> NBU-Contact-Thomas Monjalon <thomas@monjalon.net>; Neil Horman
>>>> <nhorman@tuxdriver.com>
>>>> Subject: Re: [PATCH v6 2/2] bus/auxiliary: introduce auxiliary bus
>>>>
>>>>
>>>>
>>>> On 25/06/2021 12:47, Xueming Li wrote:
>>>>> Auxiliary bus [1] provides a way to split function into
>>>>> child-devices representing sub-domains of functionality. Each
>>>>> auxiliary device represents a part of its parent functionality.
>>>>>
>>>>> Auxiliary device is identified by unique device name, sysfs path:
>>>>> /sys/bus/auxiliary/devices/<name>
>>>>>
>>>>> Devargs legacy syntax ofauxiliary device:
>>>>> -a auxiliary:<name>[,args...]
>>>>> Devargs generic syntax of auxiliary device:
>>>>> -a bus=auxiliary,name=<name>,,/class=<classs>,,/driver=<driver>,,
>>>>>
>>>>> [1] kernel auxiliary bus document:
>>>>> https://www.kernel.org/doc/html/latest/driver-api/auxiliary_bus.html
>>>>>
>>>>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
>>>>> Cc: Wang Haiyue <haiyue.wang@intel.com>
>>>>> Cc: Thomas Monjalon <thomas@monjalon.net>
>>>>> Cc: Kinsella Ray <mdr@ashroe.eu>
>>>>> ---
>>>>> MAINTAINERS | 5 +
>>>>> doc/guides/rel_notes/release_21_08.rst | 6 +
>>>>> drivers/bus/auxiliary/auxiliary_common.c | 411
>>>>> ++++++++++++++++++++++ drivers/bus/auxiliary/auxiliary_params.c |
>>>>> ++++++++++++++++++++++ 59 ++++
>>>>> drivers/bus/auxiliary/linux/auxiliary.c | 141 ++++++++
>>>>> drivers/bus/auxiliary/meson.build | 16 +
>>>>> drivers/bus/auxiliary/private.h | 74 ++++
>>>>> drivers/bus/auxiliary/rte_bus_auxiliary.h | 201 +++++++++++
>>>>> drivers/bus/auxiliary/version.map | 7 +
>>>>> drivers/bus/meson.build | 1 +
>>>>> 10 files changed, 921 insertions(+) create mode 100644
>>>>> drivers/bus/auxiliary/auxiliary_common.c
>>>>> create mode 100644 drivers/bus/auxiliary/auxiliary_params.c
>>>>> create mode 100644 drivers/bus/auxiliary/linux/auxiliary.c
>>>>> create mode 100644 drivers/bus/auxiliary/meson.build create mode
>>>>> 100644 drivers/bus/auxiliary/private.h create mode 100644
>>>>> drivers/bus/auxiliary/rte_bus_auxiliary.h
>>>>> create mode 100644 drivers/bus/auxiliary/version.map
>>>>>
>>>>
>>>> Acked-by: Ray Kinsella <mdr@ashroe.eu>
>>>
>>> Thanks, but this patch already integrated :)
>>
>> It appears in the order in which I am going through my email is incorrect. :-)
>>
>>>
>>> Would you like to have a look at another deprecation notice? Andrew reviewed RFC:
>>> https://mails.dpdk.org/archives/dev/2021-August/216007.html
>>>
>>
>> Its not strictly a depreciation notice though, you are not breaking anything right.
>> Since you are not breaking anything, don't think the notice is required in the 21.11 timeframe.
>>
>> Now if you where doing it in 21.08, it would be an ABI change and that would be a different story.
>
> Thanks for looking at this!
> Yes, it targets to 21.11. The offloading flag is fine, but the shared_group does break ABI, detail:
> https://mails.dpdk.org/archives/dev/2021-July/215575.html
Right ... its a new field, not a depreciation as such.
What I mean by this is that no existing code is broken.
21.11 is a new ABI in any case and you are not depreciating anything, so no notice is required.
> -----Original Message-----
> From: Kinsella, Ray <mdr@ashroe.eu>
> Sent: Wednesday, August 4, 2021 8:14 PM
> To: Xueming(Steven) Li <xuemingl@nvidia.com>; dpdk-dev <dev@dpdk.org>
> Subject: Re: [PATCH v6 2/2] bus/auxiliary: introduce auxiliary bus
>
>
>
> On 04/08/2021 13:11, Xueming(Steven) Li wrote:
> >
> >
> >> -----Original Message-----
> >> From: Kinsella, Ray <mdr@ashroe.eu>
> >> Sent: Wednesday, August 4, 2021 7:46 PM
> >> To: Xueming(Steven) Li <xuemingl@nvidia.com>
> >> Subject: Re: [PATCH v6 2/2] bus/auxiliary: introduce auxiliary bus
> >>
> >>
> >>
> >> On 04/08/2021 12:21, Xueming(Steven) Li wrote:
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Kinsella, Ray <mdr@ashroe.eu>
> >>>> Sent: Wednesday, August 4, 2021 6:00 PM
> >>>> To: Xueming(Steven) Li <xuemingl@nvidia.com>
> >>>> Cc: dev@dpdk.org; Wang Haiyue <haiyue.wang@intel.com>;
> >>>> NBU-Contact-Thomas Monjalon <thomas@monjalon.net>; Neil Horman
> >>>> <nhorman@tuxdriver.com>
> >>>> Subject: Re: [PATCH v6 2/2] bus/auxiliary: introduce auxiliary bus
> >>>>
> >>>>
> >>>>
> >>>> On 25/06/2021 12:47, Xueming Li wrote:
> >>>>> Auxiliary bus [1] provides a way to split function into
> >>>>> child-devices representing sub-domains of functionality. Each
> >>>>> auxiliary device represents a part of its parent functionality.
> >>>>>
> >>>>> Auxiliary device is identified by unique device name, sysfs path:
> >>>>> /sys/bus/auxiliary/devices/<name>
> >>>>>
> >>>>> Devargs legacy syntax ofauxiliary device:
> >>>>> -a auxiliary:<name>[,args...]
> >>>>> Devargs generic syntax of auxiliary device:
> >>>>> -a
> >>>>> bus=auxiliary,name=<name>,,/class=<classs>,,/driver=<driver>,,
> >>>>>
> >>>>> [1] kernel auxiliary bus document:
> >>>>> https://www.kernel.org/doc/html/latest/driver-api/auxiliary_bus.ht
> >>>>> ml
> >>>>>
> >>>>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> >>>>> Cc: Wang Haiyue <haiyue.wang@intel.com>
> >>>>> Cc: Thomas Monjalon <thomas@monjalon.net>
> >>>>> Cc: Kinsella Ray <mdr@ashroe.eu>
> >>>>> ---
> >>>>> MAINTAINERS | 5 +
> >>>>> doc/guides/rel_notes/release_21_08.rst | 6 +
> >>>>> drivers/bus/auxiliary/auxiliary_common.c | 411
> >>>>> ++++++++++++++++++++++ drivers/bus/auxiliary/auxiliary_params.c
> >>>>> ++++++++++++++++++++++ |
> >>>>> ++++++++++++++++++++++ 59 ++++
> >>>>> drivers/bus/auxiliary/linux/auxiliary.c | 141 ++++++++
> >>>>> drivers/bus/auxiliary/meson.build | 16 +
> >>>>> drivers/bus/auxiliary/private.h | 74 ++++
> >>>>> drivers/bus/auxiliary/rte_bus_auxiliary.h | 201 +++++++++++
> >>>>> drivers/bus/auxiliary/version.map | 7 +
> >>>>> drivers/bus/meson.build | 1 +
> >>>>> 10 files changed, 921 insertions(+) create mode 100644
> >>>>> drivers/bus/auxiliary/auxiliary_common.c
> >>>>> create mode 100644 drivers/bus/auxiliary/auxiliary_params.c
> >>>>> create mode 100644 drivers/bus/auxiliary/linux/auxiliary.c
> >>>>> create mode 100644 drivers/bus/auxiliary/meson.build create mode
> >>>>> 100644 drivers/bus/auxiliary/private.h create mode 100644
> >>>>> drivers/bus/auxiliary/rte_bus_auxiliary.h
> >>>>> create mode 100644 drivers/bus/auxiliary/version.map
> >>>>>
> >>>>
> >>>> Acked-by: Ray Kinsella <mdr@ashroe.eu>
> >>>
> >>> Thanks, but this patch already integrated :)
> >>
> >> It appears in the order in which I am going through my email is
> >> incorrect. :-)
> >>
> >>>
> >>> Would you like to have a look at another deprecation notice? Andrew reviewed RFC:
> >>> https://mails.dpdk.org/archives/dev/2021-August/216007.html
> >>>
> >>
> >> Its not strictly a depreciation notice though, you are not breaking anything right.
> >> Since you are not breaking anything, don't think the notice is required in the 21.11 timeframe.
> >>
> >> Now if you where doing it in 21.08, it would be an ABI change and that would be a different story.
> >
> > Thanks for looking at this!
> > Yes, it targets to 21.11. The offloading flag is fine, but the shared_group does break ABI, detail:
> > https://mails.dpdk.org/archives/dev/2021-July/215575.html
>
> Right ... its a new field, not a depreciation as such.
> What I mean by this is that no existing code is broken.
>
> 21.11 is a new ABI in any case and you are not depreciating anything, so no notice is required.
Maybe it a new process, confirmed with Thomas, it's expected:
https://doc.dpdk.org/guides/contributing/abi_policy.html#abi-changes
04/08/2021 15:00, Xueming(Steven) Li:
> From: Kinsella, Ray <mdr@ashroe.eu>
> > On 04/08/2021 13:11, Xueming(Steven) Li wrote:
> > > From: Kinsella, Ray <mdr@ashroe.eu>
> > >> Its not strictly a depreciation notice though, you are not breaking anything right.
> > >> Since you are not breaking anything, don't think the notice is required in the 21.11 timeframe.
> > >>
> > >> Now if you where doing it in 21.08, it would be an ABI change and that would be a different story.
> > >
> > > Thanks for looking at this!
> > > Yes, it targets to 21.11. The offloading flag is fine, but the shared_group does break ABI, detail:
> > > https://mails.dpdk.org/archives/dev/2021-July/215575.html
> >
> > Right ... its a new field, not a depreciation as such.
> > What I mean by this is that no existing code is broken.
> >
> > 21.11 is a new ABI in any case and you are not depreciating anything, so no notice is required.
>
> Maybe it a new process, confirmed with Thomas, it's expected:
> https://doc.dpdk.org/guides/contributing/abi_policy.html#abi-changes
I think what Ray means is that it breaks ABI but not API,
so he doesn't consider a notice is required.
My understanding of the policy is that *any* ABI change requires a notice.
But if you want to make it lighter and allow any non-announced ABI change
in an ABI-breaking release, I think I would vote for.
Cc techboard@dpdk.org
On 04/08/2021 14:12, Thomas Monjalon wrote:
> 04/08/2021 15:00, Xueming(Steven) Li:
>> From: Kinsella, Ray <mdr@ashroe.eu>
>>> On 04/08/2021 13:11, Xueming(Steven) Li wrote:
>>>> From: Kinsella, Ray <mdr@ashroe.eu>
>>>>> Its not strictly a depreciation notice though, you are not breaking anything right.
>>>>> Since you are not breaking anything, don't think the notice is required in the 21.11 timeframe.
>>>>>
>>>>> Now if you where doing it in 21.08, it would be an ABI change and that would be a different story.
>>>>
>>>> Thanks for looking at this!
>>>> Yes, it targets to 21.11. The offloading flag is fine, but the shared_group does break ABI, detail:
>>>> https://mails.dpdk.org/archives/dev/2021-July/215575.html
>>>
>>> Right ... its a new field, not a depreciation as such.
>>> What I mean by this is that no existing code is broken.
>>>
>>> 21.11 is a new ABI in any case and you are not depreciating anything, so no notice is required.
>>
>> Maybe it a new process, confirmed with Thomas, it's expected:
>> https://doc.dpdk.org/guides/contributing/abi_policy.html#abi-changes
>
> I think what Ray means is that it breaks ABI but not API,
> so he doesn't consider a notice is required.
> My understanding of the policy is that *any* ABI change requires a notice.
> But if you want to make it lighter and allow any non-announced ABI change
> in an ABI-breaking release, I think I would vote for.
Thanks for clarifying Thomas ... you are correct.
>
> Cc techboard@dpdk.org
>
04/08/2021 15:53, Kinsella, Ray:
> On 04/08/2021 14:12, Thomas Monjalon wrote:
> > 04/08/2021 15:00, Xueming(Steven) Li:
> >> From: Kinsella, Ray <mdr@ashroe.eu>
> >>> On 04/08/2021 13:11, Xueming(Steven) Li wrote:
> >>>> From: Kinsella, Ray <mdr@ashroe.eu>
> >>>>> Its not strictly a depreciation notice though, you are not breaking anything right.
> >>>>> Since you are not breaking anything, don't think the notice is required in the 21.11 timeframe.
> >>>>>
> >>>>> Now if you where doing it in 21.08, it would be an ABI change and that would be a different story.
> >>>>
> >>>> Thanks for looking at this!
> >>>> Yes, it targets to 21.11. The offloading flag is fine, but the shared_group does break ABI, detail:
> >>>> https://mails.dpdk.org/archives/dev/2021-July/215575.html
> >>>
> >>> Right ... its a new field, not a depreciation as such.
> >>> What I mean by this is that no existing code is broken.
> >>>
> >>> 21.11 is a new ABI in any case and you are not depreciating anything, so no notice is required.
> >>
> >> Maybe it a new process, confirmed with Thomas, it's expected:
> >> https://doc.dpdk.org/guides/contributing/abi_policy.html#abi-changes
> >
> > I think what Ray means is that it breaks ABI but not API,
> > so he doesn't consider a notice is required.
>
> > My understanding of the policy is that *any* ABI change requires a notice.
> > But if you want to make it lighter and allow any non-announced ABI change
> > in an ABI-breaking release, I think I would vote for.
>
> Thanks for clarifying Thomas ... you are correct.
In the meantime, let's review and ack notices, even if ABI-only change:
https://patches.dpdk.org/bundle/tmonjalo/deprecation-notices/
We'll discuss later if we can accept more ABI change,
but we should try to be on the safe side for those already announced.
@@ -525,6 +525,11 @@ F: doc/guides/mempool/octeontx2.rst
Bus Drivers
-----------
+Auxiliary bus driver
+M: Parav Pandit <parav@nvidia.com>
+M: Xueming Li <xuemingl@nvidia.com>
+F: drivers/bus/auxiliary/
+
Intel FPGA bus
M: Rosen Xu <rosen.xu@intel.com>
F: drivers/bus/ifpga/
@@ -55,6 +55,12 @@ New Features
Also, make sure to start the actual text at the margin.
=======================================================
+* **Added auxiliary bus support.**
+
+ Auxiliary bus provides a way to split function into child-devices
+ representing sub-domains of functionality. Each auxiliary device
+ represents a part of its parent functionality.
+
Removed Items
-------------
new file mode 100644
@@ -0,0 +1,411 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2021 NVIDIA Corporation & Affiliates
+ */
+
+#include <string.h>
+#include <inttypes.h>
+#include <stdint.h>
+#include <stdbool.h>
+#include <stdlib.h>
+#include <stdio.h>
+#include <sys/queue.h>
+#include <rte_errno.h>
+#include <rte_interrupts.h>
+#include <rte_log.h>
+#include <rte_bus.h>
+#include <rte_per_lcore.h>
+#include <rte_memory.h>
+#include <rte_eal.h>
+#include <rte_eal_paging.h>
+#include <rte_string_fns.h>
+#include <rte_common.h>
+#include <rte_devargs.h>
+
+#include "private.h"
+#include "rte_bus_auxiliary.h"
+
+static struct rte_devargs *
+auxiliary_devargs_lookup(const char *name)
+{
+ struct rte_devargs *devargs;
+
+ RTE_EAL_DEVARGS_FOREACH(RTE_BUS_AUXILIARY_NAME, devargs) {
+ if (strcmp(devargs->name, name) == 0)
+ return devargs;
+ }
+ return NULL;
+}
+
+/*
+ * Test whether the auxiliary device exist
+ *
+ * Stub for OS not supporting auxiliary bus.
+ */
+__rte_weak bool
+auxiliary_dev_exists(const char *name)
+{
+ RTE_SET_USED(name);
+ return false;
+}
+
+/*
+ * Scan the devices in the auxiliary bus.
+ *
+ * Stub for OS not supporting auxiliary bus.
+ */
+__rte_weak int
+auxiliary_scan(void)
+{
+ return 0;
+}
+
+/*
+ * Update a device's devargs being scanned.
+ *
+ * @param aux_dev
+ * AUXILIARY device.
+ */
+void
+auxiliary_on_scan(struct rte_auxiliary_device *aux_dev)
+{
+ aux_dev->device.devargs = auxiliary_devargs_lookup(aux_dev->name);
+}
+
+/*
+ * Match the auxiliary driver and device using driver function.
+ */
+bool
+auxiliary_match(const struct rte_auxiliary_driver *aux_drv,
+ const struct rte_auxiliary_device *aux_dev)
+{
+ if (aux_drv->match == NULL)
+ return false;
+ return aux_drv->match(aux_dev->name);
+}
+
+/*
+ * Call the probe() function of the driver.
+ */
+static int
+rte_auxiliary_probe_one_driver(struct rte_auxiliary_driver *drv,
+ struct rte_auxiliary_device *dev)
+{
+ enum rte_iova_mode iova_mode;
+ int ret;
+
+ if ((drv == NULL) || (dev == NULL))
+ return -EINVAL;
+
+ /* Check if driver supports it. */
+ if (!auxiliary_match(drv, dev))
+ /* Match of device and driver failed */
+ return 1;
+
+ /* No initialization when marked as blocked, return without error. */
+ if (dev->device.devargs != NULL &&
+ dev->device.devargs->policy == RTE_DEV_BLOCKED) {
+ AUXILIARY_LOG(INFO, "Device is blocked, not initializing");
+ return -1;
+ }
+
+ if (dev->device.numa_node < 0) {
+ AUXILIARY_LOG(INFO, "Device is not NUMA-aware, defaulting socket to 0");
+ dev->device.numa_node = 0;
+ }
+
+ iova_mode = rte_eal_iova_mode();
+ if ((drv->drv_flags & RTE_AUXILIARY_DRV_NEED_IOVA_AS_VA) > 0 &&
+ iova_mode != RTE_IOVA_VA) {
+ AUXILIARY_LOG(ERR, "Driver %s expecting VA IOVA mode but current mode is PA, not initializing",
+ drv->driver.name);
+ return -EINVAL;
+ }
+
+ dev->driver = drv;
+
+ AUXILIARY_LOG(INFO, "Probe auxiliary driver: %s device: %s (socket %i)",
+ drv->driver.name, dev->name, dev->device.numa_node);
+ ret = drv->probe(drv, dev);
+ if (ret != 0)
+ dev->driver = NULL;
+ else
+ dev->device.driver = &drv->driver;
+
+ return ret;
+}
+
+/*
+ * Call the remove() function of the driver.
+ */
+static int
+rte_auxiliary_driver_remove_dev(struct rte_auxiliary_device *dev)
+{
+ struct rte_auxiliary_driver *drv;
+ int ret = 0;
+
+ if (dev == NULL)
+ return -EINVAL;
+
+ drv = dev->driver;
+
+ AUXILIARY_LOG(DEBUG, "Driver %s remove auxiliary device %s on NUMA socket %i",
+ drv->driver.name, dev->name, dev->device.numa_node);
+
+ if (drv->remove != NULL) {
+ ret = drv->remove(dev);
+ if (ret < 0)
+ return ret;
+ }
+
+ /* clear driver structure */
+ dev->driver = NULL;
+ dev->device.driver = NULL;
+
+ return 0;
+}
+
+/*
+ * Call the probe() function of all registered driver for the given device.
+ * Return < 0 if initialization failed.
+ * Return 1 if no driver is found for this device.
+ */
+static int
+auxiliary_probe_all_drivers(struct rte_auxiliary_device *dev)
+{
+ struct rte_auxiliary_driver *drv;
+ int rc;
+
+ if (dev == NULL)
+ return -EINVAL;
+
+ FOREACH_DRIVER_ON_AUXILIARY_BUS(drv) {
+ if (!drv->match(dev->name))
+ continue;
+
+ rc = rte_auxiliary_probe_one_driver(drv, dev);
+ if (rc < 0)
+ /* negative value is an error */
+ return rc;
+ if (rc > 0)
+ /* positive value means driver doesn't support it */
+ continue;
+ return 0;
+ }
+ return 1;
+}
+
+/*
+ * Scan the content of the auxiliary bus, and call the probe function for
+ * all registered drivers to try to probe discovered devices.
+ */
+static int
+auxiliary_probe(void)
+{
+ struct rte_auxiliary_device *dev = NULL;
+ size_t probed = 0, failed = 0;
+ int ret = 0;
+
+ FOREACH_DEVICE_ON_AUXILIARY_BUS(dev) {
+ probed++;
+
+ ret = auxiliary_probe_all_drivers(dev);
+ if (ret < 0) {
+ if (ret != -EEXIST) {
+ AUXILIARY_LOG(ERR, "Requested device %s cannot be used",
+ dev->name);
+ rte_errno = errno;
+ failed++;
+ }
+ ret = 0;
+ }
+ }
+
+ return (probed && probed == failed) ? -1 : 0;
+}
+
+static int
+auxiliary_parse(const char *name, void *addr)
+{
+ struct rte_auxiliary_driver *drv = NULL;
+ const char **out = addr;
+
+ /* Allow empty device name "auxiliary:" to bypass entire bus scan. */
+ if (strlen(name) == 0)
+ return 0;
+
+ FOREACH_DRIVER_ON_AUXILIARY_BUS(drv) {
+ if (drv->match(name))
+ break;
+ }
+ if (drv != NULL && addr != NULL)
+ *out = name;
+ return drv != NULL ? 0 : -1;
+}
+
+/* Register a driver */
+void
+rte_auxiliary_register(struct rte_auxiliary_driver *driver)
+{
+ TAILQ_INSERT_TAIL(&auxiliary_bus.driver_list, driver, next);
+ driver->bus = &auxiliary_bus;
+}
+
+/* Unregister a driver */
+void
+rte_auxiliary_unregister(struct rte_auxiliary_driver *driver)
+{
+ TAILQ_REMOVE(&auxiliary_bus.driver_list, driver, next);
+ driver->bus = NULL;
+}
+
+/* Add a device to auxiliary bus */
+void
+auxiliary_add_device(struct rte_auxiliary_device *aux_dev)
+{
+ TAILQ_INSERT_TAIL(&auxiliary_bus.device_list, aux_dev, next);
+}
+
+/* Insert a device into a predefined position in auxiliary bus */
+void
+auxiliary_insert_device(struct rte_auxiliary_device *exist_aux_dev,
+ struct rte_auxiliary_device *new_aux_dev)
+{
+ TAILQ_INSERT_BEFORE(exist_aux_dev, new_aux_dev, next);
+}
+
+/* Remove a device from auxiliary bus */
+static void
+rte_auxiliary_remove_device(struct rte_auxiliary_device *auxiliary_dev)
+{
+ TAILQ_REMOVE(&auxiliary_bus.device_list, auxiliary_dev, next);
+}
+
+static struct rte_device *
+auxiliary_find_device(const struct rte_device *start, rte_dev_cmp_t cmp,
+ const void *data)
+{
+ const struct rte_auxiliary_device *pstart;
+ struct rte_auxiliary_device *adev;
+
+ if (start != NULL) {
+ pstart = RTE_DEV_TO_AUXILIARY_CONST(start);
+ adev = TAILQ_NEXT(pstart, next);
+ } else {
+ adev = TAILQ_FIRST(&auxiliary_bus.device_list);
+ }
+ while (adev != NULL) {
+ if (cmp(&adev->device, data) == 0)
+ return &adev->device;
+ adev = TAILQ_NEXT(adev, next);
+ }
+ return NULL;
+}
+
+static int
+auxiliary_plug(struct rte_device *dev)
+{
+ if (!auxiliary_dev_exists(dev->name))
+ return -ENOENT;
+ return auxiliary_probe_all_drivers(RTE_DEV_TO_AUXILIARY(dev));
+}
+
+static int
+auxiliary_unplug(struct rte_device *dev)
+{
+ struct rte_auxiliary_device *adev;
+ int ret;
+
+ adev = RTE_DEV_TO_AUXILIARY(dev);
+ ret = rte_auxiliary_driver_remove_dev(adev);
+ if (ret == 0) {
+ rte_auxiliary_remove_device(adev);
+ rte_devargs_remove(dev->devargs);
+ free(adev);
+ }
+ return ret;
+}
+
+static int
+auxiliary_dma_map(struct rte_device *dev, void *addr, uint64_t iova, size_t len)
+{
+ struct rte_auxiliary_device *aux_dev = RTE_DEV_TO_AUXILIARY(dev);
+
+ if (dev == NULL || aux_dev->driver == NULL) {
+ rte_errno = EINVAL;
+ return -1;
+ }
+ if (aux_dev->driver->dma_map == NULL) {
+ rte_errno = ENOTSUP;
+ return -1;
+ }
+ return aux_dev->driver->dma_map(aux_dev, addr, iova, len);
+}
+
+static int
+auxiliary_dma_unmap(struct rte_device *dev, void *addr, uint64_t iova,
+ size_t len)
+{
+ struct rte_auxiliary_device *aux_dev = RTE_DEV_TO_AUXILIARY(dev);
+
+ if (dev == NULL || aux_dev->driver == NULL) {
+ rte_errno = EINVAL;
+ return -1;
+ }
+ if (aux_dev->driver->dma_unmap == NULL) {
+ rte_errno = ENOTSUP;
+ return -1;
+ }
+ return aux_dev->driver->dma_unmap(aux_dev, addr, iova, len);
+}
+
+bool
+auxiliary_is_ignored_device(const char *name)
+{
+ struct rte_devargs *devargs = auxiliary_devargs_lookup(name);
+
+ switch (auxiliary_bus.bus.conf.scan_mode) {
+ case RTE_BUS_SCAN_ALLOWLIST:
+ if (devargs && devargs->policy == RTE_DEV_ALLOWED)
+ return false;
+ break;
+ case RTE_BUS_SCAN_UNDEFINED:
+ case RTE_BUS_SCAN_BLOCKLIST:
+ if (devargs == NULL || devargs->policy != RTE_DEV_BLOCKED)
+ return false;
+ break;
+ }
+ return true;
+}
+
+static enum rte_iova_mode
+auxiliary_get_iommu_class(void)
+{
+ const struct rte_auxiliary_driver *drv;
+
+ FOREACH_DRIVER_ON_AUXILIARY_BUS(drv) {
+ if ((drv->drv_flags & RTE_AUXILIARY_DRV_NEED_IOVA_AS_VA) > 0)
+ return RTE_IOVA_VA;
+ }
+
+ return RTE_IOVA_DC;
+}
+
+struct rte_auxiliary_bus auxiliary_bus = {
+ .bus = {
+ .scan = auxiliary_scan,
+ .probe = auxiliary_probe,
+ .find_device = auxiliary_find_device,
+ .plug = auxiliary_plug,
+ .unplug = auxiliary_unplug,
+ .parse = auxiliary_parse,
+ .dma_map = auxiliary_dma_map,
+ .dma_unmap = auxiliary_dma_unmap,
+ .get_iommu_class = auxiliary_get_iommu_class,
+ .dev_iterate = auxiliary_dev_iterate,
+ },
+ .device_list = TAILQ_HEAD_INITIALIZER(auxiliary_bus.device_list),
+ .driver_list = TAILQ_HEAD_INITIALIZER(auxiliary_bus.driver_list),
+};
+
+RTE_REGISTER_BUS(auxiliary, auxiliary_bus.bus);
+RTE_LOG_REGISTER_DEFAULT(auxiliary_bus_logtype, NOTICE);
new file mode 100644
@@ -0,0 +1,59 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2021 NVIDIA Corporation & Affiliates
+ */
+
+#include <string.h>
+
+#include <rte_bus.h>
+#include <rte_dev.h>
+#include <rte_errno.h>
+#include <rte_kvargs.h>
+
+#include "private.h"
+#include "rte_bus_auxiliary.h"
+
+enum auxiliary_params {
+ RTE_AUXILIARY_PARAM_NAME,
+};
+
+static const char * const auxiliary_params_keys[] = {
+ [RTE_AUXILIARY_PARAM_NAME] = "name",
+};
+
+static int
+auxiliary_dev_match(const struct rte_device *dev,
+ const void *_kvlist)
+{
+ const struct rte_kvargs *kvlist = _kvlist;
+ int ret;
+
+ ret = rte_kvargs_process(kvlist,
+ auxiliary_params_keys[RTE_AUXILIARY_PARAM_NAME],
+ rte_kvargs_strcmp, (void *)(uintptr_t)dev->name);
+
+ return ret != 0 ? -1 : 0;
+}
+
+void *
+auxiliary_dev_iterate(const void *start,
+ const char *str,
+ const struct rte_dev_iterator *it __rte_unused)
+{
+ rte_bus_find_device_t find_device;
+ struct rte_kvargs *kvargs = NULL;
+ struct rte_device *dev;
+
+ if (str != NULL) {
+ kvargs = rte_kvargs_parse(str, auxiliary_params_keys);
+ if (kvargs == NULL) {
+ AUXILIARY_LOG(ERR, "cannot parse argument list %s",
+ str);
+ rte_errno = EINVAL;
+ return NULL;
+ }
+ }
+ find_device = auxiliary_bus.bus.find_device;
+ dev = find_device(start, auxiliary_dev_match, kvargs);
+ rte_kvargs_free(kvargs);
+ return dev;
+}
new file mode 100644
@@ -0,0 +1,141 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2021 NVIDIA Corporation & Affiliates
+ */
+
+#include <string.h>
+#include <dirent.h>
+
+#include <rte_log.h>
+#include <rte_bus.h>
+#include <rte_malloc.h>
+#include <rte_devargs.h>
+#include <rte_memcpy.h>
+#include <eal_filesystem.h>
+
+#include "../rte_bus_auxiliary.h"
+#include "../private.h"
+
+#define AUXILIARY_SYSFS_PATH "/sys/bus/auxiliary/devices"
+
+/* Scan one auxiliary sysfs entry, and fill the devices list from it. */
+static int
+auxiliary_scan_one(const char *dirname, const char *name)
+{
+ struct rte_auxiliary_device *dev;
+ struct rte_auxiliary_device *dev2;
+ char filename[PATH_MAX];
+ unsigned long tmp;
+ int ret;
+
+ dev = malloc(sizeof(*dev));
+ if (dev == NULL)
+ return -1;
+
+ memset(dev, 0, sizeof(*dev));
+ if (rte_strscpy(dev->name, name, sizeof(dev->name)) < 0) {
+ free(dev);
+ return -1;
+ }
+ dev->device.name = dev->name;
+ dev->device.bus = &auxiliary_bus.bus;
+
+ /* Get NUMA node, default to 0 if not present */
+ snprintf(filename, sizeof(filename), "%s/%s/numa_node",
+ dirname, name);
+ if (access(filename, F_OK) != -1) {
+ if (eal_parse_sysfs_value(filename, &tmp) == 0)
+ dev->device.numa_node = tmp;
+ else
+ dev->device.numa_node = -1;
+ } else {
+ dev->device.numa_node = 0;
+ }
+
+ auxiliary_on_scan(dev);
+
+ /* Device is valid, add in list (sorted) */
+ TAILQ_FOREACH(dev2, &auxiliary_bus.device_list, next) {
+ ret = strcmp(dev->name, dev2->name);
+ if (ret > 0)
+ continue;
+ if (ret < 0) {
+ auxiliary_insert_device(dev2, dev);
+ } else { /* already registered */
+ if (rte_dev_is_probed(&dev2->device) &&
+ dev2->device.devargs != dev->device.devargs) {
+ /* To probe device with new devargs. */
+ rte_devargs_remove(dev2->device.devargs);
+ auxiliary_on_scan(dev2);
+ }
+ free(dev);
+ }
+ return 0;
+ }
+ auxiliary_add_device(dev);
+ return 0;
+}
+
+/*
+ * Test whether the auxiliary device exist
+ */
+bool
+auxiliary_dev_exists(const char *name)
+{
+ DIR *dir;
+ char dirname[PATH_MAX];
+
+ snprintf(dirname, sizeof(dirname), "%s/%s",
+ AUXILIARY_SYSFS_PATH, name);
+ dir = opendir(dirname);
+ if (dir == NULL)
+ return false;
+ closedir(dir);
+ return true;
+}
+
+/*
+ * Scan the devices in the auxiliary bus
+ */
+int
+auxiliary_scan(void)
+{
+ struct dirent *e;
+ DIR *dir;
+ char dirname[PATH_MAX];
+ struct rte_auxiliary_driver *drv;
+
+ dir = opendir(AUXILIARY_SYSFS_PATH);
+ if (dir == NULL) {
+ AUXILIARY_LOG(INFO, "%s not found, is auxiliary module loaded?",
+ AUXILIARY_SYSFS_PATH);
+ return 0;
+ }
+
+ while ((e = readdir(dir)) != NULL) {
+ if (e->d_name[0] == '.')
+ continue;
+
+ if (auxiliary_is_ignored_device(e->d_name))
+ continue;
+
+ snprintf(dirname, sizeof(dirname), "%s/%s",
+ AUXILIARY_SYSFS_PATH, e->d_name);
+
+ /* Ignore if no driver can handle. */
+ FOREACH_DRIVER_ON_AUXILIARY_BUS(drv) {
+ if (drv->match(e->d_name))
+ break;
+ }
+ if (drv == NULL)
+ continue;
+
+ if (auxiliary_scan_one(dirname, e->d_name) < 0)
+ goto error;
+ }
+ closedir(dir);
+ return 0;
+
+error:
+ closedir(dir);
+ return -1;
+}
new file mode 100644
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright (c) 2021 NVIDIA Corporation & Affiliates
+
+headers = files(
+ 'rte_bus_auxiliary.h',
+)
+sources = files(
+ 'auxiliary_common.c',
+ 'auxiliary_params.c',
+)
+if is_linux
+ sources += files(
+ 'linux/auxiliary.c',
+ )
+endif
+deps += ['kvargs']
new file mode 100644
@@ -0,0 +1,74 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2021 NVIDIA Corporation & Affiliates
+ */
+
+#ifndef AUXILIARY_PRIVATE_H
+#define AUXILIARY_PRIVATE_H
+
+#include <stdbool.h>
+#include <stdio.h>
+
+#include "rte_bus_auxiliary.h"
+
+extern struct rte_auxiliary_bus auxiliary_bus;
+extern int auxiliary_bus_logtype;
+
+#define AUXILIARY_LOG(level, ...) \
+ rte_log(RTE_LOG_ ## level, auxiliary_bus_logtype, \
+ RTE_FMT("auxiliary bus: " RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
+ RTE_FMT_TAIL(__VA_ARGS__,)))
+
+/* Auxiliary bus iterators */
+#define FOREACH_DEVICE_ON_AUXILIARY_BUS(p) \
+ TAILQ_FOREACH(p, &(auxiliary_bus.device_list), next)
+
+#define FOREACH_DRIVER_ON_AUXILIARY_BUS(p) \
+ TAILQ_FOREACH(p, &(auxiliary_bus.driver_list), next)
+
+bool auxiliary_dev_exists(const char *name);
+
+/*
+ * Scan the content of the auxiliary bus, and the devices in the devices
+ * list.
+ */
+int auxiliary_scan(void);
+
+/*
+ * Update a device being scanned.
+ */
+void auxiliary_on_scan(struct rte_auxiliary_device *aux_dev);
+
+/*
+ * Validate whether a device with given auxiliary device should be ignored
+ * or not.
+ */
+bool auxiliary_is_ignored_device(const char *name);
+
+/*
+ * Add an auxiliary device to the auxiliary bus (append to auxiliary device
+ * list). This function also updates the bus references of the auxiliary
+ * device and the generic device object embedded within.
+ */
+void auxiliary_add_device(struct rte_auxiliary_device *aux_dev);
+
+/*
+ * Insert an auxiliary device in the auxiliary bus at a particular location
+ * in the device list. It also updates the auxiliary bus reference of the
+ * new devices to be inserted.
+ */
+void auxiliary_insert_device(struct rte_auxiliary_device *exist_aux_dev,
+ struct rte_auxiliary_device *new_aux_dev);
+
+/*
+ * Match the auxiliary driver and device by driver function
+ */
+bool auxiliary_match(const struct rte_auxiliary_driver *aux_drv,
+ const struct rte_auxiliary_device *aux_dev);
+
+/*
+ * Iterate over devices, matching any device against the provided string
+ */
+void *auxiliary_dev_iterate(const void *start, const char *str,
+ const struct rte_dev_iterator *it);
+
+#endif /* AUXILIARY_PRIVATE_H */
new file mode 100644
@@ -0,0 +1,201 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2021 NVIDIA Corporation & Affiliates
+ */
+
+#ifndef RTE_BUS_AUXILIARY_H
+#define RTE_BUS_AUXILIARY_H
+
+/**
+ * @file
+ *
+ * Auxiliary Bus Interface.
+ */
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <limits.h>
+#include <errno.h>
+#include <sys/queue.h>
+#include <stdint.h>
+#include <inttypes.h>
+
+#include <rte_debug.h>
+#include <rte_interrupts.h>
+#include <rte_dev.h>
+#include <rte_bus.h>
+#include <rte_kvargs.h>
+
+#define RTE_BUS_AUXILIARY_NAME "auxiliary"
+
+/* Forward declarations */
+struct rte_auxiliary_driver;
+struct rte_auxiliary_bus;
+struct rte_auxiliary_device;
+
+/**
+ * Match function for the driver to decide if device can be handled.
+ *
+ * @param name
+ * Pointer to the auxiliary device name.
+ * @return
+ * Whether the driver can handle the auxiliary device.
+ */
+typedef bool(rte_auxiliary_match_t)(const char *name);
+
+/**
+ * Initialization function for the driver called during auxiliary probing.
+ *
+ * @param drv
+ * Pointer to the auxiliary driver.
+ * @param dev
+ * Pointer to the auxiliary device.
+ * @return
+ * - 0 On success.
+ * - Negative value and rte_errno is set otherwise.
+ */
+typedef int(rte_auxiliary_probe_t)(struct rte_auxiliary_driver *drv,
+ struct rte_auxiliary_device *dev);
+
+/**
+ * Uninitialization function for the driver called during hotplugging.
+ *
+ * @param dev
+ * Pointer to the auxiliary device.
+ * @return
+ * - 0 On success.
+ * - Negative value and rte_errno is set otherwise.
+ */
+typedef int (rte_auxiliary_remove_t)(struct rte_auxiliary_device *dev);
+
+/**
+ * Driver-specific DMA mapping. After a successful call the device
+ * will be able to read/write from/to this segment.
+ *
+ * @param dev
+ * Pointer to the auxiliary device.
+ * @param addr
+ * Starting virtual address of memory to be mapped.
+ * @param iova
+ * Starting IOVA address of memory to be mapped.
+ * @param len
+ * Length of memory segment being mapped.
+ * @return
+ * - 0 On success.
+ * - Negative value and rte_errno is set otherwise.
+ */
+typedef int (rte_auxiliary_dma_map_t)(struct rte_auxiliary_device *dev,
+ void *addr, uint64_t iova, size_t len);
+
+/**
+ * Driver-specific DMA un-mapping. After a successful call the device
+ * will not be able to read/write from/to this segment.
+ *
+ * @param dev
+ * Pointer to the auxiliary device.
+ * @param addr
+ * Starting virtual address of memory to be unmapped.
+ * @param iova
+ * Starting IOVA address of memory to be unmapped.
+ * @param len
+ * Length of memory segment being unmapped.
+ * @return
+ * - 0 On success.
+ * - Negative value and rte_errno is set otherwise.
+ */
+typedef int (rte_auxiliary_dma_unmap_t)(struct rte_auxiliary_device *dev,
+ void *addr, uint64_t iova, size_t len);
+
+/**
+ * A structure describing an auxiliary device.
+ */
+struct rte_auxiliary_device {
+ TAILQ_ENTRY(rte_auxiliary_device) next; /**< Next probed device. */
+ struct rte_device device; /**< Inherit core device */
+ char name[RTE_DEV_NAME_MAX_LEN + 1]; /**< ASCII device name */
+ struct rte_intr_handle intr_handle; /**< Interrupt handle */
+ struct rte_auxiliary_driver *driver; /**< Device driver */
+};
+
+/** List of auxiliary devices */
+TAILQ_HEAD(rte_auxiliary_device_list, rte_auxiliary_device);
+/** List of auxiliary drivers */
+TAILQ_HEAD(rte_auxiliary_driver_list, rte_auxiliary_driver);
+
+/**
+ * Structure describing the auxiliary bus
+ */
+struct rte_auxiliary_bus {
+ struct rte_bus bus; /**< Inherit the generic class */
+ struct rte_auxiliary_device_list device_list; /**< List of devices */
+ struct rte_auxiliary_driver_list driver_list; /**< List of drivers */
+};
+
+/**
+ * A structure describing an auxiliary driver.
+ */
+struct rte_auxiliary_driver {
+ TAILQ_ENTRY(rte_auxiliary_driver) next; /**< Next in list. */
+ struct rte_driver driver; /**< Inherit core driver. */
+ struct rte_auxiliary_bus *bus; /**< Auxiliary bus reference. */
+ rte_auxiliary_match_t *match; /**< Device match function. */
+ rte_auxiliary_probe_t *probe; /**< Device probe function. */
+ rte_auxiliary_remove_t *remove; /**< Device remove function. */
+ rte_auxiliary_dma_map_t *dma_map; /**< Device DMA map function. */
+ rte_auxiliary_dma_unmap_t *dma_unmap; /**< Device DMA unmap function. */
+ uint32_t drv_flags; /**< Flags RTE_AUXILIARY_DRV_*. */
+};
+
+/**
+ * @internal
+ * Helper macro for drivers that need to convert to struct rte_auxiliary_device.
+ */
+#define RTE_DEV_TO_AUXILIARY(ptr) \
+ container_of(ptr, struct rte_auxiliary_device, device)
+
+#define RTE_DEV_TO_AUXILIARY_CONST(ptr) \
+ container_of(ptr, const struct rte_auxiliary_device, device)
+
+#define RTE_ETH_DEV_TO_AUXILIARY(eth_dev) \
+ RTE_DEV_TO_AUXILIARY((eth_dev)->device)
+
+/** Device driver needs IOVA as VA and cannot work with IOVA as PA */
+#define RTE_AUXILIARY_DRV_NEED_IOVA_AS_VA 0x002
+
+/**
+ * Register an auxiliary driver.
+ *
+ * @param driver
+ * A pointer to a rte_auxiliary_driver structure describing the driver
+ * to be registered.
+ */
+__rte_experimental
+void rte_auxiliary_register(struct rte_auxiliary_driver *driver);
+
+/** Helper for auxiliary device registration from driver instance */
+#define RTE_PMD_REGISTER_AUXILIARY(nm, auxiliary_drv) \
+ RTE_INIT(auxiliaryinitfn_ ##nm) \
+ { \
+ (auxiliary_drv).driver.name = RTE_STR(nm); \
+ rte_auxiliary_register(&(auxiliary_drv)); \
+ } \
+ RTE_PMD_EXPORT_NAME(nm, __COUNTER__)
+
+/**
+ * Unregister an auxiliary driver.
+ *
+ * @param driver
+ * A pointer to a rte_auxiliary_driver structure describing the driver
+ * to be unregistered.
+ */
+__rte_experimental
+void rte_auxiliary_unregister(struct rte_auxiliary_driver *driver);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* RTE_BUS_AUXILIARY_H */
new file mode 100644
@@ -0,0 +1,7 @@
+EXPERIMENTAL {
+ global:
+
+ # added in 21.08
+ rte_auxiliary_register;
+ rte_auxiliary_unregister;
+};
@@ -2,6 +2,7 @@
# Copyright(c) 2017 Intel Corporation
drivers = [
+ 'auxiliary',
'dpaa',
'fslmc',
'ifpga',