[v1,1/1] bus/vdev: add get iommu class callback

  From: Vamsi Attunuru <vattunuru@marvell.com>

When DPDK application is run with vdevs, EAL decides
the iommu mode and accordingly application runs in
either of the modes.

For KNI kind of functionality with vdevs, since there
is no backed device structure in kernel, iommu_mode = PA
needs to be enforced during eal init.

Patch adds get_iommu_class callback in vdev bus driver
and returns iommu mode as PA when vdevs are used for
kni functionality, callback returns iommu mode as DC
in normal cases.

Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
---

Below support fixes the following patchset(add iova=va mode support in KNI)
which is currently failing to create KNI devices when vdevs are passed.
http://patches.dpdk.org/patch/57720/

 drivers/bus/vdev/vdev.c                 | 15 +++++++++++++++
 lib/librte_eal/common/eal_private.h     | 14 --------------
 lib/librte_eal/common/include/rte_eal.h | 14 ++++++++++++++
 3 files changed, 29 insertions(+), 14 deletions(-)
  

On Wed, Oct 16, 2019 at 7:59 PM <vattunuru@marvell.com> wrote:
>
> From: Vamsi Attunuru <vattunuru@marvell.com>
>
> When DPDK application is run with vdevs, EAL decides
> the iommu mode and accordingly application runs in
> either of the modes.
>
> For KNI kind of functionality with vdevs, since there
> is no backed device structure in kernel, iommu_mode = PA
> needs to be enforced during eal init.
>
> Patch adds get_iommu_class callback in vdev bus driver
> and returns iommu mode as PA when vdevs are used for
> kni functionality, callback returns iommu mode as DC
> in normal cases.
>
> Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
> ---
>
> Below support fixes the following patchset(add iova=va mode support in KNI)
> which is currently failing to create KNI devices when vdevs are passed.
> http://patches.dpdk.org/patch/57720/
>
>  drivers/bus/vdev/vdev.c                 | 15 +++++++++++++++
>  lib/librte_eal/common/eal_private.h     | 14 --------------
>  lib/librte_eal/common/include/rte_eal.h | 14 ++++++++++++++
>  3 files changed, 29 insertions(+), 14 deletions(-)
>
> diff --git a/drivers/bus/vdev/vdev.c b/drivers/bus/vdev/vdev.c
> index a89ea23..ab07738 100644
> --- a/drivers/bus/vdev/vdev.c
> +++ b/drivers/bus/vdev/vdev.c
> @@ -546,6 +546,20 @@ vdev_unplug(struct rte_device *dev)
>         return rte_vdev_uninit(dev->name);
>  }
>
> +static enum rte_iova_mode
> +rte_vdev_get_iommu_class(void)
> +{
> +       struct rte_devargs *devargs = NULL;
> +
> +       if (rte_eal_check_module("rte_kni") == 1) {
> +               RTE_EAL_DEVARGS_FOREACH("vdev", devargs) {
> +                       return RTE_IOVA_PA;
> +               }
> +       }
> +
> +       return RTE_IOVA_DC;
> +}

This will fix KNI + vdev issue.

# Move the check under #ifdef RTE_LIBRTE_KNI
# Please add a comment in the code such as
"vdev based KNI needs IOVA as PA to work"


>  static struct rte_bus rte_vdev_bus = {
>         .scan = vdev_scan,
>         .probe = vdev_probe,
> @@ -554,6 +568,7 @@ static struct rte_bus rte_vdev_bus = {
>         .unplug = vdev_unplug,
>         .parse = vdev_parse,
>         .dev_iterate = rte_vdev_dev_iterate,
> +       .get_iommu_class = rte_vdev_get_iommu_class,
>  };
>
>  RTE_REGISTER_BUS(vdev, rte_vdev_bus);
> diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h
> index 798ede5..4207bdb 100644
> --- a/lib/librte_eal/common/eal_private.h
> +++ b/lib/librte_eal/common/eal_private.h
> @@ -133,20 +133,6 @@ int rte_eal_intr_init(void);
>  int rte_eal_alarm_init(void);
>
>  /**
> - * Function is to check if the kernel module(like, vfio, vfio_iommu_type1,
> - * etc.) loaded.
> - *
> - * @param module_name
> - *     The module's name which need to be checked
> - *
> - * @return
> - *     -1 means some error happens(NULL pointer or open failure)
> - *     0  means the module not loaded
> - *     1  means the module loaded
> - */
> -int rte_eal_check_module(const char *module_name);
> -
> -/**
>   * Get virtual area of specified size from the OS.
>   *
>   * This function is private to the EAL.
> diff --git a/lib/librte_eal/common/include/rte_eal.h b/lib/librte_eal/common/include/rte_eal.h
> index b7cf912..4ea98e6 100644
> --- a/lib/librte_eal/common/include/rte_eal.h
> +++ b/lib/librte_eal/common/include/rte_eal.h
> @@ -531,6 +531,20 @@ rte_eal_mbuf_user_pool_ops(void);
>  const char *
>  rte_eal_get_runtime_dir(void);
>
> +/**
> + * Function is to check if the kernel module(like, vfio, vfio_iommu_type1,
> + * etc.) loaded.
> + *
> + * @param module_name
> + *     The module's name which need to be checked
> + *
> + * @return
> + *     -1 means some error happens(NULL pointer or open failure)
> + *     0  means the module not loaded
> + *     1  means the module loaded
> + */
> +int rte_eal_check_module(const char *module_name);

Please move, promoting rte_eal_check_module() as public API in a
different patch.

And update
1) lib/librte_eal/rte_eal_version.map
2) Add FreeBSD support as well, it can return always zero


> +
>  #ifdef __cplusplus
>  }
>  #endif
> --
> 2.8.4
>
  

On 10/16/2019 3:29 PM, vattunuru@marvell.com wrote:
> From: Vamsi Attunuru <vattunuru@marvell.com>
> 
> When DPDK application is run with vdevs, EAL decides
> the iommu mode and accordingly application runs in
> either of the modes.
> 
> For KNI kind of functionality with vdevs, since there
> is no backed device structure in kernel, iommu_mode = PA
> needs to be enforced during eal init.
> 
> Patch adds get_iommu_class callback in vdev bus driver
> and returns iommu mode as PA when vdevs are used for
> kni functionality, callback returns iommu mode as DC
> in normal cases.
> 
> Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>

NACK, more comment on other thread:
https://mails.dpdk.org/archives/dev/2019-October/147459.html