diff mbox series

[v2,4/4] eal: select IOVA mode as VA for default case

Message ID 20190716134609.40930-5-jerinj@marvell.com (mailing list archive)
State Superseded, archived
Delegated to: Thomas Monjalon
Headers
Series Fixes on IOVA mode selection |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation success Compilation OK

Commit Message

Jerin Jacob Kollanukkaran July 16, 2019, 1:46 p.m. UTC 
  From: Jerin Jacob <jerinj@marvell.com>

When bus layer selected the preferred mode as RTE_IOVA_DC then
select the IOVA mode as RTE_IOVA_VA.

The RTE_IOVA_VA selected as the default because,

1) All drivers work in RTE_IOVA_VA mode, irrespective of physical
address availability.

2) By default, the mempool, first asks for IOVA-contiguous memory
using RTE_MEMZONE_IOVA_CONTIG and this is slow in IOVA as PA mode
and it may affect the application boot time.

Signed-off-by: Jerin Jacob <jerinj@marvell.com>
---
 doc/guides/prog_guide/env_abstraction_layer.rst | 10 ++++++++--
 lib/librte_eal/linux/eal/eal.c                  |  6 ++----
 2 files changed, 10 insertions(+), 6 deletions(-)

Comments

Anatoly Burakov July 16, 2019, 2:33 p.m. UTC | #1 
On 16-Jul-19 2:46 PM, jerinj@marvell.com wrote:
> From: Jerin Jacob <jerinj@marvell.com>
> 
> When bus layer selected the preferred mode as RTE_IOVA_DC then
> select the IOVA mode as RTE_IOVA_VA.
> 
> The RTE_IOVA_VA selected as the default because,
> 
> 1) All drivers work in RTE_IOVA_VA mode, irrespective of physical
> address availability.
> 
> 2) By default, the mempool, first asks for IOVA-contiguous memory
> using RTE_MEMZONE_IOVA_CONTIG and this is slow in IOVA as PA mode
> and it may affect the application boot time.
> 
> Signed-off-by: Jerin Jacob <jerinj@marvell.com>
> ---

I should celebrate now :D

>   doc/guides/prog_guide/env_abstraction_layer.rst | 10 ++++++++--
>   lib/librte_eal/linux/eal/eal.c                  |  6 ++----
>   2 files changed, 10 insertions(+), 6 deletions(-)
> 
> diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst
> index 77307e3a6..1b0343eee 100644
> --- a/doc/guides/prog_guide/env_abstraction_layer.rst
> +++ b/doc/guides/prog_guide/env_abstraction_layer.rst
> @@ -445,8 +445,14 @@ kernels.
>   - if the preferred mode is RTE_IOVA_PA but there is no access to Physical
>     Addresses, then EAL init will fail early, since later probing of the devices
>     would fail anyway,
> -- if the preferred mode is RTE_IOVA_DC then based on the Physical Addresses
> -  availability, the preferred mode is adjusted to RTE_IOVA_PA or RTE_IOVA_VA.
> +- if the preferred mode is RTE_IOVA_DC then select the IOVA mode as RTE_IOVA_VA.
> +  The RTE_IOVA_VA selected as the default because,
> +
> +#. All drivers work in RTE_IOVA_VA mode, irrespective of physical address availability.

Is there anywhere we can document that any new driver must support both 
before being accepted?

> +
> +#. By default, the mempool, first asks for IOVA-contiguous memory using ``RTE_MEMZONE_IOVA_CONTIG``,
> +   and this is slow in IOVA as PA mode and it may affect the application boot time.

I would also add a point about usability improvement for use-cases which 
require large amounts of IOVA-contiguous memory.

Otherwise,

Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
Jerin Jacob Kollanukkaran July 17, 2019, 8:33 a.m. UTC | #2 
> -----Original Message-----
> From: Burakov, Anatoly <anatoly.burakov@intel.com>
> Sent: Tuesday, July 16, 2019 8:03 PM
> To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; dev@dpdk.org; John
> McNamara <john.mcnamara@intel.com>; Marko Kovacevic
> <marko.kovacevic@intel.com>
> Cc: thomas@monjalon.net; david.marchand@redhat.com
> Subject: [EXT] Re: [dpdk-dev] [PATCH v2 4/4] eal: select IOVA mode as VA for
> default case
> 
> On 16-Jul-19 2:46 PM, jerinj@marvell.com wrote:
> > From: Jerin Jacob <jerinj@marvell.com>
> >
> > When bus layer selected the preferred mode as RTE_IOVA_DC then select
> > the IOVA mode as RTE_IOVA_VA.
> >
> > The RTE_IOVA_VA selected as the default because,
> >
> > 1) All drivers work in RTE_IOVA_VA mode, irrespective of physical
> > address availability.
> >
> > 2) By default, the mempool, first asks for IOVA-contiguous memory
> > using RTE_MEMZONE_IOVA_CONTIG and this is slow in IOVA as PA mode
> and
> > it may affect the application boot time.
> >
> > Signed-off-by: Jerin Jacob <jerinj@marvell.com>
> > ---
> 
> I should celebrate now :D
> 
> >   doc/guides/prog_guide/env_abstraction_layer.rst | 10 ++++++++--
> >   lib/librte_eal/linux/eal/eal.c                  |  6 ++----
> >   2 files changed, 10 insertions(+), 6 deletions(-)
> >
> > diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst
> > b/doc/guides/prog_guide/env_abstraction_layer.rst
> > index 77307e3a6..1b0343eee 100644
> > --- a/doc/guides/prog_guide/env_abstraction_layer.rst
> > +++ b/doc/guides/prog_guide/env_abstraction_layer.rst
> > @@ -445,8 +445,14 @@ kernels.
> >   - if the preferred mode is RTE_IOVA_PA but there is no access to Physical
> >     Addresses, then EAL init will fail early, since later probing of the devices
> >     would fail anyway,
> > -- if the preferred mode is RTE_IOVA_DC then based on the Physical
> > Addresses
> > -  availability, the preferred mode is adjusted to RTE_IOVA_PA or
> RTE_IOVA_VA.
> > +- if the preferred mode is RTE_IOVA_DC then select the IOVA mode as
> RTE_IOVA_VA.
> > +  The RTE_IOVA_VA selected as the default because,
> > +
> > +#. All drivers work in RTE_IOVA_VA mode, irrespective of physical address
> availability.
> 
> Is there anywhere we can document that any new driver must support both
> before being accepted?

Not sure why new drivers need to support both PA and VA. Do you mean VA?
And not sure where to document this as well if need.

> 
> > +
> > +#. By default, the mempool, first asks for IOVA-contiguous memory using
> ``RTE_MEMZONE_IOVA_CONTIG``,
> > +   and this is slow in IOVA as PA mode and it may affect the application
> boot time.
> 
> I would also add a point about usability improvement for use-cases which
> require large amounts of IOVA-contiguous memory.

I will add in next version:
How about the following, Let me know if any change required.

#. It is easy to enable large amount of IOVA-contiguous memory use-cases with IOVA in VA mode.

> 
> Otherwise,
> 
> Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
Anatoly Burakov July 17, 2019, 12:38 p.m. UTC | #3 
On 17-Jul-19 9:33 AM, Jerin Jacob Kollanukkaran wrote:
>> -----Original Message-----
>> From: Burakov, Anatoly <anatoly.burakov@intel.com>
>> Sent: Tuesday, July 16, 2019 8:03 PM
>> To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; dev@dpdk.org; John
>> McNamara <john.mcnamara@intel.com>; Marko Kovacevic
>> <marko.kovacevic@intel.com>
>> Cc: thomas@monjalon.net; david.marchand@redhat.com
>> Subject: [EXT] Re: [dpdk-dev] [PATCH v2 4/4] eal: select IOVA mode as VA for
>> default case
>>
>> On 16-Jul-19 2:46 PM, jerinj@marvell.com wrote:
>>> From: Jerin Jacob <jerinj@marvell.com>
>>>
>>> When bus layer selected the preferred mode as RTE_IOVA_DC then select
>>> the IOVA mode as RTE_IOVA_VA.
>>>
>>> The RTE_IOVA_VA selected as the default because,
>>>
>>> 1) All drivers work in RTE_IOVA_VA mode, irrespective of physical
>>> address availability.
>>>
>>> 2) By default, the mempool, first asks for IOVA-contiguous memory
>>> using RTE_MEMZONE_IOVA_CONTIG and this is slow in IOVA as PA mode
>> and
>>> it may affect the application boot time.
>>>
>>> Signed-off-by: Jerin Jacob <jerinj@marvell.com>
>>> ---
>>
>> I should celebrate now :D
>>
>>>    doc/guides/prog_guide/env_abstraction_layer.rst | 10 ++++++++--
>>>    lib/librte_eal/linux/eal/eal.c                  |  6 ++----
>>>    2 files changed, 10 insertions(+), 6 deletions(-)
>>>
>>> diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst
>>> b/doc/guides/prog_guide/env_abstraction_layer.rst
>>> index 77307e3a6..1b0343eee 100644
>>> --- a/doc/guides/prog_guide/env_abstraction_layer.rst
>>> +++ b/doc/guides/prog_guide/env_abstraction_layer.rst
>>> @@ -445,8 +445,14 @@ kernels.
>>>    - if the preferred mode is RTE_IOVA_PA but there is no access to Physical
>>>      Addresses, then EAL init will fail early, since later probing of the devices
>>>      would fail anyway,
>>> -- if the preferred mode is RTE_IOVA_DC then based on the Physical
>>> Addresses
>>> -  availability, the preferred mode is adjusted to RTE_IOVA_PA or
>> RTE_IOVA_VA.
>>> +- if the preferred mode is RTE_IOVA_DC then select the IOVA mode as
>> RTE_IOVA_VA.
>>> +  The RTE_IOVA_VA selected as the default because,
>>> +
>>> +#. All drivers work in RTE_IOVA_VA mode, irrespective of physical address
>> availability.
>>
>> Is there anywhere we can document that any new driver must support both
>> before being accepted?
> 
> Not sure why new drivers need to support both PA and VA. Do you mean VA?
> And not sure where to document this as well if need.

We have a flaf that indicates that the driver needs IOVA as VA. Absence 
of said flag indicates that it supports both IOVA as VA and IOVA as PA. 
So, absent of this flag, any new driver must support both PA and VA, 
must it not?

> 
>>
>>> +
>>> +#. By default, the mempool, first asks for IOVA-contiguous memory using
>> ``RTE_MEMZONE_IOVA_CONTIG``,
>>> +   and this is slow in IOVA as PA mode and it may affect the application
>> boot time.
>>
>> I would also add a point about usability improvement for use-cases which
>> require large amounts of IOVA-contiguous memory.
> 
> I will add in next version:
> How about the following, Let me know if any change required.
> 
> #. It is easy to enable large amount of IOVA-contiguous memory use-cases with IOVA in VA mode.

Yes, that looks OK.

> 
>>
>> Otherwise,
>>
>> Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
Jerin Jacob Kollanukkaran July 17, 2019, 2:04 p.m. UTC | #4 
> >>>      Addresses, then EAL init will fail early, since later probing of the
> devices
> >>>      would fail anyway,
> >>> -- if the preferred mode is RTE_IOVA_DC then based on the Physical
> >>> Addresses
> >>> -  availability, the preferred mode is adjusted to RTE_IOVA_PA or
> >> RTE_IOVA_VA.
> >>> +- if the preferred mode is RTE_IOVA_DC then select the IOVA mode as
> >> RTE_IOVA_VA.
> >>> +  The RTE_IOVA_VA selected as the default because,
> >>> +
> >>> +#. All drivers work in RTE_IOVA_VA mode, irrespective of physical
> >>> +address
> >> availability.
> >>
> >> Is there anywhere we can document that any new driver must support
> >> both before being accepted?
> >
> > Not sure why new drivers need to support both PA and VA. Do you mean
> VA?
> > And not sure where to document this as well if need.
> 
> We have a flaf that indicates that the driver needs IOVA as VA. Absence of
> said flag indicates that it supports both IOVA as VA and IOVA as PA.
> So, absent of this flag, any new driver must support both PA and VA, must it
> not?

OK. I will add the following as "note" after "IOVA Mode Detection" section.
I don’t any other place to put this info in the doc. If any change needed
then let me know.

.. note::

    If the device driver needs IOVA as VA and it cannot work with IOVA as PA
    then the driver must request the PCI bus layer using  ``RTE_PCI_DRV_NEED_IOVA_AS_VA``
    requirement flag. Absence of this flag, dictates, the driver must support both IOVA as PA and VA modes.
diff mbox series

Patch

diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst
index 77307e3a6..1b0343eee 100644
--- a/doc/guides/prog_guide/env_abstraction_layer.rst
+++ b/doc/guides/prog_guide/env_abstraction_layer.rst
@@ -445,8 +445,14 @@  kernels.
 - if the preferred mode is RTE_IOVA_PA but there is no access to Physical
   Addresses, then EAL init will fail early, since later probing of the devices
   would fail anyway,
-- if the preferred mode is RTE_IOVA_DC then based on the Physical Addresses
-  availability, the preferred mode is adjusted to RTE_IOVA_PA or RTE_IOVA_VA.
+- if the preferred mode is RTE_IOVA_DC then select the IOVA mode as RTE_IOVA_VA.
+  The RTE_IOVA_VA selected as the default because,
+
+#. All drivers work in RTE_IOVA_VA mode, irrespective of physical address availability.
+
+#. By default, the mempool, first asks for IOVA-contiguous memory using ``RTE_MEMZONE_IOVA_CONTIG``,
+   and this is slow in IOVA as PA mode and it may affect the application boot time.
+
   In the case when the buses had disagreed on the IOVA Mode at the first step,
   part of the buses won't work because of this decision.
 
diff --git a/lib/librte_eal/linux/eal/eal.c b/lib/librte_eal/linux/eal/eal.c
index 2e5499f9b..34db78753 100644
--- a/lib/librte_eal/linux/eal/eal.c
+++ b/lib/librte_eal/linux/eal/eal.c
@@ -1061,10 +1061,8 @@  rte_eal_init(int argc, char **argv)
 		enum rte_iova_mode iova_mode = rte_bus_get_iommu_class();
 
 		if (iova_mode == RTE_IOVA_DC) {
-			iova_mode = phys_addrs ? RTE_IOVA_PA : RTE_IOVA_VA;
-			RTE_LOG(DEBUG, EAL,
-				"Buses did not request a specific IOVA mode, using '%s' based on physical addresses availability.\n",
-				phys_addrs ? "PA" : "VA");
+			iova_mode = RTE_IOVA_VA;
+			RTE_LOG(DEBUG, EAL, "Buses did not request a specific IOVA mode, select IOVA as VA mode.\n");
 		}
 #ifdef RTE_LIBRTE_KNI
 		/* Workaround for KNI which requires physical address to work */