doc: update TAP device features

Message ID 20240904154206.13438-1-stephen@networkplumber.org (mailing list archive)
State Changes Requested
Delegated to: Ferruh Yigit
Headers
Series doc: update TAP device features |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/loongarch-compilation success Compilation OK
ci/loongarch-unit-testing success Unit Testing PASS
ci/Intel-compilation success Compilation OK
ci/intel-Testing success Testing PASS
ci/github-robot: build success github build: passed
ci/intel-Functional success Functional PASS
ci/iol-marvell-Functional success Functional Testing PASS
ci/iol-unit-amd64-testing success Testing PASS
ci/iol-broadcom-Performance success Performance Testing PASS
ci/iol-broadcom-Functional success Functional Testing PASS
ci/iol-unit-arm64-testing success Testing PASS
ci/iol-compile-amd64-testing success Testing PASS
ci/iol-compile-arm64-testing success Testing PASS
ci/iol-sample-apps-testing success Testing PASS
ci/iol-intel-Functional success Functional Testing PASS
ci/iol-intel-Performance success Performance Testing PASS

Commit Message

Stephen Hemminger Sept. 4, 2024, 3:42 p.m. UTC
The TAP device does have per-queue stats and handles multi-process.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
 doc/guides/nics/features/tap.ini | 2 ++
 1 file changed, 2 insertions(+)
  

Comments

Ferruh Yigit Oct. 4, 2024, 1:48 a.m. UTC | #1
On 9/4/2024 4:42 PM, Stephen Hemminger wrote:
> The TAP device does have per-queue stats and handles multi-process.
> 
> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> ---
>  doc/guides/nics/features/tap.ini | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/doc/guides/nics/features/tap.ini b/doc/guides/nics/features/tap.ini
> index f26355e57f..f2ea5cd833 100644
> --- a/doc/guides/nics/features/tap.ini
> +++ b/doc/guides/nics/features/tap.ini
> @@ -14,10 +14,12 @@ Basic stats          = Y
>  L3 checksum offload  = Y
>  L4 checksum offload  = Y
>  MTU update           = Y
> +Multiprocess aware   = Y
>

ack

>  Multicast MAC filter = Y
>  Unicast MAC filter   = Y
>  Packet type parsing  = Y
>  Flow control         = Y
> +Stats per queue      = Y
>

This feature name is misleading,
it is for 'rte_eth_dev_set_[rt]x_queue_stats_mapping()' API, which is
indeed for covering limitation for some drivers.
Tap does support getting stats per queue, but doesn't support above
documented feature.

>  Linux                = Y
>  ARMv7                = Y
>  ARMv8                = Y
  
Stephen Hemminger Oct. 4, 2024, 2:26 a.m. UTC | #2
On Fri, 4 Oct 2024 02:48:21 +0100
Ferruh Yigit <ferruh.yigit@amd.com> wrote:

> On 9/4/2024 4:42 PM, Stephen Hemminger wrote:
> > The TAP device does have per-queue stats and handles multi-process.
> > 
> > Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> > ---
> >  doc/guides/nics/features/tap.ini | 2 ++
> >  1 file changed, 2 insertions(+)
> > 
> > diff --git a/doc/guides/nics/features/tap.ini b/doc/guides/nics/features/tap.ini
> > index f26355e57f..f2ea5cd833 100644
> > --- a/doc/guides/nics/features/tap.ini
> > +++ b/doc/guides/nics/features/tap.ini
> > @@ -14,10 +14,12 @@ Basic stats          = Y
> >  L3 checksum offload  = Y
> >  L4 checksum offload  = Y
> >  MTU update           = Y
> > +Multiprocess aware   = Y
> >  
> 
> ack
> 
> >  Multicast MAC filter = Y
> >  Unicast MAC filter   = Y
> >  Packet type parsing  = Y
> >  Flow control         = Y
> > +Stats per queue      = Y
> >  
> 
> This feature name is misleading,
> it is for 'rte_eth_dev_set_[rt]x_queue_stats_mapping()' API, which is
> indeed for covering limitation for some drivers.
> Tap does support getting stats per queue, but doesn't support above
> documented feature.

The stats queue mapping was a feature that was hinted at being removed.
It only exists because of HW limitations on Intel ixgbe NIC and SW
limitations from RTE_ETHDEV_QUEUE_STAT_CNTRS.

Perhaps there should be a generic SW emulation for this the mapping?
  
Ferruh Yigit Oct. 4, 2024, 4:09 a.m. UTC | #3
On 10/4/2024 3:26 AM, Stephen Hemminger wrote:
> On Fri, 4 Oct 2024 02:48:21 +0100
> Ferruh Yigit <ferruh.yigit@amd.com> wrote:
> 
>> On 9/4/2024 4:42 PM, Stephen Hemminger wrote:
>>> The TAP device does have per-queue stats and handles multi-process.
>>>
>>> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
>>> ---
>>>  doc/guides/nics/features/tap.ini | 2 ++
>>>  1 file changed, 2 insertions(+)
>>>
>>> diff --git a/doc/guides/nics/features/tap.ini b/doc/guides/nics/features/tap.ini
>>> index f26355e57f..f2ea5cd833 100644
>>> --- a/doc/guides/nics/features/tap.ini
>>> +++ b/doc/guides/nics/features/tap.ini
>>> @@ -14,10 +14,12 @@ Basic stats          = Y
>>>  L3 checksum offload  = Y
>>>  L4 checksum offload  = Y
>>>  MTU update           = Y
>>> +Multiprocess aware   = Y
>>>  
>>
>> ack
>>
>>>  Multicast MAC filter = Y
>>>  Unicast MAC filter   = Y
>>>  Packet type parsing  = Y
>>>  Flow control         = Y
>>> +Stats per queue      = Y
>>>  
>>
>> This feature name is misleading,
>> it is for 'rte_eth_dev_set_[rt]x_queue_stats_mapping()' API, which is
>> indeed for covering limitation for some drivers.
>> Tap does support getting stats per queue, but doesn't support above
>> documented feature.
> 
> The stats queue mapping was a feature that was hinted at being removed.
> It only exists because of HW limitations on Intel ixgbe NIC and SW
> limitations from RTE_ETHDEV_QUEUE_STAT_CNTRS.
> 


We have a plan to remove 'RTE_ETHDEV_QUEUE_STAT_CNTRS', by moving queue
stats to xstats.

But ixgbe limitation is there.

> Perhaps there should be a generic SW emulation for this the mapping?
>

Ack, cc'ed Bruce.
But I am not sure ROI of the effort at this stage.
  
Bruce Richardson Oct. 4, 2024, 7:54 a.m. UTC | #4
On Fri, Oct 04, 2024 at 05:09:21AM +0100, Ferruh Yigit wrote:
> On 10/4/2024 3:26 AM, Stephen Hemminger wrote:
> > On Fri, 4 Oct 2024 02:48:21 +0100
> > Ferruh Yigit <ferruh.yigit@amd.com> wrote:
> > 
> >> On 9/4/2024 4:42 PM, Stephen Hemminger wrote:
> >>> The TAP device does have per-queue stats and handles multi-process.
> >>>
> >>> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> >>> ---
> >>>  doc/guides/nics/features/tap.ini | 2 ++
> >>>  1 file changed, 2 insertions(+)
> >>>
> >>> diff --git a/doc/guides/nics/features/tap.ini b/doc/guides/nics/features/tap.ini
> >>> index f26355e57f..f2ea5cd833 100644
> >>> --- a/doc/guides/nics/features/tap.ini
> >>> +++ b/doc/guides/nics/features/tap.ini
> >>> @@ -14,10 +14,12 @@ Basic stats          = Y
> >>>  L3 checksum offload  = Y
> >>>  L4 checksum offload  = Y
> >>>  MTU update           = Y
> >>> +Multiprocess aware   = Y
> >>>  
> >>
> >> ack
> >>
> >>>  Multicast MAC filter = Y
> >>>  Unicast MAC filter   = Y
> >>>  Packet type parsing  = Y
> >>>  Flow control         = Y
> >>> +Stats per queue      = Y
> >>>  
> >>
> >> This feature name is misleading,
> >> it is for 'rte_eth_dev_set_[rt]x_queue_stats_mapping()' API, which is
> >> indeed for covering limitation for some drivers.
> >> Tap does support getting stats per queue, but doesn't support above
> >> documented feature.
> > 
> > The stats queue mapping was a feature that was hinted at being removed.
> > It only exists because of HW limitations on Intel ixgbe NIC and SW
> > limitations from RTE_ETHDEV_QUEUE_STAT_CNTRS.
> > 
> 
> 
> We have a plan to remove 'RTE_ETHDEV_QUEUE_STAT_CNTRS', by moving queue
> stats to xstats.
> 
> But ixgbe limitation is there.
> 
> > Perhaps there should be a generic SW emulation for this the mapping?
> >
> 
> Ack, cc'ed Bruce.
> But I am not sure ROI of the effort at this stage.

Not sure what the specific ask for me is here. :-) Overall, I think moving
queue stats to xstats is the best way to go.

/Bruce
  
Stephen Hemminger Oct. 4, 2024, 3:24 p.m. UTC | #5
On Fri, 4 Oct 2024 08:54:43 +0100
Bruce Richardson <bruce.richardson@intel.com> wrote:

> On Fri, Oct 04, 2024 at 05:09:21AM +0100, Ferruh Yigit wrote:
> > On 10/4/2024 3:26 AM, Stephen Hemminger wrote:  
> > > On Fri, 4 Oct 2024 02:48:21 +0100
> > > Ferruh Yigit <ferruh.yigit@amd.com> wrote:
> > >   
> > >> On 9/4/2024 4:42 PM, Stephen Hemminger wrote:  
> > >>> The TAP device does have per-queue stats and handles multi-process.
> > >>>
> > >>> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> > >>> ---
> > >>>  doc/guides/nics/features/tap.ini | 2 ++
> > >>>  1 file changed, 2 insertions(+)
> > >>>
> > >>> diff --git a/doc/guides/nics/features/tap.ini b/doc/guides/nics/features/tap.ini
> > >>> index f26355e57f..f2ea5cd833 100644
> > >>> --- a/doc/guides/nics/features/tap.ini
> > >>> +++ b/doc/guides/nics/features/tap.ini
> > >>> @@ -14,10 +14,12 @@ Basic stats          = Y
> > >>>  L3 checksum offload  = Y
> > >>>  L4 checksum offload  = Y
> > >>>  MTU update           = Y
> > >>> +Multiprocess aware   = Y
> > >>>    
> > >>
> > >> ack
> > >>  
> > >>>  Multicast MAC filter = Y
> > >>>  Unicast MAC filter   = Y
> > >>>  Packet type parsing  = Y
> > >>>  Flow control         = Y
> > >>> +Stats per queue      = Y
> > >>>    
> > >>
> > >> This feature name is misleading,
> > >> it is for 'rte_eth_dev_set_[rt]x_queue_stats_mapping()' API, which is
> > >> indeed for covering limitation for some drivers.
> > >> Tap does support getting stats per queue, but doesn't support above
> > >> documented feature.  
> > > 
> > > The stats queue mapping was a feature that was hinted at being removed.
> > > It only exists because of HW limitations on Intel ixgbe NIC and SW
> > > limitations from RTE_ETHDEV_QUEUE_STAT_CNTRS.
> > >   
> > 
> > 
> > We have a plan to remove 'RTE_ETHDEV_QUEUE_STAT_CNTRS', by moving queue
> > stats to xstats.
> > 
> > But ixgbe limitation is there.
> >   
> > > Perhaps there should be a generic SW emulation for this the mapping?
> > >  
> > 
> > Ack, cc'ed Bruce.
> > But I am not sure ROI of the effort at this stage.  
> 
> Not sure what the specific ask for me is here. :-) Overall, I think moving
> queue stats to xstats is the best way to go.
> 
> /Bruce

The problem is that xstats are free form, so it is not possible for application
to use them in device independent manner.
  
Ferruh Yigit Oct. 4, 2024, 5:26 p.m. UTC | #6
On 10/4/2024 8:54 AM, Bruce Richardson wrote:
> On Fri, Oct 04, 2024 at 05:09:21AM +0100, Ferruh Yigit wrote:
>> On 10/4/2024 3:26 AM, Stephen Hemminger wrote:
>>> On Fri, 4 Oct 2024 02:48:21 +0100
>>> Ferruh Yigit <ferruh.yigit@amd.com> wrote:
>>>
>>>> On 9/4/2024 4:42 PM, Stephen Hemminger wrote:
>>>>> The TAP device does have per-queue stats and handles multi-process.
>>>>>
>>>>> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
>>>>> ---
>>>>>  doc/guides/nics/features/tap.ini | 2 ++
>>>>>  1 file changed, 2 insertions(+)
>>>>>
>>>>> diff --git a/doc/guides/nics/features/tap.ini b/doc/guides/nics/features/tap.ini
>>>>> index f26355e57f..f2ea5cd833 100644
>>>>> --- a/doc/guides/nics/features/tap.ini
>>>>> +++ b/doc/guides/nics/features/tap.ini
>>>>> @@ -14,10 +14,12 @@ Basic stats          = Y
>>>>>  L3 checksum offload  = Y
>>>>>  L4 checksum offload  = Y
>>>>>  MTU update           = Y
>>>>> +Multiprocess aware   = Y
>>>>>  
>>>>
>>>> ack
>>>>
>>>>>  Multicast MAC filter = Y
>>>>>  Unicast MAC filter   = Y
>>>>>  Packet type parsing  = Y
>>>>>  Flow control         = Y
>>>>> +Stats per queue      = Y
>>>>>  
>>>>
>>>> This feature name is misleading,
>>>> it is for 'rte_eth_dev_set_[rt]x_queue_stats_mapping()' API, which is
>>>> indeed for covering limitation for some drivers.
>>>> Tap does support getting stats per queue, but doesn't support above
>>>> documented feature.
>>>
>>> The stats queue mapping was a feature that was hinted at being removed.
>>> It only exists because of HW limitations on Intel ixgbe NIC and SW
>>> limitations from RTE_ETHDEV_QUEUE_STAT_CNTRS.
>>>
>>
>>
>> We have a plan to remove 'RTE_ETHDEV_QUEUE_STAT_CNTRS', by moving queue
>> stats to xstats.
>>
>> But ixgbe limitation is there.
>>
>>> Perhaps there should be a generic SW emulation for this the mapping?
>>>
>>
>> Ack, cc'ed Bruce.
>> But I am not sure ROI of the effort at this stage.
> 
> Not sure what the specific ask for me is here. :-) Overall, I think moving
> queue stats to xstats is the best way to go.
> 

cc'ed because of "generic SW emulation" comment.

I was thinking if this mapping can be done transparent to the user by
driver mapping queue <-> stats_register before reading stats, but @Bruce
let me know this won't work because the stats tracking only happens
after the mapping.

@Stephen, do you have something specific in your mind for SW emulation
for mapping?
  
Stephen Hemminger Oct. 4, 2024, 8:40 p.m. UTC | #7
...
> >>> The stats queue mapping was a feature that was hinted at being removed.
> >>> It only exists because of HW limitations on Intel ixgbe NIC and SW
> >>> limitations from RTE_ETHDEV_QUEUE_STAT_CNTRS.
> >>>  
> >>
> >>
> >> We have a plan to remove 'RTE_ETHDEV_QUEUE_STAT_CNTRS', by moving queue
> >> stats to xstats.
> >>
> >> But ixgbe limitation is there.
> >>  
> >>> Perhaps there should be a generic SW emulation for this the mapping?
> >>>  
> >>
> >> Ack, cc'ed Bruce.
> >> But I am not sure ROI of the effort at this stage.  
> > 
> > Not sure what the specific ask for me is here. :-) Overall, I think moving
> > queue stats to xstats is the best way to go.
> >   
> 
> cc'ed because of "generic SW emulation" comment.
> 
> I was thinking if this mapping can be done transparent to the user by
> driver mapping queue <-> stats_register before reading stats, but @Bruce
> let me know this won't work because the stats tracking only happens
> after the mapping.
> 
> @Stephen, do you have something specific in your mind for SW emulation
> for mapping?

Probably not worth doing mapping tables for SW drivers.

My preference would be something like:
	1. Introduce new API's rte_ethdev_get_queue_stats() and rte_ethdev_reset_queue_stats().
           with callbacks in ethdev ops.
           if driver does not support this, then do a shim in ethdev that uses old stats fields.
        2. Mark the stats mapping API's as deprecated and always return error.
           Making special case for ixgbe at this point really is not worth maintaining.
        3. After all drivers converted over, drop the fields in ethdev stats for queues.
           And fix the examples, etc. Yes this would be a big API change.

Would take a year to get done (ie 25.11).
  
Stephen Hemminger Oct. 4, 2024, 8:48 p.m. UTC | #8
On Fri, 4 Oct 2024 02:48:21 +0100
Ferruh Yigit <ferruh.yigit@amd.com> wrote:

> > +Stats per queue      = Y
> >  
> 
> This feature name is misleading,
> it is for 'rte_eth_dev_set_[rt]x_queue_stats_mapping()' API, which is
> indeed for covering limitation for some drivers.
> Tap does support getting stats per queue, but doesn't support above
> documented feature.

Just looked and the current state of the feature matrix is not correct
for this part of the table. Although the feature intro says it is
stats per queue mapping, that is not what other drivers are doing.

Only these drivers have queue_stats_mapping_set:
  cnxk, enic, igc, ixgbe, txgbe, virtio

These drivers claim to have 'Stats per queue' but do not have stats mapping:
  af_xdp, ark, atlantic, avp, bnxt, cxgbe, failsafe, fm10, hinic, hns3,
  ionic, mlx4, mlx5, netvsc, nfb, nfp, ngbe, qede, thunderx, vmxnet3

So the overwhelming practice is that the features.rst description is wrong!
  
Ferruh Yigit Oct. 6, 2024, 8:36 p.m. UTC | #9
On 10/4/2024 9:48 PM, Stephen Hemminger wrote:
> On Fri, 4 Oct 2024 02:48:21 +0100
> Ferruh Yigit <ferruh.yigit@amd.com> wrote:
> 
>>> +Stats per queue      = Y
>>>  
>>
>> This feature name is misleading,
>> it is for 'rte_eth_dev_set_[rt]x_queue_stats_mapping()' API, which is
>> indeed for covering limitation for some drivers.
>> Tap does support getting stats per queue, but doesn't support above
>> documented feature.
> 
> Just looked and the current state of the feature matrix is not correct
> for this part of the table. Although the feature intro says it is
> stats per queue mapping, that is not what other drivers are doing.
> 
> Only these drivers have queue_stats_mapping_set:
>   cnxk, enic, igc, ixgbe, txgbe, virtio
> 
> These drivers claim to have 'Stats per queue' but do not have stats mapping:
>   af_xdp, ark, atlantic, avp, bnxt, cxgbe, failsafe, fm10, hinic, hns3,
>   ionic, mlx4, mlx5, netvsc, nfb, nfp, ngbe, qede, thunderx, vmxnet3
> 
> So the overwhelming practice is that the features.rst description is wrong!
>

You are right.

What about update the 'features.rst', and extend the definition of
'Stats per queue' mapping + queue stats in basic stats or extended stats.
This will be make name of the feature more intuitive.
  
Ferruh Yigit Oct. 6, 2024, 8:51 p.m. UTC | #10
On 10/4/2024 9:40 PM, Stephen Hemminger wrote:
> ...
>>>>> The stats queue mapping was a feature that was hinted at being removed.
>>>>> It only exists because of HW limitations on Intel ixgbe NIC and SW
>>>>> limitations from RTE_ETHDEV_QUEUE_STAT_CNTRS.
>>>>>  
>>>>
>>>>
>>>> We have a plan to remove 'RTE_ETHDEV_QUEUE_STAT_CNTRS', by moving queue
>>>> stats to xstats.
>>>>
>>>> But ixgbe limitation is there.
>>>>  
>>>>> Perhaps there should be a generic SW emulation for this the mapping?
>>>>>  
>>>>
>>>> Ack, cc'ed Bruce.
>>>> But I am not sure ROI of the effort at this stage.  
>>>
>>> Not sure what the specific ask for me is here. :-) Overall, I think moving
>>> queue stats to xstats is the best way to go.
>>>   
>>
>> cc'ed because of "generic SW emulation" comment.
>>
>> I was thinking if this mapping can be done transparent to the user by
>> driver mapping queue <-> stats_register before reading stats, but @Bruce
>> let me know this won't work because the stats tracking only happens
>> after the mapping.
>>
>> @Stephen, do you have something specific in your mind for SW emulation
>> for mapping?
> 
> Probably not worth doing mapping tables for SW drivers.
> 
> My preference would be something like:
> 	1. Introduce new API's rte_ethdev_get_queue_stats() and rte_ethdev_reset_queue_stats().
>            with callbacks in ethdev ops.
>            if driver does not support this, then do a shim in ethdev that uses old stats fields.
>         2. Mark the stats mapping API's as deprecated and always return error.
>            Making special case for ixgbe at this point really is not worth maintaining.
>         3. After all drivers converted over, drop the fields in ethdev stats for queues.
>            And fix the examples, etc. Yes this would be a big API change.
> 
> Would take a year to get done (ie 25.11).
>

The problem with existing a API, 'rte_eth_stats_get()', is it has queue
stats fixed sized array which limits us also for some cases causes
unnecessary memory consumption.

Our solution for now is move queue stats to xstats, as it is flexible
this solves the problem, but as you said in other thread xstats is free
form and may be hard for the application to use it.

One option is we can add some more data to the xstats, to help
application to consume it easier, like number of queues just before the
queue stats and define queue stats formatting syntax.

But if we will go with API option, instead of adding new queue stats
('rte_ethdev_get_queue_stats()'), we can update the existing
'rte_eth_stats_get()' to return dynamic number of queue stats.

And we need to find someone to own this task.


Above is for queue stats, it does not solve the mapping issue, as we
need it for a few drivers and we already have the APIs, I am for just
keeping them as it is, they are not causing any burden to us.
  
Morten Brørup Oct. 8, 2024, 9:58 a.m. UTC | #11
> From: Ferruh Yigit [mailto:ferruh.yigit@amd.com]
> Sent: Sunday, 6 October 2024 22.51
> 
> On 10/4/2024 9:40 PM, Stephen Hemminger wrote:
> > ...
> >>>>> The stats queue mapping was a feature that was hinted at being
> removed.
> >>>>> It only exists because of HW limitations on Intel ixgbe NIC and
> SW
> >>>>> limitations from RTE_ETHDEV_QUEUE_STAT_CNTRS.
> >>>>>
> >>>>
> >>>>
> >>>> We have a plan to remove 'RTE_ETHDEV_QUEUE_STAT_CNTRS', by moving
> queue
> >>>> stats to xstats.
> >>>>
> >>>> But ixgbe limitation is there.
> >>>>
> >>>>> Perhaps there should be a generic SW emulation for this the
> mapping?
> >>>>>
> >>>>
> >>>> Ack, cc'ed Bruce.
> >>>> But I am not sure ROI of the effort at this stage.
> >>>
> >>> Not sure what the specific ask for me is here. :-) Overall, I think
> moving
> >>> queue stats to xstats is the best way to go.
> >>>
> >>
> >> cc'ed because of "generic SW emulation" comment.
> >>
> >> I was thinking if this mapping can be done transparent to the user
> by
> >> driver mapping queue <-> stats_register before reading stats, but
> @Bruce
> >> let me know this won't work because the stats tracking only happens
> >> after the mapping.
> >>
> >> @Stephen, do you have something specific in your mind for SW
> emulation
> >> for mapping?
> >
> > Probably not worth doing mapping tables for SW drivers.
> >
> > My preference would be something like:
> > 	1. Introduce new API's rte_ethdev_get_queue_stats() and
> rte_ethdev_reset_queue_stats().
> >            with callbacks in ethdev ops.
> >            if driver does not support this, then do a shim in ethdev
> that uses old stats fields.
> >         2. Mark the stats mapping API's as deprecated and always
> return error.
> >            Making special case for ixgbe at this point really is not
> worth maintaining.
> >         3. After all drivers converted over, drop the fields in
> ethdev stats for queues.
> >            And fix the examples, etc. Yes this would be a big API
> change.
> >
> > Would take a year to get done (ie 25.11).
> >
> 
> The problem with existing a API, 'rte_eth_stats_get()', is it has queue
> stats fixed sized array which limits us also for some cases causes
> unnecessary memory consumption.
> 
> Our solution for now is move queue stats to xstats, as it is flexible
> this solves the problem, but as you said in other thread xstats is free
> form and may be hard for the application to use it.
> 
> One option is we can add some more data to the xstats, to help
> application to consume it easier, like number of queues just before the
> queue stats and define queue stats formatting syntax.
> 
> But if we will go with API option, instead of adding new queue stats
> ('rte_ethdev_get_queue_stats()'), we can update the existing
> 'rte_eth_stats_get()' to return dynamic number of queue stats.

Some users might not care about queue stats, so let's remove them entirely from rte_eth_stats_get(), and put them elsewhere.

> 
> And we need to find someone to own this task.
> 
> 
> Above is for queue stats, it does not solve the mapping issue, as we
> need it for a few drivers and we already have the APIs, I am for just
> keeping them as it is, they are not causing any burden to us.
  
Bruce Richardson Oct. 8, 2024, 10:04 a.m. UTC | #12
On Tue, Oct 08, 2024 at 11:58:37AM +0200, Morten Brørup wrote:
> > From: Ferruh Yigit [mailto:ferruh.yigit@amd.com]
> > Sent: Sunday, 6 October 2024 22.51
> > 
> > On 10/4/2024 9:40 PM, Stephen Hemminger wrote:
> > > ...
> > >>>>> The stats queue mapping was a feature that was hinted at being
> > removed.
> > >>>>> It only exists because of HW limitations on Intel ixgbe NIC and
> > SW
> > >>>>> limitations from RTE_ETHDEV_QUEUE_STAT_CNTRS.
> > >>>>>
> > >>>>
> > >>>>
> > >>>> We have a plan to remove 'RTE_ETHDEV_QUEUE_STAT_CNTRS', by moving
> > queue
> > >>>> stats to xstats.
> > >>>>
> > >>>> But ixgbe limitation is there.
> > >>>>
> > >>>>> Perhaps there should be a generic SW emulation for this the
> > mapping?
> > >>>>>
> > >>>>
> > >>>> Ack, cc'ed Bruce.
> > >>>> But I am not sure ROI of the effort at this stage.
> > >>>
> > >>> Not sure what the specific ask for me is here. :-) Overall, I think
> > moving
> > >>> queue stats to xstats is the best way to go.
> > >>>
> > >>
> > >> cc'ed because of "generic SW emulation" comment.
> > >>
> > >> I was thinking if this mapping can be done transparent to the user
> > by
> > >> driver mapping queue <-> stats_register before reading stats, but
> > @Bruce
> > >> let me know this won't work because the stats tracking only happens
> > >> after the mapping.
> > >>
> > >> @Stephen, do you have something specific in your mind for SW
> > emulation
> > >> for mapping?
> > >
> > > Probably not worth doing mapping tables for SW drivers.
> > >
> > > My preference would be something like:
> > > 	1. Introduce new API's rte_ethdev_get_queue_stats() and
> > rte_ethdev_reset_queue_stats().
> > >            with callbacks in ethdev ops.
> > >            if driver does not support this, then do a shim in ethdev
> > that uses old stats fields.
> > >         2. Mark the stats mapping API's as deprecated and always
> > return error.
> > >            Making special case for ixgbe at this point really is not
> > worth maintaining.
> > >         3. After all drivers converted over, drop the fields in
> > ethdev stats for queues.
> > >            And fix the examples, etc. Yes this would be a big API
> > change.
> > >
> > > Would take a year to get done (ie 25.11).
> > >
> > 
> > The problem with existing a API, 'rte_eth_stats_get()', is it has queue
> > stats fixed sized array which limits us also for some cases causes
> > unnecessary memory consumption.
> > 
> > Our solution for now is move queue stats to xstats, as it is flexible
> > this solves the problem, but as you said in other thread xstats is free
> > form and may be hard for the application to use it.
> > 
> > One option is we can add some more data to the xstats, to help
> > application to consume it easier, like number of queues just before the
> > queue stats and define queue stats formatting syntax.
> > 
> > But if we will go with API option, instead of adding new queue stats
> > ('rte_ethdev_get_queue_stats()'), we can update the existing
> > 'rte_eth_stats_get()' to return dynamic number of queue stats.
> 
> Some users might not care about queue stats, so let's remove them entirely from rte_eth_stats_get(), and put them elsewhere.
> 
+1 for this. I think separate APIs for queue stats would make more sense.

/Bruce
  

Patch

diff --git a/doc/guides/nics/features/tap.ini b/doc/guides/nics/features/tap.ini
index f26355e57f..f2ea5cd833 100644
--- a/doc/guides/nics/features/tap.ini
+++ b/doc/guides/nics/features/tap.ini
@@ -14,10 +14,12 @@  Basic stats          = Y
 L3 checksum offload  = Y
 L4 checksum offload  = Y
 MTU update           = Y
+Multiprocess aware   = Y
 Multicast MAC filter = Y
 Unicast MAC filter   = Y
 Packet type parsing  = Y
 Flow control         = Y
+Stats per queue      = Y
 Linux                = Y
 ARMv7                = Y
 ARMv8                = Y