[v6,0/2] Add l2reflect measurement application

  Dear DPDK community,

this patch provides the l2reflect measurement tool
which will be discussed in our 2022 DPDK Userspace Summit talk:
"Using DPDK OVS for deterministic low latency communication"

While the code still might need some polish, we believe it is
a good starting point for discussions about low latency networking
in DPDK.

The tool can also be used in a CI environment to contineously
measure latencies across the evolution of DPDK and Linux.

Best regards,
Felix Moessbauer
Siemens AG

Changes since v5:

- rebased against current main
- updated API to DPKD 22.x
- use RTE tracing
- rework handling of mempool
- fixes around configurable packet size
- backportet changes from out-of-tree version

Changes since v4:

All mentioned points from Thomas Monjalon's review are adressed.
This includes:

- remove obsolete makefile
- remove authors from headers
- include sbdirs in alphabetical order
- use rte functions to compare mac addrs
- use rte functions to format mac addrs
- use jansson instead of cjson for json creation
- print histogram to stderr to decouple from TUI
- add option to disable color (autodisable if redirected)
- improve documentation on how to use the tool
- improve inline documentation (mainly l2reflect_rx_filter)

This patch is still targeted towards dpdk/main until the
final decision is made to put it in DTS.
Further, currently only linux is supported due to the RT tuning
stuff and a missing windows testing environment on our side.
We would be happy about contributions to port that to other
platforms as well.

Changes since v3:

- check for sys/io.h header
- fix linking issues on gcc 10

Changes since v2:

- add missing internal dependency
- improve wording of commit message

Changes since v1:

- move to app folder, as suggested by maintainer
- fix issues reported by checkpatch



Felix Moessbauer (2):
  Fix build of apps with external dependencies
  Add l2reflect measurement application

 app/l2reflect/colors.c    |   34 ++
 app/l2reflect/colors.h    |   19 +
 app/l2reflect/l2reflect.h |   53 ++
 app/l2reflect/main.c      | 1007 +++++++++++++++++++++++++++++++++++++
 app/l2reflect/meson.build |   21 +
 app/l2reflect/payload.h   |   26 +
 app/l2reflect/stats.c     |  225 +++++++++
 app/l2reflect/stats.h     |   67 +++
 app/l2reflect/utils.c     |   67 +++
 app/l2reflect/utils.h     |   20 +
 app/meson.build           |    3 +-
 11 files changed, 1541 insertions(+), 1 deletion(-)
 create mode 100644 app/l2reflect/colors.c
 create mode 100644 app/l2reflect/colors.h
 create mode 100644 app/l2reflect/l2reflect.h
 create mode 100644 app/l2reflect/main.c
 create mode 100644 app/l2reflect/meson.build
 create mode 100644 app/l2reflect/payload.h
 create mode 100644 app/l2reflect/stats.c
 create mode 100644 app/l2reflect/stats.h
 create mode 100644 app/l2reflect/utils.c
 create mode 100644 app/l2reflect/utils.h
  

> From: Felix Moessbauer [mailto:felix.moessbauer@siemens.com]
> Sent: Friday, 2 September 2022 10.46
> 
> Dear DPDK community,
> 
> this patch provides the l2reflect measurement tool
> which will be discussed in our 2022 DPDK Userspace Summit talk:
> "Using DPDK OVS for deterministic low latency communication"
> 
> While the code still might need some polish, we believe it is
> a good starting point for discussions about low latency networking
> in DPDK.
> 
> The tool can also be used in a CI environment to contineously
> measure latencies across the evolution of DPDK and Linux.
> 
> Best regards,
> Felix Moessbauer
> Siemens AG

Dear Felix and Henning,

Great to meet you at the 2022 DPDK Userspace conference.

Have you considered using the Configuration Testing Protocol (CTP), described in chapter 8 of the Ethernet specification from 1984 [1], instead of your own packet format and the Local Experimental Ethertype?

[1]: http://decnet.ipv7.net/docs/dundas/aa-k759b-tk.pdf

The CTP an obsolete protocol, and not part of the IEEE standards for Ethernet, but I think Wireshark is able to parse such packets.

There might also be long term perspectives in building on top of a published (although obsolete) standard - e.g. the protocol could be revived and become part of the DPDK protocol library, along with LACP.

On the other hand, it might limit your ability to expand the protocol.


Med venlig hilsen / Kind regards,
-Morten Brørup
  

Am Wed, 21 Sep 2022 11:43:13 +0200
schrieb Morten Brørup <mb@smartsharesystems.com>:

> > From: Felix Moessbauer [mailto:felix.moessbauer@siemens.com]
> > Sent: Friday, 2 September 2022 10.46
> > 
> > Dear DPDK community,
> > 
> > this patch provides the l2reflect measurement tool
> > which will be discussed in our 2022 DPDK Userspace Summit talk:
> > "Using DPDK OVS for deterministic low latency communication"
> > 
> > While the code still might need some polish, we believe it is
> > a good starting point for discussions about low latency networking
> > in DPDK.
> > 
> > The tool can also be used in a CI environment to contineously
> > measure latencies across the evolution of DPDK and Linux.
> > 
> > Best regards,
> > Felix Moessbauer
> > Siemens AG  
> 
> Dear Felix and Henning,
> 
> Great to meet you at the 2022 DPDK Userspace conference.
> 
> Have you considered using the Configuration Testing Protocol (CTP),
> described in chapter 8 of the Ethernet specification from 1984 [1],
> instead of your own packet format and the Local Experimental
> Ethertype?

No we have not, first time i hear about that. First type we used must
have been 0xaffe or 0xdead, would have to dig through version control.

> [1]: http://decnet.ipv7.net/docs/dundas/aa-k759b-tk.pdf
> 
> The CTP an obsolete protocol, and not part of the IEEE standards for
> Ethernet, but I think Wireshark is able to parse such packets.

Yes ... does not seem to be a train one wants to hop on.

Maybe you can explain how one would use CTP to measure roundtrip times,
and go into detail on how that would add value.

I had a quick look at the spec and did not clearly see whether the
protocol could be used at all ... maybe "abused". And being a CTP
server one would need to implement more than just "reply". And i do not
see any value, except maybe "wireshark support" ... but i am not sure
how that would add value. The packets we send are trivial, headers are
ethernet and the content does not matter ... so wireshark support is
there for all relevant fields.

regards,
Henning

> There might also be long term perspectives in building on top of a
> published (although obsolete) standard - e.g. the protocol could be
> revived and become part of the DPDK protocol library, along with LACP.
>
> On the other hand, it might limit your ability to expand the protocol.
> 
> 
> Med venlig hilsen / Kind regards,
> -Morten Brørup
> 
> 
>
  

Am Wed, 21 Sep 2022 14:22:07 +0200
schrieb Morten Brørup <mb@smartsharesystems.com>:

> > From: Henning Schild [mailto:henning.schild@siemens.com]
> > Sent: Wednesday, 21 September 2022 13.27
> > 
> > Am Wed, 21 Sep 2022 11:43:13 +0200
> > schrieb Morten Brørup <mb@smartsharesystems.com>:
> >   
> > > > From: Felix Moessbauer [mailto:felix.moessbauer@siemens.com]
> > > > Sent: Friday, 2 September 2022 10.46
> > > >
> > > > Dear DPDK community,
> > > >
> > > > this patch provides the l2reflect measurement tool
> > > > which will be discussed in our 2022 DPDK Userspace Summit talk:
> > > > "Using DPDK OVS for deterministic low latency communication"
> > > >
> > > > While the code still might need some polish, we believe it is
> > > > a good starting point for discussions about low latency
> > > > networking in DPDK.
> > > >
> > > > The tool can also be used in a CI environment to contineously
> > > > measure latencies across the evolution of DPDK and Linux.
> > > >
> > > > Best regards,
> > > > Felix Moessbauer
> > > > Siemens AG  
> > >
> > > Dear Felix and Henning,
> > >
> > > Great to meet you at the 2022 DPDK Userspace conference.
> > >
> > > Have you considered using the Configuration Testing Protocol
> > > (CTP), described in chapter 8 of the Ethernet specification from
> > > 1984 [1], instead of your own packet format and the Local
> > > Experimental Ethertype?  
> > 
> > No we have not, first time i hear about that. First type we used
> > must have been 0xaffe or 0xdead, would have to dig through version
> > control.  
> 
> You seem to be using the correct EtherType for an experimental
> protocol like this. I was not opposing to that.
> 
> >   
> > > [1]: http://decnet.ipv7.net/docs/dundas/aa-k759b-tk.pdf
> > >
> > > The CTP an obsolete protocol, and not part of the IEEE standards
> > > for Ethernet, but I think Wireshark is able to parse such
> > > packets.  
> > 
> > Yes ... does not seem to be a train one wants to hop on.
> > 
> > Maybe you can explain how one would use CTP to measure roundtrip
> > times, and go into detail on how that would add value.  
> 
> I would only change the packet format, not the way of measuring.
> 
> > 
> > I had a quick look at the spec and did not clearly see whether the
> > protocol could be used at all ... maybe "abused". And being a CTP
> > server one would need to implement more than just "reply". And i do
> > not see any value, except maybe "wireshark support" ... but i am
> > not sure how that would add value. The packets we send are trivial,
> > headers are ethernet and the content does not matter ... so
> > wireshark support is there for all relevant fields.  
> 
> The primary - and probably only - advantage would be that the
> EtherType 0x9000 is officially allocated for CTP, so you don't need
> to use one of the EtherTypes allocated for experimental purposes,
> which might also be used for other purposes.

Yes that is something i thought about as well. In case there would be
some sort of conflict with 88B5 i would rather include some sort of
"magic start" or sub-protocol id if you want.
But until we see such a conflict i would simply stick with 88B5 which
seems to be a good fit, but not "clearly standardized". I would not
envision people running l2reflect on a very busy/big network, as the
results would become increasingly fuzzy and meaningless and other
latency benchmarks would probably be a better fit.

Any confusion with another 88B5 application is "highly unlikely". And
when jumping on 9000 we switch from one "unlikely number" to another
one ... or in fact we are now on one that is "more likely"? From what i
read cisco equipment might start acting on those 9000 packets.

I think a "highly unlikely" conflict does not justify a rewrite. For a
not well defined protocol it would not be a "conflict" really. While
9000 would be "more likely" and dictate a corset to try and fit in.
Even if we can make it fit today, future extensions might not work out.

So i think i would stay away from CTP.

Henning

> The CTP packet format is also trivial:
> 
> The "Request" packet basically contains a SkipCount=0 field followed
> a sequence of 1) a "FORWARD" structure with the MAC address of the
> Request sender, and 2) a "REPLY" structure containing only a receipt
> number (which I guess is like the ID or Sequence Number of an ICMP
> Echo packet).
> 
> The "Reply" packet is the same, but the SkipCount is increased by the
> size of the "FORWARD" structure (i.e. 8 bytes), so the SkipCount
> field now points to the "REPLY" structure.
> 
> 
> Furthermore, the protocol also allows forwarding the packet through
> multiple hops, like MPLS. In that case, the initial packet's sequence
> will contain two or more "FORWARD" structures, and the CTP server on
> each hop will pop the topmost "FORWARD" structure (by updating the
> SkipCount field) and forward the packet to the next hop. However,
> nothing prevents you from also expanding your protocol to also
> multi-hop, if the need for it should arise.
> 
> > 
> > regards,
> > Henning
> >   
> > > There might also be long term perspectives in building on top of a
> > > published (although obsolete) standard - e.g. the protocol could
> > > be revived and become part of the DPDK protocol library, along
> > > with  
> > LACP.  
> > >
> > > On the other hand, it might limit your ability to expand the  
> > protocol.  
> 
> Just wanted to let you know about the existing protocol, so you can
> give it a few thoughts. There are pros and cons.
> 
> I have no personal preferences for CTP vs. your protocol.
> 
> -Morten
>