[00/22] Add DLB2 PMD

Message ID	1599855987-25976-1-git-send-email-timothy.mcdaniel@intel.com (mailing list archive)
Headers	IronPort-SDR: qt7alPauSpHGh0qw1EY7ZzujER6eIz3D7Ne683kLhJuDt/LuT7Jpbfde2GkrKGKfoOZDzef27Q YS0nY/GiqAFw== IronPort-SDR: siFM55SqnINk/bZtOGCTDlwUZO4gqnvqaQuG+Vn1HSrjPpr3Ac+/3dB/xA+1GK23wC2ri2OHxL u75Abxj/5zvA== From: Timothy McDaniel <timothy.mcdaniel@intel.com> To: Cc: dev@dpdk.org, erik.g.carrillo@intel.com, gage.eads@intel.com, harry.van.haaren@intel.com, jerinj@marvell.com Date: Fri, 11 Sep 2020 15:26:05 -0500 Message-Id: <1599855987-25976-1-git-send-email-timothy.mcdaniel@intel.com> Subject: [dpdk-dev] [PATCH 00/22] Add DLB2 PMD Precedence: list Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org>
Series	Add DLB2 PMD \| [00/22] Add DLB2 PMD [01/22] event/dlb2: add meson build infrastructure [02/22] event/dlb2: add dynamic logging [03/22] event/dlb2: add private data structures and constants [04/22] event/dlb2: add definitions shared with LKM or shared code [05/22] event/dlb2: add inline functions [06/22] event/dlb2: add probe [07/22] event/dlb2: add xstats [08/22] event/dlb2: add infos get and configure [09/22] event/dlb2: add queue and port default conf [10/22] event/dlb2: add queue setup [11/22] event/dlb2: add port setup [12/22] event/dlb2: add port link [13/22] event/dlb2: add port unlink and port unlinks in progress [14/22] event/dlb2: add eventdev start [15/22] event/dlb2: add enqueue and its burst variants [16/22] event/dlb2: add dequeue and its burst variants [17/22] event/dlb2: add eventdev stop and close [18/22] event/dlb2: add PMD's token pop public interface [19/22] event/dlb2: add PMD self-tests [20/22] event/dlb2: add queue and port release [21/22] event/dlb2: add timeout ticks entry point [22/22] doc: add new DLB2 eventdev driver to relnotes

Message ID

1599855987-25976-1-git-send-email-timothy.mcdaniel@intel.com (mailing list archive)

Headers

IronPort-SDR: 
 qt7alPauSpHGh0qw1EY7ZzujER6eIz3D7Ne683kLhJuDt/LuT7Jpbfde2GkrKGKfoOZDzef27Q
 YS0nY/GiqAFw==
IronPort-SDR: 
 siFM55SqnINk/bZtOGCTDlwUZO4gqnvqaQuG+Vn1HSrjPpr3Ac+/3dB/xA+1GK23wC2ri2OHxL
 u75Abxj/5zvA==
From: Timothy McDaniel <timothy.mcdaniel@intel.com>
To: 
Cc: dev@dpdk.org, erik.g.carrillo@intel.com, gage.eads@intel.com,
 harry.van.haaren@intel.com, jerinj@marvell.com
Date: Fri, 11 Sep 2020 15:26:05 -0500
Message-Id: <1599855987-25976-1-git-send-email-timothy.mcdaniel@intel.com>
Subject: [dpdk-dev] [PATCH 00/22] Add DLB2 PMD
Precedence: list
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>

Series

Add DLB2 PMD |

Message

Timothy McDaniel Sept. 11, 2020, 8:26 p.m. UTC

  The following patch series adds support for a new eventdev PMD. The DLB2
PMD adds support for the Intel Dynamic Load Balancer 2.0 (DLB2)
hardware.
The DLB2 is a PCIe device that provides load-balanced, prioritized
scheduling of core-to-core communication. The device consists of
queues and arbiters that connect producer and consumer cores, and
implements load-balanced queueing features including:
- Lock-free multi-producer/multi-consumer operation.
- Multiple priority levels for varying traffic types.
- 'Direct' traffic (i.e. multi-producer/single-consumer)
- Simple unordered load-balanced distribution.
- Atomic lock-free load balancing across multiple consumers.
- Queue element reordering feature allowing ordered load-balanced
  distribution.

The DLB2 hardware supports both load balanced and directed ports and
queues. Unlike other eventdev devices already in the repo,  not all
DLB2 ports and queues are equally capable. In particular, directed
ports are limited to a single link, and must be connected to a
directed queue. Additionally, even though LDB ports may link multiple queues,
the number of queues that may be linked is limited by hardware.

While reviewing the code, please be aware that this PMD has full
control over the DLB2 hardware. Intel will be extending the DLB2 PMD
in the future (not as part of this first series) with a mode that we
refer to as the bifurcated PMD. The bifurcated PMD communicates with a
kernel driver to configure the device, ports, and queues, and memory
maps device MMIO so datapath operations occur purely in user-space.
Note that the DLB2 hardware is a successor of the DLB hardware, and
as such is structured similarly, both in terms of code layout and
implementation.

The framework to support both the PF PMD and bifurcated PMD exists in
this patchset, and is why the iface.[ch] layer is present.

Depends-on: patch-77466 ("eventdev: add PCI probe named convenience function")
Depends-on: series-12160 ("Eventdev ABI changes")
Depends-on: patch-77460 ("eal: add umonitor umwait to x86 cpuflags")

Timothy McDaniel (22):
  event/dlb2: add meson build infrastructure
  event/dlb2: add dynamic logging
  event/dlb2: add private data structures and constants
  event/dlb2: add definitions shared with LKM or shared code
  event/dlb2: add inline functions
  event/dlb2: add probe
  event/dlb2: add xstats
  event/dlb2: add infos get and configure
  event/dlb2: add queue and port default conf
  event/dlb2: add queue setup
  event/dlb2: add port setup
  event/dlb2: add port link
  event/dlb2: add port unlink and port unlinks in progress
  event/dlb2: add eventdev start
  event/dlb2: add enqueue and its burst variants
  event/dlb2: add dequeue and its burst variants
  event/dlb2: add eventdev stop and close
  event/dlb2: add PMD's token pop public interface
  event/dlb2: add PMD self-tests
  event/dlb2: add queue and port release
  event/dlb2: add timeout ticks entry point
  doc: add new DLB2 eventdev driver to relnotes

 app/test/test_eventdev.c                          |    9 +
 config/rte_config.h                               |    7 +
 doc/guides/rel_notes/release_20_11.rst            |    5 +
 drivers/event/dlb2/dlb2.c                         | 4046 ++++++++++++++
 drivers/event/dlb2/dlb2_iface.c                   |   88 +
 drivers/event/dlb2/dlb2_iface.h                   |   75 +
 drivers/event/dlb2/dlb2_inline_fns.h              |   85 +
 drivers/event/dlb2/dlb2_log.h                     |   25 +
 drivers/event/dlb2/dlb2_priv.h                    |  619 +++
 drivers/event/dlb2/dlb2_selftest.c                | 1570 ++++++
 drivers/event/dlb2/dlb2_user.h                    |  883 +++
 drivers/event/dlb2/dlb2_xstats.c                  | 1269 +++++
 drivers/event/dlb2/meson.build                    |   16 +
 drivers/event/dlb2/pf/base/dlb2_hw_types.h        |  367 ++
 drivers/event/dlb2/pf/base/dlb2_mbox.h            |  596 ++
 drivers/event/dlb2/pf/base/dlb2_osdep.h           |  248 +
 drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h    |  447 ++
 drivers/event/dlb2/pf/base/dlb2_osdep_list.h      |  131 +
 drivers/event/dlb2/pf/base/dlb2_osdep_types.h     |   31 +
 drivers/event/dlb2/pf/base/dlb2_regs.h            | 2527 +++++++++
 drivers/event/dlb2/pf/base/dlb2_resource.c        | 6023 +++++++++++++++++++++
 drivers/event/dlb2/pf/base/dlb2_resource.h        | 1913 +++++++
 drivers/event/dlb2/pf/dlb2_main.c                 |  692 +++
 drivers/event/dlb2/pf/dlb2_main.h                 |  107 +
 drivers/event/dlb2/pf/dlb2_pf.c                   |  734 +++
 drivers/event/dlb2/rte_pmd_dlb2.c                 |   39 +
 drivers/event/dlb2/rte_pmd_dlb2.h                 |   59 +
 drivers/event/dlb2/rte_pmd_dlb2_event_version.map |    9 +
 drivers/event/meson.build                         |    4 +
 29 files changed, 22624 insertions(+)
 create mode 100644 drivers/event/dlb2/dlb2.c
 create mode 100644 drivers/event/dlb2/dlb2_iface.c
 create mode 100644 drivers/event/dlb2/dlb2_iface.h
 create mode 100644 drivers/event/dlb2/dlb2_inline_fns.h
 create mode 100644 drivers/event/dlb2/dlb2_log.h
 create mode 100644 drivers/event/dlb2/dlb2_priv.h
 create mode 100644 drivers/event/dlb2/dlb2_selftest.c
 create mode 100644 drivers/event/dlb2/dlb2_user.h
 create mode 100644 drivers/event/dlb2/dlb2_xstats.c
 create mode 100644 drivers/event/dlb2/meson.build
 create mode 100644 drivers/event/dlb2/pf/base/dlb2_hw_types.h
 create mode 100644 drivers/event/dlb2/pf/base/dlb2_mbox.h
 create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep.h
 create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h
 create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep_list.h
 create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep_types.h
 create mode 100644 drivers/event/dlb2/pf/base/dlb2_regs.h
 create mode 100644 drivers/event/dlb2/pf/base/dlb2_resource.c
 create mode 100644 drivers/event/dlb2/pf/base/dlb2_resource.h
 create mode 100644 drivers/event/dlb2/pf/dlb2_main.c
 create mode 100644 drivers/event/dlb2/pf/dlb2_main.h
 create mode 100644 drivers/event/dlb2/pf/dlb2_pf.c
 create mode 100644 drivers/event/dlb2/rte_pmd_dlb2.c
 create mode 100644 drivers/event/dlb2/rte_pmd_dlb2.h
 create mode 100644 drivers/event/dlb2/rte_pmd_dlb2_event_version.map

Comments

Jerin Jacob Sept. 21, 2020, 5:11 p.m. UTC | #1

On Sat, Sep 12, 2020 at 2:00 AM Timothy McDaniel
<timothy.mcdaniel@intel.com> wrote:
>
> The following patch series adds support for a new eventdev PMD. The DLB2
> PMD adds support for the Intel Dynamic Load Balancer 2.0 (DLB2)
> hardware.
> The DLB2 is a PCIe device that provides load-balanced, prioritized
> scheduling of core-to-core communication. The device consists of
> queues and arbiters that connect producer and consumer cores, and
> implements load-balanced queueing features including:
> - Lock-free multi-producer/multi-consumer operation.
> - Multiple priority levels for varying traffic types.
> - 'Direct' traffic (i.e. multi-producer/single-consumer)
> - Simple unordered load-balanced distribution.
> - Atomic lock-free load balancing across multiple consumers.
> - Queue element reordering feature allowing ordered load-balanced
>   distribution.
>
> The DLB2 hardware supports both load balanced and directed ports and
> queues. Unlike other eventdev devices already in the repo,  not all
> DLB2 ports and queues are equally capable. In particular, directed
> ports are limited to a single link, and must be connected to a
> directed queue. Additionally, even though LDB ports may link multiple queues,
> the number of queues that may be linked is limited by hardware.
>
> While reviewing the code, please be aware that this PMD has full
> control over the DLB2 hardware. Intel will be extending the DLB2 PMD
> in the future (not as part of this first series) with a mode that we
> refer to as the bifurcated PMD. The bifurcated PMD communicates with a
> kernel driver to configure the device, ports, and queues, and memory
> maps device MMIO so datapath operations occur purely in user-space.
> Note that the DLB2 hardware is a successor of the DLB hardware, and
> as such is structured similarly, both in terms of code layout and
> implementation.
>
> The framework to support both the PF PMD and bifurcated PMD exists in
> this patchset, and is why the iface.[ch] layer is present.
>
> Depends-on: patch-77466 ("eventdev: add PCI probe named convenience function")
> Depends-on: series-12160 ("Eventdev ABI changes")
> Depends-on: patch-77460 ("eal: add umonitor umwait to x86 cpuflags")
>
> Timothy McDaniel (22):


There is two version of driver(DLB and DLB2)[1]
Both are needed?
Is two separate set of HW?

http://patches.dpdk.org/project/dpdk/list/?submitter=826


>   event/dlb2: add meson build infrastructure
>   event/dlb2: add dynamic logging
>   event/dlb2: add private data structures and constants
>   event/dlb2: add definitions shared with LKM or shared code
>   event/dlb2: add inline functions
>   event/dlb2: add probe
>   event/dlb2: add xstats
>   event/dlb2: add infos get and configure
>   event/dlb2: add queue and port default conf
>   event/dlb2: add queue setup
>   event/dlb2: add port setup
>   event/dlb2: add port link
>   event/dlb2: add port unlink and port unlinks in progress
>   event/dlb2: add eventdev start
>   event/dlb2: add enqueue and its burst variants
>   event/dlb2: add dequeue and its burst variants
>   event/dlb2: add eventdev stop and close
>   event/dlb2: add PMD's token pop public interface
>   event/dlb2: add PMD self-tests
>   event/dlb2: add queue and port release
>   event/dlb2: add timeout ticks entry point
>   doc: add new DLB2 eventdev driver to relnotes
>
>  app/test/test_eventdev.c                          |    9 +
>  config/rte_config.h                               |    7 +
>  doc/guides/rel_notes/release_20_11.rst            |    5 +
>  drivers/event/dlb2/dlb2.c                         | 4046 ++++++++++++++
>  drivers/event/dlb2/dlb2_iface.c                   |   88 +
>  drivers/event/dlb2/dlb2_iface.h                   |   75 +
>  drivers/event/dlb2/dlb2_inline_fns.h              |   85 +
>  drivers/event/dlb2/dlb2_log.h                     |   25 +
>  drivers/event/dlb2/dlb2_priv.h                    |  619 +++
>  drivers/event/dlb2/dlb2_selftest.c                | 1570 ++++++
>  drivers/event/dlb2/dlb2_user.h                    |  883 +++
>  drivers/event/dlb2/dlb2_xstats.c                  | 1269 +++++
>  drivers/event/dlb2/meson.build                    |   16 +
>  drivers/event/dlb2/pf/base/dlb2_hw_types.h        |  367 ++
>  drivers/event/dlb2/pf/base/dlb2_mbox.h            |  596 ++
>  drivers/event/dlb2/pf/base/dlb2_osdep.h           |  248 +
>  drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h    |  447 ++
>  drivers/event/dlb2/pf/base/dlb2_osdep_list.h      |  131 +
>  drivers/event/dlb2/pf/base/dlb2_osdep_types.h     |   31 +
>  drivers/event/dlb2/pf/base/dlb2_regs.h            | 2527 +++++++++
>  drivers/event/dlb2/pf/base/dlb2_resource.c        | 6023 +++++++++++++++++++++
>  drivers/event/dlb2/pf/base/dlb2_resource.h        | 1913 +++++++
>  drivers/event/dlb2/pf/dlb2_main.c                 |  692 +++
>  drivers/event/dlb2/pf/dlb2_main.h                 |  107 +
>  drivers/event/dlb2/pf/dlb2_pf.c                   |  734 +++
>  drivers/event/dlb2/rte_pmd_dlb2.c                 |   39 +
>  drivers/event/dlb2/rte_pmd_dlb2.h                 |   59 +
>  drivers/event/dlb2/rte_pmd_dlb2_event_version.map |    9 +
>  drivers/event/meson.build                         |    4 +
>  29 files changed, 22624 insertions(+)
>  create mode 100644 drivers/event/dlb2/dlb2.c
>  create mode 100644 drivers/event/dlb2/dlb2_iface.c
>  create mode 100644 drivers/event/dlb2/dlb2_iface.h
>  create mode 100644 drivers/event/dlb2/dlb2_inline_fns.h
>  create mode 100644 drivers/event/dlb2/dlb2_log.h
>  create mode 100644 drivers/event/dlb2/dlb2_priv.h
>  create mode 100644 drivers/event/dlb2/dlb2_selftest.c
>  create mode 100644 drivers/event/dlb2/dlb2_user.h
>  create mode 100644 drivers/event/dlb2/dlb2_xstats.c
>  create mode 100644 drivers/event/dlb2/meson.build
>  create mode 100644 drivers/event/dlb2/pf/base/dlb2_hw_types.h
>  create mode 100644 drivers/event/dlb2/pf/base/dlb2_mbox.h
>  create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep.h
>  create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h
>  create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep_list.h
>  create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep_types.h
>  create mode 100644 drivers/event/dlb2/pf/base/dlb2_regs.h
>  create mode 100644 drivers/event/dlb2/pf/base/dlb2_resource.c
>  create mode 100644 drivers/event/dlb2/pf/base/dlb2_resource.h
>  create mode 100644 drivers/event/dlb2/pf/dlb2_main.c
>  create mode 100644 drivers/event/dlb2/pf/dlb2_main.h
>  create mode 100644 drivers/event/dlb2/pf/dlb2_pf.c
>  create mode 100644 drivers/event/dlb2/rte_pmd_dlb2.c
>  create mode 100644 drivers/event/dlb2/rte_pmd_dlb2.h
>  create mode 100644 drivers/event/dlb2/rte_pmd_dlb2_event_version.map
>
> --
> 2.6.4
>

Timothy McDaniel Sept. 21, 2020, 5:15 p.m. UTC | #2

> -----Original Message-----
> From: Jerin Jacob <jerinjacobk@gmail.com>
> Sent: Monday, September 21, 2020 12:12 PM
> To: McDaniel, Timothy <timothy.mcdaniel@intel.com>
> Cc: dpdk-dev <dev@dpdk.org>; Carrillo, Erik G <erik.g.carrillo@intel.com>; Eads,
> Gage <gage.eads@intel.com>; Van Haaren, Harry
> <harry.van.haaren@intel.com>; Jerin Jacob <jerinj@marvell.com>
> Subject: Re: [dpdk-dev] [PATCH 00/22] Add DLB2 PMD
> 
> On Sat, Sep 12, 2020 at 2:00 AM Timothy McDaniel
> <timothy.mcdaniel@intel.com> wrote:
> >
> > The following patch series adds support for a new eventdev PMD. The DLB2
> > PMD adds support for the Intel Dynamic Load Balancer 2.0 (DLB2)
> > hardware.
> > The DLB2 is a PCIe device that provides load-balanced, prioritized
> > scheduling of core-to-core communication. The device consists of
> > queues and arbiters that connect producer and consumer cores, and
> > implements load-balanced queueing features including:
> > - Lock-free multi-producer/multi-consumer operation.
> > - Multiple priority levels for varying traffic types.
> > - 'Direct' traffic (i.e. multi-producer/single-consumer)
> > - Simple unordered load-balanced distribution.
> > - Atomic lock-free load balancing across multiple consumers.
> > - Queue element reordering feature allowing ordered load-balanced
> >   distribution.
> >
> > The DLB2 hardware supports both load balanced and directed ports and
> > queues. Unlike other eventdev devices already in the repo,  not all
> > DLB2 ports and queues are equally capable. In particular, directed
> > ports are limited to a single link, and must be connected to a
> > directed queue. Additionally, even though LDB ports may link multiple queues,
> > the number of queues that may be linked is limited by hardware.
> >
> > While reviewing the code, please be aware that this PMD has full
> > control over the DLB2 hardware. Intel will be extending the DLB2 PMD
> > in the future (not as part of this first series) with a mode that we
> > refer to as the bifurcated PMD. The bifurcated PMD communicates with a
> > kernel driver to configure the device, ports, and queues, and memory
> > maps device MMIO so datapath operations occur purely in user-space.
> > Note that the DLB2 hardware is a successor of the DLB hardware, and
> > as such is structured similarly, both in terms of code layout and
> > implementation.
> >
> > The framework to support both the PF PMD and bifurcated PMD exists in
> > this patchset, and is why the iface.[ch] layer is present.
> >
> > Depends-on: patch-77466 ("eventdev: add PCI probe named convenience
> function")
> > Depends-on: series-12160 ("Eventdev ABI changes")
> > Depends-on: patch-77460 ("eal: add umonitor umwait to x86 cpuflags")
> >
> > Timothy McDaniel (22):
> 
> 
> There is two version of driver(DLB and DLB2)[1]
> Both are needed?
> Is two separate set of HW?
> 

Yes, the two PMDs, DLB and DLB2 are both needed.  They support two different hardware devices.

Jerin Jacob Sept. 29, 2020, 6:41 p.m. UTC | #3

On Mon, Sep 21, 2020 at 10:45 PM McDaniel, Timothy
<timothy.mcdaniel@intel.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Jerin Jacob <jerinjacobk@gmail.com>
> > Sent: Monday, September 21, 2020 12:12 PM
> > To: McDaniel, Timothy <timothy.mcdaniel@intel.com>
> > Cc: dpdk-dev <dev@dpdk.org>; Carrillo, Erik G <erik.g.carrillo@intel.com>; Eads,
> > Gage <gage.eads@intel.com>; Van Haaren, Harry
> > <harry.van.haaren@intel.com>; Jerin Jacob <jerinj@marvell.com>
> > Subject: Re: [dpdk-dev] [PATCH 00/22] Add DLB2 PMD
> >
> > On Sat, Sep 12, 2020 at 2:00 AM Timothy McDaniel
> > <timothy.mcdaniel@intel.com> wrote:
> > >
>
> > >
> > > Timothy McDaniel (22):
> >
> >
> > There is two version of driver(DLB and DLB2)[1]
> > Both are needed?
> > Is two separate set of HW?
> >
>
> Yes, the two PMDs, DLB and DLB2 are both needed.  They support two different hardware devices.

There are plenty of build errors. Please see
http://mails.dpdk.org/archives/test-report/2020-September/152143.html

Also, I would like close the eventdev spec first before we merge this
driver as the driver is depended on
http://patches.dpdk.org/patch/77465/



>
>

Timothy McDaniel Sept. 29, 2020, 6:46 p.m. UTC | #4

> -----Original Message-----
> From: Jerin Jacob <jerinjacobk@gmail.com>
> Sent: Tuesday, September 29, 2020 1:41 PM
> To: McDaniel, Timothy <timothy.mcdaniel@intel.com>
> Cc: dpdk-dev <dev@dpdk.org>; Carrillo, Erik G <erik.g.carrillo@intel.com>; Eads,
> Gage <gage.eads@intel.com>; Van Haaren, Harry
> <harry.van.haaren@intel.com>; Jerin Jacob <jerinj@marvell.com>
> Subject: Re: [dpdk-dev] [PATCH 00/22] Add DLB2 PMD
> 
> On Mon, Sep 21, 2020 at 10:45 PM McDaniel, Timothy
> <timothy.mcdaniel@intel.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Jerin Jacob <jerinjacobk@gmail.com>
> > > Sent: Monday, September 21, 2020 12:12 PM
> > > To: McDaniel, Timothy <timothy.mcdaniel@intel.com>
> > > Cc: dpdk-dev <dev@dpdk.org>; Carrillo, Erik G <erik.g.carrillo@intel.com>;
> Eads,
> > > Gage <gage.eads@intel.com>; Van Haaren, Harry
> > > <harry.van.haaren@intel.com>; Jerin Jacob <jerinj@marvell.com>
> > > Subject: Re: [dpdk-dev] [PATCH 00/22] Add DLB2 PMD
> > >
> > > On Sat, Sep 12, 2020 at 2:00 AM Timothy McDaniel
> > > <timothy.mcdaniel@intel.com> wrote:
> > > >
> >
> > > >
> > > > Timothy McDaniel (22):
> > >
> > >
> > > There is two version of driver(DLB and DLB2)[1]
> > > Both are needed?
> > > Is two separate set of HW?
> > >
> >
> > Yes, the two PMDs, DLB and DLB2 are both needed.  They support two
> different hardware devices.
> 
> There are plenty of build errors. Please see
> http://mails.dpdk.org/archives/test-report/2020-September/152143.html
> 
> Also, I would like close the eventdev spec first before we merge this
> driver as the driver is depended on
> http://patches.dpdk.org/patch/77465/
> 
Thank you, Jerin.

I am working on addressing the build breaks and your other review comments. I hope to upload those changes tomorrow,
and that will include the performance comparison that you requested.

Jerin Jacob Sept. 29, 2020, 6:46 p.m. UTC | #5

On Sat, Sep 12, 2020 at 2:00 AM Timothy McDaniel
<timothy.mcdaniel@intel.com> wrote:
>
> The following patch series adds support for a new eventdev PMD. The DLB2
> PMD adds support for the Intel Dynamic Load Balancer 2.0 (DLB2)
> hardware.
> The DLB2 is a PCIe device that provides load-balanced, prioritized
> scheduling of core-to-core communication. The device consists of
> queues and arbiters that connect producer and consumer cores, and
> implements load-balanced queueing features including:
> - Lock-free multi-producer/multi-consumer operation.
> - Multiple priority levels for varying traffic types.
> - 'Direct' traffic (i.e. multi-producer/single-consumer)
> - Simple unordered load-balanced distribution.
> - Atomic lock-free load balancing across multiple consumers.
> - Queue element reordering feature allowing ordered load-balanced
>   distribution.
>
> The DLB2 hardware supports both load balanced and directed ports and
> queues. Unlike other eventdev devices already in the repo,  not all
> DLB2 ports and queues are equally capable. In particular, directed
> ports are limited to a single link, and must be connected to a
> directed queue. Additionally, even though LDB ports may link multiple queues,
> the number of queues that may be linked is limited by hardware.
>
> While reviewing the code, please be aware that this PMD has full
> control over the DLB2 hardware. Intel will be extending the DLB2 PMD
> in the future (not as part of this first series) with a mode that we
> refer to as the bifurcated PMD. The bifurcated PMD communicates with a
> kernel driver to configure the device, ports, and queues, and memory
> maps device MMIO so datapath operations occur purely in user-space.
> Note that the DLB2 hardware is a successor of the DLB hardware, and
> as such is structured similarly, both in terms of code layout and
> implementation.
>
> The framework to support both the PF PMD and bifurcated PMD exists in
> this patchset, and is why the iface.[ch] layer is present.
>
> Depends-on: patch-77466 ("eventdev: add PCI probe named convenience function")
> Depends-on: series-12160 ("Eventdev ABI changes")
> Depends-on: patch-77460 ("eal: add umonitor umwait to x86 cpuflags")

Cc: Thomas and David

There is a duplicate for patch-77460 as
https://patches.dpdk.org/patch/76554/ from different authors,
and both are open comments to be closed.
@McDaniel, Timothy and @Liang Ma , We need to have this eal patch on
master before I merge this
patch series in eventdev-next. Please check.


>

Timothy McDaniel Sept. 30, 2020, 4:10 p.m. UTC | #6

> -----Original Message-----
> From: Jerin Jacob <jerinjacobk@gmail.com>
> Sent: Tuesday, September 29, 2020 1:41 PM
> To: McDaniel, Timothy <timothy.mcdaniel@intel.com>
> Cc: dpdk-dev <dev@dpdk.org>; Carrillo, Erik G <Erik.G.Carrillo@intel.com>;
> Eads, Gage <gage.eads@intel.com>; Van Haaren, Harry
> <harry.van.haaren@intel.com>; Jerin Jacob <jerinj@marvell.com>
> Subject: Re: [dpdk-dev] [PATCH 00/22] Add DLB2 PMD
> 
> On Mon, Sep 21, 2020 at 10:45 PM McDaniel, Timothy
> <timothy.mcdaniel@intel.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Jerin Jacob <jerinjacobk@gmail.com>
> > > Sent: Monday, September 21, 2020 12:12 PM
> > > To: McDaniel, Timothy <timothy.mcdaniel@intel.com>
> > > Cc: dpdk-dev <dev@dpdk.org>; Carrillo, Erik G <erik.g.carrillo@intel.com>;
> Eads,
> > > Gage <gage.eads@intel.com>; Van Haaren, Harry
> > > <harry.van.haaren@intel.com>; Jerin Jacob <jerinj@marvell.com>
> > > Subject: Re: [dpdk-dev] [PATCH 00/22] Add DLB2 PMD
> > >
> > > On Sat, Sep 12, 2020 at 2:00 AM Timothy McDaniel
> > > <timothy.mcdaniel@intel.com> wrote:
> > > >
> >
> > > >
> > > > Timothy McDaniel (22):
> > >
> > >
> > > There is two version of driver(DLB and DLB2)[1]
> > > Both are needed?
> > > Is two separate set of HW?
> > >
> >
> > Yes, the two PMDs, DLB and DLB2 are both needed.  They support two
> different hardware devices.
> 
> There are plenty of build errors. Please see
> http://mails.dpdk.org/archives/test-report/2020-September/152143.html
> 

The build errors appear to be a result of the depends-on patches not having been applied.
Also, note that there is a 2-way dependency on the ABI changes patch and the test/apps patch.
They both depend on each other.

Timothy McDaniel Sept. 30, 2020, 4:14 p.m. UTC | #7

> -----Original Message-----
> From: Jerin Jacob <jerinjacobk@gmail.com>
> Sent: Tuesday, September 29, 2020 1:46 PM
> To: McDaniel, Timothy <timothy.mcdaniel@intel.com>; Thomas Monjalon
> <thomas@monjalon.net>; David Marchand <david.marchand@redhat.com>;
> Ma, Liang J <liang.j.ma@intel.com>
> Cc: dpdk-dev <dev@dpdk.org>; Carrillo, Erik G <Erik.G.Carrillo@intel.com>;
> Eads, Gage <gage.eads@intel.com>; Van Haaren, Harry
> <harry.van.haaren@intel.com>; Jerin Jacob <jerinj@marvell.com>
> Subject: Re: [dpdk-dev] [PATCH 00/22] Add DLB2 PMD
> 
> On Sat, Sep 12, 2020 at 2:00 AM Timothy McDaniel
> <timothy.mcdaniel@intel.com> wrote:
> >
> > The following patch series adds support for a new eventdev PMD. The DLB2
> > PMD adds support for the Intel Dynamic Load Balancer 2.0 (DLB2)
> > hardware.
> > The DLB2 is a PCIe device that provides load-balanced, prioritized
> > scheduling of core-to-core communication. The device consists of
> > queues and arbiters that connect producer and consumer cores, and
> > implements load-balanced queueing features including:
> > - Lock-free multi-producer/multi-consumer operation.
> > - Multiple priority levels for varying traffic types.
> > - 'Direct' traffic (i.e. multi-producer/single-consumer)
> > - Simple unordered load-balanced distribution.
> > - Atomic lock-free load balancing across multiple consumers.
> > - Queue element reordering feature allowing ordered load-balanced
> >   distribution.
> >
> > The DLB2 hardware supports both load balanced and directed ports and
> > queues. Unlike other eventdev devices already in the repo,  not all
> > DLB2 ports and queues are equally capable. In particular, directed
> > ports are limited to a single link, and must be connected to a
> > directed queue. Additionally, even though LDB ports may link multiple queues,
> > the number of queues that may be linked is limited by hardware.
> >
> > While reviewing the code, please be aware that this PMD has full
> > control over the DLB2 hardware. Intel will be extending the DLB2 PMD
> > in the future (not as part of this first series) with a mode that we
> > refer to as the bifurcated PMD. The bifurcated PMD communicates with a
> > kernel driver to configure the device, ports, and queues, and memory
> > maps device MMIO so datapath operations occur purely in user-space.
> > Note that the DLB2 hardware is a successor of the DLB hardware, and
> > as such is structured similarly, both in terms of code layout and
> > implementation.
> >
> > The framework to support both the PF PMD and bifurcated PMD exists in
> > this patchset, and is why the iface.[ch] layer is present.
> >
> > Depends-on: patch-77466 ("eventdev: add PCI probe named convenience
> function")
> > Depends-on: series-12160 ("Eventdev ABI changes")
> > Depends-on: patch-77460 ("eal: add umonitor umwait to x86 cpuflags")
> 
> Cc: Thomas and David
> 
> There is a duplicate for patch-77460 as
> https://patches.dpdk.org/patch/76554/ from different authors,
> and both are open comments to be closed.
> @McDaniel, Timothy and @Liang Ma , We need to have this eal patch on
> master before I merge this
> patch series in eventdev-next. Please check.
> 
> 

We have discussed this duplication with @Liang Ma, who split out just the
bit definitions into its own patch. DLB/DLB2 have been updated to use his new patch going forward,
but will duplicate it here until it is merged.