diff mbox series

[v2] doc: define qualification criteria for external library

Message ID 20230928054036.645183-1-jerinj@marvell.com (mailing list archive)
State Superseded, archived
Delegated to: Thomas Monjalon
Headers
Series [v2] doc: define qualification criteria for external library |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/loongarch-compilation success Compilation OK
ci/loongarch-unit-testing success Unit Testing PASS
ci/Intel-compilation success Compilation OK
ci/intel-Testing success Testing PASS
ci/github-robot: build success github build: passed
ci/intel-Functional success Functional PASS
ci/iol-broadcom-Functional success Functional Testing PASS
ci/iol-mellanox-Performance success Performance Testing PASS
ci/iol-intel-Performance success Performance Testing PASS
ci/iol-broadcom-Performance success Performance Testing PASS
ci/iol-intel-Functional success Functional Testing PASS
ci/iol-sample-apps-testing success Testing PASS
ci/iol-unit-arm64-testing success Testing PASS
ci/iol-compile-amd64-testing success Testing PASS
ci/iol-unit-amd64-testing success Testing PASS
ci/iol-compile-arm64-testing success Testing PASS

Commit Message

Jerin Jacob Kollanukkaran Sept. 28, 2023, 5:40 a.m. UTC 
  From: Jerin Jacob <jerinj@marvell.com>

Define qualification criteria for external library
based on a techboard meeting minutes [1] and past
learnings from mailing list discussion.

[1]
http://mails.dpdk.org/archives/dev/2019-June/135847.html

Signed-off-by: Jerin Jacob <jerinj@marvell.com>
---
v2:
- Added "Meson build integration" and "Code readability" sections.

 doc/guides/contributing/index.rst             |  1 +
 .../contributing/library_dependency.rst       | 23 +++++++++++++++++++
 2 files changed, 24 insertions(+)
 create mode 100644 doc/guides/contributing/library_dependency.rst

Comments

Jerin Jacob Nov. 17, 2023, 4:33 a.m. UTC | #1 
On Thu, Sep 28, 2023 at 11:10 AM <jerinj@marvell.com> wrote:
>
> From: Jerin Jacob <jerinj@marvell.com>
>
> Define qualification criteria for external library
> based on a techboard meeting minutes [1] and past
> learnings from mailing list discussion.
>
> [1]
> http://mails.dpdk.org/archives/dev/2019-June/135847.html
>
> Signed-off-by: Jerin Jacob <jerinj@marvell.com>

Ping for review and/or merge.


> ---
> v2:
> - Added "Meson build integration" and "Code readability" sections.
>
>  doc/guides/contributing/index.rst             |  1 +
>  .../contributing/library_dependency.rst       | 23 +++++++++++++++++++
>  2 files changed, 24 insertions(+)
>  create mode 100644 doc/guides/contributing/library_dependency.rst
>
> diff --git a/doc/guides/contributing/index.rst b/doc/guides/contributing/index.rst
> index dcb9b1fbf0..e5a8c2b0a3 100644
> --- a/doc/guides/contributing/index.rst
> +++ b/doc/guides/contributing/index.rst
> @@ -15,6 +15,7 @@ Contributor's Guidelines
>      documentation
>      unit_test
>      new_library
> +    library_dependency
>      patches
>      vulnerability
>      stable
> diff --git a/doc/guides/contributing/library_dependency.rst b/doc/guides/contributing/library_dependency.rst
> new file mode 100644
> index 0000000000..687a3b6cef
> --- /dev/null
> +++ b/doc/guides/contributing/library_dependency.rst
> @@ -0,0 +1,23 @@
> +.. SPDX-License-Identifier: BSD-3-Clause
> +   Copyright(c) 2023 Marvell.
> +
> +Library dependency
> +==================
> +
> +This document defines the qualification criteria for external libraries that may be
> +used as dependencies in DPDK drivers or libraries.
> +
> +- **Free availability**: The library must be freely available to build in either source or binary
> +  form, with a preference for source form.
> +
> +- **Compiler compatibility**: The library must be able to compile with a DPDK supported compiler
> +  for the given execution environment. For example, For Linux, the library must be able to compile
> +  with GCC and/or clang.
> +
> +- **Documentation**: Must have adequate documentation for the steps to build it.
> +
> +- **Meson build integration**: The library must have standard method like ``pkg-config``
> +  for seamless integration with DPDK's build environment.
> +
> +- **Code readability**: When the depended library is optional, use stubs to reduce the ``ifdef``
> +  clutter to enable better code readability.
> --
> 2.42.0
>
Morten Brørup Nov. 17, 2023, 8:27 a.m. UTC | #2 
> From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> Sent: Friday, 17 November 2023 05.34
> 
> On Thu, Sep 28, 2023 at 11:10 AM <jerinj@marvell.com> wrote:
> >
> > From: Jerin Jacob <jerinj@marvell.com>
> >
> > Define qualification criteria for external library
> > based on a techboard meeting minutes [1] and past
> > learnings from mailing list discussion.
> >
> > [1]
> > http://mails.dpdk.org/archives/dev/2019-June/135847.html
> >
> > Signed-off-by: Jerin Jacob <jerinj@marvell.com>
> 
> Ping for review and/or merge.
> 
> 
> > ---
> > v2:
> > - Added "Meson build integration" and "Code readability" sections.
> >
> >  doc/guides/contributing/index.rst             |  1 +
> >  .../contributing/library_dependency.rst       | 23
> +++++++++++++++++++
> >  2 files changed, 24 insertions(+)
> >  create mode 100644 doc/guides/contributing/library_dependency.rst
> >
> > diff --git a/doc/guides/contributing/index.rst
> b/doc/guides/contributing/index.rst
> > index dcb9b1fbf0..e5a8c2b0a3 100644
> > --- a/doc/guides/contributing/index.rst
> > +++ b/doc/guides/contributing/index.rst
> > @@ -15,6 +15,7 @@ Contributor's Guidelines
> >      documentation
> >      unit_test
> >      new_library
> > +    library_dependency
> >      patches
> >      vulnerability
> >      stable
> > diff --git a/doc/guides/contributing/library_dependency.rst
> b/doc/guides/contributing/library_dependency.rst
> > new file mode 100644
> > index 0000000000..687a3b6cef
> > --- /dev/null
> > +++ b/doc/guides/contributing/library_dependency.rst
> > @@ -0,0 +1,23 @@
> > +.. SPDX-License-Identifier: BSD-3-Clause
> > +   Copyright(c) 2023 Marvell.
> > +
> > +Library dependency
> > +==================
> > +
> > +This document defines the qualification criteria for external
> libraries that may be
> > +used as dependencies in DPDK drivers or libraries.
> > +
> > +- **Free availability**: The library must be freely available to
> build in either source or binary
> > +  form, with a preference for source form.

Suggest adding:

- **Free use and distribution license**: The library must be freely available to use and distribute without any attached conditions.

We must require a BSD-like license, to ensure that DPDK as a whole (including 3rd party libraries) remains BSD licensed, and can be used in commercial (closed source) applications.

> > +
> > +- **Compiler compatibility**: The library must be able to compile
> with a DPDK supported compiler
> > +  for the given execution environment. For example, For Linux, the
> library must be able to compile

Typo (after "For example,"): For -> for

> > +  with GCC and/or clang.
> > +
> > +- **Documentation**: Must have adequate documentation for the steps
> to build it.
> > +
> > +- **Meson build integration**: The library must have standard method
> like ``pkg-config``
> > +  for seamless integration with DPDK's build environment.
> > +
> > +- **Code readability**: When the depended library is optional, use
> stubs to reduce the ``ifdef``
> > +  clutter to enable better code readability.

Why does everyone keep insisting that stubs make code more readable? Sometimes #ifdef is better.

Please use something like this instead:

- **Code readability**: When the depended library is optional, use either stubs or ``#ifdef`` consistently, not a mix of both, to ensure code readability.
Bruce Richardson Nov. 17, 2023, 9:52 a.m. UTC | #3 
On Fri, Nov 17, 2023 at 09:27:02AM +0100, Morten Brørup wrote:
> > From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> > Sent: Friday, 17 November 2023 05.34
> > 
> > On Thu, Sep 28, 2023 at 11:10 AM <jerinj@marvell.com> wrote:
> > >
> > > From: Jerin Jacob <jerinj@marvell.com>
> > >
> > > Define qualification criteria for external library
> > > based on a techboard meeting minutes [1] and past
> > > learnings from mailing list discussion.
> > >
> > > [1]
> > > http://mails.dpdk.org/archives/dev/2019-June/135847.html
> > >
> > > Signed-off-by: Jerin Jacob <jerinj@marvell.com>
> > 
> > Ping for review and/or merge.
> > 
> > 
> > > ---
> > > v2:
> > > - Added "Meson build integration" and "Code readability" sections.
> > >
> > >  doc/guides/contributing/index.rst             |  1 +
> > >  .../contributing/library_dependency.rst       | 23
> > +++++++++++++++++++
> > >  2 files changed, 24 insertions(+)
> > >  create mode 100644 doc/guides/contributing/library_dependency.rst
> > >
> > > diff --git a/doc/guides/contributing/index.rst
> > b/doc/guides/contributing/index.rst
> > > index dcb9b1fbf0..e5a8c2b0a3 100644
> > > --- a/doc/guides/contributing/index.rst
> > > +++ b/doc/guides/contributing/index.rst
> > > @@ -15,6 +15,7 @@ Contributor's Guidelines
> > >      documentation
> > >      unit_test
> > >      new_library
> > > +    library_dependency
> > >      patches
> > >      vulnerability
> > >      stable
> > > diff --git a/doc/guides/contributing/library_dependency.rst
> > b/doc/guides/contributing/library_dependency.rst
> > > new file mode 100644
> > > index 0000000000..687a3b6cef
> > > --- /dev/null
> > > +++ b/doc/guides/contributing/library_dependency.rst
> > > @@ -0,0 +1,23 @@
> > > +.. SPDX-License-Identifier: BSD-3-Clause
> > > +   Copyright(c) 2023 Marvell.
> > > +
> > > +Library dependency
> > > +==================
> > > +
> > > +This document defines the qualification criteria for external
> > libraries that may be
> > > +used as dependencies in DPDK drivers or libraries.
> > > +
> > > +- **Free availability**: The library must be freely available to
> > build in either source or binary
> > > +  form, with a preference for source form.
> 
> Suggest adding:
> 
> - **Free use and distribution license**: The library must be freely available to use and distribute without any attached conditions.
> 
> We must require a BSD-like license, to ensure that DPDK as a whole (including 3rd party libraries) remains BSD licensed, and can be used in commercial (closed source) applications.
> 
I think the situation is a bit more complex. Firstly, we need to ensure
that there are no license incompatibilities. Beyond that though, the
importance of the library will depend on how strict we are going to be
about open-source licensing etc.

For example, for a particular driver - nic, crypto, whatever - we have in
the past allowed linking against non-opensource libraries in order to build
that component. That (thankfully) has not caused us any serious issues to
date, and I don't think we should change things by completely disallowing
it in future.

On the other hand, a library that becomes key for building more than just a
driver or rarely used library, e.g. one that adds key functionality to EAL,
would be held to a much higher standard. In that case we likely would look
for an open-source, appropriately licensed, version.

/Bruce
Morten Brørup Nov. 17, 2023, 10:57 a.m. UTC | #4 
> From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> Sent: Friday, 17 November 2023 10.52
> 
> On Fri, Nov 17, 2023 at 09:27:02AM +0100, Morten Brørup wrote:
> > > From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> > > Sent: Friday, 17 November 2023 05.34
> > >
> > > On Thu, Sep 28, 2023 at 11:10 AM <jerinj@marvell.com> wrote:
> > > >
> > > > From: Jerin Jacob <jerinj@marvell.com>
> > > >
> > > > Define qualification criteria for external library
> > > > based on a techboard meeting minutes [1] and past
> > > > learnings from mailing list discussion.
> > > >
> > > > [1]
> > > > http://mails.dpdk.org/archives/dev/2019-June/135847.html
> > > >
> > > > Signed-off-by: Jerin Jacob <jerinj@marvell.com>
> > >
> > > Ping for review and/or merge.
> > >
> > >
> > > > ---
> > > > v2:
> > > > - Added "Meson build integration" and "Code readability"
> sections.
> > > >
> > > >  doc/guides/contributing/index.rst             |  1 +
> > > >  .../contributing/library_dependency.rst       | 23
> > > +++++++++++++++++++
> > > >  2 files changed, 24 insertions(+)
> > > >  create mode 100644
> doc/guides/contributing/library_dependency.rst
> > > >
> > > > diff --git a/doc/guides/contributing/index.rst
> > > b/doc/guides/contributing/index.rst
> > > > index dcb9b1fbf0..e5a8c2b0a3 100644
> > > > --- a/doc/guides/contributing/index.rst
> > > > +++ b/doc/guides/contributing/index.rst
> > > > @@ -15,6 +15,7 @@ Contributor's Guidelines
> > > >      documentation
> > > >      unit_test
> > > >      new_library
> > > > +    library_dependency
> > > >      patches
> > > >      vulnerability
> > > >      stable
> > > > diff --git a/doc/guides/contributing/library_dependency.rst
> > > b/doc/guides/contributing/library_dependency.rst
> > > > new file mode 100644
> > > > index 0000000000..687a3b6cef
> > > > --- /dev/null
> > > > +++ b/doc/guides/contributing/library_dependency.rst
> > > > @@ -0,0 +1,23 @@
> > > > +.. SPDX-License-Identifier: BSD-3-Clause
> > > > +   Copyright(c) 2023 Marvell.
> > > > +
> > > > +Library dependency
> > > > +==================
> > > > +
> > > > +This document defines the qualification criteria for external
> > > libraries that may be
> > > > +used as dependencies in DPDK drivers or libraries.
> > > > +
> > > > +- **Free availability**: The library must be freely available to
> > > build in either source or binary
> > > > +  form, with a preference for source form.
> >
> > Suggest adding:
> >
> > - **Free use and distribution license**: The library must be freely
> available to use and distribute without any attached conditions.
> >
> > We must require a BSD-like license, to ensure that DPDK as a whole
> (including 3rd party libraries) remains BSD licensed, and can be used
> in commercial (closed source) applications.
> >
> I think the situation is a bit more complex. Firstly, we need to ensure
> that there are no license incompatibilities. Beyond that though, the
> importance of the library will depend on how strict we are going to be
> about open-source licensing etc.

My point about the license was not related to source code availability, it was related to conditions for use and distribution of the library.

The license needs to allow unrestricted and unconditional use and distribution, like the BSD license does.

In principle, this requirement only applies to binary form; we don't need to be able to distribute the source code of external libraries, regardless if it is open source or NDA-protected "view only" source code or other restricted access source code (e.g. Microsoft's Shared Source Initiative).

> 
> For example, for a particular driver - nic, crypto, whatever - we have
> in
> the past allowed linking against non-opensource libraries in order to
> build
> that component. That (thankfully) has not caused us any serious issues
> to
> date, and I don't think we should change things by completely
> disallowing
> it in future.
> 
> On the other hand, a library that becomes key for building more than
> just a
> driver or rarely used library, e.g. one that adds key functionality to
> EAL,
> would be held to a much higher standard. In that case we likely would
> look
> for an open-source, appropriately licensed, version.

I agree that the required degree of open source principles should vary with DPDK's dependency of the library.

We can be more lenient with hardware PMDs, where end users ultimately can choose another hardware vendor if a dependent library is unacceptable for the end user.

There is no doubt we should keep allowing opaque binary BLOBs for hardware, such as on-board firmware and FPGA images.
Jerin Jacob Nov. 19, 2023, 7:07 a.m. UTC | #5 
On Fri, Nov 17, 2023 at 1:57 PM Morten Brørup <mb@smartsharesystems.com> wrote:
>
> > From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> > Sent: Friday, 17 November 2023 05.34
> >
> > On Thu, Sep 28, 2023 at 11:10 AM <jerinj@marvell.com> wrote:
> > >
> > > From: Jerin Jacob <jerinj@marvell.com>
> > >
> > > Define qualification criteria for external library
> > > based on a techboard meeting minutes [1] and past
> > > learnings from mailing list discussion.
> > >
> > > [1]
> > > http://mails.dpdk.org/archives/dev/2019-June/135847.html
> > >
> > > Signed-off-by: Jerin Jacob <jerinj@marvell.com>
> >
> > Ping for review and/or merge.
> >
> >
> > > ---
> > > v2:
> > > - Added "Meson build integration" and "Code readability" sections.
> > >
> > >  doc/guides/contributing/index.rst             |  1 +
> > >  .../contributing/library_dependency.rst       | 23
> > +++++++++++++++++++
> > >  2 files changed, 24 insertions(+)
> > >  create mode 100644 doc/guides/contributing/library_dependency.rst
> > >
> > > diff --git a/doc/guides/contributing/index.rst
> > b/doc/guides/contributing/index.rst
> > > index dcb9b1fbf0..e5a8c2b0a3 100644
> > > --- a/doc/guides/contributing/index.rst
> > > +++ b/doc/guides/contributing/index.rst
> > > @@ -15,6 +15,7 @@ Contributor's Guidelines
> > >      documentation
> > >      unit_test
> > >      new_library
> > > +    library_dependency
> > >      patches
> > >      vulnerability
> > >      stable
> > > diff --git a/doc/guides/contributing/library_dependency.rst
> > b/doc/guides/contributing/library_dependency.rst
> > > new file mode 100644
> > > index 0000000000..687a3b6cef
> > > --- /dev/null
> > > +++ b/doc/guides/contributing/library_dependency.rst
> > > @@ -0,0 +1,23 @@
> > > +.. SPDX-License-Identifier: BSD-3-Clause
> > > +   Copyright(c) 2023 Marvell.
> > > +
> > > +Library dependency
> > > +==================
> > > +
> > > +This document defines the qualification criteria for external
> > libraries that may be
> > > +used as dependencies in DPDK drivers or libraries.
> > > +
> > > +- **Free availability**: The library must be freely available to
> > build in either source or binary
> > > +  form, with a preference for source form.
>
> Suggest adding:
>
> - **Free use and distribution license**: The library must be freely available to use and distribute without any attached conditions.
>
> We must require a BSD-like license, to ensure that DPDK as a whole (including 3rd party libraries) remains BSD licensed, and can be used in commercial (closed source) applications.

As far as I understand, The initial scope of was “free availability”
for building.
Free distribution is much wider scope. I don't think, current external
libraries[1] have free distribution rights.
[1]
https://github.com/DPDK/dpdk/blob/main/doc/guides/gpus/cuda.rst
https://gitlab.com/nvidia/headers/cuda-individual/cudart/-/blob/main/LICENSE?ref_type=heads

I am fine with either way, Feedback from others?

>
> > > +
> > > +- **Compiler compatibility**: The library must be able to compile
> > with a DPDK supported compiler
> > > +  for the given execution environment. For example, For Linux, the
> > library must be able to compile
>
> Typo (after "For example,"): For -> for

Ack. Will fix next version.

>
> > > +  with GCC and/or clang.
> > > +
> > > +- **Documentation**: Must have adequate documentation for the steps
> > to build it.
> > > +
> > > +- **Meson build integration**: The library must have standard method
> > like ``pkg-config``
> > > +  for seamless integration with DPDK's build environment.
> > > +
> > > +- **Code readability**: When the depended library is optional, use
> > stubs to reduce the ``ifdef``
> > > +  clutter to enable better code readability.
>
> Why does everyone keep insisting that stubs make code more readable? Sometimes #ifdef is better.

Could you share a case where when #ifdefs is better(Just to understand
the view).?

>
> Please use something like this instead:
>
> - **Code readability**: When the depended library is optional, use either stubs or ``#ifdef`` consistently, not a mix of both, to ensure code readability.
>
Morten Brørup Nov. 19, 2023, 8:53 a.m. UTC | #6 
> From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> Sent: Sunday, 19 November 2023 08.08
> 
> On Fri, Nov 17, 2023 at 1:57 PM Morten Brørup
> <mb@smartsharesystems.com> wrote:
> >
> > > From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> > > Sent: Friday, 17 November 2023 05.34
> > >
> > > On Thu, Sep 28, 2023 at 11:10 AM <jerinj@marvell.com> wrote:
> > > >
> > > > From: Jerin Jacob <jerinj@marvell.com>
> > > >
> > > > Define qualification criteria for external library
> > > > based on a techboard meeting minutes [1] and past
> > > > learnings from mailing list discussion.
> > > >
> > > > [1]
> > > > http://mails.dpdk.org/archives/dev/2019-June/135847.html
> > > >
> > > > Signed-off-by: Jerin Jacob <jerinj@marvell.com>
> > >
> > > Ping for review and/or merge.
> > >
> > >
> > > > ---
> > > > v2:
> > > > - Added "Meson build integration" and "Code readability"
> sections.
> > > >
> > > >  doc/guides/contributing/index.rst             |  1 +
> > > >  .../contributing/library_dependency.rst       | 23
> > > +++++++++++++++++++
> > > >  2 files changed, 24 insertions(+)
> > > >  create mode 100644
> doc/guides/contributing/library_dependency.rst
> > > >
> > > > diff --git a/doc/guides/contributing/index.rst
> > > b/doc/guides/contributing/index.rst
> > > > index dcb9b1fbf0..e5a8c2b0a3 100644
> > > > --- a/doc/guides/contributing/index.rst
> > > > +++ b/doc/guides/contributing/index.rst
> > > > @@ -15,6 +15,7 @@ Contributor's Guidelines
> > > >      documentation
> > > >      unit_test
> > > >      new_library
> > > > +    library_dependency
> > > >      patches
> > > >      vulnerability
> > > >      stable
> > > > diff --git a/doc/guides/contributing/library_dependency.rst
> > > b/doc/guides/contributing/library_dependency.rst
> > > > new file mode 100644
> > > > index 0000000000..687a3b6cef
> > > > --- /dev/null
> > > > +++ b/doc/guides/contributing/library_dependency.rst
> > > > @@ -0,0 +1,23 @@
> > > > +.. SPDX-License-Identifier: BSD-3-Clause
> > > > +   Copyright(c) 2023 Marvell.
> > > > +
> > > > +Library dependency
> > > > +==================
> > > > +
> > > > +This document defines the qualification criteria for external
> > > libraries that may be
> > > > +used as dependencies in DPDK drivers or libraries.
> > > > +
> > > > +- **Free availability**: The library must be freely available to
> > > build in either source or binary
> > > > +  form, with a preference for source form.
> >
> > Suggest adding:
> >
> > - **Free use and distribution license**: The library must be freely
> available to use and distribute without any attached conditions.
> >
> > We must require a BSD-like license, to ensure that DPDK as a whole
> (including 3rd party libraries) remains BSD licensed, and can be used
> in commercial (closed source) applications.
> 
> As far as I understand, The initial scope of was “free availability”
> for building.
> Free distribution is much wider scope. I don't think, current external
> libraries[1] have free distribution rights.
> [1]
> https://github.com/DPDK/dpdk/blob/main/doc/guides/gpus/cuda.rst
> https://gitlab.com/nvidia/headers/cuda-individual/cudart/-
> /blob/main/LICENSE?ref_type=heads

I didn't mean the library source code; I only meant the library in binary form.

It is nice if the library's header files may be distributed too, but I don't see it as a requirement, especially if they are freely available elsewhere (preferably without imposing additional restrictions on the ability to use and distribute the library in binary form).

How about this instead:

- **License permitting free use and distribution in binary form**:
The library's license must allow free and unconditional use and distribution of the library in binary form.

We might want lawyers to verify the wording when we have agreed on our intentions.

> 
> I am fine with either way, Feedback from others?
> 
> >
> > > > +
> > > > +- **Compiler compatibility**: The library must be able to
> compile
> > > with a DPDK supported compiler
> > > > +  for the given execution environment. For example, For Linux,
> the
> > > library must be able to compile
> >
> > Typo (after "For example,"): For -> for
> 
> Ack. Will fix next version.
> 
> >
> > > > +  with GCC and/or clang.
> > > > +
> > > > +- **Documentation**: Must have adequate documentation for the
> steps
> > > to build it.
> > > > +
> > > > +- **Meson build integration**: The library must have standard
> method
> > > like ``pkg-config``
> > > > +  for seamless integration with DPDK's build environment.
> > > > +
> > > > +- **Code readability**: When the depended library is optional,
> use
> > > stubs to reduce the ``ifdef``
> > > > +  clutter to enable better code readability.
> >
> > Why does everyone keep insisting that stubs make code more readable?
> Sometimes #ifdef is better.
> 
> Could you share a case where when #ifdefs is better(Just to understand
> the view).?

If an external library provides some simple functions, and a group (i.e. a subset) of functions in a DPDK library depends on that external library, then it might be more readable if those functions are enabled/disabled as a group in the DPDK library rather than individually.

E.g. the external library provides functions for statistical processing, and the DPDK library can be built with or without statistics, depending on using the external library or not. If the DPDK library is built without statistics, it should not register statistics availability towards a management module (e.g. telemetry) if it is not really implemented within the DPDK library (because the underlying functions are stubs).

It might also be a matter of personal preferences. When reviewing some source code, an #ifdef makes the availability of an underlying function perfectly clear. Blindly calling a function (of an external library) doesn't really show if the function is implemented for real, or just a stub.

My opinion on this might be tainted by my preference for building from scratch. The distro people might see it very differently!

> 
> >
> > Please use something like this instead:
> >
> > - **Code readability**: When the depended library is optional, use
> either stubs or ``#ifdef`` consistently, not a mix of both, to ensure
> code readability.
> >
Jerin Jacob Nov. 20, 2023, 5:46 p.m. UTC | #7 
On Sun, Nov 19, 2023 at 2:23 PM Morten Brørup <mb@smartsharesystems.com> wrote:
>
> > From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> > Sent: Sunday, 19 November 2023 08.08
> >
> > On Fri, Nov 17, 2023 at 1:57 PM Morten Brørup
> > <mb@smartsharesystems.com> wrote:
> > >
> > > > From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> > > > Sent: Friday, 17 November 2023 05.34
> > > >
> > > > On Thu, Sep 28, 2023 at 11:10 AM <jerinj@marvell.com> wrote:
> > > > >
> > > > > From: Jerin Jacob <jerinj@marvell.com>
> > > > >
> > > > > Define qualification criteria for external library
> > > > > based on a techboard meeting minutes [1] and past
> > > > > learnings from mailing list discussion.
> > > > >
> > > > > [1]
> > > > > http://mails.dpdk.org/archives/dev/2019-June/135847.html
> > > > >
> > > > > Signed-off-by: Jerin Jacob <jerinj@marvell.com>
> > > >
> > > > Ping for review and/or merge.
> > > >
> > > >
> > > > > ---
> > > > > v2:
> > > > > - Added "Meson build integration" and "Code readability"
> > sections.
> > > > >
> > > > >  doc/guides/contributing/index.rst             |  1 +
> > > > >  .../contributing/library_dependency.rst       | 23
> > > > +++++++++++++++++++
> > > > >  2 files changed, 24 insertions(+)
> > > > >  create mode 100644
> > doc/guides/contributing/library_dependency.rst
> > > > >
> > > > > diff --git a/doc/guides/contributing/index.rst
> > > > b/doc/guides/contributing/index.rst
> > > > > index dcb9b1fbf0..e5a8c2b0a3 100644
> > > > > --- a/doc/guides/contributing/index.rst
> > > > > +++ b/doc/guides/contributing/index.rst
> > > > > @@ -15,6 +15,7 @@ Contributor's Guidelines
> > > > >      documentation
> > > > >      unit_test
> > > > >      new_library
> > > > > +    library_dependency
> > > > >      patches
> > > > >      vulnerability
> > > > >      stable
> > > > > diff --git a/doc/guides/contributing/library_dependency.rst
> > > > b/doc/guides/contributing/library_dependency.rst
> > > > > new file mode 100644
> > > > > index 0000000000..687a3b6cef
> > > > > --- /dev/null
> > > > > +++ b/doc/guides/contributing/library_dependency.rst
> > > > > @@ -0,0 +1,23 @@
> > > > > +.. SPDX-License-Identifier: BSD-3-Clause
> > > > > +   Copyright(c) 2023 Marvell.
> > > > > +
> > > > > +Library dependency
> > > > > +==================
> > > > > +
> > > > > +This document defines the qualification criteria for external
> > > > libraries that may be
> > > > > +used as dependencies in DPDK drivers or libraries.
> > > > > +
> > > > > +- **Free availability**: The library must be freely available to
> > > > build in either source or binary
> > > > > +  form, with a preference for source form.
> > >
> > > Suggest adding:
> > >
> > > - **Free use and distribution license**: The library must be freely
> > available to use and distribute without any attached conditions.
> > >
> > > We must require a BSD-like license, to ensure that DPDK as a whole
> > (including 3rd party libraries) remains BSD licensed, and can be used
> > in commercial (closed source) applications.
> >
> > As far as I understand, The initial scope of was “free availability”
> > for building.
> > Free distribution is much wider scope. I don't think, current external
> > libraries[1] have free distribution rights.
> > [1]
> > https://github.com/DPDK/dpdk/blob/main/doc/guides/gpus/cuda.rst
> > https://gitlab.com/nvidia/headers/cuda-individual/cudart/-
> > /blob/main/LICENSE?ref_type=heads
>
> I didn't mean the library source code; I only meant the library in binary form.
>
> It is nice if the library's header files may be distributed too, but I don't see it as a requirement, especially if they are freely available elsewhere (preferably without imposing additional restrictions on the ability to use and distribute the library in binary form).
>
> How about this instead:
>
> - **License permitting free use and distribution in binary form**:
> The library's license must allow free and unconditional use and distribution of the library in binary form.

Distribution and unconditional use is not the case for existing
library dependencies such as
https://gitlab.com/nvidia/headers/cuda-individual/cudart/-/blob/main/LICENSE?ref_type=heads

So I am not sure, Which is the correct thing to do. Maybe we can
discuss more in tech board meeting if there are no other comments in
mailing list on this topic.


>
> We might want lawyers to verify the wording when we have agreed on our intentions.
>
> >
> > I am fine with either way, Feedback from others?
> >
> > >
> > > > > +
> > > > > +- **Compiler compatibility**: The library must be able to
> > compile
> > > > with a DPDK supported compiler
> > > > > +  for the given execution environment. For example, For Linux,
> > the
> > > > library must be able to compile
> > >
> > > Typo (after "For example,"): For -> for
> >
> > Ack. Will fix next version.
> >
> > >
> > > > > +  with GCC and/or clang.
> > > > > +
> > > > > +- **Documentation**: Must have adequate documentation for the
> > steps
> > > > to build it.
> > > > > +
> > > > > +- **Meson build integration**: The library must have standard
> > method
> > > > like ``pkg-config``
> > > > > +  for seamless integration with DPDK's build environment.
> > > > > +
> > > > > +- **Code readability**: When the depended library is optional,
> > use
> > > > stubs to reduce the ``ifdef``
> > > > > +  clutter to enable better code readability.
> > >
> > > Why does everyone keep insisting that stubs make code more readable?
> > Sometimes #ifdef is better.
> >
> > Could you share a case where when #ifdefs is better(Just to understand
> > the view).?
>
> If an external library provides some simple functions, and a group (i.e. a subset) of functions in a DPDK library depends on that external library, then it might be more readable if those functions are enabled/disabled as a group in the DPDK library rather than individually.
>
> E.g. the external library provides functions for statistical processing, and the DPDK library can be built with or without statistics, depending on using the external library or not. If the DPDK library is built without statistics, it should not register statistics availability towards a management module (e.g. telemetry) if it is not really implemented within the DPDK library (because the underlying functions are stubs).
>
> It might also be a matter of personal preferences. When reviewing some source code, an #ifdef makes the availability of an underlying function perfectly clear. Blindly calling a function (of an external library) doesn't really show if the function is implemented for real, or just a stub.

In general theme in DPDK code base that we are trying to avoid a  lot
of #ifdef in a given C file.
Instead, we are doing following scheme.


https://github.com/DPDK/dpdk/blob/main/drivers/ml/cnxk/meson.build#L84
https://github.com/DPDK/dpdk/blob/main/drivers/ml/cnxk/meson.build#L60
https://github.com/DPDK/dpdk/blob/main/drivers/ml/cnxk/mvtvm_ml_stubs.c


>
> My opinion on this might be tainted by my preference for building from scratch. The distro people might see it very differently!


Yeah. I don't have a strong opinion, I can change to following, if
there are no comments on this. Or we can discuss more in TB meeting if
there are no review comments in mailing list on this topic.

**Code readability**: When the depended library is optional, use
either stubs or ``#ifdef`` consistently, not a mix of both, to ensure
code readability.

>
> >
> > >
> > > Please use something like this instead:
> > >
> > > - **Code readability**: When the depended library is optional, use
> > either stubs or ``#ifdef`` consistently, not a mix of both, to ensure
> > code readability.
> > >
Thomas Monjalon Nov. 27, 2023, 4:25 p.m. UTC | #8 
20/11/2023 18:46, Jerin Jacob:
> On Sun, Nov 19, 2023 at 2:23 PM Morten Brørup <mb@smartsharesystems.com> wrote:
> > > From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> > > On Fri, Nov 17, 2023 at 1:57 PM Morten Brørup
> > > > > From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> > > > > On Thu, Sep 28, 2023 at 11:10 AM <jerinj@marvell.com> wrote:
> > > > > > From: Jerin Jacob <jerinj@marvell.com>
> > > > > > +- **Free availability**: The library must be freely available to
> > > > > build in either source or binary
> > > > > > +  form, with a preference for source form.
> > > >
> > > > Suggest adding:
> > > >
> > > > - **Free use and distribution license**: The library must be freely
> > > available to use and distribute without any attached conditions.
> > > >
> > > > We must require a BSD-like license, to ensure that DPDK as a whole
> > > (including 3rd party libraries) remains BSD licensed, and can be used
> > > in commercial (closed source) applications.
> > >
> > > As far as I understand, The initial scope of was “free availability”
> > > for building.
> > > Free distribution is much wider scope. I don't think, current external
> > > libraries[1] have free distribution rights.
> > > [1]
> > > https://github.com/DPDK/dpdk/blob/main/doc/guides/gpus/cuda.rst
> > > https://gitlab.com/nvidia/headers/cuda-individual/cudart/-
> > > /blob/main/LICENSE?ref_type=heads
> >
> > I didn't mean the library source code; I only meant the library in binary form.
> >
> > It is nice if the library's header files may be distributed too, but I don't see it as a requirement, especially if they are freely available elsewhere (preferably without imposing additional restrictions on the ability to use and distribute the library in binary form).
> >
> > How about this instead:
> >
> > - **License permitting free use and distribution in binary form**:
> > The library's license must allow free and unconditional use and distribution of the library in binary form.
> 
> Distribution and unconditional use is not the case for existing
> library dependencies such as
> https://gitlab.com/nvidia/headers/cuda-individual/cudart/-/blob/main/LICENSE?ref_type=heads
> 
> So I am not sure, Which is the correct thing to do. Maybe we can
> discuss more in tech board meeting if there are no other comments in
> mailing list on this topic.

I don't think we should make mandatory to have rights of redistribution.
To me, being to download and install the dependency without any restriction
is enough *for a driver*.
If distribution of the dependency is restricted,
then the driver will be disabled by the distro.
It is not our problem I think.

And I agree we must have stricter requirements for libraries dependencies.
I am OK to mandate free distribution for such library dependencies.


> > We might want lawyers to verify the wording when we have agreed on our intentions.
> >
> > > I am fine with either way, Feedback from others?

[...]

> > > > > > +- **Code readability**: When the depended library is optional,
> > > use
> > > > > stubs to reduce the ``ifdef``
> > > > > > +  clutter to enable better code readability.
> > > >
> > > > Why does everyone keep insisting that stubs make code more readable?
> > > Sometimes #ifdef is better.
> > >
> > > Could you share a case where when #ifdefs is better(Just to understand
> > > the view).?
> >
> > If an external library provides some simple functions, and a group (i.e. a subset) of functions in a DPDK library depends on that external library, then it might be more readable if those functions are enabled/disabled as a group in the DPDK library rather than individually.
> >
> > E.g. the external library provides functions for statistical processing, and the DPDK library can be built with or without statistics, depending on using the external library or not. If the DPDK library is built without statistics, it should not register statistics availability towards a management module (e.g. telemetry) if it is not really implemented within the DPDK library (because the underlying functions are stubs).
> >
> > It might also be a matter of personal preferences. When reviewing some source code, an #ifdef makes the availability of an underlying function perfectly clear. Blindly calling a function (of an external library) doesn't really show if the function is implemented for real, or just a stub.
> 
> In general theme in DPDK code base that we are trying to avoid a  lot
> of #ifdef in a given C file.
> Instead, we are doing following scheme.
> 
> https://github.com/DPDK/dpdk/blob/main/drivers/ml/cnxk/meson.build#L84
> https://github.com/DPDK/dpdk/blob/main/drivers/ml/cnxk/meson.build#L60
> https://github.com/DPDK/dpdk/blob/main/drivers/ml/cnxk/mvtvm_ml_stubs.c
> 
> > My opinion on this might be tainted by my preference for building from scratch. The distro people might see it very differently!
> 
> Yeah. I don't have a strong opinion, I can change to following, if
> there are no comments on this. Or we can discuss more in TB meeting if
> there are no review comments in mailing list on this topic.
> 
> **Code readability**: When the depended library is optional, use
> either stubs or ``#ifdef`` consistently, not a mix of both, to ensure
> code readability.
> 
> > > > Please use something like this instead:
> > > >
> > > > - **Code readability**: When the depended library is optional, use
> > > either stubs or ``#ifdef`` consistently, not a mix of both, to ensure
> > > code readability.

We should better focus on code organisation for technical reasons:
we don't want to mix OS-specific code in a common file.
That's why stubs are preferred to implement functions with OS-specific includes, etc.
If it's just a matter of disabling some code, #ifdef is fine.
Stephen Hemminger Nov. 27, 2023, 5:13 p.m. UTC | #9 
On Mon, 27 Nov 2023 17:25:55 +0100
Thomas Monjalon <thomas@monjalon.net> wrote:

> > 
> > Distribution and unconditional use is not the case for existing
> > library dependencies such as
> > https://gitlab.com/nvidia/headers/cuda-individual/cudart/-/blob/main/LICENSE?ref_type=heads
> > 
> > So I am not sure, Which is the correct thing to do. Maybe we can
> > discuss more in tech board meeting if there are no other comments in
> > mailing list on this topic.  
> 
> I don't think we should make mandatory to have rights of redistribution.
> To me, being to download and install the dependency without any restriction
> is enough *for a driver*.
> If distribution of the dependency is restricted,
> then the driver will be disabled by the distro.
> It is not our problem I think.
> 
> And I agree we must have stricter requirements for libraries dependencies.
> I am OK to mandate free distribution for such library dependencies.

For me, the policy should meet these goals:

1. A developer should be able to test any driver and library for free.
Ok, with any dependencies as long as they are freely available, and do not require
giving up all rights in the process (click through agreements).

2. A distro maintainer should be able to build DPDK with the component if
any dependent packages are available for that distribution.

3. DPDK must not become a sham front end for proprietary binary blobs.
That is what Open Dataplane was, just a wrapper API around proprietary binary
blob backends.
diff mbox series

Patch

diff --git a/doc/guides/contributing/index.rst b/doc/guides/contributing/index.rst
index dcb9b1fbf0..e5a8c2b0a3 100644
--- a/doc/guides/contributing/index.rst
+++ b/doc/guides/contributing/index.rst
@@ -15,6 +15,7 @@  Contributor's Guidelines
     documentation
     unit_test
     new_library
+    library_dependency
     patches
     vulnerability
     stable
diff --git a/doc/guides/contributing/library_dependency.rst b/doc/guides/contributing/library_dependency.rst
new file mode 100644
index 0000000000..687a3b6cef
--- /dev/null
+++ b/doc/guides/contributing/library_dependency.rst
@@ -0,0 +1,23 @@ 
+.. SPDX-License-Identifier: BSD-3-Clause
+   Copyright(c) 2023 Marvell.
+
+Library dependency
+==================
+
+This document defines the qualification criteria for external libraries that may be
+used as dependencies in DPDK drivers or libraries.
+
+- **Free availability**: The library must be freely available to build in either source or binary
+  form, with a preference for source form.
+
+- **Compiler compatibility**: The library must be able to compile with a DPDK supported compiler
+  for the given execution environment. For example, For Linux, the library must be able to compile
+  with GCC and/or clang.
+
+- **Documentation**: Must have adequate documentation for the steps to build it.
+
+- **Meson build integration**: The library must have standard method like ``pkg-config``
+  for seamless integration with DPDK's build environment.
+
+- **Code readability**: When the depended library is optional, use stubs to reduce the ``ifdef``
+  clutter to enable better code readability.