build: add pkg-config validation

Message ID 20201029091638.26646-1-getelson@nvidia.com (mailing list archive)
State Superseded, archived
Delegated to: Thomas Monjalon
Headers
Series build: add pkg-config validation |

Checks

Context Check Description
ci/travis-robot success Travis build: passed
ci/checkpatch warning coding style issues
ci/Intel-compilation success Compilation OK
ci/iol-intel-Functional success Functional Testing PASS
ci/iol-testing success Testing PASS
ci/iol-intel-Performance success Performance Testing PASS
ci/iol-mellanox-Performance success Performance Testing PASS

Commit Message

Etelson, Gregory Oct. 29, 2020, 9:16 a.m. UTC
DPDK relies on pkg-config(1) to provide correct parameters for
compiler and linker used in application build.
Inaccurate build parameters, produced by pkg-config from DPDK .pc
files could fail application build or cause unpredicted results
during application runtime.

This patch validates host pkg-config utility and notifies about
known issues.

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
---
 buildtools/pkg-config/meson.build           | 11 ++++++
 buildtools/pkg-config/pkgconfig-validate.sh | 38 +++++++++++++++++++++
 doc/guides/linux_gsg/sys_reqs.rst           |  5 +++
 3 files changed, 54 insertions(+)
 create mode 100755 buildtools/pkg-config/pkgconfig-validate.sh
  

Comments

Thomas Monjalon Nov. 1, 2020, 10:01 a.m. UTC | #1
29/10/2020 10:16, Gregory Etelson:
> DPDK relies on pkg-config(1) to provide correct parameters for
> compiler and linker used in application build.
> Inaccurate build parameters, produced by pkg-config from DPDK .pc
> files could fail application build or cause unpredicted results
> during application runtime.

Actually this is only for static linkage of examples with makefiles.

> This patch validates host pkg-config utility and notifies about
> known issues.
> 
> Signed-off-by: Gregory Etelson <getelson@nvidia.com>
> ---
> --- a/buildtools/pkg-config/meson.build
> +++ b/buildtools/pkg-config/meson.build
> +pkgconf = find_program('pkg-config', 'pkgconf', required: false)
> +if (pkgconf.found())
> +	cmd = run_command('./pkgconfig-validate.sh', pkgconf.path(),
> +			   check:false)
> +	if cmd.returncode() != 0
> +		version = run_command(pkgconf, '--version')
> +		warning('invalid pkg-config version @0@'.format(
> +			version.stdout().strip()))
> +	endif
> +endif

Should we restrict this warning to the example makefiles?


> --- /dev/null
> +++ b/buildtools/pkg-config/pkgconfig-validate.sh
> @@ -0,0 +1,38 @@
> +#! /bin/sh
> +# SPDX-License-Identifier: BSD-3-Clause
> +
> +if [ "$#" -ne 1 ]; then
> +	echo "$0: no pkg-config parameter"
> +	exit 1
> +fi

The default pkg-config from path could considered.

> +
> +ret=0
> +PKGCONF="$1"

PKGCONF=${1:-pkg-config}

> +
> +# take the first result only
> +pc_file=$(find "$MESON_BUILD_ROOT" -type f -name 'libdpdk.pc' -print -quit)

It does not look in PKG_CONFIG_PATH.

> +if [ ! -f "$pc_file" ]; then
> +	echo "$0: cannot locate libdpdk.pc"
> +	exit 1
> +fi
> +
> +pc_dir=$(dirname "$pc_file")
> +__pkg_config_path="$PKG_CONFIG_PATH"
> +PKG_CONFIG_PATH="${PKG_CONFIG_PATH}:$pc_dir"
> +export PKG_CONFIG_PATH
> +
> +# Statically linked private DPDK objects of form
> +# -l:file.a must be positionned between --whole-archive … --no-whole-archive
> +# linker parameters.
> +# Old pkg-config versions misplace --no-whole-archive parameter and put it
> +# next to --whole-archive.
> +"$PKGCONF" --libs --static libdpdk | \
> +grep -q 'whole-archive.*l:lib.*no-whole-archive'
> +if test "$?" -ne 0 ; then
> +	echo "WARNING: invalid pkg-config"
> +	ret=1
> +fi
> +
> +# restore PKG_CONFIG_PATH
> +export PKG_CONFIG_PATH="$__pkg_config_path"

Instead of restoring, you could just set the variable on the
command line calling PKGCONF.

> +exit $ret


> --- a/doc/guides/linux_gsg/sys_reqs.rst
> +++ b/doc/guides/linux_gsg/sys_reqs.rst
> +**Known Issues:**
> +
> +*   pkg-config v0.27 supplied with RHEL-7 does not process correctly libdpdk.pc Libs.private section.

Is it only the RHEL version or all 0.27 packages?

Is it enough to warn on issue, or do we prefer recommend a minimal version?
  
Etelson, Gregory Nov. 1, 2020, 12:06 p.m. UTC | #2
Hello Thomas,

> -----Original Message-----
> 
> 29/10/2020 10:16, Gregory Etelson:
> > DPDK relies on pkg-config(1) to provide correct parameters for
> > compiler and linker used in application build.
> > Inaccurate build parameters, produced by pkg-config from DPDK .pc
> > files could fail application build or cause unpredicted results during
> > application runtime.
> 
> Actually this is only for static linkage of examples with makefiles.
>

In our case, pkg-config produced wrong output for statically linked
external application.
General case of error in pkg-config output can lead to any kind of build
or execution fault. 
 
> > This patch validates host pkg-config utility and notifies about known
> > issues.
> >
> > Signed-off-by: Gregory Etelson <getelson@nvidia.com>
> > ---
> > --- a/buildtools/pkg-config/meson.build
> > +++ b/buildtools/pkg-config/meson.build
> > +pkgconf = find_program('pkg-config', 'pkgconf', required: false) if
> > +(pkgconf.found())
> > +     cmd = run_command('./pkgconfig-validate.sh', pkgconf.path(),
> > +                        check:false)
> > +     if cmd.returncode() != 0
> > +             version = run_command(pkgconf, '--version')
> > +             warning('invalid pkg-config version @0@'.format(
> > +                     version.stdout().strip()))
> > +     endif
> > +endif
> 
> Should we restrict this warning to the example makefiles?
>

If DPDK build infrastructure detects wrong pkg-config, it must
notify all applications, internal and external, about the
potential build fault.
 
> 
> > --- /dev/null
> > +++ b/buildtools/pkg-config/pkgconfig-validate.sh
> > @@ -0,0 +1,38 @@
> > +#! /bin/sh
> > +# SPDX-License-Identifier: BSD-3-Clause
> > +
> > +if [ "$#" -ne 1 ]; then
> > +     echo "$0: no pkg-config parameter"
> > +     exit 1
> > +fi
> 
> The default pkg-config from path could considered.
> 

Host can use ether pkg-config or pkgconf utility for
build configuration. The script let the meson build
system to query host what exact utility is used.
Meson passes the query result to the validation script. 

> > +
> > +ret=0
> > +PKGCONF="$1"
> 
> PKGCONF=${1:-pkg-config}
> 

Please see above.

> > +
> > +# take the first result only
> > +pc_file=$(find "$MESON_BUILD_ROOT" -type f -name 'libdpdk.pc' -print
> > +-quit)
> 
> It does not look in PKG_CONFIG_PATH.
>

Meson stores libdpdk.pc template under MESON_BUILD_ROOT. That template has enough
information to run validation queries.
 
> > +if [ ! -f "$pc_file" ]; then
> > +     echo "$0: cannot locate libdpdk.pc"
> > +     exit 1
> > +fi
> > +
> > +pc_dir=$(dirname "$pc_file")
> > +__pkg_config_path="$PKG_CONFIG_PATH"
> > +PKG_CONFIG_PATH="${PKG_CONFIG_PATH}:$pc_dir"
> > +export PKG_CONFIG_PATH
> > +
> > +# Statically linked private DPDK objects of form # -l:file.a must be
> > +positionned between --whole-archive … --no-whole-archive # linker
> > +parameters.
> > +# Old pkg-config versions misplace --no-whole-archive parameter and
> > +put it # next to --whole-archive.
> > +"$PKGCONF" --libs --static libdpdk | \ grep -q
> > +'whole-archive.*l:lib.*no-whole-archive'
> > +if test "$?" -ne 0 ; then
> > +     echo "WARNING: invalid pkg-config"
> > +     ret=1
> > +fi
> > +
> > +# restore PKG_CONFIG_PATH
> > +export PKG_CONFIG_PATH="$__pkg_config_path"
> 
> Instead of restoring, you could just set the variable on the command line
> calling PKGCONF.

Not sure I understand what did mean here.

> 
> > +exit $ret
> 
> 
> > --- a/doc/guides/linux_gsg/sys_reqs.rst
> > +++ b/doc/guides/linux_gsg/sys_reqs.rst
> > +**Known Issues:**
> > +
> > +*   pkg-config v0.27 supplied with RHEL-7 does not process correctly
> libdpdk.pc Libs.private section.
> 
> Is it only the RHEL version or all 0.27 packages?
> 
> Is it enough to warn on issue, or do we prefer recommend a minimal
> version?
> 

We can both warn about pkg-config versions with known issues and validate
host configuration utility during run-time.
  

Patch

diff --git a/buildtools/pkg-config/meson.build b/buildtools/pkg-config/meson.build
index 5f19304289..57ee096988 100644
--- a/buildtools/pkg-config/meson.build
+++ b/buildtools/pkg-config/meson.build
@@ -53,3 +53,14 @@  This is required for a number of static inline functions in the public headers.'
 # For static linking with dependencies as shared libraries,
 # the internal static libraries must be flagged explicitly.
 run_command(py3, 'set-static-linker-flags.py', check: true)
+
+pkgconf = find_program('pkg-config', 'pkgconf', required: false)
+if (pkgconf.found())
+	cmd = run_command('./pkgconfig-validate.sh', pkgconf.path(),
+			   check:false)
+	if cmd.returncode() != 0
+		version = run_command(pkgconf, '--version')
+		warning('invalid pkg-config version @0@'.format(
+			version.stdout().strip()))
+	endif
+endif
\ No newline at end of file
diff --git a/buildtools/pkg-config/pkgconfig-validate.sh b/buildtools/pkg-config/pkgconfig-validate.sh
new file mode 100755
index 0000000000..b606bde908
--- /dev/null
+++ b/buildtools/pkg-config/pkgconfig-validate.sh
@@ -0,0 +1,38 @@ 
+#! /bin/sh
+# SPDX-License-Identifier: BSD-3-Clause
+
+if [ "$#" -ne 1 ]; then
+	echo "$0: no pkg-config parameter"
+	exit 1
+fi
+
+ret=0
+PKGCONF="$1"
+
+# take the first result only
+pc_file=$(find "$MESON_BUILD_ROOT" -type f -name 'libdpdk.pc' -print -quit)
+if [ ! -f "$pc_file" ]; then
+	echo "$0: cannot locate libdpdk.pc"
+	exit 1
+fi
+
+pc_dir=$(dirname "$pc_file")
+__pkg_config_path="$PKG_CONFIG_PATH"
+PKG_CONFIG_PATH="${PKG_CONFIG_PATH}:$pc_dir"
+export PKG_CONFIG_PATH
+
+# Statically linked private DPDK objects of form
+# -l:file.a must be positionned between --whole-archive … --no-whole-archive
+# linker parameters.
+# Old pkg-config versions misplace --no-whole-archive parameter and put it
+# next to --whole-archive.
+"$PKGCONF" --libs --static libdpdk | \
+grep -q 'whole-archive.*l:lib.*no-whole-archive'
+if test "$?" -ne 0 ; then
+	echo "WARNING: invalid pkg-config"
+	ret=1
+fi
+
+# restore PKG_CONFIG_PATH
+export PKG_CONFIG_PATH="$__pkg_config_path"
+exit $ret
diff --git a/doc/guides/linux_gsg/sys_reqs.rst b/doc/guides/linux_gsg/sys_reqs.rst
index 6ecdc04aa9..b67da05e13 100644
--- a/doc/guides/linux_gsg/sys_reqs.rst
+++ b/doc/guides/linux_gsg/sys_reqs.rst
@@ -60,6 +60,11 @@  Compilation of the DPDK
 
 *   Linux kernel headers or sources required to build kernel modules.
 
+
+**Known Issues:**
+
+*   pkg-config v0.27 supplied with RHEL-7 does not process correctly libdpdk.pc Libs.private section.
+
 .. note::
 
    Please ensure that the latest patches are applied to third party libraries