[v5] build: prevent accidentally building without NUMA support

Message ID 20230615143854.374384-1-bruce.richardson@intel.com (mailing list archive)
State Accepted, archived
Delegated to: Thomas Monjalon
Headers
Series [v5] build: prevent accidentally building without NUMA support |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/loongarch-compilation success Compilation OK
ci/loongarch-unit-testing success Unit Testing PASS
ci/intel-Testing success Testing PASS
ci/intel-Functional success Functional PASS
ci/github-robot: build success github build: passed
ci/Intel-compilation success Compilation OK

Commit Message

Bruce Richardson June 15, 2023, 2:38 p.m. UTC
  When libnuma development package is missing on a system, DPDK can still
be built but will be missing much-needed support for NUMA memory
management. This may later cause issues at runtime if the resulting
binary is run on a NUMA system.

We can reduce the incidence of such runtime errors by ensuring that, for
native builds*, libnuma is present - unless the user actually specifies
via "max_numa_nodes" that they don't require NUMA support. Having this
as an error condition is also in keeping with what is documented in the
Linux GSG doc, where libnuma is listed as a requirement for building
DPDK [1].

* NOTE: cross-compilation builds have a different logic set, with a
  separate "numa" value indicating if numa support is necessary.

[1] https://doc.dpdk.org/guides-23.03/linux_gsg/sys_reqs.html

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Signed-off-by: David Marchand <david.marchand@redhat.com>

---
V5: Rebase on main, since dependencies merged

V4: Add Depends-on tag so CI picks up dependency

V3:
- install 32-bit libnuma packages on CI systems [thanks to David
  for the changes]
- split the patch out of the previous patchset, so it can be tracked
  separately from the more minor fixup changes.

V2: Limit check to linux only
---
 .github/workflows/build.yml | 5 ++++-
 config/meson.build          | 9 +++++++++
 2 files changed, 13 insertions(+), 1 deletion(-)
  

Comments

Thomas Monjalon June 27, 2023, 1:27 p.m. UTC | #1
15/06/2023 16:38, Bruce Richardson:
> When libnuma development package is missing on a system, DPDK can still
> be built but will be missing much-needed support for NUMA memory
> management. This may later cause issues at runtime if the resulting
> binary is run on a NUMA system.
> 
> We can reduce the incidence of such runtime errors by ensuring that, for
> native builds*, libnuma is present - unless the user actually specifies
> via "max_numa_nodes" that they don't require NUMA support. Having this
> as an error condition is also in keeping with what is documented in the
> Linux GSG doc, where libnuma is listed as a requirement for building
> DPDK [1].
> 
> * NOTE: cross-compilation builds have a different logic set, with a
>   separate "numa" value indicating if numa support is necessary.
> 
> [1] https://doc.dpdk.org/guides-23.03/linux_gsg/sys_reqs.html
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> 
> ---
> V5: Rebase on main, since dependencies merged
> 
> V4: Add Depends-on tag so CI picks up dependency
> 
> V3:
> - install 32-bit libnuma packages on CI systems [thanks to David
>   for the changes]
> - split the patch out of the previous patchset, so it can be tracked
>   separately from the more minor fixup changes.
> 
> V2: Limit check to linux only

Is the test ci/Intel-compilation fixed?
Could you send a new version for testing the CI?
  
Tu, Lijuan June 28, 2023, 3:03 a.m. UTC | #2
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Tuesday, June 27, 2023 9:27 PM
> To: Richardson, Bruce <bruce.richardson@intel.com>
> Cc: dev@dpdk.org; David Marchand <david.marchand@redhat.com>
> Subject: Re: [PATCH v5] build: prevent accidentally building without NUMA
> support
> 
> 15/06/2023 16:38, Bruce Richardson:
> > When libnuma development package is missing on a system, DPDK can
> > still be built but will be missing much-needed support for NUMA memory
> > management. This may later cause issues at runtime if the resulting
> > binary is run on a NUMA system.
> >
> > We can reduce the incidence of such runtime errors by ensuring that,
> > for native builds*, libnuma is present - unless the user actually
> > specifies via "max_numa_nodes" that they don't require NUMA support.
> > Having this as an error condition is also in keeping with what is
> > documented in the Linux GSG doc, where libnuma is listed as a
> > requirement for building DPDK [1].
> >
> > * NOTE: cross-compilation builds have a different logic set, with a
> >   separate "numa" value indicating if numa support is necessary.
> >
> > [1] https://doc.dpdk.org/guides-23.03/linux_gsg/sys_reqs.html
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > Signed-off-by: David Marchand <david.marchand@redhat.com>
> >
> > ---
> > V5: Rebase on main, since dependencies merged
> >
> > V4: Add Depends-on tag so CI picks up dependency
> >
> > V3:
> > - install 32-bit libnuma packages on CI systems [thanks to David
> >   for the changes]
> > - split the patch out of the previous patchset, so it can be tracked
> >   separately from the more minor fixup changes.
> >
> > V2: Limit check to linux only
> 
> Is the test ci/Intel-compilation fixed?
> Could you send a new version for testing the CI?
> 

Sorry, this is caused by intel testbeds lack of dep libnuma, Installed it and re-run the CI, result is PASSED.
  
Thomas Monjalon July 3, 2023, 3:14 p.m. UTC | #3
28/06/2023 05:03, Tu, Lijuan:
> From: Thomas Monjalon <thomas@monjalon.net>
> > 15/06/2023 16:38, Bruce Richardson:
> > Is the test ci/Intel-compilation fixed?
> > Could you send a new version for testing the CI?
> > 
> 
> Sorry, this is caused by intel testbeds lack of dep libnuma, Installed it and re-run the CI, result is PASSED.

Applied, thanks.
  

Patch

diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
index 3b629fcdbd..a479783bbc 100644
--- a/.github/workflows/build.yml
+++ b/.github/workflows/build.yml
@@ -91,6 +91,9 @@  jobs:
       with:
         path: reference
         key: ${{ steps.get_ref_keys.outputs.abi }}
+    - name: Configure i386 architecture
+      if: env.BUILD_32BIT == 'true'
+      run: sudo dpkg --add-architecture i386
     - name: Update APT cache
       run: sudo apt update || true
     - name: Install packages
@@ -104,7 +107,7 @@  jobs:
         pkg-config
     - name: Install i386 cross compiling packages
       if: env.BUILD_32BIT == 'true'
-      run: sudo apt install -y gcc-multilib g++-multilib
+      run: sudo apt install -y gcc-multilib g++-multilib libnuma-dev:i386
     - name: Install aarch64 cross compiling packages
       if: env.AARCH64 == 'true'
       run: sudo apt install -y crossbuild-essential-arm64
diff --git a/config/meson.build b/config/meson.build
index 22d7d908b7..d8223718e4 100644
--- a/config/meson.build
+++ b/config/meson.build
@@ -381,6 +381,15 @@  endif
 if not dpdk_conf.has('RTE_MAX_NUMA_NODES')
     error('Number of NUMA nodes not specified.')
 endif
+if (is_linux and
+        dpdk_conf.get('RTE_MAX_NUMA_NODES') > 1 and
+        not meson.is_cross_build() and
+        not has_libnuma)
+    error('''
+No NUMA library (development package) found, yet DPDK configured for multiple NUMA nodes.
+Please install libnuma, or set 'max_numa_nodes' option to '1' to build without NUMA support.
+''')
+endif
 
 # set the install path for the drivers
 dpdk_conf.set_quoted('RTE_EAL_PMD_PATH', eal_pmd_path)