diff mbox series

[v2] buildtools: ensure the NUMA nodes are counted correct

Message ID 20220922080642.315800-1-niklas.soderlund@corigine.com (mailing list archive)
State Accepted, archived
Delegated to: Thomas Monjalon
Headers
Series [v2] buildtools: ensure the NUMA nodes are counted correct |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation success Compilation OK
ci/iol-mellanox-Performance success Performance Testing PASS
ci/iol-aarch64-unit-testing success Testing PASS
ci/iol-aarch64-compile-testing success Testing PASS
ci/github-robot: build success github build: passed
ci/iol-x86_64-compile-testing success Testing PASS
ci/iol-x86_64-unit-testing success Testing PASS
ci/iol-intel-Performance success Performance Testing PASS
ci/iol-intel-Functional success Functional Testing PASS
ci/intel-Testing success Testing PASS

Commit Message

Niklas Söderlund Sept. 22, 2022, 8:06 a.m. UTC 
  From: Peng Zhang <peng.zhang@corigine.com>

The method to fetch, sort and read the last entry of a list to figure
out the total number of NUMA nodes in the system fails with 10 or more
nodes. The reason being the usage of string compare while sorting, hence
node 'node10' will be sorted before 'node2'.

Solve this by sorting the list based on integer comparison of the
numerical part of the node name.

Before this change on a system with 16 NUMA nodes,

    EAL: Detected CPU lcores: 128
    EAL: Detected NUMA nodes: 10
    EAL: Static memory layout is selected, amount of reserved memory can
	 be adjusted with -m or --socket-mem
    EAL: Detected static linkage of DPDK
    EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
    EAL: Selected IOVA mode 'VA'
    EAL: VFIO support initialized

With this change on the same system,

    EAL: Detected CPU lcores: 128
    EAL: Detected NUMA nodes: 16
    EAL: Static memory layout is selected, amount of reserved memory can
	 be adjusted with -m or --socket-mem
    EAL: Detected static linkage of DPDK
    EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
    EAL: Selected IOVA mode 'VA'
    EAL: VFIO support initialized

Fixes: 8ef09fdc506b ("build: add optional NUMA and CPU counts detection")
Cc: stable@dpdk.org

Signed-off-by: Peng Zhang <peng.zhang@corigine.com>
Signed-off-by: Chaoyong He <chaoyong.he@corigine.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@corigine.com>
---
 buildtools/get-numa-count.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

Comments

Niklas Söderlund Oct. 10, 2022, 7:52 a.m. UTC | #1 
Hi,

Genteel ping on this.

On 2022-09-22 10:06:42 +0200, Niklas Söderlund wrote:
> From: Peng Zhang <peng.zhang@corigine.com>
> 
> The method to fetch, sort and read the last entry of a list to figure
> out the total number of NUMA nodes in the system fails with 10 or more
> nodes. The reason being the usage of string compare while sorting, hence
> node 'node10' will be sorted before 'node2'.
> 
> Solve this by sorting the list based on integer comparison of the
> numerical part of the node name.
> 
> Before this change on a system with 16 NUMA nodes,
> 
>     EAL: Detected CPU lcores: 128
>     EAL: Detected NUMA nodes: 10
>     EAL: Static memory layout is selected, amount of reserved memory can
> 	 be adjusted with -m or --socket-mem
>     EAL: Detected static linkage of DPDK
>     EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
>     EAL: Selected IOVA mode 'VA'
>     EAL: VFIO support initialized
> 
> With this change on the same system,
> 
>     EAL: Detected CPU lcores: 128
>     EAL: Detected NUMA nodes: 16
>     EAL: Static memory layout is selected, amount of reserved memory can
> 	 be adjusted with -m or --socket-mem
>     EAL: Detected static linkage of DPDK
>     EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
>     EAL: Selected IOVA mode 'VA'
>     EAL: VFIO support initialized
> 
> Fixes: 8ef09fdc506b ("build: add optional NUMA and CPU counts detection")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Peng Zhang <peng.zhang@corigine.com>
> Signed-off-by: Chaoyong He <chaoyong.he@corigine.com>
> Reviewed-by: Niklas Söderlund <niklas.soderlund@corigine.com>
> ---
>  buildtools/get-numa-count.py | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/buildtools/get-numa-count.py b/buildtools/get-numa-count.py
> index 1b7787787f71..2f243886cd21 100644
> --- a/buildtools/get-numa-count.py
> +++ b/buildtools/get-numa-count.py
> @@ -6,11 +6,12 @@
>  import glob
>  import os
>  import subprocess
> +import re
>  
>  if os.name == 'posix':
>      if os.path.isdir('/sys/devices/system/node'):
>          numa_nodes = glob.glob('/sys/devices/system/node/node*')
> -        numa_nodes.sort()
> +        numa_nodes.sort(key=lambda l: int(re.findall('\d+', l)[0]))
>          print(int(os.path.basename(numa_nodes[-1])[4:]) + 1)
>      else:
>          subprocess.run(['sysctl', '-n', 'vm.ndomains'], check=False)
> -- 
> 2.37.3
>
Thomas Monjalon Oct. 10, 2022, 11:11 p.m. UTC | #2 
22/09/2022 10:06, Niklas Söderlund:
> From: Peng Zhang <peng.zhang@corigine.com>
> 
> The method to fetch, sort and read the last entry of a list to figure
> out the total number of NUMA nodes in the system fails with 10 or more
> nodes. The reason being the usage of string compare while sorting, hence
> node 'node10' will be sorted before 'node2'.
> 
> Solve this by sorting the list based on integer comparison of the
> numerical part of the node name.
> 
> Before this change on a system with 16 NUMA nodes,
> 
>     EAL: Detected CPU lcores: 128
>     EAL: Detected NUMA nodes: 10
>     EAL: Static memory layout is selected, amount of reserved memory can
> 	 be adjusted with -m or --socket-mem
>     EAL: Detected static linkage of DPDK
>     EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
>     EAL: Selected IOVA mode 'VA'
>     EAL: VFIO support initialized
> 
> With this change on the same system,
> 
>     EAL: Detected CPU lcores: 128
>     EAL: Detected NUMA nodes: 16
>     EAL: Static memory layout is selected, amount of reserved memory can
> 	 be adjusted with -m or --socket-mem
>     EAL: Detected static linkage of DPDK
>     EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
>     EAL: Selected IOVA mode 'VA'
>     EAL: VFIO support initialized
> 
> Fixes: 8ef09fdc506b ("build: add optional NUMA and CPU counts detection")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Peng Zhang <peng.zhang@corigine.com>
> Signed-off-by: Chaoyong He <chaoyong.he@corigine.com>
> Reviewed-by: Niklas Söderlund <niklas.soderlund@corigine.com>

Applied, thanks.
diff mbox series

Patch

diff --git a/buildtools/get-numa-count.py b/buildtools/get-numa-count.py
index 1b7787787f71..2f243886cd21 100644
--- a/buildtools/get-numa-count.py
+++ b/buildtools/get-numa-count.py
@@ -6,11 +6,12 @@ 
 import glob
 import os
 import subprocess
+import re
 
 if os.name == 'posix':
     if os.path.isdir('/sys/devices/system/node'):
         numa_nodes = glob.glob('/sys/devices/system/node/node*')
-        numa_nodes.sort()
+        numa_nodes.sort(key=lambda l: int(re.findall('\d+', l)[0]))
         print(int(os.path.basename(numa_nodes[-1])[4:]) + 1)
     else:
         subprocess.run(['sysctl', '-n', 'vm.ndomains'], check=False)