[v1] buildtools: ensure the NUMA nodes are counted correct
Checks
Commit Message
From: Peng Zhang <peng.zhang@corigine.com>
Sorting a list of strings with the format "node[0-9]+" in order to find the
largest integer by looking at the last item after the sort breaks. But if
there are more then 10 items as a string sort will sort "node10" before
"node2", it will get the error NUMA nodes.
Solve this by sorting the list based on the integer part of the string.
Fixes: 8ef09fdc506b ("build: add optional NUMA and CPU counts detection")
Cc: stable@dpdk.org
Signed-off-by: Peng Zhang <peng.zhang@corigine.com>
Signed-off-by: Chaoyong He <chaoyong.he@corigine.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@corigine.com>
---
buildtools/get-numa-count.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
Comments
02/08/2022 09:54, Chaoyong He:
> From: Peng Zhang <peng.zhang@corigine.com>
>
> Sorting a list of strings with the format "node[0-9]+" in order to find the
> largest integer by looking at the last item after the sort breaks. But if
> there are more then 10 items as a string sort will sort "node10" before
> "node2", it will get the error NUMA nodes.
What is the error you are seeing?
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: 2022年8月29日 21:15
> To: Nole Zhang <peng.zhang@corigine.com>; Chaoyong He
> <chaoyong.he@corigine.com>
> Subject: Re: [PATCH v1] buildtools: ensure the NUMA nodes are counted
> correct
>
> 29/08/2022 13:17, Nole Zhang:
> > From: Thomas Monjalon <thomas@monjalon.net>
> > > 02/08/2022 09:54, Chaoyong He:
> > > > From: Peng Zhang <peng.zhang@corigine.com>
> > > >
> > > > Sorting a list of strings with the format "node[0-9]+" in order
> > > > to find the largest integer by looking at the last item after
> > > > the sort breaks. But if there are more then 10 items as a string
> > > > sort will sort "node10" before "node2", it will get the error NUMA nodes.
> > >
> > > What is the error you are seeing?
> > >
> > >
> > We get the error NUMA, in this example, we get the NUMA nodes is 10,
> > But at fact, it has 11 NUMA.
>
> Please give more details, where do you see this error?
> We should know how to reproduce and check we have the same issue.
> Thanks
>
> Please reply with a detailed answer on the mailing list.
>
In the China Phytium S2500 CPU + INSPUR server, it has 16 NUMA.
The details are as follows:
~#: lscpu
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 128
On-line CPU(s) list: 0-127
Thread(s) per core: 1
Core(s) per socket: 64
Socket(s): 2
NUMA node(s): 16
Vendor ID: 0x70
Model: 3
Model name: S2500
Stepping: 0x1
BogoMIPS: 100.00
L1d cache: 4 MiB
L1i cache: 4 MiB
L2 cache: 64 MiB
L3 cache: 128 MiB
NUMA node0 CPU(s): 0-7
NUMA node1 CPU(s): 8-15
NUMA node2 CPU(s): 16-23
NUMA node3 CPU(s): 24-31
NUMA node4 CPU(s): 32-39
NUMA node5 CPU(s): 40-47
NUMA node6 CPU(s): 48-55
NUMA node7 CPU(s): 56-63
NUMA node8 CPU(s): 64-71
NUMA node9 CPU(s): 72-79
NUMA node10 CPU(s): 80-87
NUMA node11 CPU(s): 88-95
NUMA node12 CPU(s): 96-103
NUMA node13 CPU(s): 104-111
NUMA node14 CPU(s): 112-119
NUMA node15 CPU(s): 120-127
Flags: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt lpae evtstrm
I use meson build -Dmax_lcores=detect -Dmax_numa_nodes=detect to compile, then dpdk initialization only shows 10 numa.
Hi Thomas,
Have you checked if this address the same issue you where seeing? Do you
think we can move forward with this fix?
On 2022-08-31 10:47:24 +0200, Nole Zhang wrote:
>
>
>
> > -----Original Message-----
> > From: Thomas Monjalon <thomas@monjalon.net>
> > Sent: 2022年8月29日 21:15
> > To: Nole Zhang <peng.zhang@corigine.com>; Chaoyong He
> > <chaoyong.he@corigine.com>
> > Subject: Re: [PATCH v1] buildtools: ensure the NUMA nodes are counted
> > correct
> >
> > 29/08/2022 13:17, Nole Zhang:
> > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > 02/08/2022 09:54, Chaoyong He:
> > > > > From: Peng Zhang <peng.zhang@corigine.com>
> > > > >
> > > > > Sorting a list of strings with the format "node[0-9]+" in order
> > > > > to find the largest integer by looking at the last item after
> > > > > the sort breaks. But if there are more then 10 items as a string
> > > > > sort will sort "node10" before "node2", it will get the error NUMA nodes.
> > > >
> > > > What is the error you are seeing?
> > > >
> > > >
> > > We get the error NUMA, in this example, we get the NUMA nodes is 10,
> > > But at fact, it has 11 NUMA.
> >
> > Please give more details, where do you see this error?
> > We should know how to reproduce and check we have the same issue.
> > Thanks
> >
> > Please reply with a detailed answer on the mailing list.
> >
> In the China Phytium S2500 CPU + INSPUR server, it has 16 NUMA.
> The details are as follows:
>
> ~#: lscpu
>
> Architecture: aarch64
> CPU op-mode(s): 64-bit
> Byte Order: Little Endian
> CPU(s): 128
> On-line CPU(s) list: 0-127
> Thread(s) per core: 1
> Core(s) per socket: 64
> Socket(s): 2
> NUMA node(s): 16
> Vendor ID: 0x70
> Model: 3
> Model name: S2500
> Stepping: 0x1
> BogoMIPS: 100.00
> L1d cache: 4 MiB
> L1i cache: 4 MiB
> L2 cache: 64 MiB
> L3 cache: 128 MiB
> NUMA node0 CPU(s): 0-7
> NUMA node1 CPU(s): 8-15
> NUMA node2 CPU(s): 16-23
> NUMA node3 CPU(s): 24-31
> NUMA node4 CPU(s): 32-39
> NUMA node5 CPU(s): 40-47
> NUMA node6 CPU(s): 48-55
> NUMA node7 CPU(s): 56-63
> NUMA node8 CPU(s): 64-71
> NUMA node9 CPU(s): 72-79
> NUMA node10 CPU(s): 80-87
> NUMA node11 CPU(s): 88-95
> NUMA node12 CPU(s): 96-103
> NUMA node13 CPU(s): 104-111
> NUMA node14 CPU(s): 112-119
> NUMA node15 CPU(s): 120-127
> Flags: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt lpae evtstrm
>
>
> I use meson build -Dmax_lcores=detect -Dmax_numa_nodes=detect to compile, then dpdk initialization only shows 10 numa.
Hi,
20/09/2022 12:11, Niklas Soderlund:
> Hi Thomas,
>
> Have you checked if this address the same issue you where seeing? Do you
> think we can move forward with this fix?
Yes. No need to show lscpu output,
but the output of the DPDK initialization would be more interesting.
> On 2022-08-31 10:47:24 +0200, Nole Zhang wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Thomas Monjalon <thomas@monjalon.net>
> > > Sent: 2022年8月29日 21:15
> > > To: Nole Zhang <peng.zhang@corigine.com>; Chaoyong He
> > > <chaoyong.he@corigine.com>
> > > Subject: Re: [PATCH v1] buildtools: ensure the NUMA nodes are counted
> > > correct
> > >
> > > 29/08/2022 13:17, Nole Zhang:
> > > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > > 02/08/2022 09:54, Chaoyong He:
> > > > > > From: Peng Zhang <peng.zhang@corigine.com>
> > > > > >
> > > > > > Sorting a list of strings with the format "node[0-9]+" in order
> > > > > > to find the largest integer by looking at the last item after
> > > > > > the sort breaks. But if there are more then 10 items as a string
> > > > > > sort will sort "node10" before "node2", it will get the error NUMA nodes.
> > > > >
> > > > > What is the error you are seeing?
> > > > >
> > > > >
> > > > We get the error NUMA, in this example, we get the NUMA nodes is 10,
> > > > But at fact, it has 11 NUMA.
> > >
> > > Please give more details, where do you see this error?
> > > We should know how to reproduce and check we have the same issue.
> > > Thanks
> > >
> > > Please reply with a detailed answer on the mailing list.
> > >
> > In the China Phytium S2500 CPU + INSPUR server, it has 16 NUMA.
> > The details are as follows:
> >
> > ~#: lscpu
> >
> > Architecture: aarch64
> > CPU op-mode(s): 64-bit
> > Byte Order: Little Endian
> > CPU(s): 128
> > On-line CPU(s) list: 0-127
> > Thread(s) per core: 1
> > Core(s) per socket: 64
> > Socket(s): 2
> > NUMA node(s): 16
> > Vendor ID: 0x70
> > Model: 3
> > Model name: S2500
> > Stepping: 0x1
> > BogoMIPS: 100.00
> > L1d cache: 4 MiB
> > L1i cache: 4 MiB
> > L2 cache: 64 MiB
> > L3 cache: 128 MiB
> > NUMA node0 CPU(s): 0-7
> > NUMA node1 CPU(s): 8-15
> > NUMA node2 CPU(s): 16-23
> > NUMA node3 CPU(s): 24-31
> > NUMA node4 CPU(s): 32-39
> > NUMA node5 CPU(s): 40-47
> > NUMA node6 CPU(s): 48-55
> > NUMA node7 CPU(s): 56-63
> > NUMA node8 CPU(s): 64-71
> > NUMA node9 CPU(s): 72-79
> > NUMA node10 CPU(s): 80-87
> > NUMA node11 CPU(s): 88-95
> > NUMA node12 CPU(s): 96-103
> > NUMA node13 CPU(s): 104-111
> > NUMA node14 CPU(s): 112-119
> > NUMA node15 CPU(s): 120-127
> > Flags: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt lpae evtstrm
> >
> >
> > I use meson build -Dmax_lcores=detect -Dmax_numa_nodes=detect to compile, then dpdk initialization only shows 10 numa.
>
>
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: 2022年9月20日 21:23
> To: Niklas Soderlund <niklas.soderlund@corigine.com>
> Cc: Nole Zhang <peng.zhang@corigine.com>; dev@dpdk.org; Chaoyong He
> <chaoyong.he@corigine.com>; oss-drivers <oss-drivers@corigine.com>
> Subject: Re: FW: [PATCH v1] buildtools: ensure the NUMA nodes are counted
> correct
>
> Hi,
>
> 20/09/2022 12:11, Niklas Soderlund:
> > Hi Thomas,
> >
> > Have you checked if this address the same issue you where seeing? Do
> > you think we can move forward with this fix?
>
> Yes. No need to show lscpu output,
> but the output of the DPDK initialization would be more interesting.
>
OK, this is the DPDK initialization.
The original code, the dpdkd initialized info:
EAL: Detected CPU lcores: 128
EAL: Detected NUMA nodes: 10
EAL: Static memory layout is selected, amount of reserved memory can be adjusted with -m or --socket-mem
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: VFIO support initialized
Revised code, the dpdkd initialized info:
EAL: Detected CPU lcores: 128
EAL: Detected NUMA nodes: 16
EAL: Static memory layout is selected, amount of reserved memory can be adjusted with -m or --socket-mem
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: VFIO support initialized
>
> > On 2022-08-31 10:47:24 +0200, Nole Zhang wrote:
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > Sent: 2022年8月29日 21:15
> > > > To: Nole Zhang <peng.zhang@corigine.com>; Chaoyong He
> > > > <chaoyong.he@corigine.com>
> > > > Subject: Re: [PATCH v1] buildtools: ensure the NUMA nodes are
> > > > counted correct
> > > >
> > > > 29/08/2022 13:17, Nole Zhang:
> > > > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > > > 02/08/2022 09:54, Chaoyong He:
> > > > > > > From: Peng Zhang <peng.zhang@corigine.com>
> > > > > > >
> > > > > > > Sorting a list of strings with the format "node[0-9]+" in
> > > > > > > order to find the largest integer by looking at the last
> > > > > > > item after the sort breaks. But if there are more then 10
> > > > > > > items as a string sort will sort "node10" before "node2", it will get
> the error NUMA nodes.
> > > > > >
> > > > > > What is the error you are seeing?
> > > > > >
> > > > > >
> > > > > We get the error NUMA, in this example, we get the NUMA nodes is
> > > > > 10, But at fact, it has 11 NUMA.
> > > >
> > > > Please give more details, where do you see this error?
> > > > We should know how to reproduce and check we have the same issue.
> > > > Thanks
> > > >
> > > > Please reply with a detailed answer on the mailing list.
> > > >
> > > In the China Phytium S2500 CPU + INSPUR server, it has 16 NUMA.
> > > The details are as follows:
> > >
> > > ~#: lscpu
> > >
> > > Architecture: aarch64
> > > CPU op-mode(s): 64-bit
> > > Byte Order: Little Endian
> > > CPU(s): 128
> > > On-line CPU(s) list: 0-127
> > > Thread(s) per core: 1
> > > Core(s) per socket: 64
> > > Socket(s): 2
> > > NUMA node(s): 16
> > > Vendor ID: 0x70
> > > Model: 3
> > > Model name: S2500
> > > Stepping: 0x1
> > > BogoMIPS: 100.00
> > > L1d cache: 4 MiB
> > > L1i cache: 4 MiB
> > > L2 cache: 64 MiB
> > > L3 cache: 128 MiB
> > > NUMA node0 CPU(s): 0-7
> > > NUMA node1 CPU(s): 8-15
> > > NUMA node2 CPU(s): 16-23
> > > NUMA node3 CPU(s): 24-31
> > > NUMA node4 CPU(s): 32-39
> > > NUMA node5 CPU(s): 40-47
> > > NUMA node6 CPU(s): 48-55
> > > NUMA node7 CPU(s): 56-63
> > > NUMA node8 CPU(s): 64-71
> > > NUMA node9 CPU(s): 72-79
> > > NUMA node10 CPU(s): 80-87
> > > NUMA node11 CPU(s): 88-95
> > > NUMA node12 CPU(s): 96-103
> > > NUMA node13 CPU(s): 104-111
> > > NUMA node14 CPU(s): 112-119
> > > NUMA node15 CPU(s): 120-127
> > > Flags: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt
> lpae evtstrm
> > >
> > >
> > > I use meson build -Dmax_lcores=detect -Dmax_numa_nodes=detect to
> compile, then dpdk initialization only shows 10 numa.
> >
> >
>
>
>
>
Please send a v2 with the proper explanations.
21/09/2022 09:19, Nole Zhang:
>
> > -----Original Message-----
> > From: Thomas Monjalon <thomas@monjalon.net>
> > Sent: 2022年9月20日 21:23
> > To: Niklas Soderlund <niklas.soderlund@corigine.com>
> > Cc: Nole Zhang <peng.zhang@corigine.com>; dev@dpdk.org; Chaoyong He
> > <chaoyong.he@corigine.com>; oss-drivers <oss-drivers@corigine.com>
> > Subject: Re: FW: [PATCH v1] buildtools: ensure the NUMA nodes are counted
> > correct
> >
> > Hi,
> >
> > 20/09/2022 12:11, Niklas Soderlund:
> > > Hi Thomas,
> > >
> > > Have you checked if this address the same issue you where seeing? Do
> > > you think we can move forward with this fix?
> >
> > Yes. No need to show lscpu output,
> > but the output of the DPDK initialization would be more interesting.
> >
>
> OK, this is the DPDK initialization.
>
> The original code, the dpdkd initialized info:
> EAL: Detected CPU lcores: 128
> EAL: Detected NUMA nodes: 10
> EAL: Static memory layout is selected, amount of reserved memory can be adjusted with -m or --socket-mem
> EAL: Detected static linkage of DPDK
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Selected IOVA mode 'VA'
> EAL: VFIO support initialized
>
> Revised code, the dpdkd initialized info:
> EAL: Detected CPU lcores: 128
> EAL: Detected NUMA nodes: 16
> EAL: Static memory layout is selected, amount of reserved memory can be adjusted with -m or --socket-mem
> EAL: Detected static linkage of DPDK
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Selected IOVA mode 'VA'
> EAL: VFIO support initialized
>
> >
> > > On 2022-08-31 10:47:24 +0200, Nole Zhang wrote:
> > > >
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > > Sent: 2022年8月29日 21:15
> > > > > To: Nole Zhang <peng.zhang@corigine.com>; Chaoyong He
> > > > > <chaoyong.he@corigine.com>
> > > > > Subject: Re: [PATCH v1] buildtools: ensure the NUMA nodes are
> > > > > counted correct
> > > > >
> > > > > 29/08/2022 13:17, Nole Zhang:
> > > > > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > > > > 02/08/2022 09:54, Chaoyong He:
> > > > > > > > From: Peng Zhang <peng.zhang@corigine.com>
> > > > > > > >
> > > > > > > > Sorting a list of strings with the format "node[0-9]+" in
> > > > > > > > order to find the largest integer by looking at the last
> > > > > > > > item after the sort breaks. But if there are more then 10
> > > > > > > > items as a string sort will sort "node10" before "node2", it will get
> > the error NUMA nodes.
> > > > > > >
> > > > > > > What is the error you are seeing?
> > > > > > >
> > > > > > >
> > > > > > We get the error NUMA, in this example, we get the NUMA nodes is
> > > > > > 10, But at fact, it has 11 NUMA.
> > > > >
> > > > > Please give more details, where do you see this error?
> > > > > We should know how to reproduce and check we have the same issue.
> > > > > Thanks
> > > > >
> > > > > Please reply with a detailed answer on the mailing list.
> > > > >
> > > > In the China Phytium S2500 CPU + INSPUR server, it has 16 NUMA.
> > > > The details are as follows:
> > > >
> > > > ~#: lscpu
> > > >
> > > > Architecture: aarch64
> > > > CPU op-mode(s): 64-bit
> > > > Byte Order: Little Endian
> > > > CPU(s): 128
> > > > On-line CPU(s) list: 0-127
> > > > Thread(s) per core: 1
> > > > Core(s) per socket: 64
> > > > Socket(s): 2
> > > > NUMA node(s): 16
> > > > Vendor ID: 0x70
> > > > Model: 3
> > > > Model name: S2500
> > > > Stepping: 0x1
> > > > BogoMIPS: 100.00
> > > > L1d cache: 4 MiB
> > > > L1i cache: 4 MiB
> > > > L2 cache: 64 MiB
> > > > L3 cache: 128 MiB
> > > > NUMA node0 CPU(s): 0-7
> > > > NUMA node1 CPU(s): 8-15
> > > > NUMA node2 CPU(s): 16-23
> > > > NUMA node3 CPU(s): 24-31
> > > > NUMA node4 CPU(s): 32-39
> > > > NUMA node5 CPU(s): 40-47
> > > > NUMA node6 CPU(s): 48-55
> > > > NUMA node7 CPU(s): 56-63
> > > > NUMA node8 CPU(s): 64-71
> > > > NUMA node9 CPU(s): 72-79
> > > > NUMA node10 CPU(s): 80-87
> > > > NUMA node11 CPU(s): 88-95
> > > > NUMA node12 CPU(s): 96-103
> > > > NUMA node13 CPU(s): 104-111
> > > > NUMA node14 CPU(s): 112-119
> > > > NUMA node15 CPU(s): 120-127
> > > > Flags: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt
> > lpae evtstrm
> > > >
> > > >
> > > > I use meson build -Dmax_lcores=detect -Dmax_numa_nodes=detect to
> > compile, then dpdk initialization only shows 10 numa.
> > >
> > >
> >
> >
> >
> >
>
>
@@ -6,11 +6,12 @@
import glob
import os
import subprocess
+import re
if os.name == 'posix':
if os.path.isdir('/sys/devices/system/node'):
numa_nodes = glob.glob('/sys/devices/system/node/node*')
- numa_nodes.sort()
+ numa_nodes.sort(key=lambda l: int(re.findall('\d+', l)[0]))
print(int(os.path.basename(numa_nodes[-1])[4:]) + 1)
else:
subprocess.run(['sysctl', '-n', 'vm.ndomains'], check=False)