diff mbox series

[v1] buildtools: ensure the NUMA nodes are counted correct

Message ID	1659426855-11014-1-git-send-email-chaoyong.he@corigine.com (mailing list archive)
State	Superseded, archived
Delegated to:	Thomas Monjalon
Headers	From: Chaoyong He <chaoyong.he@corigine.com> To: dev@dpdk.org Cc: oss-drivers@corigine.com, niklas.soderlund@corigine.com, Peng Zhang <peng.zhang@corigine.com>, stable@dpdk.org, Chaoyong He <chaoyong.he@corigine.com> Subject: [PATCH v1] buildtools: ensure the NUMA nodes are counted correct Date: Tue, 2 Aug 2022 15:54:15 +0800 Message-Id: <1659426855-11014-1-git-send-email-chaoyong.he@corigine.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MIME-Version: 1.0 Precedence: list Errors-To: dev-bounces@dpdk.org
Series	[v1] buildtools: ensure the NUMA nodes are counted correct \| [v1] buildtools: ensure the NUMA nodes are counted correct

Checks

Context	Check	Description
ci/checkpatch	success	coding style OK
ci/Intel-compilation	success	Compilation OK
ci/intel-Testing	success	Testing PASS
ci/github-robot: build	success	github build: passed
ci/iol-aarch64-compile-testing	success	Testing PASS
ci/iol-aarch64-unit-testing	success	Testing PASS
ci/iol-mellanox-Performance	success	Performance Testing PASS
ci/iol-x86_64-unit-testing	success	Testing PASS
ci/iol-x86_64-compile-testing	success	Testing PASS
ci/iol-intel-Functional	success	Functional Testing PASS
ci/iol-intel-Performance	success	Performance Testing PASS

Commit Message

Chaoyong He Aug. 2, 2022, 7:54 a.m. UTC

  From: Peng Zhang <peng.zhang@corigine.com>

Sorting a list of strings with the format "node[0-9]+" in order to find the
largest integer by looking at the last item after the sort breaks. But if
there are more then 10 items as a string sort will sort "node10" before
"node2", it will get the error NUMA nodes.

Solve this by sorting the list based on the integer part of the string.

Fixes: 8ef09fdc506b ("build: add optional NUMA and CPU counts detection")
Cc: stable@dpdk.org

Signed-off-by: Peng Zhang <peng.zhang@corigine.com>
Signed-off-by: Chaoyong He <chaoyong.he@corigine.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@corigine.com>
---
 buildtools/get-numa-count.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

Comments

Thomas Monjalon Aug. 29, 2022, 10:41 a.m. UTC | #1

02/08/2022 09:54, Chaoyong He:
> From: Peng Zhang <peng.zhang@corigine.com>
> 
> Sorting a list of strings with the format "node[0-9]+" in order to find the
> largest integer by looking at the last item after the sort breaks. But if
> there are more then 10 items as a string sort will sort "node10" before
> "node2", it will get the error NUMA nodes.

What is the error you are seeing?

Nole Zhang Aug. 31, 2022, 8:47 a.m. UTC | #2

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: 2022年8月29日 21:15
> To: Nole Zhang <peng.zhang@corigine.com>; Chaoyong He 
> <chaoyong.he@corigine.com>
> Subject: Re: [PATCH v1] buildtools: ensure the NUMA nodes are counted 
> correct
> 
> 29/08/2022 13:17, Nole Zhang:
> > From: Thomas Monjalon <thomas@monjalon.net>
> > > 02/08/2022 09:54, Chaoyong He:
> > > > From: Peng Zhang <peng.zhang@corigine.com>
> > > >
> > > > Sorting a list of strings with the format "node[0-9]+" in order 
> > > > to find the largest integer by looking at the last item after 
> > > > the sort breaks. But if there are more then 10 items as a string 
> > > > sort will sort "node10" before "node2", it will get the error NUMA nodes.
> > >
> > > What is the error you are seeing?
> > >
> > >
> > We get the error NUMA, in this example, we get the NUMA nodes is 10, 
> > But at fact, it has 11 NUMA.
> 
> Please give more details, where do you see this error?
> We should know how to reproduce and check we have the same issue.
> Thanks
> 
> Please reply with a detailed answer on the mailing list.
> 
In the China Phytium S2500 CPU + INSPUR server, it has 16 NUMA.
The details are as follows:

~#: lscpu

Architecture:        aarch64
CPU op-mode(s):      64-bit
Byte Order:          Little Endian
CPU(s):              128
On-line CPU(s) list: 0-127
Thread(s) per core:  1
Core(s) per socket:  64
Socket(s):           2
NUMA node(s):        16
Vendor ID:           0x70
Model:               3
Model name:          S2500
Stepping:            0x1
BogoMIPS:            100.00
L1d cache:           4 MiB
L1i cache:           4 MiB
L2 cache:            64 MiB
L3 cache:            128 MiB
NUMA node0 CPU(s):   0-7
NUMA node1 CPU(s):   8-15
NUMA node2 CPU(s):   16-23
NUMA node3 CPU(s):   24-31
NUMA node4 CPU(s):   32-39
NUMA node5 CPU(s):   40-47
NUMA node6 CPU(s):   48-55
NUMA node7 CPU(s):   56-63
NUMA node8 CPU(s):   64-71
NUMA node9 CPU(s):   72-79
NUMA node10 CPU(s):  80-87
NUMA node11 CPU(s):  88-95
NUMA node12 CPU(s):  96-103
NUMA node13 CPU(s):  104-111
NUMA node14 CPU(s):  112-119
NUMA node15 CPU(s):  120-127
Flags:               half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt lpae evtstrm


I use  meson build -Dmax_lcores=detect -Dmax_numa_nodes=detect to compile, then dpdk initialization only shows 10 numa.

Niklas Söderlund Sept. 20, 2022, 10:11 a.m. UTC | #3

Hi Thomas,

Have you checked if this address the same issue you where seeing? Do you 
think we can move forward with this fix?

On 2022-08-31 10:47:24 +0200, Nole Zhang wrote:
> 
> 
> 
> > -----Original Message-----
> > From: Thomas Monjalon <thomas@monjalon.net>
> > Sent: 2022年8月29日 21:15
> > To: Nole Zhang <peng.zhang@corigine.com>; Chaoyong He 
> > <chaoyong.he@corigine.com>
> > Subject: Re: [PATCH v1] buildtools: ensure the NUMA nodes are counted 
> > correct
> > 
> > 29/08/2022 13:17, Nole Zhang:
> > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > 02/08/2022 09:54, Chaoyong He:
> > > > > From: Peng Zhang <peng.zhang@corigine.com>
> > > > >
> > > > > Sorting a list of strings with the format "node[0-9]+" in order 
> > > > > to find the largest integer by looking at the last item after 
> > > > > the sort breaks. But if there are more then 10 items as a string 
> > > > > sort will sort "node10" before "node2", it will get the error NUMA nodes.
> > > >
> > > > What is the error you are seeing?
> > > >
> > > >
> > > We get the error NUMA, in this example, we get the NUMA nodes is 10, 
> > > But at fact, it has 11 NUMA.
> > 
> > Please give more details, where do you see this error?
> > We should know how to reproduce and check we have the same issue.
> > Thanks
> > 
> > Please reply with a detailed answer on the mailing list.
> > 
> In the China Phytium S2500 CPU + INSPUR server, it has 16 NUMA.
> The details are as follows:
> 
> ~#: lscpu
> 
> Architecture:        aarch64
> CPU op-mode(s):      64-bit
> Byte Order:          Little Endian
> CPU(s):              128
> On-line CPU(s) list: 0-127
> Thread(s) per core:  1
> Core(s) per socket:  64
> Socket(s):           2
> NUMA node(s):        16
> Vendor ID:           0x70
> Model:               3
> Model name:          S2500
> Stepping:            0x1
> BogoMIPS:            100.00
> L1d cache:           4 MiB
> L1i cache:           4 MiB
> L2 cache:            64 MiB
> L3 cache:            128 MiB
> NUMA node0 CPU(s):   0-7
> NUMA node1 CPU(s):   8-15
> NUMA node2 CPU(s):   16-23
> NUMA node3 CPU(s):   24-31
> NUMA node4 CPU(s):   32-39
> NUMA node5 CPU(s):   40-47
> NUMA node6 CPU(s):   48-55
> NUMA node7 CPU(s):   56-63
> NUMA node8 CPU(s):   64-71
> NUMA node9 CPU(s):   72-79
> NUMA node10 CPU(s):  80-87
> NUMA node11 CPU(s):  88-95
> NUMA node12 CPU(s):  96-103
> NUMA node13 CPU(s):  104-111
> NUMA node14 CPU(s):  112-119
> NUMA node15 CPU(s):  120-127
> Flags:               half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt lpae evtstrm
> 
> 
> I use  meson build -Dmax_lcores=detect -Dmax_numa_nodes=detect to compile, then dpdk initialization only shows 10 numa.

Thomas Monjalon Sept. 20, 2022, 1:22 p.m. UTC | #4

Hi,

20/09/2022 12:11, Niklas Soderlund:
> Hi Thomas,
> 
> Have you checked if this address the same issue you where seeing? Do you 
> think we can move forward with this fix?

Yes. No need to show lscpu output,
but the output of the DPDK initialization would be more interesting.


> On 2022-08-31 10:47:24 +0200, Nole Zhang wrote:
> > 
> > 
> > 
> > > -----Original Message-----
> > > From: Thomas Monjalon <thomas@monjalon.net>
> > > Sent: 2022年8月29日 21:15
> > > To: Nole Zhang <peng.zhang@corigine.com>; Chaoyong He 
> > > <chaoyong.he@corigine.com>
> > > Subject: Re: [PATCH v1] buildtools: ensure the NUMA nodes are counted 
> > > correct
> > > 
> > > 29/08/2022 13:17, Nole Zhang:
> > > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > > 02/08/2022 09:54, Chaoyong He:
> > > > > > From: Peng Zhang <peng.zhang@corigine.com>
> > > > > >
> > > > > > Sorting a list of strings with the format "node[0-9]+" in order 
> > > > > > to find the largest integer by looking at the last item after 
> > > > > > the sort breaks. But if there are more then 10 items as a string 
> > > > > > sort will sort "node10" before "node2", it will get the error NUMA nodes.
> > > > >
> > > > > What is the error you are seeing?
> > > > >
> > > > >
> > > > We get the error NUMA, in this example, we get the NUMA nodes is 10, 
> > > > But at fact, it has 11 NUMA.
> > > 
> > > Please give more details, where do you see this error?
> > > We should know how to reproduce and check we have the same issue.
> > > Thanks
> > > 
> > > Please reply with a detailed answer on the mailing list.
> > > 
> > In the China Phytium S2500 CPU + INSPUR server, it has 16 NUMA.
> > The details are as follows:
> > 
> > ~#: lscpu
> > 
> > Architecture:        aarch64
> > CPU op-mode(s):      64-bit
> > Byte Order:          Little Endian
> > CPU(s):              128
> > On-line CPU(s) list: 0-127
> > Thread(s) per core:  1
> > Core(s) per socket:  64
> > Socket(s):           2
> > NUMA node(s):        16
> > Vendor ID:           0x70
> > Model:               3
> > Model name:          S2500
> > Stepping:            0x1
> > BogoMIPS:            100.00
> > L1d cache:           4 MiB
> > L1i cache:           4 MiB
> > L2 cache:            64 MiB
> > L3 cache:            128 MiB
> > NUMA node0 CPU(s):   0-7
> > NUMA node1 CPU(s):   8-15
> > NUMA node2 CPU(s):   16-23
> > NUMA node3 CPU(s):   24-31
> > NUMA node4 CPU(s):   32-39
> > NUMA node5 CPU(s):   40-47
> > NUMA node6 CPU(s):   48-55
> > NUMA node7 CPU(s):   56-63
> > NUMA node8 CPU(s):   64-71
> > NUMA node9 CPU(s):   72-79
> > NUMA node10 CPU(s):  80-87
> > NUMA node11 CPU(s):  88-95
> > NUMA node12 CPU(s):  96-103
> > NUMA node13 CPU(s):  104-111
> > NUMA node14 CPU(s):  112-119
> > NUMA node15 CPU(s):  120-127
> > Flags:               half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt lpae evtstrm
> > 
> > 
> > I use  meson build -Dmax_lcores=detect -Dmax_numa_nodes=detect to compile, then dpdk initialization only shows 10 numa. 
> 
>

Nole Zhang Sept. 21, 2022, 7:19 a.m. UTC | #5

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: 2022年9月20日 21:23
> To: Niklas Soderlund <niklas.soderlund@corigine.com>
> Cc: Nole Zhang <peng.zhang@corigine.com>; dev@dpdk.org; Chaoyong He
> <chaoyong.he@corigine.com>; oss-drivers <oss-drivers@corigine.com>
> Subject: Re: FW: [PATCH v1] buildtools: ensure the NUMA nodes are counted
> correct
> 
> Hi,
> 
> 20/09/2022 12:11, Niklas Soderlund:
> > Hi Thomas,
> >
> > Have you checked if this address the same issue you where seeing? Do
> > you think we can move forward with this fix?
> 
> Yes. No need to show lscpu output,
> but the output of the DPDK initialization would be more interesting.
> 

OK, this is the DPDK initialization.

The original code, the dpdkd initialized info:
EAL: Detected CPU lcores: 128
EAL: Detected NUMA nodes: 10
EAL: Static memory layout is selected, amount of reserved memory can be adjusted with -m or --socket-mem
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: VFIO support initialized

Revised code, the dpdkd initialized info:
EAL: Detected CPU lcores: 128
EAL: Detected NUMA nodes: 16
EAL: Static memory layout is selected, amount of reserved memory can be adjusted with -m or --socket-mem
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: VFIO support initialized

> 
> > On 2022-08-31 10:47:24 +0200, Nole Zhang wrote:
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > Sent: 2022年8月29日 21:15
> > > > To: Nole Zhang <peng.zhang@corigine.com>; Chaoyong He
> > > > <chaoyong.he@corigine.com>
> > > > Subject: Re: [PATCH v1] buildtools: ensure the NUMA nodes are
> > > > counted correct
> > > >
> > > > 29/08/2022 13:17, Nole Zhang:
> > > > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > > > 02/08/2022 09:54, Chaoyong He:
> > > > > > > From: Peng Zhang <peng.zhang@corigine.com>
> > > > > > >
> > > > > > > Sorting a list of strings with the format "node[0-9]+" in
> > > > > > > order to find the largest integer by looking at the last
> > > > > > > item after the sort breaks. But if there are more then 10
> > > > > > > items as a string sort will sort "node10" before "node2", it will get
> the error NUMA nodes.
> > > > > >
> > > > > > What is the error you are seeing?
> > > > > >
> > > > > >
> > > > > We get the error NUMA, in this example, we get the NUMA nodes is
> > > > > 10, But at fact, it has 11 NUMA.
> > > >
> > > > Please give more details, where do you see this error?
> > > > We should know how to reproduce and check we have the same issue.
> > > > Thanks
> > > >
> > > > Please reply with a detailed answer on the mailing list.
> > > >
> > > In the China Phytium S2500 CPU + INSPUR server, it has 16 NUMA.
> > > The details are as follows:
> > >
> > > ~#: lscpu
> > >
> > > Architecture:        aarch64
> > > CPU op-mode(s):      64-bit
> > > Byte Order:          Little Endian
> > > CPU(s):              128
> > > On-line CPU(s) list: 0-127
> > > Thread(s) per core:  1
> > > Core(s) per socket:  64
> > > Socket(s):           2
> > > NUMA node(s):        16
> > > Vendor ID:           0x70
> > > Model:               3
> > > Model name:          S2500
> > > Stepping:            0x1
> > > BogoMIPS:            100.00
> > > L1d cache:           4 MiB
> > > L1i cache:           4 MiB
> > > L2 cache:            64 MiB
> > > L3 cache:            128 MiB
> > > NUMA node0 CPU(s):   0-7
> > > NUMA node1 CPU(s):   8-15
> > > NUMA node2 CPU(s):   16-23
> > > NUMA node3 CPU(s):   24-31
> > > NUMA node4 CPU(s):   32-39
> > > NUMA node5 CPU(s):   40-47
> > > NUMA node6 CPU(s):   48-55
> > > NUMA node7 CPU(s):   56-63
> > > NUMA node8 CPU(s):   64-71
> > > NUMA node9 CPU(s):   72-79
> > > NUMA node10 CPU(s):  80-87
> > > NUMA node11 CPU(s):  88-95
> > > NUMA node12 CPU(s):  96-103
> > > NUMA node13 CPU(s):  104-111
> > > NUMA node14 CPU(s):  112-119
> > > NUMA node15 CPU(s):  120-127
> > > Flags:               half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt
> lpae evtstrm
> > >
> > >
> > > I use  meson build -Dmax_lcores=detect -Dmax_numa_nodes=detect to
> compile, then dpdk initialization only shows 10 numa.
> >
> >
> 
> 
> 
>

Thomas Monjalon Sept. 21, 2022, 8:07 a.m. UTC | #6

Please send a v2 with the proper explanations.


21/09/2022 09:19, Nole Zhang:
> 
> > -----Original Message-----
> > From: Thomas Monjalon <thomas@monjalon.net>
> > Sent: 2022年9月20日 21:23
> > To: Niklas Soderlund <niklas.soderlund@corigine.com>
> > Cc: Nole Zhang <peng.zhang@corigine.com>; dev@dpdk.org; Chaoyong He
> > <chaoyong.he@corigine.com>; oss-drivers <oss-drivers@corigine.com>
> > Subject: Re: FW: [PATCH v1] buildtools: ensure the NUMA nodes are counted
> > correct
> > 
> > Hi,
> > 
> > 20/09/2022 12:11, Niklas Soderlund:
> > > Hi Thomas,
> > >
> > > Have you checked if this address the same issue you where seeing? Do
> > > you think we can move forward with this fix?
> > 
> > Yes. No need to show lscpu output,
> > but the output of the DPDK initialization would be more interesting.
> > 
> 
> OK, this is the DPDK initialization.
> 
> The original code, the dpdkd initialized info:
> EAL: Detected CPU lcores: 128
> EAL: Detected NUMA nodes: 10
> EAL: Static memory layout is selected, amount of reserved memory can be adjusted with -m or --socket-mem
> EAL: Detected static linkage of DPDK
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Selected IOVA mode 'VA'
> EAL: VFIO support initialized
> 
> Revised code, the dpdkd initialized info:
> EAL: Detected CPU lcores: 128
> EAL: Detected NUMA nodes: 16
> EAL: Static memory layout is selected, amount of reserved memory can be adjusted with -m or --socket-mem
> EAL: Detected static linkage of DPDK
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Selected IOVA mode 'VA'
> EAL: VFIO support initialized
> 
> > 
> > > On 2022-08-31 10:47:24 +0200, Nole Zhang wrote:
> > > >
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > > Sent: 2022年8月29日 21:15
> > > > > To: Nole Zhang <peng.zhang@corigine.com>; Chaoyong He
> > > > > <chaoyong.he@corigine.com>
> > > > > Subject: Re: [PATCH v1] buildtools: ensure the NUMA nodes are
> > > > > counted correct
> > > > >
> > > > > 29/08/2022 13:17, Nole Zhang:
> > > > > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > > > > 02/08/2022 09:54, Chaoyong He:
> > > > > > > > From: Peng Zhang <peng.zhang@corigine.com>
> > > > > > > >
> > > > > > > > Sorting a list of strings with the format "node[0-9]+" in
> > > > > > > > order to find the largest integer by looking at the last
> > > > > > > > item after the sort breaks. But if there are more then 10
> > > > > > > > items as a string sort will sort "node10" before "node2", it will get
> > the error NUMA nodes.
> > > > > > >
> > > > > > > What is the error you are seeing?
> > > > > > >
> > > > > > >
> > > > > > We get the error NUMA, in this example, we get the NUMA nodes is
> > > > > > 10, But at fact, it has 11 NUMA.
> > > > >
> > > > > Please give more details, where do you see this error?
> > > > > We should know how to reproduce and check we have the same issue.
> > > > > Thanks
> > > > >
> > > > > Please reply with a detailed answer on the mailing list.
> > > > >
> > > > In the China Phytium S2500 CPU + INSPUR server, it has 16 NUMA.
> > > > The details are as follows:
> > > >
> > > > ~#: lscpu
> > > >
> > > > Architecture:        aarch64
> > > > CPU op-mode(s):      64-bit
> > > > Byte Order:          Little Endian
> > > > CPU(s):              128
> > > > On-line CPU(s) list: 0-127
> > > > Thread(s) per core:  1
> > > > Core(s) per socket:  64
> > > > Socket(s):           2
> > > > NUMA node(s):        16
> > > > Vendor ID:           0x70
> > > > Model:               3
> > > > Model name:          S2500
> > > > Stepping:            0x1
> > > > BogoMIPS:            100.00
> > > > L1d cache:           4 MiB
> > > > L1i cache:           4 MiB
> > > > L2 cache:            64 MiB
> > > > L3 cache:            128 MiB
> > > > NUMA node0 CPU(s):   0-7
> > > > NUMA node1 CPU(s):   8-15
> > > > NUMA node2 CPU(s):   16-23
> > > > NUMA node3 CPU(s):   24-31
> > > > NUMA node4 CPU(s):   32-39
> > > > NUMA node5 CPU(s):   40-47
> > > > NUMA node6 CPU(s):   48-55
> > > > NUMA node7 CPU(s):   56-63
> > > > NUMA node8 CPU(s):   64-71
> > > > NUMA node9 CPU(s):   72-79
> > > > NUMA node10 CPU(s):  80-87
> > > > NUMA node11 CPU(s):  88-95
> > > > NUMA node12 CPU(s):  96-103
> > > > NUMA node13 CPU(s):  104-111
> > > > NUMA node14 CPU(s):  112-119
> > > > NUMA node15 CPU(s):  120-127
> > > > Flags:               half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt
> > lpae evtstrm
> > > >
> > > >
> > > > I use  meson build -Dmax_lcores=detect -Dmax_numa_nodes=detect to
> > compile, then dpdk initialization only shows 10 numa.
> > >
> > >
> > 
> > 
> > 
> > 
> 
>

diff mbox series

Patch

diff --git a/buildtools/get-numa-count.py b/buildtools/get-numa-count.py
index 1b7787787f..2f243886cd 100644
--- a/buildtools/get-numa-count.py
+++ b/buildtools/get-numa-count.py
@@ -6,11 +6,12 @@ 
 import glob
 import os
 import subprocess
+import re
 
 if os.name == 'posix':
     if os.path.isdir('/sys/devices/system/node'):
         numa_nodes = glob.glob('/sys/devices/system/node/node*')
-        numa_nodes.sort()
+        numa_nodes.sort(key=lambda l: int(re.findall('\d+', l)[0]))
         print(int(os.path.basename(numa_nodes[-1])[4:]) + 1)
     else:
         subprocess.run(['sysctl', '-n', 'vm.ndomains'], check=False)