usertools: enhance logic to display NUMA
Checks
Commit Message
enhance python logic to accomadate NUMA information. Current logic
considers physical socket with CPU threads to core map. With new
AMD SKU vairant NUMA is no longer same as SOCKET. Single physical
socket can be partitioned to variant of 1,2 and 4.
The changes address the new mapping with Socket-NUMA to CPU cores.
Signed-off-by: Vipin Varghese <vipin.varghese@amd.com>
---
usertools/cpu_layout.py | 76 +++++++++++++++++++++++++----------------
1 file changed, 47 insertions(+), 29 deletions(-)
Comments
26/03/2022 08:32, Vipin Varghese:
> enhance python logic to accomadate NUMA information. Current logic
> considers physical socket with CPU threads to core map. With new
> AMD SKU vairant NUMA is no longer same as SOCKET. Single physical
> socket can be partitioned to variant of 1,2 and 4.
>
> The changes address the new mapping with Socket-NUMA to CPU cores.
>
> Signed-off-by: Vipin Varghese <vipin.varghese@amd.com>
> ---
> usertools/cpu_layout.py | 76 +++++++++++++++++++++++++----------------
> 1 file changed, 47 insertions(+), 29 deletions(-)
Honestly, I'm not sure it is a good idea to keep this script in the DPDK repo.
Can it be replaced with hwloc usage?
What is the output on the new AMD SKU for this command?
lstopo-no-graphics --merge
[AMD Official Use Only]
Hi Thomas,
<snipp>
26/03/2022 08:32, Vipin Varghese:
> enhance python logic to accomadate NUMA information. Current logic
> considers physical socket with CPU threads to core map. With new AMD
> SKU vairant NUMA is no longer same as SOCKET. Single physical socket
> can be partitioned to variant of 1,2 and 4.
>
> The changes address the new mapping with Socket-NUMA to CPU cores.
>
> Signed-off-by: Vipin Varghese <vipin.varghese@amd.com>
> ---
> usertools/cpu_layout.py | 76
> +++++++++++++++++++++++++----------------
> 1 file changed, 47 insertions(+), 29 deletions(-)
Honestly, I'm not sure it is a good idea to keep this script in the DPDK repo.
Can it be replaced with hwloc usage?
thanks for the suggestion, it is genuine and useful. Following is my observations
- It takes some effort to identify the NUMA with `Group`
- One needs to install ` lstopo-no-graphics` on distro and manually build and add on custom Linux.
What is the output on the new AMD SKU for this command?
lstopo-no-graphics --merge
I have tried ` lstopo-no-graphics --merge` on a ` 2 Socket AMD EPYC 7713 64-Core Processor` with
possible NUMA configuration such as 1, 2 and 4.
```
$ lstopo-no-graphics --merge
Machine (503GB total)
Package L#0
Group0 L#0
NUMANode L#0 (P#0 126GB)
L3 L#0 (32MB)
Core L#0
PU L#0 (P#0)
PU L#1 (P#128)
Core L#1
PU L#2 (P#1)
PU L#3 (P#129)
Core L#2
PU L#4 (P#2)
PU L#5 (P#130)
Core L#3
PU L#6 (P#3)
PU L#7 (P#131)
Core L#4
PU L#8 (P#4)
PU L#9 (P#132)
Core L#5
PU L#10 (P#5)
PU L#11 (P#133)
Core L#6
PU L#12 (P#6)
PU L#13 (P#134)
Core L#7
PU L#14 (P#7)
PU L#15 (P#135)
L3 L#1 (32MB)
Core L#8
PU L#16 (P#8)
PU L#17 (P#136)
Core L#9
PU L#18 (P#9)
PU L#19 (P#137)
Core L#10
PU L#20 (P#10)
PU L#21 (P#138)
Core L#11
PU L#22 (P#11)
PU L#23 (P#139)
Core L#12
PU L#24 (P#12)
PU L#25 (P#140)
Core L#13
PU L#26 (P#13)
PU L#27 (P#141)
Core L#14
PU L#28 (P#14)
PU L#29 (P#142)
Core L#15
PU L#30 (P#15)
PU L#31 (P#143)
L3 L#2 (32MB)
Core L#16
PU L#32 (P#16)
PU L#33 (P#144)
Core L#17
PU L#34 (P#17)
PU L#35 (P#145)
Core L#18
PU L#36 (P#18)
PU L#37 (P#146)
Core L#19
PU L#38 (P#19)
PU L#39 (P#147)
Core L#20
PU L#40 (P#20)
PU L#41 (P#148)
Core L#21
PU L#42 (P#21)
PU L#43 (P#149)
Core L#22
PU L#44 (P#22)
PU L#45 (P#150)
Core L#23
PU L#46 (P#23)
PU L#47 (P#151)
L3 L#3 (32MB)
Core L#24
PU L#48 (P#24)
PU L#49 (P#152)
Core L#25
PU L#50 (P#25)
PU L#51 (P#153)
Core L#26
PU L#52 (P#26)
PU L#53 (P#154)
Core L#27
PU L#54 (P#27)
PU L#55 (P#155)
Core L#28
PU L#56 (P#28)
PU L#57 (P#156)
Core L#29
PU L#58 (P#29)
PU L#59 (P#157)
Core L#30
PU L#60 (P#30)
PU L#61 (P#158)
Core L#31
PU L#62 (P#31)
PU L#63 (P#159)
HostBridge
PCIBridge
PCI 41:00.0 (Ethernet)
Net "ens1f0"
OpenFabrics "mlx5_0"
PCI 41:00.1 (Ethernet)
Net "ens1f1"
OpenFabrics "mlx5_1"
HostBridge
PCIBridge
PCI 63:00.0 (Ethernet)
Net "eno12399np0"
PCI 63:00.1 (Ethernet)
Net "eno12409np1"
PCIBridge
PCIBridge
PCI 62:00.0 (VGA)
Group0 L#1
NUMANode L#1 (P#1 126GB)
L3 L#4 (32MB)
Core L#32
PU L#64 (P#32)
PU L#65 (P#160)
Core L#33
PU L#66 (P#33)
PU L#67 (P#161)
Core L#34
PU L#68 (P#34)
PU L#69 (P#162)
Core L#35
PU L#70 (P#35)
PU L#71 (P#163)
Core L#36
PU L#72 (P#36)
PU L#73 (P#164)
Core L#37
PU L#74 (P#37)
PU L#75 (P#165)
Core L#38
PU L#76 (P#38)
PU L#77 (P#166)
Core L#39
PU L#78 (P#39)
PU L#79 (P#167)
L3 L#5 (32MB)
Core L#40
PU L#80 (P#40)
PU L#81 (P#168)
Core L#41
PU L#82 (P#41)
PU L#83 (P#169)
Core L#42
PU L#84 (P#42)
PU L#85 (P#170)
Core L#43
PU L#86 (P#43)
PU L#87 (P#171)
Core L#44
PU L#88 (P#44)
PU L#89 (P#172)
Core L#45
PU L#90 (P#45)
PU L#91 (P#173)
Core L#46
PU L#92 (P#46)
PU L#93 (P#174)
Core L#47
PU L#94 (P#47)
PU L#95 (P#175)
L3 L#6 (32MB)
Core L#48
PU L#96 (P#48)
PU L#97 (P#176)
Core L#49
PU L#98 (P#49)
PU L#99 (P#177)
Core L#50
PU L#100 (P#50)
PU L#101 (P#178)
Core L#51
PU L#102 (P#51)
PU L#103 (P#179)
Core L#52
PU L#104 (P#52)
PU L#105 (P#180)
Core L#53
PU L#106 (P#53)
PU L#107 (P#181)
Core L#54
PU L#108 (P#54)
PU L#109 (P#182)
Core L#55
PU L#110 (P#55)
PU L#111 (P#183)
L3 L#7 (32MB)
Core L#56
PU L#112 (P#56)
PU L#113 (P#184)
Core L#57
PU L#114 (P#57)
PU L#115 (P#185)
Core L#58
PU L#116 (P#58)
PU L#117 (P#186)
Core L#59
PU L#118 (P#59)
PU L#119 (P#187)
Core L#60
PU L#120 (P#60)
PU L#121 (P#188)
Core L#61
PU L#122 (P#61)
PU L#123 (P#189)
Core L#62
PU L#124 (P#62)
PU L#125 (P#190)
Core L#63
PU L#126 (P#63)
PU L#127 (P#191)
HostBridge
PCIBridge
PCI 01:00.0 (RAID)
PCIBridge
PCI 02:00.0 (SATA)
Block(Disk) "sda"
HostBridge
PCIBridge
PCI 22:00.0 (NVMExp)
Block(Disk) "nvme0n1"
Package L#1
Group0 L#2
NUMANode L#2 (P#2 126GB)
L3 L#8 (32MB)
Core L#64
PU L#128 (P#64)
PU L#129 (P#192)
Core L#65
PU L#130 (P#65)
PU L#131 (P#193)
Core L#66
PU L#132 (P#66)
PU L#133 (P#194)
Core L#67
PU L#134 (P#67)
PU L#135 (P#195)
Core L#68
PU L#136 (P#68)
PU L#137 (P#196)
Core L#69
PU L#138 (P#69)
PU L#139 (P#197)
Core L#70
PU L#140 (P#70)
PU L#141 (P#198)
Core L#71
PU L#142 (P#71)
PU L#143 (P#199)
L3 L#9 (32MB)
Core L#72
PU L#144 (P#72)
PU L#145 (P#200)
Core L#73
PU L#146 (P#73)
PU L#147 (P#201)
Core L#74
PU L#148 (P#74)
PU L#149 (P#202)
Core L#75
PU L#150 (P#75)
PU L#151 (P#203)
Core L#76
PU L#152 (P#76)
PU L#153 (P#204)
Core L#77
PU L#154 (P#77)
PU L#155 (P#205)
Core L#78
PU L#156 (P#78)
PU L#157 (P#206)
Core L#79
PU L#158 (P#79)
PU L#159 (P#207)
L3 L#10 (32MB)
Core L#80
PU L#160 (P#80)
PU L#161 (P#208)
Core L#81
PU L#162 (P#81)
PU L#163 (P#209)
Core L#82
PU L#164 (P#82)
PU L#165 (P#210)
Core L#83
PU L#166 (P#83)
PU L#167 (P#211)
Core L#84
PU L#168 (P#84)
PU L#169 (P#212)
Core L#85
PU L#170 (P#85)
PU L#171 (P#213)
Core L#86
PU L#172 (P#86)
PU L#173 (P#214)
Core L#87
PU L#174 (P#87)
PU L#175 (P#215)
L3 L#11 (32MB)
Core L#88
PU L#176 (P#88)
PU L#177 (P#216)
Core L#89
PU L#178 (P#89)
PU L#179 (P#217)
Core L#90
PU L#180 (P#90)
PU L#181 (P#218)
Core L#91
PU L#182 (P#91)
PU L#183 (P#219)
Core L#92
PU L#184 (P#92)
PU L#185 (P#220)
Core L#93
PU L#186 (P#93)
PU L#187 (P#221)
Core L#94
PU L#188 (P#94)
PU L#189 (P#222)
Core L#95
PU L#190 (P#95)
PU L#191 (P#223)
HostBridge
PCIBridge
PCI c5:00.0 (SATA)
HostBridge
PCIBridge
PCI e1:00.0 (Ethernet)
Net "eno8303"
PCI e1:00.1 (Ethernet)
Net "eno8403"
Group0 L#3
NUMANode L#3 (P#3 126GB)
L3 L#12 (32MB)
Core L#96
PU L#192 (P#96)
PU L#193 (P#224)
Core L#97
PU L#194 (P#97)
PU L#195 (P#225)
Core L#98
PU L#196 (P#98)
PU L#197 (P#226)
Core L#99
PU L#198 (P#99)
PU L#199 (P#227)
Core L#100
PU L#200 (P#100)
PU L#201 (P#228)
Core L#101
PU L#202 (P#101)
PU L#203 (P#229)
Core L#102
PU L#204 (P#102)
PU L#205 (P#230)
Core L#103
PU L#206 (P#103)
PU L#207 (P#231)
L3 L#13 (32MB)
Core L#104
PU L#208 (P#104)
PU L#209 (P#232)
Core L#105
PU L#210 (P#105)
PU L#211 (P#233)
Core L#106
PU L#212 (P#106)
PU L#213 (P#234)
Core L#107
PU L#214 (P#107)
PU L#215 (P#235)
Core L#108
PU L#216 (P#108)
PU L#217 (P#236)
Core L#109
PU L#218 (P#109)
PU L#219 (P#237)
Core L#110
PU L#220 (P#110)
PU L#221 (P#238)
Core L#111
PU L#222 (P#111)
PU L#223 (P#239)
L3 L#14 (32MB)
Core L#112
PU L#224 (P#112)
PU L#225 (P#240)
Core L#113
PU L#226 (P#113)
PU L#227 (P#241)
Core L#114
PU L#228 (P#114)
PU L#229 (P#242)
Core L#115
PU L#230 (P#115)
PU L#231 (P#243)
Core L#116
PU L#232 (P#116)
PU L#233 (P#244)
Core L#117
PU L#234 (P#117)
PU L#235 (P#245)
Core L#118
PU L#236 (P#118)
PU L#237 (P#246)
Core L#119
PU L#238 (P#119)
PU L#239 (P#247)
L3 L#15 (32MB)
Core L#120
PU L#240 (P#120)
PU L#241 (P#248)
Core L#121
PU L#242 (P#121)
PU L#243 (P#249)
Core L#122
PU L#244 (P#122)
PU L#245 (P#250)
Core L#123
PU L#246 (P#123)
PU L#247 (P#251)
Core L#124
PU L#248 (P#124)
PU L#249 (P#252)
Core L#125
PU L#250 (P#125)
PU L#251 (P#253)
Core L#126
PU L#252 (P#126)
PU L#253 (P#254)
Core L#127
PU L#254 (P#127)
PU L#255 (P#255)
HostBridge
PCIBridge
PCI 81:00.0 (Ethernet)
Net "ens3f0"
OpenFabrics "mlx5_2"
PCI 81:00.1 (Ethernet)
Net "ens3f1"
OpenFabrics "mlx5_3"
```
The current result gives
```
$ ./usertools/cpu_layout.py
======================================================================
Core and Socket Information (as reported by '/sys/devices/system/cpu')
======================================================================
cores = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63]
numa nodes per socket = [0, 1]
sockets = [0, 1]
Socket 0 Numa 0
-------------------
Core 0 [0, 128]
Core 1 [1, 129]
Core 2 [2, 130]
Core 3 [3, 131]
Core 4 [4, 132]
Core 5 [5, 133]
Core 6 [6, 134]
Core 7 [7, 135]
Core 8 [8, 136]
Core 9 [9, 137]
Core 10 [10, 138]
Core 11 [11, 139]
Core 12 [12, 140]
Core 13 [13, 141]
Core 14 [14, 142]
Core 15 [15, 143]
Core 16 [16, 144]
Core 17 [17, 145]
Core 18 [18, 146]
Core 19 [19, 147]
Core 20 [20, 148]
Core 21 [21, 149]
Core 22 [22, 150]
Core 23 [23, 151]
Core 24 [24, 152]
Core 25 [25, 153]
Core 26 [26, 154]
Core 27 [27, 155]
Core 28 [28, 156]
Core 29 [29, 157]
Core 30 [30, 158]
Core 31 [31, 159]
Socket 0 Numa 1
-------------------
Core 0 [32, 160]
Core 1 [33, 161]
Core 2 [34, 162]
Core 3 [35, 163]
Core 4 [36, 164]
Core 5 [37, 165]
Core 6 [38, 166]
Core 7 [39, 167]
Core 8 [40, 168]
Core 9 [41, 169]
Core 10 [42, 170]
Core 11 [43, 171]
Core 12 [44, 172]
Core 13 [45, 173]
Core 14 [46, 174]
Core 15 [47, 175]
Core 16 [48, 176]
Core 17 [49, 177]
Core 18 [50, 178]
Core 19 [51, 179]
Core 20 [52, 180]
Core 21 [53, 181]
Core 22 [54, 182]
Core 23 [55, 183]
Core 24 [56, 184]
Core 25 [57, 185]
Core 26 [58, 186]
Core 27 [59, 187]
Core 28 [60, 188]
Core 29 [61, 189]
Core 30 [62, 190]
Core 31 [63, 191]
Socket 1 Numa 2
-------------------
Core 0 [64, 192]
Core 1 [65, 193]
Core 2 [66, 194]
Core 3 [67, 195]
Core 4 [68, 196]
Core 5 [69, 197]
Core 6 [70, 198]
Core 7 [71, 199]
Core 8 [72, 200]
Core 9 [73, 201]
Core 10 [74, 202]
Core 11 [75, 203]
Core 12 [76, 204]
Core 13 [77, 205]
Core 14 [78, 206]
Core 15 [79, 207]
Core 16 [80, 208]
Core 17 [81, 209]
Core 18 [82, 210]
Core 19 [83, 211]
Core 20 [84, 212]
Core 21 [85, 213]
Core 22 [86, 214]
Core 23 [87, 215]
Core 24 [88, 216]
Core 25 [89, 217]
Core 26 [90, 218]
Core 27 [91, 219]
Core 28 [92, 220]
Core 29 [93, 221]
Core 30 [94, 222]
Core 31 [95, 223]
Socket 1 Numa 3
-------------------
Core 0 [96, 224]
Core 1 [97, 225]
Core 2 [98, 226]
Core 3 [99, 227]
Core 4 [100, 228]
Core 5 [101, 229]
Core 6 [102, 230]
Core 7 [103, 231]
Core 8 [104, 232]
Core 9 [105, 233]
Core 10 [106, 234]
Core 11 [107, 235]
Core 12 [108, 236]
Core 13 [109, 237]
Core 14 [110, 238]
Core 15 [111, 239]
Core 16 [112, 240]
Core 17 [113, 241]
Core 18 [114, 242]
Core 19 [115, 243]
Core 20 [116, 244]
Core 21 [117, 245]
Core 22 [118, 246]
Core 23 [119, 247]
Core 24 [120, 248]
Core 25 [121, 249]
Core 26 [122, 250]
Core 27 [123, 251]
Core 28 [124, 252]
Core 29 [125, 253]
Core 30 [126, 254]
Core 31 [127, 255]
```
[AMD Official Use Only]
Validated the changes on:
- AMD EPYC 7713
- AMD EPYC 7543P
- Intel(R) Xeon(R) Platinum 8352V
-----Original Message-----
From: Varghese, Vipin <Vipin.Varghese@amd.com>
Sent: Monday, March 28, 2022 8:27 AM
To: Thomas Monjalon <thomas@monjalon.net>
Cc: david.marchand@redhat.com; Tummala, Sivaprasad <Sivaprasad.Tummala@amd.com>; dev@dpdk.org
Subject: RE: [PATCH] usertools: enhance logic to display NUMA
[AMD Official Use Only]
Hi Thomas,
<snipp>
26/03/2022 08:32, Vipin Varghese:
> enhance python logic to accomadate NUMA information. Current logic
> considers physical socket with CPU threads to core map. With new AMD
> SKU vairant NUMA is no longer same as SOCKET. Single physical socket
> can be partitioned to variant of 1,2 and 4.
>
> The changes address the new mapping with Socket-NUMA to CPU cores.
>
> Signed-off-by: Vipin Varghese <vipin.varghese@amd.com>
> ---
> usertools/cpu_layout.py | 76
> +++++++++++++++++++++++++----------------
> 1 file changed, 47 insertions(+), 29 deletions(-)
Honestly, I'm not sure it is a good idea to keep this script in the DPDK repo.
Can it be replaced with hwloc usage?
thanks for the suggestion, it is genuine and useful. Following is my observations
- It takes some effort to identify the NUMA with `Group`
- One needs to install ` lstopo-no-graphics` on distro and manually build and add on custom Linux.
What is the output on the new AMD SKU for this command?
lstopo-no-graphics --merge
I have tried ` lstopo-no-graphics --merge` on a ` 2 Socket AMD EPYC 7713 64-Core Processor` with possible NUMA configuration such as 1, 2 and 4.
```
$ lstopo-no-graphics --merge
Machine (503GB total)
Package L#0
Group0 L#0
NUMANode L#0 (P#0 126GB)
L3 L#0 (32MB)
Core L#0
PU L#0 (P#0)
PU L#1 (P#128)
Core L#1
PU L#2 (P#1)
PU L#3 (P#129)
Core L#2
PU L#4 (P#2)
PU L#5 (P#130)
Core L#3
PU L#6 (P#3)
PU L#7 (P#131)
Core L#4
PU L#8 (P#4)
PU L#9 (P#132)
Core L#5
PU L#10 (P#5)
PU L#11 (P#133)
Core L#6
PU L#12 (P#6)
PU L#13 (P#134)
Core L#7
PU L#14 (P#7)
PU L#15 (P#135)
L3 L#1 (32MB)
Core L#8
PU L#16 (P#8)
PU L#17 (P#136)
Core L#9
PU L#18 (P#9)
PU L#19 (P#137)
Core L#10
PU L#20 (P#10)
PU L#21 (P#138)
Core L#11
PU L#22 (P#11)
PU L#23 (P#139)
Core L#12
PU L#24 (P#12)
PU L#25 (P#140)
Core L#13
PU L#26 (P#13)
PU L#27 (P#141)
Core L#14
PU L#28 (P#14)
PU L#29 (P#142)
Core L#15
PU L#30 (P#15)
PU L#31 (P#143)
L3 L#2 (32MB)
Core L#16
PU L#32 (P#16)
PU L#33 (P#144)
Core L#17
PU L#34 (P#17)
PU L#35 (P#145)
Core L#18
PU L#36 (P#18)
PU L#37 (P#146)
Core L#19
PU L#38 (P#19)
PU L#39 (P#147)
Core L#20
PU L#40 (P#20)
PU L#41 (P#148)
Core L#21
PU L#42 (P#21)
PU L#43 (P#149)
Core L#22
PU L#44 (P#22)
PU L#45 (P#150)
Core L#23
PU L#46 (P#23)
PU L#47 (P#151)
L3 L#3 (32MB)
Core L#24
PU L#48 (P#24)
PU L#49 (P#152)
Core L#25
PU L#50 (P#25)
PU L#51 (P#153)
Core L#26
PU L#52 (P#26)
PU L#53 (P#154)
Core L#27
PU L#54 (P#27)
PU L#55 (P#155)
Core L#28
PU L#56 (P#28)
PU L#57 (P#156)
Core L#29
PU L#58 (P#29)
PU L#59 (P#157)
Core L#30
PU L#60 (P#30)
PU L#61 (P#158)
Core L#31
PU L#62 (P#31)
PU L#63 (P#159)
HostBridge
PCIBridge
PCI 41:00.0 (Ethernet)
Net "ens1f0"
OpenFabrics "mlx5_0"
PCI 41:00.1 (Ethernet)
Net "ens1f1"
OpenFabrics "mlx5_1"
HostBridge
PCIBridge
PCI 63:00.0 (Ethernet)
Net "eno12399np0"
PCI 63:00.1 (Ethernet)
Net "eno12409np1"
PCIBridge
PCIBridge
PCI 62:00.0 (VGA)
Group0 L#1
NUMANode L#1 (P#1 126GB)
L3 L#4 (32MB)
Core L#32
PU L#64 (P#32)
PU L#65 (P#160)
Core L#33
PU L#66 (P#33)
PU L#67 (P#161)
Core L#34
PU L#68 (P#34)
PU L#69 (P#162)
Core L#35
PU L#70 (P#35)
PU L#71 (P#163)
Core L#36
PU L#72 (P#36)
PU L#73 (P#164)
Core L#37
PU L#74 (P#37)
PU L#75 (P#165)
Core L#38
PU L#76 (P#38)
PU L#77 (P#166)
Core L#39
PU L#78 (P#39)
PU L#79 (P#167)
L3 L#5 (32MB)
Core L#40
PU L#80 (P#40)
PU L#81 (P#168)
Core L#41
PU L#82 (P#41)
PU L#83 (P#169)
Core L#42
PU L#84 (P#42)
PU L#85 (P#170)
Core L#43
PU L#86 (P#43)
PU L#87 (P#171)
Core L#44
PU L#88 (P#44)
PU L#89 (P#172)
Core L#45
PU L#90 (P#45)
PU L#91 (P#173)
Core L#46
PU L#92 (P#46)
PU L#93 (P#174)
Core L#47
PU L#94 (P#47)
PU L#95 (P#175)
L3 L#6 (32MB)
Core L#48
PU L#96 (P#48)
PU L#97 (P#176)
Core L#49
PU L#98 (P#49)
PU L#99 (P#177)
Core L#50
PU L#100 (P#50)
PU L#101 (P#178)
Core L#51
PU L#102 (P#51)
PU L#103 (P#179)
Core L#52
PU L#104 (P#52)
PU L#105 (P#180)
Core L#53
PU L#106 (P#53)
PU L#107 (P#181)
Core L#54
PU L#108 (P#54)
PU L#109 (P#182)
Core L#55
PU L#110 (P#55)
PU L#111 (P#183)
L3 L#7 (32MB)
Core L#56
PU L#112 (P#56)
PU L#113 (P#184)
Core L#57
PU L#114 (P#57)
PU L#115 (P#185)
Core L#58
PU L#116 (P#58)
PU L#117 (P#186)
Core L#59
PU L#118 (P#59)
PU L#119 (P#187)
Core L#60
PU L#120 (P#60)
PU L#121 (P#188)
Core L#61
PU L#122 (P#61)
PU L#123 (P#189)
Core L#62
PU L#124 (P#62)
PU L#125 (P#190)
Core L#63
PU L#126 (P#63)
PU L#127 (P#191)
HostBridge
PCIBridge
PCI 01:00.0 (RAID)
PCIBridge
PCI 02:00.0 (SATA)
Block(Disk) "sda"
HostBridge
PCIBridge
PCI 22:00.0 (NVMExp)
Block(Disk) "nvme0n1"
Package L#1
Group0 L#2
NUMANode L#2 (P#2 126GB)
L3 L#8 (32MB)
Core L#64
PU L#128 (P#64)
PU L#129 (P#192)
Core L#65
PU L#130 (P#65)
PU L#131 (P#193)
Core L#66
PU L#132 (P#66)
PU L#133 (P#194)
Core L#67
PU L#134 (P#67)
PU L#135 (P#195)
Core L#68
PU L#136 (P#68)
PU L#137 (P#196)
Core L#69
PU L#138 (P#69)
PU L#139 (P#197)
Core L#70
PU L#140 (P#70)
PU L#141 (P#198)
Core L#71
PU L#142 (P#71)
PU L#143 (P#199)
L3 L#9 (32MB)
Core L#72
PU L#144 (P#72)
PU L#145 (P#200)
Core L#73
PU L#146 (P#73)
PU L#147 (P#201)
Core L#74
PU L#148 (P#74)
PU L#149 (P#202)
Core L#75
PU L#150 (P#75)
PU L#151 (P#203)
Core L#76
PU L#152 (P#76)
PU L#153 (P#204)
Core L#77
PU L#154 (P#77)
PU L#155 (P#205)
Core L#78
PU L#156 (P#78)
PU L#157 (P#206)
Core L#79
PU L#158 (P#79)
PU L#159 (P#207)
L3 L#10 (32MB)
Core L#80
PU L#160 (P#80)
PU L#161 (P#208)
Core L#81
PU L#162 (P#81)
PU L#163 (P#209)
Core L#82
PU L#164 (P#82)
PU L#165 (P#210)
Core L#83
PU L#166 (P#83)
PU L#167 (P#211)
Core L#84
PU L#168 (P#84)
PU L#169 (P#212)
Core L#85
PU L#170 (P#85)
PU L#171 (P#213)
Core L#86
PU L#172 (P#86)
PU L#173 (P#214)
Core L#87
PU L#174 (P#87)
PU L#175 (P#215)
L3 L#11 (32MB)
Core L#88
PU L#176 (P#88)
PU L#177 (P#216)
Core L#89
PU L#178 (P#89)
PU L#179 (P#217)
Core L#90
PU L#180 (P#90)
PU L#181 (P#218)
Core L#91
PU L#182 (P#91)
PU L#183 (P#219)
Core L#92
PU L#184 (P#92)
PU L#185 (P#220)
Core L#93
PU L#186 (P#93)
PU L#187 (P#221)
Core L#94
PU L#188 (P#94)
PU L#189 (P#222)
Core L#95
PU L#190 (P#95)
PU L#191 (P#223)
HostBridge
PCIBridge
PCI c5:00.0 (SATA)
HostBridge
PCIBridge
PCI e1:00.0 (Ethernet)
Net "eno8303"
PCI e1:00.1 (Ethernet)
Net "eno8403"
Group0 L#3
NUMANode L#3 (P#3 126GB)
L3 L#12 (32MB)
Core L#96
PU L#192 (P#96)
PU L#193 (P#224)
Core L#97
PU L#194 (P#97)
PU L#195 (P#225)
Core L#98
PU L#196 (P#98)
PU L#197 (P#226)
Core L#99
PU L#198 (P#99)
PU L#199 (P#227)
Core L#100
PU L#200 (P#100)
PU L#201 (P#228)
Core L#101
PU L#202 (P#101)
PU L#203 (P#229)
Core L#102
PU L#204 (P#102)
PU L#205 (P#230)
Core L#103
PU L#206 (P#103)
PU L#207 (P#231)
L3 L#13 (32MB)
Core L#104
PU L#208 (P#104)
PU L#209 (P#232)
Core L#105
PU L#210 (P#105)
PU L#211 (P#233)
Core L#106
PU L#212 (P#106)
PU L#213 (P#234)
Core L#107
PU L#214 (P#107)
PU L#215 (P#235)
Core L#108
PU L#216 (P#108)
PU L#217 (P#236)
Core L#109
PU L#218 (P#109)
PU L#219 (P#237)
Core L#110
PU L#220 (P#110)
PU L#221 (P#238)
Core L#111
PU L#222 (P#111)
PU L#223 (P#239)
L3 L#14 (32MB)
Core L#112
PU L#224 (P#112)
PU L#225 (P#240)
Core L#113
PU L#226 (P#113)
PU L#227 (P#241)
Core L#114
PU L#228 (P#114)
PU L#229 (P#242)
Core L#115
PU L#230 (P#115)
PU L#231 (P#243)
Core L#116
PU L#232 (P#116)
PU L#233 (P#244)
Core L#117
PU L#234 (P#117)
PU L#235 (P#245)
Core L#118
PU L#236 (P#118)
PU L#237 (P#246)
Core L#119
PU L#238 (P#119)
PU L#239 (P#247)
L3 L#15 (32MB)
Core L#120
PU L#240 (P#120)
PU L#241 (P#248)
Core L#121
PU L#242 (P#121)
PU L#243 (P#249)
Core L#122
PU L#244 (P#122)
PU L#245 (P#250)
Core L#123
PU L#246 (P#123)
PU L#247 (P#251)
Core L#124
PU L#248 (P#124)
PU L#249 (P#252)
Core L#125
PU L#250 (P#125)
PU L#251 (P#253)
Core L#126
PU L#252 (P#126)
PU L#253 (P#254)
Core L#127
PU L#254 (P#127)
PU L#255 (P#255)
HostBridge
PCIBridge
PCI 81:00.0 (Ethernet)
Net "ens3f0"
OpenFabrics "mlx5_2"
PCI 81:00.1 (Ethernet)
Net "ens3f1"
OpenFabrics "mlx5_3"
```
The current result gives
```
$ ./usertools/cpu_layout.py
======================================================================
Core and Socket Information (as reported by '/sys/devices/system/cpu') ======================================================================
cores = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63] numa nodes per socket = [0, 1] sockets = [0, 1]
Socket 0 Numa 0
-------------------
Core 0 [0, 128]
Core 1 [1, 129]
Core 2 [2, 130]
Core 3 [3, 131]
Core 4 [4, 132]
Core 5 [5, 133]
Core 6 [6, 134]
Core 7 [7, 135]
Core 8 [8, 136]
Core 9 [9, 137]
Core 10 [10, 138]
Core 11 [11, 139]
Core 12 [12, 140]
Core 13 [13, 141]
Core 14 [14, 142]
Core 15 [15, 143]
Core 16 [16, 144]
Core 17 [17, 145]
Core 18 [18, 146]
Core 19 [19, 147]
Core 20 [20, 148]
Core 21 [21, 149]
Core 22 [22, 150]
Core 23 [23, 151]
Core 24 [24, 152]
Core 25 [25, 153]
Core 26 [26, 154]
Core 27 [27, 155]
Core 28 [28, 156]
Core 29 [29, 157]
Core 30 [30, 158]
Core 31 [31, 159]
Socket 0 Numa 1
-------------------
Core 0 [32, 160]
Core 1 [33, 161]
Core 2 [34, 162]
Core 3 [35, 163]
Core 4 [36, 164]
Core 5 [37, 165]
Core 6 [38, 166]
Core 7 [39, 167]
Core 8 [40, 168]
Core 9 [41, 169]
Core 10 [42, 170]
Core 11 [43, 171]
Core 12 [44, 172]
Core 13 [45, 173]
Core 14 [46, 174]
Core 15 [47, 175]
Core 16 [48, 176]
Core 17 [49, 177]
Core 18 [50, 178]
Core 19 [51, 179]
Core 20 [52, 180]
Core 21 [53, 181]
Core 22 [54, 182]
Core 23 [55, 183]
Core 24 [56, 184]
Core 25 [57, 185]
Core 26 [58, 186]
Core 27 [59, 187]
Core 28 [60, 188]
Core 29 [61, 189]
Core 30 [62, 190]
Core 31 [63, 191]
Socket 1 Numa 2
-------------------
Core 0 [64, 192]
Core 1 [65, 193]
Core 2 [66, 194]
Core 3 [67, 195]
Core 4 [68, 196]
Core 5 [69, 197]
Core 6 [70, 198]
Core 7 [71, 199]
Core 8 [72, 200]
Core 9 [73, 201]
Core 10 [74, 202]
Core 11 [75, 203]
Core 12 [76, 204]
Core 13 [77, 205]
Core 14 [78, 206]
Core 15 [79, 207]
Core 16 [80, 208]
Core 17 [81, 209]
Core 18 [82, 210]
Core 19 [83, 211]
Core 20 [84, 212]
Core 21 [85, 213]
Core 22 [86, 214]
Core 23 [87, 215]
Core 24 [88, 216]
Core 25 [89, 217]
Core 26 [90, 218]
Core 27 [91, 219]
Core 28 [92, 220]
Core 29 [93, 221]
Core 30 [94, 222]
Core 31 [95, 223]
Socket 1 Numa 3
-------------------
Core 0 [96, 224]
Core 1 [97, 225]
Core 2 [98, 226]
Core 3 [99, 227]
Core 4 [100, 228]
Core 5 [101, 229]
Core 6 [102, 230]
Core 7 [103, 231]
Core 8 [104, 232]
Core 9 [105, 233]
Core 10 [106, 234]
Core 11 [107, 235]
Core 12 [108, 236]
Core 13 [109, 237]
Core 14 [110, 238]
Core 15 [111, 239]
Core 16 [112, 240]
Core 17 [113, 241]
Core 18 [114, 242]
Core 19 [115, 243]
Core 20 [116, 244]
Core 21 [117, 245]
Core 22 [118, 246]
Core 23 [119, 247]
Core 24 [120, 248]
Core 25 [121, 249]
Core 26 [122, 250]
Core 27 [123, 251]
Core 28 [124, 252]
Core 29 [125, 253]
Core 30 [126, 254]
Core 31 [127, 255]
```
On 3/28/2022 3:56 AM, Varghese, Vipin wrote:
> Hi Thomas,
>
> <snipp>
>
> 26/03/2022 08:32, Vipin Varghese:
>>> enhance python logic to accomadate NUMA information. Current logic
>>> considers physical socket with CPU threads to core map. With new AMD
>>> SKU vairant NUMA is no longer same as SOCKET. Single physical socket
>>> can be partitioned to variant of 1,2 and 4.
>>>
>>> The changes address the new mapping with Socket-NUMA to CPU cores.
>>>
>>> Signed-off-by: Vipin Varghese <vipin.varghese@amd.com>
>>> ---
>>> usertools/cpu_layout.py | 76
>>> +++++++++++++++++++++++++----------------
>>> 1 file changed, 47 insertions(+), 29 deletions(-)
>> Honestly, I'm not sure it is a good idea to keep this script in the DPDK repo.
>> Can it be replaced with hwloc usage?
>
> thanks for the suggestion, it is genuine and useful. Following is my observations
>
> - It takes some effort to identify the NUMA with `Group`
> - One needs to install ` lstopo-no-graphics` on distro and manually build and add on custom Linux.
>
>
>> What is the output on the new AMD SKU for this command?
>> lstopo-no-graphics --merge
>
> I have tried ` lstopo-no-graphics --merge` on a ` 2 Socket AMD EPYC 7713 64-Core Processor` with
> possible NUMA configuration such as 1, 2 and 4.
>
Hi Thomas,
Should we document preferred tool and usage in DPDK, if so where is good
location for it?
Also @Dmitry, is there solution for Windows for this issue (a tool to
replace cpu_layout.py)?
Thanks,
ferruh
Hi,
2023-06-14 11:48 (UTC+0100), Ferruh Yigit:
> Also @Dmitry, is there solution for Windows for this issue (a tool to
> replace cpu_layout.py)?
hwloc binaries, including lstopo-no-graphics.exe,
are packaged for Windows by the authors:
https://www.open-mpi.org/software/hwloc/current/
14/06/2023 12:48, Ferruh Yigit:
> Should we document preferred tool and usage in DPDK, if so where is good
> location for it?
I think it should be in the user guide.
We are starting to merge the 3 user guides in 1.
You can add it to the Linux guide only for now.
[AMD Official Use Only - General]
@Yigit, Ferruh thanks for bringing this to my attention,
The output of the modified
```
Socket 0 Numa 0
-------------------
Core 0 [0, 128]
Core 1 [1, 129]
Core 2 [2, 130]
Core 3 [3, 131]
.....
Core 29 [29, 157]
Core 30 [30, 158]
Core 31 [31, 159]
Socket 0 Numa 1
-------------------
Core 0 [32, 160]
Core 1 [33, 161]
Core 2 [34, 162]
Core 3 [35, 163]
.....
Core 30 [62, 190]
Core 31 [63, 191]
Socket 1 Numa 2
-------------------
Core 0 [64, 192]
Core 1 [65, 193]
Core 2 [66, 194]
Core 3 [67, 195]
.....
Core 31 [95, 223]
Socket 1 Numa 3
-------------------
Core 0 [96, 224]
Core 1 [97, 225]
Core 2 [98, 226]
.....
Core 29 [125, 253]
Core 30 [126, 254]
Core 31 [127, 255]
```
The intention of the changes is clearly highlighted the NUMA divisioning and CPU pinning within single or multiple sockets. But as pointed out @Dmitry Kozlyuk if the end user will use hwloc on both linux & windows to identify these, we should promote and document the changes provided the existing tool is phased out.
Note: there were questions asked in forums, stackoverflow and github on NUMA to CPU pinning, hence enhanced the tool to accommodate the changes.
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Wednesday, June 14, 2023 8:00 PM
> To: Yigit, Ferruh <Ferruh.Yigit@amd.com>
> Cc: Varghese, Vipin <Vipin.Varghese@amd.com>; Dmitry Kozlyuk
> <dmitry.kozliuk@gmail.com>; david.marchand@redhat.com; Tummala,
> Sivaprasad <Sivaprasad.Tummala@amd.com>; dev@dpdk.org; Narcisa Ana
> Maria Vasile <navasile@linux.microsoft.com>; Dmitry Malloy
> <dmitrym@microsoft.com>; Pallavi Kadam <pallavi.kadam@intel.com>
> Subject: Re: [PATCH] usertools: enhance logic to display NUMA
>
> Caution: This message originated from an External Source. Use proper
> caution when opening attachments, clicking links, or responding.
>
>
> 14/06/2023 12:48, Ferruh Yigit:
> > Should we document preferred tool and usage in DPDK, if so where is
> > good location for it?
>
> I think it should be in the user guide.
> We are starting to merge the 3 user guides in 1.
> You can add it to the Linux guide only for now.
>
>
On Sat, 26 Mar 2022 00:32:07 -0700
Vipin Varghese <vipin.varghese@amd.com> wrote:
> +
> + output = " Socket " + str(socket).ljust(3, ' ') + " Numa " + str(numa).zfill(1) + " "
> + #output = " Socket " + str(socket).zfill(1) + " Numa " + str(numa).zfill(1) + " "
> + print(output)
> + print(format("-" * len(output)))
> +
> + for index,coreSibling in enumerate(numa_map[keys]):
> + print ("Core " + str(index).ljust(3, ' ') + " " + str(coreSibling))
> + #print ("Core " + str(index).zfill(3) + " " + str(coreSibling))
> +print("")
> +
> --
Git complains because you added new blank line at end of file.
One wording suggestion would be to use the term "Node" instead of "Numa" in the table.
And fix heading alignment. The new headings don't look right.
For the case with only single socket, single node, some of the headings could be
dropped as well.
I.e:
Socket 0 Node 0
-------------------
Core 0 [0, 4]
Core 1 [1, 5]
Core 2 [2, 6]
Core 3 [3, 7]
11/07/2023 17:42, Stephen Hemminger:
> On Sat, 26 Mar 2022 00:32:07 -0700
> Vipin Varghese <vipin.varghese@amd.com> wrote:
>
> > +
> > + output = " Socket " + str(socket).ljust(3, ' ') + " Numa " + str(numa).zfill(1) + " "
> > + #output = " Socket " + str(socket).zfill(1) + " Numa " + str(numa).zfill(1) + " "
> > + print(output)
> > + print(format("-" * len(output)))
> > +
> > + for index,coreSibling in enumerate(numa_map[keys]):
> > + print ("Core " + str(index).ljust(3, ' ') + " " + str(coreSibling))
> > + #print ("Core " + str(index).zfill(3) + " " + str(coreSibling))
> > +print("")
> > +
>
> Git complains because you added new blank line at end of file.
>
> One wording suggestion would be to use the term "Node" instead of "Numa" in the table.
> And fix heading alignment. The new headings don't look right.
>
> For the case with only single socket, single node, some of the headings could be
> dropped as well.
I don't understand why we continue working on this script.
I thought we agreed it should be removed in favor of lstopo.
[AMD Official Use Only - General]
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Tuesday, July 11, 2023 9:56 PM
> To: Varghese, Vipin <Vipin.Varghese@amd.com>; Stephen Hemminger
> <stephen@networkplumber.org>
> Cc: david.marchand@redhat.com; Tummala, Sivaprasad
> <Sivaprasad.Tummala@amd.com>; dev@dpdk.org
> Subject: Re: [PATCH] usertools: enhance logic to display NUMA
>
> Caution: This message originated from an External Source. Use proper
> caution when opening attachments, clicking links, or responding.
>
>
> 11/07/2023 17:42, Stephen Hemminger:
> > On Sat, 26 Mar 2022 00:32:07 -0700
> > Vipin Varghese <vipin.varghese@amd.com> wrote:
> >
> > > +
> > > + output = " Socket " + str(socket).ljust(3, ' ') + " Numa " +
> str(numa).zfill(1) + " "
> > > + #output = " Socket " + str(socket).zfill(1) + " Numa " + str(numa).zfill(1) +
> " "
> > > + print(output)
> > > + print(format("-" * len(output)))
> > > +
> > > + for index,coreSibling in enumerate(numa_map[keys]):
> > > + print ("Core " + str(index).ljust(3, ' ') + " " + str(coreSibling))
> > > + #print ("Core " + str(index).zfill(3) + " " + str(coreSibling))
> > > +print("")
> > > +
> >
> > Git complains because you added new blank line at end of file.
> >
> > One wording suggestion would be to use the term "Node" instead of
> "Numa" in the table.
> > And fix heading alignment. The new headings don't look right.
> >
> > For the case with only single socket, single node, some of the
> > headings could be dropped as well.
>
> I don't understand why we continue working on this script.
> I thought we agreed it should be removed in favor of lstopo.
>
Sorry Thomas, I did not follow your ` I don't understand why we continue working on this script. I thought we agreed it should be removed in favor of lstopo.`
From last email from my end `we should promote and document the changes provided the existing tool is phased out and use lstopo`.
Note:
1. This is with assumption that both Linux and Windows `lstopo` is modified and handles `ACPI L3 SRAT NUMA` and `Node per Socket NUMA`.
2. I have not seen a depreciation notice for cpu_layout.py too.
14/07/2023 11:14, Varghese, Vipin:
> From: Thomas Monjalon <thomas@monjalon.net>
> > 11/07/2023 17:42, Stephen Hemminger:
> > > On Sat, 26 Mar 2022 00:32:07 -0700
> > > Vipin Varghese <vipin.varghese@amd.com> wrote:
> > >
> > > > +
> > > > + output = " Socket " + str(socket).ljust(3, ' ') + " Numa " +
> > str(numa).zfill(1) + " "
> > > > + #output = " Socket " + str(socket).zfill(1) + " Numa " + str(numa).zfill(1) +
> > " "
> > > > + print(output)
> > > > + print(format("-" * len(output)))
> > > > +
> > > > + for index,coreSibling in enumerate(numa_map[keys]):
> > > > + print ("Core " + str(index).ljust(3, ' ') + " " + str(coreSibling))
> > > > + #print ("Core " + str(index).zfill(3) + " " + str(coreSibling))
> > > > +print("")
> > > > +
> > >
> > > Git complains because you added new blank line at end of file.
> > >
> > > One wording suggestion would be to use the term "Node" instead of
> > "Numa" in the table.
> > > And fix heading alignment. The new headings don't look right.
> > >
> > > For the case with only single socket, single node, some of the
> > > headings could be dropped as well.
> >
> > I don't understand why we continue working on this script.
> > I thought we agreed it should be removed in favor of lstopo.
> >
>
> Sorry Thomas, I did not follow your ` I don't understand why we continue working on this script. I thought we agreed it should be removed in favor of lstopo.`
>
> From last email from my end `we should promote and document the changes provided the existing tool is phased out and use lstopo`.
>
> Note:
> 1. This is with assumption that both Linux and Windows `lstopo` is modified and handles `ACPI L3 SRAT NUMA` and `Node per Socket NUMA`.
> 2. I have not seen a depreciation notice for cpu_layout.py too.
My take is that we should stop working on this script.
It is better to contribute to lstopo. Did you try?
We probably won't deprecate cpu_layout.py easily.
The first step would be to add a message when running the script,
recommending to use lstopo.
Vipin would you like to write such a patch?
[AMD Official Use Only - General]
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Monday, July 17, 2023 8:37 PM
> To: Stephen Hemminger <stephen@networkplumber.org>; Varghese, Vipin
> <Vipin.Varghese@amd.com>
> Cc: david.marchand@redhat.com; Tummala, Sivaprasad
> <Sivaprasad.Tummala@amd.com>; dev@dpdk.org
> Subject: Re: [PATCH] usertools: enhance logic to display NUMA
>
> Caution: This message originated from an External Source. Use proper
> caution when opening attachments, clicking links, or responding.
>
>
> 14/07/2023 11:14, Varghese, Vipin:
> > From: Thomas Monjalon <thomas@monjalon.net>
> > > 11/07/2023 17:42, Stephen Hemminger:
> > > > On Sat, 26 Mar 2022 00:32:07 -0700 Vipin Varghese
> > > > <vipin.varghese@amd.com> wrote:
> > > >
<<Snipped>>
> > > >
> > > > Git complains because you added new blank line at end of file.
> > > >
> > > > One wording suggestion would be to use the term "Node" instead of
> > > "Numa" in the table.
> > > > And fix heading alignment. The new headings don't look right.
> > > >
> > > > For the case with only single socket, single node, some of the
> > > > headings could be dropped as well.
> > >
> > > I don't understand why we continue working on this script.
> > > I thought we agreed it should be removed in favor of lstopo.
> > >
> >
> > Sorry Thomas, I did not follow your ` I don't understand why we
> > continue working on this script. I thought we agreed it should be
> > removed in favor of lstopo.`
> >
> > From last email from my end `we should promote and document the
> changes provided the existing tool is phased out and use lstopo`.
> >
> > Note:
> > 1. This is with assumption that both Linux and Windows `lstopo` is modified
> and handles `ACPI L3 SRAT NUMA` and `Node per Socket NUMA`.
> > 2. I have not seen a depreciation notice for cpu_layout.py too.
>
> My take is that we should stop working on this script.
> It is better to contribute to lstopo. Did you try?
>
> We probably won't deprecate cpu_layout.py easily.
> The first step would be to add a message when running the script,
> recommending to use lstopo.
> Vipin would you like to write such a patch?
Sure, we can do this. Based on the discussion with Ferruh, we as AMD find a need to update documentation for ` Platform Specific Guides`, which will help to capture various combinations of `NPS` and `L3 for Compute Tiles` using lstopo and other tools as required.
>
18/07/2023 12:37, Varghese, Vipin:
> > 14/07/2023 11:14, Varghese, Vipin:
> > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > 11/07/2023 17:42, Stephen Hemminger:
> > > > > On Sat, 26 Mar 2022 00:32:07 -0700 Vipin Varghese
> > > > > <vipin.varghese@amd.com> wrote:
> > > > >
>
> <<Snipped>>
>
> > > > >
> > > > > Git complains because you added new blank line at end of file.
> > > > >
> > > > > One wording suggestion would be to use the term "Node" instead of
> > > > "Numa" in the table.
> > > > > And fix heading alignment. The new headings don't look right.
> > > > >
> > > > > For the case with only single socket, single node, some of the
> > > > > headings could be dropped as well.
> > > >
> > > > I don't understand why we continue working on this script.
> > > > I thought we agreed it should be removed in favor of lstopo.
> > > >
> > >
> > > Sorry Thomas, I did not follow your ` I don't understand why we
> > > continue working on this script. I thought we agreed it should be
> > > removed in favor of lstopo.`
> > >
> > > From last email from my end `we should promote and document the
> > changes provided the existing tool is phased out and use lstopo`.
> > >
> > > Note:
> > > 1. This is with assumption that both Linux and Windows `lstopo` is modified
> > and handles `ACPI L3 SRAT NUMA` and `Node per Socket NUMA`.
> > > 2. I have not seen a depreciation notice for cpu_layout.py too.
> >
> > My take is that we should stop working on this script.
> > It is better to contribute to lstopo. Did you try?
> >
> > We probably won't deprecate cpu_layout.py easily.
> > The first step would be to add a message when running the script,
> > recommending to use lstopo.
> > Vipin would you like to write such a patch?
>
> Sure, we can do this. Based on the discussion with Ferruh, we as AMD find a need to update documentation for ` Platform Specific Guides`, which will help to capture various combinations of `NPS` and `L3 for Compute Tiles` using lstopo and other tools as required.
OK thanks
[AMD Official Use Only - General]
<snipped>
> > > >
> > > > From last email from my end `we should promote and document the
> > > changes provided the existing tool is phased out and use lstopo`.
> > > >
> > > > Note:
> > > > 1. This is with assumption that both Linux and Windows `lstopo` is
> > > > modified
> > > and handles `ACPI L3 SRAT NUMA` and `Node per Socket NUMA`.
> > > > 2. I have not seen a depreciation notice for cpu_layout.py too.
> > >
> > > My take is that we should stop working on this script.
> > > It is better to contribute to lstopo. Did you try?
> > >
> > > We probably won't deprecate cpu_layout.py easily.
> > > The first step would be to add a message when running the script,
> > > recommending to use lstopo.
> > > Vipin would you like to write such a patch?
https://patchwork.dpdk.org/project/dpdk/patch/20230812005720.997-1-vipin.varghese@amd.com/
> >
> > Sure, we can do this. Based on the discussion with Ferruh, we as AMD find a
> need to update documentation for ` Platform Specific Guides`, which will help
> to capture various combinations of `NPS` and `L3 for Compute Tiles` using
> lstopo and other tools as required.
Updating the documentation section for AMD EPYC shortly.
>
> OK thanks
>
>
@@ -3,13 +3,27 @@
# Copyright(c) 2010-2014 Intel Corporation
# Copyright(c) 2017 Cavium, Inc. All rights reserved.
+import glob
+import os
+
sockets = []
cores = []
+numaNodes = []
core_map = {}
+numa_map = {}
+node_path = "/sys/devices/system/node"
base_path = "/sys/devices/system/cpu"
-fd = open("{}/kernel_max".format(base_path))
-max_cpus = int(fd.read())
-fd.close()
+max_cpus = 0
+
+if os.path.isdir(base_path):
+ temp_maxCpu = glob.glob(base_path + '/cpu[0-9]*')
+ max_cpus = len(temp_maxCpu)
+
+if os.path.isdir(node_path):
+ temp_numaNodes = glob.glob(node_path + '/node*')
+ for numaId in range(0, int(os.path.basename(temp_numaNodes[-1])[4:]) + 1):
+ numaNodes.append(numaId)
+
for cpu in range(max_cpus + 1):
try:
fd = open("{}/cpu{}/topology/core_id".format(base_path, cpu))
@@ -17,48 +31,52 @@
continue
core = int(fd.read())
fd.close()
+
+ tempGet_cpuNuma = glob.glob("{}/cpu{}/node*".format(base_path, cpu))
+ temp_cpuNuma = tempGet_cpuNuma[-1].split("{}/cpu{}/".format(base_path, cpu))[-1]
+ numa = temp_cpuNuma.split("node")[-1]
+
fd = open("{}/cpu{}/topology/physical_package_id".format(base_path, cpu))
socket = int(fd.read())
fd.close()
+
if core not in cores:
cores.append(core)
+
if socket not in sockets:
sockets.append(socket)
+
key = (socket, core)
if key not in core_map:
core_map[key] = []
core_map[key].append(cpu)
+ key = (socket, numa)
+ if key not in numa_map:
+ numa_map[key] = []
+
+ if (core_map[(socket, core)] not in numa_map[key]):
+ numa_map[key].append(core_map[(socket, core)])
+
print(format("=" * (47 + len(base_path))))
print("Core and Socket Information (as reported by '{}')".format(base_path))
print("{}\n".format("=" * (47 + len(base_path))))
print("cores = ", cores)
+print("numa nodes per socket = ", numaNodes)
print("sockets = ", sockets)
print("")
-max_processor_len = len(str(len(cores) * len(sockets) * 2 - 1))
-max_thread_count = len(list(core_map.values())[0])
-max_core_map_len = (max_processor_len * max_thread_count) \
- + len(", ") * (max_thread_count - 1) \
- + len('[]') + len('Socket ')
-max_core_id_len = len(str(max(cores)))
-
-output = " ".ljust(max_core_id_len + len('Core '))
-for s in sockets:
- output += " Socket %s" % str(s).ljust(max_core_map_len - len('Socket '))
-print(output)
-
-output = " ".ljust(max_core_id_len + len('Core '))
-for s in sockets:
- output += " --------".ljust(max_core_map_len)
- output += " "
-print(output)
-
-for c in cores:
- output = "Core %s" % str(c).ljust(max_core_id_len)
- for s in sockets:
- if (s, c) in core_map:
- output += " " + str(core_map[(s, c)]).ljust(max_core_map_len)
- else:
- output += " " * (max_core_map_len + 1)
- print(output)
+for keys in numa_map:
+ print ("")
+ socket,numa = keys
+
+ output = " Socket " + str(socket).ljust(3, ' ') + " Numa " + str(numa).zfill(1) + " "
+ #output = " Socket " + str(socket).zfill(1) + " Numa " + str(numa).zfill(1) + " "
+ print(output)
+ print(format("-" * len(output)))
+
+ for index,coreSibling in enumerate(numa_map[keys]):
+ print ("Core " + str(index).ljust(3, ' ') + " " + str(coreSibling))
+ #print ("Core " + str(index).zfill(3) + " " + str(coreSibling))
+print("")
+