@@ -31,7 +31,7 @@ If I execute "l2fwd -l 0-3 -m 64 -n 3 -- -p 3", I get the following output, indi
I have set up a total of 1024 Hugepages (that is, allocated 512 2M pages to each NUMA node).
The -m command line parameter does not guarantee that huge pages will be reserved on specific sockets. Therefore, allocated huge pages may not be on socket 0.
-To request memory to be reserved on a specific socket, please use the --socket-mem command-line parameter instead of -m.
+To request memory to be reserved on a specific socket, please use the --numa-mem command-line parameter instead of -m.
I am running a 32-bit DPDK application on a NUMA system, and sometimes the application initializes fine but cannot allocate memory. Why is that happening?
@@ -54,7 +54,7 @@ For example, if your EAL coremask is 0xff0, the main core will usually be the fi
.. Note: Instead of '-c 0xff0' use the '-l 4-11' as a cleaner way to define lcores.
In this way, the hugepages have a greater chance of being allocated to the correct socket.
-Additionally, a ``--socket-mem`` option could be used to ensure the availability of memory for each socket, so that if hugepages were allocated on
+Additionally, a ``--numa-mem`` option could be used to ensure the availability of memory for each socket, so that if hugepages were allocated on
the wrong socket, the application simply will not start.
@@ -614,7 +614,7 @@ Run testpmd in the Virtual Machine.
# use for bonding of virtio and vf tests in VM
/root/dpdk/<build_dir>/app/dpdk-testpmd \
- -l 0-3 -n 4 --socket-mem 350 -- --i --port-topology=chained
+ -l 0-3 -n 4 --numa-mem 350 -- --i --port-topology=chained
.. _lm_bond_virtio_sriov_switch_conf:
@@ -438,4 +438,4 @@ run_testpmd_in_vm.sh
# test system has 8 cpus (0-7), use cpus 2-7 for VM
/root/dpdk/<build_dir>/app/dpdk-testpmd \
- -l 0-5 -n 4 --socket-mem 350 -- --burst=64 --i
+ -l 0-5 -n 4 --numa-mem 350 -- --burst=64 --i
@@ -122,7 +122,7 @@ Testpmd launch
.. code-block:: console
- <build_dir>/app/dpdk-testpmd -l 0,2,3,4,5 --socket-mem=1024 -n 4 \
+ <build_dir>/app/dpdk-testpmd -l 0,2,3,4,5 --numa-mem=1024 -n 4 \
--vdev 'net_vhost0,iface=/tmp/vhost-user1' \
--vdev 'net_vhost1,iface=/tmp/vhost-user2' -- \
--portmask=f -i --rxq=1 --txq=1 \
@@ -332,7 +332,7 @@ Start testpmd:
.. code-block:: console
- <build_dir>/app/dpdk-testpmd -l 0,1,2 --socket-mem 1024 -n 4 \
+ <build_dir>/app/dpdk-testpmd -l 0,1,2 --numa-mem 1024 -n 4 \
--proc-type auto --file-prefix pg -- \
--portmask=3 --forward-mode=macswap --port-topology=chained \
--disable-rss -i --rxq=1 --txq=1 \
@@ -77,7 +77,7 @@ some minor changes.
.. code-block:: console
- $(testpmd) -l 0-1 -n 4 --socket-mem 1024,1024 \
+ $(testpmd) -l 0-1 -n 4 --numa-mem 1024,1024 \
--vdev 'eth_vhost0,iface=/tmp/sock0' \
--file-prefix=host --no-pci -- -i
@@ -34,7 +34,7 @@ The following is the list of options that can be given to the EAL:
.. code-block:: console
./rte-app [-c COREMASK | -l CORELIST] [-n NUM] [-b <domain:bus:devid.func>] \
- [--socket-mem=MB,...] [-d LIB.so|DIR] [-m MB] [-r NUM] [-v] [--file-prefix] \
+ [--numa-mem=MB,...] [-d LIB.so|DIR] [-m MB] [-r NUM] [-v] [--file-prefix] \
[--proc-type <primary|secondary|auto>]
The EAL options are as follows:
@@ -55,12 +55,12 @@ The EAL options are as follows:
use the specified Ethernet device(s) only. Use comma-separate
``[domain:]bus:devid.func`` values. Cannot be used with ``-b`` option.
-* ``--socket-mem``:
+* ``--numa-mem``:
Memory to allocate from hugepages on specific sockets. In dynamic memory mode,
this memory will also be pinned (i.e. not released back to the system until
application closes).
-* ``--socket-limit``:
+* ``--numa-limit``:
Limit maximum memory available for allocation on each socket. Does not support
legacy memory mode.
@@ -71,7 +71,7 @@ The EAL options are as follows:
* ``-m MB``:
Memory to allocate from hugepages, regardless of processor socket. It is
- recommended that ``--socket-mem`` be used instead of this option.
+ recommended that ``--numa-mem`` be used instead of this option.
* ``-r NUM``:
Number of memory ranks.
@@ -158,9 +158,9 @@ Hugepage Memory Use by Applications
When running an application, it is recommended to use the same amount of memory as that allocated for hugepages.
This is done automatically by the DPDK application at startup,
-if no ``-m`` or ``--socket-mem`` parameter is passed to it when run.
+if no ``-m`` or ``--numa-mem`` parameter is passed to it when run.
-If more memory is requested by explicitly passing a ``-m`` or ``--socket-mem`` value, the application fails.
+If more memory is requested by explicitly passing a ``-m`` or ``--numa-mem`` value, the application fails.
However, the application itself can also fail if the user requests less memory than the reserved amount of hugepage-memory, particularly if using the ``-m`` option.
The reason is as follows.
Suppose the system has 1024 reserved 2 MB pages in socket 0 and 1024 in socket 1.
@@ -168,15 +168,15 @@ If the user requests 128 MB of memory, the 64 pages may not match the constraint
* The hugepage memory by be given to the application by the kernel in socket 1 only.
In this case, if the application attempts to create an object, such as a ring or memory pool in socket 0, it fails.
- To avoid this issue, it is recommended that the ``--socket-mem`` option be used instead of the ``-m`` option.
+ To avoid this issue, it is recommended that the ``--numa-mem`` option be used instead of the ``-m`` option.
* These pages can be located anywhere in physical memory, and, although the DPDK EAL will attempt to allocate memory in contiguous blocks,
it is possible that the pages will not be contiguous. In this case, the application is not able to allocate big memory pools.
The socket-mem option can be used to request specific amounts of memory for specific sockets.
-This is accomplished by supplying the ``--socket-mem`` flag followed by amounts of memory requested on each socket,
-for example, supply ``--socket-mem=0,512`` to try and reserve 512 MB for socket 1 only.
-Similarly, on a four socket system, to allocate 1 GB memory on each of sockets 0 and 2 only, the parameter ``--socket-mem=1024,0,1024`` can be used.
+This is accomplished by supplying the ``--numa-mem`` flag followed by amounts of memory requested on each socket,
+for example, supply ``--numa-mem=0,512`` to try and reserve 512 MB for socket 1 only.
+Similarly, on a four socket system, to allocate 1 GB memory on each of sockets 0 and 2 only, the parameter ``--numa-mem=1024,0,1024`` can be used.
No memory will be reserved on any CPU socket that is not explicitly referenced, for example, socket 3 in this case.
If the DPDK cannot allocate enough memory on each socket, the EAL initialization fails.
@@ -60,20 +60,20 @@ Memory-related options
Use legacy DPDK memory allocation mode.
-* ``--socket-mem <amounts of memory per socket>``
+* ``--numa-mem <amounts of memory per NUMA node>``
- Preallocate specified amounts of memory per socket. The parameter is a
+ Preallocate specified amounts of memory per NUMA node. The parameter is a
comma-separated list of values. For example::
- --socket-mem 1024,2048
+ --numa-mem 1024,2048
- This will allocate 1 gigabyte of memory on socket 0, and 2048 megabytes of
- memory on socket 1.
+ This will allocate 1 gigabyte of memory on NUMA node 0, and 2048 megabytes of
+ memory on NUMA node 1.
-* ``--socket-limit <amounts of memory per socket>``
+* ``--numa-limit <amounts of memory per NUMA node>``
- Place a per-socket upper limit on memory use (non-legacy memory mode only).
- 0 will disable the limit for a particular socket.
+ Place a per-NUMA node upper limit on memory use (non-legacy memory mode only).
+ 0 will disable the limit for a particular NUMA node.
* ``--single-file-segments``
@@ -364,7 +364,7 @@ Performance tuning
#. To minimize overhead of searching Memory Regions:
- - '--socket-mem' is recommended to pin memory by predictable amount.
+ - '--numa-mem' is recommended to pin memory by predictable amount.
- Configure per-lcore cache when creating Mempools for packet buffer.
- Refrain from dynamically allocating/freeing memory in run-time.
@@ -1763,7 +1763,7 @@ Performance tuning
#. To minimize overhead of searching Memory Regions:
- - '--socket-mem' is recommended to pin memory by predictable amount.
+ - '--numa-mem' is recommended to pin memory by predictable amount.
- Configure per-lcore cache when creating Mempools for packet buffer.
- Refrain from dynamically allocating/freeing memory in run-time.
@@ -119,11 +119,11 @@ in use, either reserved memory will satisfy the requirements, or the allocation
will fail.
There is no need to preallocate any memory at startup using ``-m`` or
-``--socket-mem`` command-line parameters, however it is still possible to do so,
+``--numa-mem`` command-line parameters, however it is still possible to do so,
in which case preallocate memory will be "pinned" (i.e. will never be released
by the application back to the system). It will be possible to allocate more
hugepages, and deallocate those, but any preallocated pages will not be freed.
-If neither ``-m`` nor ``--socket-mem`` were specified, no memory will be
+If neither ``-m`` nor ``--numa-mem`` were specified, no memory will be
preallocated, and all memory will be allocated at runtime, as needed.
Another available option to use in dynamic memory mode is
@@ -143,7 +143,7 @@ to deny them), allocation validator callbacks are also available via
``rte_mem_alloc_validator_callback_register()`` function.
A default validator callback is provided by EAL, which can be enabled with a
-``--socket-limit`` command-line option, for a simple way to limit maximum amount
+``--numa-limit`` command-line option, for a simple way to limit maximum amount
of memory that can be used by DPDK application.
.. warning::
@@ -164,7 +164,7 @@ This mode mimics historical behavior of EAL. That is, EAL will reserve all
memory at startup, sort all memory into large IOVA-contiguous chunks, and will
not allow acquiring or releasing hugepages from the system at runtime.
-If neither ``-m`` nor ``--socket-mem`` were specified, the entire available
+If neither ``-m`` nor ``--numa-mem`` were specified, the entire available
hugepage memory will be preallocated.
Hugepage Allocation Matching
@@ -187,7 +187,7 @@ very dependent on the memory allocation patterns of the application.
Additional restrictions are present when running in 32-bit mode. In dynamic
memory mode, by default maximum of 2 gigabytes of VA space will be preallocated,
-and all of it will be on main lcore NUMA node unless ``--socket-mem`` flag is
+and all of it will be on main lcore NUMA node unless ``--numa-mem`` flag is
used.
In legacy mode, VA space will only be preallocated for segments that were
@@ -1183,7 +1183,7 @@ into a single block.
If deallocating pages at runtime is supported, and the free element encloses
one or more pages, those pages can be deallocated and be removed from the heap.
If DPDK was started with command-line parameters for preallocating memory
-(``-m`` or ``--socket-mem``), then those pages that were allocated at startup
+(``-m`` or ``--numa-mem``), then those pages that were allocated at startup
will not be deallocated.
Any successful deallocation event will trigger a callback, for which user
@@ -127,7 +127,7 @@ allocate more memory than they need. However if ``--legacy-mem`` is used, DPDK
will attempt to preallocate all memory it can get to, and memory use must be
explicitly limited. This is done by passing the ``-m`` flag to each process to
specify how much hugepage memory, in megabytes, each process can use (or passing
-``--socket-mem`` to specify how much hugepage memory on each socket each process
+``--numa-mem`` to specify how much hugepage memory on each socket each process
can use).
.. note::
@@ -67,7 +67,7 @@ issue the command:
.. code-block:: console
$ ./<build_dir>/examples/dpdk-bbdev --vdev='baseband_turbo_sw' -a <NIC0PCIADDR> \
- -c 0x38 --socket-mem=2,2 --file-prefix=bbdev -- -e 0x10 -d 0x20
+ -c 0x38 --numa-mem=2,2 --file-prefix=bbdev -- -e 0x10 -d 0x20
where, NIC0PCIADDR is the PCI address of the Rx port
@@ -99,12 +99,12 @@ ports.
.. code-block:: console
$ ./pktgen-3.4.0/app/x86_64-native-linux-gcc/pktgen -c 0x3 \
- --socket-mem=1,1 --file-prefix=pg -a <NIC1PCIADDR> -- -m 1.0 -P
+ --numa-mem=1,1 --file-prefix=pg -a <NIC1PCIADDR> -- -m 1.0 -P
where:
* ``-c COREMASK``: A hexadecimal bitmask of cores to run on
-* ``--socket-mem``: Memory to allocate on specific sockets (use comma separated values)
+* ``--numa-mem``: Memory to allocate on specific sockets (use comma separated values)
* ``--file-prefix``: Prefix for hugepage filenames
* ``-a <NIC1PCIADDR>``: Add a PCI device in allow list. The argument format is <[domain:]bus:devid.func>.
* ``-m <string>``: Matrix for mapping ports to logical cores.
@@ -270,7 +270,7 @@ The mapping of lcores to port/queues is similar to other l3fwd applications.
For example, given the following command line to run application in poll mode::
- ./<build_dir>/examples/dpdk-ipsec-secgw -l 20,21 -n 4 --socket-mem 0,2048 \
+ ./<build_dir>/examples/dpdk-ipsec-secgw -l 20,21 -n 4 --numa-mem 0,2048 \
--vdev "crypto_null" -- -p 0xf -P -u 0x3 \
--config="(0,0,20),(1,0,20),(2,0,21),(3,0,21)" \
-f /path/to/config_file --transfer-mode poll \
@@ -281,7 +281,7 @@ where each option means:
* The ``-n`` option sets memory 4 channels.
-* The ``--socket-mem`` to use 2GB on socket 1.
+* The ``--numa-mem`` to use 2GB on socket 1.
* The ``--vdev "crypto_null"`` option creates virtual NULL cryptodev PMD.
@@ -368,7 +368,7 @@ For example, something like the following command line:
.. code-block:: console
- ./<build_dir>/examples/dpdk-ipsec-secgw -l 20,21 -n 4 --socket-mem 0,2048 \
+ ./<build_dir>/examples/dpdk-ipsec-secgw -l 20,21 -n 4 --numa-mem 0,2048 \
-a 81:00.0 -a 81:00.1 -a 81:00.2 -a 81:00.3 \
--vdev "crypto_aesni_mb" --vdev "crypto_null" \
-- \
@@ -54,7 +54,7 @@ Take IFCVF driver for example:
.. code-block:: console
- ./dpdk-vdpa -c 0x2 -n 4 --socket-mem 1024,1024 \
+ ./dpdk-vdpa -c 0x2 -n 4 --numa-mem 1024,1024 \
-a 0000:06:00.3,vdpa=1 -a 0000:06:00.4,vdpa=1 \
-- --interactive
@@ -60,7 +60,7 @@ Start the vswitch example
.. code-block:: console
- ./dpdk-vhost -l 0-3 -n 4 --socket-mem 1024 \
+ ./dpdk-vhost -l 0-3 -n 4 --numa-mem 1024 \
-- --socket-file /tmp/sock0 --client \
...
@@ -218,5 +218,5 @@ Common Issues
pre-allocation
The builtin example doesn't support dynamic memory allocation. When vhost
- backend enables "builtin-net-driver", "--socket-mem" option should be
+ backend enables "builtin-net-driver", "--numa-mem" option should be
added at virtio-user PMD side as a startup item.
@@ -91,8 +91,11 @@ eal_long_options[] = {
{OPT_DEV_BLOCK, 1, NULL, OPT_DEV_BLOCK_NUM },
{OPT_DEV_ALLOW, 1, NULL, OPT_DEV_ALLOW_NUM },
{OPT_PROC_TYPE, 1, NULL, OPT_PROC_TYPE_NUM },
- {OPT_SOCKET_MEM, 1, NULL, OPT_SOCKET_MEM_NUM },
- {OPT_SOCKET_LIMIT, 1, NULL, OPT_SOCKET_LIMIT_NUM },
+ /* socket-mem/socket-limit are kept for backwards compatiblity */
+ {OPT_SOCKET_MEM, 1, NULL, OPT_NUMA_MEM_NUM },
+ {OPT_SOCKET_LIMIT, 1, NULL, OPT_NUMA_LIMIT_NUM },
+ {OPT_NUMA_MEM, 1, NULL, OPT_NUMA_MEM_NUM },
+ {OPT_NUMA_LIMIT, 1, NULL, OPT_NUMA_LIMIT_NUM },
{OPT_SYSLOG, 1, NULL, OPT_SYSLOG_NUM },
{OPT_VDEV, 1, NULL, OPT_VDEV_NUM },
{OPT_VFIO_INTR, 1, NULL, OPT_VFIO_INTR_NUM },
@@ -2083,12 +2086,12 @@ eal_check_common_options(struct internal_config *internal_cfg)
return -1;
}
if (mem_parsed && internal_cfg->force_numa == 1) {
- EAL_LOG(ERR, "Options -m and --"OPT_SOCKET_MEM" cannot "
+ EAL_LOG(ERR, "Options -m and --"OPT_NUMA_MEM" cannot "
"be specified at the same time");
return -1;
}
if (internal_cfg->no_hugetlbfs && internal_cfg->force_numa == 1) {
- EAL_LOG(ERR, "Option --"OPT_SOCKET_MEM" cannot "
+ EAL_LOG(ERR, "Option --"OPT_NUMA_MEM" cannot "
"be specified together with --"OPT_NO_HUGE);
return -1;
}
@@ -2106,7 +2109,7 @@ eal_check_common_options(struct internal_config *internal_cfg)
return -1;
}
if (internal_conf->force_numa_limits && internal_conf->legacy_mem) {
- EAL_LOG(ERR, "Option --"OPT_SOCKET_LIMIT
+ EAL_LOG(ERR, "Option --"OPT_NUMA_LIMIT
" is only supported in non-legacy memory mode");
}
if (internal_cfg->single_file_segments &&
@@ -2141,7 +2144,7 @@ eal_check_common_options(struct internal_config *internal_cfg)
if (internal_cfg->legacy_mem && internal_cfg->memory == 0) {
EAL_LOG(NOTICE, "Static memory layout is selected, "
"amount of reserved memory can be adjusted with "
- "-m or --"OPT_SOCKET_MEM);
+ "-m or --"OPT_NUMA_MEM);
}
return 0;
@@ -2194,7 +2197,7 @@ eal_common_usage(void)
" --"OPT_MAIN_LCORE" ID Core ID that is used as main\n"
" --"OPT_MBUF_POOL_OPS_NAME" Pool ops name for mbuf to use\n"
" -n CHANNELS Number of memory channels\n"
- " -m MB Memory to allocate (see also --"OPT_SOCKET_MEM")\n"
+ " -m MB Memory to allocate (see also --"OPT_NUMA_MEM")\n"
" -r RANKS Force number of memory ranks (don't detect)\n"
" -b, --block Add a device to the blocked list.\n"
" Prevent EAL from using this device. The argument\n"
@@ -60,9 +60,11 @@ enum {
#define OPT_IN_MEMORY "in-memory"
OPT_IN_MEMORY_NUM,
#define OPT_SOCKET_MEM "socket-mem"
- OPT_SOCKET_MEM_NUM,
-#define OPT_SOCKET_LIMIT "socket-limit"
- OPT_SOCKET_LIMIT_NUM,
+#define OPT_NUMA_MEM "numa-mem"
+ OPT_NUMA_MEM_NUM,
+#define OPT_SOCKET_LIMIT "socket-limit"
+#define OPT_NUMA_LIMIT "numa-limit"
+ OPT_NUMA_LIMIT_NUM,
#define OPT_SYSLOG "syslog"
OPT_SYSLOG_NUM,
#define OPT_VDEV "vdev"
@@ -443,8 +443,8 @@ eal_usage(const char *prgname)
printf("\nUsage: %s ", prgname);
eal_common_usage();
printf("EAL Linux options:\n"
- " --"OPT_SOCKET_MEM" Memory to allocate on sockets (comma separated values)\n"
- " --"OPT_SOCKET_LIMIT" Limit memory allocation on sockets (comma separated values)\n"
+ " --"OPT_NUMA_MEM" Memory to allocate on NUMA nodes (comma separated values)\n"
+ " --"OPT_NUMA_LIMIT" Limit memory allocation on NUMA nodes (comma separated values)\n"
" --"OPT_HUGE_DIR" Directory where hugetlbfs is mounted\n"
" --"OPT_FILE_PREFIX" Prefix for hugepage filenames\n"
" --"OPT_CREATE_UIO_DEV" Create /dev/uioX (usually done by hotplug)\n"
@@ -693,11 +693,11 @@ eal_parse_args(int argc, char **argv)
}
break;
}
- case OPT_SOCKET_MEM_NUM:
+ case OPT_NUMA_MEM_NUM:
if (eal_parse_socket_arg(optarg,
internal_conf->numa_mem) < 0) {
EAL_LOG(ERR, "invalid parameters for --"
- OPT_SOCKET_MEM);
+ OPT_NUMA_MEM);
eal_usage(prgname);
ret = -1;
goto out;
@@ -705,11 +705,11 @@ eal_parse_args(int argc, char **argv)
internal_conf->force_numa = 1;
break;
- case OPT_SOCKET_LIMIT_NUM:
+ case OPT_NUMA_LIMIT_NUM:
if (eal_parse_socket_arg(optarg,
internal_conf->numa_limit) < 0) {
EAL_LOG(ERR, "invalid parameters for --"
- OPT_SOCKET_LIMIT);
+ OPT_NUMA_LIMIT);
eal_usage(prgname);
ret = -1;
goto out;