Patch Comment Detail
get:
Show a patch comment.
patch:
Update a patch comment.
put:
Update a patch comment.
GET /api/patches/139713/comments/169789/?format=api
{ "id": 169789, "web_url": "http://patchwork.dpdk.org/comment/169789/", "msgid": "<31aec717-e498-45eb-9b61-82b0556490fb@amd.com>", "list_archive_url": "https://inbox.dpdk.org/dev/31aec717-e498-45eb-9b61-82b0556490fb@amd.com", "date": "2024-05-01T12:36:41", "subject": "Re: [PATCH v9 8/9] doc: update documentation of TAP PMD", "submitter": { "id": 2700, "url": "http://patchwork.dpdk.org/api/people/2700/?format=api", "name": "Ferruh Yigit", "email": "ferruh.yigit@amd.com" }, "content": "On 4/26/2024 4:48 PM, Stephen Hemminger wrote:\n> The driver support of flows has changed and the wording in\n> the guide was awkward.\n> \n> Drop references to DPDK pktgen in this documentation since\n> it is not required and confusing.\n> \n> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>\n> ---\n> doc/guides/linux_gsg/sys_reqs.rst | 3 +\n> doc/guides/nics/tap.rst | 274 +++++++++---------------------\n> 2 files changed, 80 insertions(+), 197 deletions(-)\n> \n> diff --git a/doc/guides/linux_gsg/sys_reqs.rst b/doc/guides/linux_gsg/sys_reqs.rst\n> index 13be715933..0254568517 100644\n> --- a/doc/guides/linux_gsg/sys_reqs.rst\n> +++ b/doc/guides/linux_gsg/sys_reqs.rst\n> @@ -101,6 +101,9 @@ Running DPDK Applications\n> \n> To run a DPDK application, some customization may be required on the target machine.\n> \n> +.. _linux_gsg_kernel_version:\n> +\n> +\n>\n\nI guess there was an intention to use this document as reference, but I\ndon't see the link. Perhaps anchor is no more required?\n\n> System Software\n> ~~~~~~~~~~~~~~~\n> \n> diff --git a/doc/guides/nics/tap.rst b/doc/guides/nics/tap.rst\n> index d4f45c02a1..55e38fb25b 100644\n> --- a/doc/guides/nics/tap.rst\n> +++ b/doc/guides/nics/tap.rst\n> @@ -1,47 +1,51 @@\n> .. SPDX-License-Identifier: BSD-3-Clause\n> Copyright(c) 2016 Intel Corporation.\n> \n> -Tun|Tap Poll Mode Driver\n> -========================\n> +TAP Poll Mode Driver\n> +====================\n> \n> -The ``rte_eth_tap.c`` PMD creates a device using TAP interfaces on the\n> -local host. The PMD allows for DPDK and the host to communicate using a raw\n> -device interface on the host and in the DPDK application.\n> +The TAP Poll Mode Driver (PMD) is a virtual device for injecting packets to be processed\n> +by the Linux kernel. This PMD is useful when writing DPDK application\n> +for offloading network functionality (such as tunneling) from the kernel.\n> \n> -The device created is a TAP device, which sends/receives packet in a raw\n> -format with a L2 header. The usage for a TAP PMD is for connectivity to the\n> -local host using a TAP interface. When the TAP PMD is initialized it will\n> -create a number of tap devices in the host accessed via ``ifconfig -a`` or\n> -``ip`` command. The commands can be used to assign and query the virtual like\n> -device.\n> +From the kernel point of view, the TAP device looks like a regular network interface.\n> +The network device can be managed by standard tools such as ``ip`` and ``ethtool`` commands.\n> +It is also possible to use existing packet tools such as ``wireshark`` or ``tcpdump``.\n> \n> -These TAP interfaces can be used with Wireshark or tcpdump or Pktgen-DPDK\n> -along with being able to be used as a network connection to the DPDK\n> -application. The method enable one or more interfaces is to use the\n> -``--vdev=net_tap0`` option on the DPDK application command line. Each\n> -``--vdev=net_tap1`` option given will create an interface named dtap0, dtap1,\n> -and so on.\n> +From the DPDK application, the TAP device looks like a DPDK ethdev.\n> +Packets are sent and received in L2 (Ethernet) format. The standare DPDK\n> +API's to query for information, statistics and send and receive packets\n> +work as expected.\n> \n> -The interface name can be changed by adding the ``iface=foo0``, for example::\n> +Requirements\n> +~~~~~~~~~~~~\n> +\n> +The TAP PMD requires kernel support for multiple queues in TAP device as\n> +well as the multi-queue ``multiq`` and incoming ``ingress`` queue disciplines.\n> +These are standard kernel features in most Linux distributions.\n> +\n> +Arguments\n> +---------\n> +\n> +TAP devices are created with the command line\n> +``--vdev=net_tap0`` option. This option maybe specified more the once by repeating\n> +with a different ``net_tapX`` device.\n> +\n> +By default, the Linux interfaces are named ``dtap0``, ``dtap1``, etc.\n> +The interface name can be specified by adding the ``iface=foo0``, for example::\n> \n> --vdev=net_tap0,iface=foo0 --vdev=net_tap1,iface=foo1, ...\n> \n> -Normally the PMD will generate a random MAC address, but when testing or with\n> -a static configuration the developer may need a fixed MAC address style.\n> -Using the option ``mac=fixed`` you can create a fixed known MAC address::\n> +Normally the PMD will generate a random MAC address.\n> +If a static address is desired instead, the ``mac=fixed`` can be used.\n> \n> --vdev=net_tap0,mac=fixed\n> \n> -The MAC address will have a fixed value with the last octet incrementing by one\n> -for each interface string containing ``mac=fixed``. The MAC address is formatted\n> -as 02:'d':'t':'a':'p':[00-FF]. Convert the characters to hex and you get the\n> -actual MAC address: ``02:64:74:61:70:[00-FF]``.\n> -\n> - --vdev=net_tap0,mac=\"02:64:74:61:70:11\"\n> +With the fixed option, the MAC address will have the first octets:\n> +as 02:'d':'t':'a':'p':[00-FF] and the last octets are the interface number.\n> \n> -The MAC address will have a user value passed as string. The MAC address is in\n> -format with delimiter ``:``. The string is byte converted to hex and you get\n> -the actual MAC address: ``02:64:74:61:70:11``.\n> +To specify a specific MAC address use the conventional representation.\n> +The string is byte converted to hex, the result is MAC address: ``02:64:74:61:70:11``.\n> \n> It is possible to specify a remote netdevice to capture packets from by adding\n> ``remote=foo1``, for example::\n> @@ -59,40 +63,20 @@ netdevice that has no support in the DPDK. It is possible to add explicit\n> rte_flow rules on the tap PMD to capture specific traffic (see next section for\n> examples).\n> \n> -After the DPDK application is started you can send and receive packets on the\n> -interface using the standard rx_burst/tx_burst APIs in DPDK. From the host\n> -point of view you can use any host tool like tcpdump, Wireshark, ping, Pktgen\n> -and others to communicate with the DPDK application. The DPDK application may\n> -not understand network protocols like IPv4/6, UDP or TCP unless the\n> -application has been written to understand these protocols.\n> -\n> -If you need the interface as a real network interface meaning running and has\n> -a valid IP address then you can do this with the following commands::\n> -\n> - sudo ip link set dtap0 up; sudo ip addr add 192.168.0.250/24 dev dtap0\n> - sudo ip link set dtap1 up; sudo ip addr add 192.168.1.250/24 dev dtap1\n> -\n> -Please change the IP addresses as you see fit.\n> -\n> -If routing is enabled on the host you can also communicate with the DPDK App\n> -over the internet via a standard socket layer application as long as you\n> -account for the protocol handling in the application.\n> -\n> -If you have a Network Stack in your DPDK application or something like it you\n> -can utilize that stack to handle the network protocols. Plus you would be able\n> -to address the interface using an IP address assigned to the internal\n> -interface.\n> -\n> Normally, when the DPDK application exits,\n> the TAP device is marked down and is removed.\n> -But this behaviour can be overridden by the use of the persist flag, example::\n> +But this behavior can be overridden by the use of the persist flag, example::\n> \n> --vdev=net_tap0,iface=tap0,persist ...\n> \n> -The TUN PMD allows user to create a TUN device on host. The PMD allows user\n> -to transmit and receive packets via DPDK API calls with L3 header and payload.\n> -The devices in host can be accessed via ``ifconfig`` or ``ip`` command. TUN\n> -interfaces are passed to DPDK ``rte_eal_init`` arguments as ``--vdev=net_tunX``,\n> +TUN devices\n> +-----------\n> +\n> +The TAP device can be used an L3 tunnel only device (TUN).\n> +This type of device does not include the Ethernet (L2) header; all packets\n> +are sent and received as IP packets.\n> +\n> +TUN devices are created with the command line arguments ``--vdev=net_tunX``,\n> where X stands for unique id, example::\n> \n> --vdev=net_tun0 --vdev=net_tun1,iface=foo1, ...\n> @@ -103,27 +87,33 @@ options. Default interface name is ``dtunX``, where X stands for unique id.\n> Flow API support\n> ----------------\n> \n> -The tap PMD supports major flow API pattern items and actions, when running on\n> -linux kernels above 4.2 (\"Flower\" classifier required).\n> -The kernel support can be checked with this command::\n> +The TAP PMD supports major flow API pattern items and actions.\n> +\n> +Requirements\n> +~~~~~~~~~~~~\n> \n> - zcat /proc/config.gz | ( grep 'CLS_FLOWER=' || echo 'not supported' ) |\n> - tee -a /dev/stderr | grep -q '=m' &&\n> - lsmod | ( grep cls_flower || echo 'try modprobe cls_flower' )\n> +Flow support in TAP driver requires the Linux kernel support of flow based\n> +traffic control filter ``flower``. This was added in Linux 4.3 kernel.\n> \n> -Supported items:\n> +The implementation of RSS action uses an eBPF module that requires additional\n> +libraries and tools. Building the RSS support requires the ``clang``\n> +compiler to compile the C code to BPF target; ``bpftool`` to convert the\n> +compiled BPF object to a header file; and ``libbpf`` to load the eBPF\n> +action into the kernel.\n> \n> -- eth: src and dst (with variable masks), and eth_type (0xffff mask).\n> -- vlan: vid, pcp, but not eid. (requires kernel 4.9)\n> -- ipv4/6: src and dst (with variable masks), and ip_proto (0xffff mask).\n> -- udp/tcp: src and dst port (0xffff) mask.\n> +Supported match items:\n> +\n> + - eth: src and dst (with variable masks), and eth_type (0xffff mask).\n> + - vlan: vid, pcp, but not eid. (requires kernel 4.9)\n> + - ipv4/6: src and dst (with variable masks), and ip_proto (0xffff mask).\n> + - udp/tcp: src and dst port (0xffff) mask.\n> \n> Supported actions:\n> \n> - DROP\n> - QUEUE\n> - PASSTHRU\n> -- RSS (requires kernel 4.9)\n> +- RSS\n> \n> It is generally not possible to provide a \"last\" item. However, if the \"last\"\n> item, once masked, is identical to the masked spec, then it is supported.\n> @@ -133,7 +123,7 @@ full mask (exact match).\n> \n> As rules are translated to TC, it is possible to show them with something like::\n> \n> - tc -s filter show dev tap1 parent 1:\n> + tc -s filter show dev dtap1 parent 1:\n> \n> Examples of testpmd flow rules\n> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n> @@ -174,135 +164,25 @@ The IPC synchronization of Rx/Tx queues is currently limited:\n> - Maximum 8 queues shared\n> - Synchronized on probing, but not on later port update\n> \n> -Example\n> --------\n> -\n> -The following is a simple example of using the TAP PMD with the Pktgen\n> -packet generator. It requires that the ``socat`` utility is installed on the\n> -test system.\n> -\n> -Build DPDK, then pull down Pktgen and build pktgen using the DPDK SDK/Target\n> -used to build the dpdk you pulled down.\n> -\n> -Run pktgen from the pktgen directory in a terminal with a commandline like the\n> -following::\n> -\n> - sudo ./app/app/x86_64-native-linux-gcc/app/pktgen -l 1-5 -n 4 \\\n> - --proc-type auto --log-level debug --socket-mem 512,512 --file-prefix pg \\\n> - --vdev=net_tap0 --vdev=net_tap1 -b 05:00.0 -b 05:00.1 \\\n> - -b 04:00.0 -b 04:00.1 -b 04:00.2 -b 04:00.3 \\\n> - -b 81:00.0 -b 81:00.1 -b 81:00.2 -b 81:00.3 \\\n> - -b 82:00.0 -b 83:00.0 -- -T -P -m [2:3].0 -m [4:5].1 \\\n> - -f themes/black-yellow.theme\n> -\n> -.. Note:\n> -\n> - Change the ``-b`` options to exclude all of your physical ports. The\n> - following command line is all one line.\n> -\n> - Also, ``-f themes/black-yellow.theme`` is optional if the default colors\n> - work on your system configuration. See the Pktgen docs for more\n> - information.\n> -\n> -Verify with ``ifconfig -a`` command in a different xterm window, should have a\n> -``dtap0`` and ``dtap1`` interfaces created.\n> -\n> -Next set the links for the two interfaces to up via the commands below::\n> -\n> - sudo ip link set dtap0 up; sudo ip addr add 192.168.0.250/24 dev dtap0\n> - sudo ip link set dtap1 up; sudo ip addr add 192.168.1.250/24 dev dtap1\n> -\n> -Then use socat to create a loopback for the two interfaces::\n> -\n> - sudo socat interface:dtap0 interface:dtap1\n> -\n> -Then on the Pktgen command line interface you can start sending packets using\n> -the commands ``start 0`` and ``start 1`` or you can start both at the same\n> -time with ``start all``. The command ``str`` is an alias for ``start all`` and\n> -``stp`` is an alias for ``stop all``.\n> -\n> -While running you should see the 64 byte counters increasing to verify the\n> -traffic is being looped back. You can use ``set all size XXX`` to change the\n> -size of the packets after you stop the traffic. Use pktgen ``help``\n> -command to see a list of all commands. You can also use the ``-f`` option to\n> -load commands at startup in command line or Lua script in pktgen.\n> \n> RSS specifics\n> -------------\n> -Packet distribution in TAP is done by the kernel which has a default\n> -distribution. This feature is adding RSS distribution based on eBPF code.\n> -The default eBPF code calculates RSS hash based on Toeplitz algorithm for\n> -a fixed RSS key. It is calculated on fixed packet offsets. For IPv4 and IPv6 it\n> -is calculated over src/dst addresses (8 or 32 bytes for IPv4 or IPv6\n> -respectively) and src/dst TCP/UDP ports (4 bytes).\n> -\n> -The RSS algorithm is written in file ``tap_bpf_program.c`` which\n> -does not take part in TAP PMD compilation. Instead this file is compiled\n> -in advance to eBPF object file. The eBPF object file is then parsed and\n> -translated into eBPF byte code in the format of C arrays of eBPF\n> -instructions. The C array of eBPF instructions is part of TAP PMD tree and\n> -is taking part in TAP PMD compilation. At run time the C arrays are uploaded to\n> -the kernel via BPF system calls and the RSS hash is calculated by the\n> -kernel.\n> -\n> -It is possible to support different RSS hash algorithms by updating file\n> -``tap_bpf_program.c`` In order to add a new RSS hash algorithm follow these\n> -steps:\n> -\n> -#. Write the new RSS implementation in file ``tap_bpf_program.c``\n> -\n> - BPF programs which are uploaded to the kernel correspond to\n> - C functions under different ELF sections.\n> -\n> -#. Install ``LLVM`` library and ``clang`` compiler versions 3.7 and above\n> -\n> -#. Use make to compile `tap_bpf_program.c`` via ``LLVM`` into an object file\n> - and extract the resulting instructions into ``tap_bpf_insn.h``::\n> -\n> - cd bpf; make\n> -\n> -#. Recompile the TAP PMD.\n> -\n> -The C arrays are uploaded to the kernel using BPF system calls.\n> -\n> -``tc`` (traffic control) is a well known user space utility program used to\n> -configure the Linux kernel packet scheduler. It is usually packaged as\n> -part of the ``iproute2`` package.\n> -Since commit 11c39b5e9 (\"tc: add eBPF support to f_bpf\") ``tc`` can be used\n> -to uploads eBPF code to the kernel and can be patched in order to print the\n> -C arrays of eBPF instructions just before calling the BPF system call.\n> -Please refer to ``iproute2`` package file ``lib/bpf.c`` function\n> -``bpf_prog_load()``.\n> -\n> -An example utility for eBPF instruction generation in the format of C arrays will\n> -be added in next releases\n> -\n> -TAP reports on supported RSS functions as part of dev_infos_get callback:\n> -``RTE_ETH_RSS_IP``, ``RTE_ETH_RSS_UDP`` and ``RTE_ETH_RSS_TCP``.\n> -**Known limitation:** TAP supports all of the above hash functions together\n> -and not in partial combinations.\n> -\n> -Systems supporting flow API\n> ----------------------------\n> -\n> -- \"tc flower\" classifier requires linux kernel above 4.2\n> -- eBPF/RSS requires linux kernel above 4.9\n> -\n> -+--------------------+-----------------------+\n> -| RH7.3 | No flow rule support |\n> -+--------------------+-----------------------+\n> -| RH7.4 | No RSS action support |\n> -+--------------------+-----------------------+\n> -| RH7.5 | No RSS action support |\n> -+--------------------+-----------------------+\n> -| SLES 15, | No limitation |\n> -| kernel 4.12 | |\n> -+--------------------+-----------------------+\n> -| Azure Ubuntu 16.04,| No limitation |\n> -| kernel 4.13 | |\n> -+--------------------+-----------------------+\n> +The default packet distribution in TAP without flow rules is done by the\n> +kernel which has a default flow based distribution.\n> +When flow rules are used to distribute packets across a set of queues\n> +an eBPF program is used to calculate the RSS based on Toeplitz algorithm for\n> +with the given key.\n> +\n> +The hash is calculated for IPv4 and IPv6, over src/dst addresses\n> +(8 or 32 bytes for IPv4 or IPv6 respectively) and\n> +optionally the src/dst TCP/UDP ports (4 bytes).\n> +\n> \n> Limitations\n> -----------\n> \n> -* Rx/Tx must have the same number of queues.\n> +- Since TAP device uses a file descriptors to talk to the kernel.\n> + The same number of queues must be specified for receive and transmit.\n> +\n> +- The RSS algorithm only support L3 or L4 functions. It does not support\n> + finer grain selections (for example: only IPV6 packets with extension headers).\n\nSection headers in the document seems mixed up, it is currently as\nfollowing, I don't think this was the intention:\n\nTAP\n Requirements\n Arguments\n TUN devices\n Flow API support\n Requirements\n Examples of testpmd flow rules\n Multi-process sharing\n RSS specifics\n Limitations", "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from mails.dpdk.org (mails.dpdk.org [217.70.189.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id 6569743F60;\n\tWed, 1 May 2024 14:36:50 +0200 (CEST)", "from mails.dpdk.org (localhost [127.0.0.1])\n\tby mails.dpdk.org (Postfix) with ESMTP id DD44B402A7;\n\tWed, 1 May 2024 14:36:49 +0200 (CEST)", "from NAM10-MW2-obe.outbound.protection.outlook.com\n (mail-mw2nam10on2061.outbound.protection.outlook.com [40.107.94.61])\n by mails.dpdk.org (Postfix) with ESMTP id ED00E4021E\n for <dev@dpdk.org>; Wed, 1 May 2024 14:36:47 +0200 (CEST)", "from CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11)\n by CY8PR12MB7730.namprd12.prod.outlook.com (2603:10b6:930:85::17)\n with Microsoft SMTP Server (version=TLS1_2,\n cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.29; Wed, 1 May\n 2024 12:36:44 +0000", "from CH2PR12MB4294.namprd12.prod.outlook.com\n ([fe80::282f:29d3:cac1:cde3]) by CH2PR12MB4294.namprd12.prod.outlook.com\n ([fe80::282f:29d3:cac1:cde3%7]) with mapi id 15.20.7519.035; Wed, 1 May 2024\n 12:36:44 +0000" ], "ARC-Seal": "i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;\n b=lFX1dJ9urFPuLi8frsDleTGCTlISPPhE/StNuRg+alrpVpsA8JX0KFrHnBHwiKfdnLji0rk5ZR5b0ZlmkkdFVzs2AcxDQaCtXljHcmr15hncLJ3Jl0a1SvCmMcFIebKE2a0V9LM3X1aT5q3Yqccd+72u7hh8iKJhBEawdsLVodSwyOmSElDR7iLJtvcEHhZv/6HFpzT8+ks8V4l/2eFN+cz0comsoegRdiD5GGEUXQXOZ2nnr0jCP7ZfG/1m7NYMfarPoT4k6jkteb2EzDZPwjZfoONVU/zXPT/baWnAbeUp+xV1+ilRTJEEWjHNuIlX44DS8WmhZnbKTftxSfU5+Q==", "ARC-Message-Signature": "i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;\n s=arcselector9901;\n h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;\n bh=67nMbp9EvFqinriLAyq/jck4L2w9s9eAb3OqYhUlnPI=;\n b=nt+RXYgoX5TPfkfKPfK0YingLxxCw9aAcwOnPkGwg7e67iIzisHb7+sc1s7qXcJ5ANH4Ap4Rnxa3IsVVg2F8Z//h9g6NxCKXwWcAMY+h1uWg9gIpklo7M5qu0P6+yCKghlVoUOfJfnRIlgozflA4SSJmtM8imwHXUz1xxB1ChDpuPvRG/zEJFvUgGOPwq0I6c5KCwk2STWx5cBhWIzE1akHVBgbRZbH675/g4j9rzgOITds8gkG7BcVycPHDdBC829PLyW/ky7/BdjvPJAFFQ0P6ALToMsmGxGomqH4f9ShLzxcWknz+PdaBjoY3hv9ZvWzndHnW3KH4k3JQ70Gw+Q==", "ARC-Authentication-Results": "i=1; mx.microsoft.com 1; spf=pass\n smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass\n header.d=amd.com; arc=none", "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;\n h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;\n bh=67nMbp9EvFqinriLAyq/jck4L2w9s9eAb3OqYhUlnPI=;\n b=sOwfCnpkXnnzs6fbzPsH8N/PBuhkfKPZekEeiq5jihWR5j+ohd8dhuihQcmlMsiHQ0GD9AmipzlROf+ei8EYQ9R6D4D6Pd3WFYu1QqTMjr2m6Kt7CJf60gElbsddNNl1rnq8P8Lswid5ADwuPo07ur44dx9z4kTt8uoQS4Xctxs=", "Authentication-Results": "dkim=none (message not signed)\n header.d=none;dmarc=none action=none header.from=amd.com;", "Message-ID": "<31aec717-e498-45eb-9b61-82b0556490fb@amd.com>", "Date": "Wed, 1 May 2024 13:36:41 +0100", "User-Agent": "Mozilla Thunderbird", "Subject": "Re: [PATCH v9 8/9] doc: update documentation of TAP PMD", "To": "Stephen Hemminger <stephen@networkplumber.org>, dev@dpdk.org", "References": "<20240130034925.44869-1-stephen@networkplumber.org>\n <20240426155125.125910-1-stephen@networkplumber.org>\n <20240426155125.125910-9-stephen@networkplumber.org>", "Content-Language": "en-US", "From": "Ferruh Yigit <ferruh.yigit@amd.com>", "Autocrypt": "addr=ferruh.yigit@amd.com; keydata=\n xsFNBGJDD3EBEAC/M7Tk/DfQSmP1K96vyzdhfSBzlCaGtcxNXorq4fALruqVsD3oi0yfyEz9\n 4YN8x7py0o9EL8ZdpOX0skc0AMCDAaw033uWhCn0GLMeGRKUbfOAPvL6ecSDvGD7CJIO9j0J\n eZUvasBgPdM/435PEr9DmC6Ggzdzt8IuG4PoLi5jpFSfcqxZFCCxLUDEo/w0nuguk2FTuYJg\n B2zEZ4JTBZrw7hIHiFh8D8hr6YA6a5uTofq1tr+l048lbtdFUl8TR0aIExVzE4Z8qKZlcE+9\n RQaewjK5Al1jLE4sHdmd3GN+IvgDF3D/fLsi25SKJDeGSdeHkOmaX0qGeM4WKIfU6iARRCiQ\n N3AmBIxZ/A7UXBKLaOyZ+/i3sE6Wb53nrO4i8+0K2Qwyh6LjTeiJAIjYKN43ppxz3DaI+QwQ\n vI+uyHr4Gg0Da9EPPz/YyKauSeOZCfCB5gIfICO0j6x0SCl8uQ2nLpjxcZkf0gjcwUzP3h+S\n 3x6NfDji9YEij0zczW/dcSpGgZ6vsFpPrtnP9ZXy6J53yp0kJtOJoOlkEFFdU2yCZnCDseum\n CoudmGLZVvS0/DzHDJejq+3kK3FDGktZBOxZIIpal+nFqS7lVgOZc4+huVv3jyhzoAUOEyXA\n XK5j6o7g8STUY+z33QNnHpdLvecMwuzmvqy0jR54yAbZ64mB9QARAQABzSNGZXJydWggWWln\n aXQgPGZlcnJ1aC55aWdpdEBhbWQuY29tPsLBlwQTAQgAQQIbAwULCQgHAgYVCgkICwIEFgID\n AQIeAQIXgAIZARYhBEm7aYjps5XGsPHCElRTPtCKKm/6BQJkdyEEBQkE3meNAAoJEFRTPtCK\n Km/6UdcP/0/kEp49aIUhkRnQfmKmNVpcBEs4NqceNCWTQlaXdEwL1lxf1L49dsF5Jz1yvWi3\n tMtq0Mk1o68mQ7q8iZAzIeLxGQAlievMNE0BzLWPFmuX+ac98ITBqKdnUAn6ig5ezR+jxrAU\n 58utUszDl16eMabtCu76sINL5izB8zCWcDEUB4UqM8iBSQZ7/a7TSBVS0jVBldAORg1qfFIs\n cGMPQn/skhy3QqbK3u3Rhc44zRxvzrQJmhY6T1rpeniHSyGOeIYqjpbpnMU5n1VWzQ4NXvAD\n VDkZ4NDw6CpvF4S2h2Ds7w7GKvT6RRTddrl672IaLcaWRiqBNCPm+eKh4q5/XkOXTgUqYBVg\n Ors8uS9EbQC/SAcp9VHF9fB+3nadxZm4CLPe5ZDJnSmgu/ea7xjWQYR8ouo2THxqNZtkercc\n GOxGFxIaLcJIR/XChh9d0LKgc1FfVARTMW8UrPgINVEmVSFmAVSgVfsWIV+NSpG9/e90E4SV\n gMLPABn1YpJ8ca/IwqovctqDDXfxZOvCPOVWTzQe/ut767W+ctGR1kRkxWcz470SycOcY+PW\n VRPJd91Af0GdLFkwzZgNzkd6Gyc9XXcv4lwwqBLhWrBhqPYB0aZXIG1E/cVTiRp4dWpFHAFD\n DcuLldjIw93lCDsIeEDM9rBizGVMWEoeFmqSe7pzGTPXzsFNBGJDD3EBEAC8fBFQHej8qgIG\n CBzoIEd1cZgPIARlIhRudODXoNDbwA+zJMKtOVwol3Hh1qJ2/yZP11nZsqrP4fyUvMxrwhDe\n WBWFVDbWHLnqXMnKuUU1vQMujbzgq/4Rb9wSMW5vBL6YxhZng+h71JgS/9nVtzyaTtsOTrJi\n 6nzFSDx6Wbza2jYvL9rlK0yxJcMEiKwZQ/if4KcOesD0rtxomU/iSEv6DATcJbGXP6T93nPl\n 90XksijRKAmOwvdu3A8IIlxiSSVRP0lxiHOeR35y6PjHY2usfEDZZOVOfDfhlCVAIBZUZALv\n VmFOVSTYXeKgYa6Ooaf72+cHM3SgJIbYnevJfFv8YQW0MEAJ/IXE7B1Lk+pHNxwU3VBCrKnA\n fd/PTvviesuYRkrRD6qqZnINeu3b2DouVGGt2fVcGA38BujCd3p8i7azoGc7A6cgF7z9ETnr\n ANrbg1/dJyDmkDxOxVrVquTBbxJbDy2HaIe9wyJTEK2Sznpy62DaHVY+gfDQzexBXM10geHC\n IIUhEnOUYVaq65X3ZDjyAQnNDBQ4uMqSHZk8DpJ22X+T+IMzWzWl+VyU4UZXjkLKPvlqPjJk\n 1RbKScek5L2GhxHQbPaD76Hx4Jiel0vm2G+4wei8Ay1+0YRFkhySxogU/uQVXHTv63KzQMak\n oIfnN/V2R0ucarsvMBW+gwARAQABwsF8BBgBCAAmAhsMFiEESbtpiOmzlcaw8cISVFM+0Ioq\n b/oFAmR3IPsFCQTeZ44ACgkQVFM+0Ioqb/qINhAAtcor9bevHy22HvJvXX17IOpPSklZJAeQ\n Az43ZEo5kRlJ8mElc2g3RzYCvL/V3fSiIATxIsLq/MDtYhO8AAvklxND/u2zeBd7BkRZTZZX\n W1V1cM3oTvfx3LOhDu4f2ExQzCGdkzbXTRswSJIe1W0qwsDp+YPekbrsKp1maZArGeu+6FuW\n honeosIrWS98QJmscEhP8ooyJkLDCCOgEk+mJ/JBjzcJGuYn6+Iy/ApMw/vqiLGL1UWekcTA\n g18mREHqIR+A3ZvypIufSFB52oIs1zD/uh/MgmL62bY/Cw6M2SxiVxLRsav9TNkF6ZaNQCgn\n GqifliCEMvEuLZRBOZSYH2A/PfwjYW0Ss0Gyfywmb2IA990gcQsXxuCLG7pAbWaeYazoYYEQ\n NYmWatZNMAs68ERI2zvrVxdJ/fBWAllIEd0uQ4P05GtAHPdTIDQYp545+TPV7oyF0LfXcsQs\n SFVZE6igdvkjfYmh+QOrHGZvpWXLTmffVf/AQ81wspzbfxJ7sYM4P8Mg5kKOsaoUdyA/2qVe\n cMh1CLUHXF1GlofpGbe1lj4KUJVse5g3qwV7i9VrseA8c4VIZewdIjkzAhmmbxl+8rM/LKBH\n dZUMTzME5PFCXJIZ83qkZQ795MTe2YScp9dIV7fsS5tpDwIs7BZNVM1l3NAdK+DLHqNxKuyO 8Zk=", "In-Reply-To": "<20240426155125.125910-9-stephen@networkplumber.org>", "Content-Type": "text/plain; charset=UTF-8", "Content-Transfer-Encoding": "7bit", "X-ClientProxiedBy": "DUZPR01CA0207.eurprd01.prod.exchangelabs.com\n (2603:10a6:10:4b6::9) To CH2PR12MB4294.namprd12.prod.outlook.com\n (2603:10b6:610:a9::11)", "MIME-Version": "1.0", "X-MS-PublicTrafficType": "Email", "X-MS-TrafficTypeDiagnostic": "CH2PR12MB4294:EE_|CY8PR12MB7730:EE_", "X-MS-Office365-Filtering-Correlation-Id": "34d5538e-bef5-40a2-bb50-08dc69db5c0f", "X-MS-Exchange-SenderADCheck": "1", "X-MS-Exchange-AntiSpam-Relay": "0", "X-Microsoft-Antispam": "BCL:0;ARA:13230031|366007|376005|1800799015;", "X-Microsoft-Antispam-Message-Info": "=?utf-8?q?IwaYO6FvxsvsVycdWrpozqIZF5uq+os?=\n\t=?utf-8?q?+BqMPLqpGNDtVsNUKo6Lkemgx4zNzYiRdo5QGJ8Z8GYipB99amMkwpBmYJ53d+emg?=\n\t=?utf-8?q?jnF+2IcA1GcQxYLjIX/1uE9Gj5wQFnFBH/JLJb+ljxVngUkk5CRA3V/8XKNu8HBiX?=\n\t=?utf-8?q?tSNBdKmuzwDy6owxROHbBS7MvABrmh87vyFehz4WFZYwXu8wZdmx9EXAz5IsjamcB?=\n\t=?utf-8?q?r3nnF3m7ri+trJ82C2ZNHiFbwVGWomEytv3sJQRYay/+8xB3QxxC6QEQf7FBxLX6G?=\n\t=?utf-8?q?EOIlip/QC5j3kpXdBYZr8NP64SIZGq+CfXqm0nGeUdMLNLHCLkY4e9b1FcB1/kl1P?=\n\t=?utf-8?q?AAw9vaObgDoZiaN7yEXd9RacVPRVTzaBkkKLwpyuSXBAiP63XnlLtOepvI8KLIdDy?=\n\t=?utf-8?q?KU1DYuJbNzZgV3eTPpAlw3OkH+UeSWE1xhwuysRLhXLPlXtlo+dU/BRuN4oAuta8J?=\n\t=?utf-8?q?GqMKyRq9l9qRL+uo9xEEL7AWZkY7bnurr6i8ljz4JxpwgQS3X+oh3I05nAGkl+3aJ?=\n\t=?utf-8?q?Nc/fRxIcjxYDXGczxGqk47n+svuQHbO2Umt5yaKWpGFBlZeX1wO0uIPpcZtg2I1C4?=\n\t=?utf-8?q?CkjFBO17O0eNI5Qr64HFcXzMiuF4/b7cYESwrYZVL8RenGUvzSaiyoba49anwj1AQ?=\n\t=?utf-8?q?pgza7WnOMziigbANLU1fIP12NM5HkHw+/WpEKt7fBSNNqa/6br1b1fdtaroiaa6sI?=\n\t=?utf-8?q?logEIfVCEQL//7TRGOqC83+Oz1XOj8lTnEA3B76p943HzzMe1qF1NdNTKYMEi5c//?=\n\t=?utf-8?q?wBnPCpgg0eyCCTgTRFoD8UO5VvvaU0mF9K5hoXZTiyWAGV+nwEDge2oIOqmrga8f5?=\n\t=?utf-8?q?L3cbVqOqqb4FFEwO0XdH4U89hsAj1ehlX2RozxDE8m4NJnVXtyujBU6jh8NU/5bUR?=\n\t=?utf-8?q?Xf4AFcaAsHse/tL2HnAyy3l1oSL5+mpvYEV6NNlrNwRNGdwyZ6a5q4IKdyLirihfy?=\n\t=?utf-8?q?2On5GC6qea26tP1bdxDDCYycTsUF7jZWwwlxe1jM9h99p1UCfQTPzYwoX4fHy3l4d?=\n\t=?utf-8?q?6aB1Pc/+eJMl1j9dwDuB2SGBZCLWOEe3qSYTv5JTOg+6FZkLVcxdBu48/WLU5jr0W?=\n\t=?utf-8?q?nFR0CdkFRJ0hJD6Pa2KQ7PuzdifIEJNSWCnK7b3pkRomtc1Blj6yy1rO22wIxInhx?=\n\t=?utf-8?q?YgYmcfey1FtzoN1c4xE4P39Q5u6f7fy2QSgEDCdJSStMXB55URZ2BVLTUKjL5mCSt?=\n\t=?utf-8?q?V+HZl1eHotMZnIS2QdSGwSTT8pNfeSLzsWBw=3D=3D?=", "X-Forefront-Antispam-Report": "CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;\n IPV:NLI; SFV:NSPM; H:CH2PR12MB4294.namprd12.prod.outlook.com; PTR:; CAT:NONE;\n SFS:(13230031)(366007)(376005)(1800799015); DIR:OUT; SFP:1101;", "X-MS-Exchange-AntiSpam-MessageData-ChunkCount": "1", "X-MS-Exchange-AntiSpam-MessageData-0": "=?utf-8?q?IPnx92PoCFptt/rML7KNyR7estAn?=\n\t=?utf-8?q?mSM758ZP4nXh/t7JKvYXHjIdv78PUR8HFAHTgXw0/9ketSuWy04GDJph4YYcPd61k?=\n\t=?utf-8?q?5NYVKhvFQ33zAhV9RJD+vRxBePYVnqSQKaOrmiLeTGUWIBHvXU5/HShqrTVrH+DkV?=\n\t=?utf-8?q?3VQumUVmMmAanZw0hi3o/sEmj40EsrZh0LIO/b+5G4emxUUAQ1Y4vIUkK9GuBJA2g?=\n\t=?utf-8?q?bDet3hdoUcxHq3H/vAtrbahm97oGI1Yz7GfVOeUbXTizCSL2xfFFIgIQiJ9IfX5PM?=\n\t=?utf-8?q?52lU8xaFBzoD9j6ARV9T39g8gmh+rGXuy6dkzRklGhkixyoPB1LuE1rOJCpyR2rLR?=\n\t=?utf-8?q?PBGqh2Ln92tt1Iamlqla2vIJXkzQjgtrzwFgBdQT2fSArEQhuiMG4pCJvJottOqZn?=\n\t=?utf-8?q?NvVy1vHQl7hxkC2ON9cwwrFSpRI+ejLorgW04MVlgavJOL5QH27bbIbul0YnVOlSA?=\n\t=?utf-8?q?2uZVcUz7h9xKR6l6Z6ivPptINGHdlvVBcoIn/O+two7gcSFCWLweS2WXFGOQdFAzw?=\n\t=?utf-8?q?capJExS5nJBhnyHNeILas6mO1cK6osoI7EVdALVOPQXcnTJTo27dbkE4zMQdmPI9f?=\n\t=?utf-8?q?xwJ0AvQTLTWXC/NjL5rC9kBzIjhSsYX34Yg9FYLIY7btAQ5RzSmBCugCy2jRPMN6z?=\n\t=?utf-8?q?CCtK9wjdSFR82g4b3WBOj4RYHq4ALjlpCFozVv+5L2righHrYWSKij3BA0F6WbCup?=\n\t=?utf-8?q?Jo+sGwGIi/PX0OIaUQFrzGF7+2/f/No9jfwQfle5CrPSsY+mYNIMRC4aPlaO2kXv8?=\n\t=?utf-8?q?vOjF8os24/ZKQE9323vwrJKAf1Sm+kkdbqm/U7tPHuD1RtjGDN4vzida7LWR7igmh?=\n\t=?utf-8?q?NYQ0MW72jZWLpJmErY2lPZ7myE+vmYR1sfDSaMuPZnKhh/iceBsQL2NbQ3ab4CJnY?=\n\t=?utf-8?q?bYv1qq0sxCUf9KC5P85qA75lm2Uf0KHoSFxjuCrBqxjlevFfXO6swnLJs49SqD77M?=\n\t=?utf-8?q?4dGC275+6fpILKaRGh/7ZIWoNMWeq0Bo6DpWXzfD0FsdMzaLUAp7MZ38jr9auG/nP?=\n\t=?utf-8?q?cR6dcHXJsNBF/+cSZxLNGc9/tpRsab/45GAYQI9No/14l8HxSPHWStRRud0+e6/r7?=\n\t=?utf-8?q?qq4DRjaOuL59qhRdTV9xHK3Jo03sWmF3YXdk4fksJDmIagQuJY1F/HYjh/hGb3HDD?=\n\t=?utf-8?q?nIiYZ+jqbqougjJxQF+LHHdyGxMYchQfLQukOzw7bt7XI1o7AaGd0Z1xhaRa/2m1X?=\n\t=?utf-8?q?utC+kzZ/BpaCAWD95nlTge9njtWVSKr2KAmrVoDi5lXAJNvln3nYlF8D9g5h34y5o?=\n\t=?utf-8?q?bDkRBS7pkjoqfErCb1Kz5EvVZ9VQc46lsthxUuPXuh0x98ZccBXR49NM+0do9Vjch?=\n\t=?utf-8?q?UHCLE4Wp0Q4y/CJE7q+lG+iz9LHpkvAa56eC0AKvcyG1jUHTUZsMGe7JX4zfh6YV1?=\n\t=?utf-8?q?nzPe2DMgYIDcKGxgXdopPOHmylVnVjh3AE4xTCJ/kyKPK60flhxKbKZAVxaRptMPg?=\n\t=?utf-8?q?wlYOl4rrhCxc8CEKbXtloPI4U225LfIMG+hVDv2TEn+2346qfdmK/Ugo7BcDVJEGf?=\n\t=?utf-8?q?T6+9GJ0+WKhr?=", "X-OriginatorOrg": "amd.com", "X-MS-Exchange-CrossTenant-Network-Message-Id": "\n 34d5538e-bef5-40a2-bb50-08dc69db5c0f", "X-MS-Exchange-CrossTenant-AuthSource": "CH2PR12MB4294.namprd12.prod.outlook.com", "X-MS-Exchange-CrossTenant-AuthAs": "Internal", "X-MS-Exchange-CrossTenant-OriginalArrivalTime": "01 May 2024 12:36:44.5839 (UTC)", "X-MS-Exchange-CrossTenant-FromEntityHeader": "Hosted", "X-MS-Exchange-CrossTenant-Id": "3dd8961f-e488-4e60-8e11-a82d994e183d", "X-MS-Exchange-CrossTenant-MailboxType": "HOSTED", "X-MS-Exchange-CrossTenant-UserPrincipalName": "\n xlqCgZptv+v7smmmvMgeZAMgB5Er+GT6yYnHRECLN15PeRQ6Y4eNhfkiNB5AKUxI", "X-MS-Exchange-Transport-CrossTenantHeadersStamped": "CY8PR12MB7730", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org" }, "addressed": null }