From patchwork Thu Oct 5 17:12:06 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ananyev, Konstantin" X-Patchwork-Id: 29768 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9D2D31B20E; Thu, 5 Oct 2017 19:12:16 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id D11581B1C7 for ; Thu, 5 Oct 2017 19:12:14 +0200 (CEST) Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 05 Oct 2017 10:12:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.42,481,1500966000"; d="scan'208";a="135582413" Received: from irsmsx105.ger.corp.intel.com ([163.33.3.28]) by orsmga004.jf.intel.com with ESMTP; 05 Oct 2017 10:12:12 -0700 Received: from irsmsx103.ger.corp.intel.com ([169.254.3.49]) by irsmsx105.ger.corp.intel.com ([169.254.7.75]) with mapi id 14.03.0319.002; Thu, 5 Oct 2017 18:12:06 +0100 From: "Ananyev, Konstantin" To: "Kavanagh, Mark B" , "dev@dpdk.org" CC: "Hu, Jiayu" , "Tan, Jianfeng" , "Yigit, Ferruh" , "thomas@monjalon.net" Thread-Topic: [PATCH v8 0/6] Support TCP/IPv4, VxLAN, and GRE GSO in DPDK Thread-Index: AQHTPfDXu3vQEeptxU6k893mTc4BuKLVfCeg Date: Thu, 5 Oct 2017 17:12:06 +0000 Message-ID: <2601191342CEEE43887BDE71AB9772585FAA4E94@IRSMSX103.ger.corp.intel.com> References: <1507201331-228465-1-git-send-email-mark.b.kavanagh@intel.com> <1507218244-29568-1-git-send-email-mark.b.kavanagh@intel.com> In-Reply-To: <1507218244-29568-1-git-send-email-mark.b.kavanagh@intel.com> Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiMmIxYzI2MjItMDhhYi00OWNmLWE3YTMtYzA1MTM3MzkwYWFhIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX0lDIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE2LjUuOS4zIiwiVHJ1c3RlZExhYmVsSGFzaCI6ImV6RitVY3dWUE1qN05tTTdGaUVpOTVneGxGWlNydkRRZGE5elwvWVBJc3g4PSJ9 x-ctpclassification: CTP_IC dlp-product: dlpe-windows dlp-version: 11.0.0.116 dlp-reaction: no-action x-originating-ip: [163.33.239.181] MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v8 0/6] Support TCP/IPv4, VxLAN, and GRE GSO in DPDK X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Mark, > -----Original Message----- > From: Kavanagh, Mark B > Sent: Thursday, October 5, 2017 4:44 PM > To: dev@dpdk.org > Cc: Hu, Jiayu ; Tan, Jianfeng ; Ananyev, Konstantin ; Yigit, > Ferruh ; thomas@monjalon.net; Kavanagh, Mark B > Subject: [PATCH v8 0/6] Support TCP/IPv4, VxLAN, and GRE GSO in DPDK > > Generic Segmentation Offload (GSO) is a SW technique to split large > packets into small ones. Akin to TSO, GSO enables applications to > operate on large packets, thus reducing per-packet processing overhead. > > To enable more flexibility to applications, DPDK GSO is implemented > as a standalone library. Applications explicitly use the GSO library > to segment packets. This patch adds GSO support to DPDK for specific > packet types: specifically, TCP/IPv4, VxLAN, and GRE. > > The first patch introduces the GSO API framework. The second patch > adds GSO support for TCP/IPv4 packets (containing an optional VLAN > tag). The third patch adds GSO support for VxLAN packets that contain > outer IPv4, and inner TCP/IPv4 headers (plus optional inner and/or > outer VLAN tags). The fourth patch adds GSO support for GRE packets > that contain outer IPv4, and inner TCP/IPv4 headers (with optional > outer VLAN tag). The fifth patch in the series enables TCP/IPv4, VxLAN, > and GRE GSO in testpmd's checksum forwarding engine. The final patch > in the series adds GSO documentation to the programmer's guide. > > Performance Testing > =================== > The performance of TCP/IPv4 GSO on a 10Gbps link is demonstrated using > iperf. Setup for the test is described as follows: > > a. Connect 2 x 10Gbps physical ports (P0, P1), which are in the same > machine, together physically. > b. Launch testpmd with P0 and a vhost-user port, and use csum > forwarding engine with "retry". > c. Select IP and TCP HW checksum calculation for P0; select TCP HW > checksum calculation for vhost-user port. > d. Launch a VM with csum and tso offloading enabled. > e. Run iperf-client on virtio-net port in the VM to send TCP packets. > With enabling csum and tso, the VM can send large TCP/IPv4 packets > (mss is up to 64KB). > f. P1 is assigned to linux kernel and enabled kernel GRO. Run > iperf-server on P1. > > We conduct three iperf tests: > > test-1: enable GSO for P0 in testpmd, and set max GSO segment length > to 1518B. Run two iperf-client in the VM. > test-2: enable TSO for P0 in testpmd, and set TSO segsz to 1518B. Run > two iperf-client in the VM. > test-3: disable GSO and TSO in testpmd. Run two iperf-client in the VM. > > Throughput of the above three tests: > > test-1: 9.4Gbps > test-2: 9.5Gbps > test-3: 3Mbps > > Functional Testing > ================== > Unlike TCP packets, VMs can't send large VxLAN or GRE packets. The max > length of tunneled packets from VMs is 1514B. So current experiment > method can't be used to measure VxLAN and GRE GSO performance, but simply > test the functionality via setting small GSO segment length (e.g. 500B). > > VxLAN > ----- > To test VxLAN GSO functionality, we use the following setup: > > a. Connect 2 x 10Gbps physical ports (P0, P1), which are in the same > machine, together physically. > b. Launch testpmd with P0 and a vhost-user port, and use csum forwarding > engine with "retry". > c. Testpmd commands: > - csum parse_tunnel on "P0" > - csum parse_tunnel on "vhost-user port" > - csum set outer-ip hw "P0" > - csum set ip hw "P0" > - csum set tcp hw "P0" > - csum set tcp hw "vhost-user port" > - set port "P0" gso on > - set gso segsz 500 > d. Launch a VM with csum and tso offloading enabled. > e. Create a vxlan port for the virtio-net port in the VM. Run iperf-client > on the VxLAN port, so TCP packets are VxLAN encapsulated. However, the > max packet length is 1514B. > f. P1 is assigned to linux kernel and kernel GRO is disabled. Similarly, > create a VxLAN port for P1, and run iperf-server on the VxLAN port. > > In testpmd, we can see the length of all packets sent from P0 is smaller > than or equal to 500B. Additionally, the packets arriving in P1 is > encapsulated and is smaller than or equal to 500B. > > GRE > --- > The same process may be used to test GRE functionality, with the exception that > the tunnel type created for both the guest's virtio-net, and the host's kernel > interfaces is GRE: > `ip tunnel add mode gre remote local ` > > As in the VxLAN testcase, the length of packets sent from P0, and received on > P1, is less than 500B. > > Change log > ========== > v8: > - resolve coding style infractions (indentation). > - centralize invalid parameter checking for rte_gso_segment() into a single > 'if' statement. > - don't clear PKT_TX_TCP_SEG flag for packets that don't qualify for GSO > on account of invalid params. > - allow GSO for tunneled packets only via gso_ctx (by correcting 'if' > statement condition). Last (hopefully :)) few nits from me: 1. [dpdk-dev,v8,5/6] app/testpmd: enable TCP/IPv4, VxLAN and GRE GSO I686 build fails for me, I think you need to: 2. [dpdk-dev,v8,2/6] gso: add TCP/IPv4 GSO support int rte_gso_segment(struct rte_mbuf *pkt, ... + } else { + pkts_out[0] = pkt; + RTE_LOG(WARNING, GSO, "Unsupported packet type\n"); + return 1; + } + I still think that log level should be DEBUG here. Konstantin > > v7: > - add RTE_GSO_SEG_SIZE_MIN macro; use this to validate gso_ctx.gso_segsz. > - rename 'ipid_flag' member of gso_ctx to 'flag'. > - remove mention of VLAN tags in supported packet types. > - don't clear PKT_TX_TCP_SEG flag if GSO fails. > - take all packet overhead into account when checking for empty packet. > - ensure that only enabled GSO types are enacted upon (i.e. no fall-through to > TCP/IPv4 case from tunneled case). > - validate user-supplied gso segsz arg against RTE_GSO_SEG_SIZE_MIN in testpmd. > - simplify error-checking/handling for GSO failure case in testpmd csum engine. > - use 0 instead of !RTE_GSO_IPID_FIXED in testpmd. > > v6: > - rebase to HEAD of master (i5dce9fcA) > - remove 'l3_offset' parameter from 'update_ipv4_tcp_headers' > > v5: > - add GSO section to the programmer's guide. > - use MF or (previously 'and') offset to check if a packet is IP > fragmented. > - move 'update_header' helper functions to gso_common.h. > - move txp/ipv4 'update_header' function to gso_tcp4.c. > - move tunnel 'update_header' function to gso_tunnel_tcp4.c. > - add offset parameter to 'update_header' functions. > - combine GRE and VxLAN tunnel header update functions into a single > function. > - correct typos and errors in comments/commit messages. > > v4: > - use ol_flags instead of packet_type to decide which segmentation > function to use. > - use MF and offset to check if a packet is IP fragmented, instead of > using DF. > - remove ETHER_CRC_LEN from gso segment payload length calculation. > - refactor internal header update and other functions. > - remove RTE_GSO_IPID_INCREASE. > - add some of GSO documents. > - set the default GSO length to 1514 and fill PKT_TX_TCP_SEG for the > packets sent from GSO-enabled ports in testpmd. > v3: > - support all IPv4 header flags, including RTE_PTYPE_(INNER_)L3_IPV4, > RTE_PTYPE_(INNER_)L3_IPV4_EXT and RTE_PTYPE_(INNER_)L3_IPV4_EXT_ > UNKNOWN. > - fill mbuf->packet_type instead of using rte_net_get_ptype() in > csumonly.c, since rte_net_get_ptype() doesn't support vxlan. > - store the input packet into pkts_out inside gso_tcp4_segment() and > gso_tunnel_tcp4_segment() instead of rte_gso_segment(), when no GSO > is performed. > - add missing incldues. > - optimize file names, function names and function description. > - fix one bug in testpmd. > v2: > - merge data segments whose data_len is less than mss into a large data > segment in gso_do_segment(). > - use mbuf->packet_type/l2_len/l3_len etc. instead of parsing the packet > header in rte_gso_segment(). > - provide IP id macros for applications to select fixed or incremental IP > ids. > > Jiayu Hu (3): > gso: add Generic Segmentation Offload API framework > gso: add TCP/IPv4 GSO support > app/testpmd: enable TCP/IPv4, VxLAN and GRE GSO > > Mark Kavanagh (3): > gso: add VxLAN GSO support > gso: add GRE GSO support > doc: add GSO programmer's guide > > MAINTAINERS | 6 + > app/test-pmd/cmdline.c | 179 ++++++++ > app/test-pmd/config.c | 24 ++ > app/test-pmd/csumonly.c | 42 +- > app/test-pmd/testpmd.c | 13 + > app/test-pmd/testpmd.h | 10 + > config/common_base | 5 + > doc/api/doxy-api-index.md | 1 + > doc/api/doxy-api.conf | 1 + > .../generic_segmentation_offload_lib.rst | 256 +++++++++++ > .../prog_guide/img/gso-output-segment-format.svg | 313 ++++++++++++++ > doc/guides/prog_guide/img/gso-three-seg-mbuf.svg | 477 +++++++++++++++++++++ > doc/guides/prog_guide/index.rst | 1 + > doc/guides/rel_notes/release_17_11.rst | 17 + > doc/guides/testpmd_app_ug/testpmd_funcs.rst | 46 ++ > lib/Makefile | 2 + > lib/librte_eal/common/include/rte_log.h | 1 + > lib/librte_gso/Makefile | 52 +++ > lib/librte_gso/gso_common.c | 153 +++++++ > lib/librte_gso/gso_common.h | 171 ++++++++ > lib/librte_gso/gso_tcp4.c | 104 +++++ > lib/librte_gso/gso_tcp4.h | 74 ++++ > lib/librte_gso/gso_tunnel_tcp4.c | 126 ++++++ > lib/librte_gso/gso_tunnel_tcp4.h | 75 ++++ > lib/librte_gso/rte_gso.c | 110 +++++ > lib/librte_gso/rte_gso.h | 148 +++++++ > lib/librte_gso/rte_gso_version.map | 7 + > mk/rte.app.mk | 1 + > 28 files changed, 2411 insertions(+), 4 deletions(-) > create mode 100644 doc/guides/prog_guide/generic_segmentation_offload_lib.rst > create mode 100644 doc/guides/prog_guide/img/gso-output-segment-format.svg > create mode 100644 doc/guides/prog_guide/img/gso-three-seg-mbuf.svg > create mode 100644 lib/librte_gso/Makefile > create mode 100644 lib/librte_gso/gso_common.c > create mode 100644 lib/librte_gso/gso_common.h > create mode 100644 lib/librte_gso/gso_tcp4.c > create mode 100644 lib/librte_gso/gso_tcp4.h > create mode 100644 lib/librte_gso/gso_tunnel_tcp4.c > create mode 100644 lib/librte_gso/gso_tunnel_tcp4.h > create mode 100644 lib/librte_gso/rte_gso.c > create mode 100644 lib/librte_gso/rte_gso.h > create mode 100644 lib/librte_gso/rte_gso_version.map > > -- > 1.9.3 --- a/app/test-pmd/cmdline.c +++ b/app/test-pmd/cmdline.c @@ -4052,7 +4052,7 @@ cmd_gso_size_parsed(void *parsed_result, if (!strcmp(res->cmd_keyword, "gso") && !strcmp(res->cmd_segsz, "segsz")) { if (res->cmd_size < RTE_GSO_SEG_SIZE_MIN) - printf("gso_size should be larger than %lu." + printf("gso_size should be larger than %zu." " Please input a legal value\n", RTE_GSO_SEG_SIZE_MIN); else