From patchwork Sat Jun 23 23:17:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ophir Munk X-Patchwork-Id: 41430 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id F3B4F1BB42; Sun, 24 Jun 2018 01:18:00 +0200 (CEST) Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-he1eur01on0082.outbound.protection.outlook.com [104.47.0.82]) by dpdk.org (Postfix) with ESMTP id 79DE11BB3E for ; Sun, 24 Jun 2018 01:17:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RTvTTMhy8ihwxJ/LuKRHUiH3Ibn4SrpJ8s1MXNz03qQ=; b=KYBAWnCedyldlNEfdN9RBIsubiqrWTvHbb/1lvXzT+njB3JGggptgu9kHD1KjtsXxHdr+D6wck27n6DmUSf74RebWdIT25zGHEuMs3436uJvammcCUF4uVbAkJCE23FeTEr+8wmXf77QJVvzrJoE1d728RkegtWKyLTnTbiNkTQ= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=ophirmu@mellanox.com; Received: from mellanox.com (37.142.13.130) by HE1PR0501MB2314.eurprd05.prod.outlook.com (2603:10a6:3:27::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.884.23; Sat, 23 Jun 2018 23:17:56 +0000 From: Ophir Munk To: dev@dpdk.org, Keith Wiles Cc: Thomas Monjalon , Olga Shern , Ophir Munk Date: Sat, 23 Jun 2018 23:17:40 +0000 Message-Id: <1529795861-1361-2-git-send-email-ophirmu@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1529795861-1361-1-git-send-email-ophirmu@mellanox.com> References: <1528821108-12405-3-git-send-email-ophirmu@mellanox.com> <1529795861-1361-1-git-send-email-ophirmu@mellanox.com> MIME-Version: 1.0 X-Originating-IP: [37.142.13.130] X-ClientProxiedBy: VI1PR0901CA0090.eurprd09.prod.outlook.com (2603:10a6:800:7e::16) To HE1PR0501MB2314.eurprd05.prod.outlook.com (2603:10a6:3:27::19) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3570cae4-1903-48a2-c756-08d5d95f8e41 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(8989117)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600026)(711020)(48565401081)(2017052603328)(7153060)(7193020); SRVR:HE1PR0501MB2314; X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2314; 3:OdBz+2fkGh596TlXaSGxWLLRCmGkXw+ER3/F3vuU5HjIyjQNBSkHWhmXo5dwAYq9b/mvOIVOLx94qTTPCdR8vSPqJGP6dy0wIunN2ze5P2EEjXO0Jcwmy8oJ8AtOSP0tUQWQLYIedfBh94Q5ityYhCJmPi3UfwoKpwcfLseCQUTfU5JO9V7bRkPmPC8gaLQSbqc7tlVcc+9osGYu9shiReuJr2ZD25CBJ8aCF7qIav3ho6CNf9m4w6AlLudO/FBn; 25:MXzcCiEFpzIqCgWfzL0KDc4dp1xpTXdgT0UxT7ax6fx7J/JbAK8/COoo10vSoko6QsZRkrYKyUKOgvlkDTTzAtkKiwHXIyKIKq6srVk2j9Q9odhl1kCw3JYUSJAMINydN7Wl7MDYB4bTH7XqmpJENhbfpwpx5+8TEGVZ8Py1b7NnW2y0tDgLMrtdW8aO4s25vLg6liRMn5gW6EVU0UQ6NTlhDS/VyFa0W6VeSuEKo/u2DGHGuePHi8R/fsNWuwCxFY81HWOOZdYhDrXWmLCAK/6xOR32fNzyoORZis7ld+j4sXB8XjMgbQgSKRLOhufaGHv0doByP+NFAIHm9mA/Jw==; 31:6uafbfOU6P3tghHZca+L+usKXG/kuA4ygHqEn3f5RbdPTUPWsN3OpUpgOZSZ676qOjjrd5/uKju70XRW6evYlXCrwqJOMkgp+fxLbTOTLQmbsaeTRXE1wiYYPQCn+P6nnjRxhjilhRa8syZwIxrlGWjSDFx7wrv/tH/ANUnem+OmskoVIuIwXybRJtizZgA1diQWwgM7heonobPZrOOt8zZQjl/27BrMm5LiJhYMXHs= X-MS-TrafficTypeDiagnostic: HE1PR0501MB2314: X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2314; 20:VdPBq/I/2maOMe8WNKFuVC6CZZMuGQBFEVaRuBCsqIcpOWLnkAS1/zACPaBzVIHWn3F0Dcy/HYMK6TZeSZi2/PwAoq6eITtuxluQFpLI97KumM4DkyTJGEgTZ2pBtCD5d/yR+VMcXj7RDOTrt69/5PWwrtC5P1WVmkb+kfrSAvpvWfEkTmCQkRjsCFpLBX6wk0K+tnGqf/oabXlvwoyf+XVLF+OeaTV8SfFwiGYLuZO2+JBaebhBEPgymh+HpwMR5PndsRHVARfd7I0UqL3S3V4tNW/av2qCL1rNCEn3HcxCousDMohkbAJdgwlUdu2KX47QcakXTWo+yx8/BvD3RNLa6ILVGmSRRCn0cnmzX7lkATR5MKZA+qlo1TIAx6WznV4wYDSZp+k3m+8CN17zPqhIlZnu4ZTxf05LNkc+LxHKO/Wxv1cYfLdgP0QpMFF+pAy0282LxXKyAaKOgIQJAED3ev/z3bw1fk2riCvgPTO1QZ5t+d7jothSEcViibXl; 4:xl1sR9vdJwH61Exx7s2oIRiKWUEXffZflIayngh/srlQsNp6LhqPveU9lBe+IfBD920754mBt15CU55HIjfRcwRQH/ZlWVqTSrfWCBQQoWyqEekFcwUGChwVyOfUwmGmFGy1065I/nPbD/ntkGL2JUwlebamcHAkPoGWew25nRZkcjDs+tEdbKrDn88sbTktB+RyfFeneQV5A63oqVJskg3s3qZkwUfDg4wBKZ9XXStAr00vpFmlVmXycBW9qzVDSBP9wW0116knS+xkMiXxvA== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(3231254)(944501410)(52105095)(93006095)(93001095)(10201501046)(3002001)(6055026)(149027)(150027)(6041310)(20161123558120)(20161123562045)(20161123564045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011)(7699016); SRVR:HE1PR0501MB2314; BCL:0; PCL:0; RULEID:; SRVR:HE1PR0501MB2314; X-Forefront-PRVS: 07126E493C X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(366004)(396003)(39860400002)(346002)(39380400002)(376002)(199004)(189003)(26005)(86362001)(7696005)(16526019)(51416003)(7736002)(52116002)(55016002)(6666003)(2906002)(107886003)(4326008)(8936002)(4720700003)(16586007)(21086003)(186003)(36756003)(69596002)(3846002)(6116002)(59450400001)(76176011)(386003)(66066001)(47776003)(5660300001)(106356001)(316002)(105586002)(6916009)(53936002)(956004)(476003)(68736007)(48376002)(446003)(486006)(54906003)(50466002)(8676002)(81156014)(2616005)(81166006)(478600001)(11346002)(50226002)(25786009)(305945005)(33026002)(97736004); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR0501MB2314; H:mellanox.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; HE1PR0501MB2314; 23:XjeC/aaqJudYaAi6JcgEGmZViHEl3SxnewdIwy4?= W/4735VV0wuYstQw0zaFh8MmKTqGYVcYY1xSN1GrLYuh+CjXAB/fmijcKPfMplMRWWs8eOpzrsfrUWouIkSIS/Xd0U9MrQbCYcQ3fWAAuJ6DOEX0UJ5YcbCToJCisJutVGVxnAKd4DbjBQZCCE/1Zehbpi7cuPNxIeVXrLdSBy0XdnidPlYst7iWon2By5MK29Wf88aFrwqqSV/akpvKP6sCbLhC0kVR9t6gqOpzr5svnOiyPsHW/V6Zw0vglJmEK2sduWr0QuOWD3QPY04VETSmKLoeQJ97/ivCj/M42q169iOKMhx1wm6U5UvMXJ8H1mBae+LXmm/0OXE4i+9Jrroxkv/eN6LTDV8y0FU4aokvleumb0fVolwv49Uq2vIuOPTfBhdGelcit9P3yt57IMAW9a+QcOirXvUc6bUTzMMpIgICYERTpEMkU8+ZvL1FgwfSb9JD15/eDwhhZJxfLvQtDDaK6snSnueviLjbEWldCBP2guFa0tmr9QBJFFy0FaM4d3xiAOVMopH/gw9Yaw5rrIwn8lZ2e94AytUCZb1AgWjMGZYHDR6YT33ovHP1M1l8aVmMe2NM+oubak3QuG/456ltxspvC7wOlZZkxIU5WJHGe5rfNOYu/s2bm7nJdDDXUT1yTFulKx+d56ql7IZVEvNs2pc0N6rMa1yjIGwLpM8zokwPv4Hpskce0V3UDM3malFr/wr46dc3AU/BYKhPX4HaKZAcdab81bn6m3IBdGpjTkZIQcd/Kx3QTaUsOAynueaaRfKnYbVCjkGuPxSPN6DiX3oIjTgvpJ4n5a14TMrc+wKx/OdUYdCfZi1GAW2kgZBrLI+h6RoD/Za2fvN3weymVXKElAOGB4ymmrKGVoS4Sx6XgPaVKuetNrbk96eEjjZBeg2+U/C3frimoIqGB0SLI/JSbOWLV0B0ZaABZ/6wY7UUxHDyVv1sIr0U1nUXFH5vztfPs5ur6ppoc+xClPUuy3HkISpSoxHOes5Kj+0PBklZVT+rglA0F6OjGXBgDZD//E4aZsNGtOH93wu0SAoZCO8v2TJ3yZc46XJAwUOMPDSm9nGgfSaGgCiTpS7T0wGwrJvhZYBra92vt8pNG8Nih2s5sYGkzeBWM2Rz4+m3g2jrjC93w0IIYvm6oNXVUjLNkNbwoNopw1NEN+Tqdh053t69/O5makflBaAjGGkwn1tfHfx2H9W+/rf6UnKdcFzOrc8Zqw5rzarbXmW4MhqqIw0pyfGKjT5AGHJw+BV9E5LG44Cft8Rxjyuc2LIs= X-Microsoft-Antispam-Message-Info: F7Rd+1E3WdRgIOELqnvcyJgty8AbDdzPF+k5YcPxvLFd3rUQS58rurxLox/KjfGBGfpiOEyGTu6+CmB10kNC8oFGYfrPFigokZ/+qdRkEd63pG+NZS+emvdn/PX3WmlSWw5yLNLazkbJoBuQVE5kpB3iJYEP1xyu/s5PigkP+HucurqjWcHJvMuuuEMwDbkcIkoMQFAUhYFP3Ja4l1fen6WBw2VkM3/b/vgP2clExIGzwXFVxAksDZepDjUFfn3YHysCPWnHuEmYd+K4zUqX/Ybk7hKDwsh/IQR5d8Wz1nZxO4A2xcOJxV+DR5kWx6OyTcBk9eg/hjOmv5sJzGqMj2Lq0l41efrwjrOFqKW6rNQ= X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2314; 6:yxf7IXv85obs0Znkp/PYCAv6A7jDjb8eFL2dFVuVJasn8ladQZCBtRaJUzhLBX0/HNqIMcgpdaVKP0xXP8sxndDvJVZtpQOpomQPAVUlFSPjsRxRFbZhoJbr9JF0dJAhk0N+GrN9/uMYCaoOwpij25yJ2hlhbkwVpjLAw0F+M7Z80aR8lNbRfPH94iKZqRtCGNv06ZaHHPe2RJQZ/jd+uiXpCdwzzrCf7t+WuvIXRH+7z6vyjUjBWZfUcT+xuD6SkB26jLEHiNubgjL7sEr2Y+ykEHOuHsn6ctUclQe4G3QVi6NPF/QOhWPeohfNe2iVKiadqule6RBstMm/5gCDq29NAJ3kxgG3EQmrOECg69ox7K8JcsYg3CrJdeVaEf+E7EZT23N8nftrph7zy75P4jlCrYSGoKId5UZ6faZgfKXSElgiTz3eszj2yBAkN3jL8tv+TspWKvx8kR9yUksJYg==; 5:mEcIPpsOQzfQrsrZYvwKMK53ZBT0AEZ/nf1x4j4fWItsafTbIDB/jBFOY8QJU+SAW73cQG/tPI5Lx1qsJyAL2vimLH+Xqrt25bYZxHP9sjjmS6YYm7JUbxujD+6bHuUmEc7suOQg4jEDEPuMoaxkmLmniFNZBD5OjsU/8OL4i9o=; 24:9MnMQUHXrEcxTLTf2yBAL4N7OLyyz9qtgsJeZaS7wp3AUi86qrrfvsRshzpKawJv55mQ8gy2+eSrG99LeK73MAAmjzei5ILetHmhuBasjNU= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2314; 7:lBaJjkalWmK+RBrN52Su2+xjVwHxYs/18e1OQBpuXnkGzHSUR3WMTxOq1z96oQQ+FF8+QUpqNdvf2Gzz8a2rJvbLZHW14r8NIMg9HS0z6w8vy9cnkq6x78lUGd4zf/2OXJsJUv6c+Aem0O83ccULKHMJ2SmlUQd3kXhQKY/+R8vFZaYbUmGzjDLJi3ITdTxy5VccZveWe3Vjtai3NAFC45MgVNK3Mt08xCVwAcSeyAaY9oEBSVaZM0Lz2vn+9i5P X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Jun 2018 23:17:56.4480 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3570cae4-1903-48a2-c756-08d5d95f8e41 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0501MB2314 Subject: [dpdk-dev] [PATCH v5 1/2] net/tap: calculate checksums of multi segs packets X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Prior to this commit IP/UDP/TCP checksum offload calculations were skipped in case of a multi segments packet. This commit enables TAP checksum calculations for multi segments packets. The only restriction is that the first segment must contain headers of layers 3 (IP) and 4 (UDP or TCP) Reviewed-by: Raslan Darawsheh Signed-off-by: Ophir Munk Acked-by: Keith Wiles for Series --- drivers/net/tap/rte_eth_tap.c | 161 ++++++++++++++++++++++++++++++------------ 1 file changed, 114 insertions(+), 47 deletions(-) diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c index df396bf..8903646 100644 --- a/drivers/net/tap/rte_eth_tap.c +++ b/drivers/net/tap/rte_eth_tap.c @@ -415,9 +415,41 @@ tap_tx_offload_get_queue_capa(void) DEV_TX_OFFLOAD_TCP_CKSUM; } +/* Finalize l4 checksum calculation */ static void -tap_tx_offload(char *packet, uint64_t ol_flags, unsigned int l2_len, - unsigned int l3_len) +tap_tx_l4_cksum(uint16_t *l4_cksum, uint16_t l4_phdr_cksum, + uint32_t l4_raw_cksum) +{ + if (l4_cksum) { + uint32_t cksum; + + cksum = __rte_raw_cksum_reduce(l4_raw_cksum); + cksum += l4_phdr_cksum; + + cksum = ((cksum & 0xffff0000) >> 16) + (cksum & 0xffff); + cksum = (~cksum) & 0xffff; + if (cksum == 0) + cksum = 0xffff; + *l4_cksum = cksum; + } +} + +/* Accumaulate L4 raw checksums */ +static void +tap_tx_l4_add_rcksum(char *l4_data, unsigned int l4_len, uint16_t *l4_cksum, + uint32_t *l4_raw_cksum) +{ + if (l4_cksum == NULL) + return; + + *l4_raw_cksum = __rte_raw_cksum(l4_data, l4_len, *l4_raw_cksum); +} + +/* L3 and L4 pseudo headers checksum offloads */ +static void +tap_tx_l3_cksum(char *packet, uint64_t ol_flags, unsigned int l2_len, + unsigned int l3_len, unsigned int l4_len, uint16_t **l4_cksum, + uint16_t *l4_phdr_cksum, uint32_t *l4_raw_cksum) { void *l3_hdr = packet + l2_len; @@ -430,38 +462,21 @@ tap_tx_offload(char *packet, uint64_t ol_flags, unsigned int l2_len, iph->hdr_checksum = (cksum == 0xffff) ? cksum : ~cksum; } if (ol_flags & PKT_TX_L4_MASK) { - uint16_t l4_len; - uint32_t cksum; - uint16_t *l4_cksum; void *l4_hdr; l4_hdr = packet + l2_len + l3_len; if ((ol_flags & PKT_TX_L4_MASK) == PKT_TX_UDP_CKSUM) - l4_cksum = &((struct udp_hdr *)l4_hdr)->dgram_cksum; + *l4_cksum = &((struct udp_hdr *)l4_hdr)->dgram_cksum; else if ((ol_flags & PKT_TX_L4_MASK) == PKT_TX_TCP_CKSUM) - l4_cksum = &((struct tcp_hdr *)l4_hdr)->cksum; + *l4_cksum = &((struct tcp_hdr *)l4_hdr)->cksum; else return; - *l4_cksum = 0; - if (ol_flags & PKT_TX_IPV4) { - struct ipv4_hdr *iph = l3_hdr; - - l4_len = rte_be_to_cpu_16(iph->total_length) - l3_len; - cksum = rte_ipv4_phdr_cksum(l3_hdr, 0); - } else { - struct ipv6_hdr *ip6h = l3_hdr; - - /* payload_len does not include ext headers */ - l4_len = rte_be_to_cpu_16(ip6h->payload_len) - - l3_len + sizeof(struct ipv6_hdr); - cksum = rte_ipv6_phdr_cksum(l3_hdr, 0); - } - cksum += rte_raw_cksum(l4_hdr, l4_len); - cksum = ((cksum & 0xffff0000) >> 16) + (cksum & 0xffff); - cksum = (~cksum) & 0xffff; - if (cksum == 0) - cksum = 0xffff; - *l4_cksum = cksum; + **l4_cksum = 0; + if (ol_flags & PKT_TX_IPV4) + *l4_phdr_cksum = rte_ipv4_phdr_cksum(l3_hdr, 0); + else + *l4_phdr_cksum = rte_ipv6_phdr_cksum(l3_hdr, 0); + *l4_raw_cksum = __rte_raw_cksum(l4_hdr, l4_len, 0); } } @@ -482,17 +497,27 @@ pmd_tx_burst(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) max_size = *txq->mtu + (ETHER_HDR_LEN + ETHER_CRC_LEN + 4); for (i = 0; i < nb_pkts; i++) { struct rte_mbuf *mbuf = bufs[num_tx]; - struct iovec iovecs[mbuf->nb_segs + 1]; + struct iovec iovecs[mbuf->nb_segs + 2]; struct tun_pi pi = { .flags = 0, .proto = 0x00 }; struct rte_mbuf *seg = mbuf; char m_copy[mbuf->data_len]; + int proto; int n; int j; + int k; /* first index in iovecs for copying segments */ + uint16_t l234_hlen; /* length of layers 2,3,4 headers */ + uint16_t seg_len; /* length of first segment */ + uint16_t nb_segs; + uint16_t *l4_cksum; /* l4 checksum (pseudo header + payload) */ + uint32_t l4_raw_cksum = 0; /* TCP/UDP payload raw checksum */ + uint16_t l4_phdr_cksum = 0; /* TCP/UDP pseudo header checksum */ + uint16_t is_cksum = 0; /* in case cksum should be offloaded */ /* stats.errs will be incremented */ if (rte_pktmbuf_pkt_len(mbuf) > max_size) break; + l4_cksum = NULL; if (txq->type == ETH_TUNTAP_TYPE_TUN) { /* * TUN and TAP are created with IFF_NO_PI disabled. @@ -505,35 +530,77 @@ pmd_tx_burst(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) * is 4 or 6, then protocol field is updated. */ char *buff_data = rte_pktmbuf_mtod(seg, void *); - j = (*buff_data & 0xf0); - pi.proto = (j == 0x40) ? rte_cpu_to_be_16(ETHER_TYPE_IPv4) : - (j == 0x60) ? rte_cpu_to_be_16(ETHER_TYPE_IPv6) : 0x00; + proto = (*buff_data & 0xf0); + pi.proto = (proto == 0x40) ? + rte_cpu_to_be_16(ETHER_TYPE_IPv4) : + ((proto == 0x60) ? + rte_cpu_to_be_16(ETHER_TYPE_IPv6) : + 0x00); } - iovecs[0].iov_base = π - iovecs[0].iov_len = sizeof(pi); - for (j = 1; j <= mbuf->nb_segs; j++) { - iovecs[j].iov_len = rte_pktmbuf_data_len(seg); - iovecs[j].iov_base = - rte_pktmbuf_mtod(seg, void *); - seg = seg->next; - } + k = 0; + iovecs[k].iov_base = π + iovecs[k].iov_len = sizeof(pi); + k++; + + nb_segs = mbuf->nb_segs; if (txq->csum && ((mbuf->ol_flags & (PKT_TX_IP_CKSUM | PKT_TX_IPV4) || (mbuf->ol_flags & PKT_TX_L4_MASK) == PKT_TX_UDP_CKSUM || (mbuf->ol_flags & PKT_TX_L4_MASK) == PKT_TX_TCP_CKSUM))) { - /* Support only packets with all data in the same seg */ - if (mbuf->nb_segs > 1) + is_cksum = 1; + + /* Support only packets with at least layer 4 + * header included in the first segment + */ + seg_len = rte_pktmbuf_data_len(mbuf); + l234_hlen = mbuf->l2_len + mbuf->l3_len + mbuf->l4_len; + if (seg_len < l234_hlen) break; - /* To change checksums, work on a copy of data. */ + + /* To change checksums, work on a + * copy of l2, l3 l4 headers. + */ rte_memcpy(m_copy, rte_pktmbuf_mtod(mbuf, void *), - rte_pktmbuf_data_len(mbuf)); - tap_tx_offload(m_copy, mbuf->ol_flags, - mbuf->l2_len, mbuf->l3_len); - iovecs[1].iov_base = m_copy; + l234_hlen); + tap_tx_l3_cksum(m_copy, mbuf->ol_flags, + mbuf->l2_len, mbuf->l3_len, mbuf->l4_len, + &l4_cksum, &l4_phdr_cksum, + &l4_raw_cksum); + iovecs[k].iov_base = m_copy; + iovecs[k].iov_len = l234_hlen; + k++; + + /* Update next iovecs[] beyond l2, l3, l4 headers */ + if (seg_len > l234_hlen) { + iovecs[k].iov_len = seg_len - l234_hlen; + iovecs[k].iov_base = + rte_pktmbuf_mtod(seg, char *) + + l234_hlen; + tap_tx_l4_add_rcksum(iovecs[k].iov_base, + iovecs[k].iov_len, l4_cksum, + &l4_raw_cksum); + k++; + nb_segs++; + } + seg = seg->next; + } + + for (j = k; j <= nb_segs; j++) { + iovecs[j].iov_len = rte_pktmbuf_data_len(seg); + iovecs[j].iov_base = rte_pktmbuf_mtod(seg, void *); + if (is_cksum) + tap_tx_l4_add_rcksum(iovecs[j].iov_base, + iovecs[j].iov_len, l4_cksum, + &l4_raw_cksum); + seg = seg->next; } + + if (is_cksum) + tap_tx_l4_cksum(l4_cksum, l4_phdr_cksum, l4_raw_cksum); + /* copy the tx frame data */ - n = writev(txq->fd, iovecs, mbuf->nb_segs + 1); + n = writev(txq->fd, iovecs, j); if (n <= 0) break; From patchwork Sat Jun 23 23:17:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ophir Munk X-Patchwork-Id: 41432 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DAAD41BB52; Sun, 24 Jun 2018 01:18:09 +0200 (CEST) Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-he1eur01hn0235.outbound.protection.outlook.com [104.47.0.235]) by dpdk.org (Postfix) with ESMTP id 52AA51BB6F for ; Sun, 24 Jun 2018 01:18:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YUzKGuEagx4Hi9ZUM6dBn0g+6IGQzEVb6L3UQvk0cMA=; b=WMTxGa2ZEM3DNG5gNjKg2cZpimlMxTxy1Tnhqg+5jkgk/iahVcAdtGmSXA7HZtAHvA9VJh2WmHrZuPFxLD3vXhmx0vPg4AVm/htro/2x2beAueR52nyzPnN9OUZQIC4IYbaP2bzrOZjchJem2TlP1H+Wc4+7JvUUemLoLeX8rU4= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=ophirmu@mellanox.com; Received: from mellanox.com (37.142.13.130) by HE1PR0501MB2314.eurprd05.prod.outlook.com (2603:10a6:3:27::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.884.23; Sat, 23 Jun 2018 23:17:58 +0000 From: Ophir Munk To: dev@dpdk.org, Keith Wiles Cc: Thomas Monjalon , Olga Shern , Ophir Munk Date: Sat, 23 Jun 2018 23:17:41 +0000 Message-Id: <1529795861-1361-3-git-send-email-ophirmu@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1529795861-1361-1-git-send-email-ophirmu@mellanox.com> References: <1528821108-12405-3-git-send-email-ophirmu@mellanox.com> <1529795861-1361-1-git-send-email-ophirmu@mellanox.com> MIME-Version: 1.0 X-Originating-IP: [37.142.13.130] X-ClientProxiedBy: VI1PR0901CA0090.eurprd09.prod.outlook.com (2603:10a6:800:7e::16) To HE1PR0501MB2314.eurprd05.prod.outlook.com (2603:10a6:3:27::19) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 926d9c86-0df7-41c9-5930-08d5d95f8f56 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(8989117)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600026)(711020)(48565401081)(2017052603328)(7153060)(7193020); SRVR:HE1PR0501MB2314; X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2314; 3:qHAP5vRaV9Jo4NVNBb6J2xOmsYZ45coCzjOdrPqqF5hp5xYgXzDVTXOfJvpaSoSzidO9e9KSUPxeFwVDVctLv9jT2+Byt1pJbF7i1TyumSb68rOnSGn/tn53eDdSP+0q9MoTGNPEavDWMsKbrCLE0KNR3iWGXfGPRLvt87AyzCEpBCKtrVew/EMU3nkcjn2G8D+l2yl0ZHkhb2xpRZHLxUrgnb6Lp7qTYVaNmOINVx6ukEL93imwfGWoA6dzogjl; 25:Zt/ar63mRinay9LYCbcH7B7XD9fUb5uEHsColVgc/XHMSn7Cnic8uvhlt+3tYVd7XWEI27+aZdkDAZ/Jn+KjrvsqD/853geew5DsNiV5mMbQmZ41/6gJk0h60vMxEDupliNMVL4ReNNMfyatB3u+AYLxNy95HlqeVGcz5gOs3pm+G1mWQHf+fvCSeiNR/L5cNVOssMABjdzi5Ta798/hZjNo08bmhslNkvew76Tpc4UnI/PXbbWqp0/ExYHHMwvyr1q+JlFVZbCew+dZK+UzOyh8+Y8elgZFH+0iLgSezY8fIGxRiUukV2UAPF/34ophVA1EKXYwltd870m7ojFXXg==; 31:W734Ju/ToGmgdrLjGCYGg/fmQVNQvRbzbest1/CrT5OBG2FncvhKoAbGT1beFXzw84MzI9Y1Lzq1uzwqkPvxQ5GskWOZWDvdLwwLTij6uaoVqiRvpTMyPZ3QmzdbzVTUYhiLnCsuWMwP/80DQy/obWjCoV7yCs6EethHJiuMiaU7LT6UrVRucdUfBOiZOiySpXbGw7NQsnMGBbqZtjibe4uiuOu8Ne1QvhsFvG5aEFk= X-MS-TrafficTypeDiagnostic: HE1PR0501MB2314:|HE1PR0501MB2314: X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2314; 20:787nLNCAT/NMmUQyyXNPiBia0PuYRwqhQtMVu7FHCxsDV476Fq5XFZjdxR1/Ged40cFDZymFUqVSoJYox04/gRYxhX0pqc/iave0DG7lsAbitALNtYylPqqlpYgu8TZcQZZ5RcmEsMezn7csaUYeRwr8yiFRO2JaArk6uSA4udWvxAGF+PLl796GBmGq+BoubCfTPDWxWrdi/xJXz85li5cpgUPlXPlDQQxaPzcoDdQGLnhjzIqzYKHLXi4W3gmnHiPK35FqSFgtdEg1rZTwRUnYLQpr8eE6zXQa0FfCCOR26Hx2ioHWEdF7O2Rl2tG4YvUYEPQ+j94jac65oJEYfrbNZguF3rfBKVXai+GTl22jgvF8IDRZqoU5tZYnmHpRZHH8y6rPV3CS+OiLkfhTOu92yOu6VwRKBw5osBynOyN/YsC2V+eef1Iovdma1Ypmygsgoy9QjEnBQbSJh2o/EaZsyDGpvlrCk/d9E+8uuDQayVXKWv+bMIQvNQdgxz6i; 4:bDDgDNYCwQdOpXplSNFXjnEkgoN/Npko/TG5gDtp7klhmR0Il0VirbgotEbiCWKb99EPznoTeEPLpIIXmzTI80/3plNFt/EpVYebAxCpRAexGnIMN9fwTwJJnKEK9MfgZJQ418OTSnf6RIlev3rVteUhmJOWdf5ebTDTfffOAyaohsoaNFobVEfiedon4QFvq39d5E9V/Z8U4KTp+gjx91dD+1iEbOlcBnbbAgdap25viY+t+rPfhwk1bXCrRE/2xHoLMezIu3HvXlqF+PAsIel59aCm0ISx2CPosHdzw1PMu151kLhl1+ZTBa/+2cwn X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(66839620246622); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(3231254)(2232096)(944501410)(52105095)(93006095)(93001095)(10201501046)(3002001)(6055026)(149027)(150027)(6041310)(20161123558120)(20161123562045)(20161123564045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011)(7699016); SRVR:HE1PR0501MB2314; BCL:0; PCL:0; RULEID:; SRVR:HE1PR0501MB2314; X-Forefront-PRVS: 07126E493C X-Forefront-Antispam-Report: SFV:SPM; SFS:(10009020)(366004)(396003)(39860400002)(346002)(39380400002)(376002)(199004)(189003)(575784001)(26005)(86362001)(7696005)(16526019)(51416003)(7736002)(52116002)(55016002)(6666003)(2906002)(107886003)(4326008)(8936002)(4720700003)(16586007)(21086003)(186003)(36756003)(69596002)(3846002)(6116002)(76176011)(386003)(66066001)(47776003)(5660300001)(106356001)(316002)(105586002)(6916009)(53936002)(956004)(476003)(68736007)(48376002)(446003)(486006)(54906003)(50466002)(8676002)(81156014)(2616005)(81166006)(478600001)(11346002)(50226002)(25786009)(305945005)(33026002)(97736004)(59010400001); DIR:OUT; SFP:1501; SCL:5; SRVR:HE1PR0501MB2314; H:mellanox.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; HE1PR0501MB2314; 23:D67tKjeY7HiJpQKnTLRWDXcQe89kzTJsaAN1pxx?= 7Wl88OUV2cnmBkB8ApZGf+K1oUgLk+VIOhH2aY/fEbvMlJarhiKaTRVYCMO6z9kJmGVDVyk2+I6Plia790JZ+LK7MxLZfa6SP98WdDypwz/ul6Gp4KG/UxpR/9KLJPEid0nb1SMoWhRFjt2HH7+7RPSHTO31O2sfcSAnwhPW89nAFUJsd6qQFOpfSHziJexRGDX6t0/KzaEo95QMgpUbM1CEGmIgQ8JAd/KbSJBthWOJ1Bt95aunn1MbdE9U1tc2RnlsuiK+GnynL0LKXBmhTDJCYg5UXLkgP1+EjD3U2NPm6yo43YjExRb6Etb4JqccS5RyzAIEodqcvnduyfeguuF7vC/C8tEZmlVc7yNuLFclK8EYIY2N6SMlUFmRpCBlb1EVaeZzvojVSVcove7fEn+Bhv8vF63Dz/Ochwfwn4qqjQ3A25lvTWHK8yghH4WRIP+f2xt656k+v9UQQ2+UmeAiK2a8BQVXOppCMR5wa5sFqBK+XGK0z2gePLuE90EJcest5qEKlKM67l4oIVTKq/SNtzodGq36OA3pqfhlhqbrWfiLW68SiBT6mPrn3pObAJfLjVvKHQaczjEk1FwiqEtVqTms6U4o7acrgIAXXd6rjb9hpeK2TXo9OEU8G+EEL+VKgLBLB3dRH41Ghay0PnyRsO+95f/Jztwjb6ROR19Hr8o/wAbBJVkYfif0ai0S8bloI+3uBkWqkjPCUONQDVGtvi9UCghz56A3KywFBTnNmkKz6UFEdnjjKjgczOSCFBRdz54xX8uZVvHAk22PmpWcaHSe3yDzLE+Xw8wgb1MbCpnLJ8ZoG6wj7duuNOBTcnxA/Wi7QPuGYYH4O6wt09j+C0OQR/IfR9rGAJU/uszPsPzRzFZmR3UT5jeFubDO6SW6y/n+cdTwSoCuX78+0LDq2/9HRd5TTxWD95dX1h9Jr/IK/LDmLmhgRi/cHqZ/9zpqh7x9JP1FZkWpsqcj1vDOQzQEkOmXZpU2AtM861pxdRrfNzxN0+LGgHQQ2yhRdHithxE3gapLGEFttkQsfx2Obq8h2SvX7O8syeTNEk48H6mnkKIC1ngKIOyssqJoFu2yjMwz8yOPzwTtuorif+Z4aHCK40xd6cY+3ZihFuKc6uUAixx0Vs2Am8rV8oU8H5f5RWt8enr54ZcyZgh5KjjT9VaFWFuLmTv2QqmDyjSo/v9jfx4rxLZ02oFIDJl+oo0xyde9UsfSzu8klS8nmdZfs83gvII5SGy508FMNCNa+wnfaeFi0ppFMKvmfrx4UXOTJZ/AX470la8pjp1oo24KeM2fn60aHIIc40rdxpkaggG5nFccCzMEn53BgGuI2hFU= X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2314; 23:KyfGf76CBb/u1x3LCfQ7xdZ8ytDNh3EijwdnINFZt4Mn2a9HLvZh7AKjAveY8U273uaBeBY2/77c3sLMLoofpgMn9ThQ6jAB81Hp2Mr19mWhCKENoIYTSBLRcUUMDjOOxdZnlx48RmcsQhnurLJYcg==; 6:RiwWnmm6DZAObtiJ5t2rotxJ2XKmw3CHJAWz+h++I1CrXdqpRLCP9RHBZchTtQ3cUYF78/X1oqep+pcTd+jdgatThaRCFYsdGK0P2HlgZSFNCZYsNxq7qLglT9bH1E/lr51V/5bocGkbRdbS66Pk8/oIK+mnOPgD9y9DGX+odz5dNme9fFwfSFhlPoWWbwj/05bTjTMvNKdRdlPGpI14ZrhDT+xb3BfW242dFdYzWCkxxeMc1xfSpmQKq3eP6aqmBd2REt/dwmr2bSytxeJ0xlz0mj06+r8mDw5r7nZLkPp15OJ/KLrzyP036rEi96c7oqlvvkz3NF56rlRAqPqG67dt+X3AorrwLaaAK1mKLC2x0xtZA9lQu/1IjuiKOnfHUtPgPO0TeXumQ9/P0lWFriqd1DYXe7TkwQ1f2qJeXRATl11q0LeKpE3DyV5s0fcLL2IsteA9AqDdy/5bmLJqZHZTatFgNx2v4Wco3Ql9VePcuuY3C2oCfua/d1SLODpc; 5:9zNlFrkkDa5+vJDdxDKfSs1L+3dqzqfdwzeqPTejOJjfr1JeZzMSmuK/QsWI2FUSJDqvYe2DQMoBuoQ9aP2uuoio5KZ9TSvCksXrLGOdoT6PbWLGd7TVKoT0PROKstoHRBdte6U83qnuhYNBbgza9h44g6XEE9HfrAYYzcrGywA= X-Microsoft-Antispam-Message-Info: tQyKgTp5RxTQRYsD4fWHpDkf9eqX8HN/a2bcQtoBNtScln/idBbcoCqq3p8KH/jpXNx+wfzmWkI8A1gPB7iMCWgFUP+3IbsigaEQds1G9ZeQ0wixIuzwiGBp+I6PFP4yTFfsSqtB/etSoeAvhYAwfAD+FpZSoX5AnhsYF7jg1le2pvLJ/1paS48O+yulaXCuIGp+y++wOpngkK1g62SNesRlFHNMHr7p3KD4KywV/TOD7OoQrf9NSkNohN+Siwt4BuOUNvK50sWqjvDr74esLt6oVHd0wjJb5CBAYYADWi/jQw8LJIIm0/bQQVIpZXs7Q/AOBNLCadmRcjBmdz3CEikux6Lug+ZCMoG20TyS6rY47wwcMBJeFkbQJadpqA9dvwk86JhjW4ewVfwjKgzHD8YrfWEllBkcZ9imxtfY4cFBjxKB4i1YFXFuhSKh0XVFQeKi9gvhV4D7EpSA6IBgCu3fhGwEG/25vdkwALhATjOiUt8Ni/+jcvu7jGP5kM0UAz96h8LprbtAqa3Qbcw4n+YbUleAkBgB/jp/JdAQMKjx70ud+lG0Tx7gRae+4VgrzPpW4ifPkmNcQD0kEg9bQccAPabvtMbeHrbIxxO53cO9282oSfthMY5i0hXHU/e1qmOw7uTDV3Y+LspteVDjbtcvSZoSg4sBoE31ak5jI38r0CC/CO8vs+Rv3Zn0zNqwa56EKCdQ77lKPooiGNSDbg== SpamDiagnosticOutput: 1:22 X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2314; 7:ofmOQz69/KowKgkyxfrbccWs/3+33JbaghuLP8lz5B7oNYhpa32MgRfHR1VRweYN5mwXUYsmMULA1TXJxZ9AoRotSc7XIN15CVKV9Izvo243ZKFpgrenRIVqanm8LNWRGmQsIC5OjTXoQYitmSBYtVKwYHOY1b72Os8rNis60qXwu2sWMQBZDQ/pnoX72BHAND7oyzk8M8u84iF68LujBpFxbFrwQK4fPqJPvdq+OUdGklVj1/4EvNlF+LXzavMS X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Jun 2018 23:17:58.2606 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 926d9c86-0df7-41c9-5930-08d5d95f8f56 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0501MB2314 Subject: [dpdk-dev] [PATCH v5 2/2] net/tap: support TSO (TCP Segment Offload) X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit implements TCP segmentation offload in TAP. librte_gso library is used to segment large TCP payloads (e.g. packets of 64K bytes size) into smaller MTU size buffers. By supporting TSO offload capability in software a TAP device can be used as a failsafe sub device and be paired with another PCI device which supports TSO capability in HW. For more details on librte_gso implementation please refer to dpdk documentation. The number of newly generated TCP TSO segments is limited to 64. Reviewed-by: Raslan Darawsheh Signed-off-by: Ophir Munk --- drivers/net/tap/Makefile | 2 +- drivers/net/tap/rte_eth_tap.c | 174 +++++++++++++++++++++++++++++++++++------- drivers/net/tap/rte_eth_tap.h | 3 + mk/rte.app.mk | 4 +- 4 files changed, 154 insertions(+), 29 deletions(-) diff --git a/drivers/net/tap/Makefile b/drivers/net/tap/Makefile index ccc5c5f..3243365 100644 --- a/drivers/net/tap/Makefile +++ b/drivers/net/tap/Makefile @@ -24,7 +24,7 @@ CFLAGS += -I. CFLAGS += $(WERROR_FLAGS) LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring LDLIBS += -lrte_ethdev -lrte_net -lrte_kvargs -lrte_hash -LDLIBS += -lrte_bus_vdev +LDLIBS += -lrte_bus_vdev -lrte_gso CFLAGS += -DTAP_MAX_QUEUES=$(TAP_MAX_QUEUES) diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c index 8903646..d137f58 100644 --- a/drivers/net/tap/rte_eth_tap.c +++ b/drivers/net/tap/rte_eth_tap.c @@ -17,6 +17,7 @@ #include #include +#include #include #include #include @@ -55,6 +56,12 @@ #define ETH_TAP_CMP_MAC_FMT "0123456789ABCDEFabcdef" #define ETH_TAP_MAC_ARG_FMT ETH_TAP_MAC_FIXED "|" ETH_TAP_USR_MAC_FMT +#define TAP_GSO_MBUFS_PER_CORE 128 +#define TAP_GSO_MBUF_SEG_SIZE 128 +#define TAP_GSO_MBUF_CACHE_SIZE 4 +#define TAP_GSO_MBUFS_NUM \ + (TAP_GSO_MBUFS_PER_CORE * TAP_GSO_MBUF_CACHE_SIZE) + static struct rte_vdev_driver pmd_tap_drv; static struct rte_vdev_driver pmd_tun_drv; @@ -412,7 +419,8 @@ tap_tx_offload_get_queue_capa(void) return DEV_TX_OFFLOAD_MULTI_SEGS | DEV_TX_OFFLOAD_IPV4_CKSUM | DEV_TX_OFFLOAD_UDP_CKSUM | - DEV_TX_OFFLOAD_TCP_CKSUM; + DEV_TX_OFFLOAD_TCP_CKSUM | + DEV_TX_OFFLOAD_TCP_TSO; } /* Finalize l4 checksum calculation */ @@ -480,23 +488,16 @@ tap_tx_l3_cksum(char *packet, uint64_t ol_flags, unsigned int l2_len, } } -/* Callback to handle sending packets from the tap interface - */ -static uint16_t -pmd_tx_burst(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) +static inline void +tap_write_mbufs(struct tx_queue *txq, uint16_t num_mbufs, + struct rte_mbuf **pmbufs, + uint16_t *num_packets, unsigned long *num_tx_bytes) { - struct tx_queue *txq = queue; - uint16_t num_tx = 0; - unsigned long num_tx_bytes = 0; - uint32_t max_size; int i; + uint16_t l234_hlen; - if (unlikely(nb_pkts == 0)) - return 0; - - max_size = *txq->mtu + (ETHER_HDR_LEN + ETHER_CRC_LEN + 4); - for (i = 0; i < nb_pkts; i++) { - struct rte_mbuf *mbuf = bufs[num_tx]; + for (i = 0; i < num_mbufs; i++) { + struct rte_mbuf *mbuf = pmbufs[i]; struct iovec iovecs[mbuf->nb_segs + 2]; struct tun_pi pi = { .flags = 0, .proto = 0x00 }; struct rte_mbuf *seg = mbuf; @@ -504,8 +505,7 @@ pmd_tx_burst(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) int proto; int n; int j; - int k; /* first index in iovecs for copying segments */ - uint16_t l234_hlen; /* length of layers 2,3,4 headers */ + int k; /* current index in iovecs for copying segments */ uint16_t seg_len; /* length of first segment */ uint16_t nb_segs; uint16_t *l4_cksum; /* l4 checksum (pseudo header + payload) */ @@ -513,10 +513,6 @@ pmd_tx_burst(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) uint16_t l4_phdr_cksum = 0; /* TCP/UDP pseudo header checksum */ uint16_t is_cksum = 0; /* in case cksum should be offloaded */ - /* stats.errs will be incremented */ - if (rte_pktmbuf_pkt_len(mbuf) > max_size) - break; - l4_cksum = NULL; if (txq->type == ETH_TUNTAP_TYPE_TUN) { /* @@ -558,8 +554,8 @@ pmd_tx_burst(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) if (seg_len < l234_hlen) break; - /* To change checksums, work on a - * copy of l2, l3 l4 headers. + /* To change checksums, work on a * copy of l2, l3 + * headers + l4 pseudo header */ rte_memcpy(m_copy, rte_pktmbuf_mtod(mbuf, void *), l234_hlen); @@ -603,13 +599,90 @@ pmd_tx_burst(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) n = writev(txq->fd, iovecs, j); if (n <= 0) break; + (*num_packets)++; + (*num_tx_bytes) += rte_pktmbuf_pkt_len(mbuf); + } +} + +/* Callback to handle sending packets from the tap interface + */ +static uint16_t +pmd_tx_burst(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) +{ + struct tx_queue *txq = queue; + uint16_t num_tx = 0; + uint16_t num_packets = 0; + unsigned long num_tx_bytes = 0; + uint32_t max_size; + int i; + + if (unlikely(nb_pkts == 0)) + return 0; + struct rte_mbuf *gso_mbufs[MAX_GSO_MBUFS]; + max_size = *txq->mtu + (ETHER_HDR_LEN + ETHER_CRC_LEN + 4); + for (i = 0; i < nb_pkts; i++) { + struct rte_mbuf *mbuf_in = bufs[num_tx]; + struct rte_mbuf **mbuf; + uint16_t num_mbufs = 0; + uint16_t tso_segsz = 0; + int ret; + uint16_t hdrs_len; + int j; + uint64_t tso; + + tso = mbuf_in->ol_flags & PKT_TX_TCP_SEG; + if (tso) { + struct rte_gso_ctx *gso_ctx = &txq->gso_ctx; + + assert(gso_ctx != NULL); + + /* TCP segmentation implies TCP checksum offload */ + mbuf_in->ol_flags |= PKT_TX_TCP_CKSUM; + + /* gso size is calculated without ETHER_CRC_LEN */ + hdrs_len = mbuf_in->l2_len + mbuf_in->l3_len + + mbuf_in->l4_len; + tso_segsz = mbuf_in->tso_segsz + hdrs_len; + if (unlikely(tso_segsz == hdrs_len) || + tso_segsz > *txq->mtu) { + txq->stats.errs++; + break; + } + gso_ctx->gso_size = tso_segsz; + ret = rte_gso_segment(mbuf_in, /* packet to segment */ + gso_ctx, /* gso control block */ + (struct rte_mbuf **)&gso_mbufs, /* out mbufs */ + RTE_DIM(gso_mbufs)); /* max tso mbufs */ + + /* ret contains the number of new created mbufs */ + if (ret < 0) + break; + + mbuf = gso_mbufs; + num_mbufs = ret; + } else { + /* stats.errs will be incremented */ + if (rte_pktmbuf_pkt_len(mbuf_in) > max_size) + break; + + /* ret 0 indicates no new mbufs were created */ + ret = 0; + mbuf = &mbuf_in; + num_mbufs = 1; + } + + tap_write_mbufs(txq, num_mbufs, mbuf, + &num_packets, &num_tx_bytes); num_tx++; - num_tx_bytes += mbuf->pkt_len; - rte_pktmbuf_free(mbuf); + /* free original mbuf */ + rte_pktmbuf_free(mbuf_in); + /* free tso mbufs */ + for (j = 0; j < ret; j++) + rte_pktmbuf_free(mbuf[j]); } - txq->stats.opackets += num_tx; + txq->stats.opackets += num_packets; txq->stats.errs += nb_pkts - num_tx; txq->stats.obytes += num_tx_bytes; @@ -1071,31 +1144,75 @@ tap_mac_set(struct rte_eth_dev *dev, struct ether_addr *mac_addr) } static int +tap_gso_ctx_setup(struct rte_gso_ctx *gso_ctx, struct rte_eth_dev *dev) +{ + uint32_t gso_types; + char pool_name[64]; + + /* + * Create private mbuf pool with TAP_GSO_MBUF_SEG_SIZE bytes + * size per mbuf use this pool for both direct and indirect mbufs + */ + + struct rte_mempool *mp; /* Mempool for GSO packets */ + + /* initialize GSO context */ + gso_types = DEV_TX_OFFLOAD_TCP_TSO; + snprintf(pool_name, sizeof(pool_name), "mp_%s", dev->device->name); + mp = rte_mempool_lookup((const char *)pool_name); + if (!mp) { + mp = rte_pktmbuf_pool_create(pool_name, TAP_GSO_MBUFS_NUM, + TAP_GSO_MBUF_CACHE_SIZE, 0, + RTE_PKTMBUF_HEADROOM + TAP_GSO_MBUF_SEG_SIZE, + SOCKET_ID_ANY); + if (!mp) { + struct pmd_internals *pmd = dev->data->dev_private; + RTE_LOG(DEBUG, PMD, "%s: failed to create mbuf pool for device %s\n", + pmd->name, dev->device->name); + return -1; + } + } + + gso_ctx->direct_pool = mp; + gso_ctx->indirect_pool = mp; + gso_ctx->gso_types = gso_types; + gso_ctx->gso_size = 0; /* gso_size is set in tx_burst() per packet */ + gso_ctx->flag = 0; + + return 0; +} + +static int tap_setup_queue(struct rte_eth_dev *dev, struct pmd_internals *internals, uint16_t qid, int is_rx) { + int ret; int *fd; int *other_fd; const char *dir; struct pmd_internals *pmd = dev->data->dev_private; struct rx_queue *rx = &internals->rxq[qid]; struct tx_queue *tx = &internals->txq[qid]; + struct rte_gso_ctx *gso_ctx; if (is_rx) { fd = &rx->fd; other_fd = &tx->fd; dir = "rx"; + gso_ctx = NULL; } else { fd = &tx->fd; other_fd = &rx->fd; dir = "tx"; + gso_ctx = &tx->gso_ctx; } if (*fd != -1) { /* fd for this queue already exists */ TAP_LOG(DEBUG, "%s: fd %d for %s queue qid %d exists", pmd->name, *fd, dir, qid); + gso_ctx = NULL; } else if (*other_fd != -1) { /* Only other_fd exists. dup it */ *fd = dup(*other_fd); @@ -1120,6 +1237,11 @@ tap_setup_queue(struct rte_eth_dev *dev, tx->mtu = &dev->data->mtu; rx->rxmode = &dev->data->dev_conf.rxmode; + if (gso_ctx) { + ret = tap_gso_ctx_setup(gso_ctx, dev); + if (ret) + return -1; + } tx->type = pmd->type; diff --git a/drivers/net/tap/rte_eth_tap.h b/drivers/net/tap/rte_eth_tap.h index 7b21d0d..44e2773 100644 --- a/drivers/net/tap/rte_eth_tap.h +++ b/drivers/net/tap/rte_eth_tap.h @@ -15,6 +15,7 @@ #include #include +#include #include "tap_log.h" #ifdef IFF_MULTI_QUEUE @@ -22,6 +23,7 @@ #else #define RTE_PMD_TAP_MAX_QUEUES 1 #endif +#define MAX_GSO_MBUFS 64 enum rte_tuntap_type { ETH_TUNTAP_TYPE_UNKNOWN, @@ -59,6 +61,7 @@ struct tx_queue { uint16_t *mtu; /* Pointer to MTU from dev_data */ uint16_t csum:1; /* Enable checksum offloading */ struct pkt_stats stats; /* Stats for this TX queue */ + struct rte_gso_ctx gso_ctx; /* GSO context */ }; struct pmd_internals { diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 1e32c83..e2ee879 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -38,8 +38,6 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PORT) += -lrte_port _LDLIBS-$(CONFIG_RTE_LIBRTE_PDUMP) += -lrte_pdump _LDLIBS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR) += -lrte_distributor _LDLIBS-$(CONFIG_RTE_LIBRTE_IP_FRAG) += -lrte_ip_frag -_LDLIBS-$(CONFIG_RTE_LIBRTE_GRO) += -lrte_gro -_LDLIBS-$(CONFIG_RTE_LIBRTE_GSO) += -lrte_gso _LDLIBS-$(CONFIG_RTE_LIBRTE_METER) += -lrte_meter _LDLIBS-$(CONFIG_RTE_LIBRTE_LPM) += -lrte_lpm # librte_acl needs --whole-archive because of weak functions @@ -61,6 +59,8 @@ endif _LDLIBS-y += --whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_CFGFILE) += -lrte_cfgfile +_LDLIBS-$(CONFIG_RTE_LIBRTE_GRO) += -lrte_gro +_LDLIBS-$(CONFIG_RTE_LIBRTE_GSO) += -lrte_gso _LDLIBS-$(CONFIG_RTE_LIBRTE_HASH) += -lrte_hash _LDLIBS-$(CONFIG_RTE_LIBRTE_MEMBER) += -lrte_member _LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST) += -lrte_vhost