From patchwork Tue Jun 12 16:31:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ophir Munk X-Patchwork-Id: 41011 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 441F51E9CE; Tue, 12 Jun 2018 18:32:07 +0200 (CEST) Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-eopbgr20064.outbound.protection.outlook.com [40.107.2.64]) by dpdk.org (Postfix) with ESMTP id 3E7A21E9CB for ; Tue, 12 Jun 2018 18:32:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9MXRO3iseg0+w/km3pHbhw32f0D61DuoHaBiZITj8S8=; b=MXpCRDhWfr72rNSks30zOLMkBExewqk5R3gCVy3+JHeMDnseKV4BeZBcQPhJ65VIbCbiWYKM17vlRsip/EiAXQesdhegt8ugZeQiCrA4XeOpvxmBtaKKgmc+OF7Tzo1wWMm+riKrsDRJPTCuVtcHgCEWZQ+yUpAysykzGVAsJXo= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=ophirmu@mellanox.com; Received: from mellanox.com (37.142.13.130) by HE1PR0501MB2314.eurprd05.prod.outlook.com (2603:10a6:3:27::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.841.17; Tue, 12 Jun 2018 16:32:02 +0000 From: Ophir Munk To: dev@dpdk.org, Pascal Mazon , Keith Wiles Cc: Thomas Monjalon , Olga Shern , Ophir Munk Date: Tue, 12 Jun 2018 16:31:47 +0000 Message-Id: <1528821108-12405-2-git-send-email-ophirmu@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1528821108-12405-1-git-send-email-ophirmu@mellanox.com> References: <1520629826-23055-2-git-send-email-ophirmu@mellanox.com> <1528821108-12405-1-git-send-email-ophirmu@mellanox.com> MIME-Version: 1.0 X-Originating-IP: [37.142.13.130] X-ClientProxiedBy: VI1PR08CA0183.eurprd08.prod.outlook.com (2603:10a6:800:d2::13) To HE1PR0501MB2314.eurprd05.prod.outlook.com (2603:10a6:3:27::19) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(48565401081)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:HE1PR0501MB2314; X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2314; 3:3+XO4bqUxH3WCJlgYKBOd0JPp8S/Qb7D3vzxrrgH+jtxdWegYoJnjz4X9wdETM4cHX5wPgn972C7GVWknMWTR5UdkXcw1vKX0sPRH/9rK3pYzcHe0RTL1QrdZHJZG8U8hTEDXu0nof1QRCeqFBiwcg57HC6IgRGvkIyjAgPNoWKmsmjhsfiSEU/E5mQh+nK2Fn5r1ai4uqB0UQKjT+M88H+mU006UdwSmxNcYEOtOSbneQy/70D3QjvNlun4rgJf; 25:QnEtjoXuqE6WV9bLY69ZVIUQyX8P90ErGQT4pne+wFd1piWH8Drx+EqPT5PUxb+JMak8XbwHcxGuMGkolxgcvyA2RkTkss07JRYFwgQACAPHTZp3WnU7yXRu3yopfWBSdfmj3Kdt0Vua+YUNB4Wal4fQhv1eqbh+fCZ1EyN8BBc5DI/yHbnWfj0h7r7DmL/xBt/GkP1WK0TXpoJQUaIohYXBDWiQAl7B/UUlp+EFgxolFqTu+5ZOKgR/NCifPvreBri0PA+0iwpdhVthcW7y4V6Zp27ItqyVAJpsiW3Al/jrMKm+IT78YkTgjoBK2IMwlSpPNZwMjKJzIZxYzBTpWQ==; 31:wMRa92XMVP+lmW6N1m8YinnEztPaSLyg4ICksdKNZpxBuEBHZG3hrtRr1WO+NSbw1PHABXtyueqHAAGAvJ7Bl+ZNK9lPmhEVQX7n/Tax+9egq/GWoUlRwDDLiEvVOQYr7XUEDWSTU0uwkzXH39B4Iq1c+FKobVRXmdmS+z0XwAV0VmY6HIkrHc1SNk1E3W+5QySkK1Qkt5vZmVLMsgv6QaSDUW6lFKrCf7Z2bNRluuc= X-MS-TrafficTypeDiagnostic: HE1PR0501MB2314: X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2314; 20:weLXagz6kC+rO9Fyzdgca5q648/6ZAnyDsqD2KaUJ9hmd3YZX/7/2tOywuWSOTNeI9bB6wx57anGXXai+xeNIvRYltmWhL3zATAEQZiTZ8+O7DN30+dcTWirOwhUAyrGA0WciHR8Q23GJ9MTqQJl5YzXJytrXMmSefdv6kc/kh5GXEy8Kg4KvaC5qD1YEi9g52xz397l9p75RpJO+oyXilcg52hIPuvrisbcwzWsYwoNx57ZSE1loDOO1ZPQ4flP9m91Z/7iCIE1io4CC3CrJ6J/ELnMYHccfFKgXeqitLJPMyFwBQaYL0rHxKiddYySiuD+E/23YnngpSbA9fzb2uS8JlACuQwqdQQb3uEE7YGhuHhy03A4AiGuoNtJ7enxby0SHgeoWQIyhC3jRP/pahLLmYsN4H8owno89UyZU46gPrYgIJii7fPTZGfCbX7qYXNcKnn6nXxPfGJtEq1MVDKCiCkO7hgUG9gnyn2yz4qJlI/rR+8UkQ0n1ayv0h85; 4:pUSt+iP3NtPXNbc+SG9UyvX+ll5/ArL7wx0DK2EQv5QOM/IGxBUJoXuZFYP8cKacXBotrIWT7PGsk5dej0JekaKeIB1T46mv8rwT1Xsz9RcEPJcTiIA0m93mcDNfFcUZFdXhmkKiSLdclUhdLh6RDzzcinf9+iSbRVGo9GvJZUyCURWjD62oE5CqjC+0t/xz5C+uuAeijtpp0UW0AunanQqOfmSBtzAOm3ODyBgcaEnMDmSdOD+Kfx1Na1j+aoxmVjUoVM5laPU6LFH5N6sWtQ== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(10201501046)(93006095)(93001095)(3002001)(3231254)(944501410)(52105095)(6055026)(149027)(150027)(6041310)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123558120)(20161123562045)(6072148)(201708071742011)(7699016); SRVR:HE1PR0501MB2314; BCL:0; PCL:0; RULEID:; SRVR:HE1PR0501MB2314; X-Forefront-PRVS: 07013D7479 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(376002)(366004)(346002)(396003)(39860400002)(39380400002)(199004)(189003)(478600001)(2906002)(33026002)(69596002)(106356001)(16586007)(316002)(48376002)(110136005)(86362001)(105586002)(54906003)(21086003)(66066001)(47776003)(50466002)(6116002)(3846002)(68736007)(446003)(55016002)(186003)(81156014)(107886003)(2616005)(52116002)(956004)(11346002)(7736002)(476003)(76176011)(59450400001)(7696005)(26005)(486006)(51416003)(4720700003)(53936002)(386003)(36756003)(5660300001)(97736004)(8676002)(16526019)(81166006)(8936002)(305945005)(50226002)(25786009)(6666003)(4326008); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR0501MB2314; H:mellanox.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; HE1PR0501MB2314; 23:mMPEVFC4CzPoK9PZDPQIlnp/cxE7MgPRlBCtqdg?= WlDUhk8zEzgKXywAjvCe8NTvQ9SVqsL8bofawcPW1V9kFRa5omD8hwIMii31zF2P94jlwN5uTRhB23Ewemr3cp3njRyJ4lrPOydDJoF4/DqWl8kKpf97vTxbGrzXBLy6wSl3siSDp+EyUFCrpzajnLt4koz9m6/hN87Ex1ZIJG+TS8Jm63DL7eEQ6PkIz6FMAa2q1WXladFGUHEtwAv6N5BkgpOhHze2BAv3+pBv32SGAhze5O1ZZjNpiRJ4kOX+jCXoh51ao/FfPRyT4wnV7tpk790wABw/ARCzkYyG6gjcy7uxad4/u4DuHhhZ9EfJalJDVJa42eKozULyEmhYyw2hXBiZcHAuvqVvzbg9qthZbssrL2tbpf23bqjgFCXk0Cx8SHmQomTDEZsw9GhpQjsqdi25tezgtayby63V9jFofV0qlIbf/h8N/WUWtAOgZ/qaK88eV0IfiMhGBtlTmuAu6kBqgYKs3QtsGcKOYDpkMOTqAcKqJotPKmZY7PaLFFXFZPG67Wc0F7fsPXoTx+1LkbsS4LLSKWba2vk/lEyIB+QWvnqdrPFaxcVLVXUSu6uTbLA5GiGFC4ihxlroi58msM5SQWHLWUoyw6Xk4kedhTOmlgtqQQXQ67hrNAgS5oVnGOSHMI4HoOU+zHfb/DxNuLhwEU5kE8TSdKZxcZt4bU30L8GcREGIy1DmQsuP5YqrF229Ia7hRYFxSR2cqYFVKNAxex37/Gx8PrgeqX7CxAexV8+S0qHfI0FsM45ioWLmd9LGmAp+W1AaQfufvaIh/t8Y+cN2v7FIJcRNmTUDO2ex1zMnE1MUjpOQf4JYf4hUvcGGzsSI8bt31D82nXLCkWCZAMoXdjTeXH5orOkflzZPUn9k6b6WeZj0AlR75hS6PpSGiMdIJ1b+VV8kMsspyYt3KHcEb50mtavqpaERK5qyY8c8DWOOugJC4E5FI/geF+Zs7cfhAYYoKFzYFsPOlYNgLNgYOtlQ6uw8Eg2so/QqC/Gc2WZ2/eX2sQSbIOGcRCDy/8aD4T8l83sAqgGqZYfVQCpkPz+2nQ0OER8pZ6rhBYH6c7a/2OUwFXEquPhO6GOfrzgyap3ndBjs0P+6HqA0D7nIW71FyTQLsoVgZ7G1fiq4blsT1Ezfw7SUB8W50WhEFuz557bSh93d7VileRx2l2Ch83X/7FOOtNXFqlf1DVaT5mflJKl2oQGEL4IL8D70R6bmw+HDx48vNVwnFAX+bhX3YG3gjUhnsVXsq1xOHGBExMhLlj5UBtIt8s7i+Fiw0sFfBwNFiREyz7OZw X-Microsoft-Antispam-Message-Info: JiBdtPX2Y7gvvoZcG6rBiNx4AveDirXq6gKv7sv25lkPxuRl3/Uu9A7/Qkx8mIDrO2B1RFow4FH/kzeNwVQ/su8N7vRJQDXVVqXcan5TPbnta55lvRt1UuPkCUoYtY7cP6zjk86taJrgB8rau3Cd88P5cNFm3XSJ5re/Gyoaj401vP54zrzMi0enQ+inxTj0 X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2314; 6:ug+kwLKdgf1ZpuOAylgPhPZvyJTa8G0F+SZFMSwr4kvG3CJIBBtIDONO7afOH5xupUZzO1Op/C8iWKW80niargM29edQLpOU2zigQ1Srkkf6BDY+1KXovbOzosQcCAI26pBbJnAOq3gDfxEWeR1kj2vn0CfRaBKe+R0zzO4OkD/AXeRLBjD09ZQGllGwKBQsby6N7lyoZ3VAqu/vLYltYHgGjXHEmHH0OA5LJ6SKAvXq4X06HWU7EMM2Nc+fZLl+1l+vf2YRYVn9ifklz2vmQwcdJy5NUrfGx7F/CwkbPdu5i3KONWa3qpG1vPUQcCnCyzQLBfaEWduT/7CfdNTkkFgUUzlX05/go4Whdle3lpz8pf8UJs+1TL18t4bCTSWJnqQKabo+Ufmm41HwelsQKeAurCUClGigux7ejxYz8mJzqlCgBELDCy4UOaV0yMoy4aWeD6ETWaTenhoCqra0XA==; 5:/6Exb6I3BhKLPHCQDSe9oDCnQrfxWz5M8lb+9HenAF3kfx1ny1OpLXSWIENaEr3Lw0qe1pGbLWylSmS+1DmoVjmcN6g2wlor0oxLuwH/dHtmKH7Hf1eVOE882dqGDqmrPkQkKrrL1894iXP+VhB+BRmoebwfXS2LabDHCaCVZ3k=; 24:XdTcpgFRcHLtm5iazHq2/7EHCrH3BWKzL+Cd0r5ZI7jYbNRqbbj+dIicTOdju8A30m0KOol5Wkrxb7gjqEngp/FuapMT0BzMZWqGyB/zlEg= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2314; 7:XWaEL8DoDg9vR8fRXx4KNbUFqpke+CAZKX0CGJgR1j4GiUEQfqxvSTJa2FpyM/Yadc0M3tSsXwF0zh80Ma3FHOQ4c1o+w77FF8igie+kmmAvYAvGuT8N4ZJDTmt4ole6CF0J6arQkrNqSWO028em82Ou3NDFCDczKXjgYjY77JZuMBs4jzZ+4BhwDiAZ+OMivwxEpzB5AIWiJ2DRX8vViivSpsTkzEPE7ckuKUwpHq8MJjaUcx1R2WMbwLdxwXgs X-MS-Office365-Filtering-Correlation-Id: 1ba76217-08f9-4385-b07f-08d5d08207ef X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Jun 2018 16:32:02.8705 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1ba76217-08f9-4385-b07f-08d5d08207ef X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0501MB2314 Subject: [dpdk-dev] [PATCH v4 1/2] net/tap: calculate checksums of multi segs packets X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Prior to this commit IP/UDP/TCP checksum offload calculations were skipped in case of a multi segments packet. This commit enables TAP checksum calculations for multi segments packets. The only restriction is that the first segment must contain headers of layers 3 (IP) and 4 (UDP or TCP) Reviewed-by: Raslan Darawsheh Signed-off-by: Ophir Munk --- drivers/net/tap/rte_eth_tap.c | 158 +++++++++++++++++++++++++++++------------- 1 file changed, 110 insertions(+), 48 deletions(-) diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c index df396bf..c19f053 100644 --- a/drivers/net/tap/rte_eth_tap.c +++ b/drivers/net/tap/rte_eth_tap.c @@ -415,12 +415,43 @@ tap_tx_offload_get_queue_capa(void) DEV_TX_OFFLOAD_TCP_CKSUM; } +/* Finalize l4 checksum calculation */ static void -tap_tx_offload(char *packet, uint64_t ol_flags, unsigned int l2_len, - unsigned int l3_len) +tap_tx_l4_cksum(uint16_t *l4_cksum, uint16_t l4_phdr_cksum, + uint32_t l4_raw_cksum) { - void *l3_hdr = packet + l2_len; + if (l4_cksum) { + uint32_t cksum; + + cksum = __rte_raw_cksum_reduce(l4_raw_cksum); + cksum += l4_phdr_cksum; + + cksum = ((cksum & 0xffff0000) >> 16) + (cksum & 0xffff); + cksum = (~cksum) & 0xffff; + if (cksum == 0) + cksum = 0xffff; + *l4_cksum = cksum; + } +} +/* Accumaulate L4 raw checksums */ +static void +tap_tx_l4_add_rcksum(char *l4_data, unsigned int l4_len, uint16_t *l4_cksum, + uint32_t *l4_raw_cksum) +{ + if (l4_cksum == NULL) + return; + + *l4_raw_cksum = __rte_raw_cksum(l4_data, l4_len, *l4_raw_cksum); +} + +/* L3 and L4 pseudo headers checksum offloads */ +static void +tap_tx_l3_cksum(char *packet, uint64_t ol_flags, unsigned int l2_len, + unsigned int l3_len, unsigned int l4_len, uint16_t **l4_cksum, + uint16_t *l4_phdr_cksum, uint32_t *l4_raw_cksum) +{ + void *l3_hdr = packet + l2_len; if (ol_flags & (PKT_TX_IP_CKSUM | PKT_TX_IPV4)) { struct ipv4_hdr *iph = l3_hdr; uint16_t cksum; @@ -430,38 +461,21 @@ tap_tx_offload(char *packet, uint64_t ol_flags, unsigned int l2_len, iph->hdr_checksum = (cksum == 0xffff) ? cksum : ~cksum; } if (ol_flags & PKT_TX_L4_MASK) { - uint16_t l4_len; - uint32_t cksum; - uint16_t *l4_cksum; void *l4_hdr; l4_hdr = packet + l2_len + l3_len; if ((ol_flags & PKT_TX_L4_MASK) == PKT_TX_UDP_CKSUM) - l4_cksum = &((struct udp_hdr *)l4_hdr)->dgram_cksum; + *l4_cksum = &((struct udp_hdr *)l4_hdr)->dgram_cksum; else if ((ol_flags & PKT_TX_L4_MASK) == PKT_TX_TCP_CKSUM) - l4_cksum = &((struct tcp_hdr *)l4_hdr)->cksum; + *l4_cksum = &((struct tcp_hdr *)l4_hdr)->cksum; else return; - *l4_cksum = 0; - if (ol_flags & PKT_TX_IPV4) { - struct ipv4_hdr *iph = l3_hdr; - - l4_len = rte_be_to_cpu_16(iph->total_length) - l3_len; - cksum = rte_ipv4_phdr_cksum(l3_hdr, 0); - } else { - struct ipv6_hdr *ip6h = l3_hdr; - - /* payload_len does not include ext headers */ - l4_len = rte_be_to_cpu_16(ip6h->payload_len) - - l3_len + sizeof(struct ipv6_hdr); - cksum = rte_ipv6_phdr_cksum(l3_hdr, 0); - } - cksum += rte_raw_cksum(l4_hdr, l4_len); - cksum = ((cksum & 0xffff0000) >> 16) + (cksum & 0xffff); - cksum = (~cksum) & 0xffff; - if (cksum == 0) - cksum = 0xffff; - *l4_cksum = cksum; + **l4_cksum = 0; + if (ol_flags & PKT_TX_IPV4) + *l4_phdr_cksum = rte_ipv4_phdr_cksum(l3_hdr, 0); + else + *l4_phdr_cksum = rte_ipv6_phdr_cksum(l3_hdr, 0); + *l4_raw_cksum = __rte_raw_cksum(l4_hdr, l4_len, 0); } } @@ -482,17 +496,27 @@ pmd_tx_burst(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) max_size = *txq->mtu + (ETHER_HDR_LEN + ETHER_CRC_LEN + 4); for (i = 0; i < nb_pkts; i++) { struct rte_mbuf *mbuf = bufs[num_tx]; - struct iovec iovecs[mbuf->nb_segs + 1]; + struct iovec iovecs[mbuf->nb_segs + 2]; struct tun_pi pi = { .flags = 0, .proto = 0x00 }; struct rte_mbuf *seg = mbuf; char m_copy[mbuf->data_len]; + int proto; int n; int j; + int k; /* first index in iovecs for copying segments */ + uint16_t l234_hlen; /* length of layers 2,3,4 headers */ + uint16_t seg_len; /* length of first segment */ + uint16_t nb_segs; + uint16_t *l4_cksum; /* l4 checksum (pseudo header + payload) */ + uint32_t l4_raw_cksum = 0; /* TCP/UDP payload raw checksum */ + uint16_t l4_phdr_cksum = 0; /* TCP/UDP pseudo header checksum */ + uint16_t is_cksum = 0; /* in case cksum should be offloaded */ /* stats.errs will be incremented */ if (rte_pktmbuf_pkt_len(mbuf) > max_size) break; + l4_cksum = NULL; if (txq->type == ETH_TUNTAP_TYPE_TUN) { /* * TUN and TAP are created with IFF_NO_PI disabled. @@ -505,35 +529,73 @@ pmd_tx_burst(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) * is 4 or 6, then protocol field is updated. */ char *buff_data = rte_pktmbuf_mtod(seg, void *); - j = (*buff_data & 0xf0); - pi.proto = (j == 0x40) ? rte_cpu_to_be_16(ETHER_TYPE_IPv4) : - (j == 0x60) ? rte_cpu_to_be_16(ETHER_TYPE_IPv6) : 0x00; + proto = (*buff_data & 0xf0); + pi.proto = (proto == 0x40) ? + rte_cpu_to_be_16(ETHER_TYPE_IPv4) : + ((proto == 0x60) ? + rte_cpu_to_be_16(ETHER_TYPE_IPv6) : + 0x00); } - iovecs[0].iov_base = π - iovecs[0].iov_len = sizeof(pi); - for (j = 1; j <= mbuf->nb_segs; j++) { - iovecs[j].iov_len = rte_pktmbuf_data_len(seg); - iovecs[j].iov_base = - rte_pktmbuf_mtod(seg, void *); - seg = seg->next; - } + k = 0; + iovecs[k].iov_base = π + iovecs[k].iov_len = sizeof(pi); + k++; + nb_segs = mbuf->nb_segs; if (txq->csum && ((mbuf->ol_flags & (PKT_TX_IP_CKSUM | PKT_TX_IPV4) || (mbuf->ol_flags & PKT_TX_L4_MASK) == PKT_TX_UDP_CKSUM || (mbuf->ol_flags & PKT_TX_L4_MASK) == PKT_TX_TCP_CKSUM))) { - /* Support only packets with all data in the same seg */ - if (mbuf->nb_segs > 1) + is_cksum = 1; + /* Support only packets with at least layer 4 + * header included in the first segment + */ + seg_len = rte_pktmbuf_data_len(mbuf); + l234_hlen = mbuf->l2_len + mbuf->l3_len + mbuf->l4_len; + if (seg_len < l234_hlen) break; - /* To change checksums, work on a copy of data. */ + + /* To change checksums, work on a + * copy of l2, l3 l4 headers. + */ rte_memcpy(m_copy, rte_pktmbuf_mtod(mbuf, void *), - rte_pktmbuf_data_len(mbuf)); - tap_tx_offload(m_copy, mbuf->ol_flags, - mbuf->l2_len, mbuf->l3_len); - iovecs[1].iov_base = m_copy; + l234_hlen); + tap_tx_l3_cksum(m_copy, mbuf->ol_flags, + mbuf->l2_len, mbuf->l3_len, mbuf->l4_len, + &l4_cksum, &l4_phdr_cksum, + &l4_raw_cksum); + iovecs[k].iov_base = m_copy; + iovecs[k].iov_len = l234_hlen; + k++; + /* Update next iovecs[] beyond l2, l3, l4 headers */ + if (seg_len > l234_hlen) { + iovecs[k].iov_len = seg_len - l234_hlen; + iovecs[k].iov_base = + rte_pktmbuf_mtod(seg, char *) + + l234_hlen; + tap_tx_l4_add_rcksum(iovecs[k].iov_base, + iovecs[k].iov_len, l4_cksum, + &l4_raw_cksum); + k++; + nb_segs++; + } + seg = seg->next; } + for (j = k; j <= nb_segs; j++) { + iovecs[j].iov_len = rte_pktmbuf_data_len(seg); + iovecs[j].iov_base = rte_pktmbuf_mtod(seg, void *); + if (is_cksum) + tap_tx_l4_add_rcksum(iovecs[j].iov_base, + iovecs[j].iov_len, l4_cksum, + &l4_raw_cksum); + seg = seg->next; + } + + if (is_cksum) + tap_tx_l4_cksum(l4_cksum, l4_phdr_cksum, l4_raw_cksum); + /* copy the tx frame data */ - n = writev(txq->fd, iovecs, mbuf->nb_segs + 1); + n = writev(txq->fd, iovecs, j); if (n <= 0) break; From patchwork Tue Jun 12 16:31:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ophir Munk X-Patchwork-Id: 41013 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 718921EA0D; Tue, 12 Jun 2018 18:32:17 +0200 (CEST) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01hn0230.outbound.protection.outlook.com [104.47.1.230]) by dpdk.org (Postfix) with ESMTP id 4C4671E9FA for ; Tue, 12 Jun 2018 18:32:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=42EwGFjp2tC4fj862VMaoOIe5NRiSs0q5nc8n2xGHqk=; b=nDw+nw86DrGP9SDf3VhXJM1gixjy5MCE7pBN74xTI3Ki5SUBGKOPaK7B2eJSt5Aa5gGerDFEUTRSz7kPJtSfz0qaUZhJDXwJtoe4h6F5HwPVIOuhtkgh+8xCMKalws0TGB9QhVr0SPgamjs0w0jxyf8nphqsMhl7wdKgLIy56dw= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=ophirmu@mellanox.com; Received: from mellanox.com (37.142.13.130) by HE1PR0501MB2314.eurprd05.prod.outlook.com (2603:10a6:3:27::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.841.17; Tue, 12 Jun 2018 16:32:04 +0000 From: Ophir Munk To: dev@dpdk.org, Pascal Mazon , Keith Wiles Cc: Thomas Monjalon , Olga Shern , Ophir Munk Date: Tue, 12 Jun 2018 16:31:48 +0000 Message-Id: <1528821108-12405-3-git-send-email-ophirmu@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1528821108-12405-1-git-send-email-ophirmu@mellanox.com> References: <1520629826-23055-2-git-send-email-ophirmu@mellanox.com> <1528821108-12405-1-git-send-email-ophirmu@mellanox.com> MIME-Version: 1.0 X-Originating-IP: [37.142.13.130] X-ClientProxiedBy: VI1PR08CA0183.eurprd08.prod.outlook.com (2603:10a6:800:d2::13) To HE1PR0501MB2314.eurprd05.prod.outlook.com (2603:10a6:3:27::19) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(48565401081)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:HE1PR0501MB2314; X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2314; 3:XKSjyjzKa1ye5UbYJ3bzsw3iRpMWjR4xYgq4pOsesa46DwJ5hmlojLe2L7l0bSLQOEjkR94+JzWd827U/W+UuGUEArn2uzFaUM/UqbRNgJlneL7tS/de6YDJijlA/1ijUkMqCTnzxevWnOsBbon1jvxyc+7B+yAzwtoryeUgZvq+sp+RpAnm/mSzoEMRXTJP4XOBRw2SXWHUGM78iMmyUy/wHQkMpx7RVVzDZps0/Mn55AVpTr+qT9b5GTGSJQM1; 25:1QnuY92LMVB5/poKkjhd6p2zxD2n9/n39OywSzN8Qxss5x6pOOCIG2GEzgKCtBGsY5RtQzEnk9en+BX68G5kVaQ+M5oCTcjdxm/UX3VIaOyomS+CHHRkYxS9sMN6kL7go0RmywKlBxqbprZji9vdY7zJNOdLUyJFL9+oYSiYfQnqG8vemDYunja/S6VX7x2eD/CSQ9B9OuiMPdtoTm+JbS7B/7r9DrRNJQHKVR2MDvIvsjG3ElwFPTp1JI5blnvC/Z0vIGxXFhyfIyYLIU7fGfhAvYnKsO5FNoN8ioFEnP8rf2aPaauoZTLbwi0+4ryLPwZ7e0/qLzI3lN4bzPsZ0Q==; 31:VTqdS/bMQNFW73TqQnhBCYaanOkaWfwoCDgB4dap8AJaCLblMGYHAqPOSshYmAiaLqoMn6nbSv6IeaPUblpf1rQ0gZneHVG0a7OENO7uHJHtvNEFwtdSXlqAzPdIrL2mEx+CurPmYVf6ZErdILbL7aWChLLQMA1xp+uCj7FuAer8AS4XqkhJkhu/pk0HfGW6Rab/Xt8huD/YnbJO1HuiwU0fTSiBIQkOIEvjq7ysLMg= X-MS-TrafficTypeDiagnostic: HE1PR0501MB2314:|HE1PR0501MB2314: X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2314; 20:Ud9Atc1BT5reQsKNtRWidUCHRla3sXvhT9LLZ6DTEIgd3bUb2ohyEFFRFJmLoMojpZVezDgYIJl3FTKVqKrObibl+rFLcFjtlZg/KTcNsw/9UpbUb8KbpwYCZCYNQNOmOB+Jq5T1xyX65bsDoUtvYqj8w0lM4DIi250kcmDMLm8beBOf7Whon2LkzIue4N8vXoCYzxM80Q4BhCZgHM0gmdKN/pDVJx8JuNoC3xqR2X0qXeRNHUpTuoZ+Vmt9fG29qXTKJ0rWcuXtnbXMvbumkC+o33vUB0Pcyvm7spNys6s74K6XdQ5lujojc06BLwFOVSFO0joSfS4VFD6OdgRnZLkyfdSGVPzZkArUZG+ltnmwuV/VCIjIuwzscQ+YGnHXj85ZnrEjfvY38QFdmK+OxN7YcnwzxBr4Mw1+PHW1wPnviS721FtUTEZF91kG+HSWWdXQnfpDJQCPnE807hZMc1WoWS6Ttcc+bMo+rsp+7ELqCO5bmT3g5hbN5Qvq6//O; 4:KFMlAglAAE4vyvAt5lyqcI7a4ILSKihwQEHtxXFKbH0aQC4/ZTOf4dqnstVFpOLiRL4ypDp6tkjwCD/eOmdqEMyQn5vWg9OjGAdfz3omb1lS6ADjDez5fdvvcZAlzGWBv7hcd7sKqcdQ4BaJin21jrtrFVD53GppTTTNkZgUN2A1oUnnR4jCn+n9Wu5ni7uT8lNxyI7iyuyxnFXyN+gphrPl9srMNvH5JBFVvZJOGqRQIlN93+RyaLIWpfZDuCrcwqUSvenEfGk18IAp+tYzLNO/ZFeHYTukN9MABfqd2VkIuRml9Ob+CmZAPMTadxia X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(66839620246622); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(10201501046)(93006095)(93001095)(3002001)(3231254)(2232076)(944501410)(52105095)(6055026)(149027)(150027)(6041310)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123558120)(20161123562045)(6072148)(201708071742011)(7699016); SRVR:HE1PR0501MB2314; BCL:0; PCL:0; RULEID:; SRVR:HE1PR0501MB2314; X-Forefront-PRVS: 07013D7479 X-Forefront-Antispam-Report: SFV:SPM; SFS:(10009020)(376002)(366004)(346002)(396003)(39860400002)(39380400002)(199004)(189003)(575784001)(478600001)(2906002)(33026002)(69596002)(106356001)(16586007)(316002)(48376002)(110136005)(86362001)(105586002)(54906003)(21086003)(66066001)(47776003)(50466002)(6116002)(3846002)(68736007)(446003)(55016002)(186003)(81156014)(107886003)(2616005)(52116002)(956004)(11346002)(7736002)(476003)(76176011)(7696005)(26005)(486006)(51416003)(4720700003)(53936002)(386003)(36756003)(5660300001)(97736004)(8676002)(16526019)(81166006)(8936002)(305945005)(50226002)(25786009)(6666003)(4326008)(59010400001); DIR:OUT; SFP:1501; SCL:5; SRVR:HE1PR0501MB2314; H:mellanox.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; HE1PR0501MB2314; 23:VDdo8UiJ86b92Tdo0MzH/n62BlvGSckojAwgtHQ?= nc13N6VD+B1jmI3ZFEavhttCr1HWr+PBJBnB1fQhAvaUV4ilnDaMFv3/d2NRi6EEFxZDCnewHMK+wcj23gad2vliNf5k+7fFGxem+HKeXcIpCwrdNaVfLUlimM42wywTaZflL346kd4vIpwvw9j3f0VLysYmr49gwN4wQfSHkW8ED4rUD/EbDMl4TJahlXowxLhPKERv1iaMzk66c64C1tBzTCc4VcoavEkQLLXXWcH8ZE2+9Av9nnPAnBddeeyrHYw7+yxmseDC1o2U2bhVJf5h+lepJU4joXm0BnjE/Ll7j6ungjmb3N+oy8ZFrbTy+7xCiTk5watFubficZA54SoKTEwNKJ9733vpYGOyOHslDGWYZ4b6+lAWCPS9MRAyOm6wEgUBzQUte3TfXh6caltSfwtKjukQyu3UxglwTZmedAW0Nfo9LgAnxXVngKuYockWeotK8weWT2S+YtUt6pBw+0rIAj+8OzxlXuqg19JUPNF4/FyUruEAzIJhtFhEXosavxllZLJ0z2yC4Jhtrjfb4Hg+ZmGfDd8wrZ9jFAp42gG43v3oHb/vEIfTLAzXkiwfua1gwJWQA1do3sf1Byjvo23BiwVJ/+bzFRiu+IEQpsi3TvdiFvDm1QXYrMbPgsTbVOfHKRC7Dh4DEG3Cz2Lw1gGp6O0vT4VLHA9rmtVVvAb2mw/NX/TgRdk8aGIQ+bRI0eVxPPrRWTSUA6n5Apfa250PTo2e5IzzJogVZMPQkwW4TQSTEPrPwPzYcrXDDJ1pn4T/lQAuNVGTSY9rico5F/vHm7/qnD114/3dUc2XgWWZnlILU53iGgrR0VPhjSjFzajg7tNfXBJjU2uOLo4DqoldyElVSpyjNVFQOkgVKxf5I/8KQrRUVFV4+gq9VuGtJWfxT4GqUkJeszu+mq8zV+CNYKgbNnZDOtJsxSOXbNrmO2rwELXtzrNVrvwMIazvrmniYbmkQBLbRv3vpr0ad6WEUz7T0DmSYHqkfGKuxuTephQ9gzbKl7bydx1nYuIm+2Nmhucko52EeH52Z1FJpgdcorROL1TDl1DpHuRXphk99JIOTlHI/4AMaP8g8hTflzfbpFAnO5yDb/C+iG5ayedz3SRwn0T+YhOOO71XoSFiFi+fGlq85UhZDWRW9l0zWyM6R12GqWlAbz/GhbODa92ZNHCXXwYd9eKemi6XAniCsYqQK50eKJTlhCrEVMkHX1YEp4PsbERBC0LvSl444YSel5wP9x7zmFl9bLKOmnn4VNbUz6ogRcxa/Xuo2u4JylRkRZdRacUInu/Rg4BS/3E7uzT4K99GJN1aaEhSbz2czehgj4ZNADXMPkdi2lmo= X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2314; 23:xAvzfhmEXZcrTmTv9GPyrvFUyIYN+7rEXcWS1L7Bb+W33B7MdI9ps51fU20g+PdaVZEwH8Ouz8ep8/JKyHsmqoVn8HSpw22YIQ+3ZWXsdSL00UNhys8pglqZPd4G/syrQriXPNxmVy7HckGr69w6NQ==; 6:csTA0F0feeFG6NL+XoFS3a1Pzq8QJ+2TQjAS+3zdfqOObfgQls0PfjSZfCVjbYGKWD5OWSm4VbnmtOLyxhZfzuoOSRrilK1ODajR+RHQcT0twRWaWtHyphuUwjWnlCoPUcYd2VFvf02TYBpdnhm4ayvP/9Vy1vW1Dp4wnirlU17+ZuvQ8ceFRBfcUFOhUuH+bvRKmdDnRkvNvzYamQB6JOma90ETqqEY+P7Cnw767/o6xou0Jm6AtH4yGqErxESVlkufaXx5vU3ttX0jUBaAka5J6Uc2dR6eCePDlu4dj+BjX5/1fBDbw5bdA9aemKi6n7eVUvSESXQ0G8xbBSszcOsHZ1KzHckPhSEEGHsGQ5WAjJdkduNheDlotUQ8Nc3mIqcan5U5c8WB0NtSAxuYM96XCqoE6Nf7gurDh1ulozx9cHOqViIa4khhOgTE8aTJdhntA2pPHYfZGdCQdj0vTlDrSFO3FazI5jxABVkN5zc534Ik5O75eULAVPeEDKMU; 5:7PadP97HsZHRxaa7WIqINJfBqZuS1zL7jrM+rkWRnCY4GxJ+asV28rn66ZFu+QaxjBQ/CXkm76KryHC26AOB5n2kmiumHtZK42DlGG2R3F60B2l2Zay7K06DAXqdMcO0RFEfGvM/1d7IMfOGSN5m/zEh3jX0h29sTktCtz1ER9A= X-Microsoft-Antispam-Message-Info: NFApwuy+lUQHSFQ/5sEc5ktMxkg8OPbFwZ+sFZS5Ogf98yxrHQnR/Ggv6//xi84hZAtcpd238nnvujFGCQJMFSBC1BT4V8H+I4x/t6AqZlASzXyu6UknS+VNo83AyzrBzKvLv6CLiKSZ+hsSiTbLSuHhxSd1xDzyKIwAJKkDpn6hm2p7GQ7Toofyuh6I8OANvGZuf4jNogAASgAMkOx1uv3dgxuv6EheQleb6tEnjDRMGs51gF+vjw9FcZSB+Njwk+a+MAeIKP0eFOhRsP6KN2D9YIkaqqjMNg8HwfkL1nsiAkibhjiaS3Hz+IEfV1Ne0wVSKuyAM/u5jZnXqSZ1p4HI64kVpcawRW+PvK6rKeNqvfh8S9BX9jhrXqIOZIe8NqH0tdirre+8ZaBONtQacc8+visI/T3qfMcS8B+5oVjc8VDhE+ehZ48HKnHJAXTmbKBOiULGOO1szgodWOSxsPHA76OT+rgww3iuVfEA+saiKwNjP4rmDrkAOA/4E7trG8ldJ9yTk0YveWfNH+hkkwwVoZRNHcfncXjYho/EIqQ= SpamDiagnosticOutput: 1:22 X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2314; 7:aEM2QG/z5BG/0CKiYxSSi4DQF2QTwwpXbHcTZXx6SFPOVAGjzvcVfvvSmeIrof57XNeZNlc0LEs4VbrNs0nGk5io09mA+A8G40/P4gJfHROGKVr3UI9ydLFr0FLUDqNIUK5NEsrjhse2X99DOf25nYft7Vj8S6TX4oieUW1QdvzTYQxWKoHUCsbhFFizzFNh0pfoz5n+3vjJ5wvWunBjpJ9J1FiquVzn11P/HjwksaxX5Jk/5cDwyztl99e69jSN X-MS-Office365-Filtering-Correlation-Id: 7be90f73-1ed7-4015-56da-08d5d08208f8 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Jun 2018 16:32:04.5111 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7be90f73-1ed7-4015-56da-08d5d08208f8 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0501MB2314 Subject: [dpdk-dev] [PATCH v4 2/2] net/tap: support TSO (TCP Segment Offload) X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This commit implements TCP segmentation offload in TAP. librte_gso library is used to segment large TCP payloads (e.g. packets of 64K bytes size) into smaller MTU size buffers. By supporting TSO offload capability in software a TAP device can be used as a failsafe sub device and be paired with another PCI device which supports TSO capability in HW. For more details on librte_gso implementation please refer to dpdk documentation. The number of newly generated TCP TSO segments is limited to 64. Reviewed-by: Raslan Darawsheh Signed-off-by: Ophir Munk --- drivers/net/tap/Makefile | 2 +- drivers/net/tap/rte_eth_tap.c | 159 +++++++++++++++++++++++++++++++++++------- drivers/net/tap/rte_eth_tap.h | 3 + mk/rte.app.mk | 4 +- 4 files changed, 138 insertions(+), 30 deletions(-) diff --git a/drivers/net/tap/Makefile b/drivers/net/tap/Makefile index ccc5c5f..3243365 100644 --- a/drivers/net/tap/Makefile +++ b/drivers/net/tap/Makefile @@ -24,7 +24,7 @@ CFLAGS += -I. CFLAGS += $(WERROR_FLAGS) LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring LDLIBS += -lrte_ethdev -lrte_net -lrte_kvargs -lrte_hash -LDLIBS += -lrte_bus_vdev +LDLIBS += -lrte_bus_vdev -lrte_gso CFLAGS += -DTAP_MAX_QUEUES=$(TAP_MAX_QUEUES) diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c index c19f053..62b931f 100644 --- a/drivers/net/tap/rte_eth_tap.c +++ b/drivers/net/tap/rte_eth_tap.c @@ -17,6 +17,7 @@ #include #include +#include #include #include #include @@ -55,6 +56,9 @@ #define ETH_TAP_CMP_MAC_FMT "0123456789ABCDEFabcdef" #define ETH_TAP_MAC_ARG_FMT ETH_TAP_MAC_FIXED "|" ETH_TAP_USR_MAC_FMT +#define TAP_GSO_MBUFS_NUM 128 +#define TAP_GSO_MBUF_SEG_SIZE 128 + static struct rte_vdev_driver pmd_tap_drv; static struct rte_vdev_driver pmd_tun_drv; @@ -412,7 +416,8 @@ tap_tx_offload_get_queue_capa(void) return DEV_TX_OFFLOAD_MULTI_SEGS | DEV_TX_OFFLOAD_IPV4_CKSUM | DEV_TX_OFFLOAD_UDP_CKSUM | - DEV_TX_OFFLOAD_TCP_CKSUM; + DEV_TX_OFFLOAD_TCP_CKSUM | + DEV_TX_OFFLOAD_TCP_TSO; } /* Finalize l4 checksum calculation */ @@ -479,23 +484,15 @@ tap_tx_l3_cksum(char *packet, uint64_t ol_flags, unsigned int l2_len, } } -/* Callback to handle sending packets from the tap interface - */ -static uint16_t -pmd_tx_burst(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) +static inline void +tap_write_mbufs(struct tx_queue *txq, uint16_t num_mbufs, + struct rte_mbuf **pmbufs, uint16_t l234_hlen, + uint16_t *num_packets, unsigned long *num_tx_bytes) { - struct tx_queue *txq = queue; - uint16_t num_tx = 0; - unsigned long num_tx_bytes = 0; - uint32_t max_size; int i; - if (unlikely(nb_pkts == 0)) - return 0; - - max_size = *txq->mtu + (ETHER_HDR_LEN + ETHER_CRC_LEN + 4); - for (i = 0; i < nb_pkts; i++) { - struct rte_mbuf *mbuf = bufs[num_tx]; + for (i = 0; i < num_mbufs; i++) { + struct rte_mbuf *mbuf = pmbufs[i]; struct iovec iovecs[mbuf->nb_segs + 2]; struct tun_pi pi = { .flags = 0, .proto = 0x00 }; struct rte_mbuf *seg = mbuf; @@ -503,8 +500,7 @@ pmd_tx_burst(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) int proto; int n; int j; - int k; /* first index in iovecs for copying segments */ - uint16_t l234_hlen; /* length of layers 2,3,4 headers */ + int k; /* current index in iovecs for copying segments */ uint16_t seg_len; /* length of first segment */ uint16_t nb_segs; uint16_t *l4_cksum; /* l4 checksum (pseudo header + payload) */ @@ -512,10 +508,6 @@ pmd_tx_burst(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) uint16_t l4_phdr_cksum = 0; /* TCP/UDP pseudo header checksum */ uint16_t is_cksum = 0; /* in case cksum should be offloaded */ - /* stats.errs will be incremented */ - if (rte_pktmbuf_pkt_len(mbuf) > max_size) - break; - l4_cksum = NULL; if (txq->type == ETH_TUNTAP_TYPE_TUN) { /* @@ -554,9 +546,8 @@ pmd_tx_burst(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) l234_hlen = mbuf->l2_len + mbuf->l3_len + mbuf->l4_len; if (seg_len < l234_hlen) break; - - /* To change checksums, work on a - * copy of l2, l3 l4 headers. + /* To change checksums, work on a * copy of l2, l3 + * headers + l4 pseudo header */ rte_memcpy(m_copy, rte_pktmbuf_mtod(mbuf, void *), l234_hlen); @@ -598,13 +589,80 @@ pmd_tx_burst(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) n = writev(txq->fd, iovecs, j); if (n <= 0) break; + (*num_packets)++; + (*num_tx_bytes) += rte_pktmbuf_pkt_len(mbuf); + } +} + +/* Callback to handle sending packets from the tap interface + */ +static uint16_t +pmd_tx_burst(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) +{ + struct tx_queue *txq = queue; + uint16_t num_tx = 0; + uint16_t num_packets = 0; + unsigned long num_tx_bytes = 0; + uint16_t tso_segsz = 0; + uint16_t hdrs_len; + uint32_t max_size; + int i; + uint64_t tso; + int ret; + if (unlikely(nb_pkts == 0)) + return 0; + + struct rte_mbuf *gso_mbufs[MAX_GSO_MBUFS]; + max_size = *txq->mtu + (ETHER_HDR_LEN + ETHER_CRC_LEN + 4); + for (i = 0; i < nb_pkts; i++) { + struct rte_mbuf *mbuf_in = bufs[num_tx]; + struct rte_mbuf **mbuf; + uint16_t num_mbufs; + + tso = mbuf_in->ol_flags & PKT_TX_TCP_SEG; + if (tso) { + struct rte_gso_ctx *gso_ctx = &txq->gso_ctx; + assert(gso_ctx != NULL); + /* TCP segmentation implies TCP checksum offload */ + mbuf_in->ol_flags |= PKT_TX_TCP_CKSUM; + /* gso size is calculated without ETHER_CRC_LEN */ + hdrs_len = mbuf_in->l2_len + mbuf_in->l3_len + + mbuf_in->l4_len; + tso_segsz = mbuf_in->tso_segsz + hdrs_len; + if (unlikely(tso_segsz == hdrs_len) || + tso_segsz > *txq->mtu) { + txq->stats.errs++; + break; + } + gso_ctx->gso_size = tso_segsz; + ret = rte_gso_segment(mbuf_in, /* packet to segment */ + gso_ctx, /* gso control block */ + (struct rte_mbuf **)&gso_mbufs, /* out mbufs */ + RTE_DIM(gso_mbufs)); /* max tso mbufs */ + + /* ret contains the number of new created mbufs */ + if (ret < 0) + break; + + mbuf = gso_mbufs; + num_mbufs = ret; + } else { + /* stats.errs will be incremented */ + if (rte_pktmbuf_pkt_len(mbuf_in) > max_size) + break; + + mbuf = &mbuf_in; + num_mbufs = 1; + } + + tap_write_mbufs(txq, num_mbufs, mbuf, hdrs_len, + &num_packets, &num_tx_bytes); num_tx++; - num_tx_bytes += mbuf->pkt_len; - rte_pktmbuf_free(mbuf); + rte_pktmbuf_free(mbuf_in); } - txq->stats.opackets += num_tx; + txq->stats.opackets += num_packets; txq->stats.errs += nb_pkts - num_tx; txq->stats.obytes += num_tx_bytes; @@ -1066,31 +1124,73 @@ tap_mac_set(struct rte_eth_dev *dev, struct ether_addr *mac_addr) } static int +tap_gso_ctx_setup(struct rte_gso_ctx *gso_ctx, struct rte_eth_dev *dev) +{ + uint32_t gso_types; + char pool_name[64]; + + /* + * Create private mbuf pool with TAP_GSO_MBUF_SEG_SIZE bytes + * size per mbuf use this pool for both direct and indirect mbufs + */ + + struct rte_mempool *mp; /* Mempool for GSO packets */ + /* initialize GSO context */ + gso_types = DEV_TX_OFFLOAD_TCP_TSO; + snprintf(pool_name, sizeof(pool_name), "mp_%s", dev->device->name); + mp = rte_mempool_lookup((const char *)pool_name); + if (!mp) { + mp = rte_pktmbuf_pool_create(pool_name, TAP_GSO_MBUFS_NUM, + 0, 0, RTE_PKTMBUF_HEADROOM + TAP_GSO_MBUF_SEG_SIZE, + SOCKET_ID_ANY); + if (!mp) { + struct pmd_internals *pmd = dev->data->dev_private; + RTE_LOG(DEBUG, PMD, "%s: failed to create mbuf pool for device %s\n", + pmd->name, dev->device->name); + return -1; + } + } + + gso_ctx->direct_pool = mp; + gso_ctx->indirect_pool = mp; + gso_ctx->gso_types = gso_types; + gso_ctx->gso_size = 0; /* gso_size is set in tx_burst() per packet */ + gso_ctx->flag = 0; + + return 0; +} + +static int tap_setup_queue(struct rte_eth_dev *dev, struct pmd_internals *internals, uint16_t qid, int is_rx) { + int ret; int *fd; int *other_fd; const char *dir; struct pmd_internals *pmd = dev->data->dev_private; struct rx_queue *rx = &internals->rxq[qid]; struct tx_queue *tx = &internals->txq[qid]; + struct rte_gso_ctx *gso_ctx; if (is_rx) { fd = &rx->fd; other_fd = &tx->fd; dir = "rx"; + gso_ctx = NULL; } else { fd = &tx->fd; other_fd = &rx->fd; dir = "tx"; + gso_ctx = &tx->gso_ctx; } if (*fd != -1) { /* fd for this queue already exists */ TAP_LOG(DEBUG, "%s: fd %d for %s queue qid %d exists", pmd->name, *fd, dir, qid); + gso_ctx = NULL; } else if (*other_fd != -1) { /* Only other_fd exists. dup it */ *fd = dup(*other_fd); @@ -1115,6 +1215,11 @@ tap_setup_queue(struct rte_eth_dev *dev, tx->mtu = &dev->data->mtu; rx->rxmode = &dev->data->dev_conf.rxmode; + if (gso_ctx) { + ret = tap_gso_ctx_setup(gso_ctx, dev); + if (ret) + return -1; + } tx->type = pmd->type; diff --git a/drivers/net/tap/rte_eth_tap.h b/drivers/net/tap/rte_eth_tap.h index 7b21d0d..44e2773 100644 --- a/drivers/net/tap/rte_eth_tap.h +++ b/drivers/net/tap/rte_eth_tap.h @@ -15,6 +15,7 @@ #include #include +#include #include "tap_log.h" #ifdef IFF_MULTI_QUEUE @@ -22,6 +23,7 @@ #else #define RTE_PMD_TAP_MAX_QUEUES 1 #endif +#define MAX_GSO_MBUFS 64 enum rte_tuntap_type { ETH_TUNTAP_TYPE_UNKNOWN, @@ -59,6 +61,7 @@ struct tx_queue { uint16_t *mtu; /* Pointer to MTU from dev_data */ uint16_t csum:1; /* Enable checksum offloading */ struct pkt_stats stats; /* Stats for this TX queue */ + struct rte_gso_ctx gso_ctx; /* GSO context */ }; struct pmd_internals { diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 1e32c83..e2ee879 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -38,8 +38,6 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PORT) += -lrte_port _LDLIBS-$(CONFIG_RTE_LIBRTE_PDUMP) += -lrte_pdump _LDLIBS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR) += -lrte_distributor _LDLIBS-$(CONFIG_RTE_LIBRTE_IP_FRAG) += -lrte_ip_frag -_LDLIBS-$(CONFIG_RTE_LIBRTE_GRO) += -lrte_gro -_LDLIBS-$(CONFIG_RTE_LIBRTE_GSO) += -lrte_gso _LDLIBS-$(CONFIG_RTE_LIBRTE_METER) += -lrte_meter _LDLIBS-$(CONFIG_RTE_LIBRTE_LPM) += -lrte_lpm # librte_acl needs --whole-archive because of weak functions @@ -61,6 +59,8 @@ endif _LDLIBS-y += --whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_CFGFILE) += -lrte_cfgfile +_LDLIBS-$(CONFIG_RTE_LIBRTE_GRO) += -lrte_gro +_LDLIBS-$(CONFIG_RTE_LIBRTE_GSO) += -lrte_gso _LDLIBS-$(CONFIG_RTE_LIBRTE_HASH) += -lrte_hash _LDLIBS-$(CONFIG_RTE_LIBRTE_MEMBER) += -lrte_member _LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST) += -lrte_vhost