From patchwork Thu Jun 7 09:26:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 40708 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A12FD1B010; Thu, 7 Jun 2018 11:26:33 +0200 (CEST) Received: from mx1.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by dpdk.org (Postfix) with ESMTP id 9DEBC7ED8 for ; Thu, 7 Jun 2018 11:26:32 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id BC01479D36; Thu, 7 Jun 2018 09:26:31 +0000 (UTC) Received: from localhost.localdomain (ovpn-112-46.ams2.redhat.com [10.36.112.46]) by smtp.corp.redhat.com (Postfix) with ESMTP id D0DB920357CA; Thu, 7 Jun 2018 09:26:30 +0000 (UTC) From: Maxime Coquelin To: zhihong.wang@intel.com, tiwei.bie@intel.com, dev@dpdk.org Cc: Maxime Coquelin Date: Thu, 7 Jun 2018 11:26:11 +0200 Message-Id: <20180607092616.27720-1-maxime.coquelin@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Thu, 07 Jun 2018 09:26:31 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Thu, 07 Jun 2018 09:26:31 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'maxime.coquelin@redhat.com' RCPT:'' Subject: [dpdk-dev] [PATCH v3 0/5] net/virtio: Tx path selection and offload improvements X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Since v2, the series introduce a new devarg to disable the Tx simple path default. Indeed, as reported by Tiwei, simple Tx path does not comply with the Virtio spec, and so should not be selected automatically as it might break with some Virtio device implementations. In the default setup, it has an impact on performance when mergeable buffers are disabled, as the standard path will now be used. This series initially addressed a problem seen when running benchmarks, where we see a big difference in Tx performance depending on whether Rx mergeable is enabled or not. With patch 2, Tx simple path is selected even when Rx-mrg is negotiated. Digging a bit further, I found that Tx simple path could be selected even when offload features had been negotiated. With patch 3, guest does not try to negotiate Tx offload features that haven't been selected by the application. It means that without restarting the guest, we can now switch from Tx simple path when application do no request offloads, to Tx standard path when the application request offloads. Note that to do so, one must stop the port first [0]. Another advantage of doing this than using Tx simple path when guest application does not use offload features, is that on host side, the virtio net header parsing is skipped in dequeue function. Patch 4 fixes an issue with Rx offload, as simple path was be used if application requested TCP LRO only. Finally, patch 5 aims at improving the offload feature checks in the standard paths. Below are benchmarks results when offloads are enabled at device level and simple Tx is disabled (default). Rx-mrg=off benchmarks: +------------+-------+-------------+-------------+----------+ | Run | PVP | Guest->Host | Host->Guest | Loopback | +------------+-------+-------------+-------------+----------+ | v18.05 | 14.47 | 17.02 | 17.57 | 13.15 | | + series | 11.73 | 13.90 | 17.50 | 13.19 | +------------+-------+-------------+-------------+----------+ Rx-mrg=on benchmarks: +------------+-------+-------------+-------------+----------+ | Run | PVP | Guest->Host | Host->Guest | Loopback | +------------+-------+-------------+-------------+----------+ | v18.05 | 9.53 | 13.80 | 16.70 | 13.11 | | + series | 9.58 | 13.93 | 16.70 | 13.11 | +------------+-------+-------------+-------------+----------+ Below are benchmarks results when offloads are enabled at device level and simple Tx is unlocked (with passing 'simple_tx_support=1' as Virtio device devarg). Rx-mrg=off benchmarks: +------------+-------+-------------+-------------+----------+ | Run | PVP | Guest->Host | Host->Guest | Loopback | +------------+-------+-------------+-------------+----------+ | v18.05 | 14.47 | 17.02 | 17.57 | 13.15 | | + series | 14.88 | 19.64 | 17.50 | 13.14 | +------------+-------+-------------+-------------+----------+ Rx-mrg=on benchmarks: +------------+-------+-------------+-------------+----------+ | Run | PVP | Guest->Host | Host->Guest | Loopback | +------------+-------+-------------+-------------+----------+ | v18.05 | 9.53 | 13.80 | 16.70 | 13.11 | | + series | 12.62 | 19.69 | 16.70 | 13.11 | +------------+-------+-------------+-------------+----------+ [0]: testpmd> port start 0 EAL: Error disabling MSI-X interrupts for fd 17 set_rxtx_funcs(): virtio: using mergeable buffer Rx path on port 0 set_rxtx_funcs(): virtio: using simple Tx path on port 0 Port 0: 52:54:00:58:D7:01 Checking link statuses... Done testpmd> show port 0 tx_offload capabilities Tx Offloading Capabilities of port 0 : Per Queue : Per Port : VLAN_INSERT UDP_CKSUM TCP_CKSUM TCP_TSO MULTI_SEGS testpmd> show port 0 tx_offload configuration Tx Offloading Configuration of port 0 : Port : Queue[ 0] : testpmd> port stop 0 Stopping ports... Checking link statuses... Done testpmd> csum set tcp hw 0 Parse tunnel is off IP checksum offload is sw UDP checksum offload is sw TCP checksum offload is hw SCTP checksum offload is sw Outer-Ip checksum offload is sw testpmd> port start 0 Configuring Port 0 (socket 0) EAL: Error disabling MSI-X interrupts for fd 17 set_rxtx_funcs(): virtio: using mergeable buffer Rx path on port 0 set_rxtx_funcs(): virtio: using standard Tx path on port 0 Port 0: 52:54:00:58:D7:01 Checking link statuses... Done testpmd> show port 0 tx_offload configuration Tx Offloading Configuration of port 0 : Port : TCP_CKSUM Queue[ 0] : Changes in v3: ============== - Fix devargs parsing not to break if other args are passed (Tiwei) - Various comestic fixes (Tiwei) Changes in v2: ============== - Introduce a devarg to disable simple Tx path by default (Tiwei) - Use standard PATH if VLAN strip or insert offloads are requested (Tiwei) - Use boolean instead of int/uint8_t for has_tx/rx_offload (Tiwei) Maxime Coquelin (5): net/virtio: forbid simple Tx path by default net/virtio: use simple path for Tx even if Rx mergeable net/vhost: improve Tx path selection net/virtio: don't use simple Rx if TCP LRO or VLAN strip requested net/virtio: improve offload check performance doc/guides/nics/virtio.rst | 9 +++ drivers/net/virtio/virtio_ethdev.c | 121 ++++++++++++++++++++++++++++++++++--- drivers/net/virtio/virtio_ethdev.h | 3 - drivers/net/virtio/virtio_pci.h | 4 ++ drivers/net/virtio/virtio_rxtx.c | 30 ++------- 5 files changed, 132 insertions(+), 35 deletions(-)