Cover Detail
Show a cover letter.
GET /api/covers/127469/?format=api
{ "id": 127469, "url": "http://patchwork.dpdk.org/api/covers/127469/?format=api", "web_url": "http://patchwork.dpdk.org/project/dpdk/cover/20230525162551.70359-1-maxime.coquelin@redhat.com/", "project": { "id": 1, "url": "http://patchwork.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<20230525162551.70359-1-maxime.coquelin@redhat.com>", "list_archive_url": "https://inbox.dpdk.org/dev/20230525162551.70359-1-maxime.coquelin@redhat.com", "date": "2023-05-25T16:25:23", "name": "[v3,00/28] Add VDUSE support to Vhost library", "submitter": { "id": 512, "url": "http://patchwork.dpdk.org/api/people/512/?format=api", "name": "Maxime Coquelin", "email": "maxime.coquelin@redhat.com" }, "mbox": "http://patchwork.dpdk.org/project/dpdk/cover/20230525162551.70359-1-maxime.coquelin@redhat.com/mbox/", "series": [ { "id": 28187, "url": "http://patchwork.dpdk.org/api/series/28187/?format=api", "web_url": "http://patchwork.dpdk.org/project/dpdk/list/?series=28187", "date": "2023-05-25T16:25:23", "name": "Add VDUSE support to Vhost library", "version": 3, "mbox": "http://patchwork.dpdk.org/series/28187/mbox/" } ], "comments": "http://patchwork.dpdk.org/api/covers/127469/comments/", "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from mails.dpdk.org (mails.dpdk.org [217.70.189.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id 6ED6642B9D;\n\tThu, 25 May 2023 18:26:01 +0200 (CEST)", "from mails.dpdk.org (localhost [127.0.0.1])\n\tby mails.dpdk.org (Postfix) with ESMTP id 5601C40DF8;\n\tThu, 25 May 2023 18:26:01 +0200 (CEST)", "from us-smtp-delivery-124.mimecast.com\n (us-smtp-delivery-124.mimecast.com [170.10.129.124])\n by mails.dpdk.org (Postfix) with ESMTP id 881C140DDB\n for <dev@dpdk.org>; Thu, 25 May 2023 18:25:59 +0200 (CEST)", "from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com\n [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS\n (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id\n us-mta-59-LKg5iX9hNe2ri9iOZeX3lw-1; Thu, 25 May 2023 12:25:55 -0400", "from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com\n [10.11.54.7])\n (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n (No client certificate requested)\n by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4B40C1C05136;\n Thu, 25 May 2023 16:25:55 +0000 (UTC)", "from max-t490s.redhat.com (unknown [10.39.208.23])\n by smtp.corp.redhat.com (Postfix) with ESMTP id B7E31140E95D;\n Thu, 25 May 2023 16:25:52 +0000 (UTC)" ], "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n s=mimecast20190719; t=1685031959;\n h=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n to:to:cc:cc:mime-version:mime-version:content-type:content-type:\n content-transfer-encoding:content-transfer-encoding;\n bh=kIcglijttQxA0GBQN+FIGslAGKkZmrjOnzzJPsNfNgk=;\n b=irTn61liiHfoyXw3v0EFgSA/Jfyh9YX+vZybBB8XYAYv1e8qs+6xrIc1ZCkIvkNKrWBINp\n a+f75/4enKYSUviAcqyTfgsnn5NEthYog632powfyAGUJkbqcoNXvupTpa2rmRNQGql47k\n lwK86Arnb1HwxQdzVoRm1kaD+dDjbGk=", "X-MC-Unique": "LKg5iX9hNe2ri9iOZeX3lw-1", "From": "Maxime Coquelin <maxime.coquelin@redhat.com>", "To": "dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com,\n mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com,\n cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com,\n eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com", "Cc": "Maxime Coquelin <maxime.coquelin@redhat.com>", "Subject": "[PATCH v3 00/28] Add VDUSE support to Vhost library", "Date": "Thu, 25 May 2023 18:25:23 +0200", "Message-Id": "<20230525162551.70359-1-maxime.coquelin@redhat.com>", "MIME-Version": "1.0", "X-Scanned-By": "MIMEDefang 3.1 on 10.11.54.7", "X-Mimecast-Spam-Score": "0", "X-Mimecast-Originator": "redhat.com", "Content-Transfer-Encoding": "8bit", "Content-Type": "text/plain; charset=\"US-ASCII\"; x-default=true", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org" }, "content": "Note: v2 is identical to v3, it is just a resend because\nof an issue when posting v2 breaking the series in patchwork.\n\nThis series introduces a new type of backend, VDUSE,\nto the Vhost library.\n\nVDUSE stands for vDPA device in Userspace, it enables\nimplementing a Virtio device in userspace and have it\nattached to the Kernel vDPA bus.\n\nOnce attached to the vDPA bus, it can be used either by\nKernel Virtio drivers, like virtio-net in our case, via\nthe virtio-vdpa driver. Doing that, the device is visible\nto the Kernel networking stack and is exposed to userspace\nas a regular netdev.\n\nIt can also be exposed to userspace thanks to the\nvhost-vdpa driver, via a vhost-vdpa chardev that can be\npassed to QEMU or Virtio-user PMD.\n\nWhile VDUSE support is already available in upstream\nKernel, a couple of patches are required to support\nnetwork device type:\n\nhttps://gitlab.com/mcoquelin/linux/-/tree/vduse_networking_rfc\n\nIn order to attach the created VDUSE device to the vDPA\nbus, a recent iproute2 version containing the vdpa tool is\nrequired.\n\nBenchmark results:\n==================\n\nOn this v2, PVP reference benchmark has been run & compared with\nVhost-user.\n\nWhen doing macswap forwarding in the worload, no difference is seen.\nWhen doing io forwarding in the workload, we see 4% performance\ndegradation with VDUSE, comapred to Vhost-user/Virtio-user. It is\nexplained by the use of the IOTLB layer in the Vhost-library when using\nVDUSE, whereas Vhost-user/Virtio-user does not make use of it.\n\nUsage:\n======\n\n1. Probe required Kernel modules\n# modprobe vdpa\n# modprobe vduse\n# modprobe virtio-vdpa\n\n2. Build (require vduse kernel headers to be available)\n# meson build\n# ninja -C build\n\n3. Create a VDUSE device (vduse0) using Vhost PMD with\ntestpmd (with 4 queue pairs in this example)\n# ./build/app/dpdk-testpmd --no-pci --vdev=net_vhost0,iface=/dev/vduse/vduse0,queues=4 --log-level=*:9 -- -i --txq=4 --rxq=4\n \n4. Attach the VDUSE device to the vDPA bus\n# vdpa dev add name vduse0 mgmtdev vduse\n=> The virtio-net netdev shows up (eth0 here)\n# ip l show eth0\n21: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000\n link/ether c2:73:ea:a7:68:6d brd ff:ff:ff:ff:ff:ff\n\n5. Start/stop traffic in testpmd\ntestpmd> start\ntestpmd> show port stats 0\n ######################## NIC statistics for port 0 ########################\n RX-packets: 11 RX-missed: 0 RX-bytes: 1482\n RX-errors: 0\n RX-nombuf: 0\n TX-packets: 1 TX-errors: 0 TX-bytes: 62\n\n Throughput (since last show)\n Rx-pps: 0 Rx-bps: 0\n Tx-pps: 0 Tx-bps: 0\n ############################################################################\ntestpmd> stop\n\n6. Detach the VDUSE device from the vDPA bus\n# vdpa dev del vduse0\n\n7. Quit testpmd\ntestpmd> quit\n\nKnown issues & remaining work:\n==============================\n- Fix issue in FD manager (still polling while FD has been removed)\n- Add Netlink support in Vhost library\n- Support device reconnection\n -> a temporary patch to support reconnection via a tmpfs file is available,\n upstream solution would be in-kernel and is being developed.\n -> https://gitlab.com/mcoquelin/dpdk-next-virtio/-/commit/5ad06ce14159a9ce36ee168dd13ef389cec91137\n- Support packed ring\n- Provide more performance benchmark results\n\nChanges in v2/v3:\n=================\n- Fixed mem_set_dump() parameter (patch 4)\n- Fixed accidental comment change (patch 7, Chenbo)\n- Change from __builtin_ctz to __builtin_ctzll (patch 9, Chenbo)\n- move change from patch 12 to 13 (Chenbo)\n- Enable locks annotation for control queue (Patch 17)\n- Send control queue notification when used descriptors enqueued (Patch 17)\n- Lock control queue IOTLB lock (Patch 17)\n- Fix error path in virtio_net_ctrl_pop() (Patch 17, Chenbo)\n- Set VDUSE dev FD as NONBLOCK (Patch 18)\n- Enable more Virtio features (Patch 18)\n- Remove calls to pthread_setcancelstate() (Patch 22)\n- Add calls to fdset_pipe_notify() when adding and deleting FDs from a set (Patch 22)\n- Use RTE_DIM() to get requests string array size (Patch 22)\n- Set reply result for IOTLB update message (Patch 25, Chenbo)\n- Fix queues enablement with multiqueue (Patch 26)\n- Move kickfd creation for better logging (Patch 26)\n- Improve logging (Patch 26)\n- Uninstall cvq kickfd in case of handler installation failure (Patch 27)\n- Enable CVQ notifications once handler is installed (Patch 27)\n- Don't advertise multiqueue and control queue if app only request single queue pair (Patch 27)\n- Add release notes\n\nMaxime Coquelin (28):\n vhost: fix missing guest notif stat increment\n vhost: fix invalid call FD handling\n vhost: fix IOTLB entries overlap check with previous entry\n vhost: add helper of IOTLB entries coredump\n vhost: add helper for IOTLB entries shared page check\n vhost: don't dump unneeded pages with IOTLB\n vhost: change to single IOTLB cache per device\n vhost: add offset field to IOTLB entries\n vhost: add page size info to IOTLB entry\n vhost: retry translating IOVA after IOTLB miss\n vhost: introduce backend ops\n vhost: add IOTLB cache entry removal callback\n vhost: add helper for IOTLB misses\n vhost: add helper for interrupt injection\n vhost: add API to set max queue pairs\n net/vhost: use API to set max queue pairs\n vhost: add control virtqueue support\n vhost: add VDUSE device creation and destruction\n vhost: add VDUSE callback for IOTLB miss\n vhost: add VDUSE callback for IOTLB entry removal\n vhost: add VDUSE callback for IRQ injection\n vhost: add VDUSE events handler\n vhost: add support for virtqueue state get event\n vhost: add support for VDUSE status set event\n vhost: add support for VDUSE IOTLB update event\n vhost: add VDUSE device startup\n vhost: add multiqueue support to VDUSE\n vhost: add VDUSE device stop\n\n doc/guides/prog_guide/vhost_lib.rst | 4 +\n doc/guides/rel_notes/release_23_07.rst | 11 +\n drivers/net/vhost/rte_eth_vhost.c | 3 +\n lib/vhost/iotlb.c | 333 +++++++------\n lib/vhost/iotlb.h | 45 +-\n lib/vhost/meson.build | 5 +\n lib/vhost/rte_vhost.h | 17 +\n lib/vhost/socket.c | 72 ++-\n lib/vhost/vduse.c | 646 +++++++++++++++++++++++++\n lib/vhost/vduse.h | 33 ++\n lib/vhost/version.map | 3 +\n lib/vhost/vhost.c | 51 +-\n lib/vhost/vhost.h | 90 ++--\n lib/vhost/vhost_user.c | 51 +-\n lib/vhost/vhost_user.h | 2 +-\n lib/vhost/virtio_net_ctrl.c | 286 +++++++++++\n lib/vhost/virtio_net_ctrl.h | 10 +\n 17 files changed, 1424 insertions(+), 238 deletions(-)\n create mode 100644 lib/vhost/vduse.c\n create mode 100644 lib/vhost/vduse.h\n create mode 100644 lib/vhost/virtio_net_ctrl.c\n create mode 100644 lib/vhost/virtio_net_ctrl.h" }