From patchwork Tue Jun 6 08:18:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128191 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2A77B42C3E; Tue, 6 Jun 2023 10:21:49 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5892D42C76; Tue, 6 Jun 2023 10:20:18 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id C9D2B41141 for ; Tue, 6 Jun 2023 10:20:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039616; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IIyMCcgg6MKZkCLy0ze0X6ty1Yk1sayPpDrb/zVwqQw=; b=G+Z7bqKhUrtMyh8UQ9pT4I+S+A/q1WG+37iZFEPNELkmEil2n60m0+uohw8IB3kWOy322+ VJ20U5IapecLKESRtC2OHGxepJqvg6ZrygWbRHgbF9OmEWKsw5MHRXAjKdBrvrn/SiqkHe Q1tXGsLNwqVcnflJw1Eu8gYrEWPPFR4= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-470-ynkhvemNNpu3CZeDebpe0w-1; Tue, 06 Jun 2023 04:20:13 -0400 X-MC-Unique: ynkhvemNNpu3CZeDebpe0w-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 95A42811E86; Tue, 6 Jun 2023 08:20:12 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1A21840CF8F6; Tue, 6 Jun 2023 08:20:09 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 26/26] vhost: add VDUSE device stop Date: Tue, 6 Jun 2023 10:18:52 +0200 Message-Id: <20230606081852.71003-27-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds VDUSE device stop and cleanup of its virtqueues. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- doc/guides/rel_notes/release_23_07.rst | 7 +++ lib/vhost/vduse.c | 72 +++++++++++++++++++++++--- 2 files changed, 71 insertions(+), 8 deletions(-) diff --git a/doc/guides/rel_notes/release_23_07.rst b/doc/guides/rel_notes/release_23_07.rst index 7034fb664c..0f44c859da 100644 --- a/doc/guides/rel_notes/release_23_07.rst +++ b/doc/guides/rel_notes/release_23_07.rst @@ -67,6 +67,13 @@ New Features Introduced ``rte_vhost_driver_set_max_queue_num()`` to be able to limit the maximum number of supported queue pairs, required for VDUSE support. +* **Added VDUSE support into Vhost library.** + + VDUSE aims at implementing vDPA devices in userspace. It can be used as an + alternative to Vhost-user when using Vhost-vDPA, but also enable providing a + virtio-net netdev to the host when using Virtio-vDPA driver. A limitation in + this release is the lack of reconnection support. + Removed Items ------------- diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c index d3759077ff..a509daf80c 100644 --- a/lib/vhost/vduse.c +++ b/lib/vhost/vduse.c @@ -252,6 +252,44 @@ vduse_vring_setup(struct virtio_net *dev, unsigned int index) } } +static void +vduse_vring_cleanup(struct virtio_net *dev, unsigned int index) +{ + struct vhost_virtqueue *vq = dev->virtqueue[index]; + struct vduse_vq_eventfd vq_efd; + int ret; + + if (vq == dev->cvq && vq->kickfd >= 0) { + fdset_del(&vduse.fdset, vq->kickfd); + fdset_pipe_notify(&vduse.fdset); + } + + vq_efd.index = index; + vq_efd.fd = VDUSE_EVENTFD_DEASSIGN; + + ret = ioctl(dev->vduse_dev_fd, VDUSE_VQ_SETUP_KICKFD, &vq_efd); + if (ret) + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to cleanup kickfd for VQ %u: %s\n", + index, strerror(errno)); + + close(vq->kickfd); + vq->kickfd = VIRTIO_UNINITIALIZED_EVENTFD; + + vring_invalidate(dev, vq); + + rte_free(vq->batch_copy_elems); + vq->batch_copy_elems = NULL; + + rte_free(vq->shadow_used_split); + vq->shadow_used_split = NULL; + + vq->enabled = false; + vq->ready = false; + vq->size = 0; + vq->last_used_idx = 0; + vq->last_avail_idx = 0; +} + static void vduse_device_start(struct virtio_net *dev) { @@ -304,6 +342,23 @@ vduse_device_start(struct virtio_net *dev) } } +static void +vduse_device_stop(struct virtio_net *dev) +{ + unsigned int i; + + VHOST_LOG_CONFIG(dev->ifname, INFO, "Stopping device...\n"); + + vhost_destroy_device_notify(dev); + + dev->flags &= ~VIRTIO_DEV_READY; + + for (i = 0; i < dev->nr_vring; i++) + vduse_vring_cleanup(dev, i); + + vhost_user_iotlb_flush_all(dev); +} + static void vduse_events_handler(int fd, void *arg, int *remove __rte_unused) { @@ -311,6 +366,7 @@ vduse_events_handler(int fd, void *arg, int *remove __rte_unused) struct vduse_dev_request req; struct vduse_dev_response resp; struct vhost_virtqueue *vq; + uint8_t old_status = dev->status; int ret; memset(&resp, 0, sizeof(resp)); @@ -339,6 +395,7 @@ vduse_events_handler(int fd, void *arg, int *remove __rte_unused) case VDUSE_SET_STATUS: VHOST_LOG_CONFIG(dev->ifname, INFO, "\tnew status: 0x%08x\n", req.s.status); + old_status = dev->status; dev->status = req.s.status; resp.result = VDUSE_REQ_RESULT_OK; break; @@ -363,8 +420,12 @@ vduse_events_handler(int fd, void *arg, int *remove __rte_unused) return; } - if (dev->status & VIRTIO_DEVICE_STATUS_DRIVER_OK) - vduse_device_start(dev); + if ((old_status ^ dev->status) & VIRTIO_DEVICE_STATUS_DRIVER_OK) { + if (dev->status & VIRTIO_DEVICE_STATUS_DRIVER_OK) + vduse_device_start(dev); + else + vduse_device_stop(dev); + } VHOST_LOG_CONFIG(dev->ifname, INFO, "Request %s (%u) handled successfully\n", vduse_req_id_to_str(req.type), req.type); @@ -560,12 +621,7 @@ vduse_device_destroy(const char *path) if (vid == RTE_MAX_VHOST_DEVICE) return -1; - if (dev->cvq && dev->cvq->kickfd >= 0) { - fdset_del(&vduse.fdset, dev->cvq->kickfd); - fdset_pipe_notify(&vduse.fdset); - close(dev->cvq->kickfd); - dev->cvq->kickfd = VIRTIO_UNINITIALIZED_EVENTFD; - } + vduse_device_stop(dev); fdset_del(&vduse.fdset, dev->vduse_dev_fd); fdset_pipe_notify(&vduse.fdset);