From patchwork Thu May 25 16:25:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127495 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E845842B9D; Thu, 25 May 2023 18:28:35 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A25D942F94; Thu, 25 May 2023 18:27:33 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id C6E8A40E6E for ; Thu, 25 May 2023 18:27:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685032051; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YwlzYHwt7IM+hV5pg2MDq7egkCHBIQihzJGivnXO/ZE=; b=Lp16kaHApyiFwygqLfUnPwVUhrv2BNq+DvsSNaLoP22V/3+fUzrW6zUHDxcFYMJ48Scy9G NbB0voIOjqcod6/WDuen89L+0zNsQBQl5/yMk6wryluDGu2iIrr4i6THhLUoAJzXIp4quY bLwQwQmohfLMmUr/w9MFmYgA5jkncXk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-10-1vcObvENMD2kBhpEjzRFZg-1; Thu, 25 May 2023 12:27:28 -0400 X-MC-Unique: 1vcObvENMD2kBhpEjzRFZg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F122C185A792; Thu, 25 May 2023 16:27:27 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.23]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6CEF8140E95D; Thu, 25 May 2023 16:27:25 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v3 26/28] vhost: add VDUSE device startup Date: Thu, 25 May 2023 18:25:49 +0200 Message-Id: <20230525162551.70359-27-maxime.coquelin@redhat.com> In-Reply-To: <20230525162551.70359-1-maxime.coquelin@redhat.com> References: <20230525162551.70359-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds the device and its virtqueues initialization once the Virtio driver has set the DRIVER_OK in the Virtio status register. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vduse.c | 126 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 126 insertions(+) diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c index 110654ec68..a10dc24d38 100644 --- a/lib/vhost/vduse.c +++ b/lib/vhost/vduse.c @@ -141,6 +141,128 @@ static struct vhost_backend_ops vduse_backend_ops = { .inject_irq = vduse_inject_irq, }; +static void +vduse_vring_setup(struct virtio_net *dev, unsigned int index) +{ + struct vhost_virtqueue *vq = dev->virtqueue[index]; + struct vhost_vring_addr *ra = &vq->ring_addrs; + struct vduse_vq_info vq_info; + struct vduse_vq_eventfd vq_efd; + int ret; + + vq_info.index = index; + ret = ioctl(dev->vduse_dev_fd, VDUSE_VQ_GET_INFO, &vq_info); + if (ret) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to get VQ %u info: %s\n", + index, strerror(errno)); + return; + } + + VHOST_LOG_CONFIG(dev->ifname, INFO, "VQ %u info:\n", index); + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tnum: %u\n", vq_info.num); + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tdesc_addr: %llx\n", vq_info.desc_addr); + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tdriver_addr: %llx\n", vq_info.driver_addr); + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tdevice_addr: %llx\n", vq_info.device_addr); + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tavail_idx: %u\n", vq_info.split.avail_index); + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tready: %u\n", vq_info.ready); + + vq->last_avail_idx = vq_info.split.avail_index; + vq->size = vq_info.num; + vq->ready = true; + vq->enabled = vq_info.ready; + ra->desc_user_addr = vq_info.desc_addr; + ra->avail_user_addr = vq_info.driver_addr; + ra->used_user_addr = vq_info.device_addr; + + vq->kickfd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); + if (vq->kickfd < 0) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to init kickfd for VQ %u: %s\n", + index, strerror(errno)); + vq->kickfd = VIRTIO_INVALID_EVENTFD; + return; + } + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tkick fd: %d\n", vq->kickfd); + + vq->shadow_used_split = rte_malloc_socket(NULL, + vq->size * sizeof(struct vring_used_elem), + RTE_CACHE_LINE_SIZE, 0); + vq->batch_copy_elems = rte_malloc_socket(NULL, + vq->size * sizeof(struct batch_copy_elem), + RTE_CACHE_LINE_SIZE, 0); + + vhost_user_iotlb_rd_lock(vq); + if (vring_translate(dev, vq)) + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to translate vring %d addresses\n", + index); + + if (vhost_enable_guest_notification(dev, vq, 0)) + VHOST_LOG_CONFIG(dev->ifname, ERR, + "Failed to disable guest notifications on vring %d\n", + index); + vhost_user_iotlb_rd_unlock(vq); + + vq_efd.index = index; + vq_efd.fd = vq->kickfd; + + ret = ioctl(dev->vduse_dev_fd, VDUSE_VQ_SETUP_KICKFD, &vq_efd); + if (ret) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to setup kickfd for VQ %u: %s\n", + index, strerror(errno)); + close(vq->kickfd); + vq->kickfd = VIRTIO_UNINITIALIZED_EVENTFD; + return; + } +} + +static void +vduse_device_start(struct virtio_net *dev) +{ + unsigned int i, ret; + + VHOST_LOG_CONFIG(dev->ifname, INFO, "Starting device...\n"); + + dev->notify_ops = vhost_driver_callback_get(dev->ifname); + if (!dev->notify_ops) { + VHOST_LOG_CONFIG(dev->ifname, ERR, + "Failed to get callback ops for driver\n"); + return; + } + + ret = ioctl(dev->vduse_dev_fd, VDUSE_DEV_GET_FEATURES, &dev->features); + if (ret) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to get features: %s\n", + strerror(errno)); + return; + } + + VHOST_LOG_CONFIG(dev->ifname, INFO, "Negotiated Virtio features: 0x%" PRIx64 "\n", + dev->features); + + if (dev->features & + ((1ULL << VIRTIO_NET_F_MRG_RXBUF) | + (1ULL << VIRTIO_F_VERSION_1) | + (1ULL << VIRTIO_F_RING_PACKED))) { + dev->vhost_hlen = sizeof(struct virtio_net_hdr_mrg_rxbuf); + } else { + dev->vhost_hlen = sizeof(struct virtio_net_hdr); + } + + for (i = 0; i < dev->nr_vring; i++) + vduse_vring_setup(dev, i); + + dev->flags |= VIRTIO_DEV_READY; + + if (dev->notify_ops->new_device(dev->vid) == 0) + dev->flags |= VIRTIO_DEV_RUNNING; + + for (i = 0; i < dev->nr_vring; i++) { + struct vhost_virtqueue *vq = dev->virtqueue[i]; + + if (dev->notify_ops->vring_state_changed) + dev->notify_ops->vring_state_changed(dev->vid, i, vq->enabled); + } +} + static void vduse_events_handler(int fd, void *arg, int *remove __rte_unused) { @@ -177,6 +299,10 @@ vduse_events_handler(int fd, void *arg, int *remove __rte_unused) VHOST_LOG_CONFIG(dev->ifname, INFO, "\tnew status: 0x%08x\n", req.s.status); dev->status = req.s.status; + + if (dev->status & VIRTIO_DEVICE_STATUS_DRIVER_OK) + vduse_device_start(dev); + resp.result = VDUSE_REQ_RESULT_OK; break; case VDUSE_UPDATE_IOTLB: