From patchwork Tue Jun 6 08:18:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 128181 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4CFF542C3E; Tue, 6 Jun 2023 10:20:49 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2A26942D3C; Tue, 6 Jun 2023 10:19:49 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 48C3442D56 for ; Tue, 6 Jun 2023 10:19:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686039585; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Srz5SyFp3lpfq/AC319jSE9Sw5G0wmsLmdf65yy9f0g=; b=CJKmgQSkqGD3EFOGPOL8ph0rM2e+Mf+oZse7RNjkLM68q51MpYV/kBnOa2NC9jgpW9BczS 9r4X0ZCWHMI9Hchyvhqf/dqiy5RzQyycKT5HN7VP3OG6z6Fs1kzYA9v/rbYR2xvRM0vkZ+ SQv9+Z+z7yFlz6NL32bzPB5FFZPcXtg= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-98-9ndNdiRLMcii-QtNqsUPTA-1; Tue, 06 Jun 2023 04:19:44 -0400 X-MC-Unique: 9ndNdiRLMcii-QtNqsUPTA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 40D33185A78B; Tue, 6 Jun 2023 08:19:44 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id BA18940CF8F7; Tue, 6 Jun 2023 08:19:41 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v5 16/26] vhost: add VDUSE device creation and destruction Date: Tue, 6 Jun 2023 10:18:42 +0200 Message-Id: <20230606081852.71003-17-maxime.coquelin@redhat.com> In-Reply-To: <20230606081852.71003-1-maxime.coquelin@redhat.com> References: <20230606081852.71003-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds initial support for VDUSE, which includes the device creation and destruction. It does not include the virtqueues configuration, so this is not functionnal at this point. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/meson.build | 4 + lib/vhost/socket.c | 34 ++++--- lib/vhost/vduse.c | 201 ++++++++++++++++++++++++++++++++++++++++++ lib/vhost/vduse.h | 33 +++++++ lib/vhost/vhost.h | 2 + 5 files changed, 262 insertions(+), 12 deletions(-) create mode 100644 lib/vhost/vduse.c create mode 100644 lib/vhost/vduse.h diff --git a/lib/vhost/meson.build b/lib/vhost/meson.build index d211a0bd37..13e0382c92 100644 --- a/lib/vhost/meson.build +++ b/lib/vhost/meson.build @@ -29,6 +29,10 @@ sources = files( 'virtio_net.c', 'virtio_net_ctrl.c', ) +if cc.has_header('linux/vduse.h') + sources += files('vduse.c') + cflags += '-DVHOST_HAS_VDUSE' +endif headers = files( 'rte_vdpa.h', 'rte_vhost.h', diff --git a/lib/vhost/socket.c b/lib/vhost/socket.c index 29f7a8cece..19a7469e45 100644 --- a/lib/vhost/socket.c +++ b/lib/vhost/socket.c @@ -19,6 +19,7 @@ #include #include "fd_man.h" +#include "vduse.h" #include "vhost.h" #include "vhost_user.h" @@ -36,6 +37,7 @@ struct vhost_user_socket { int socket_fd; struct sockaddr_un un; bool is_server; + bool is_vduse; bool reconnect; bool iommu_support; bool use_builtin_virtio_net; @@ -991,18 +993,21 @@ rte_vhost_driver_register(const char *path, uint64_t flags) #endif } - if ((flags & RTE_VHOST_USER_CLIENT) != 0) { - vsocket->reconnect = !(flags & RTE_VHOST_USER_NO_RECONNECT); - if (vsocket->reconnect && reconn_tid == 0) { - if (vhost_user_reconnect_init() != 0) - goto out_mutex; - } + if (!strncmp("/dev/vduse/", path, strlen("/dev/vduse/"))) { + vsocket->is_vduse = true; } else { - vsocket->is_server = true; - } - ret = create_unix_socket(vsocket); - if (ret < 0) { - goto out_mutex; + if ((flags & RTE_VHOST_USER_CLIENT) != 0) { + vsocket->reconnect = !(flags & RTE_VHOST_USER_NO_RECONNECT); + if (vsocket->reconnect && reconn_tid == 0) { + if (vhost_user_reconnect_init() != 0) + goto out_mutex; + } + } else { + vsocket->is_server = true; + } + ret = create_unix_socket(vsocket); + if (ret < 0) + goto out_mutex; } vhost_user.vsockets[vhost_user.vsocket_cnt++] = vsocket; @@ -1067,7 +1072,9 @@ rte_vhost_driver_unregister(const char *path) if (strcmp(vsocket->path, path)) continue; - if (vsocket->is_server) { + if (vsocket->is_vduse) { + vduse_device_destroy(path); + } else if (vsocket->is_server) { /* * If r/wcb is executing, release vhost_user's * mutex lock, and try again since the r/wcb @@ -1218,6 +1225,9 @@ rte_vhost_driver_start(const char *path) if (!vsocket) return -1; + if (vsocket->is_vduse) + return vduse_device_create(path); + if (fdset_tid == 0) { /** * create a pipe which will be waited by poll and notified to diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c new file mode 100644 index 0000000000..d67818bfb5 --- /dev/null +++ b/lib/vhost/vduse.c @@ -0,0 +1,201 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Red Hat, Inc. + */ + +#include +#include +#include +#include + + +#include +#include + +#include +#include + +#include + +#include "vduse.h" +#include "vhost.h" + +#define VHOST_VDUSE_API_VERSION 0 +#define VDUSE_CTRL_PATH "/dev/vduse/control" + +#define VDUSE_NET_SUPPORTED_FEATURES ((1ULL << VIRTIO_NET_F_MRG_RXBUF) | \ + (1ULL << VIRTIO_F_ANY_LAYOUT) | \ + (1ULL << VIRTIO_F_VERSION_1) | \ + (1ULL << VIRTIO_NET_F_GSO) | \ + (1ULL << VIRTIO_NET_F_HOST_TSO4) | \ + (1ULL << VIRTIO_NET_F_HOST_TSO6) | \ + (1ULL << VIRTIO_NET_F_HOST_UFO) | \ + (1ULL << VIRTIO_NET_F_HOST_ECN) | \ + (1ULL << VIRTIO_NET_F_CSUM) | \ + (1ULL << VIRTIO_NET_F_GUEST_CSUM) | \ + (1ULL << VIRTIO_NET_F_GUEST_TSO4) | \ + (1ULL << VIRTIO_NET_F_GUEST_TSO6) | \ + (1ULL << VIRTIO_NET_F_GUEST_UFO) | \ + (1ULL << VIRTIO_NET_F_GUEST_ECN) | \ + (1ULL << VIRTIO_RING_F_INDIRECT_DESC) | \ + (1ULL << VIRTIO_F_IN_ORDER) | \ + (1ULL << VIRTIO_F_IOMMU_PLATFORM)) + +static struct vhost_backend_ops vduse_backend_ops = { +}; + +int +vduse_device_create(const char *path) +{ + int control_fd, dev_fd, vid, ret; + uint32_t i; + struct virtio_net *dev; + uint64_t ver = VHOST_VDUSE_API_VERSION; + struct vduse_dev_config *dev_config = NULL; + const char *name = path + strlen("/dev/vduse/"); + + control_fd = open(VDUSE_CTRL_PATH, O_RDWR); + if (control_fd < 0) { + VHOST_LOG_CONFIG(name, ERR, "Failed to open %s: %s\n", + VDUSE_CTRL_PATH, strerror(errno)); + return -1; + } + + if (ioctl(control_fd, VDUSE_SET_API_VERSION, &ver)) { + VHOST_LOG_CONFIG(name, ERR, "Failed to set API version: %" PRIu64 ": %s\n", + ver, strerror(errno)); + ret = -1; + goto out_ctrl_close; + } + + dev_config = malloc(offsetof(struct vduse_dev_config, config)); + if (!dev_config) { + VHOST_LOG_CONFIG(name, ERR, "Failed to allocate VDUSE config\n"); + ret = -1; + goto out_ctrl_close; + } + + memset(dev_config, 0, sizeof(struct vduse_dev_config)); + + strncpy(dev_config->name, name, VDUSE_NAME_MAX - 1); + dev_config->device_id = VIRTIO_ID_NET; + dev_config->vendor_id = 0; + dev_config->features = VDUSE_NET_SUPPORTED_FEATURES; + dev_config->vq_num = 2; + dev_config->vq_align = sysconf(_SC_PAGE_SIZE); + dev_config->config_size = 0; + + ret = ioctl(control_fd, VDUSE_CREATE_DEV, dev_config); + if (ret < 0) { + VHOST_LOG_CONFIG(name, ERR, "Failed to create VDUSE device: %s\n", + strerror(errno)); + goto out_free; + } + + dev_fd = open(path, O_RDWR); + if (dev_fd < 0) { + VHOST_LOG_CONFIG(name, ERR, "Failed to open device %s: %s\n", + path, strerror(errno)); + ret = -1; + goto out_dev_close; + } + + ret = fcntl(dev_fd, F_SETFL, O_NONBLOCK); + if (ret < 0) { + VHOST_LOG_CONFIG(name, ERR, "Failed to set chardev as non-blocking: %s\n", + strerror(errno)); + goto out_dev_close; + } + + vid = vhost_new_device(&vduse_backend_ops); + if (vid < 0) { + VHOST_LOG_CONFIG(name, ERR, "Failed to create new Vhost device\n"); + ret = -1; + goto out_dev_close; + } + + dev = get_device(vid); + if (!dev) { + ret = -1; + goto out_dev_close; + } + + strncpy(dev->ifname, path, IF_NAME_SZ - 1); + dev->vduse_ctrl_fd = control_fd; + dev->vduse_dev_fd = dev_fd; + vhost_setup_virtio_net(dev->vid, true, true, true, true); + + for (i = 0; i < 2; i++) { + struct vduse_vq_config vq_cfg = { 0 }; + + ret = alloc_vring_queue(dev, i); + if (ret) { + VHOST_LOG_CONFIG(name, ERR, "Failed to alloc vring %d metadata\n", i); + goto out_dev_destroy; + } + + vq_cfg.index = i; + vq_cfg.max_size = 1024; + + ret = ioctl(dev->vduse_dev_fd, VDUSE_VQ_SETUP, &vq_cfg); + if (ret) { + VHOST_LOG_CONFIG(name, ERR, "Failed to set-up VQ %d\n", i); + goto out_dev_destroy; + } + } + + free(dev_config); + + return 0; + +out_dev_destroy: + vhost_destroy_device(vid); +out_dev_close: + if (dev_fd >= 0) + close(dev_fd); + ioctl(control_fd, VDUSE_DESTROY_DEV, name); +out_free: + free(dev_config); +out_ctrl_close: + close(control_fd); + + return ret; +} + +int +vduse_device_destroy(const char *path) +{ + const char *name = path + strlen("/dev/vduse/"); + struct virtio_net *dev; + int vid, ret; + + for (vid = 0; vid < RTE_MAX_VHOST_DEVICE; vid++) { + dev = vhost_devices[vid]; + + if (dev == NULL) + continue; + + if (!strcmp(path, dev->ifname)) + break; + } + + if (vid == RTE_MAX_VHOST_DEVICE) + return -1; + + if (dev->vduse_dev_fd >= 0) { + close(dev->vduse_dev_fd); + dev->vduse_dev_fd = -1; + } + + if (dev->vduse_ctrl_fd >= 0) { + ret = ioctl(dev->vduse_ctrl_fd, VDUSE_DESTROY_DEV, name); + if (ret) + VHOST_LOG_CONFIG(name, ERR, "Failed to destroy VDUSE device: %s\n", + strerror(errno)); + close(dev->vduse_ctrl_fd); + dev->vduse_ctrl_fd = -1; + } + + vhost_destroy_device(vid); + + return 0; +} diff --git a/lib/vhost/vduse.h b/lib/vhost/vduse.h new file mode 100644 index 0000000000..a15e5d4c16 --- /dev/null +++ b/lib/vhost/vduse.h @@ -0,0 +1,33 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Red Hat, Inc. + */ + +#ifndef _VDUSE_H +#define _VDUSE_H + +#include "vhost.h" + +#ifdef VHOST_HAS_VDUSE + +int vduse_device_create(const char *path); +int vduse_device_destroy(const char *path); + +#else + +static inline int +vduse_device_create(const char *path) +{ + VHOST_LOG_CONFIG(path, ERR, "VDUSE support disabled at build time\n"); + return -1; +} + +static inline int +vduse_device_destroy(const char *path) +{ + VHOST_LOG_CONFIG(path, ERR, "VDUSE support disabled at build time\n"); + return -1; +} + +#endif /* VHOST_HAS_VDUSE */ + +#endif /* _VDUSE_H */ diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h index 04267a3ac5..9ecede0f30 100644 --- a/lib/vhost/vhost.h +++ b/lib/vhost/vhost.h @@ -526,6 +526,8 @@ struct virtio_net { int postcopy_ufd; int postcopy_listening; + int vduse_ctrl_fd; + int vduse_dev_fd; struct vhost_virtqueue *cvq;