From patchwork Thu Jun 1 20:07:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127909 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EC83042C08; Thu, 1 Jun 2023 22:08:29 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4C89842D12; Thu, 1 Jun 2023 22:08:27 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id F254C42BC9 for ; Thu, 1 Jun 2023 22:08:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650105; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kdk7A4dPJ3aSyRCNhnDx2xvj/ieo1OEJTR+LhLYwHNo=; b=bRmSSZv7gKpE0yV84uvquNigC7iCWprzeQP6En7FoPyICDcnAyTKc/y7NmDJbPxM/YdPiN BTLDyYv0gfGS7C5an4TXQ81Mnd0IxUEWIa2M82jGkpYhoK3F1gStoerzni02LeWLMU8WHq ABuNRBouM5n2HNyAeFfVs51O231LiHE= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-558-NBP6OI9CPB2qsaOrU-SOhw-1; Thu, 01 Jun 2023 16:08:22 -0400 X-MC-Unique: NBP6OI9CPB2qsaOrU-SOhw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EBFB03806723; Thu, 1 Jun 2023 20:08:21 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2B6EF202696C; Thu, 1 Jun 2023 20:08:19 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin , stable@dpdk.org Subject: [PATCH v4 01/26] vhost: fix IOTLB entries overlap check with previous entry Date: Thu, 1 Jun 2023 22:07:47 +0200 Message-Id: <20230601200812.672233-2-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Commit 22b6d0ac691a ("vhost: fix madvise IOTLB entries pages overlap check") fixed the check to ensure the entry to be removed does not overlap with the next one in the IOTLB cache before marking it as DONTDUMP with madvise(). This is not enough, because the same issue is present when comparing with the previous entry in the cache, where the end address of the previous entry should be used, not the start one. Fixes: dea092d0addb ("vhost: fix madvise arguments alignment") Cc: stable@dpdk.org Signed-off-by: Maxime Coquelin Acked-by: Mike Pattrick Reviewed-by: Chenbo Xia --- lib/vhost/iotlb.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/lib/vhost/iotlb.c b/lib/vhost/iotlb.c index 3f45bc6061..870c8acb88 100644 --- a/lib/vhost/iotlb.c +++ b/lib/vhost/iotlb.c @@ -178,8 +178,8 @@ vhost_user_iotlb_cache_random_evict(struct virtio_net *dev, struct vhost_virtque mask = ~(alignment - 1); /* Don't disable coredump if the previous node is in the same page */ - if (prev_node == NULL || - (node->uaddr & mask) != (prev_node->uaddr & mask)) { + if (prev_node == NULL || (node->uaddr & mask) != + ((prev_node->uaddr + prev_node->size - 1) & mask)) { next_node = RTE_TAILQ_NEXT(node, next); /* Don't disable coredump if the next node is in the same page */ if (next_node == NULL || ((node->uaddr + node->size - 1) & mask) != @@ -283,8 +283,8 @@ vhost_user_iotlb_cache_remove(struct virtio_net *dev, struct vhost_virtqueue *vq mask = ~(alignment-1); /* Don't disable coredump if the previous node is in the same page */ - if (prev_node == NULL || - (node->uaddr & mask) != (prev_node->uaddr & mask)) { + if (prev_node == NULL || (node->uaddr & mask) != + ((prev_node->uaddr + prev_node->size - 1) & mask)) { next_node = RTE_TAILQ_NEXT(node, next); /* Don't disable coredump if the next node is in the same page */ if (next_node == NULL || ((node->uaddr + node->size - 1) & mask) != From patchwork Thu Jun 1 20:07:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127911 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C63E542C08; Thu, 1 Jun 2023 22:08:40 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B800742D44; Thu, 1 Jun 2023 22:08:33 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 2F78342BC9 for ; Thu, 1 Jun 2023 22:08:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650109; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nghe3pVt71XrFPUd8X5i89nN3A/M9CrhzX8c+jD0QKw=; b=HWh6BOYXIi9fXXYfDh5Xw9EpApiC0mfVTjVvvzTuYpdoIlmOFfuzij93fBk/3YBZJOYeMb RAnXuysZIWDjwjxkE4hUeySEpVrPjXCHqDhfKDBDjEy8Ef/HL+GiSK9Hj5Of2jU/zvKwJz tPtbpSk4Dpcq6b+u841teEK2V1jzbGA= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-665-GkX6WxP3OKCwKVOq9sByKw-1; Thu, 01 Jun 2023 16:08:25 -0400 X-MC-Unique: GkX6WxP3OKCwKVOq9sByKw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F091638294A0; Thu, 1 Jun 2023 20:08:24 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 41E3A2029F6D; Thu, 1 Jun 2023 20:08:22 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v4 02/26] vhost: add helper of IOTLB entries coredump Date: Thu, 1 Jun 2023 22:07:48 +0200 Message-Id: <20230601200812.672233-3-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch reworks IOTLB code to extract madvise-related bits into dedicated helper. This refactoring improves code sharing. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/iotlb.c | 77 +++++++++++++++++++++++++---------------------- 1 file changed, 41 insertions(+), 36 deletions(-) diff --git a/lib/vhost/iotlb.c b/lib/vhost/iotlb.c index 870c8acb88..51d45de446 100644 --- a/lib/vhost/iotlb.c +++ b/lib/vhost/iotlb.c @@ -23,6 +23,34 @@ struct vhost_iotlb_entry { #define IOTLB_CACHE_SIZE 2048 +static void +vhost_user_iotlb_set_dump(struct virtio_net *dev, struct vhost_iotlb_entry *node) +{ + uint64_t align; + + align = hua_to_alignment(dev->mem, (void *)(uintptr_t)node->uaddr); + + mem_set_dump((void *)(uintptr_t)node->uaddr, node->size, true, align); +} + +static void +vhost_user_iotlb_clear_dump(struct virtio_net *dev, struct vhost_iotlb_entry *node, + struct vhost_iotlb_entry *prev, struct vhost_iotlb_entry *next) +{ + uint64_t align, mask; + + align = hua_to_alignment(dev->mem, (void *)(uintptr_t)node->uaddr); + mask = ~(align - 1); + + /* Don't disable coredump if the previous node is in the same page */ + if (prev == NULL || (node->uaddr & mask) != ((prev->uaddr + prev->size - 1) & mask)) { + /* Don't disable coredump if the next node is in the same page */ + if (next == NULL || + ((node->uaddr + node->size - 1) & mask) != (next->uaddr & mask)) + mem_set_dump((void *)(uintptr_t)node->uaddr, node->size, false, align); + } +} + static struct vhost_iotlb_entry * vhost_user_iotlb_pool_get(struct vhost_virtqueue *vq) { @@ -149,8 +177,8 @@ vhost_user_iotlb_cache_remove_all(struct virtio_net *dev, struct vhost_virtqueue rte_rwlock_write_lock(&vq->iotlb_lock); RTE_TAILQ_FOREACH_SAFE(node, &vq->iotlb_list, next, temp_node) { - mem_set_dump((void *)(uintptr_t)node->uaddr, node->size, false, - hua_to_alignment(dev->mem, (void *)(uintptr_t)node->uaddr)); + vhost_user_iotlb_clear_dump(dev, node, NULL, NULL); + TAILQ_REMOVE(&vq->iotlb_list, node, next); vhost_user_iotlb_pool_put(vq, node); } @@ -164,7 +192,6 @@ static void vhost_user_iotlb_cache_random_evict(struct virtio_net *dev, struct vhost_virtqueue *vq) { struct vhost_iotlb_entry *node, *temp_node, *prev_node = NULL; - uint64_t alignment, mask; int entry_idx; rte_rwlock_write_lock(&vq->iotlb_lock); @@ -173,20 +200,10 @@ vhost_user_iotlb_cache_random_evict(struct virtio_net *dev, struct vhost_virtque RTE_TAILQ_FOREACH_SAFE(node, &vq->iotlb_list, next, temp_node) { if (!entry_idx) { - struct vhost_iotlb_entry *next_node; - alignment = hua_to_alignment(dev->mem, (void *)(uintptr_t)node->uaddr); - mask = ~(alignment - 1); - - /* Don't disable coredump if the previous node is in the same page */ - if (prev_node == NULL || (node->uaddr & mask) != - ((prev_node->uaddr + prev_node->size - 1) & mask)) { - next_node = RTE_TAILQ_NEXT(node, next); - /* Don't disable coredump if the next node is in the same page */ - if (next_node == NULL || ((node->uaddr + node->size - 1) & mask) != - (next_node->uaddr & mask)) - mem_set_dump((void *)(uintptr_t)node->uaddr, node->size, - false, alignment); - } + struct vhost_iotlb_entry *next_node = RTE_TAILQ_NEXT(node, next); + + vhost_user_iotlb_clear_dump(dev, node, prev_node, next_node); + TAILQ_REMOVE(&vq->iotlb_list, node, next); vhost_user_iotlb_pool_put(vq, node); vq->iotlb_cache_nr--; @@ -240,16 +257,16 @@ vhost_user_iotlb_cache_insert(struct virtio_net *dev, struct vhost_virtqueue *vq vhost_user_iotlb_pool_put(vq, new_node); goto unlock; } else if (node->iova > new_node->iova) { - mem_set_dump((void *)(uintptr_t)new_node->uaddr, new_node->size, true, - hua_to_alignment(dev->mem, (void *)(uintptr_t)new_node->uaddr)); + vhost_user_iotlb_set_dump(dev, new_node); + TAILQ_INSERT_BEFORE(node, new_node, next); vq->iotlb_cache_nr++; goto unlock; } } - mem_set_dump((void *)(uintptr_t)new_node->uaddr, new_node->size, true, - hua_to_alignment(dev->mem, (void *)(uintptr_t)new_node->uaddr)); + vhost_user_iotlb_set_dump(dev, new_node); + TAILQ_INSERT_TAIL(&vq->iotlb_list, new_node, next); vq->iotlb_cache_nr++; @@ -265,7 +282,6 @@ vhost_user_iotlb_cache_remove(struct virtio_net *dev, struct vhost_virtqueue *vq uint64_t iova, uint64_t size) { struct vhost_iotlb_entry *node, *temp_node, *prev_node = NULL; - uint64_t alignment, mask; if (unlikely(!size)) return; @@ -278,20 +294,9 @@ vhost_user_iotlb_cache_remove(struct virtio_net *dev, struct vhost_virtqueue *vq break; if (iova < node->iova + node->size) { - struct vhost_iotlb_entry *next_node; - alignment = hua_to_alignment(dev->mem, (void *)(uintptr_t)node->uaddr); - mask = ~(alignment-1); - - /* Don't disable coredump if the previous node is in the same page */ - if (prev_node == NULL || (node->uaddr & mask) != - ((prev_node->uaddr + prev_node->size - 1) & mask)) { - next_node = RTE_TAILQ_NEXT(node, next); - /* Don't disable coredump if the next node is in the same page */ - if (next_node == NULL || ((node->uaddr + node->size - 1) & mask) != - (next_node->uaddr & mask)) - mem_set_dump((void *)(uintptr_t)node->uaddr, node->size, - false, alignment); - } + struct vhost_iotlb_entry *next_node = RTE_TAILQ_NEXT(node, next); + + vhost_user_iotlb_clear_dump(dev, node, prev_node, next_node); TAILQ_REMOVE(&vq->iotlb_list, node, next); vhost_user_iotlb_pool_put(vq, node); From patchwork Thu Jun 1 20:07:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127910 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B94BE42C08; Thu, 1 Jun 2023 22:08:34 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 86EA342BC9; Thu, 1 Jun 2023 22:08:32 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 1083440EF0 for ; Thu, 1 Jun 2023 22:08:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650109; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xtWU2swtOfVhpOdXEIx7YzgGQtuh2JboI/uCu+jZY8U=; b=D7eEqlb9JzKvvjVcJU6P4fwcaYYl8r5LGmC5Bg7ye+dLFC8l6v/NvYlASzcVtwjdYTNa1Q hmOC4zcMjxFVwOMlNTJ873pE7WMVP8wn6tvdn5XvHRiBds3oTU3zAYuBTIKrS6Aixb+clD IQXkpLulf8G/GZ+fdge/UG/N3q5al+Q= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-122-eL0ZHQFWOji4Z9kq_EAS2Q-1; Thu, 01 Jun 2023 16:08:28 -0400 X-MC-Unique: eL0ZHQFWOji4Z9kq_EAS2Q-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D64651C05152; Thu, 1 Jun 2023 20:08:27 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 444B02029F6D; Thu, 1 Jun 2023 20:08:25 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v4 03/26] vhost: add helper for IOTLB entries shared page check Date: Thu, 1 Jun 2023 22:07:49 +0200 Message-Id: <20230601200812.672233-4-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch introduces a helper to check whether two IOTLB entries share a page. Signed-off-by: Maxime Coquelin Acked-by: Mike Pattrick Reviewed-by: Chenbo Xia --- lib/vhost/iotlb.c | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/lib/vhost/iotlb.c b/lib/vhost/iotlb.c index 51d45de446..4ef038adff 100644 --- a/lib/vhost/iotlb.c +++ b/lib/vhost/iotlb.c @@ -23,6 +23,23 @@ struct vhost_iotlb_entry { #define IOTLB_CACHE_SIZE 2048 +static bool +vhost_user_iotlb_share_page(struct vhost_iotlb_entry *a, struct vhost_iotlb_entry *b, + uint64_t align) +{ + uint64_t a_end, b_start; + + if (a == NULL || b == NULL) + return false; + + /* Assumes entry a lower than entry b */ + RTE_ASSERT(a->uaddr < b->uaddr); + a_end = RTE_ALIGN_CEIL(a->uaddr + a->size, align); + b_start = RTE_ALIGN_FLOOR(b->uaddr, align); + + return a_end > b_start; +} + static void vhost_user_iotlb_set_dump(struct virtio_net *dev, struct vhost_iotlb_entry *node) { @@ -37,16 +54,14 @@ static void vhost_user_iotlb_clear_dump(struct virtio_net *dev, struct vhost_iotlb_entry *node, struct vhost_iotlb_entry *prev, struct vhost_iotlb_entry *next) { - uint64_t align, mask; + uint64_t align; align = hua_to_alignment(dev->mem, (void *)(uintptr_t)node->uaddr); - mask = ~(align - 1); /* Don't disable coredump if the previous node is in the same page */ - if (prev == NULL || (node->uaddr & mask) != ((prev->uaddr + prev->size - 1) & mask)) { + if (!vhost_user_iotlb_share_page(prev, node, align)) { /* Don't disable coredump if the next node is in the same page */ - if (next == NULL || - ((node->uaddr + node->size - 1) & mask) != (next->uaddr & mask)) + if (!vhost_user_iotlb_share_page(node, next, align)) mem_set_dump((void *)(uintptr_t)node->uaddr, node->size, false, align); } } From patchwork Thu Jun 1 20:07:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127912 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0501042C08; Thu, 1 Jun 2023 22:08:49 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 815EF42D74; Thu, 1 Jun 2023 22:08:36 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 0FA0A42D6C for ; Thu, 1 Jun 2023 22:08:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650114; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3viU0ucDifS7vbgVuPxlajnSbkTJ5nflHILVn2DSYNM=; b=L5Y9q8NS9DyypauUPuMqyrykQcWpK5hRAjy8J4M/wCTYdijQlNWlG9duk2QF1yuTHmHqpS 2UXDk3bz/JhjEqtdMHEZN50ftqfBLqIUF38VKnU5sXaodNqWGP+AcBgCAaTJ9vETLJ1Z6V FSQ5o2D004tcLUGlFn/V86c5aXdtX64= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-307-DvfBpUnlMeyGUepVdFtfbA-1; Thu, 01 Jun 2023 16:08:31 -0400 X-MC-Unique: DvfBpUnlMeyGUepVdFtfbA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F1AAF185A795; Thu, 1 Jun 2023 20:08:30 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2BC1C2029F6D; Thu, 1 Jun 2023 20:08:28 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin , stable@dpdk.org Subject: [PATCH v4 04/26] vhost: don't dump unneeded pages with IOTLB Date: Thu, 1 Jun 2023 22:07:50 +0200 Message-Id: <20230601200812.672233-5-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On IOTLB entry removal, previous fixes took care of not marking pages shared with other IOTLB entries as DONTDUMP. However, if an IOTLB entry is spanned on multiple pages, the other pages were kept as DODUMP while they might not have been shared with other entries, increasing needlessly the coredump size. This patch addresses this issue by excluding only the shared pages from madvise's DONTDUMP. Fixes: dea092d0addb ("vhost: fix madvise arguments alignment") Cc: stable@dpdk.org Signed-off-by: Maxime Coquelin Acked-by: Mike Pattrick Reviewed-by: Chenbo Xia --- lib/vhost/iotlb.c | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/lib/vhost/iotlb.c b/lib/vhost/iotlb.c index 4ef038adff..95d67ac832 100644 --- a/lib/vhost/iotlb.c +++ b/lib/vhost/iotlb.c @@ -54,16 +54,23 @@ static void vhost_user_iotlb_clear_dump(struct virtio_net *dev, struct vhost_iotlb_entry *node, struct vhost_iotlb_entry *prev, struct vhost_iotlb_entry *next) { - uint64_t align; + uint64_t align, start, end; + + start = node->uaddr; + end = node->uaddr + node->size; align = hua_to_alignment(dev->mem, (void *)(uintptr_t)node->uaddr); - /* Don't disable coredump if the previous node is in the same page */ - if (!vhost_user_iotlb_share_page(prev, node, align)) { - /* Don't disable coredump if the next node is in the same page */ - if (!vhost_user_iotlb_share_page(node, next, align)) - mem_set_dump((void *)(uintptr_t)node->uaddr, node->size, false, align); - } + /* Skip first page if shared with previous entry. */ + if (vhost_user_iotlb_share_page(prev, node, align)) + start = RTE_ALIGN_CEIL(start, align); + + /* Skip last page if shared with next entry. */ + if (vhost_user_iotlb_share_page(node, next, align)) + end = RTE_ALIGN_FLOOR(end, align); + + if (end > start) + mem_set_dump((void *)(uintptr_t)start, end - start, false, align); } static struct vhost_iotlb_entry * From patchwork Thu Jun 1 20:07:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127913 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E260B42C08; Thu, 1 Jun 2023 22:08:54 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 93F4442D5D; Thu, 1 Jun 2023 22:08:37 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 2BE8642D3F for ; Thu, 1 Jun 2023 22:08:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650115; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=REctACCwjaix+IZinDTY/dmf8cyAct9mknHtWiNZYgU=; b=LJOg6kC4tGUUexuQFLWlYCVZX9sYc+Urq3kIIGMFTB80s3WAcl50LZLG8zSgx5ef9BFHdt iJ+zjnplFzdeR5I502RqhpjK+wHvuI6zjdru6d0pFE6K9KHpMdbVyyYX7O57aGGRkjthrR qr8/wY0WrxA+mSBXsknEA6Nuct9iHpY= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-495-VgRrnlivNU-n-q0wx70laA-1; Thu, 01 Jun 2023 16:08:34 -0400 X-MC-Unique: VgRrnlivNU-n-q0wx70laA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 074433C025AD; Thu, 1 Jun 2023 20:08:34 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 45D19200AE6F; Thu, 1 Jun 2023 20:08:31 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v4 05/26] vhost: change to single IOTLB cache per device Date: Thu, 1 Jun 2023 22:07:51 +0200 Message-Id: <20230601200812.672233-6-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch simplifies IOTLB implementation and improves IOTLB memory consumption by having a single IOTLB cache per device, instead of having one per queue. In order to not impact performance, it keeps an IOTLB lock per virtqueue, so that there is no contention between multiple queue trying to acquire it. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/iotlb.c | 212 +++++++++++++++++++---------------------- lib/vhost/iotlb.h | 43 ++++++--- lib/vhost/vhost.c | 18 ++-- lib/vhost/vhost.h | 16 ++-- lib/vhost/vhost_user.c | 23 +++-- 5 files changed, 159 insertions(+), 153 deletions(-) diff --git a/lib/vhost/iotlb.c b/lib/vhost/iotlb.c index 95d67ac832..6d49bf6b30 100644 --- a/lib/vhost/iotlb.c +++ b/lib/vhost/iotlb.c @@ -74,86 +74,81 @@ vhost_user_iotlb_clear_dump(struct virtio_net *dev, struct vhost_iotlb_entry *no } static struct vhost_iotlb_entry * -vhost_user_iotlb_pool_get(struct vhost_virtqueue *vq) +vhost_user_iotlb_pool_get(struct virtio_net *dev) { struct vhost_iotlb_entry *node; - rte_spinlock_lock(&vq->iotlb_free_lock); - node = SLIST_FIRST(&vq->iotlb_free_list); + rte_spinlock_lock(&dev->iotlb_free_lock); + node = SLIST_FIRST(&dev->iotlb_free_list); if (node != NULL) - SLIST_REMOVE_HEAD(&vq->iotlb_free_list, next_free); - rte_spinlock_unlock(&vq->iotlb_free_lock); + SLIST_REMOVE_HEAD(&dev->iotlb_free_list, next_free); + rte_spinlock_unlock(&dev->iotlb_free_lock); return node; } static void -vhost_user_iotlb_pool_put(struct vhost_virtqueue *vq, - struct vhost_iotlb_entry *node) +vhost_user_iotlb_pool_put(struct virtio_net *dev, struct vhost_iotlb_entry *node) { - rte_spinlock_lock(&vq->iotlb_free_lock); - SLIST_INSERT_HEAD(&vq->iotlb_free_list, node, next_free); - rte_spinlock_unlock(&vq->iotlb_free_lock); + rte_spinlock_lock(&dev->iotlb_free_lock); + SLIST_INSERT_HEAD(&dev->iotlb_free_list, node, next_free); + rte_spinlock_unlock(&dev->iotlb_free_lock); } static void -vhost_user_iotlb_cache_random_evict(struct virtio_net *dev, struct vhost_virtqueue *vq); +vhost_user_iotlb_cache_random_evict(struct virtio_net *dev); static void -vhost_user_iotlb_pending_remove_all(struct vhost_virtqueue *vq) +vhost_user_iotlb_pending_remove_all(struct virtio_net *dev) { struct vhost_iotlb_entry *node, *temp_node; - rte_rwlock_write_lock(&vq->iotlb_pending_lock); + rte_rwlock_write_lock(&dev->iotlb_pending_lock); - RTE_TAILQ_FOREACH_SAFE(node, &vq->iotlb_pending_list, next, temp_node) { - TAILQ_REMOVE(&vq->iotlb_pending_list, node, next); - vhost_user_iotlb_pool_put(vq, node); + RTE_TAILQ_FOREACH_SAFE(node, &dev->iotlb_pending_list, next, temp_node) { + TAILQ_REMOVE(&dev->iotlb_pending_list, node, next); + vhost_user_iotlb_pool_put(dev, node); } - rte_rwlock_write_unlock(&vq->iotlb_pending_lock); + rte_rwlock_write_unlock(&dev->iotlb_pending_lock); } bool -vhost_user_iotlb_pending_miss(struct vhost_virtqueue *vq, uint64_t iova, - uint8_t perm) +vhost_user_iotlb_pending_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm) { struct vhost_iotlb_entry *node; bool found = false; - rte_rwlock_read_lock(&vq->iotlb_pending_lock); + rte_rwlock_read_lock(&dev->iotlb_pending_lock); - TAILQ_FOREACH(node, &vq->iotlb_pending_list, next) { + TAILQ_FOREACH(node, &dev->iotlb_pending_list, next) { if ((node->iova == iova) && (node->perm == perm)) { found = true; break; } } - rte_rwlock_read_unlock(&vq->iotlb_pending_lock); + rte_rwlock_read_unlock(&dev->iotlb_pending_lock); return found; } void -vhost_user_iotlb_pending_insert(struct virtio_net *dev, struct vhost_virtqueue *vq, - uint64_t iova, uint8_t perm) +vhost_user_iotlb_pending_insert(struct virtio_net *dev, uint64_t iova, uint8_t perm) { struct vhost_iotlb_entry *node; - node = vhost_user_iotlb_pool_get(vq); + node = vhost_user_iotlb_pool_get(dev); if (node == NULL) { VHOST_LOG_CONFIG(dev->ifname, DEBUG, - "IOTLB pool for vq %"PRIu32" empty, clear entries for pending insertion\n", - vq->index); - if (!TAILQ_EMPTY(&vq->iotlb_pending_list)) - vhost_user_iotlb_pending_remove_all(vq); + "IOTLB pool empty, clear entries for pending insertion\n"); + if (!TAILQ_EMPTY(&dev->iotlb_pending_list)) + vhost_user_iotlb_pending_remove_all(dev); else - vhost_user_iotlb_cache_random_evict(dev, vq); - node = vhost_user_iotlb_pool_get(vq); + vhost_user_iotlb_cache_random_evict(dev); + node = vhost_user_iotlb_pool_get(dev); if (node == NULL) { VHOST_LOG_CONFIG(dev->ifname, ERR, - "IOTLB pool vq %"PRIu32" still empty, pending insertion failure\n", - vq->index); + "IOTLB pool still empty, pending insertion failure\n"); return; } } @@ -161,22 +156,21 @@ vhost_user_iotlb_pending_insert(struct virtio_net *dev, struct vhost_virtqueue * node->iova = iova; node->perm = perm; - rte_rwlock_write_lock(&vq->iotlb_pending_lock); + rte_rwlock_write_lock(&dev->iotlb_pending_lock); - TAILQ_INSERT_TAIL(&vq->iotlb_pending_list, node, next); + TAILQ_INSERT_TAIL(&dev->iotlb_pending_list, node, next); - rte_rwlock_write_unlock(&vq->iotlb_pending_lock); + rte_rwlock_write_unlock(&dev->iotlb_pending_lock); } void -vhost_user_iotlb_pending_remove(struct vhost_virtqueue *vq, - uint64_t iova, uint64_t size, uint8_t perm) +vhost_user_iotlb_pending_remove(struct virtio_net *dev, uint64_t iova, uint64_t size, uint8_t perm) { struct vhost_iotlb_entry *node, *temp_node; - rte_rwlock_write_lock(&vq->iotlb_pending_lock); + rte_rwlock_write_lock(&dev->iotlb_pending_lock); - RTE_TAILQ_FOREACH_SAFE(node, &vq->iotlb_pending_list, next, + RTE_TAILQ_FOREACH_SAFE(node, &dev->iotlb_pending_list, next, temp_node) { if (node->iova < iova) continue; @@ -184,81 +178,78 @@ vhost_user_iotlb_pending_remove(struct vhost_virtqueue *vq, continue; if ((node->perm & perm) != node->perm) continue; - TAILQ_REMOVE(&vq->iotlb_pending_list, node, next); - vhost_user_iotlb_pool_put(vq, node); + TAILQ_REMOVE(&dev->iotlb_pending_list, node, next); + vhost_user_iotlb_pool_put(dev, node); } - rte_rwlock_write_unlock(&vq->iotlb_pending_lock); + rte_rwlock_write_unlock(&dev->iotlb_pending_lock); } static void -vhost_user_iotlb_cache_remove_all(struct virtio_net *dev, struct vhost_virtqueue *vq) +vhost_user_iotlb_cache_remove_all(struct virtio_net *dev) { struct vhost_iotlb_entry *node, *temp_node; - rte_rwlock_write_lock(&vq->iotlb_lock); + vhost_user_iotlb_wr_lock_all(dev); - RTE_TAILQ_FOREACH_SAFE(node, &vq->iotlb_list, next, temp_node) { + RTE_TAILQ_FOREACH_SAFE(node, &dev->iotlb_list, next, temp_node) { vhost_user_iotlb_clear_dump(dev, node, NULL, NULL); - TAILQ_REMOVE(&vq->iotlb_list, node, next); - vhost_user_iotlb_pool_put(vq, node); + TAILQ_REMOVE(&dev->iotlb_list, node, next); + vhost_user_iotlb_pool_put(dev, node); } - vq->iotlb_cache_nr = 0; + dev->iotlb_cache_nr = 0; - rte_rwlock_write_unlock(&vq->iotlb_lock); + vhost_user_iotlb_wr_unlock_all(dev); } static void -vhost_user_iotlb_cache_random_evict(struct virtio_net *dev, struct vhost_virtqueue *vq) +vhost_user_iotlb_cache_random_evict(struct virtio_net *dev) { struct vhost_iotlb_entry *node, *temp_node, *prev_node = NULL; int entry_idx; - rte_rwlock_write_lock(&vq->iotlb_lock); + vhost_user_iotlb_wr_lock_all(dev); - entry_idx = rte_rand() % vq->iotlb_cache_nr; + entry_idx = rte_rand() % dev->iotlb_cache_nr; - RTE_TAILQ_FOREACH_SAFE(node, &vq->iotlb_list, next, temp_node) { + RTE_TAILQ_FOREACH_SAFE(node, &dev->iotlb_list, next, temp_node) { if (!entry_idx) { struct vhost_iotlb_entry *next_node = RTE_TAILQ_NEXT(node, next); vhost_user_iotlb_clear_dump(dev, node, prev_node, next_node); - TAILQ_REMOVE(&vq->iotlb_list, node, next); - vhost_user_iotlb_pool_put(vq, node); - vq->iotlb_cache_nr--; + TAILQ_REMOVE(&dev->iotlb_list, node, next); + vhost_user_iotlb_pool_put(dev, node); + dev->iotlb_cache_nr--; break; } prev_node = node; entry_idx--; } - rte_rwlock_write_unlock(&vq->iotlb_lock); + vhost_user_iotlb_wr_unlock_all(dev); } void -vhost_user_iotlb_cache_insert(struct virtio_net *dev, struct vhost_virtqueue *vq, - uint64_t iova, uint64_t uaddr, +vhost_user_iotlb_cache_insert(struct virtio_net *dev, uint64_t iova, uint64_t uaddr, uint64_t size, uint8_t perm) { struct vhost_iotlb_entry *node, *new_node; - new_node = vhost_user_iotlb_pool_get(vq); + new_node = vhost_user_iotlb_pool_get(dev); if (new_node == NULL) { VHOST_LOG_CONFIG(dev->ifname, DEBUG, - "IOTLB pool vq %"PRIu32" empty, clear entries for cache insertion\n", - vq->index); - if (!TAILQ_EMPTY(&vq->iotlb_list)) - vhost_user_iotlb_cache_random_evict(dev, vq); + "IOTLB pool empty, clear entries for cache insertion\n"); + if (!TAILQ_EMPTY(&dev->iotlb_list)) + vhost_user_iotlb_cache_random_evict(dev); else - vhost_user_iotlb_pending_remove_all(vq); - new_node = vhost_user_iotlb_pool_get(vq); + vhost_user_iotlb_pending_remove_all(dev); + new_node = vhost_user_iotlb_pool_get(dev); if (new_node == NULL) { VHOST_LOG_CONFIG(dev->ifname, ERR, - "IOTLB pool vq %"PRIu32" still empty, cache insertion failed\n", - vq->index); + "IOTLB pool still empty, cache insertion failed\n"); return; } } @@ -268,49 +259,47 @@ vhost_user_iotlb_cache_insert(struct virtio_net *dev, struct vhost_virtqueue *vq new_node->size = size; new_node->perm = perm; - rte_rwlock_write_lock(&vq->iotlb_lock); + vhost_user_iotlb_wr_lock_all(dev); - TAILQ_FOREACH(node, &vq->iotlb_list, next) { + TAILQ_FOREACH(node, &dev->iotlb_list, next) { /* * Entries must be invalidated before being updated. * So if iova already in list, assume identical. */ if (node->iova == new_node->iova) { - vhost_user_iotlb_pool_put(vq, new_node); + vhost_user_iotlb_pool_put(dev, new_node); goto unlock; } else if (node->iova > new_node->iova) { vhost_user_iotlb_set_dump(dev, new_node); TAILQ_INSERT_BEFORE(node, new_node, next); - vq->iotlb_cache_nr++; + dev->iotlb_cache_nr++; goto unlock; } } vhost_user_iotlb_set_dump(dev, new_node); - TAILQ_INSERT_TAIL(&vq->iotlb_list, new_node, next); - vq->iotlb_cache_nr++; + TAILQ_INSERT_TAIL(&dev->iotlb_list, new_node, next); + dev->iotlb_cache_nr++; unlock: - vhost_user_iotlb_pending_remove(vq, iova, size, perm); - - rte_rwlock_write_unlock(&vq->iotlb_lock); + vhost_user_iotlb_pending_remove(dev, iova, size, perm); + vhost_user_iotlb_wr_unlock_all(dev); } void -vhost_user_iotlb_cache_remove(struct virtio_net *dev, struct vhost_virtqueue *vq, - uint64_t iova, uint64_t size) +vhost_user_iotlb_cache_remove(struct virtio_net *dev, uint64_t iova, uint64_t size) { struct vhost_iotlb_entry *node, *temp_node, *prev_node = NULL; if (unlikely(!size)) return; - rte_rwlock_write_lock(&vq->iotlb_lock); + vhost_user_iotlb_wr_lock_all(dev); - RTE_TAILQ_FOREACH_SAFE(node, &vq->iotlb_list, next, temp_node) { + RTE_TAILQ_FOREACH_SAFE(node, &dev->iotlb_list, next, temp_node) { /* Sorted list */ if (unlikely(iova + size < node->iova)) break; @@ -320,19 +309,19 @@ vhost_user_iotlb_cache_remove(struct virtio_net *dev, struct vhost_virtqueue *vq vhost_user_iotlb_clear_dump(dev, node, prev_node, next_node); - TAILQ_REMOVE(&vq->iotlb_list, node, next); - vhost_user_iotlb_pool_put(vq, node); - vq->iotlb_cache_nr--; - } else + TAILQ_REMOVE(&dev->iotlb_list, node, next); + vhost_user_iotlb_pool_put(dev, node); + dev->iotlb_cache_nr--; + } else { prev_node = node; + } } - rte_rwlock_write_unlock(&vq->iotlb_lock); + vhost_user_iotlb_wr_unlock_all(dev); } uint64_t -vhost_user_iotlb_cache_find(struct vhost_virtqueue *vq, uint64_t iova, - uint64_t *size, uint8_t perm) +vhost_user_iotlb_cache_find(struct virtio_net *dev, uint64_t iova, uint64_t *size, uint8_t perm) { struct vhost_iotlb_entry *node; uint64_t offset, vva = 0, mapped = 0; @@ -340,7 +329,7 @@ vhost_user_iotlb_cache_find(struct vhost_virtqueue *vq, uint64_t iova, if (unlikely(!*size)) goto out; - TAILQ_FOREACH(node, &vq->iotlb_list, next) { + TAILQ_FOREACH(node, &dev->iotlb_list, next) { /* List sorted by iova */ if (unlikely(iova < node->iova)) break; @@ -373,60 +362,57 @@ vhost_user_iotlb_cache_find(struct vhost_virtqueue *vq, uint64_t iova, } void -vhost_user_iotlb_flush_all(struct virtio_net *dev, struct vhost_virtqueue *vq) +vhost_user_iotlb_flush_all(struct virtio_net *dev) { - vhost_user_iotlb_cache_remove_all(dev, vq); - vhost_user_iotlb_pending_remove_all(vq); + vhost_user_iotlb_cache_remove_all(dev); + vhost_user_iotlb_pending_remove_all(dev); } int -vhost_user_iotlb_init(struct virtio_net *dev, struct vhost_virtqueue *vq) +vhost_user_iotlb_init(struct virtio_net *dev) { unsigned int i; int socket = 0; - if (vq->iotlb_pool) { + if (dev->iotlb_pool) { /* * The cache has already been initialized, * just drop all cached and pending entries. */ - vhost_user_iotlb_flush_all(dev, vq); - rte_free(vq->iotlb_pool); + vhost_user_iotlb_flush_all(dev); + rte_free(dev->iotlb_pool); } #ifdef RTE_LIBRTE_VHOST_NUMA - if (get_mempolicy(&socket, NULL, 0, vq, MPOL_F_NODE | MPOL_F_ADDR) != 0) + if (get_mempolicy(&socket, NULL, 0, dev, MPOL_F_NODE | MPOL_F_ADDR) != 0) socket = 0; #endif - rte_spinlock_init(&vq->iotlb_free_lock); - rte_rwlock_init(&vq->iotlb_lock); - rte_rwlock_init(&vq->iotlb_pending_lock); + rte_spinlock_init(&dev->iotlb_free_lock); + rte_rwlock_init(&dev->iotlb_pending_lock); - SLIST_INIT(&vq->iotlb_free_list); - TAILQ_INIT(&vq->iotlb_list); - TAILQ_INIT(&vq->iotlb_pending_list); + SLIST_INIT(&dev->iotlb_free_list); + TAILQ_INIT(&dev->iotlb_list); + TAILQ_INIT(&dev->iotlb_pending_list); if (dev->flags & VIRTIO_DEV_SUPPORT_IOMMU) { - vq->iotlb_pool = rte_calloc_socket("iotlb", IOTLB_CACHE_SIZE, + dev->iotlb_pool = rte_calloc_socket("iotlb", IOTLB_CACHE_SIZE, sizeof(struct vhost_iotlb_entry), 0, socket); - if (!vq->iotlb_pool) { - VHOST_LOG_CONFIG(dev->ifname, ERR, - "Failed to create IOTLB cache pool for vq %"PRIu32"\n", - vq->index); + if (!dev->iotlb_pool) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to create IOTLB cache pool\n"); return -1; } for (i = 0; i < IOTLB_CACHE_SIZE; i++) - vhost_user_iotlb_pool_put(vq, &vq->iotlb_pool[i]); + vhost_user_iotlb_pool_put(dev, &dev->iotlb_pool[i]); } - vq->iotlb_cache_nr = 0; + dev->iotlb_cache_nr = 0; return 0; } void -vhost_user_iotlb_destroy(struct vhost_virtqueue *vq) +vhost_user_iotlb_destroy(struct virtio_net *dev) { - rte_free(vq->iotlb_pool); + rte_free(dev->iotlb_pool); } diff --git a/lib/vhost/iotlb.h b/lib/vhost/iotlb.h index 73b5465b41..3490b9e6be 100644 --- a/lib/vhost/iotlb.h +++ b/lib/vhost/iotlb.h @@ -37,20 +37,37 @@ vhost_user_iotlb_wr_unlock(struct vhost_virtqueue *vq) rte_rwlock_write_unlock(&vq->iotlb_lock); } -void vhost_user_iotlb_cache_insert(struct virtio_net *dev, struct vhost_virtqueue *vq, - uint64_t iova, uint64_t uaddr, +static __rte_always_inline void +vhost_user_iotlb_wr_lock_all(struct virtio_net *dev) + __rte_no_thread_safety_analysis +{ + uint32_t i; + + for (i = 0; i < dev->nr_vring; i++) + rte_rwlock_write_lock(&dev->virtqueue[i]->iotlb_lock); +} + +static __rte_always_inline void +vhost_user_iotlb_wr_unlock_all(struct virtio_net *dev) + __rte_no_thread_safety_analysis +{ + uint32_t i; + + for (i = 0; i < dev->nr_vring; i++) + rte_rwlock_write_unlock(&dev->virtqueue[i]->iotlb_lock); +} + +void vhost_user_iotlb_cache_insert(struct virtio_net *dev, uint64_t iova, uint64_t uaddr, uint64_t size, uint8_t perm); -void vhost_user_iotlb_cache_remove(struct virtio_net *dev, struct vhost_virtqueue *vq, - uint64_t iova, uint64_t size); -uint64_t vhost_user_iotlb_cache_find(struct vhost_virtqueue *vq, uint64_t iova, +void vhost_user_iotlb_cache_remove(struct virtio_net *dev, uint64_t iova, uint64_t size); +uint64_t vhost_user_iotlb_cache_find(struct virtio_net *dev, uint64_t iova, uint64_t *size, uint8_t perm); -bool vhost_user_iotlb_pending_miss(struct vhost_virtqueue *vq, uint64_t iova, - uint8_t perm); -void vhost_user_iotlb_pending_insert(struct virtio_net *dev, struct vhost_virtqueue *vq, - uint64_t iova, uint8_t perm); -void vhost_user_iotlb_pending_remove(struct vhost_virtqueue *vq, uint64_t iova, +bool vhost_user_iotlb_pending_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm); +void vhost_user_iotlb_pending_insert(struct virtio_net *dev, uint64_t iova, uint8_t perm); +void vhost_user_iotlb_pending_remove(struct virtio_net *dev, uint64_t iova, uint64_t size, uint8_t perm); -void vhost_user_iotlb_flush_all(struct virtio_net *dev, struct vhost_virtqueue *vq); -int vhost_user_iotlb_init(struct virtio_net *dev, struct vhost_virtqueue *vq); -void vhost_user_iotlb_destroy(struct vhost_virtqueue *vq); +void vhost_user_iotlb_flush_all(struct virtio_net *dev); +int vhost_user_iotlb_init(struct virtio_net *dev); +void vhost_user_iotlb_destroy(struct virtio_net *dev); + #endif /* _VHOST_IOTLB_H_ */ diff --git a/lib/vhost/vhost.c b/lib/vhost/vhost.c index 79e88f986e..3ddd2a963f 100644 --- a/lib/vhost/vhost.c +++ b/lib/vhost/vhost.c @@ -67,7 +67,7 @@ __vhost_iova_to_vva(struct virtio_net *dev, struct vhost_virtqueue *vq, tmp_size = *size; - vva = vhost_user_iotlb_cache_find(vq, iova, &tmp_size, perm); + vva = vhost_user_iotlb_cache_find(dev, iova, &tmp_size, perm); if (tmp_size == *size) { if (dev->flags & VIRTIO_DEV_STATS_ENABLED) vq->stats.iotlb_hits++; @@ -79,7 +79,7 @@ __vhost_iova_to_vva(struct virtio_net *dev, struct vhost_virtqueue *vq, iova += tmp_size; - if (!vhost_user_iotlb_pending_miss(vq, iova, perm)) { + if (!vhost_user_iotlb_pending_miss(dev, iova, perm)) { /* * iotlb_lock is read-locked for a full burst, * but it only protects the iotlb cache. @@ -89,12 +89,12 @@ __vhost_iova_to_vva(struct virtio_net *dev, struct vhost_virtqueue *vq, */ vhost_user_iotlb_rd_unlock(vq); - vhost_user_iotlb_pending_insert(dev, vq, iova, perm); + vhost_user_iotlb_pending_insert(dev, iova, perm); if (vhost_user_iotlb_miss(dev, iova, perm)) { VHOST_LOG_DATA(dev->ifname, ERR, "IOTLB miss req failed for IOVA 0x%" PRIx64 "\n", iova); - vhost_user_iotlb_pending_remove(vq, iova, 1, perm); + vhost_user_iotlb_pending_remove(dev, iova, 1, perm); } vhost_user_iotlb_rd_lock(vq); @@ -401,7 +401,6 @@ free_vq(struct virtio_net *dev, struct vhost_virtqueue *vq) vhost_free_async_mem(vq); rte_rwlock_write_unlock(&vq->access_lock); rte_free(vq->batch_copy_elems); - vhost_user_iotlb_destroy(vq); rte_free(vq->log_cache); rte_free(vq); } @@ -579,7 +578,7 @@ vring_invalidate(struct virtio_net *dev __rte_unused, struct vhost_virtqueue *vq } static void -init_vring_queue(struct virtio_net *dev, struct vhost_virtqueue *vq, +init_vring_queue(struct virtio_net *dev __rte_unused, struct vhost_virtqueue *vq, uint32_t vring_idx) { int numa_node = SOCKET_ID_ANY; @@ -599,8 +598,6 @@ init_vring_queue(struct virtio_net *dev, struct vhost_virtqueue *vq, } #endif vq->numa_node = numa_node; - - vhost_user_iotlb_init(dev, vq); } static void @@ -635,6 +632,7 @@ alloc_vring_queue(struct virtio_net *dev, uint32_t vring_idx) dev->virtqueue[i] = vq; init_vring_queue(dev, vq, i); rte_rwlock_init(&vq->access_lock); + rte_rwlock_init(&vq->iotlb_lock); vq->avail_wrap_counter = 1; vq->used_wrap_counter = 1; vq->signalled_used_valid = false; @@ -799,6 +797,10 @@ vhost_setup_virtio_net(int vid, bool enable, bool compliant_ol_flags, bool stats dev->flags |= VIRTIO_DEV_SUPPORT_IOMMU; else dev->flags &= ~VIRTIO_DEV_SUPPORT_IOMMU; + + if (vhost_user_iotlb_init(dev) < 0) + VHOST_LOG_CONFIG("device", ERR, "failed to init IOTLB\n"); + } void diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h index eaf3b0d392..ee952de175 100644 --- a/lib/vhost/vhost.h +++ b/lib/vhost/vhost.h @@ -305,13 +305,6 @@ struct vhost_virtqueue { struct log_cache_entry *log_cache; rte_rwlock_t iotlb_lock; - rte_rwlock_t iotlb_pending_lock; - struct vhost_iotlb_entry *iotlb_pool; - TAILQ_HEAD(, vhost_iotlb_entry) iotlb_list; - TAILQ_HEAD(, vhost_iotlb_entry) iotlb_pending_list; - int iotlb_cache_nr; - rte_spinlock_t iotlb_free_lock; - SLIST_HEAD(, vhost_iotlb_entry) iotlb_free_list; /* Used to notify the guest (trigger interrupt) */ int callfd; @@ -486,6 +479,15 @@ struct virtio_net { int extbuf; int linearbuf; struct vhost_virtqueue *virtqueue[VHOST_MAX_QUEUE_PAIRS * 2]; + + rte_rwlock_t iotlb_pending_lock; + struct vhost_iotlb_entry *iotlb_pool; + TAILQ_HEAD(, vhost_iotlb_entry) iotlb_list; + TAILQ_HEAD(, vhost_iotlb_entry) iotlb_pending_list; + int iotlb_cache_nr; + rte_spinlock_t iotlb_free_lock; + SLIST_HEAD(, vhost_iotlb_entry) iotlb_free_list; + struct inflight_mem_info *inflight_info; #define IF_NAME_SZ (PATH_MAX > IFNAMSIZ ? PATH_MAX : IFNAMSIZ) char ifname[IF_NAME_SZ]; diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index c9454ce3d9..f2fe7ebc93 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -237,6 +237,8 @@ vhost_backend_cleanup(struct virtio_net *dev) } dev->postcopy_listening = 0; + + vhost_user_iotlb_destroy(dev); } static void @@ -539,7 +541,6 @@ numa_realloc(struct virtio_net **pdev, struct vhost_virtqueue **pvq) if (vq != dev->virtqueue[vq->index]) { VHOST_LOG_CONFIG(dev->ifname, INFO, "reallocated virtqueue on node %d\n", node); dev->virtqueue[vq->index] = vq; - vhost_user_iotlb_init(dev, vq); } if (vq_is_packed(dev)) { @@ -664,6 +665,8 @@ numa_realloc(struct virtio_net **pdev, struct vhost_virtqueue **pvq) return; } dev->guest_pages = gp; + + vhost_user_iotlb_init(dev); } #else static void @@ -1360,8 +1363,7 @@ vhost_user_set_mem_table(struct virtio_net **pdev, /* Flush IOTLB cache as previous HVAs are now invalid */ if (dev->features & (1ULL << VIRTIO_F_IOMMU_PLATFORM)) - for (i = 0; i < dev->nr_vring; i++) - vhost_user_iotlb_flush_all(dev, dev->virtqueue[i]); + vhost_user_iotlb_flush_all(dev); free_mem_region(dev); rte_free(dev->mem); @@ -2194,7 +2196,7 @@ vhost_user_get_vring_base(struct virtio_net **pdev, ctx->msg.size = sizeof(ctx->msg.payload.state); ctx->fd_num = 0; - vhost_user_iotlb_flush_all(dev, vq); + vhost_user_iotlb_flush_all(dev); vring_invalidate(dev, vq); @@ -2639,15 +2641,14 @@ vhost_user_iotlb_msg(struct virtio_net **pdev, if (!vva) return RTE_VHOST_MSG_RESULT_ERR; + vhost_user_iotlb_cache_insert(dev, imsg->iova, vva, len, imsg->perm); + for (i = 0; i < dev->nr_vring; i++) { struct vhost_virtqueue *vq = dev->virtqueue[i]; if (!vq) continue; - vhost_user_iotlb_cache_insert(dev, vq, imsg->iova, vva, - len, imsg->perm); - if (is_vring_iotlb(dev, vq, imsg)) { rte_rwlock_write_lock(&vq->access_lock); translate_ring_addresses(&dev, &vq); @@ -2657,15 +2658,14 @@ vhost_user_iotlb_msg(struct virtio_net **pdev, } break; case VHOST_IOTLB_INVALIDATE: + vhost_user_iotlb_cache_remove(dev, imsg->iova, imsg->size); + for (i = 0; i < dev->nr_vring; i++) { struct vhost_virtqueue *vq = dev->virtqueue[i]; if (!vq) continue; - vhost_user_iotlb_cache_remove(dev, vq, imsg->iova, - imsg->size); - if (is_vring_iotlb(dev, vq, imsg)) { rte_rwlock_write_lock(&vq->access_lock); vring_invalidate(dev, vq); @@ -2674,8 +2674,7 @@ vhost_user_iotlb_msg(struct virtio_net **pdev, } break; default: - VHOST_LOG_CONFIG(dev->ifname, ERR, - "invalid IOTLB message type (%d)\n", + VHOST_LOG_CONFIG(dev->ifname, ERR, "invalid IOTLB message type (%d)\n", imsg->type); return RTE_VHOST_MSG_RESULT_ERR; } From patchwork Thu Jun 1 20:07:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127914 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D20C842C08; Thu, 1 Jun 2023 22:09:02 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 14B5F42D85; Thu, 1 Jun 2023 22:08:44 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id DD9C442D73 for ; Thu, 1 Jun 2023 22:08:41 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650121; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pRtT2U6/p62Goq2GE+Yawodl2EdFsJLJutV0Bqwp8lg=; b=Np2P1iKzfxT097EVXk/LpqK54JYzZoYXl3sXh6ZDKFhYeqGRHrdtVOVzYTqGXK1XXEon3B i+TBmOd0LRSUlPKkYxX/BMzR0t7PUn7NWsuMqtURSfiYScGSi+EeRH2pR3k1ppH+9+jdlt +hdR4orV1oc32m3O+uSePuwAWeb+QTw= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-511-fT18vM8KO2yZJy3-BSe2Bw-1; Thu, 01 Jun 2023 16:08:37 -0400 X-MC-Unique: fT18vM8KO2yZJy3-BSe2Bw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 181653C025CE; Thu, 1 Jun 2023 20:08:37 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 58C2A2029F6D; Thu, 1 Jun 2023 20:08:34 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v4 06/26] vhost: add offset field to IOTLB entries Date: Thu, 1 Jun 2023 22:07:52 +0200 Message-Id: <20230601200812.672233-7-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch is a preliminary work to prepare for VDUSE support, for which we need to keep track of the mmaped base address and offset in order to be able to unmap it later when IOTLB entry is invalidated. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/iotlb.c | 30 ++++++++++++++++++------------ lib/vhost/iotlb.h | 2 +- lib/vhost/vhost_user.c | 2 +- 3 files changed, 20 insertions(+), 14 deletions(-) diff --git a/lib/vhost/iotlb.c b/lib/vhost/iotlb.c index 6d49bf6b30..aa5100e6e7 100644 --- a/lib/vhost/iotlb.c +++ b/lib/vhost/iotlb.c @@ -17,6 +17,7 @@ struct vhost_iotlb_entry { uint64_t iova; uint64_t uaddr; + uint64_t uoffset; uint64_t size; uint8_t perm; }; @@ -27,15 +28,18 @@ static bool vhost_user_iotlb_share_page(struct vhost_iotlb_entry *a, struct vhost_iotlb_entry *b, uint64_t align) { - uint64_t a_end, b_start; + uint64_t a_start, a_end, b_start; if (a == NULL || b == NULL) return false; + a_start = a->uaddr + a->uoffset; + b_start = b->uaddr + b->uoffset; + /* Assumes entry a lower than entry b */ - RTE_ASSERT(a->uaddr < b->uaddr); - a_end = RTE_ALIGN_CEIL(a->uaddr + a->size, align); - b_start = RTE_ALIGN_FLOOR(b->uaddr, align); + RTE_ASSERT(a_start < b_start); + a_end = RTE_ALIGN_CEIL(a_start + a->size, align); + b_start = RTE_ALIGN_FLOOR(b_start, align); return a_end > b_start; } @@ -43,11 +47,12 @@ vhost_user_iotlb_share_page(struct vhost_iotlb_entry *a, struct vhost_iotlb_entr static void vhost_user_iotlb_set_dump(struct virtio_net *dev, struct vhost_iotlb_entry *node) { - uint64_t align; + uint64_t align, start; - align = hua_to_alignment(dev->mem, (void *)(uintptr_t)node->uaddr); + start = node->uaddr + node->uoffset; + align = hua_to_alignment(dev->mem, (void *)(uintptr_t)start); - mem_set_dump((void *)(uintptr_t)node->uaddr, node->size, true, align); + mem_set_dump((void *)(uintptr_t)start, node->size, true, align); } static void @@ -56,10 +61,10 @@ vhost_user_iotlb_clear_dump(struct virtio_net *dev, struct vhost_iotlb_entry *no { uint64_t align, start, end; - start = node->uaddr; - end = node->uaddr + node->size; + start = node->uaddr + node->uoffset; + end = start + node->size; - align = hua_to_alignment(dev->mem, (void *)(uintptr_t)node->uaddr); + align = hua_to_alignment(dev->mem, (void *)(uintptr_t)start); /* Skip first page if shared with previous entry. */ if (vhost_user_iotlb_share_page(prev, node, align)) @@ -234,7 +239,7 @@ vhost_user_iotlb_cache_random_evict(struct virtio_net *dev) void vhost_user_iotlb_cache_insert(struct virtio_net *dev, uint64_t iova, uint64_t uaddr, - uint64_t size, uint8_t perm) + uint64_t uoffset, uint64_t size, uint8_t perm) { struct vhost_iotlb_entry *node, *new_node; @@ -256,6 +261,7 @@ vhost_user_iotlb_cache_insert(struct virtio_net *dev, uint64_t iova, uint64_t ua new_node->iova = iova; new_node->uaddr = uaddr; + new_node->uoffset = uoffset; new_node->size = size; new_node->perm = perm; @@ -344,7 +350,7 @@ vhost_user_iotlb_cache_find(struct virtio_net *dev, uint64_t iova, uint64_t *siz offset = iova - node->iova; if (!vva) - vva = node->uaddr + offset; + vva = node->uaddr + node->uoffset + offset; mapped += node->size - offset; iova = node->iova + node->size; diff --git a/lib/vhost/iotlb.h b/lib/vhost/iotlb.h index 3490b9e6be..bee36c5903 100644 --- a/lib/vhost/iotlb.h +++ b/lib/vhost/iotlb.h @@ -58,7 +58,7 @@ vhost_user_iotlb_wr_unlock_all(struct virtio_net *dev) } void vhost_user_iotlb_cache_insert(struct virtio_net *dev, uint64_t iova, uint64_t uaddr, - uint64_t size, uint8_t perm); + uint64_t uoffset, uint64_t size, uint8_t perm); void vhost_user_iotlb_cache_remove(struct virtio_net *dev, uint64_t iova, uint64_t size); uint64_t vhost_user_iotlb_cache_find(struct virtio_net *dev, uint64_t iova, uint64_t *size, uint8_t perm); diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index f2fe7ebc93..7f88a8754f 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -2641,7 +2641,7 @@ vhost_user_iotlb_msg(struct virtio_net **pdev, if (!vva) return RTE_VHOST_MSG_RESULT_ERR; - vhost_user_iotlb_cache_insert(dev, imsg->iova, vva, len, imsg->perm); + vhost_user_iotlb_cache_insert(dev, imsg->iova, vva, 0, len, imsg->perm); for (i = 0; i < dev->nr_vring; i++) { struct vhost_virtqueue *vq = dev->virtqueue[i]; From patchwork Thu Jun 1 20:07:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127915 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6F2D242C08; Thu, 1 Jun 2023 22:09:08 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 44A2E42DA0; Thu, 1 Jun 2023 22:08:46 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id D18CD42D53 for ; Thu, 1 Jun 2023 22:08:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650123; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sGEJh1nMhfTRX7+OJKRQZyCAdFbDueQwqgvaVGPX80o=; b=OtgGQoZcyrRKashSwnmaUOGnRRumXkiFZ0OpWNs8ydSV1oP+m4Wcs8kC9fpymH22hq8L9P T0YQYHytU/DI/MUO1BQb3PjwYVukSnqanclnR295NcexwnJIF1ecqVxzkII3X8LvnsTx6h o/T1x0y1js+e8xO6Mt3tXDJSNbxL3JE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-427-ngUh9MP5MwOjwTPae5c1pQ-1; Thu, 01 Jun 2023 16:08:40 -0400 X-MC-Unique: ngUh9MP5MwOjwTPae5c1pQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 00F78802E58; Thu, 1 Jun 2023 20:08:40 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5F167202696C; Thu, 1 Jun 2023 20:08:37 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v4 07/26] vhost: add page size info to IOTLB entry Date: Thu, 1 Jun 2023 22:07:53 +0200 Message-Id: <20230601200812.672233-8-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org VDUSE will close the file descriptor after having mapped the shared memory, so it will not be possible to get the page size afterwards. This patch adds an new page_shift field to the IOTLB entry, so that the information will be passed at IOTLB cache insertion time. The information is stored as a bit shift value so that IOTLB entry keeps fitting in a single cacheline. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/iotlb.c | 46 ++++++++++++++++++++---------------------- lib/vhost/iotlb.h | 2 +- lib/vhost/vhost.h | 1 - lib/vhost/vhost_user.c | 8 +++++--- 4 files changed, 28 insertions(+), 29 deletions(-) diff --git a/lib/vhost/iotlb.c b/lib/vhost/iotlb.c index aa5100e6e7..87986f2489 100644 --- a/lib/vhost/iotlb.c +++ b/lib/vhost/iotlb.c @@ -19,14 +19,14 @@ struct vhost_iotlb_entry { uint64_t uaddr; uint64_t uoffset; uint64_t size; + uint8_t page_shift; uint8_t perm; }; #define IOTLB_CACHE_SIZE 2048 static bool -vhost_user_iotlb_share_page(struct vhost_iotlb_entry *a, struct vhost_iotlb_entry *b, - uint64_t align) +vhost_user_iotlb_share_page(struct vhost_iotlb_entry *a, struct vhost_iotlb_entry *b) { uint64_t a_start, a_end, b_start; @@ -38,44 +38,41 @@ vhost_user_iotlb_share_page(struct vhost_iotlb_entry *a, struct vhost_iotlb_entr /* Assumes entry a lower than entry b */ RTE_ASSERT(a_start < b_start); - a_end = RTE_ALIGN_CEIL(a_start + a->size, align); - b_start = RTE_ALIGN_FLOOR(b_start, align); + a_end = RTE_ALIGN_CEIL(a_start + a->size, RTE_BIT64(a->page_shift)); + b_start = RTE_ALIGN_FLOOR(b_start, RTE_BIT64(b->page_shift)); return a_end > b_start; } static void -vhost_user_iotlb_set_dump(struct virtio_net *dev, struct vhost_iotlb_entry *node) +vhost_user_iotlb_set_dump(struct vhost_iotlb_entry *node) { - uint64_t align, start; + uint64_t start; start = node->uaddr + node->uoffset; - align = hua_to_alignment(dev->mem, (void *)(uintptr_t)start); - - mem_set_dump((void *)(uintptr_t)start, node->size, true, align); + mem_set_dump((void *)(uintptr_t)start, node->size, true, RTE_BIT64(node->page_shift)); } static void -vhost_user_iotlb_clear_dump(struct virtio_net *dev, struct vhost_iotlb_entry *node, +vhost_user_iotlb_clear_dump(struct vhost_iotlb_entry *node, struct vhost_iotlb_entry *prev, struct vhost_iotlb_entry *next) { - uint64_t align, start, end; + uint64_t start, end; start = node->uaddr + node->uoffset; end = start + node->size; - align = hua_to_alignment(dev->mem, (void *)(uintptr_t)start); - /* Skip first page if shared with previous entry. */ - if (vhost_user_iotlb_share_page(prev, node, align)) - start = RTE_ALIGN_CEIL(start, align); + if (vhost_user_iotlb_share_page(prev, node)) + start = RTE_ALIGN_CEIL(start, RTE_BIT64(node->page_shift)); /* Skip last page if shared with next entry. */ - if (vhost_user_iotlb_share_page(node, next, align)) - end = RTE_ALIGN_FLOOR(end, align); + if (vhost_user_iotlb_share_page(node, next)) + end = RTE_ALIGN_FLOOR(end, RTE_BIT64(node->page_shift)); if (end > start) - mem_set_dump((void *)(uintptr_t)start, end - start, false, align); + mem_set_dump((void *)(uintptr_t)start, end - start, false, + RTE_BIT64(node->page_shift)); } static struct vhost_iotlb_entry * @@ -198,7 +195,7 @@ vhost_user_iotlb_cache_remove_all(struct virtio_net *dev) vhost_user_iotlb_wr_lock_all(dev); RTE_TAILQ_FOREACH_SAFE(node, &dev->iotlb_list, next, temp_node) { - vhost_user_iotlb_clear_dump(dev, node, NULL, NULL); + vhost_user_iotlb_clear_dump(node, NULL, NULL); TAILQ_REMOVE(&dev->iotlb_list, node, next); vhost_user_iotlb_pool_put(dev, node); @@ -223,7 +220,7 @@ vhost_user_iotlb_cache_random_evict(struct virtio_net *dev) if (!entry_idx) { struct vhost_iotlb_entry *next_node = RTE_TAILQ_NEXT(node, next); - vhost_user_iotlb_clear_dump(dev, node, prev_node, next_node); + vhost_user_iotlb_clear_dump(node, prev_node, next_node); TAILQ_REMOVE(&dev->iotlb_list, node, next); vhost_user_iotlb_pool_put(dev, node); @@ -239,7 +236,7 @@ vhost_user_iotlb_cache_random_evict(struct virtio_net *dev) void vhost_user_iotlb_cache_insert(struct virtio_net *dev, uint64_t iova, uint64_t uaddr, - uint64_t uoffset, uint64_t size, uint8_t perm) + uint64_t uoffset, uint64_t size, uint64_t page_size, uint8_t perm) { struct vhost_iotlb_entry *node, *new_node; @@ -263,6 +260,7 @@ vhost_user_iotlb_cache_insert(struct virtio_net *dev, uint64_t iova, uint64_t ua new_node->uaddr = uaddr; new_node->uoffset = uoffset; new_node->size = size; + new_node->page_shift = __builtin_ctzll(page_size); new_node->perm = perm; vhost_user_iotlb_wr_lock_all(dev); @@ -276,7 +274,7 @@ vhost_user_iotlb_cache_insert(struct virtio_net *dev, uint64_t iova, uint64_t ua vhost_user_iotlb_pool_put(dev, new_node); goto unlock; } else if (node->iova > new_node->iova) { - vhost_user_iotlb_set_dump(dev, new_node); + vhost_user_iotlb_set_dump(new_node); TAILQ_INSERT_BEFORE(node, new_node, next); dev->iotlb_cache_nr++; @@ -284,7 +282,7 @@ vhost_user_iotlb_cache_insert(struct virtio_net *dev, uint64_t iova, uint64_t ua } } - vhost_user_iotlb_set_dump(dev, new_node); + vhost_user_iotlb_set_dump(new_node); TAILQ_INSERT_TAIL(&dev->iotlb_list, new_node, next); dev->iotlb_cache_nr++; @@ -313,7 +311,7 @@ vhost_user_iotlb_cache_remove(struct virtio_net *dev, uint64_t iova, uint64_t si if (iova < node->iova + node->size) { struct vhost_iotlb_entry *next_node = RTE_TAILQ_NEXT(node, next); - vhost_user_iotlb_clear_dump(dev, node, prev_node, next_node); + vhost_user_iotlb_clear_dump(node, prev_node, next_node); TAILQ_REMOVE(&dev->iotlb_list, node, next); vhost_user_iotlb_pool_put(dev, node); diff --git a/lib/vhost/iotlb.h b/lib/vhost/iotlb.h index bee36c5903..81ca04df21 100644 --- a/lib/vhost/iotlb.h +++ b/lib/vhost/iotlb.h @@ -58,7 +58,7 @@ vhost_user_iotlb_wr_unlock_all(struct virtio_net *dev) } void vhost_user_iotlb_cache_insert(struct virtio_net *dev, uint64_t iova, uint64_t uaddr, - uint64_t uoffset, uint64_t size, uint8_t perm); + uint64_t uoffset, uint64_t size, uint64_t page_size, uint8_t perm); void vhost_user_iotlb_cache_remove(struct virtio_net *dev, uint64_t iova, uint64_t size); uint64_t vhost_user_iotlb_cache_find(struct virtio_net *dev, uint64_t iova, uint64_t *size, uint8_t perm); diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h index ee952de175..de84c115b7 100644 --- a/lib/vhost/vhost.h +++ b/lib/vhost/vhost.h @@ -1032,7 +1032,6 @@ mbuf_is_consumed(struct rte_mbuf *m) return true; } -uint64_t hua_to_alignment(struct rte_vhost_memory *mem, void *ptr); void mem_set_dump(void *ptr, size_t size, bool enable, uint64_t alignment); /* Versioned functions */ diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index 7f88a8754f..98d8b8ac79 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -743,7 +743,7 @@ log_addr_to_gpa(struct virtio_net *dev, struct vhost_virtqueue *vq) return log_gpa; } -uint64_t +static uint64_t hua_to_alignment(struct rte_vhost_memory *mem, void *ptr) { struct rte_vhost_mem_region *r; @@ -2632,7 +2632,7 @@ vhost_user_iotlb_msg(struct virtio_net **pdev, struct virtio_net *dev = *pdev; struct vhost_iotlb_msg *imsg = &ctx->msg.payload.iotlb; uint16_t i; - uint64_t vva, len; + uint64_t vva, len, pg_sz; switch (imsg->type) { case VHOST_IOTLB_UPDATE: @@ -2641,7 +2641,9 @@ vhost_user_iotlb_msg(struct virtio_net **pdev, if (!vva) return RTE_VHOST_MSG_RESULT_ERR; - vhost_user_iotlb_cache_insert(dev, imsg->iova, vva, 0, len, imsg->perm); + pg_sz = hua_to_alignment(dev->mem, (void *)(uintptr_t)vva); + + vhost_user_iotlb_cache_insert(dev, imsg->iova, vva, 0, len, pg_sz, imsg->perm); for (i = 0; i < dev->nr_vring; i++) { struct vhost_virtqueue *vq = dev->virtqueue[i]; From patchwork Thu Jun 1 20:07:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127916 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5E66A42C08; Thu, 1 Jun 2023 22:09:15 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 51CF742DAB; Thu, 1 Jun 2023 22:08:50 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 0546642DAB for ; Thu, 1 Jun 2023 22:08:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650127; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YVqmLHMAAAMHCW9ouzdoXrU9qsZSPl5hdtEviSt13uc=; b=adFmiTZ5l2+5TYz+aSpJxTsNU1PTvuhIZ72JhjqzidU6XuFkVrVaxExyIPe0YSxl6jmYP3 LJ0zQBNr7raimk8Qgvb4B5sD1dw5zw3XLcAPrWKLDF58gSvbsrhl+vq0SplyeCKrs3INuU 9PlVqkpaOOb1Bm4PDxfWofvCaxweScE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-396-aYb8ss6nPLeFvkrCZvaaHg-1; Thu, 01 Jun 2023 16:08:43 -0400 X-MC-Unique: aYb8ss6nPLeFvkrCZvaaHg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E81DC800B2A; Thu, 1 Jun 2023 20:08:42 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4A38D202696C; Thu, 1 Jun 2023 20:08:40 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v4 08/26] vhost: retry translating IOVA after IOTLB miss Date: Thu, 1 Jun 2023 22:07:54 +0200 Message-Id: <20230601200812.672233-9-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Vhost-user backend IOTLB misses and updates are asynchronous, so IOVA address translation function just fails after having sent an IOTLB miss update if needed entry was not in the IOTLB cache. This is not the case for VDUSE, for which the needed IOTLB update is returned directly when sending an IOTLB miss. This patch retry again finding the needed entry in the IOTLB cache after having sent an IOTLB miss. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vhost.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/lib/vhost/vhost.c b/lib/vhost/vhost.c index 3ddd2a963f..7e1af487c1 100644 --- a/lib/vhost/vhost.c +++ b/lib/vhost/vhost.c @@ -100,6 +100,12 @@ __vhost_iova_to_vva(struct virtio_net *dev, struct vhost_virtqueue *vq, vhost_user_iotlb_rd_lock(vq); } + tmp_size = *size; + /* Retry in case of VDUSE, as it is synchronous */ + vva = vhost_user_iotlb_cache_find(dev, iova, &tmp_size, perm); + if (tmp_size == *size) + return vva; + return 0; } From patchwork Thu Jun 1 20:07:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127934 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 342E642C08; Thu, 1 Jun 2023 22:11:36 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 240C242D61; Thu, 1 Jun 2023 22:11:36 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 54AB842D61 for ; Thu, 1 Jun 2023 22:11:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650293; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RAJbhtRtDIaak0TIE345YBErRVtJxh6iqZrYiMJd/lE=; b=G/TSSAXQSpmq0/wHC/g2s4/X7RMIOJ069o2xp0zsFe1hVpTHWVgo/kIhmHE97GS8ic8wrn Jsgs6RX49jSnn+meVo4+UmnBmL4t9VptElkCXeZ+y4EHf9EKEg1LlJQlvlcka34J9P6tX2 r3kjrWZR6qZu04wWx4x94mChHYKhWb8= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-497-OfgzEwKwO92LcWqFhhwh2w-1; Thu, 01 Jun 2023 16:08:46 -0400 X-MC-Unique: OfgzEwKwO92LcWqFhhwh2w-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D72123806634; Thu, 1 Jun 2023 20:08:45 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 45ED0202696C; Thu, 1 Jun 2023 20:08:43 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v4 09/26] vhost: introduce backend ops Date: Thu, 1 Jun 2023 22:07:55 +0200 Message-Id: <20230601200812.672233-10-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch introduces backend ops struct, that will enable calling backend specifics callbacks (Vhost-user, VDUSE), in shared code. This is an empty shell for now, it will be filled in later patches. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/socket.c | 2 +- lib/vhost/vhost.c | 8 +++++++- lib/vhost/vhost.h | 10 +++++++++- lib/vhost/vhost_user.c | 8 ++++++++ lib/vhost/vhost_user.h | 1 + 5 files changed, 26 insertions(+), 3 deletions(-) diff --git a/lib/vhost/socket.c b/lib/vhost/socket.c index cb0218b7bc..407d0011c3 100644 --- a/lib/vhost/socket.c +++ b/lib/vhost/socket.c @@ -223,7 +223,7 @@ vhost_user_add_connection(int fd, struct vhost_user_socket *vsocket) return; } - vid = vhost_new_device(); + vid = vhost_user_new_device(); if (vid == -1) { goto err; } diff --git a/lib/vhost/vhost.c b/lib/vhost/vhost.c index 7e1af487c1..d054772bf8 100644 --- a/lib/vhost/vhost.c +++ b/lib/vhost/vhost.c @@ -680,11 +680,16 @@ reset_device(struct virtio_net *dev) * there is a new virtio device being attached). */ int -vhost_new_device(void) +vhost_new_device(struct vhost_backend_ops *ops) { struct virtio_net *dev; int i; + if (ops == NULL) { + VHOST_LOG_CONFIG("device", ERR, "missing backend ops.\n"); + return -1; + } + pthread_mutex_lock(&vhost_dev_lock); for (i = 0; i < RTE_MAX_VHOST_DEVICE; i++) { if (vhost_devices[i] == NULL) @@ -712,6 +717,7 @@ vhost_new_device(void) dev->backend_req_fd = -1; dev->postcopy_ufd = -1; rte_spinlock_init(&dev->backend_req_lock); + dev->backend_ops = ops; return i; } diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h index de84c115b7..966b6dd67a 100644 --- a/lib/vhost/vhost.h +++ b/lib/vhost/vhost.h @@ -89,6 +89,12 @@ for (iter = val; iter < num; iter++) #endif +/** + * Structure that contains backend-specific ops. + */ +struct vhost_backend_ops { +}; + /** * Structure contains buffer address, length and descriptor index * from vring to do scatter RX. @@ -516,6 +522,8 @@ struct virtio_net { void *extern_data; /* pre and post vhost user message handlers for the device */ struct rte_vhost_user_extern_ops extern_ops; + + struct vhost_backend_ops *backend_ops; } __rte_cache_aligned; static inline void @@ -815,7 +823,7 @@ get_device(int vid) return dev; } -int vhost_new_device(void); +int vhost_new_device(struct vhost_backend_ops *ops); void cleanup_device(struct virtio_net *dev, int destroy); void reset_device(struct virtio_net *dev); void vhost_destroy_device(int); diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index 98d8b8ac79..2cd86686a5 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -3464,3 +3464,11 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) return ret; } + +static struct vhost_backend_ops vhost_user_backend_ops; + +int +vhost_user_new_device(void) +{ + return vhost_new_device(&vhost_user_backend_ops); +} diff --git a/lib/vhost/vhost_user.h b/lib/vhost/vhost_user.h index a0987a58f9..61456049c8 100644 --- a/lib/vhost/vhost_user.h +++ b/lib/vhost/vhost_user.h @@ -185,5 +185,6 @@ int vhost_user_iotlb_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm); int read_fd_message(char *ifname, int sockfd, char *buf, int buflen, int *fds, int max_fds, int *fd_num); int send_fd_message(char *ifname, int sockfd, char *buf, int buflen, int *fds, int fd_num); +int vhost_user_new_device(void); #endif From patchwork Thu Jun 1 20:07:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127917 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9374242C08; Thu, 1 Jun 2023 22:09:21 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5B4F542DB3; Thu, 1 Jun 2023 22:08:53 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 11D3442DB3 for ; Thu, 1 Jun 2023 22:08:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650130; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=E1CAwqEqZ0aCaZX14KXIsKgiIikY251g0neJCz5ALdQ=; b=XCMbRVvNOXlO5BedHVXPjDz8ve2BVlIX9/ojANsBB0A1Uc0xA3oHdbgUr4NGI2x02kDywJ 74LvtxHQvHgGFp7wofhJba95hFrceaWwyAkfIqOktO1jJRGEGwRaNe/Qt3lRpeqh2aVbaB 3ub+sYflnzsePL6Q5sjmA6OGQlty3fA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-275-J3EyKKZmPQG1zTcBV2SItw-1; Thu, 01 Jun 2023 16:08:49 -0400 X-MC-Unique: J3EyKKZmPQG1zTcBV2SItw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DC672185A78F; Thu, 1 Jun 2023 20:08:48 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2AE442029F6D; Thu, 1 Jun 2023 20:08:46 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v4 10/26] vhost: add IOTLB cache entry removal callback Date: Thu, 1 Jun 2023 22:07:56 +0200 Message-Id: <20230601200812.672233-11-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org VDUSE will need to munmap() the IOTLB entry on removal from the cache, as it performs mmap() before insertion. This patch introduces a callback that VDUSE layer will implement to achieve this. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/iotlb.c | 12 ++++++++++++ lib/vhost/vhost.h | 3 +++ 2 files changed, 15 insertions(+) diff --git a/lib/vhost/iotlb.c b/lib/vhost/iotlb.c index 87986f2489..424121cc00 100644 --- a/lib/vhost/iotlb.c +++ b/lib/vhost/iotlb.c @@ -25,6 +25,15 @@ struct vhost_iotlb_entry { #define IOTLB_CACHE_SIZE 2048 +static void +vhost_user_iotlb_remove_notify(struct virtio_net *dev, struct vhost_iotlb_entry *entry) +{ + if (dev->backend_ops->iotlb_remove_notify == NULL) + return; + + dev->backend_ops->iotlb_remove_notify(entry->uaddr, entry->uoffset, entry->size); +} + static bool vhost_user_iotlb_share_page(struct vhost_iotlb_entry *a, struct vhost_iotlb_entry *b) { @@ -198,6 +207,7 @@ vhost_user_iotlb_cache_remove_all(struct virtio_net *dev) vhost_user_iotlb_clear_dump(node, NULL, NULL); TAILQ_REMOVE(&dev->iotlb_list, node, next); + vhost_user_iotlb_remove_notify(dev, node); vhost_user_iotlb_pool_put(dev, node); } @@ -223,6 +233,7 @@ vhost_user_iotlb_cache_random_evict(struct virtio_net *dev) vhost_user_iotlb_clear_dump(node, prev_node, next_node); TAILQ_REMOVE(&dev->iotlb_list, node, next); + vhost_user_iotlb_remove_notify(dev, node); vhost_user_iotlb_pool_put(dev, node); dev->iotlb_cache_nr--; break; @@ -314,6 +325,7 @@ vhost_user_iotlb_cache_remove(struct virtio_net *dev, uint64_t iova, uint64_t si vhost_user_iotlb_clear_dump(node, prev_node, next_node); TAILQ_REMOVE(&dev->iotlb_list, node, next); + vhost_user_iotlb_remove_notify(dev, node); vhost_user_iotlb_pool_put(dev, node); dev->iotlb_cache_nr--; } else { diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h index 966b6dd67a..69df8b14c0 100644 --- a/lib/vhost/vhost.h +++ b/lib/vhost/vhost.h @@ -89,10 +89,13 @@ for (iter = val; iter < num; iter++) #endif +typedef void (*vhost_iotlb_remove_notify)(uint64_t addr, uint64_t off, uint64_t size); + /** * Structure that contains backend-specific ops. */ struct vhost_backend_ops { + vhost_iotlb_remove_notify iotlb_remove_notify; }; /** From patchwork Thu Jun 1 20:07:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127918 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4F91B42C08; Thu, 1 Jun 2023 22:09:29 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 751FC42DBC; Thu, 1 Jun 2023 22:08:54 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 9E6C242DBC for ; Thu, 1 Jun 2023 22:08:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650133; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MzQdgogwAu1MPoz6zTxH904RRXYwVsGzzVYeHbwQb+8=; b=AFdT++QwAA3RC2jRB5sUlhA6EmKR3kcOU95p2YJ/0G2c+SBLzCzY8E1odt2yHgSYvTtokW MAghN+nAiGZMW4a6r5tOxUo0oUSKSeaFwi+Kl0Hg4YLlPvfHniHxpB6Pbox5ABCwja6VoI wa4+8qwzhtMuWIcx5npZFEuIojTtmu8= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-671-TYLXni0kNRee5sR_M_MWAg-1; Thu, 01 Jun 2023 16:08:52 -0400 X-MC-Unique: TYLXni0kNRee5sR_M_MWAg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C2A752800C2B; Thu, 1 Jun 2023 20:08:51 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2EF3A202696C; Thu, 1 Jun 2023 20:08:49 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v4 11/26] vhost: add helper for IOTLB misses Date: Thu, 1 Jun 2023 22:07:57 +0200 Message-Id: <20230601200812.672233-12-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds a helper for sending IOTLB misses as VDUSE will use an ioctl while Vhost-user use a dedicated Vhost-user backend request. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vhost.c | 13 ++++++++++++- lib/vhost/vhost.h | 4 ++++ lib/vhost/vhost_user.c | 6 ++++-- lib/vhost/vhost_user.h | 1 - 4 files changed, 20 insertions(+), 4 deletions(-) diff --git a/lib/vhost/vhost.c b/lib/vhost/vhost.c index d054772bf8..f77f30d674 100644 --- a/lib/vhost/vhost.c +++ b/lib/vhost/vhost.c @@ -56,6 +56,12 @@ static const struct vhost_vq_stats_name_off vhost_vq_stat_strings[] = { #define VHOST_NB_VQ_STATS RTE_DIM(vhost_vq_stat_strings) +static int +vhost_iotlb_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm) +{ + return dev->backend_ops->iotlb_miss(dev, iova, perm); +} + uint64_t __vhost_iova_to_vva(struct virtio_net *dev, struct vhost_virtqueue *vq, uint64_t iova, uint64_t *size, uint8_t perm) @@ -90,7 +96,7 @@ __vhost_iova_to_vva(struct virtio_net *dev, struct vhost_virtqueue *vq, vhost_user_iotlb_rd_unlock(vq); vhost_user_iotlb_pending_insert(dev, iova, perm); - if (vhost_user_iotlb_miss(dev, iova, perm)) { + if (vhost_iotlb_miss(dev, iova, perm)) { VHOST_LOG_DATA(dev->ifname, ERR, "IOTLB miss req failed for IOVA 0x%" PRIx64 "\n", iova); @@ -690,6 +696,11 @@ vhost_new_device(struct vhost_backend_ops *ops) return -1; } + if (ops->iotlb_miss == NULL) { + VHOST_LOG_CONFIG("device", ERR, "missing IOTLB miss backend op.\n"); + return -1; + } + pthread_mutex_lock(&vhost_dev_lock); for (i = 0; i < RTE_MAX_VHOST_DEVICE; i++) { if (vhost_devices[i] == NULL) diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h index 69df8b14c0..25255aaea8 100644 --- a/lib/vhost/vhost.h +++ b/lib/vhost/vhost.h @@ -89,13 +89,17 @@ for (iter = val; iter < num; iter++) #endif +struct virtio_net; typedef void (*vhost_iotlb_remove_notify)(uint64_t addr, uint64_t off, uint64_t size); +typedef int (*vhost_iotlb_miss_cb)(struct virtio_net *dev, uint64_t iova, uint8_t perm); + /** * Structure that contains backend-specific ops. */ struct vhost_backend_ops { vhost_iotlb_remove_notify iotlb_remove_notify; + vhost_iotlb_miss_cb iotlb_miss; }; /** diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index 2cd86686a5..30ad63aba0 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -3305,7 +3305,7 @@ vhost_user_msg_handler(int vid, int fd) return ret; } -int +static int vhost_user_iotlb_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm) { int ret; @@ -3465,7 +3465,9 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) return ret; } -static struct vhost_backend_ops vhost_user_backend_ops; +static struct vhost_backend_ops vhost_user_backend_ops = { + .iotlb_miss = vhost_user_iotlb_miss, +}; int vhost_user_new_device(void) diff --git a/lib/vhost/vhost_user.h b/lib/vhost/vhost_user.h index 61456049c8..1ffeca92f3 100644 --- a/lib/vhost/vhost_user.h +++ b/lib/vhost/vhost_user.h @@ -179,7 +179,6 @@ struct __rte_packed vhu_msg_context { /* vhost_user.c */ int vhost_user_msg_handler(int vid, int fd); -int vhost_user_iotlb_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm); /* socket.c */ int read_fd_message(char *ifname, int sockfd, char *buf, int buflen, int *fds, int max_fds, From patchwork Thu Jun 1 20:07:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127919 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AD97842C08; Thu, 1 Jun 2023 22:09:38 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C63AA42D3D; Thu, 1 Jun 2023 22:08:57 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id E38A042B8C for ; Thu, 1 Jun 2023 22:08:56 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650136; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BOFgfKVmUk5DBDxg658RC36571AMByiB0QftAZCajvA=; b=JSNXQ58I663L7KgZ9QADZiL+BhYAJs5OvqvA5iSbD7BWP4hW7xqXwI5PiZEYcbT2rjAklo xbnOoQiaO/wlafr0+UyjQiuyTciK7A+qBLr8VNC4XXDDprUWC+TG8ocEy9sxvX7/zgSmIE YVbHVCHacxk5n0k5zAU5v6FsKXBiVdU= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-638-HTbyjlyiN4eaWdnhBRaXSw-1; Thu, 01 Jun 2023 16:08:55 -0400 X-MC-Unique: HTbyjlyiN4eaWdnhBRaXSw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C6650800159; Thu, 1 Jun 2023 20:08:54 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 15A882029F6E; Thu, 1 Jun 2023 20:08:51 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v4 12/26] vhost: add helper for interrupt injection Date: Thu, 1 Jun 2023 22:07:58 +0200 Message-Id: <20230601200812.672233-13-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Vhost-user uses eventfd to inject IRQs, but VDUSE uses an ioctl. This patch prepares vhost_vring_call_split() and vhost_vring_call_packed() to support VDUSE by introducing a new helper. It also adds a new counter for guest notification failures, which could happen in case of uninitialized call file descriptor for example. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vhost.c | 25 +++++++++++++------------ lib/vhost/vhost.h | 19 +++++++++---------- lib/vhost/vhost_user.c | 10 ++++++++++ 3 files changed, 32 insertions(+), 22 deletions(-) diff --git a/lib/vhost/vhost.c b/lib/vhost/vhost.c index f77f30d674..eb6309b681 100644 --- a/lib/vhost/vhost.c +++ b/lib/vhost/vhost.c @@ -701,6 +701,11 @@ vhost_new_device(struct vhost_backend_ops *ops) return -1; } + if (ops->inject_irq == NULL) { + VHOST_LOG_CONFIG("device", ERR, "missing IRQ injection backend op.\n"); + return -1; + } + pthread_mutex_lock(&vhost_dev_lock); for (i = 0; i < RTE_MAX_VHOST_DEVICE; i++) { if (vhost_devices[i] == NULL) @@ -1511,20 +1516,16 @@ rte_vhost_notify_guest(int vid, uint16_t queue_id) rte_rwlock_read_lock(&vq->access_lock); - if (vq->callfd >= 0) { - int ret = eventfd_write(vq->callfd, (eventfd_t)1); - - if (ret) { - if (dev->flags & VIRTIO_DEV_STATS_ENABLED) - __atomic_fetch_add(&vq->stats.guest_notifications_error, + if (dev->backend_ops->inject_irq(dev, vq)) { + if (dev->flags & VIRTIO_DEV_STATS_ENABLED) + __atomic_fetch_add(&vq->stats.guest_notifications_error, 1, __ATOMIC_RELAXED); - } else { - if (dev->flags & VIRTIO_DEV_STATS_ENABLED) - __atomic_fetch_add(&vq->stats.guest_notifications, + } else { + if (dev->flags & VIRTIO_DEV_STATS_ENABLED) + __atomic_fetch_add(&vq->stats.guest_notifications, 1, __ATOMIC_RELAXED); - if (dev->notify_ops->guest_notified) - dev->notify_ops->guest_notified(dev->vid); - } + if (dev->notify_ops->guest_notified) + dev->notify_ops->guest_notified(dev->vid); } rte_rwlock_read_unlock(&vq->access_lock); diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h index 25255aaea8..ea2798b0bf 100644 --- a/lib/vhost/vhost.h +++ b/lib/vhost/vhost.h @@ -90,16 +90,20 @@ #endif struct virtio_net; +struct vhost_virtqueue; + typedef void (*vhost_iotlb_remove_notify)(uint64_t addr, uint64_t off, uint64_t size); typedef int (*vhost_iotlb_miss_cb)(struct virtio_net *dev, uint64_t iova, uint8_t perm); +typedef int (*vhost_vring_inject_irq_cb)(struct virtio_net *dev, struct vhost_virtqueue *vq); /** * Structure that contains backend-specific ops. */ struct vhost_backend_ops { vhost_iotlb_remove_notify iotlb_remove_notify; vhost_iotlb_miss_cb iotlb_miss; + vhost_vring_inject_irq_cb inject_irq; }; /** @@ -906,8 +910,6 @@ vhost_need_event(uint16_t event_idx, uint16_t new_idx, uint16_t old) static __rte_always_inline void vhost_vring_inject_irq(struct virtio_net *dev, struct vhost_virtqueue *vq) { - int ret; - if (dev->notify_ops->guest_notify && dev->notify_ops->guest_notify(dev->vid, vq->index)) { if (dev->flags & VIRTIO_DEV_STATS_ENABLED) @@ -916,8 +918,7 @@ vhost_vring_inject_irq(struct virtio_net *dev, struct vhost_virtqueue *vq) return; } - ret = eventfd_write(vq->callfd, (eventfd_t) 1); - if (ret) { + if (dev->backend_ops->inject_irq(dev, vq)) { if (dev->flags & VIRTIO_DEV_STATS_ENABLED) __atomic_fetch_add(&vq->stats.guest_notifications_error, 1, __ATOMIC_RELAXED); @@ -950,14 +951,12 @@ vhost_vring_call_split(struct virtio_net *dev, struct vhost_virtqueue *vq) "%s: used_event_idx=%d, old=%d, new=%d\n", __func__, vhost_used_event(vq), old, new); - if ((vhost_need_event(vhost_used_event(vq), new, old) || - unlikely(!signalled_used_valid)) && - vq->callfd >= 0) + if (vhost_need_event(vhost_used_event(vq), new, old) || + unlikely(!signalled_used_valid)) vhost_vring_inject_irq(dev, vq); } else { /* Kick the guest if necessary. */ - if (!(vq->avail->flags & VRING_AVAIL_F_NO_INTERRUPT) && - (vq->callfd >= 0)) + if (!(vq->avail->flags & VRING_AVAIL_F_NO_INTERRUPT)) vhost_vring_inject_irq(dev, vq); } } @@ -1009,7 +1008,7 @@ vhost_vring_call_packed(struct virtio_net *dev, struct vhost_virtqueue *vq) if (vhost_need_event(off, new, old)) kick = true; kick: - if (kick && vq->callfd >= 0) + if (kick) vhost_vring_inject_irq(dev, vq); } diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index 30ad63aba0..901a80bbaa 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -3465,8 +3465,18 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) return ret; } +static int +vhost_user_inject_irq(struct virtio_net *dev __rte_unused, struct vhost_virtqueue *vq) +{ + if (vq->callfd < 0) + return -1; + + return eventfd_write(vq->callfd, (eventfd_t)1); +} + static struct vhost_backend_ops vhost_user_backend_ops = { .iotlb_miss = vhost_user_iotlb_miss, + .inject_irq = vhost_user_inject_irq, }; int From patchwork Thu Jun 1 20:07:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127920 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A968F42C08; Thu, 1 Jun 2023 22:09:44 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E68E142D35; Thu, 1 Jun 2023 22:09:02 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 7C50D42D13 for ; Thu, 1 Jun 2023 22:09:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650141; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=efX2tscuvNWnUeV8epCwn18SwAwA5eHT1MXkbWj5H6c=; b=jL5FkhreU93uFuvR2VO13eEI18ZGtVFnkrHL0ZukqCEDgC9or/w9OyWtAUQu5XaOkSzTKm 0T1RZVnDy3DdSjOEY2O//yBCYTYlDNyuCp2cDPZroNMx7uxtSgMubZS7yGuolZDvjuIk9f 9Pro6Y4Y3Sor7xZCoZwhO/poOF2qYpU= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-523-tWF5Msa6ORm2MdSWcMyZDg-1; Thu, 01 Jun 2023 16:08:58 -0400 X-MC-Unique: tWF5Msa6ORm2MdSWcMyZDg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A74CA85A5AA; Thu, 1 Jun 2023 20:08:57 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1A9D0200AE6F; Thu, 1 Jun 2023 20:08:54 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v4 13/26] vhost: add API to set max queue pairs Date: Thu, 1 Jun 2023 22:07:59 +0200 Message-Id: <20230601200812.672233-14-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch introduces a new rte_vhost_driver_set_max_queues API as preliminary work for multiqueue support with VDUSE. Indeed, with VDUSE we need to pre-allocate the vrings at device creation time, so we need such API not to allocate the 128 queue pairs supported by the Vhost library. Calling the API is optional, 128 queue pairs remaining the default. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- doc/guides/prog_guide/vhost_lib.rst | 4 +++ doc/guides/rel_notes/release_23_07.rst | 5 ++++ lib/vhost/rte_vhost.h | 17 ++++++++++++ lib/vhost/socket.c | 36 ++++++++++++++++++++++++-- lib/vhost/version.map | 1 + 5 files changed, 61 insertions(+), 2 deletions(-) diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst index 0f964d7a4a..0c2b4d020a 100644 --- a/doc/guides/prog_guide/vhost_lib.rst +++ b/doc/guides/prog_guide/vhost_lib.rst @@ -339,6 +339,10 @@ The following is an overview of some key Vhost API functions: Inject the offloaded interrupt received by the 'guest_notify' callback, into the vhost device's queue. +* ``rte_vhost_driver_set_max_queue_num(const char *path, uint32_t max_queue_pairs)`` + + Set the maximum number of queue pairs supported by the device. + Vhost-user Implementations -------------------------- diff --git a/doc/guides/rel_notes/release_23_07.rst b/doc/guides/rel_notes/release_23_07.rst index 3eed8ac561..7034fb664c 100644 --- a/doc/guides/rel_notes/release_23_07.rst +++ b/doc/guides/rel_notes/release_23_07.rst @@ -62,6 +62,11 @@ New Features rte_vhost_notify_guest(), is added to raise the interrupt outside of the fast path. +* **Added Vhost API to set maximum queue pairs supported.** + + Introduced ``rte_vhost_driver_set_max_queue_num()`` to be able to limit the + maximum number of supported queue pairs, required for VDUSE support. + Removed Items ------------- diff --git a/lib/vhost/rte_vhost.h b/lib/vhost/rte_vhost.h index 7a10bc36cf..7844c9f142 100644 --- a/lib/vhost/rte_vhost.h +++ b/lib/vhost/rte_vhost.h @@ -609,6 +609,23 @@ rte_vhost_driver_get_protocol_features(const char *path, int rte_vhost_driver_get_queue_num(const char *path, uint32_t *queue_num); +/** + * @warning + * @b EXPERIMENTAL: this API may change, or be removed, without prior notice. + * + * Set the maximum number of queue pairs supported by the device. + * + * @param path + * The vhost-user socket file path + * @param max_queue_pairs + * The maximum number of queue pairs + * @return + * 0 on success, -1 on failure + */ +__rte_experimental +int +rte_vhost_driver_set_max_queue_num(const char *path, uint32_t max_queue_pairs); + /** * Get the feature bits after negotiation * diff --git a/lib/vhost/socket.c b/lib/vhost/socket.c index 407d0011c3..29f7a8cece 100644 --- a/lib/vhost/socket.c +++ b/lib/vhost/socket.c @@ -57,6 +57,8 @@ struct vhost_user_socket { uint64_t protocol_features; + uint32_t max_queue_pairs; + struct rte_vdpa_device *vdpa_dev; struct rte_vhost_device_ops const *notify_ops; @@ -823,7 +825,7 @@ rte_vhost_driver_get_queue_num(const char *path, uint32_t *queue_num) vdpa_dev = vsocket->vdpa_dev; if (!vdpa_dev) { - *queue_num = VHOST_MAX_QUEUE_PAIRS; + *queue_num = vsocket->max_queue_pairs; goto unlock_exit; } @@ -833,7 +835,36 @@ rte_vhost_driver_get_queue_num(const char *path, uint32_t *queue_num) goto unlock_exit; } - *queue_num = RTE_MIN((uint32_t)VHOST_MAX_QUEUE_PAIRS, vdpa_queue_num); + *queue_num = RTE_MIN(vsocket->max_queue_pairs, vdpa_queue_num); + +unlock_exit: + pthread_mutex_unlock(&vhost_user.mutex); + return ret; +} + +int +rte_vhost_driver_set_max_queue_num(const char *path, uint32_t max_queue_pairs) +{ + struct vhost_user_socket *vsocket; + int ret = 0; + + VHOST_LOG_CONFIG(path, INFO, "Setting max queue pairs to %u\n", max_queue_pairs); + + if (max_queue_pairs > VHOST_MAX_QUEUE_PAIRS) { + VHOST_LOG_CONFIG(path, ERR, "Library only supports up to %u queue pairs\n", + VHOST_MAX_QUEUE_PAIRS); + return -1; + } + + pthread_mutex_lock(&vhost_user.mutex); + vsocket = find_vhost_user_socket(path); + if (!vsocket) { + VHOST_LOG_CONFIG(path, ERR, "socket file is not registered yet.\n"); + ret = -1; + goto unlock_exit; + } + + vsocket->max_queue_pairs = max_queue_pairs; unlock_exit: pthread_mutex_unlock(&vhost_user.mutex); @@ -889,6 +920,7 @@ rte_vhost_driver_register(const char *path, uint64_t flags) goto out_free; } vsocket->vdpa_dev = NULL; + vsocket->max_queue_pairs = VHOST_MAX_QUEUE_PAIRS; vsocket->extbuf = flags & RTE_VHOST_USER_EXTBUF_SUPPORT; vsocket->linearbuf = flags & RTE_VHOST_USER_LINEARBUF_SUPPORT; vsocket->async_copy = flags & RTE_VHOST_USER_ASYNC_COPY; diff --git a/lib/vhost/version.map b/lib/vhost/version.map index 7bcbfd12cf..5051c29022 100644 --- a/lib/vhost/version.map +++ b/lib/vhost/version.map @@ -107,6 +107,7 @@ EXPERIMENTAL { # added in 23.07 rte_vhost_notify_guest; + rte_vhost_driver_set_max_queue_num; }; INTERNAL { From patchwork Thu Jun 1 20:08:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127921 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 417D242C08; Thu, 1 Jun 2023 22:09:50 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id EFF4242D70; Thu, 1 Jun 2023 22:09:05 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id B46E142D52 for ; Thu, 1 Jun 2023 22:09:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650144; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HKy6hSXmK5VsHBV/QWhIAznNiUc9xtI8d5uBiU8cXMs=; b=iUT0IU7okB4L+Oa60QRmZxOM2NYoChaSHtC7fHU4xl082iovwbV9F72KxbIKaWMjWpHKz4 JkSThVFabekJCfHyw3oeqGS1WnLLpuprwtfLEgTIfs6GyTvSuxr58VJk2exUdiz+kC9OYL WdH7xlxOnMoAmeE5j8DmK7nySLHVcpA= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-477-aNnYKTY-PfaT_x0U3jRrpg-1; Thu, 01 Jun 2023 16:09:01 -0400 X-MC-Unique: aNnYKTY-PfaT_x0U3jRrpg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9AC7B3C10680; Thu, 1 Jun 2023 20:09:00 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id F01B42029F6D; Thu, 1 Jun 2023 20:08:57 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v4 14/26] net/vhost: use API to set max queue pairs Date: Thu, 1 Jun 2023 22:08:00 +0200 Message-Id: <20230601200812.672233-15-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org In order to support multiqueue with VDUSE, we need to be able to limit the maximum number of queue pairs, to avoid unnecessary memory consumption since the maximum number of queue pairs need to be allocated at device creation time, as opposed to Vhost-user which allocate only when the frontend initialize them. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- drivers/net/vhost/rte_eth_vhost.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c index 62ef955ebc..8d37ec9775 100644 --- a/drivers/net/vhost/rte_eth_vhost.c +++ b/drivers/net/vhost/rte_eth_vhost.c @@ -1013,6 +1013,9 @@ vhost_driver_setup(struct rte_eth_dev *eth_dev) goto drv_unreg; } + if (rte_vhost_driver_set_max_queue_num(internal->iface_name, internal->max_queues)) + goto drv_unreg; + if (rte_vhost_driver_callback_register(internal->iface_name, &vhost_ops) < 0) { VHOST_LOG(ERR, "Can't register callbacks\n"); From patchwork Thu Jun 1 20:08:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127922 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1E8B742C08; Thu, 1 Jun 2023 22:09:56 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0AD4542D52; Thu, 1 Jun 2023 22:09:10 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.145.221.124]) by mails.dpdk.org (Postfix) with ESMTP id 7E2F240EF0 for ; Thu, 1 Jun 2023 22:09:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650148; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YPQYGUKXd3T5G0nzSO7UYxof2jO85vBggzbd+oY0ZrI=; b=CtnlkUCsPQuqT8g/0iw/Lytnk7tEPqXiIINvsDc/MEfx7C868Yu2c0qrMRIsOYl8WqVCXN Aq7220CUmk8l5g2bHoY1Fc+KUXCAiiN/sho4Hfo574P8WvFgys0vBui8ZAYFyEwmcVUVGo F1fTIA1OlDKMreZ/0CySOdn9LTxPmRI= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-2-e6-MpEAUP_y0A1hVxhjFDQ-1; Thu, 01 Jun 2023 16:09:04 -0400 X-MC-Unique: e6-MpEAUP_y0A1hVxhjFDQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A37B9811E78; Thu, 1 Jun 2023 20:09:03 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 107BF200AE6F; Thu, 1 Jun 2023 20:09:00 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v4 15/26] vhost: add control virtqueue support Date: Thu, 1 Jun 2023 22:08:01 +0200 Message-Id: <20230601200812.672233-16-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org In order to support multi-queue with VDUSE, having control queue support is required. This patch adds control queue implementation, it will be used later when adding VDUSE support. Only split ring layout is supported for now, packed ring support will be added later. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/meson.build | 1 + lib/vhost/vhost.h | 2 + lib/vhost/virtio_net_ctrl.c | 286 ++++++++++++++++++++++++++++++++++++ lib/vhost/virtio_net_ctrl.h | 10 ++ 4 files changed, 299 insertions(+) create mode 100644 lib/vhost/virtio_net_ctrl.c create mode 100644 lib/vhost/virtio_net_ctrl.h diff --git a/lib/vhost/meson.build b/lib/vhost/meson.build index 05679447db..d211a0bd37 100644 --- a/lib/vhost/meson.build +++ b/lib/vhost/meson.build @@ -27,6 +27,7 @@ sources = files( 'vhost_crypto.c', 'vhost_user.c', 'virtio_net.c', + 'virtio_net_ctrl.c', ) headers = files( 'rte_vdpa.h', diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h index ea2798b0bf..04267a3ac5 100644 --- a/lib/vhost/vhost.h +++ b/lib/vhost/vhost.h @@ -527,6 +527,8 @@ struct virtio_net { int postcopy_ufd; int postcopy_listening; + struct vhost_virtqueue *cvq; + struct rte_vdpa_device *vdpa_dev; /* context data for the external message handlers */ diff --git a/lib/vhost/virtio_net_ctrl.c b/lib/vhost/virtio_net_ctrl.c new file mode 100644 index 0000000000..6b583a0ce6 --- /dev/null +++ b/lib/vhost/virtio_net_ctrl.c @@ -0,0 +1,286 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Red Hat, Inc. + */ + +#include +#include +#include + +#include "iotlb.h" +#include "vhost.h" +#include "virtio_net_ctrl.h" + +struct virtio_net_ctrl { + uint8_t class; + uint8_t command; + uint8_t command_data[]; +}; + +struct virtio_net_ctrl_elem { + struct virtio_net_ctrl *ctrl_req; + uint16_t head_idx; + uint16_t n_descs; + uint8_t *desc_ack; +}; + +static int +virtio_net_ctrl_pop(struct virtio_net *dev, struct vhost_virtqueue *cvq, + struct virtio_net_ctrl_elem *ctrl_elem) + __rte_shared_locks_required(&cvq->iotlb_lock) +{ + uint16_t avail_idx, desc_idx, n_descs = 0; + uint64_t desc_len, desc_addr, desc_iova, data_len = 0; + uint8_t *ctrl_req; + struct vring_desc *descs; + + avail_idx = __atomic_load_n(&cvq->avail->idx, __ATOMIC_ACQUIRE); + if (avail_idx == cvq->last_avail_idx) { + VHOST_LOG_CONFIG(dev->ifname, DEBUG, "Control queue empty\n"); + return 0; + } + + desc_idx = cvq->avail->ring[cvq->last_avail_idx]; + if (desc_idx >= cvq->size) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Out of range desc index, dropping\n"); + goto err; + } + + ctrl_elem->head_idx = desc_idx; + + if (cvq->desc[desc_idx].flags & VRING_DESC_F_INDIRECT) { + desc_len = cvq->desc[desc_idx].len; + desc_iova = cvq->desc[desc_idx].addr; + + descs = (struct vring_desc *)(uintptr_t)vhost_iova_to_vva(dev, cvq, + desc_iova, &desc_len, VHOST_ACCESS_RO); + if (!descs || desc_len != cvq->desc[desc_idx].len) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to map ctrl indirect descs\n"); + goto err; + } + + desc_idx = 0; + } else { + descs = cvq->desc; + } + + while (1) { + desc_len = descs[desc_idx].len; + desc_iova = descs[desc_idx].addr; + + n_descs++; + + if (descs[desc_idx].flags & VRING_DESC_F_WRITE) { + if (ctrl_elem->desc_ack) { + VHOST_LOG_CONFIG(dev->ifname, ERR, + "Unexpected ctrl chain layout\n"); + goto err; + } + + if (desc_len != sizeof(uint8_t)) { + VHOST_LOG_CONFIG(dev->ifname, ERR, + "Invalid ack size for ctrl req, dropping\n"); + goto err; + } + + ctrl_elem->desc_ack = (uint8_t *)(uintptr_t)vhost_iova_to_vva(dev, cvq, + desc_iova, &desc_len, VHOST_ACCESS_WO); + if (!ctrl_elem->desc_ack || desc_len != sizeof(uint8_t)) { + VHOST_LOG_CONFIG(dev->ifname, ERR, + "Failed to map ctrl ack descriptor\n"); + goto err; + } + } else { + if (ctrl_elem->desc_ack) { + VHOST_LOG_CONFIG(dev->ifname, ERR, + "Unexpected ctrl chain layout\n"); + goto err; + } + + data_len += desc_len; + } + + if (!(descs[desc_idx].flags & VRING_DESC_F_NEXT)) + break; + + desc_idx = descs[desc_idx].next; + } + + desc_idx = ctrl_elem->head_idx; + + if (cvq->desc[desc_idx].flags & VRING_DESC_F_INDIRECT) + ctrl_elem->n_descs = 1; + else + ctrl_elem->n_descs = n_descs; + + if (!ctrl_elem->desc_ack) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Missing ctrl ack descriptor\n"); + goto err; + } + + if (data_len < sizeof(ctrl_elem->ctrl_req->class) + sizeof(ctrl_elem->ctrl_req->command)) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Invalid control header size\n"); + goto err; + } + + ctrl_elem->ctrl_req = malloc(data_len); + if (!ctrl_elem->ctrl_req) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to alloc ctrl request\n"); + goto err; + } + + ctrl_req = (uint8_t *)ctrl_elem->ctrl_req; + + if (cvq->desc[desc_idx].flags & VRING_DESC_F_INDIRECT) { + desc_len = cvq->desc[desc_idx].len; + desc_iova = cvq->desc[desc_idx].addr; + + descs = (struct vring_desc *)(uintptr_t)vhost_iova_to_vva(dev, cvq, + desc_iova, &desc_len, VHOST_ACCESS_RO); + if (!descs || desc_len != cvq->desc[desc_idx].len) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to map ctrl indirect descs\n"); + goto free_err; + } + + desc_idx = 0; + } else { + descs = cvq->desc; + } + + while (!(descs[desc_idx].flags & VRING_DESC_F_WRITE)) { + desc_len = descs[desc_idx].len; + desc_iova = descs[desc_idx].addr; + + desc_addr = vhost_iova_to_vva(dev, cvq, desc_iova, &desc_len, VHOST_ACCESS_RO); + if (!desc_addr || desc_len < descs[desc_idx].len) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to map ctrl descriptor\n"); + goto free_err; + } + + memcpy(ctrl_req, (void *)(uintptr_t)desc_addr, desc_len); + ctrl_req += desc_len; + + if (!(descs[desc_idx].flags & VRING_DESC_F_NEXT)) + break; + + desc_idx = descs[desc_idx].next; + } + + cvq->last_avail_idx++; + if (cvq->last_avail_idx >= cvq->size) + cvq->last_avail_idx -= cvq->size; + + if (dev->features & (1ULL << VIRTIO_RING_F_EVENT_IDX)) + vhost_avail_event(cvq) = cvq->last_avail_idx; + + return 1; + +free_err: + free(ctrl_elem->ctrl_req); +err: + cvq->last_avail_idx++; + if (cvq->last_avail_idx >= cvq->size) + cvq->last_avail_idx -= cvq->size; + + if (dev->features & (1ULL << VIRTIO_RING_F_EVENT_IDX)) + vhost_avail_event(cvq) = cvq->last_avail_idx; + + return -1; +} + +static uint8_t +virtio_net_ctrl_handle_req(struct virtio_net *dev, struct virtio_net_ctrl *ctrl_req) +{ + uint8_t ret = VIRTIO_NET_ERR; + + if (ctrl_req->class == VIRTIO_NET_CTRL_MQ && + ctrl_req->command == VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET) { + uint16_t queue_pairs; + uint32_t i; + + queue_pairs = *(uint16_t *)(uintptr_t)ctrl_req->command_data; + VHOST_LOG_CONFIG(dev->ifname, INFO, "Ctrl req: MQ %u queue pairs\n", queue_pairs); + ret = VIRTIO_NET_OK; + + for (i = 0; i < dev->nr_vring; i++) { + struct vhost_virtqueue *vq = dev->virtqueue[i]; + bool enable; + + if (vq == dev->cvq) + continue; + + if (i < queue_pairs * 2) + enable = true; + else + enable = false; + + vq->enabled = enable; + if (dev->notify_ops->vring_state_changed) + dev->notify_ops->vring_state_changed(dev->vid, i, enable); + } + } + + return ret; +} + +static int +virtio_net_ctrl_push(struct virtio_net *dev, struct virtio_net_ctrl_elem *ctrl_elem) +{ + struct vhost_virtqueue *cvq = dev->cvq; + struct vring_used_elem *used_elem; + + used_elem = &cvq->used->ring[cvq->last_used_idx]; + used_elem->id = ctrl_elem->head_idx; + used_elem->len = ctrl_elem->n_descs; + + cvq->last_used_idx++; + if (cvq->last_used_idx >= cvq->size) + cvq->last_used_idx -= cvq->size; + + __atomic_store_n(&cvq->used->idx, cvq->last_used_idx, __ATOMIC_RELEASE); + + vhost_vring_call_split(dev, dev->cvq); + + free(ctrl_elem->ctrl_req); + + return 0; +} + +int +virtio_net_ctrl_handle(struct virtio_net *dev) +{ + int ret = 0; + + if (dev->features & (1ULL << VIRTIO_F_RING_PACKED)) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Packed ring not supported yet\n"); + return -1; + } + + if (!dev->cvq) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "missing control queue\n"); + return -1; + } + + rte_rwlock_read_lock(&dev->cvq->access_lock); + vhost_user_iotlb_rd_lock(dev->cvq); + + while (1) { + struct virtio_net_ctrl_elem ctrl_elem; + + memset(&ctrl_elem, 0, sizeof(struct virtio_net_ctrl_elem)); + + ret = virtio_net_ctrl_pop(dev, dev->cvq, &ctrl_elem); + if (ret <= 0) + break; + + *ctrl_elem.desc_ack = virtio_net_ctrl_handle_req(dev, ctrl_elem.ctrl_req); + + ret = virtio_net_ctrl_push(dev, &ctrl_elem); + if (ret < 0) + break; + } + + vhost_user_iotlb_rd_unlock(dev->cvq); + rte_rwlock_read_unlock(&dev->cvq->access_lock); + + return ret; +} diff --git a/lib/vhost/virtio_net_ctrl.h b/lib/vhost/virtio_net_ctrl.h new file mode 100644 index 0000000000..9a90f4b9da --- /dev/null +++ b/lib/vhost/virtio_net_ctrl.h @@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Red Hat, Inc. + */ + +#ifndef _VIRTIO_NET_CTRL_H +#define _VIRTIO_NET_CTRL_H + +int virtio_net_ctrl_handle(struct virtio_net *dev); + +#endif From patchwork Thu Jun 1 20:08:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127923 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B49A242C08; Thu, 1 Jun 2023 22:10:03 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6899A42FC4; Thu, 1 Jun 2023 22:09:12 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id DFB8542FA6 for ; Thu, 1 Jun 2023 22:09:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650150; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Srz5SyFp3lpfq/AC319jSE9Sw5G0wmsLmdf65yy9f0g=; b=PMl+re1ES0xiDPxzDm+NcsA72kiD74lJiLx3n/j++k0HrUD3grlk0mLvCleT071b6FMFTB AwL4dhTc/sXvGCorg636Zp7Lf4jo2t5ItJcEmXEVw4w1maIJuq27x0Hh+0iRkv0l6Rw4xx UvNGluo/rls0niF8+Hr6h5OiWNY3cvs= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-643-ZTafkQDONvCDIW6oFTcm6w-1; Thu, 01 Jun 2023 16:09:07 -0400 X-MC-Unique: ZTafkQDONvCDIW6oFTcm6w-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8B7073806634; Thu, 1 Jun 2023 20:09:06 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id EA4702029F6D; Thu, 1 Jun 2023 20:09:03 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v4 16/26] vhost: add VDUSE device creation and destruction Date: Thu, 1 Jun 2023 22:08:02 +0200 Message-Id: <20230601200812.672233-17-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds initial support for VDUSE, which includes the device creation and destruction. It does not include the virtqueues configuration, so this is not functionnal at this point. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/meson.build | 4 + lib/vhost/socket.c | 34 ++++--- lib/vhost/vduse.c | 201 ++++++++++++++++++++++++++++++++++++++++++ lib/vhost/vduse.h | 33 +++++++ lib/vhost/vhost.h | 2 + 5 files changed, 262 insertions(+), 12 deletions(-) create mode 100644 lib/vhost/vduse.c create mode 100644 lib/vhost/vduse.h diff --git a/lib/vhost/meson.build b/lib/vhost/meson.build index d211a0bd37..13e0382c92 100644 --- a/lib/vhost/meson.build +++ b/lib/vhost/meson.build @@ -29,6 +29,10 @@ sources = files( 'virtio_net.c', 'virtio_net_ctrl.c', ) +if cc.has_header('linux/vduse.h') + sources += files('vduse.c') + cflags += '-DVHOST_HAS_VDUSE' +endif headers = files( 'rte_vdpa.h', 'rte_vhost.h', diff --git a/lib/vhost/socket.c b/lib/vhost/socket.c index 29f7a8cece..19a7469e45 100644 --- a/lib/vhost/socket.c +++ b/lib/vhost/socket.c @@ -19,6 +19,7 @@ #include #include "fd_man.h" +#include "vduse.h" #include "vhost.h" #include "vhost_user.h" @@ -36,6 +37,7 @@ struct vhost_user_socket { int socket_fd; struct sockaddr_un un; bool is_server; + bool is_vduse; bool reconnect; bool iommu_support; bool use_builtin_virtio_net; @@ -991,18 +993,21 @@ rte_vhost_driver_register(const char *path, uint64_t flags) #endif } - if ((flags & RTE_VHOST_USER_CLIENT) != 0) { - vsocket->reconnect = !(flags & RTE_VHOST_USER_NO_RECONNECT); - if (vsocket->reconnect && reconn_tid == 0) { - if (vhost_user_reconnect_init() != 0) - goto out_mutex; - } + if (!strncmp("/dev/vduse/", path, strlen("/dev/vduse/"))) { + vsocket->is_vduse = true; } else { - vsocket->is_server = true; - } - ret = create_unix_socket(vsocket); - if (ret < 0) { - goto out_mutex; + if ((flags & RTE_VHOST_USER_CLIENT) != 0) { + vsocket->reconnect = !(flags & RTE_VHOST_USER_NO_RECONNECT); + if (vsocket->reconnect && reconn_tid == 0) { + if (vhost_user_reconnect_init() != 0) + goto out_mutex; + } + } else { + vsocket->is_server = true; + } + ret = create_unix_socket(vsocket); + if (ret < 0) + goto out_mutex; } vhost_user.vsockets[vhost_user.vsocket_cnt++] = vsocket; @@ -1067,7 +1072,9 @@ rte_vhost_driver_unregister(const char *path) if (strcmp(vsocket->path, path)) continue; - if (vsocket->is_server) { + if (vsocket->is_vduse) { + vduse_device_destroy(path); + } else if (vsocket->is_server) { /* * If r/wcb is executing, release vhost_user's * mutex lock, and try again since the r/wcb @@ -1218,6 +1225,9 @@ rte_vhost_driver_start(const char *path) if (!vsocket) return -1; + if (vsocket->is_vduse) + return vduse_device_create(path); + if (fdset_tid == 0) { /** * create a pipe which will be waited by poll and notified to diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c new file mode 100644 index 0000000000..d67818bfb5 --- /dev/null +++ b/lib/vhost/vduse.c @@ -0,0 +1,201 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Red Hat, Inc. + */ + +#include +#include +#include +#include + + +#include +#include + +#include +#include + +#include + +#include "vduse.h" +#include "vhost.h" + +#define VHOST_VDUSE_API_VERSION 0 +#define VDUSE_CTRL_PATH "/dev/vduse/control" + +#define VDUSE_NET_SUPPORTED_FEATURES ((1ULL << VIRTIO_NET_F_MRG_RXBUF) | \ + (1ULL << VIRTIO_F_ANY_LAYOUT) | \ + (1ULL << VIRTIO_F_VERSION_1) | \ + (1ULL << VIRTIO_NET_F_GSO) | \ + (1ULL << VIRTIO_NET_F_HOST_TSO4) | \ + (1ULL << VIRTIO_NET_F_HOST_TSO6) | \ + (1ULL << VIRTIO_NET_F_HOST_UFO) | \ + (1ULL << VIRTIO_NET_F_HOST_ECN) | \ + (1ULL << VIRTIO_NET_F_CSUM) | \ + (1ULL << VIRTIO_NET_F_GUEST_CSUM) | \ + (1ULL << VIRTIO_NET_F_GUEST_TSO4) | \ + (1ULL << VIRTIO_NET_F_GUEST_TSO6) | \ + (1ULL << VIRTIO_NET_F_GUEST_UFO) | \ + (1ULL << VIRTIO_NET_F_GUEST_ECN) | \ + (1ULL << VIRTIO_RING_F_INDIRECT_DESC) | \ + (1ULL << VIRTIO_F_IN_ORDER) | \ + (1ULL << VIRTIO_F_IOMMU_PLATFORM)) + +static struct vhost_backend_ops vduse_backend_ops = { +}; + +int +vduse_device_create(const char *path) +{ + int control_fd, dev_fd, vid, ret; + uint32_t i; + struct virtio_net *dev; + uint64_t ver = VHOST_VDUSE_API_VERSION; + struct vduse_dev_config *dev_config = NULL; + const char *name = path + strlen("/dev/vduse/"); + + control_fd = open(VDUSE_CTRL_PATH, O_RDWR); + if (control_fd < 0) { + VHOST_LOG_CONFIG(name, ERR, "Failed to open %s: %s\n", + VDUSE_CTRL_PATH, strerror(errno)); + return -1; + } + + if (ioctl(control_fd, VDUSE_SET_API_VERSION, &ver)) { + VHOST_LOG_CONFIG(name, ERR, "Failed to set API version: %" PRIu64 ": %s\n", + ver, strerror(errno)); + ret = -1; + goto out_ctrl_close; + } + + dev_config = malloc(offsetof(struct vduse_dev_config, config)); + if (!dev_config) { + VHOST_LOG_CONFIG(name, ERR, "Failed to allocate VDUSE config\n"); + ret = -1; + goto out_ctrl_close; + } + + memset(dev_config, 0, sizeof(struct vduse_dev_config)); + + strncpy(dev_config->name, name, VDUSE_NAME_MAX - 1); + dev_config->device_id = VIRTIO_ID_NET; + dev_config->vendor_id = 0; + dev_config->features = VDUSE_NET_SUPPORTED_FEATURES; + dev_config->vq_num = 2; + dev_config->vq_align = sysconf(_SC_PAGE_SIZE); + dev_config->config_size = 0; + + ret = ioctl(control_fd, VDUSE_CREATE_DEV, dev_config); + if (ret < 0) { + VHOST_LOG_CONFIG(name, ERR, "Failed to create VDUSE device: %s\n", + strerror(errno)); + goto out_free; + } + + dev_fd = open(path, O_RDWR); + if (dev_fd < 0) { + VHOST_LOG_CONFIG(name, ERR, "Failed to open device %s: %s\n", + path, strerror(errno)); + ret = -1; + goto out_dev_close; + } + + ret = fcntl(dev_fd, F_SETFL, O_NONBLOCK); + if (ret < 0) { + VHOST_LOG_CONFIG(name, ERR, "Failed to set chardev as non-blocking: %s\n", + strerror(errno)); + goto out_dev_close; + } + + vid = vhost_new_device(&vduse_backend_ops); + if (vid < 0) { + VHOST_LOG_CONFIG(name, ERR, "Failed to create new Vhost device\n"); + ret = -1; + goto out_dev_close; + } + + dev = get_device(vid); + if (!dev) { + ret = -1; + goto out_dev_close; + } + + strncpy(dev->ifname, path, IF_NAME_SZ - 1); + dev->vduse_ctrl_fd = control_fd; + dev->vduse_dev_fd = dev_fd; + vhost_setup_virtio_net(dev->vid, true, true, true, true); + + for (i = 0; i < 2; i++) { + struct vduse_vq_config vq_cfg = { 0 }; + + ret = alloc_vring_queue(dev, i); + if (ret) { + VHOST_LOG_CONFIG(name, ERR, "Failed to alloc vring %d metadata\n", i); + goto out_dev_destroy; + } + + vq_cfg.index = i; + vq_cfg.max_size = 1024; + + ret = ioctl(dev->vduse_dev_fd, VDUSE_VQ_SETUP, &vq_cfg); + if (ret) { + VHOST_LOG_CONFIG(name, ERR, "Failed to set-up VQ %d\n", i); + goto out_dev_destroy; + } + } + + free(dev_config); + + return 0; + +out_dev_destroy: + vhost_destroy_device(vid); +out_dev_close: + if (dev_fd >= 0) + close(dev_fd); + ioctl(control_fd, VDUSE_DESTROY_DEV, name); +out_free: + free(dev_config); +out_ctrl_close: + close(control_fd); + + return ret; +} + +int +vduse_device_destroy(const char *path) +{ + const char *name = path + strlen("/dev/vduse/"); + struct virtio_net *dev; + int vid, ret; + + for (vid = 0; vid < RTE_MAX_VHOST_DEVICE; vid++) { + dev = vhost_devices[vid]; + + if (dev == NULL) + continue; + + if (!strcmp(path, dev->ifname)) + break; + } + + if (vid == RTE_MAX_VHOST_DEVICE) + return -1; + + if (dev->vduse_dev_fd >= 0) { + close(dev->vduse_dev_fd); + dev->vduse_dev_fd = -1; + } + + if (dev->vduse_ctrl_fd >= 0) { + ret = ioctl(dev->vduse_ctrl_fd, VDUSE_DESTROY_DEV, name); + if (ret) + VHOST_LOG_CONFIG(name, ERR, "Failed to destroy VDUSE device: %s\n", + strerror(errno)); + close(dev->vduse_ctrl_fd); + dev->vduse_ctrl_fd = -1; + } + + vhost_destroy_device(vid); + + return 0; +} diff --git a/lib/vhost/vduse.h b/lib/vhost/vduse.h new file mode 100644 index 0000000000..a15e5d4c16 --- /dev/null +++ b/lib/vhost/vduse.h @@ -0,0 +1,33 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Red Hat, Inc. + */ + +#ifndef _VDUSE_H +#define _VDUSE_H + +#include "vhost.h" + +#ifdef VHOST_HAS_VDUSE + +int vduse_device_create(const char *path); +int vduse_device_destroy(const char *path); + +#else + +static inline int +vduse_device_create(const char *path) +{ + VHOST_LOG_CONFIG(path, ERR, "VDUSE support disabled at build time\n"); + return -1; +} + +static inline int +vduse_device_destroy(const char *path) +{ + VHOST_LOG_CONFIG(path, ERR, "VDUSE support disabled at build time\n"); + return -1; +} + +#endif /* VHOST_HAS_VDUSE */ + +#endif /* _VDUSE_H */ diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h index 04267a3ac5..9ecede0f30 100644 --- a/lib/vhost/vhost.h +++ b/lib/vhost/vhost.h @@ -526,6 +526,8 @@ struct virtio_net { int postcopy_ufd; int postcopy_listening; + int vduse_ctrl_fd; + int vduse_dev_fd; struct vhost_virtqueue *cvq; From patchwork Thu Jun 1 20:08:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127924 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6AD9542C08; Thu, 1 Jun 2023 22:10:09 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8BFD442FB4; Thu, 1 Jun 2023 22:09:15 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 9E47842FB4 for ; Thu, 1 Jun 2023 22:09:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650153; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bCjb9bPrkWjcMRuH9nhbBJ1yUcyb/M8upG8Hsux6BOI=; b=ECuAortDLyAqeEExquoH5MVI1yZzYOsY04CQnE270EYyNFqYKbk7UnM0A1IQu51vnZIA9p oQARTBWmr6qGXZLxI273guxanzjdhTU8dFbnqV76pPIQrp7KV+d+MPLAP9bv/GJD1x+5Kk ZylRWI606MWs90lp7u3rIvQeIz+iQoM= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-617-1Y0Lg3bEOA230f_5oH27MA-1; Thu, 01 Jun 2023 16:09:10 -0400 X-MC-Unique: 1Y0Lg3bEOA230f_5oH27MA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 820FE2800C2B; Thu, 1 Jun 2023 20:09:09 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id E53A3202696C; Thu, 1 Jun 2023 20:09:06 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v4 17/26] vhost: add VDUSE callback for IOTLB miss Date: Thu, 1 Jun 2023 22:08:03 +0200 Message-Id: <20230601200812.672233-18-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch implements the VDUSE callback for IOTLB misses, which is done by using the VDUSE VDUSE_IOTLB_GET_FD ioctl. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vduse.c | 58 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c index d67818bfb5..f72c7bf6ab 100644 --- a/lib/vhost/vduse.c +++ b/lib/vhost/vduse.c @@ -13,9 +13,11 @@ #include #include +#include #include +#include "iotlb.h" #include "vduse.h" #include "vhost.h" @@ -40,7 +42,63 @@ (1ULL << VIRTIO_F_IN_ORDER) | \ (1ULL << VIRTIO_F_IOMMU_PLATFORM)) +static int +vduse_iotlb_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm __rte_unused) +{ + struct vduse_iotlb_entry entry; + uint64_t size, page_size; + struct stat stat; + void *mmap_addr; + int fd, ret; + + entry.start = iova; + entry.last = iova + 1; + + ret = ioctl(dev->vduse_dev_fd, VDUSE_IOTLB_GET_FD, &entry); + if (ret < 0) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to get IOTLB entry for 0x%" PRIx64 "\n", + iova); + return -1; + } + + fd = ret; + + VHOST_LOG_CONFIG(dev->ifname, DEBUG, "New IOTLB entry:\n"); + VHOST_LOG_CONFIG(dev->ifname, DEBUG, "\tIOVA: %" PRIx64 " - %" PRIx64 "\n", + (uint64_t)entry.start, (uint64_t)entry.last); + VHOST_LOG_CONFIG(dev->ifname, DEBUG, "\toffset: %" PRIx64 "\n", (uint64_t)entry.offset); + VHOST_LOG_CONFIG(dev->ifname, DEBUG, "\tfd: %d\n", fd); + VHOST_LOG_CONFIG(dev->ifname, DEBUG, "\tperm: %x\n", entry.perm); + + size = entry.last - entry.start + 1; + mmap_addr = mmap(0, size + entry.offset, entry.perm, MAP_SHARED, fd, 0); + if (!mmap_addr) { + VHOST_LOG_CONFIG(dev->ifname, ERR, + "Failed to mmap IOTLB entry for 0x%" PRIx64 "\n", iova); + ret = -1; + goto close_fd; + } + + ret = fstat(fd, &stat); + if (ret < 0) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to get page size.\n"); + munmap(mmap_addr, entry.offset + size); + goto close_fd; + } + page_size = (uint64_t)stat.st_blksize; + + vhost_user_iotlb_cache_insert(dev, entry.start, (uint64_t)(uintptr_t)mmap_addr, + entry.offset, size, page_size, entry.perm); + + ret = 0; +close_fd: + close(fd); + + return ret; +} + static struct vhost_backend_ops vduse_backend_ops = { + .iotlb_miss = vduse_iotlb_miss, }; int From patchwork Thu Jun 1 20:08:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127925 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2E94C42C08; Thu, 1 Jun 2023 22:10:15 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8911042FC0; Thu, 1 Jun 2023 22:09:20 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id F269C42D61 for ; Thu, 1 Jun 2023 22:09:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650158; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KaBO5Glxw4sHd0jMxJSKcFBrQo1PSSFcgrz7LOJWFWo=; b=a4LzseKAAy6h5Fdfdn34YvNyknc+1mXrJB27Vgq+Gzte2fdle/+6kEdEF25Mlao/TvjqZx jZPKWVj054+gZUf7WPQWe5G5jsLn+dxUgycTg+EFHgiAJPb7UCRfc+ZqOsBqvXB3lRaWZM ZXGf93lg5aYqAHzcGJ/IkPR++x2VUvM= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-272-U-MaNeDtOMaOqQE4xc7EpQ-1; Thu, 01 Jun 2023 16:09:15 -0400 X-MC-Unique: U-MaNeDtOMaOqQE4xc7EpQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 52D001C05152; Thu, 1 Jun 2023 20:09:14 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id C86072029F6D; Thu, 1 Jun 2023 20:09:09 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v4 18/26] vhost: add VDUSE callback for IOTLB entry removal Date: Thu, 1 Jun 2023 22:08:04 +0200 Message-Id: <20230601200812.672233-19-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch implements the VDUSE callback for IOTLB entry removal, where it unmaps the pages from the invalidated IOTLB entry. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vduse.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c index f72c7bf6ab..58c1b384a8 100644 --- a/lib/vhost/vduse.c +++ b/lib/vhost/vduse.c @@ -42,6 +42,12 @@ (1ULL << VIRTIO_F_IN_ORDER) | \ (1ULL << VIRTIO_F_IOMMU_PLATFORM)) +static void +vduse_iotlb_remove_notify(uint64_t addr, uint64_t offset, uint64_t size) +{ + munmap((void *)(uintptr_t)addr, offset + size); +} + static int vduse_iotlb_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm __rte_unused) { @@ -99,6 +105,7 @@ vduse_iotlb_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm __rte_unuse static struct vhost_backend_ops vduse_backend_ops = { .iotlb_miss = vduse_iotlb_miss, + .iotlb_remove_notify = vduse_iotlb_remove_notify, }; int From patchwork Thu Jun 1 20:08:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127926 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9C69C42C08; Thu, 1 Jun 2023 22:10:20 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8686042FD8; Thu, 1 Jun 2023 22:09:22 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 56ACD42FA6 for ; Thu, 1 Jun 2023 22:09:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650159; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aYGAMnOnshRq1wFFVMK1JfUyFWNr5tDK5GwYNj+Lrw8=; b=E/PqtHl6cP1WYRYH3r+RY+8dIcbIvHsViUAvBjnwOEza36uaceAiMi7g9AAoFVhxPw/+EA o7zZuDsnfmLpSpNXsAqWlSu2wi+pvBLaO0iLfiu1gTtGEY+DmDMqNW6aSa8oeUo429eHc8 NSSxGV/2vztk4Gy9CgJBsE171AsxsNA= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-221-rZaEsxYTPKqxv93_vLOn5g-1; Thu, 01 Jun 2023 16:09:17 -0400 X-MC-Unique: rZaEsxYTPKqxv93_vLOn5g-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5A34E3C00088; Thu, 1 Jun 2023 20:09:17 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id A3D8B2029F6D; Thu, 1 Jun 2023 20:09:14 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v4 19/26] vhost: add VDUSE callback for IRQ injection Date: Thu, 1 Jun 2023 22:08:05 +0200 Message-Id: <20230601200812.672233-20-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch implements the VDUSE callback for kicking virtqueues. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vduse.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c index 58c1b384a8..d39e39b9dc 100644 --- a/lib/vhost/vduse.c +++ b/lib/vhost/vduse.c @@ -42,6 +42,12 @@ (1ULL << VIRTIO_F_IN_ORDER) | \ (1ULL << VIRTIO_F_IOMMU_PLATFORM)) +static int +vduse_inject_irq(struct virtio_net *dev, struct vhost_virtqueue *vq) +{ + return ioctl(dev->vduse_dev_fd, VDUSE_VQ_INJECT_IRQ, &vq->index); +} + static void vduse_iotlb_remove_notify(uint64_t addr, uint64_t offset, uint64_t size) { @@ -106,6 +112,7 @@ vduse_iotlb_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm __rte_unuse static struct vhost_backend_ops vduse_backend_ops = { .iotlb_miss = vduse_iotlb_miss, .iotlb_remove_notify = vduse_iotlb_remove_notify, + .inject_irq = vduse_inject_irq, }; int From patchwork Thu Jun 1 20:08:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127927 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C463A42C08; Thu, 1 Jun 2023 22:10:25 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9CB5842FE0; Thu, 1 Jun 2023 22:09:26 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 0647442BDA for ; Thu, 1 Jun 2023 22:09:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650164; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/NaxkaDcGVWBez8isP823L/8Qgqwaypg58E+DhHMgoE=; b=NdZ/+gOJUY3GdWy1aVe+8+cwtyLHv911CmP/u+BjGWzEcGQPJhGaW1zI6U36bmbVeWzuKl KaswwlkpebA9lrzrgldwEWuJzoxNYhR5JQG+wYw30EKuK7ebJMe2sa/3T7bd/rAp3qV9X7 nGC3MV7ENEmLxfH0aI008r7dFFQv5iQ= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-475-CEui7vqoMhWSEDhCPuT5ZA-1; Thu, 01 Jun 2023 16:09:20 -0400 X-MC-Unique: CEui7vqoMhWSEDhCPuT5ZA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 539D9185A79B; Thu, 1 Jun 2023 20:09:20 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id AC5412029F6D; Thu, 1 Jun 2023 20:09:17 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v4 20/26] vhost: add VDUSE events handler Date: Thu, 1 Jun 2023 22:08:06 +0200 Message-Id: <20230601200812.672233-21-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch makes use of Vhost lib's FD manager to install a handler for VDUSE events occurring on the VDUSE device FD. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vduse.c | 101 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+) diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c index d39e39b9dc..92c515cff2 100644 --- a/lib/vhost/vduse.c +++ b/lib/vhost/vduse.c @@ -17,6 +17,7 @@ #include +#include "fd_man.h" #include "iotlb.h" #include "vduse.h" #include "vhost.h" @@ -42,6 +43,31 @@ (1ULL << VIRTIO_F_IN_ORDER) | \ (1ULL << VIRTIO_F_IOMMU_PLATFORM)) +struct vduse { + struct fdset fdset; +}; + +static struct vduse vduse = { + .fdset = { + .fd = { [0 ... MAX_FDS - 1] = {-1, NULL, NULL, NULL, 0} }, + .fd_mutex = PTHREAD_MUTEX_INITIALIZER, + .fd_pooling_mutex = PTHREAD_MUTEX_INITIALIZER, + .num = 0 + }, +}; + +static bool vduse_events_thread; + +static const char * const vduse_reqs_str[] = { + "VDUSE_GET_VQ_STATE", + "VDUSE_SET_STATUS", + "VDUSE_UPDATE_IOTLB", +}; + +#define vduse_req_id_to_str(id) \ + (id < RTE_DIM(vduse_reqs_str) ? \ + vduse_reqs_str[id] : "Unknown") + static int vduse_inject_irq(struct virtio_net *dev, struct vhost_virtqueue *vq) { @@ -115,16 +141,80 @@ static struct vhost_backend_ops vduse_backend_ops = { .inject_irq = vduse_inject_irq, }; +static void +vduse_events_handler(int fd, void *arg, int *remove __rte_unused) +{ + struct virtio_net *dev = arg; + struct vduse_dev_request req; + struct vduse_dev_response resp; + int ret; + + memset(&resp, 0, sizeof(resp)); + + ret = read(fd, &req, sizeof(req)); + if (ret < 0) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to read request: %s\n", + strerror(errno)); + return; + } else if (ret < (int)sizeof(req)) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Incomplete to read request %d\n", ret); + return; + } + + VHOST_LOG_CONFIG(dev->ifname, INFO, "New request: %s (%u)\n", + vduse_req_id_to_str(req.type), req.type); + + switch (req.type) { + default: + resp.result = VDUSE_REQ_RESULT_FAILED; + break; + } + + resp.request_id = req.request_id; + + ret = write(dev->vduse_dev_fd, &resp, sizeof(resp)); + if (ret != sizeof(resp)) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to write response %s\n", + strerror(errno)); + return; + } + VHOST_LOG_CONFIG(dev->ifname, INFO, "Request %s (%u) handled successfully\n", + vduse_req_id_to_str(req.type), req.type); +} + int vduse_device_create(const char *path) { int control_fd, dev_fd, vid, ret; + pthread_t fdset_tid; uint32_t i; struct virtio_net *dev; uint64_t ver = VHOST_VDUSE_API_VERSION; struct vduse_dev_config *dev_config = NULL; const char *name = path + strlen("/dev/vduse/"); + /* If first device, create events dispatcher thread */ + if (vduse_events_thread == false) { + /** + * create a pipe which will be waited by poll and notified to + * rebuild the wait list of poll. + */ + if (fdset_pipe_init(&vduse.fdset) < 0) { + VHOST_LOG_CONFIG(path, ERR, "failed to create pipe for vduse fdset\n"); + return -1; + } + + ret = rte_ctrl_thread_create(&fdset_tid, "vduse-events", NULL, + fdset_event_dispatch, &vduse.fdset); + if (ret != 0) { + VHOST_LOG_CONFIG(path, ERR, "failed to create vduse fdset handling thread\n"); + fdset_pipe_uninit(&vduse.fdset); + return -1; + } + + vduse_events_thread = true; + } + control_fd = open(VDUSE_CTRL_PATH, O_RDWR); if (control_fd < 0) { VHOST_LOG_CONFIG(name, ERR, "Failed to open %s: %s\n", @@ -215,6 +305,14 @@ vduse_device_create(const char *path) } } + ret = fdset_add(&vduse.fdset, dev->vduse_dev_fd, vduse_events_handler, NULL, dev); + if (ret) { + VHOST_LOG_CONFIG(name, ERR, "Failed to add fd %d to vduse fdset\n", + dev->vduse_dev_fd); + goto out_dev_destroy; + } + fdset_pipe_notify(&vduse.fdset); + free(dev_config); return 0; @@ -253,6 +351,9 @@ vduse_device_destroy(const char *path) if (vid == RTE_MAX_VHOST_DEVICE) return -1; + fdset_del(&vduse.fdset, dev->vduse_dev_fd); + fdset_pipe_notify(&vduse.fdset); + if (dev->vduse_dev_fd >= 0) { close(dev->vduse_dev_fd); dev->vduse_dev_fd = -1; From patchwork Thu Jun 1 20:08:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127928 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E87AE42C08; Thu, 1 Jun 2023 22:10:32 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1CCFC43008; Thu, 1 Jun 2023 22:09:28 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 139F242FE4 for ; Thu, 1 Jun 2023 22:09:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650166; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/mMbfPtlgp+8d/URc01bG4SThKEEoWRRLfDp81aBQ+U=; b=Gr6/ixEkyCiATP7Zg5FeSkM58TTItenxr7/j2+UabzPdpZZvRgOrCuzP6VlrV0M1s+jXLj 86lpj3RecUVVBMUlWQF2oE+NeO8u/rhyL0mMbqBFGXh+mKjTnuNcsv4EGzZbbjorg1OfDk lKUVg7wzYn87R6qqVZyamlxraf1Fj5o= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-255-geSlF2dCO7CeiFYrrVP6hQ-1; Thu, 01 Jun 2023 16:09:23 -0400 X-MC-Unique: geSlF2dCO7CeiFYrrVP6hQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 53EE03C025BF; Thu, 1 Jun 2023 20:09:23 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9BCFF2029F6D; Thu, 1 Jun 2023 20:09:20 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v4 21/26] vhost: add support for virtqueue state get event Date: Thu, 1 Jun 2023 22:08:07 +0200 Message-Id: <20230601200812.672233-22-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds support for VDUSE_GET_VQ_STATE event handling, which consists in providing the backend last available index for the specified virtqueue. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vduse.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c index 92c515cff2..7e36c50b6c 100644 --- a/lib/vhost/vduse.c +++ b/lib/vhost/vduse.c @@ -147,6 +147,7 @@ vduse_events_handler(int fd, void *arg, int *remove __rte_unused) struct virtio_net *dev = arg; struct vduse_dev_request req; struct vduse_dev_response resp; + struct vhost_virtqueue *vq; int ret; memset(&resp, 0, sizeof(resp)); @@ -165,6 +166,13 @@ vduse_events_handler(int fd, void *arg, int *remove __rte_unused) vduse_req_id_to_str(req.type), req.type); switch (req.type) { + case VDUSE_GET_VQ_STATE: + vq = dev->virtqueue[req.vq_state.index]; + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tvq index: %u, avail_index: %u\n", + req.vq_state.index, vq->last_avail_idx); + resp.vq_state.split.avail_index = vq->last_avail_idx; + resp.result = VDUSE_REQ_RESULT_OK; + break; default: resp.result = VDUSE_REQ_RESULT_FAILED; break; From patchwork Thu Jun 1 20:08:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127929 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 51F5A42C08; Thu, 1 Jun 2023 22:10:38 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2D06643006; Thu, 1 Jun 2023 22:09:31 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 0090242D8D for ; Thu, 1 Jun 2023 22:09:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650169; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+AtyqhGEYrXQHh7YOVU0PAAJ8QpP4RzlHjniUfhJWNo=; b=Fo5MrTRUKV0OFVOhcNlrWzx7U9QFUYsksMBF7w+zTGw7fd8/a/QZeHuD6aYCrLz4ZOwGlB 5HTDV3Zk+JJtR5NrDOHhi5/z27/UU7ytUK7lMcHkrgkRNImK/AVdMZ6Iw+qH0F0LQ5mKFy Yqirl98hoI477CJ5OGfXu2hTArPgOxs= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-481-C-evGYJXO8GtoKj3Nuzdbg-1; Thu, 01 Jun 2023 16:09:26 -0400 X-MC-Unique: C-evGYJXO8GtoKj3Nuzdbg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 31DB2380673A; Thu, 1 Jun 2023 20:09:26 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9AF8C2029F6D; Thu, 1 Jun 2023 20:09:23 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v4 22/26] vhost: add support for VDUSE status set event Date: Thu, 1 Jun 2023 22:08:08 +0200 Message-Id: <20230601200812.672233-23-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds support for VDUSE_SET_STATUS event handling, which consists in updating the Virtio device status set by the Virtio driver. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vduse.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c index 7e36c50b6c..3bf65d4b8b 100644 --- a/lib/vhost/vduse.c +++ b/lib/vhost/vduse.c @@ -173,6 +173,12 @@ vduse_events_handler(int fd, void *arg, int *remove __rte_unused) resp.vq_state.split.avail_index = vq->last_avail_idx; resp.result = VDUSE_REQ_RESULT_OK; break; + case VDUSE_SET_STATUS: + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tnew status: 0x%08x\n", + req.s.status); + dev->status = req.s.status; + resp.result = VDUSE_REQ_RESULT_OK; + break; default: resp.result = VDUSE_REQ_RESULT_FAILED; break; From patchwork Thu Jun 1 20:08:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127930 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9AEB942C08; Thu, 1 Jun 2023 22:10:43 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4080342FF2; Thu, 1 Jun 2023 22:09:40 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 3336142D82 for ; Thu, 1 Jun 2023 22:09:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650178; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yyWxx4cOr/d1MsHouVd421UBe4fkggg/wLUy230+ehE=; b=dn31y10EQSPciXHo+PpkvHYmkkSpHEv0UQl+zu/EnPhViScOOO2Nh534igTM031wSQ5Pfg yLTlm9gZkhl/F6E1lIoh5kkZL2pLuXA53kpafYC1T3Z+PaE1KXHhbOQP9080l/Nz65dGxB wPQ0uzLn94BBU+phUgwDRtFPkHafjBE= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-624-qouYlEDkNl-PNI0aF4_jmg-1; Thu, 01 Jun 2023 16:09:37 -0400 X-MC-Unique: qouYlEDkNl-PNI0aF4_jmg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 26F3A1C05AB3; Thu, 1 Jun 2023 20:09:37 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 83A2B2029F6D; Thu, 1 Jun 2023 20:09:26 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v4 23/26] vhost: add support for VDUSE IOTLB update event Date: Thu, 1 Jun 2023 22:08:09 +0200 Message-Id: <20230601200812.672233-24-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds support for VDUSE_UPDATE_IOTLB event handling, which consists in invaliding IOTLB entries for the range specified in the request. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vduse.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c index 3bf65d4b8b..110654ec68 100644 --- a/lib/vhost/vduse.c +++ b/lib/vhost/vduse.c @@ -179,6 +179,13 @@ vduse_events_handler(int fd, void *arg, int *remove __rte_unused) dev->status = req.s.status; resp.result = VDUSE_REQ_RESULT_OK; break; + case VDUSE_UPDATE_IOTLB: + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tIOVA range: %" PRIx64 " - %" PRIx64 "\n", + (uint64_t)req.iova.start, (uint64_t)req.iova.last); + vhost_user_iotlb_cache_remove(dev, req.iova.start, + req.iova.last - req.iova.start + 1); + resp.result = VDUSE_REQ_RESULT_OK; + break; default: resp.result = VDUSE_REQ_RESULT_FAILED; break; From patchwork Thu Jun 1 20:08:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127931 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id F3ED942C08; Thu, 1 Jun 2023 22:10:48 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 641B242D6D; Thu, 1 Jun 2023 22:09:45 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 1098542C4D for ; Thu, 1 Jun 2023 22:09:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650183; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YwlzYHwt7IM+hV5pg2MDq7egkCHBIQihzJGivnXO/ZE=; b=E37ftuJUj/lPfAXsVgDrPRP5dkDouEaFALylQcMLCxwk/9z9lMok9UH2/+62MUqjh3eDAa 55+UvywQzAJqhwwsNt2t8ToAr01fLmnGnHavQrZvNMRerxMh57sCt9giA2dANZ879WWuqJ vE2gNHfw+bqNR4Nnqd64e9pJP0Rv36c= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-630-J50FCxGvNT-xcC8ozGuS3w-1; Thu, 01 Jun 2023 16:09:40 -0400 X-MC-Unique: J50FCxGvNT-xcC8ozGuS3w-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 204E7811E85; Thu, 1 Jun 2023 20:09:40 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7A5C4202696C; Thu, 1 Jun 2023 20:09:37 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v4 24/26] vhost: add VDUSE device startup Date: Thu, 1 Jun 2023 22:08:10 +0200 Message-Id: <20230601200812.672233-25-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds the device and its virtqueues initialization once the Virtio driver has set the DRIVER_OK in the Virtio status register. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vduse.c | 126 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 126 insertions(+) diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c index 110654ec68..a10dc24d38 100644 --- a/lib/vhost/vduse.c +++ b/lib/vhost/vduse.c @@ -141,6 +141,128 @@ static struct vhost_backend_ops vduse_backend_ops = { .inject_irq = vduse_inject_irq, }; +static void +vduse_vring_setup(struct virtio_net *dev, unsigned int index) +{ + struct vhost_virtqueue *vq = dev->virtqueue[index]; + struct vhost_vring_addr *ra = &vq->ring_addrs; + struct vduse_vq_info vq_info; + struct vduse_vq_eventfd vq_efd; + int ret; + + vq_info.index = index; + ret = ioctl(dev->vduse_dev_fd, VDUSE_VQ_GET_INFO, &vq_info); + if (ret) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to get VQ %u info: %s\n", + index, strerror(errno)); + return; + } + + VHOST_LOG_CONFIG(dev->ifname, INFO, "VQ %u info:\n", index); + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tnum: %u\n", vq_info.num); + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tdesc_addr: %llx\n", vq_info.desc_addr); + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tdriver_addr: %llx\n", vq_info.driver_addr); + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tdevice_addr: %llx\n", vq_info.device_addr); + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tavail_idx: %u\n", vq_info.split.avail_index); + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tready: %u\n", vq_info.ready); + + vq->last_avail_idx = vq_info.split.avail_index; + vq->size = vq_info.num; + vq->ready = true; + vq->enabled = vq_info.ready; + ra->desc_user_addr = vq_info.desc_addr; + ra->avail_user_addr = vq_info.driver_addr; + ra->used_user_addr = vq_info.device_addr; + + vq->kickfd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); + if (vq->kickfd < 0) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to init kickfd for VQ %u: %s\n", + index, strerror(errno)); + vq->kickfd = VIRTIO_INVALID_EVENTFD; + return; + } + VHOST_LOG_CONFIG(dev->ifname, INFO, "\tkick fd: %d\n", vq->kickfd); + + vq->shadow_used_split = rte_malloc_socket(NULL, + vq->size * sizeof(struct vring_used_elem), + RTE_CACHE_LINE_SIZE, 0); + vq->batch_copy_elems = rte_malloc_socket(NULL, + vq->size * sizeof(struct batch_copy_elem), + RTE_CACHE_LINE_SIZE, 0); + + vhost_user_iotlb_rd_lock(vq); + if (vring_translate(dev, vq)) + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to translate vring %d addresses\n", + index); + + if (vhost_enable_guest_notification(dev, vq, 0)) + VHOST_LOG_CONFIG(dev->ifname, ERR, + "Failed to disable guest notifications on vring %d\n", + index); + vhost_user_iotlb_rd_unlock(vq); + + vq_efd.index = index; + vq_efd.fd = vq->kickfd; + + ret = ioctl(dev->vduse_dev_fd, VDUSE_VQ_SETUP_KICKFD, &vq_efd); + if (ret) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to setup kickfd for VQ %u: %s\n", + index, strerror(errno)); + close(vq->kickfd); + vq->kickfd = VIRTIO_UNINITIALIZED_EVENTFD; + return; + } +} + +static void +vduse_device_start(struct virtio_net *dev) +{ + unsigned int i, ret; + + VHOST_LOG_CONFIG(dev->ifname, INFO, "Starting device...\n"); + + dev->notify_ops = vhost_driver_callback_get(dev->ifname); + if (!dev->notify_ops) { + VHOST_LOG_CONFIG(dev->ifname, ERR, + "Failed to get callback ops for driver\n"); + return; + } + + ret = ioctl(dev->vduse_dev_fd, VDUSE_DEV_GET_FEATURES, &dev->features); + if (ret) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to get features: %s\n", + strerror(errno)); + return; + } + + VHOST_LOG_CONFIG(dev->ifname, INFO, "Negotiated Virtio features: 0x%" PRIx64 "\n", + dev->features); + + if (dev->features & + ((1ULL << VIRTIO_NET_F_MRG_RXBUF) | + (1ULL << VIRTIO_F_VERSION_1) | + (1ULL << VIRTIO_F_RING_PACKED))) { + dev->vhost_hlen = sizeof(struct virtio_net_hdr_mrg_rxbuf); + } else { + dev->vhost_hlen = sizeof(struct virtio_net_hdr); + } + + for (i = 0; i < dev->nr_vring; i++) + vduse_vring_setup(dev, i); + + dev->flags |= VIRTIO_DEV_READY; + + if (dev->notify_ops->new_device(dev->vid) == 0) + dev->flags |= VIRTIO_DEV_RUNNING; + + for (i = 0; i < dev->nr_vring; i++) { + struct vhost_virtqueue *vq = dev->virtqueue[i]; + + if (dev->notify_ops->vring_state_changed) + dev->notify_ops->vring_state_changed(dev->vid, i, vq->enabled); + } +} + static void vduse_events_handler(int fd, void *arg, int *remove __rte_unused) { @@ -177,6 +299,10 @@ vduse_events_handler(int fd, void *arg, int *remove __rte_unused) VHOST_LOG_CONFIG(dev->ifname, INFO, "\tnew status: 0x%08x\n", req.s.status); dev->status = req.s.status; + + if (dev->status & VIRTIO_DEVICE_STATUS_DRIVER_OK) + vduse_device_start(dev); + resp.result = VDUSE_REQ_RESULT_OK; break; case VDUSE_UPDATE_IOTLB: From patchwork Thu Jun 1 20:08:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127933 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 970E542C08; Thu, 1 Jun 2023 22:10:58 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A74F043018; Thu, 1 Jun 2023 22:09:52 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 4535942FFE for ; Thu, 1 Jun 2023 22:09:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650189; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GB/AU+Vbvek3rhMkceixizYLiCXsskLFM3Hzutrpcc4=; b=JRXL6Fnxe1G2u/3cqkFoqyNC6aMmpg16olKpVQDMFwTy2QbUb3entvOwmzgq501pSNbHPs eiCv1vmxCUm4IXAwANzs9R14dYjB6OAZPaZKUUrT+LxbKgFC8sOM96BUjHvhOCGx2VL8tY eiG8RLHkngZZ3dyNcku7y7NU+mEJbPU= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-658--Uu0AXkeNqqkiHJWB77tUw-1; Thu, 01 Jun 2023 16:09:45 -0400 X-MC-Unique: -Uu0AXkeNqqkiHJWB77tUw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 19B2F380673A; Thu, 1 Jun 2023 20:09:43 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 677502029F6D; Thu, 1 Jun 2023 20:09:40 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v4 25/26] vhost: add multiqueue support to VDUSE Date: Thu, 1 Jun 2023 22:08:11 +0200 Message-Id: <20230601200812.672233-26-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch enables control queue support in order to support multiqueue. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vduse.c | 83 +++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 76 insertions(+), 7 deletions(-) diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c index a10dc24d38..699cfed9e3 100644 --- a/lib/vhost/vduse.c +++ b/lib/vhost/vduse.c @@ -21,6 +21,7 @@ #include "iotlb.h" #include "vduse.h" #include "vhost.h" +#include "virtio_net_ctrl.h" #define VHOST_VDUSE_API_VERSION 0 #define VDUSE_CTRL_PATH "/dev/vduse/control" @@ -41,7 +42,9 @@ (1ULL << VIRTIO_NET_F_GUEST_ECN) | \ (1ULL << VIRTIO_RING_F_INDIRECT_DESC) | \ (1ULL << VIRTIO_F_IN_ORDER) | \ - (1ULL << VIRTIO_F_IOMMU_PLATFORM)) + (1ULL << VIRTIO_F_IOMMU_PLATFORM) | \ + (1ULL << VIRTIO_NET_F_CTRL_VQ) | \ + (1ULL << VIRTIO_NET_F_MQ)) struct vduse { struct fdset fdset; @@ -141,6 +144,25 @@ static struct vhost_backend_ops vduse_backend_ops = { .inject_irq = vduse_inject_irq, }; +static void +vduse_control_queue_event(int fd, void *arg, int *remove __rte_unused) +{ + struct virtio_net *dev = arg; + uint64_t buf; + int ret; + + ret = read(fd, &buf, sizeof(buf)); + if (ret < 0) { + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to read control queue event: %s\n", + strerror(errno)); + return; + } + + VHOST_LOG_CONFIG(dev->ifname, DEBUG, "Control queue kicked\n"); + if (virtio_net_ctrl_handle(dev)) + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to handle ctrl request\n"); +} + static void vduse_vring_setup(struct virtio_net *dev, unsigned int index) { @@ -212,6 +234,22 @@ vduse_vring_setup(struct virtio_net *dev, unsigned int index) vq->kickfd = VIRTIO_UNINITIALIZED_EVENTFD; return; } + + if (vq == dev->cvq) { + ret = fdset_add(&vduse.fdset, vq->kickfd, vduse_control_queue_event, NULL, dev); + if (ret) { + VHOST_LOG_CONFIG(dev->ifname, ERR, + "Failed to setup kickfd handler for VQ %u: %s\n", + index, strerror(errno)); + vq_efd.fd = VDUSE_EVENTFD_DEASSIGN; + ioctl(dev->vduse_dev_fd, VDUSE_VQ_SETUP_KICKFD, &vq_efd); + close(vq->kickfd); + vq->kickfd = VIRTIO_UNINITIALIZED_EVENTFD; + } + fdset_pipe_notify(&vduse.fdset); + vhost_enable_guest_notification(dev, vq, 1); + VHOST_LOG_CONFIG(dev->ifname, INFO, "Ctrl queue event handler installed\n"); + } } static void @@ -258,6 +296,9 @@ vduse_device_start(struct virtio_net *dev) for (i = 0; i < dev->nr_vring; i++) { struct vhost_virtqueue *vq = dev->virtqueue[i]; + if (vq == dev->cvq) + continue; + if (dev->notify_ops->vring_state_changed) dev->notify_ops->vring_state_changed(dev->vid, i, vq->enabled); } @@ -334,9 +375,11 @@ vduse_device_create(const char *path) { int control_fd, dev_fd, vid, ret; pthread_t fdset_tid; - uint32_t i; + uint32_t i, max_queue_pairs, total_queues; struct virtio_net *dev; + struct virtio_net_config vnet_config = { 0 }; uint64_t ver = VHOST_VDUSE_API_VERSION; + uint64_t features = VDUSE_NET_SUPPORTED_FEATURES; struct vduse_dev_config *dev_config = NULL; const char *name = path + strlen("/dev/vduse/"); @@ -376,22 +419,39 @@ vduse_device_create(const char *path) goto out_ctrl_close; } - dev_config = malloc(offsetof(struct vduse_dev_config, config)); + dev_config = malloc(offsetof(struct vduse_dev_config, config) + + sizeof(vnet_config)); if (!dev_config) { VHOST_LOG_CONFIG(name, ERR, "Failed to allocate VDUSE config\n"); ret = -1; goto out_ctrl_close; } + ret = rte_vhost_driver_get_queue_num(path, &max_queue_pairs); + if (ret < 0) { + VHOST_LOG_CONFIG(name, ERR, "Failed to get max queue pairs\n"); + goto out_free; + } + + VHOST_LOG_CONFIG(path, INFO, "VDUSE max queue pairs: %u\n", max_queue_pairs); + total_queues = max_queue_pairs * 2; + + if (max_queue_pairs == 1) + features &= ~(RTE_BIT64(VIRTIO_NET_F_CTRL_VQ) | RTE_BIT64(VIRTIO_NET_F_MQ)); + else + total_queues += 1; /* Includes ctrl queue */ + + vnet_config.max_virtqueue_pairs = max_queue_pairs; memset(dev_config, 0, sizeof(struct vduse_dev_config)); strncpy(dev_config->name, name, VDUSE_NAME_MAX - 1); dev_config->device_id = VIRTIO_ID_NET; dev_config->vendor_id = 0; - dev_config->features = VDUSE_NET_SUPPORTED_FEATURES; - dev_config->vq_num = 2; + dev_config->features = features; + dev_config->vq_num = total_queues; dev_config->vq_align = sysconf(_SC_PAGE_SIZE); - dev_config->config_size = 0; + dev_config->config_size = sizeof(struct virtio_net_config); + memcpy(dev_config->config, &vnet_config, sizeof(vnet_config)); ret = ioctl(control_fd, VDUSE_CREATE_DEV, dev_config); if (ret < 0) { @@ -433,7 +493,7 @@ vduse_device_create(const char *path) dev->vduse_dev_fd = dev_fd; vhost_setup_virtio_net(dev->vid, true, true, true, true); - for (i = 0; i < 2; i++) { + for (i = 0; i < total_queues; i++) { struct vduse_vq_config vq_cfg = { 0 }; ret = alloc_vring_queue(dev, i); @@ -452,6 +512,8 @@ vduse_device_create(const char *path) } } + dev->cvq = dev->virtqueue[max_queue_pairs * 2]; + ret = fdset_add(&vduse.fdset, dev->vduse_dev_fd, vduse_events_handler, NULL, dev); if (ret) { VHOST_LOG_CONFIG(name, ERR, "Failed to add fd %d to vduse fdset\n", @@ -498,6 +560,13 @@ vduse_device_destroy(const char *path) if (vid == RTE_MAX_VHOST_DEVICE) return -1; + if (dev->cvq && dev->cvq->kickfd >= 0) { + fdset_del(&vduse.fdset, dev->cvq->kickfd); + fdset_pipe_notify(&vduse.fdset); + close(dev->cvq->kickfd); + dev->cvq->kickfd = VIRTIO_UNINITIALIZED_EVENTFD; + } + fdset_del(&vduse.fdset, dev->vduse_dev_fd); fdset_pipe_notify(&vduse.fdset); From patchwork Thu Jun 1 20:08:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 127932 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E2FA842C08; Thu, 1 Jun 2023 22:10:53 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9198943012; Thu, 1 Jun 2023 22:09:51 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id E6D7542FA2 for ; Thu, 1 Jun 2023 22:09:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685650189; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=z5Q8mdzoqgAPpj9w6VIVQmHquuECEspqe72O1F9vM9s=; b=HWElXy4kBXhgoE7lSy2KSaX3iwN0n/G7MeOte+afyA0ym7uSKyKlvOZDdP2s8uK6vLGCLW XPGAzEzBi02gqr/CcxDkZNLN4wQBu4NuaqKrPe4vZcSYVU/CQqA3BYi2CBQ4DzbBGQuEl/ Yv1HlE9Vv/qNxOXHMfZm/1IjUfvQuCA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-16-cx5refUtN4SvJtFoQKCgsw-1; Thu, 01 Jun 2023 16:09:46 -0400 X-MC-Unique: cx5refUtN4SvJtFoQKCgsw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 020DD802E58; Thu, 1 Jun 2023 20:09:46 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id 611432029F6D; Thu, 1 Jun 2023 20:09:43 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com Cc: Maxime Coquelin Subject: [PATCH v4 26/26] vhost: add VDUSE device stop Date: Thu, 1 Jun 2023 22:08:12 +0200 Message-Id: <20230601200812.672233-27-maxime.coquelin@redhat.com> In-Reply-To: <20230601200812.672233-1-maxime.coquelin@redhat.com> References: <20230601200812.672233-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds VDUSE device stop and cleanup of its virtqueues. Signed-off-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- doc/guides/rel_notes/release_23_07.rst | 6 +++ lib/vhost/vduse.c | 72 +++++++++++++++++++++++--- 2 files changed, 70 insertions(+), 8 deletions(-) diff --git a/doc/guides/rel_notes/release_23_07.rst b/doc/guides/rel_notes/release_23_07.rst index 7034fb664c..6f43e8e633 100644 --- a/doc/guides/rel_notes/release_23_07.rst +++ b/doc/guides/rel_notes/release_23_07.rst @@ -67,6 +67,12 @@ New Features Introduced ``rte_vhost_driver_set_max_queue_num()`` to be able to limit the maximum number of supported queue pairs, required for VDUSE support. +* **Added VDUSE support into Vhost library.** + + VDUSE aims at implementing vDPA devices in userspace. It can be used as an + alternative to Vhost-user when using Vhost-vDPA, but also enable providing a + virtio-net netdev to the host when using Virtio-vDPA driver. + Removed Items ------------- diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c index 699cfed9e3..f421b1cf4c 100644 --- a/lib/vhost/vduse.c +++ b/lib/vhost/vduse.c @@ -252,6 +252,44 @@ vduse_vring_setup(struct virtio_net *dev, unsigned int index) } } +static void +vduse_vring_cleanup(struct virtio_net *dev, unsigned int index) +{ + struct vhost_virtqueue *vq = dev->virtqueue[index]; + struct vduse_vq_eventfd vq_efd; + int ret; + + if (vq == dev->cvq && vq->kickfd >= 0) { + fdset_del(&vduse.fdset, vq->kickfd); + fdset_pipe_notify(&vduse.fdset); + } + + vq_efd.index = index; + vq_efd.fd = VDUSE_EVENTFD_DEASSIGN; + + ret = ioctl(dev->vduse_dev_fd, VDUSE_VQ_SETUP_KICKFD, &vq_efd); + if (ret) + VHOST_LOG_CONFIG(dev->ifname, ERR, "Failed to cleanup kickfd for VQ %u: %s\n", + index, strerror(errno)); + + close(vq->kickfd); + vq->kickfd = VIRTIO_UNINITIALIZED_EVENTFD; + + vring_invalidate(dev, vq); + + rte_free(vq->batch_copy_elems); + vq->batch_copy_elems = NULL; + + rte_free(vq->shadow_used_split); + vq->shadow_used_split = NULL; + + vq->enabled = false; + vq->ready = false; + vq->size = 0; + vq->last_used_idx = 0; + vq->last_avail_idx = 0; +} + static void vduse_device_start(struct virtio_net *dev) { @@ -304,6 +342,23 @@ vduse_device_start(struct virtio_net *dev) } } +static void +vduse_device_stop(struct virtio_net *dev) +{ + unsigned int i; + + VHOST_LOG_CONFIG(dev->ifname, INFO, "Stopping device...\n"); + + vhost_destroy_device_notify(dev); + + dev->flags &= ~VIRTIO_DEV_READY; + + for (i = 0; i < dev->nr_vring; i++) + vduse_vring_cleanup(dev, i); + + vhost_user_iotlb_flush_all(dev); +} + static void vduse_events_handler(int fd, void *arg, int *remove __rte_unused) { @@ -311,6 +366,7 @@ vduse_events_handler(int fd, void *arg, int *remove __rte_unused) struct vduse_dev_request req; struct vduse_dev_response resp; struct vhost_virtqueue *vq; + uint8_t old_status; int ret; memset(&resp, 0, sizeof(resp)); @@ -339,10 +395,15 @@ vduse_events_handler(int fd, void *arg, int *remove __rte_unused) case VDUSE_SET_STATUS: VHOST_LOG_CONFIG(dev->ifname, INFO, "\tnew status: 0x%08x\n", req.s.status); + old_status = dev->status; dev->status = req.s.status; - if (dev->status & VIRTIO_DEVICE_STATUS_DRIVER_OK) - vduse_device_start(dev); + if ((old_status ^ dev->status) & VIRTIO_DEVICE_STATUS_DRIVER_OK) { + if (dev->status & VIRTIO_DEVICE_STATUS_DRIVER_OK) + vduse_device_start(dev); + else + vduse_device_stop(dev); + } resp.result = VDUSE_REQ_RESULT_OK; break; @@ -560,12 +621,7 @@ vduse_device_destroy(const char *path) if (vid == RTE_MAX_VHOST_DEVICE) return -1; - if (dev->cvq && dev->cvq->kickfd >= 0) { - fdset_del(&vduse.fdset, dev->cvq->kickfd); - fdset_pipe_notify(&vduse.fdset); - close(dev->cvq->kickfd); - dev->cvq->kickfd = VIRTIO_UNINITIALIZED_EVENTFD; - } + vduse_device_stop(dev); fdset_del(&vduse.fdset, dev->vduse_dev_fd); fdset_pipe_notify(&vduse.fdset);