From patchwork Fri Aug 27 16:12:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 97469 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0F72FA0C54; Fri, 27 Aug 2021 18:12:46 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 933944067C; Fri, 27 Aug 2021 18:12:45 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 34D8140150 for ; Fri, 27 Aug 2021 18:12:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630080763; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=BxLeIxe+MPetr4fS6B0k3aKinJvGateeoipmaUYIdhc=; b=SZarJKzNjgLawSRH5onD4aeB8JIDTJl4Nx6ijxVAcqMmTq5dNM7YLZtjvQPDxDN0zaMOZy JBj4Vw2M2+AzGLIqu/+3BG8FmH4mhKo94muT46CwthRVj+3uVRM5ClYBr6JmjhGWaaWugQ NKJN3hdw7gZGwz52m4k30lCS17kqDXE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-76-UX0WTZJHPZyhL2zcB71ZqQ-1; Fri, 27 Aug 2021 12:12:42 -0400 X-MC-Unique: UX0WTZJHPZyhL2zcB71ZqQ-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3F9D8107ACF5; Fri, 27 Aug 2021 16:12:41 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.236]) by smtp.corp.redhat.com (Postfix) with ESMTP id DD86C19C46; Fri, 27 Aug 2021 16:12:36 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: Maxime Coquelin , Chenbo Xia Cc: dev@dpdk.org, Pei Zhang , Jason Wang Date: Fri, 27 Aug 2021 18:12:31 +0200 Message-Id: <20210827161231.579968-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [PATCH] vhost: Clean iotlb cache on vring stop X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Old IOVA cache entries are left when there is a change on virtio driver in VM. In case that all these old entries have iova addresses lesser than new iova entries, vhost code will need to iterate all the cache to find the new ones. In case of just a new iova entry needed for the new translations, this condition will last forever. This has been observed in virtio-net to testpmd's vfio-pci driver transition, reducing the performance from more than 10Mpps to less than 0.07Mpps if the hugepage address was higher than the networking buffers. Since all new buffers are contained in this new gigantic page, vhost needs to scan IOTLB_CACHE_SIZE - 1 for each translation at worst. Signed-off-by: Eugenio Pérez Reported-by: Pei Zhang Reviewed-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/vhost/vhost_user.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index 29a4c9af60..7de48f5333 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -2113,6 +2113,8 @@ vhost_user_get_vring_base(struct virtio_net **pdev, msg->size = sizeof(msg->payload.state); msg->fd_num = 0; + vhost_user_iotlb_flush_all(vq); + vring_invalidate(dev, vq); return RTE_VHOST_MSG_RESULT_REPLY;