From patchwork Tue Aug 23 04:34:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Pei, Andy" X-Patchwork-Id: 115341 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A2BD0A0093; Tue, 23 Aug 2022 07:24:47 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9D0A1410F2; Tue, 23 Aug 2022 07:24:43 +0200 (CEST) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 45AFD40DDE for ; Tue, 23 Aug 2022 07:24:42 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1661232282; x=1692768282; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=OSxntA4BTV8/dX1BCMaoUhNiAfGhgvAYpHMkgnNiExc=; b=OKnb1pHqRdLnwGtXJmtWaeeBLuM3M0lg7IN4FMbdXY9KNJQH+UavdPnp mqtDqae7lEpT4xXaEHAU27/lQbAofSmLlz3nBXO1hYGM1+WNL7HTlW4Sl G4k/SUl1jyHKYu3pdP8vLv3tuZi9Wjl5gnwgpsss/1Q6+8+/RIf2Ynv8c 3CFYM1zRvhJM2qZtgMjRLVkc1XNR+Yd0nnOTHPVgA6EB5JGs1KqsJEPdZ 2ThEgFK6KWYWy+QLg+VowmfKhTu0JZYqwiHT9Y/Ooa2fAYVwD3AOdXXhQ bQIC33nhbf0sQMxYKtloTprrwzCmfyZ3FS/3WDlhHdRJJd1pbZ6sQfn+f A==; X-IronPort-AV: E=McAfee;i="6500,9779,10447"; a="355325280" X-IronPort-AV: E=Sophos;i="5.93,256,1654585200"; d="scan'208";a="355325280" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Aug 2022 22:24:41 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,256,1654585200"; d="scan'208";a="669864688" Received: from dpdk-dipei.sh.intel.com ([10.67.110.251]) by fmsmga008.fm.intel.com with ESMTP; 22 Aug 2022 22:24:39 -0700 From: Andy Pei To: dev@dpdk.org Cc: chenbo.xia@intel.com, rosen.xu@intel.com, wei.huang@intel.com, gang.cao@intel.com, maxime.coquelin@redhat.com, Huang Wei Subject: [PATCH 1/8] vdpa/ifc: add new device ID Date: Tue, 23 Aug 2022 12:34:58 +0800 Message-Id: <1661229305-240952-2-git-send-email-andy.pei@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1661229305-240952-1-git-send-email-andy.pei@intel.com> References: <1661229305-240952-1-git-send-email-andy.pei@intel.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Huang Wei Add new device id to support IFCVF_NET_TRANSITIONAL_DEVICE_ID (0x1000). Signed-off-by: Huang Wei Signed-off-by: Andy Pei --- drivers/vdpa/ifc/base/ifcvf.h | 4 +++- drivers/vdpa/ifc/ifcvf_vdpa.c | 9 ++++++++- 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/drivers/vdpa/ifc/base/ifcvf.h b/drivers/vdpa/ifc/base/ifcvf.h index 9d95aac..7ede738 100644 --- a/drivers/vdpa/ifc/base/ifcvf.h +++ b/drivers/vdpa/ifc/base/ifcvf.h @@ -12,11 +12,13 @@ #define IFCVF_BLK 1 #define IFCVF_VENDOR_ID 0x1AF4 -#define IFCVF_NET_DEVICE_ID 0x1041 +#define IFCVF_NET_MODERN_DEVICE_ID 0x1041 #define IFCVF_BLK_MODERN_DEVICE_ID 0x1042 +#define IFCVF_NET_TRANSITIONAL_DEVICE_ID 0x1000 #define IFCVF_BLK_TRANSITIONAL_DEVICE_ID 0x1001 #define IFCVF_SUBSYS_VENDOR_ID 0x8086 #define IFCVF_SUBSYS_DEVICE_ID 0x001A +#define IFCVF_NET_DEVICE_ID 0x0001 #define IFCVF_BLK_DEVICE_ID 0x0002 #define IFCVF_MAX_QUEUES 1 diff --git a/drivers/vdpa/ifc/ifcvf_vdpa.c b/drivers/vdpa/ifc/ifcvf_vdpa.c index ac42de9..61d0250 100644 --- a/drivers/vdpa/ifc/ifcvf_vdpa.c +++ b/drivers/vdpa/ifc/ifcvf_vdpa.c @@ -1684,13 +1684,20 @@ struct rte_vdpa_dev_info dev_info[] = { static const struct rte_pci_id pci_id_ifcvf_map[] = { { .class_id = RTE_CLASS_ANY_ID, .vendor_id = IFCVF_VENDOR_ID, - .device_id = IFCVF_NET_DEVICE_ID, + .device_id = IFCVF_NET_MODERN_DEVICE_ID, .subsystem_vendor_id = IFCVF_SUBSYS_VENDOR_ID, .subsystem_device_id = IFCVF_SUBSYS_DEVICE_ID, }, { .class_id = RTE_CLASS_ANY_ID, .vendor_id = IFCVF_VENDOR_ID, + .device_id = IFCVF_NET_TRANSITIONAL_DEVICE_ID, + .subsystem_vendor_id = IFCVF_SUBSYS_VENDOR_ID, + .subsystem_device_id = IFCVF_NET_DEVICE_ID, + }, + + { .class_id = RTE_CLASS_ANY_ID, + .vendor_id = IFCVF_VENDOR_ID, .device_id = IFCVF_BLK_TRANSITIONAL_DEVICE_ID, .subsystem_vendor_id = IFCVF_SUBSYS_VENDOR_ID, .subsystem_device_id = IFCVF_BLK_DEVICE_ID, From patchwork Tue Aug 23 04:34:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Pei, Andy" X-Patchwork-Id: 115342 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 719E8A0093; Tue, 23 Aug 2022 07:24:54 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7170F427ED; Tue, 23 Aug 2022 07:24:46 +0200 (CEST) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id BAF62427EC for ; Tue, 23 Aug 2022 07:24:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1661232284; x=1692768284; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=Zot4Am17jNHQAQS8iST1qki7PQWNspdi4CIszg6n2Bo=; b=egPp/Y7CzzaBT9ynGMvLybpinGGJuEmxKOtRci/Nb41MAvI9XpSk0S5Y DdYIMaCEuJE0RrV0+SyasHuRbcSS5aUt7oAgy51xsmV2OY+wPy9KwYttl DGciPRfc8eBDnCc3d9cBRn04ze4G4zWVM16qDwKkg1mVu+KftG7rUCBVO RjRxkpDRfr2jjHo17Uh0qeAKZHowsIdChIeFk5vyR38P4f/lCZ3RzJ0Aq 6jFSzeHDsiSKaUxk1Yc9uEhWVa0eKgBW+xLeCo96TBfnEHas3dKM/R6nf Y/orzqZ+mfQvqL20A4cEqzOJQanZPnHram/caboj2zGbeoTnpH7R3LCV5 g==; X-IronPort-AV: E=McAfee;i="6500,9779,10447"; a="355325286" X-IronPort-AV: E=Sophos;i="5.93,256,1654585200"; d="scan'208";a="355325286" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Aug 2022 22:24:44 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,256,1654585200"; d="scan'208";a="669864714" Received: from dpdk-dipei.sh.intel.com ([10.67.110.251]) by fmsmga008.fm.intel.com with ESMTP; 22 Aug 2022 22:24:42 -0700 From: Andy Pei To: dev@dpdk.org Cc: chenbo.xia@intel.com, rosen.xu@intel.com, wei.huang@intel.com, gang.cao@intel.com, maxime.coquelin@redhat.com, Huang Wei Subject: [PATCH 2/8] vdpa/ifc: add multi queue suppoort Date: Tue, 23 Aug 2022 12:34:59 +0800 Message-Id: <1661229305-240952-3-git-send-email-andy.pei@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1661229305-240952-1-git-send-email-andy.pei@intel.com> References: <1661229305-240952-1-git-send-email-andy.pei@intel.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Enable VHOST_USER_PROTOCOL_F_MQ feature. ExposeIFCVF_MQ_OFFSET register to enable multi queue. Signed-off-by: Andy Pei Signed-off-by: Huang Wei --- drivers/vdpa/ifc/base/ifcvf.c | 5 +++++ drivers/vdpa/ifc/base/ifcvf.h | 2 ++ drivers/vdpa/ifc/ifcvf_vdpa.c | 1 + 3 files changed, 8 insertions(+) diff --git a/drivers/vdpa/ifc/base/ifcvf.c b/drivers/vdpa/ifc/base/ifcvf.c index f1e1474..34c8226 100644 --- a/drivers/vdpa/ifc/base/ifcvf.c +++ b/drivers/vdpa/ifc/base/ifcvf.c @@ -90,6 +90,11 @@ if (!hw->lm_cfg) WARNINGOUT("HW support live migration not support!\n"); + if (hw->mem_resource[4].addr) + hw->mq_cfg = hw->mem_resource[4].addr + IFCVF_MQ_OFFSET; + else + hw->mq_cfg = NULL; + if (hw->common_cfg == NULL || hw->notify_base == NULL || hw->isr == NULL || hw->dev_cfg == NULL) { DEBUGOUT("capability incomplete\n"); diff --git a/drivers/vdpa/ifc/base/ifcvf.h b/drivers/vdpa/ifc/base/ifcvf.h index 7ede738..ad505f1 100644 --- a/drivers/vdpa/ifc/base/ifcvf.h +++ b/drivers/vdpa/ifc/base/ifcvf.h @@ -50,6 +50,7 @@ #define IFCVF_LM_CFG_SIZE 0x40 #define IFCVF_LM_RING_STATE_OFFSET 0x20 +#define IFCVF_MQ_OFFSET 0x28 #define IFCVF_LM_LOGGING_CTRL 0x0 @@ -149,6 +150,7 @@ struct ifcvf_hw { u16 *notify_base; u16 *notify_addr[IFCVF_MAX_QUEUES * 2]; u8 *lm_cfg; + u8 *mq_cfg; struct vring_info vring[IFCVF_MAX_QUEUES * 2]; u8 nr_vring; int device_type; diff --git a/drivers/vdpa/ifc/ifcvf_vdpa.c b/drivers/vdpa/ifc/ifcvf_vdpa.c index 61d0250..2d165c0 100644 --- a/drivers/vdpa/ifc/ifcvf_vdpa.c +++ b/drivers/vdpa/ifc/ifcvf_vdpa.c @@ -1248,6 +1248,7 @@ struct rte_vdpa_dev_info { 1ULL << VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD | \ 1ULL << VHOST_USER_PROTOCOL_F_HOST_NOTIFIER | \ 1ULL << VHOST_USER_PROTOCOL_F_LOG_SHMFD | \ + 1ULL << VHOST_USER_PROTOCOL_F_MQ | \ 1ULL << VHOST_USER_PROTOCOL_F_STATUS) #define VDPA_BLK_PROTOCOL_FEATURES \ From patchwork Tue Aug 23 04:35:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Pei, Andy" X-Patchwork-Id: 115343 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 09ABFA0093; Tue, 23 Aug 2022 07:25:02 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 808234281B; Tue, 23 Aug 2022 07:24:49 +0200 (CEST) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id CFFC74281B for ; Tue, 23 Aug 2022 07:24:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1661232288; x=1692768288; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=MVNbXBjBKTeHXv4sSe9YudTelWhvDBk09Y5qNYuM5N4=; b=jHC03Kpy/SXEtIWNgD1qw1K4Xg7RBfemOMnyVS/S5BK5/QaE4rnyvAjO j0UjUDa+dgTPmxNaiZvvwtZn41uwE2UlSb4TDgpmOImqJ6c0guizgbSrE B576B0aASSYM2DSgXfe3xhDAhT2x73xlDrIsM2lHh9ddtVwGF1v/WMpht Sy11WlsM4Cd2m3E9+yAKHwJloYdxEh+klTTrHYlrDo0N40N09EJEofBqW SSBiGuu+pzintpI9LcNy0DE6ho9Yzd3GKSoc/VU4mCLf2P1n7ZCkxtkp/ 8RsXhZrI5Gw7ctLu0J3GsjNAb0QV85dVr82Qwu3syderuwGKVqtp6p9ZZ w==; X-IronPort-AV: E=McAfee;i="6500,9779,10447"; a="355325298" X-IronPort-AV: E=Sophos;i="5.93,256,1654585200"; d="scan'208";a="355325298" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Aug 2022 22:24:47 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,256,1654585200"; d="scan'208";a="669864735" Received: from dpdk-dipei.sh.intel.com ([10.67.110.251]) by fmsmga008.fm.intel.com with ESMTP; 22 Aug 2022 22:24:45 -0700 From: Andy Pei To: dev@dpdk.org Cc: chenbo.xia@intel.com, rosen.xu@intel.com, wei.huang@intel.com, gang.cao@intel.com, maxime.coquelin@redhat.com, Huang Wei Subject: [PATCH 3/8] vdpa/ifc: set max queues according to HW spec Date: Tue, 23 Aug 2022 12:35:00 +0800 Message-Id: <1661229305-240952-4-git-send-email-andy.pei@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1661229305-240952-1-git-send-email-andy.pei@intel.com> References: <1661229305-240952-1-git-send-email-andy.pei@intel.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Set max_queues according to virtio HW spec. For virtio BLK device, set max_queues to the value of "num_queues". "num_queues" is element of struct virtio_blk_config. Signed-off-by: Andy Pei Signed-off-by: Huang Wei --- drivers/vdpa/ifc/base/ifcvf.h | 2 +- drivers/vdpa/ifc/ifcvf_vdpa.c | 18 +++++++++++++++++- 2 files changed, 18 insertions(+), 2 deletions(-) diff --git a/drivers/vdpa/ifc/base/ifcvf.h b/drivers/vdpa/ifc/base/ifcvf.h index ad505f1..c17bf2a 100644 --- a/drivers/vdpa/ifc/base/ifcvf.h +++ b/drivers/vdpa/ifc/base/ifcvf.h @@ -21,7 +21,7 @@ #define IFCVF_NET_DEVICE_ID 0x0001 #define IFCVF_BLK_DEVICE_ID 0x0002 -#define IFCVF_MAX_QUEUES 1 +#define IFCVF_MAX_QUEUES 32 #ifndef VIRTIO_F_IOMMU_PLATFORM #define VIRTIO_F_IOMMU_PLATFORM 33 diff --git a/drivers/vdpa/ifc/ifcvf_vdpa.c b/drivers/vdpa/ifc/ifcvf_vdpa.c index 2d165c0..34aea6c 100644 --- a/drivers/vdpa/ifc/ifcvf_vdpa.c +++ b/drivers/vdpa/ifc/ifcvf_vdpa.c @@ -26,6 +26,18 @@ #include "base/ifcvf.h" +/** +** RTE_MAX() and RTE_MIN() cannot be used since braced-group within +** expression allowed only inside a function, but MAX() is used as +** a number of elements in array. +**/ +#ifndef MAX +#define MAX(v1, v2) ((v1) > (v2) ? (v1) : (v2)) +#endif +#ifndef MIN +#define MIN(v1, v2) ((v1) < (v2) ? (v1) : (v2)) +#endif + RTE_LOG_REGISTER(ifcvf_vdpa_logtype, pmd.vdpa.ifcvf, NOTICE); #define DRV_LOG(level, fmt, args...) \ rte_log(RTE_LOG_ ## level, ifcvf_vdpa_logtype, \ @@ -1559,7 +1571,6 @@ struct rte_vdpa_dev_info dev_info[] = { } internal->configured = 0; - internal->max_queues = IFCVF_MAX_QUEUES; features = ifcvf_get_features(&internal->hw); device_id = ifcvf_pci_get_device_type(pci_dev); @@ -1570,6 +1581,8 @@ struct rte_vdpa_dev_info dev_info[] = { if (device_id == VIRTIO_ID_NET) { internal->hw.device_type = IFCVF_NET; + internal->max_queues = MIN(IFCVF_MAX_QUEUES, + (internal->hw.common_cfg->num_queues - 1)/2); internal->features = features & ~(1ULL << VIRTIO_F_IOMMU_PLATFORM); internal->features |= dev_info[IFCVF_NET].features; @@ -1609,6 +1622,9 @@ struct rte_vdpa_dev_info dev_info[] = { internal->hw.blk_cfg->geometry.sectors); DRV_LOG(DEBUG, "num_queues: 0x%08x", internal->hw.blk_cfg->num_queues); + + internal->max_queues = MIN(IFCVF_MAX_QUEUES, + internal->hw.blk_cfg->num_queues); } list->internal = internal; From patchwork Tue Aug 23 04:35:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Pei, Andy" X-Patchwork-Id: 115344 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C07BFA0093; Tue, 23 Aug 2022 07:25:07 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 57D4D427F6; Tue, 23 Aug 2022 07:24:52 +0200 (CEST) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id AC0F3427EC for ; Tue, 23 Aug 2022 07:24:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1661232291; x=1692768291; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=Lg21sk7WwAa4CwaYlMhDdhlRhVL+rg6motdShcLYfKI=; b=YsRW+9pBC73HPIE3wbnATr/c/dlkXd8jVfNYt/9IRZuM5hHb7xlsapUx ERGEm8nwf3zq6rbQRU8DjkpWmuqNuSaA4XiZ7UsSg2lb24aOHgA0uZ6wW QiQML0WNStgu2scTGIo1GeseJOe30E4OdXlgz0Z5w9o0bsPPc52+9YeqN Gpg2hAIrRj7dGEBckkVNgjnwl/i42vk+RYlfl8ucR4DNBVn34w2Pn1h/Q 2Yw26PA+Qh+TO92k2fRYlE3pdKe96GMBcbhLxeeN398114JTlVphTyUr1 PT8RmUsgax7MhWivPBujfNR8ZHfnYXFANinw2WD39hHf9bRGmsOrDEm25 w==; X-IronPort-AV: E=McAfee;i="6500,9779,10447"; a="355325306" X-IronPort-AV: E=Sophos;i="5.93,256,1654585200"; d="scan'208";a="355325306" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Aug 2022 22:24:51 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,256,1654585200"; d="scan'208";a="669864746" Received: from dpdk-dipei.sh.intel.com ([10.67.110.251]) by fmsmga008.fm.intel.com with ESMTP; 22 Aug 2022 22:24:49 -0700 From: Andy Pei To: dev@dpdk.org Cc: chenbo.xia@intel.com, rosen.xu@intel.com, wei.huang@intel.com, gang.cao@intel.com, maxime.coquelin@redhat.com, Huang Wei Subject: [PATCH 4/8] vdpa/ifc: write queue count to MQ register Date: Tue, 23 Aug 2022 12:35:01 +0800 Message-Id: <1661229305-240952-5-git-send-email-andy.pei@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1661229305-240952-1-git-send-email-andy.pei@intel.com> References: <1661229305-240952-1-git-send-email-andy.pei@intel.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Write queue count to IFCVF_MQ_OFFSET register to enable multi queue feature. Signed-off-by: Andy Pei Signed-off-by: Huang Wei --- drivers/vdpa/ifc/base/ifcvf.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/drivers/vdpa/ifc/base/ifcvf.c b/drivers/vdpa/ifc/base/ifcvf.c index 34c8226..1b50df6 100644 --- a/drivers/vdpa/ifc/base/ifcvf.c +++ b/drivers/vdpa/ifc/base/ifcvf.c @@ -198,6 +198,19 @@ IFCVF_WRITE_REG32(val >> 32, hi); } +STATIC void +ifcvf_enable_multiqueue(struct ifcvf_hw *hw, u16 nr_queue_pair) +{ + u8 *mq_cfg; + + if (hw->device_type == IFCVF_NET) + nr_queue_pair = (nr_queue_pair + 1) / 2; + + mq_cfg = hw->mq_cfg; + if (mq_cfg) + *(u32 *)mq_cfg = nr_queue_pair; +} + STATIC int ifcvf_hw_enable(struct ifcvf_hw *hw) { @@ -215,6 +228,7 @@ return -1; } + ifcvf_enable_multiqueue(hw, hw->nr_vring); for (i = 0; i < hw->nr_vring; i++) { IFCVF_WRITE_REG16(i, &cfg->queue_select); io_write64_twopart(hw->vring[i].desc, &cfg->queue_desc_lo, From patchwork Tue Aug 23 04:35:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Pei, Andy" X-Patchwork-Id: 115345 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1C265A0093; Tue, 23 Aug 2022 07:25:13 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1186B427F5; Tue, 23 Aug 2022 07:24:57 +0200 (CEST) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 4084442829 for ; Tue, 23 Aug 2022 07:24:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1661232295; x=1692768295; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=Sc/sPWJi9OSN3YevCIG004RgnfnYrh6TbhSvcoAVYVI=; b=SetI3LK+6kfIckSmBTag4vg33/f1z+M7HA7kQ0pbLklyI/R6UjnID2MF jjkbAwC4RDfvksaWAjoSJf0DCDH14MlCbl8nSYE447h8e87oRM6gKACkv EBlyNU+SoOwyr8UvwebqMUWAgtZXoCzXMNLiBMsERa+/DCK6aKhAA68ll Clztl6rUPBvN3OrCKNAptnoxGB0Cx1wYyQwpUic+FcTPPrPv+5xN6RWeL WWPa2niCrIWVqrwrOKLNzXVtVYJlgWuBjCr5A4rM0zFqqxDnP5Iq7kUEF lIgTsb27N543ZVtQ3rYJ5ITwZqndOA5ULa9nNwyiRzqftdFWfbCnQug19 Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10447"; a="355325316" X-IronPort-AV: E=Sophos;i="5.93,256,1654585200"; d="scan'208";a="355325316" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Aug 2022 22:24:54 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,256,1654585200"; d="scan'208";a="669864760" Received: from dpdk-dipei.sh.intel.com ([10.67.110.251]) by fmsmga008.fm.intel.com with ESMTP; 22 Aug 2022 22:24:53 -0700 From: Andy Pei To: dev@dpdk.org Cc: chenbo.xia@intel.com, rosen.xu@intel.com, wei.huang@intel.com, gang.cao@intel.com, maxime.coquelin@redhat.com Subject: [PATCH 5/8] vdpa/ifc: only configure enabled queue Date: Tue, 23 Aug 2022 12:35:02 +0800 Message-Id: <1661229305-240952-6-git-send-email-andy.pei@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1661229305-240952-1-git-send-email-andy.pei@intel.com> References: <1661229305-240952-1-git-send-email-andy.pei@intel.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org when configure the hardware queue, we only configure queues which have been enabled by vhost. Signed-off-by: Andy Pei --- drivers/vdpa/ifc/base/ifcvf.c | 5 ++++- drivers/vdpa/ifc/ifcvf_vdpa.c | 4 ++++ 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/drivers/vdpa/ifc/base/ifcvf.c b/drivers/vdpa/ifc/base/ifcvf.c index 1b50df6..ca5f677 100644 --- a/drivers/vdpa/ifc/base/ifcvf.c +++ b/drivers/vdpa/ifc/base/ifcvf.c @@ -230,6 +230,8 @@ ifcvf_enable_multiqueue(hw, hw->nr_vring); for (i = 0; i < hw->nr_vring; i++) { + if (!hw->vring[i].enable) + continue; IFCVF_WRITE_REG16(i, &cfg->queue_select); io_write64_twopart(hw->vring[i].desc, &cfg->queue_desc_lo, &cfg->queue_desc_hi); @@ -264,7 +266,8 @@ notify_off = IFCVF_READ_REG16(&cfg->queue_notify_off); hw->notify_addr[i] = (void *)((u8 *)hw->notify_base + notify_off * hw->notify_off_multiplier); - IFCVF_WRITE_REG16(1, &cfg->queue_enable); + if (hw->vring[i].enable) + IFCVF_WRITE_REG16(1, &cfg->queue_enable); } return 0; diff --git a/drivers/vdpa/ifc/ifcvf_vdpa.c b/drivers/vdpa/ifc/ifcvf_vdpa.c index 34aea6c..a62bcec 100644 --- a/drivers/vdpa/ifc/ifcvf_vdpa.c +++ b/drivers/vdpa/ifc/ifcvf_vdpa.c @@ -290,6 +290,8 @@ struct rte_vdpa_dev_info { rte_vhost_get_negotiated_features(vid, &hw->req_features); for (i = 0; i < nr_vring; i++) { + if (!hw->vring[i].enable) + continue; rte_vhost_get_vhost_vring(vid, i, &vq); gpa = hva_to_gpa(vid, (uint64_t)(uintptr_t)vq.desc); if (gpa == 0) { @@ -505,6 +507,8 @@ struct rte_vdpa_dev_info { vring.kickfd = -1; for (qid = 0; qid < q_num; qid++) { + if (!hw->vring[qid].enable) + continue; ev.events = EPOLLIN | EPOLLPRI; rte_vhost_get_vhost_vring(internal->vid, qid, &vring); ev.data.u64 = qid | (uint64_t)vring.kickfd << 32; From patchwork Tue Aug 23 04:35:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Pei, Andy" X-Patchwork-Id: 115346 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9C6CCA0093; Tue, 23 Aug 2022 07:25:18 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 04329427EC; Tue, 23 Aug 2022 07:25:00 +0200 (CEST) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 7BE664282D for ; Tue, 23 Aug 2022 07:24:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1661232298; x=1692768298; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=5AXOpeePn1UowgYWzlVi6v5rkeYV+FgRpf/ICfZ+F4U=; b=JHuxwm6vznsDGceqjU6BJNZcfxaIfOry9Ez/Lvm+JaV54Cl2RgFgh9Up wB8+VJZtOuxDW1yoTjZ0it+ghhGbt3R1xn8DJrquOLtOj4Ut2ECDAoqnZ SWpIzJEKo4l1VuuTqcG3ecJw+mattWAt3rcq/MMhEpvgzrzSI8O4ZF1Jq rGVzBXZwudKLWMgmspSOLtlolsG38eRO3/DUugJFOee2m/WBB2wYeOYyy vetRmDufAwg5RszDx2F7FDl0+cSFeaekUzJj23fddtBNYJ27hnyK8AbgS maOLiWux74lYAEiFY6Zq2azr/Avj+H2RACCKBry5fGpNhbDFl7GzI6wjF g==; X-IronPort-AV: E=McAfee;i="6500,9779,10447"; a="355325322" X-IronPort-AV: E=Sophos;i="5.93,256,1654585200"; d="scan'208";a="355325322" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Aug 2022 22:24:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,256,1654585200"; d="scan'208";a="669864781" Received: from dpdk-dipei.sh.intel.com ([10.67.110.251]) by fmsmga008.fm.intel.com with ESMTP; 22 Aug 2022 22:24:56 -0700 From: Andy Pei To: dev@dpdk.org Cc: chenbo.xia@intel.com, rosen.xu@intel.com, wei.huang@intel.com, gang.cao@intel.com, maxime.coquelin@redhat.com Subject: [PATCH 6/8] vdpa/ifc: set vring state callback update data path Date: Tue, 23 Aug 2022 12:35:03 +0800 Message-Id: <1661229305-240952-7-git-send-email-andy.pei@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1661229305-240952-1-git-send-email-andy.pei@intel.com> References: <1661229305-240952-1-git-send-email-andy.pei@intel.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org To support multi queue, in the case that first queue is ready and device is configured, when more queues need to be configured, we just close and restart data path. This also fix the situation that using set_vring_state callback to disable one queue will cause all vfio interrupts being disabled. Signed-off-by: Andy Pei --- drivers/vdpa/ifc/ifcvf_vdpa.c | 27 +++++++++------------------ 1 file changed, 9 insertions(+), 18 deletions(-) diff --git a/drivers/vdpa/ifc/ifcvf_vdpa.c b/drivers/vdpa/ifc/ifcvf_vdpa.c index a62bcec..94c8ef1 100644 --- a/drivers/vdpa/ifc/ifcvf_vdpa.c +++ b/drivers/vdpa/ifc/ifcvf_vdpa.c @@ -1286,8 +1286,6 @@ struct rte_vdpa_dev_info { struct internal_list *list; struct ifcvf_internal *internal; struct ifcvf_hw *hw; - struct ifcvf_pci_common_cfg *cfg; - int ret = 0; vdev = rte_vhost_get_vdpa_device(vid); list = find_internal_resource_by_vdev(vdev); @@ -1303,27 +1301,20 @@ struct rte_vdpa_dev_info { } hw = &internal->hw; - if (!internal->configured) - goto exit; - cfg = hw->common_cfg; - IFCVF_WRITE_REG16(vring, &cfg->queue_select); - IFCVF_WRITE_REG16(!!state, &cfg->queue_enable); + hw->vring[vring].enable = !!state; - if (!state && hw->vring[vring].enable) { - ret = vdpa_disable_vfio_intr(internal); - if (ret) - return ret; - } + if (!internal->configured) + goto exit; - if (state && !hw->vring[vring].enable) { - ret = vdpa_enable_vfio_intr(internal, false); - if (ret) - return ret; - } + /* close data path */ + rte_atomic32_set(&internal->dev_attached, 0); + update_datapath(internal); + /* restart data path */ + rte_atomic32_set(&internal->dev_attached, 1); + update_datapath(internal); exit: - hw->vring[vring].enable = !!state; return 0; } From patchwork Tue Aug 23 04:35:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Pei, Andy" X-Patchwork-Id: 115347 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CEFC3A0093; Tue, 23 Aug 2022 07:25:23 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E164742905; Tue, 23 Aug 2022 07:25:02 +0200 (CEST) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 05A454284D for ; Tue, 23 Aug 2022 07:25:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1661232301; x=1692768301; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=B5lziUqtBtLk67ML+WiHfj0sJ1taOXpoF1Gt2nBE048=; b=aCXSuw5Ve4nPHQ9h0MpBd+NUp47uRs19bOhrT7mO5C6DO4K3iLVXAapb vu9TZY05Pxq09cfqhnANWTxOn8wFv6SR4qvlratHvbvsLzz1x8dKu+WOn +W9Kwlivff3kmqy/iA6E9yD296L4u+MYNSRq1e5RnjFnZXAnQP6rXIt9h Dbv9ms3WjXEEF+yUVHrkfk4kj+iIAT7Qfa/zQ3X12vQ3euH1BwmbZmF60 azvE43w+zs41teo7keFFJtGCE9DFWcud3MIMpUFgOdMPo5IjA3pIXQup8 yPjyRQToweCbiGJNhExAXPGN78KnlbpzAEYEeG225hQYOKLA94DaSPT25 g==; X-IronPort-AV: E=McAfee;i="6500,9779,10447"; a="355325329" X-IronPort-AV: E=Sophos;i="5.93,256,1654585200"; d="scan'208";a="355325329" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Aug 2022 22:25:00 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,256,1654585200"; d="scan'208";a="669864787" Received: from dpdk-dipei.sh.intel.com ([10.67.110.251]) by fmsmga008.fm.intel.com with ESMTP; 22 Aug 2022 22:24:59 -0700 From: Andy Pei To: dev@dpdk.org Cc: chenbo.xia@intel.com, rosen.xu@intel.com, wei.huang@intel.com, gang.cao@intel.com, maxime.coquelin@redhat.com Subject: [PATCH 7/8] vhost: configure device when any queue is ready for BLK device Date: Tue, 23 Aug 2022 12:35:04 +0800 Message-Id: <1661229305-240952-8-git-send-email-andy.pei@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1661229305-240952-1-git-send-email-andy.pei@intel.com> References: <1661229305-240952-1-git-send-email-andy.pei@intel.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org When boot from virtio blk device, seabois in QEMU only enables one queue. To work in this scenario, vDPA BLK device back-end conf_dev when any queue is ready. Signed-off-by: Andy Pei --- lib/vhost/vhost_user.c | 56 +++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 44 insertions(+), 12 deletions(-) diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index 4ad28ba..b65fba3 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -1451,6 +1451,25 @@ #define VIRTIO_BUILTIN_NUM_VQS_TO_BE_READY 2u static int +virtio_has_queue_ready(struct virtio_net *dev) +{ + struct vhost_virtqueue *vq; + uint32_t i, nr_vring = dev->nr_vring; + + if (!dev->nr_vring) + return 0; + + for (i = 0; i < nr_vring; i++) { + vq = dev->virtqueue[i]; + + if (vq_is_ready(dev, vq)) + return 1; + } + + return 0; +} + +static int virtio_is_ready(struct virtio_net *dev) { struct vhost_virtqueue *vq; @@ -3167,9 +3186,33 @@ static int is_vring_iotlb(struct virtio_net *dev, if (unlock_required) vhost_user_unlock_all_queue_pairs(dev); - if (ret != 0 || !virtio_is_ready(dev)) + if (ret != 0) goto out; + vdpa_dev = dev->vdpa_dev; + if (vdpa_dev) { + if (vdpa_dev->ops->get_dev_type) { + ret = vdpa_dev->ops->get_dev_type(vdpa_dev, &vdpa_type); + if (ret) { + VHOST_LOG_CONFIG(dev->ifname, ERR, + "failed to get vdpa dev type.\n"); + ret = -1; + goto out; + } + } else { + vdpa_type = RTE_VHOST_VDPA_DEVICE_TYPE_NET; + } + } + + if (!virtio_is_ready(dev)) { + if (vdpa_type == RTE_VHOST_VDPA_DEVICE_TYPE_BLK) { + if (!virtio_has_queue_ready(dev)) + goto out; + } else { + goto out; + } + } + /* * Virtio is now ready. If not done already, it is time * to notify the application it can process the rings and @@ -3181,20 +3224,9 @@ static int is_vring_iotlb(struct virtio_net *dev, dev->flags |= VIRTIO_DEV_RUNNING; } - vdpa_dev = dev->vdpa_dev; if (!vdpa_dev) goto out; - if (vdpa_dev->ops->get_dev_type) { - ret = vdpa_dev->ops->get_dev_type(vdpa_dev, &vdpa_type); - if (ret) { - VHOST_LOG_CONFIG(dev->ifname, ERR, "failed to get vdpa dev type.\n"); - ret = -1; - goto out; - } - } else { - vdpa_type = RTE_VHOST_VDPA_DEVICE_TYPE_NET; - } if (vdpa_type == RTE_VHOST_VDPA_DEVICE_TYPE_BLK && request != VHOST_USER_SET_VRING_CALL) goto out; From patchwork Tue Aug 23 04:35:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Pei, Andy" X-Patchwork-Id: 115348 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4E9CEA0093; Tue, 23 Aug 2022 07:25:29 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CE31942B70; Tue, 23 Aug 2022 07:25:04 +0200 (CEST) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 724BA42B6F for ; Tue, 23 Aug 2022 07:25:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1661232303; x=1692768303; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=58gDnf8ey0Q8kguTYs8CrVnN3GCNq7cMGJ9BV4Hrs1w=; b=nTHoRhfYGV/TyRnH0csKjN+sSxI/R/M4pu5bwVKYBjM0sQbcGe//TQDx cMMoYSCf7ioY4m8Sg92nHhQpcW+c2SxDIlfjiirZi4FkdSA1bjEHCC0wb tN/ho+1DzoprJA1o58gsuw+1I/mmRxGKLAkzI1iTkEaWxHH5oVb32eDWY oiBSxMlZ60t/HyPvUjACw0yR3r+yArLuoUB2O/WAiv9LQ8p9EatnUapjF UQ/QJnxRHy0vyMnIyXO24GwSs1+C41F8wPF1QoohQTawXrXGcjYcUpqi+ EFI6cSuwAmLb4GIQqVtEBPD9t0fxqDmmk+F12WcgbUxnAyx0LucxMSCH0 Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10447"; a="355325335" X-IronPort-AV: E=Sophos;i="5.93,256,1654585200"; d="scan'208";a="355325335" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Aug 2022 22:25:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,256,1654585200"; d="scan'208";a="669864808" Received: from dpdk-dipei.sh.intel.com ([10.67.110.251]) by fmsmga008.fm.intel.com with ESMTP; 22 Aug 2022 22:25:01 -0700 From: Andy Pei To: dev@dpdk.org Cc: chenbo.xia@intel.com, rosen.xu@intel.com, wei.huang@intel.com, gang.cao@intel.com, maxime.coquelin@redhat.com Subject: [PATCH 8/8] vhost: vDPA BLK devices configure device when all queue callfds are set Date: Tue, 23 Aug 2022 12:35:05 +0800 Message-Id: <1661229305-240952-9-git-send-email-andy.pei@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1661229305-240952-1-git-send-email-andy.pei@intel.com> References: <1661229305-240952-1-git-send-email-andy.pei@intel.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org In the virtio blk vDPA live migration use case, before the live migration process, QEMU will set call fd to vDPA back-end. QEMU and vDPA back-end stand by until live migration starts. During live migration process, QEMU sets kick fd and a new call fd. However, after the kick fd is set to the vDPA back-end, the vDPA back-end configures device and data path starts. The new call fd will cause some kind of "re-configuration", this kind of "re-configuration" cause IO drop. After this patch, vDPA back-end configures device after kick fd and call fd are well set and make sure no IO drops. This patch only impact virtio blk vDPA device and does not impact net device. Signed-off-by: Andy Pei --- lib/vhost/vhost_user.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index b65fba3..568030a 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -2994,6 +2994,7 @@ static int is_vring_iotlb(struct virtio_net *dev, uint32_t vdpa_type = 0; uint32_t request; uint32_t i; + uint16_t blk_call_fd; dev = get_device(vid); if (dev == NULL) @@ -3227,9 +3228,15 @@ static int is_vring_iotlb(struct virtio_net *dev, if (!vdpa_dev) goto out; - if (vdpa_type == RTE_VHOST_VDPA_DEVICE_TYPE_BLK - && request != VHOST_USER_SET_VRING_CALL) - goto out; + if (vdpa_type == RTE_VHOST_VDPA_DEVICE_TYPE_BLK) { + if (request == VHOST_USER_SET_VRING_CALL) { + blk_call_fd = ctx.msg.payload.u64 & VHOST_USER_VRING_IDX_MASK; + if (blk_call_fd != dev->nr_vring - 1) + goto out; + } else { + goto out; + } + } if (!(dev->flags & VIRTIO_DEV_VDPA_CONFIGURED)) { if (vdpa_dev->ops->dev_conf(dev->vid))