From patchwork Thu Jun 25 13:38:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matan Azrad X-Patchwork-Id: 72188 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id F1EFFA0350; Thu, 25 Jun 2020 15:38:30 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 476241B75C; Thu, 25 Jun 2020 15:38:29 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 0E1D01B5E1 for ; Thu, 25 Jun 2020 15:38:26 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from matan@mellanox.com) with SMTP; 25 Jun 2020 16:38:23 +0300 Received: from pegasus25.mtr.labs.mlnx. (pegasus25.mtr.labs.mlnx [10.210.16.10]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 05PDcK3v009858; Thu, 25 Jun 2020 16:38:23 +0300 From: Matan Azrad To: Maxime Coquelin Cc: dev@dpdk.org, Xiao Wang Date: Thu, 25 Jun 2020 13:38:14 +0000 Message-Id: <1593092298-52257-2-git-send-email-matan@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593092298-52257-1-git-send-email-matan@mellanox.com> References: <1592497686-433697-1-git-send-email-matan@mellanox.com> <1593092298-52257-1-git-send-email-matan@mellanox.com> Subject: [dpdk-dev] [PATCH v2 1/5] vhost: skip access lock when vDPA is configured X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" No need to take access lock in the vhost-user message handler when vDPA driver controls all the data-path of the vhost device. It allows the vDPA set_vring_state operation callback to configure guest notifications. Signed-off-by: Matan Azrad Reviewed-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/librte_vhost/vhost_user.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c index 4e1af91..8d8050b 100644 --- a/lib/librte_vhost/vhost_user.c +++ b/lib/librte_vhost/vhost_user.c @@ -2690,8 +2690,10 @@ typedef int (*vhost_message_handler_t)(struct virtio_net **pdev, case VHOST_USER_SEND_RARP: case VHOST_USER_NET_SET_MTU: case VHOST_USER_SET_SLAVE_REQ_FD: - vhost_user_lock_all_queue_pairs(dev); - unlock_required = 1; + if (!(dev->flags & VIRTIO_DEV_VDPA_CONFIGURED)) { + vhost_user_lock_all_queue_pairs(dev); + unlock_required = 1; + } break; default: break; From patchwork Thu Jun 25 13:38:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matan Azrad X-Patchwork-Id: 72189 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 84692A0350; Thu, 25 Jun 2020 15:38:38 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 698F31BE80; Thu, 25 Jun 2020 15:38:31 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 1BCF21B6B4 for ; Thu, 25 Jun 2020 15:38:27 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from matan@mellanox.com) with SMTP; 25 Jun 2020 16:38:26 +0300 Received: from pegasus25.mtr.labs.mlnx. (pegasus25.mtr.labs.mlnx [10.210.16.10]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 05PDcK3w009858; Thu, 25 Jun 2020 16:38:26 +0300 From: Matan Azrad To: Maxime Coquelin Cc: dev@dpdk.org, Xiao Wang Date: Thu, 25 Jun 2020 13:38:15 +0000 Message-Id: <1593092298-52257-3-git-send-email-matan@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593092298-52257-1-git-send-email-matan@mellanox.com> References: <1592497686-433697-1-git-send-email-matan@mellanox.com> <1593092298-52257-1-git-send-email-matan@mellanox.com> Subject: [dpdk-dev] [PATCH v2 2/5] vhost: improve device readiness notifications X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Some guest drivers may not configure disabled virtio queues. In this case, the vhost management never notifies the application and the vDPA device readiness because it waits to the device to be ready. The current ready state means that all the virtio queues should be configured regardless the enablement status. In order to support this case, this patch changes the ready state: The device is ready when at least 1 queue pair is configured and enabled. So, now, the application and vDPA driver are notifies when the first queue pair is configured and enabled. Also the queue notifications will be triggered according to the new ready definition. Signed-off-by: Matan Azrad Reviewed-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/librte_vhost/vhost.h | 1 + lib/librte_vhost/vhost_user.c | 55 +++++++++++++++++++++++++++++-------------- 2 files changed, 38 insertions(+), 18 deletions(-) diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h index 17f1e9a..8a74f33 100644 --- a/lib/librte_vhost/vhost.h +++ b/lib/librte_vhost/vhost.h @@ -151,6 +151,7 @@ struct vhost_virtqueue { int backend; int enabled; int access_ok; + int ready; rte_spinlock_t access_lock; /* Used to notify the guest (trigger interrupt) */ diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c index 8d8050b..b90fc78 100644 --- a/lib/librte_vhost/vhost_user.c +++ b/lib/librte_vhost/vhost_user.c @@ -228,6 +228,21 @@ dev->postcopy_listening = 0; } +static void +vhost_user_notify_queue_state(struct virtio_net *dev, uint16_t index, + int enable) +{ + int did = dev->vdpa_dev_id; + struct rte_vdpa_device *vdpa_dev = rte_vdpa_get_device(did); + + if (vdpa_dev && vdpa_dev->ops->set_vring_state) + vdpa_dev->ops->set_vring_state(dev->vid, index, enable); + + if (dev->notify_ops->vring_state_changed) + dev->notify_ops->vring_state_changed(dev->vid, + index, enable); +} + /* * This function just returns success at the moment unless * the device hasn't been initialised. @@ -1306,27 +1321,31 @@ return rings_ok && vq->kickfd != VIRTIO_UNINITIALIZED_EVENTFD && - vq->callfd != VIRTIO_UNINITIALIZED_EVENTFD; + vq->callfd != VIRTIO_UNINITIALIZED_EVENTFD && + vq->enabled; } +#define VIRTIO_DEV_NUM_VQS_TO_BE_READY 2u + static int virtio_is_ready(struct virtio_net *dev) { struct vhost_virtqueue *vq; uint32_t i; - if (dev->nr_vring == 0) + if (dev->nr_vring < VIRTIO_DEV_NUM_VQS_TO_BE_READY) return 0; - for (i = 0; i < dev->nr_vring; i++) { + for (i = 0; i < VIRTIO_DEV_NUM_VQS_TO_BE_READY; i++) { vq = dev->virtqueue[i]; if (!vq_is_ready(dev, vq)) return 0; } - VHOST_LOG_CONFIG(INFO, - "virtio is now ready for processing.\n"); + if (!(dev->flags & VIRTIO_DEV_RUNNING)) + VHOST_LOG_CONFIG(INFO, + "virtio is now ready for processing.\n"); return 1; } @@ -1970,8 +1989,6 @@ static int vhost_user_set_vring_err(struct virtio_net **pdev __rte_unused, struct virtio_net *dev = *pdev; int enable = (int)msg->payload.state.num; int index = (int)msg->payload.state.index; - struct rte_vdpa_device *vdpa_dev; - int did = -1; if (validate_msg_fds(msg, 0) != 0) return RTE_VHOST_MSG_RESULT_ERR; @@ -1980,15 +1997,6 @@ static int vhost_user_set_vring_err(struct virtio_net **pdev __rte_unused, "set queue enable: %d to qp idx: %d\n", enable, index); - did = dev->vdpa_dev_id; - vdpa_dev = rte_vdpa_get_device(did); - if (vdpa_dev && vdpa_dev->ops->set_vring_state) - vdpa_dev->ops->set_vring_state(dev->vid, index, enable); - - if (dev->notify_ops->vring_state_changed) - dev->notify_ops->vring_state_changed(dev->vid, - index, enable); - /* On disable, rings have to be stopped being processed. */ if (!enable && dev->dequeue_zero_copy) drain_zmbuf_list(dev->virtqueue[index]); @@ -2618,6 +2626,7 @@ typedef int (*vhost_message_handler_t)(struct virtio_net **pdev, int unlock_required = 0; bool handled; int request; + uint32_t i; dev = get_device(vid); if (dev == NULL) @@ -2793,6 +2802,17 @@ typedef int (*vhost_message_handler_t)(struct virtio_net **pdev, return -1; } + for (i = 0; i < dev->nr_vring; i++) { + struct vhost_virtqueue *vq = dev->virtqueue[i]; + bool cur_ready = vq_is_ready(dev, vq); + + if (cur_ready != (vq && vq->ready)) { + vhost_user_notify_queue_state(dev, i, cur_ready); + vq->ready = cur_ready; + } + } + + if (!(dev->flags & VIRTIO_DEV_RUNNING) && virtio_is_ready(dev)) { dev->flags |= VIRTIO_DEV_READY; @@ -2810,8 +2830,7 @@ typedef int (*vhost_message_handler_t)(struct virtio_net **pdev, did = dev->vdpa_dev_id; vdpa_dev = rte_vdpa_get_device(did); if (vdpa_dev && virtio_is_ready(dev) && - !(dev->flags & VIRTIO_DEV_VDPA_CONFIGURED) && - msg.request.master == VHOST_USER_SET_VRING_CALL) { + !(dev->flags & VIRTIO_DEV_VDPA_CONFIGURED)) { if (vdpa_dev->ops->dev_conf) vdpa_dev->ops->dev_conf(dev->vid); dev->flags |= VIRTIO_DEV_VDPA_CONFIGURED; From patchwork Thu Jun 25 13:38:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matan Azrad X-Patchwork-Id: 72190 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A2CC7A0350; Thu, 25 Jun 2020 15:38:46 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 23D351BE9C; Thu, 25 Jun 2020 15:38:34 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 407BA1BE9C for ; Thu, 25 Jun 2020 15:38:32 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from matan@mellanox.com) with SMTP; 25 Jun 2020 16:38:29 +0300 Received: from pegasus25.mtr.labs.mlnx. (pegasus25.mtr.labs.mlnx [10.210.16.10]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 05PDcK3x009858; Thu, 25 Jun 2020 16:38:29 +0300 From: Matan Azrad To: Maxime Coquelin Cc: dev@dpdk.org, Xiao Wang Date: Thu, 25 Jun 2020 13:38:16 +0000 Message-Id: <1593092298-52257-4-git-send-email-matan@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593092298-52257-1-git-send-email-matan@mellanox.com> References: <1592497686-433697-1-git-send-email-matan@mellanox.com> <1593092298-52257-1-git-send-email-matan@mellanox.com> Subject: [dpdk-dev] [PATCH v2 3/5] vhost: handle memory hotplug with vDPA devices X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Some vDPA drivers' basic configurations should be updated when the guest memory is hotplugged. Close vDPA device before hotplug operation and recreate it after the hotplug operation is done. Signed-off-by: Matan Azrad Reviewed-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/librte_vhost/vhost_user.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c index b90fc78..f690fdb 100644 --- a/lib/librte_vhost/vhost_user.c +++ b/lib/librte_vhost/vhost_user.c @@ -1073,6 +1073,15 @@ } if (dev->mem) { + if (dev->flags & VIRTIO_DEV_VDPA_CONFIGURED) { + int did = dev->vdpa_dev_id; + struct rte_vdpa_device *vdpa_dev = + rte_vdpa_get_device(did); + + if (vdpa_dev && vdpa_dev->ops->dev_close) + vdpa_dev->ops->dev_close(dev->vid); + dev->flags &= ~VIRTIO_DEV_VDPA_CONFIGURED; + } free_mem_region(dev); rte_free(dev->mem); dev->mem = NULL; From patchwork Thu Jun 25 13:38:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matan Azrad X-Patchwork-Id: 72192 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5F853A0350; Thu, 25 Jun 2020 15:39:02 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C03F41BECC; Thu, 25 Jun 2020 15:38:39 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 1C3641BEB3 for ; Thu, 25 Jun 2020 15:38:37 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from matan@mellanox.com) with SMTP; 25 Jun 2020 16:38:31 +0300 Received: from pegasus25.mtr.labs.mlnx. (pegasus25.mtr.labs.mlnx [10.210.16.10]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 05PDcK40009858; Thu, 25 Jun 2020 16:38:31 +0300 From: Matan Azrad To: Maxime Coquelin Cc: dev@dpdk.org, Xiao Wang Date: Thu, 25 Jun 2020 13:38:17 +0000 Message-Id: <1593092298-52257-5-git-send-email-matan@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593092298-52257-1-git-send-email-matan@mellanox.com> References: <1592497686-433697-1-git-send-email-matan@mellanox.com> <1593092298-52257-1-git-send-email-matan@mellanox.com> Subject: [dpdk-dev] [PATCH v2 4/5] vhost: notify virtq file descriptor update X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When virtq call or kick file descriptors are changed in the device configuration when the queue is ready, the application and the vDPA driver should be notified to be aligned to the new file descriptors. Notify the state to be disabled before the file descriptor update and return it back to be enabled after the update. Signed-off-by: Matan Azrad Reviewed-by: Maxime Coquelin Reviewed-by: Chenbo Xia --- lib/librte_vhost/vhost_user.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c index f690fdb..f3966b6 100644 --- a/lib/librte_vhost/vhost_user.c +++ b/lib/librte_vhost/vhost_user.c @@ -1624,6 +1624,12 @@ "vring call idx:%d file:%d\n", file.index, file.fd); vq = dev->virtqueue[file.index]; + + if (vq->ready) { + vhost_user_notify_queue_state(dev, file.index, 0); + vq->ready = 0; + } + if (vq->callfd >= 0) close(vq->callfd); @@ -1882,6 +1888,11 @@ static int vhost_user_set_vring_err(struct virtio_net **pdev __rte_unused, dev->vid, file.index, 1); } + if (vq->ready) { + vhost_user_notify_queue_state(dev, file.index, 0); + vq->ready = 0; + } + if (vq->kickfd >= 0) close(vq->kickfd); vq->kickfd = file.fd; From patchwork Thu Jun 25 13:38:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matan Azrad X-Patchwork-Id: 72191 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 071E0A0350; Thu, 25 Jun 2020 15:38:54 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 808E51BEB7; Thu, 25 Jun 2020 15:38:38 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 1CCA01BEB5 for ; Thu, 25 Jun 2020 15:38:37 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from matan@mellanox.com) with SMTP; 25 Jun 2020 16:38:33 +0300 Received: from pegasus25.mtr.labs.mlnx. (pegasus25.mtr.labs.mlnx [10.210.16.10]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 05PDcK41009858; Thu, 25 Jun 2020 16:38:33 +0300 From: Matan Azrad To: Maxime Coquelin Cc: dev@dpdk.org, Xiao Wang Date: Thu, 25 Jun 2020 13:38:18 +0000 Message-Id: <1593092298-52257-6-git-send-email-matan@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1593092298-52257-1-git-send-email-matan@mellanox.com> References: <1592497686-433697-1-git-send-email-matan@mellanox.com> <1593092298-52257-1-git-send-email-matan@mellanox.com> Subject: [dpdk-dev] [PATCH v2 5/5] vdpa/mlx5: support queue update X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Last changes in vDPA device management by vhost library may cause queue ready state update after the device configuration. So, there is chance that some queue configuration information will be known only after the device was configured. Add support to reconfigure a queue after the device configuration according to the queue state update and the configuration changes. Adjust the host notifier and the guest notification configuration to be per queue and to be applied in the enablement process. Signed-off-by: Matan Azrad Reviewed-by: Maxime Coquelin --- drivers/vdpa/mlx5/mlx5_vdpa.c | 26 ------------ drivers/vdpa/mlx5/mlx5_vdpa.h | 8 +++- drivers/vdpa/mlx5/mlx5_vdpa_virtq.c | 80 ++++++++++++++++++++++++++----------- 3 files changed, 64 insertions(+), 50 deletions(-) diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.c b/drivers/vdpa/mlx5/mlx5_vdpa.c index 51f3fe8..a2b1816 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa.c @@ -141,31 +141,6 @@ } static int -mlx5_vdpa_direct_db_prepare(struct mlx5_vdpa_priv *priv) -{ - int ret; - - if (priv->direct_notifier) { - ret = rte_vhost_host_notifier_ctrl(priv->vid, - RTE_VHOST_QUEUE_ALL, false); - if (ret != 0) { - DRV_LOG(INFO, "Direct HW notifier FD cannot be " - "destroyed for device %d: %d.", priv->vid, ret); - return -1; - } - priv->direct_notifier = 0; - } - ret = rte_vhost_host_notifier_ctrl(priv->vid, RTE_VHOST_QUEUE_ALL, - true); - if (ret != 0) - DRV_LOG(INFO, "Direct HW notifier FD cannot be configured for" - " device %d: %d.", priv->vid, ret); - else - priv->direct_notifier = 1; - return 0; -} - -static int mlx5_vdpa_features_set(int vid) { int did = rte_vhost_get_vdpa_device_id(vid); @@ -330,7 +305,6 @@ if (mlx5_vdpa_mtu_set(priv)) DRV_LOG(WARNING, "MTU cannot be set on device %d.", did); if (mlx5_vdpa_pd_create(priv) || mlx5_vdpa_mem_register(priv) || - mlx5_vdpa_direct_db_prepare(priv) || mlx5_vdpa_virtqs_prepare(priv) || mlx5_vdpa_steer_setup(priv) || mlx5_vdpa_cqe_event_setup(priv)) { mlx5_vdpa_dev_close(vid); diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.h b/drivers/vdpa/mlx5/mlx5_vdpa.h index c0228b2..d4e405a 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa.h +++ b/drivers/vdpa/mlx5/mlx5_vdpa.h @@ -73,11 +73,18 @@ struct mlx5_vdpa_query_mr { int is_indirect; }; +enum { + MLX5_VDPA_NOTIFIER_STATE_DISABLED, + MLX5_VDPA_NOTIFIER_STATE_ENABLED, + MLX5_VDPA_NOTIFIER_STATE_ERR +}; + struct mlx5_vdpa_virtq { SLIST_ENTRY(mlx5_vdpa_virtq) next; uint8_t enable; uint16_t index; uint16_t vq_size; + uint8_t notifier_state; struct mlx5_vdpa_priv *priv; struct mlx5_devx_obj *virtq; struct mlx5_devx_obj *counters; @@ -112,7 +119,6 @@ enum { struct mlx5_vdpa_priv { TAILQ_ENTRY(mlx5_vdpa_priv) next; uint8_t configured; - uint8_t direct_notifier; /* Whether direct notifier is on or off. */ uint64_t last_traffic_tic; uint32_t last_total; pthread_t timer_tid; diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c b/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c index 4b4d019..3e61264 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c @@ -36,6 +36,17 @@ break; } while (1); rte_write32(virtq->index, priv->virtq_db_addr); + if (virtq->notifier_state == MLX5_VDPA_NOTIFIER_STATE_DISABLED) { + if (rte_vhost_host_notifier_ctrl(priv->vid, virtq->index, true)) + virtq->notifier_state = MLX5_VDPA_NOTIFIER_STATE_ERR; + else + virtq->notifier_state = + MLX5_VDPA_NOTIFIER_STATE_ENABLED; + DRV_LOG(INFO, "Virtq %u notifier state is %s.", virtq->index, + virtq->notifier_state == + MLX5_VDPA_NOTIFIER_STATE_ENABLED ? "enabled" : + "disabled"); + } DRV_LOG(DEBUG, "Ring virtq %u doorbell.", virtq->index); } @@ -79,6 +90,7 @@ memset(&virtq->reset, 0, sizeof(virtq->reset)); if (virtq->eqp.fw_qp) mlx5_vdpa_event_qp_destroy(&virtq->eqp); + virtq->notifier_state = MLX5_VDPA_NOTIFIER_STATE_DISABLED; return 0; } @@ -87,10 +99,8 @@ { int i; - for (i = 0; i < priv->nr_virtqs; i++) { + for (i = 0; i < priv->nr_virtqs; i++) mlx5_vdpa_virtq_unset(&priv->virtqs[i]); - priv->virtqs[i].enable = 0; - } if (priv->tis) { claim_zero(mlx5_devx_cmd_destroy(priv->tis)); priv->tis = NULL; @@ -143,6 +153,7 @@ DRV_LOG(ERR, "Failed to set virtq %d base.", index); return -1; } + DRV_LOG(DEBUG, "vid %u virtq %u was stopped.", priv->vid, index); return 0; } @@ -289,6 +300,7 @@ virtq->priv = priv; if (!virtq->virtq) goto error; + claim_zero(rte_vhost_enable_guest_notification(priv->vid, index, 1)); if (mlx5_vdpa_virtq_modify(virtq, 1)) goto error; virtq->priv = priv; @@ -297,10 +309,6 @@ virtq->intr_handle.fd = vq.kickfd; if (virtq->intr_handle.fd == -1) { DRV_LOG(WARNING, "Virtq %d kickfd is invalid.", index); - if (!priv->direct_notifier) { - DRV_LOG(ERR, "Virtq %d cannot be notified.", index); - goto error; - } } else { virtq->intr_handle.type = RTE_INTR_HANDLE_EXT; if (rte_intr_callback_register(&virtq->intr_handle, @@ -315,6 +323,8 @@ virtq->intr_handle.fd, index); } } + DRV_LOG(DEBUG, "vid %u virtq %u was created successfully.", priv->vid, + index); return 0; error: mlx5_vdpa_virtq_unset(virtq); @@ -418,18 +428,35 @@ goto error; } priv->nr_virtqs = nr_vring; - for (i = 0; i < nr_vring; i++) { - claim_zero(rte_vhost_enable_guest_notification(priv->vid, i, - 1)); - if (mlx5_vdpa_virtq_setup(priv, i)) + for (i = 0; i < nr_vring; i++) + if (priv->virtqs[i].enable && mlx5_vdpa_virtq_setup(priv, i)) goto error; - } return 0; error: mlx5_vdpa_virtqs_release(priv); return -1; } +static int +mlx5_vdpa_virtq_is_modified(struct mlx5_vdpa_priv *priv, + struct mlx5_vdpa_virtq *virtq) +{ + struct rte_vhost_vring vq; + int ret = rte_vhost_get_vhost_vring(priv->vid, virtq->index, &vq); + + if (ret) + return -1; + if (vq.size != virtq->vq_size || vq.kickfd != virtq->intr_handle.fd) + return 1; + if (virtq->eqp.cq.cq) { + if (vq.callfd != virtq->eqp.cq.callfd) + return 1; + } else if (vq.callfd != -1) { + return 1; + } + return 0; +} + int mlx5_vdpa_virtq_enable(struct mlx5_vdpa_priv *priv, int index, int enable) { @@ -438,26 +465,33 @@ DRV_LOG(INFO, "Update virtq %d status %sable -> %sable.", index, virtq->enable ? "en" : "dis", enable ? "en" : "dis"); - if (virtq->enable == !!enable) - return 0; if (!priv->configured) { virtq->enable = !!enable; return 0; } - if (enable) { - /* Configuration might have been updated - reconfigure virtq. */ - if (virtq->virtq) { - ret = mlx5_vdpa_virtq_stop(priv, index); - if (ret) - DRV_LOG(WARNING, "Failed to stop virtq %d.", - index); - mlx5_vdpa_virtq_unset(virtq); + if (virtq->enable == !!enable) { + if (!enable) + return 0; + ret = mlx5_vdpa_virtq_is_modified(priv, virtq); + if (ret < 0) { + DRV_LOG(ERR, "Virtq %d modify check failed.", index); + return -1; } + if (ret == 0) + return 0; + DRV_LOG(INFO, "Virtq %d was modified, recreate it.", index); + } + if (virtq->virtq) { + ret = mlx5_vdpa_virtq_stop(priv, index); + if (ret) + DRV_LOG(WARNING, "Failed to stop virtq %d.", index); + mlx5_vdpa_virtq_unset(virtq); + } + if (enable) { ret = mlx5_vdpa_virtq_setup(priv, index); if (ret) { DRV_LOG(ERR, "Failed to setup virtq %d.", index); return ret; - /* The only case virtq can stay invalid. */ } } virtq->enable = !!enable;