From patchwork Fri Mar 11 17:34:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, YuanX" X-Patchwork-Id: 108683 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7E5E3A0093; Fri, 11 Mar 2022 10:41:55 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6E4A141141; Fri, 11 Mar 2022 10:41:55 +0100 (CET) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by mails.dpdk.org (Postfix) with ESMTP id E5A8F41141 for ; Fri, 11 Mar 2022 10:41:52 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1646991713; x=1678527713; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Qez/7uxXWCC4C4HuY8DW+jJT6AAOjo6onP61QJtjmHU=; b=VlZGzIi7bGuR4sMtCWM90oR+W1UpokvpzUMb1jnhb/Ax6d5jWzrscDuL lkLQs7h0qKI/7+kDxYYWptolQaURUqnrF77/u1shej23In48SesDUeMjM PIqy7lSriVCqZWjP2/w7Unwx1VnqA5GWyeonOnzcUfdxvWepTiMY9vFih h76UEbQUGkgJVyX0aCxsnkQxuVzN0lBJQh/WJvSzBngNSebHKlzsoFd5G WjzzenUVfJwN+zSP36Fg0JJztqxxBpAXD41sI3hV8/RemmCkvNPImGPTT 9ajHajOgPkoUmLiutdDDYqh5kkYR8NDOnVBqaX2oI/hlSTWGGCaVUKcUM g==; X-IronPort-AV: E=McAfee;i="6200,9189,10282"; a="236143679" X-IronPort-AV: E=Sophos;i="5.90,173,1643702400"; d="scan'208";a="236143679" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Mar 2022 01:41:52 -0800 X-IronPort-AV: E=Sophos;i="5.90,173,1643702400"; d="scan'208";a="555238293" Received: from unknown (HELO localhost.localdomain) ([10.239.251.55]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Mar 2022 01:41:49 -0800 From: Yuan Wang To: maxime.coquelin@redhat.com, chenbo.xia@intel.com Cc: dev@dpdk.org, jiayu.hu@intel.com, xuan.ding@intel.com, yvonnex.yang@intel.com, sunil.pai.g@intel.com, yuanx.wang@intel.com Subject: [RFC v2 1/2] vhost: support clear in-flight packets for async dequeue Date: Sat, 12 Mar 2022 01:34:13 +0800 Message-Id: <20220311173414.93216-2-yuanx.wang@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220311173414.93216-1-yuanx.wang@intel.com> References: <20220117132847.884998-1-yuanx.wang@intel.com> <20220311173414.93216-1-yuanx.wang@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org rte_vhost_clear_queue_thread_unsafe() supports to clear in-flight packets for async enqueue only. But after supporting async dequeue, this API should support async dequeue too. This patch also adds the thread-safe version of this API, the difference between the two API is that thread safety uses lock. These APIs maybe used to clean up packets in the async channel to prevent packet loss when the device state changes or when the device is destroyed. Signed-off-by: Yuan Wang --- doc/guides/prog_guide/vhost_lib.rst | 7 ++- doc/guides/rel_notes/release_22_03.rst | 4 ++ lib/vhost/rte_vhost_async.h | 25 ++++++++ lib/vhost/version.map | 1 + lib/vhost/virtio_net.c | 79 +++++++++++++++++++++++++- 5 files changed, 113 insertions(+), 3 deletions(-) diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst index 886f8f5e72..bbb7333d03 100644 --- a/doc/guides/prog_guide/vhost_lib.rst +++ b/doc/guides/prog_guide/vhost_lib.rst @@ -273,7 +273,12 @@ The following is an overview of some key Vhost API functions: * ``rte_vhost_clear_queue_thread_unsafe(vid, queue_id, **pkts, count, dma_id, vchan_id)`` - Clear inflight packets which are submitted to DMA engine in vhost async data + Clear in-flight packets which are submitted to async channel in vhost + async data path without performing any locking. Completed packets are + returned to applications through ``pkts``. + +* ``rte_vhost_clear_queue(vid, queue_id, **pkts, count, dma_id, vchan_id)`` + Clear in-flight packets which are submitted to async channel in vhost async data path. Completed packets are returned to applications through ``pkts``. Vhost-user Implementations diff --git a/doc/guides/rel_notes/release_22_03.rst b/doc/guides/rel_notes/release_22_03.rst index 60e5b4f9aa..ab754056b9 100644 --- a/doc/guides/rel_notes/release_22_03.rst +++ b/doc/guides/rel_notes/release_22_03.rst @@ -232,6 +232,10 @@ New Features * Crypto producer mode helps to measure performance of OP_NEW and OP_FORWARD modes of event crypto adapter. +* **Added thread-safe version of inflight packet clear API in vhost library.** + + * Added an API which can clear the inflight packets submitted to + the async channel in a thread-safe manner in the vhost async data path. Removed Items ------------- diff --git a/lib/vhost/rte_vhost_async.h b/lib/vhost/rte_vhost_async.h index b6ab0b06a2..eb1ddb2ec1 100644 --- a/lib/vhost/rte_vhost_async.h +++ b/lib/vhost/rte_vhost_async.h @@ -166,6 +166,31 @@ uint16_t rte_vhost_clear_queue_thread_unsafe(int vid, uint16_t queue_id, struct rte_mbuf **pkts, uint16_t count, int16_t dma_id, uint16_t vchan_id); +/** + * This function checks async completion status and clear packets for + * a specific vhost device queue. Packets which are inflight will be + * returned in an array. + * + * @param vid + * ID of vhost device to clear data + * @param queue_id + * Queue id to clear data + * @param pkts + * Blank array to get return packet pointer + * @param count + * Size of the packet array + * @param dma_id + * The identifier of the DMA device + * @param vchan_id + * The identifier of virtual DMA channel + * @return + * Number of packets returned + */ +__rte_experimental +uint16_t rte_vhost_clear_queue(int vid, uint16_t queue_id, + struct rte_mbuf **pkts, uint16_t count, int16_t dma_id, + uint16_t vchan_id); + /** * The DMA vChannels used in asynchronous data path must be configured * first. So this function needs to be called before enabling DMA diff --git a/lib/vhost/version.map b/lib/vhost/version.map index 968d6d4290..11c5fa635d 100644 --- a/lib/vhost/version.map +++ b/lib/vhost/version.map @@ -88,6 +88,7 @@ EXPERIMENTAL { # added in 22.03 rte_vhost_async_dma_configure; rte_vhost_async_try_dequeue_burst; + rte_vhost_clear_queue; }; INTERNAL { diff --git a/lib/vhost/virtio_net.c b/lib/vhost/virtio_net.c index 3816caca79..ca96632ee5 100644 --- a/lib/vhost/virtio_net.c +++ b/lib/vhost/virtio_net.c @@ -26,6 +26,11 @@ #define MAX_BATCH_LEN 256 +static __rte_always_inline uint16_t +async_poll_dequeue_completed_split(struct virtio_net *dev, uint16_t queue_id, + struct rte_mbuf **pkts, uint16_t count, uint16_t dma_id, + uint16_t vchan_id, bool legacy_ol_flags); + /* DMA device copy operation tracking array. */ struct async_dma_info dma_copy_track[RTE_DMADEV_DEFAULT_MAX]; @@ -2083,7 +2088,7 @@ rte_vhost_clear_queue_thread_unsafe(int vid, uint16_t queue_id, return 0; VHOST_LOG_DATA(DEBUG, "(%s) %s\n", dev->ifname, __func__); - if (unlikely(!is_valid_virt_queue_idx(queue_id, 0, dev->nr_vring))) { + if (unlikely(queue_id >= dev->nr_vring)) { VHOST_LOG_DATA(ERR, "(%s) %s: invalid virtqueue idx %d.\n", dev->ifname, __func__, queue_id); return 0; @@ -2104,11 +2109,81 @@ rte_vhost_clear_queue_thread_unsafe(int vid, uint16_t queue_id, return 0; } - n_pkts_cpl = vhost_poll_enqueue_completed(dev, queue_id, pkts, count, dma_id, vchan_id); + if (queue_id % 2 == 0) + n_pkts_cpl = vhost_poll_enqueue_completed(dev, queue_id, + pkts, count, dma_id, vchan_id); + else { + if (unlikely(vq_is_packed(dev))) + VHOST_LOG_DATA(ERR, + "(%d) %s: async dequeue does not support packed ring.\n", + dev->vid, __func__); + else + n_pkts_cpl = async_poll_dequeue_completed_split(dev, queue_id, pkts, count, + dma_id, vchan_id, dev->flags & VIRTIO_DEV_LEGACY_OL_FLAGS); + } + + return n_pkts_cpl; +} + +uint16_t +rte_vhost_clear_queue(int vid, uint16_t queue_id, struct rte_mbuf **pkts, + uint16_t count, int16_t dma_id, uint16_t vchan_id) +{ + struct virtio_net *dev = get_device(vid); + struct vhost_virtqueue *vq; + uint16_t n_pkts_cpl = 0; + + if (!dev) + return 0; + + VHOST_LOG_DATA(DEBUG, "(%s) %s\n", dev->ifname, __func__); + if (unlikely(queue_id >= dev->nr_vring)) { + VHOST_LOG_DATA(ERR, "(%s) %s: invalid virtqueue idx %d.\n", + dev->ifname, __func__, queue_id); + return 0; + } + + vq = dev->virtqueue[queue_id]; + + if (unlikely(!vq->async)) { + VHOST_LOG_DATA(ERR, "(%s) %s: async not registered for queue id %d.\n", + dev->ifname, __func__, queue_id); + return 0; + } + + if (unlikely(!dma_copy_track[dma_id].vchans || + !dma_copy_track[dma_id].vchans[vchan_id].pkts_cmpl_flag_addr)) { + VHOST_LOG_DATA(ERR, "(%s) %s: invalid channel %d:%u.\n", dev->ifname, __func__, + dma_id, vchan_id); + return 0; + } + + if (!rte_spinlock_trylock(&vq->access_lock)) { + VHOST_LOG_DATA(ERR, + "(%d) %s: failed to clear async queue id %d, virtqueue busy.\n", + dev->vid, __func__, queue_id); + return 0; + } + + if (queue_id % 2 == 0) + n_pkts_cpl = vhost_poll_enqueue_completed(dev, queue_id, + pkts, count, dma_id, vchan_id); + else { + if (unlikely(vq_is_packed(dev))) + VHOST_LOG_DATA(ERR, + "(%d) %s: async dequeue does not support packed ring.\n", + dev->vid, __func__); + else + n_pkts_cpl = async_poll_dequeue_completed_split(dev, queue_id, pkts, count, + dma_id, vchan_id, dev->flags & VIRTIO_DEV_LEGACY_OL_FLAGS); + } + + rte_spinlock_unlock(&vq->access_lock); return n_pkts_cpl; } + static __rte_always_inline uint32_t virtio_dev_rx_async_submit(struct virtio_net *dev, uint16_t queue_id, struct rte_mbuf **pkts, uint32_t count, int16_t dma_id, uint16_t vchan_id) From patchwork Fri Mar 11 17:34:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, YuanX" X-Patchwork-Id: 108684 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7D571A0093; Fri, 11 Mar 2022 10:42:00 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 679D141152; Fri, 11 Mar 2022 10:41:57 +0100 (CET) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by mails.dpdk.org (Postfix) with ESMTP id CCE2741143 for ; Fri, 11 Mar 2022 10:41:55 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1646991716; x=1678527716; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=KQx7Amk1D6tp+az+1cUgv2Qnehq+ZfuHPb4KNRmYyP4=; b=HR6b65Lywvuz8ugUN/1aQP/fETJTJcYn4d03GtFxU6ln1mpb+GEDazxR Jq9wUzpGSiwSnQtpTdGepF803xZAdNfRm7ho0JedZbJiyVzhZsvfb+c2n 7oPot8T9u3a0hAPTkvP+62mLn6hp3PH6V8QM+pHGhrE8+W6VTnLXSdyIU b25UtZ77mb8KWdhcRJyCnDrhDTNiIVqhzxdOn34Xy+PU/owmAH2Wo/HQz gWRrBAMtngUbMCWCL++vSulCmisZDZAzrAhF9mboKciiiLRJ4wvF3jgsh AE04lQNthNFNunuAze7e2i64U0HhAslaneyOZl0t9wnBF0QuawE+2EFBG A==; X-IronPort-AV: E=McAfee;i="6200,9189,10282"; a="236143683" X-IronPort-AV: E=Sophos;i="5.90,173,1643702400"; d="scan'208";a="236143683" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Mar 2022 01:41:55 -0800 X-IronPort-AV: E=Sophos;i="5.90,173,1643702400"; d="scan'208";a="555238308" Received: from unknown (HELO localhost.localdomain) ([10.239.251.55]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Mar 2022 01:41:52 -0800 From: Yuan Wang To: maxime.coquelin@redhat.com, chenbo.xia@intel.com Cc: dev@dpdk.org, jiayu.hu@intel.com, xuan.ding@intel.com, yvonnex.yang@intel.com, sunil.pai.g@intel.com, yuanx.wang@intel.com Subject: [RFC v2 2/2] example/vhost: support to clear in-flight packets for async dequeue Date: Sat, 12 Mar 2022 01:34:14 +0800 Message-Id: <20220311173414.93216-3-yuanx.wang@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220311173414.93216-1-yuanx.wang@intel.com> References: <20220117132847.884998-1-yuanx.wang@intel.com> <20220311173414.93216-1-yuanx.wang@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch allows vring_state_changed() to clear in-flight dequeue packets. Signed-off-by: Yuan Wang --- examples/vhost/main.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/examples/vhost/main.c b/examples/vhost/main.c index d26e40ab73..04e7821322 100644 --- a/examples/vhost/main.c +++ b/examples/vhost/main.c @@ -1767,9 +1767,6 @@ vring_state_changed(int vid, uint16_t queue_id, int enable) if (!vdev) return -1; - if (queue_id != VIRTIO_RXQ) - return 0; - if (dma_bind[vid2socketid[vid]].dmas[queue_id].async_enabled) { if (!enable) vhost_clear_queue_thread_unsafe(vdev, queue_id);