From patchwork Mon Nov 13 23:12:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joshua Washington X-Patchwork-Id: 134185 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7BD3C43320; Tue, 14 Nov 2023 00:12:16 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D17F4402D6; Tue, 14 Nov 2023 00:12:15 +0100 (CET) Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) by mails.dpdk.org (Postfix) with ESMTP id 249E44027B for ; Tue, 14 Nov 2023 00:12:15 +0100 (CET) Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-1cc1397321fso55333615ad.3 for ; Mon, 13 Nov 2023 15:12:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1699917134; x=1700521934; darn=dpdk.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=LVomfWC+bu93sole4ZQE0Nja/v6i6FRVq0gmHXuimBM=; b=UaDiqPdkGOr801/LaZ26x2togLaPtVjVHRGjrXRUI1Gq+9V2hLBqyBTLKhlF/sTYzQ 2OZLxuuuTzikS0LBil7aVzMDvR53Ft2jDInOI1Ib/ca5UYvOSPsPXFcuvRrSP2LhOPeu jRHIceUPVVm3iVgeed8cvNsuqsBzp+fDycGf95YMeyiNVNl+gQAbb1JE+lhdEcJ0wm23 pCW7KHHDZyFOJYPcAzCiOQkIXzG7odk3mSfv+5fExHWG0DLxGltrs4cJzAhwIWPrKMcL IFMMaiogZoak6GmzFd1VwLzHtkf5XVlgmEF9Cnuz8w7OJulo5/tf8AzxKKZwgUWE6n2b ZGcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699917134; x=1700521934; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=LVomfWC+bu93sole4ZQE0Nja/v6i6FRVq0gmHXuimBM=; b=qF8lArWlbY7TeeQuARMqIPQYbfyyI0r52ZLljbPTMLAbP9qdZpqouBgcoF7KYzHu3I 5ePUUzcsSqSBdYg2qmEyjhWnmyG2eWphm3hRkcXeMEqMtpoCS55dC2+AZXesZv1ToR12 fomAK5xydzZ88ka/QKORbOV3B00hMi2qELr2d866EhqjeRyDN54s/VdLwF8r/HFYu/eZ rvK7uG//yVNS3RzqeDJ8FqIZ6CZn4bEp0fYd3DJvNEoXYlFjfONkuJU0J/LAAocw510d bjS3oJgvRvDeY8EsGpedOnJAd5EYMsUujjnQygprSC47mNJk5b9/VDXytpt6w1ke0siI OwNQ== X-Gm-Message-State: AOJu0YyeiZNY2Zhcbtyn8jXyKcStDr0+sgazMYp6zG0AS1duhCVZRtVV dWj1jO5LYk5iwp6efUgvUAF+8C8MLBTMTA== X-Google-Smtp-Source: AGHT+IFt4/lTJMGYyatvuIwZx9u6ez7QyC+sgES5PdOb6NTUfSPlUvgx2HT/5yF0t76aM7xOJ7f/EIXSPHNUBA== X-Received: from joshwash.sea.corp.google.com ([2620:15c:11c:202:35c2:a552:d23a:aba4]) (user=joshwash job=sendgmr) by 2002:a17:903:2581:b0:1cc:533e:f512 with SMTP id jb1-20020a170903258100b001cc533ef512mr176285plb.6.1699917133892; Mon, 13 Nov 2023 15:12:13 -0800 (PST) Date: Mon, 13 Nov 2023 15:12:06 -0800 In-Reply-To: <20231111003410.2950594-1-joshwash@google.com> Mime-Version: 1.0 References: <20231111003410.2950594-1-joshwash@google.com> X-Mailer: git-send-email 2.42.0.869.gea05f2083d-goog Message-ID: <20231113231206.376452-1-joshwash@google.com> Subject: [PATCH v2] net/gve: fix RX buffer size alignment From: Joshua Washington To: Junfeng Guo , Jeroen de Borst , Rushil Gupta , Joshua Washington , Xiaoyun Li Cc: dev@dpdk.org, stable@dpdk.org, Ferruh Yigit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org In GVE, both queue formats have RX buffer size alignment requirements which will not always be respected when a user specifies an mbuf size. Assuming that an mbuf size is greater than the DPDK recommended default (2048 + 128), if the buffer size is not properly aligned with what the device expects, the device will silently fail to create any transmit or receive queues. Because no queues are created, there is no network traffic for the DPDK program, and errors like the following are returned when attempting to destroy queues: gve_adminq_parse_err(): AQ command failed with status -11 gve_stop_tx_queues(): failed to destroy txqs gve_adminq_parse_err(): AQ command failed with status -11 gve_stop_rx_queues(): failed to destroy rxqs This change aims to remedy this by restricting the RX receive buffer sizes to valid sizes for their respective queue formats, including both alignment and minimum and maximum supported buffer sizes. Fixes: 4bec2d0b5572 ("net/gve: support queue operations") Fixes: 1dc00f4fc74b ("net/gve: add Rx queue setup for DQO") Cc: junfeng.guo@intel.com Cc: stable@dpdk.org Signed-off-by: Joshua Washington Reviewed-by: Rushil Gupta Acked-by: Junfeng Guo --- drivers/net/gve/gve_ethdev.c | 5 ++++- drivers/net/gve/gve_ethdev.h | 8 +++++++- drivers/net/gve/gve_rx.c | 7 ++++++- drivers/net/gve/gve_rx_dqo.c | 6 +++++- 4 files changed, 22 insertions(+), 4 deletions(-) diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c index eb3bc7e151..43b4ab523d 100644 --- a/drivers/net/gve/gve_ethdev.c +++ b/drivers/net/gve/gve_ethdev.c @@ -296,7 +296,10 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->max_mac_addrs = 1; dev_info->max_rx_queues = priv->max_nb_rxq; dev_info->max_tx_queues = priv->max_nb_txq; - dev_info->min_rx_bufsize = GVE_MIN_BUF_SIZE; + if (gve_is_gqi(priv)) + dev_info->min_rx_bufsize = GVE_RX_MIN_BUF_SIZE_GQI; + else + dev_info->min_rx_bufsize = GVE_RX_MIN_BUF_SIZE_DQO; dev_info->max_rx_pktlen = priv->max_mtu + RTE_ETHER_HDR_LEN; dev_info->max_mtu = priv->max_mtu; dev_info->min_mtu = RTE_ETHER_MIN_MTU; diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h index 755ee8ad15..37f2b60845 100644 --- a/drivers/net/gve/gve_ethdev.h +++ b/drivers/net/gve/gve_ethdev.h @@ -20,7 +20,13 @@ #define GVE_DEFAULT_TX_RS_THRESH 32 #define GVE_TX_MAX_FREE_SZ 512 -#define GVE_MIN_BUF_SIZE 1024 +#define GVE_RX_BUF_ALIGN_DQO 128 +#define GVE_RX_MIN_BUF_SIZE_DQO 1024 +#define GVE_RX_MAX_BUF_SIZE_DQO ((16 * 1024) - GVE_RX_BUF_ALIGN_DQO) + +#define GVE_RX_BUF_ALIGN_GQI 2048 +#define GVE_RX_MIN_BUF_SIZE_GQI 2048 +#define GVE_RX_MAX_BUF_SIZE_GQI 4096 #define GVE_TX_CKSUM_OFFLOAD_MASK ( \ RTE_MBUF_F_TX_L4_MASK | \ diff --git a/drivers/net/gve/gve_rx.c b/drivers/net/gve/gve_rx.c index b8c92ccda0..36a1b73c65 100644 --- a/drivers/net/gve/gve_rx.c +++ b/drivers/net/gve/gve_rx.c @@ -301,6 +301,7 @@ gve_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id, const struct rte_memzone *mz; struct gve_rx_queue *rxq; uint16_t free_thresh; + uint32_t mbuf_len; int err = 0; if (nb_desc != hw->rx_desc_cnt) { @@ -344,7 +345,11 @@ gve_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id, rxq->hw = hw; rxq->ntfy_addr = &hw->db_bar2[rte_be_to_cpu_32(hw->irq_dbs[rxq->ntfy_id].id)]; - rxq->rx_buf_len = rte_pktmbuf_data_room_size(rxq->mpool) - RTE_PKTMBUF_HEADROOM; + mbuf_len = + rte_pktmbuf_data_room_size(rxq->mpool) - RTE_PKTMBUF_HEADROOM; + rxq->rx_buf_len = + RTE_MIN((uint16_t)GVE_RX_MAX_BUF_SIZE_GQI, + RTE_ALIGN_FLOOR(mbuf_len, GVE_RX_BUF_ALIGN_GQI)); /* Allocate software ring */ rxq->sw_ring = rte_zmalloc_socket("gve rx sw ring", sizeof(struct rte_mbuf *) * nb_desc, diff --git a/drivers/net/gve/gve_rx_dqo.c b/drivers/net/gve/gve_rx_dqo.c index 7e7ddac48e..422784e7e0 100644 --- a/drivers/net/gve/gve_rx_dqo.c +++ b/drivers/net/gve/gve_rx_dqo.c @@ -220,6 +220,7 @@ gve_rx_queue_setup_dqo(struct rte_eth_dev *dev, uint16_t queue_id, const struct rte_memzone *mz; struct gve_rx_queue *rxq; uint16_t free_thresh; + uint32_t mbuf_len; int err = 0; if (nb_desc != hw->rx_desc_cnt) { @@ -264,8 +265,11 @@ gve_rx_queue_setup_dqo(struct rte_eth_dev *dev, uint16_t queue_id, rxq->hw = hw; rxq->ntfy_addr = &hw->db_bar2[rte_be_to_cpu_32(hw->irq_dbs[rxq->ntfy_id].id)]; - rxq->rx_buf_len = + mbuf_len = rte_pktmbuf_data_room_size(rxq->mpool) - RTE_PKTMBUF_HEADROOM; + rxq->rx_buf_len = + RTE_MIN((uint16_t)GVE_RX_MAX_BUF_SIZE_DQO, + RTE_ALIGN_FLOOR(mbuf_len, GVE_RX_BUF_ALIGN_DQO)); /* Allocate software ring */ rxq->sw_ring = rte_zmalloc_socket("gve rx sw ring",