From patchwork Mon Apr 1 06:56:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chaoyong He X-Patchwork-Id: 139014 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BBF1F43D9F; Mon, 1 Apr 2024 08:56:43 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4E0294026F; Mon, 1 Apr 2024 08:56:43 +0200 (CEST) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2093.outbound.protection.outlook.com [40.107.237.93]) by mails.dpdk.org (Postfix) with ESMTP id E4AB740265 for ; Mon, 1 Apr 2024 08:56:41 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TKqPdtcMbFGPABFwFnaEbVyrNkPyEtcjrAjLOz+nRKIzilL9xfq8fhVM2gxc1MqJXXCItMZdw8KbSg3Q7/1ye/N8xGwM8nBenqd/rhGhv6aunVFZvL/GkfW8g1eET1greL+eahaYdvCb09joGhLQQRB8S7244oTwnG+mIMsoRhGzATP4J18MKcUcPOyKgUPpCDMqIfD9JuSqlQ9kkHVTa3VBs/T0UAnbPcwdnioOUaNKli0S5/CBd3PP/Hc05WRH9xQ4QOqEzeliEbx03cj69e15vAzO9MWKEYD1xdaBwy8xFGNvZPDv/qT0K4GdqNtkfEr4N7WGePu7msw93ILfvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KNbCL47eU2kfbQl8/HRBPnIINLH66hQp8P6VLSC96ww=; b=GHr1wTV6vpBo4/HVeYD3vMOz9Y2klF9zrktXjXZeHNuVqFHnSz8pfeKMVS5jOnJLcj6jA3mxHQdTzlfkJfzyFwXikmrNltub8PVW1bpod25ptQE4kAg+kCbG8T1PGl5Kq2hD4uTkbak0+Vu0QGBySJ/XGgFP7VIe3DYUOwuN3qTbUd6VAip7pnGoiiWngCfNXaCopcTr6wMGjneejMcyg2rxQZB+ZnLBZp0UzNrRebOSnWFjKaoLLqHIau3o0zBq98mSlanWR756ZYd0uosyI3QYPaBxybbtNz941TkQZjvmlQKS3CHGW2NH5pBOakORywFlFIG/cCT6/pL6DlPwLA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KNbCL47eU2kfbQl8/HRBPnIINLH66hQp8P6VLSC96ww=; b=VREtRur8UuqreeRlQp4p5IUmo5rK9jNUgOVZQh45ITmfskBb0G/hlDyHMR0Ya6W92VXVpzJl5b4Q+LX4BrAF1UupSQ9mnyup52FEsIXYtc34H5csDhg7SG6yIHUSM/xkNg+CayAnowx9zI97Xn77A8iPMfiz/dKuHGa6sFaZCb4= Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by SA0PR13MB3984.namprd13.prod.outlook.com (2603:10b6:806:93::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7409.46; Mon, 1 Apr 2024 06:56:39 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::ec12:7411:559a:850e]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::ec12:7411:559a:850e%5]) with mapi id 15.20.7409.042; Mon, 1 Apr 2024 06:56:39 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, Long Wu , Chaoyong He Subject: [PATCH] net/nfp: use Tx ring pointer write back Date: Mon, 1 Apr 2024 14:56:22 +0800 Message-Id: <20240401065622.106253-1-chaoyong.he@corigine.com> X-Mailer: git-send-email 2.39.1 X-ClientProxiedBy: PH2PEPF00003859.namprd17.prod.outlook.com (2603:10b6:518:1::7b) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|SA0PR13MB3984:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: R/2ZZHpSe5RT2ngRz2uCvGbUNjFtCwCEo455fVS3M6GmO5xcRxelZrXW+SSi3Ruw4ZNzh0+tWv4zu81804yyI+mUbEMGSQjuzogFN5mYjiz7M1ohdEeK8xXzTornjbztG3lPYQQyvr+wdqsEr3KTNE3UOHySIheiSKeURTL9zhAsmvPHwGadq7e6M8RHmxpRa80zB+YzCMtVspDTL52zrMJpQx/bpImN3WT5lIabrqmuoTvKNeTe51QXYfgh5GGy6K3Xd7+zl8V1z3wQXa2g9/mtydjXbr70joGkssFBI2nx/XhxSkCpZOd4dJiq46hFSqUMqq2zXVzQH6sr9bElM6QNQKXRVU+WNwpBDVvF69DLwHSfuSEwd0qIO15NExC+I2Wzf5bNig/18Mn0WyQsHTaZ/DcCoBeYTanqgekdkE+mbXmnegN61kDWCx8oiT5hVmf9OMiL/TySiA3WLCShQwZxVcs7ts8q/zl+x7ohNyszs8fqS51GUN567cLOxIVPos5ItjJ3dtKjr1bzWuq4IYTATDfUMw1uuIGwrxLbpDip+GQmD7d9ejFJROtJQ5/2NXFDb4lIFU3bgLVjQw8EcZjc+j6oWvoXU83Xhb2tJbOK4fzX4cmsvPLk6Pv+IZXWQrrJzoTqcHWyk9gJu9i24mGljvP1JKGsBng6YS8wSnu2kMhR/SfDNAPayNfFLnvApDkCXqsXlJq1IqNs63DtwQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(376005)(52116005)(1800799015)(366007)(38350700005); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: UuJDzp26nIH6evZUqQ2CkMJ6QhUFU0wO+2WL1hq4D5xy+OuCKGQDHkLq7OyhbLTWnS3G3C0Pzmtgz8WwQ1CIKrgm3oX/ZvVWcPte7yFTBWh6ZcNfU8ZYjxLe9smNun1CUmBe9sOlhDnY0bl4CVekzAnvFxpX4NA5kJW96TbLI7tOep9DonSLpeavqsF9zavNuiO77Ky84nFpFbu1DqG5EVgFPrua+p5r094svIEbBiVKx2q5yGtC8iIhOSx0wTDcuKWOpBJ6dPK6uswlpLFF//Op03UIyIA75bq/bD0dBcGSY01VWNFLXgTfuIov2/VJYPDGx/TQl4j5ysvwKjqr+7KOaFZMmpDAyyd4JgAF6OIcBllOi9TyuT9dvc6gKvAeQhfrV/e6uKM5V9vU0HoiswKniPDkNb8ENjCFwGXnq1rGn3Ca/WpGq7/eKSYmdoy1Mfv3xBXQylZUtGrv3vJUg9/OaEN3ljOdrzeTnwKTvmsBMQ7LLmw2LGSW6A7yb3YcpV3wdpotOgR7Fv4EEDYl//CKhI/7KLQAkILVAxZw0brUiMmKZtuiqIEFScLfpJLFBQ/0wo3fUG0UMUJbEGSczg1ji256ik3f/CRGUny9kFlNQ5jiNujqFG3E3zVG0IzH8SaCe+g7Dt+Ig+lb67vvcvAJJhQtUjshX9r5up5Uh0AjdYZ3hs5j91lKGV9fA167JAeZlMR4d3qhvbjZH9pMvgx1/+Ax8XjNagLO4r149oI2JxrQLriOhw9f6z0Qy802ksokLYWyVuf2OqAUB3jzKQKeIyzYYSsdCzTKDuM5YvrGfk/4FvVIOi4/s9fiBiZmzk/bkjye6FBTXriSb0PrWsSm/qK7HRqjkQug2bzQ5rgm46UMF0thAacmg9Saxd+iwVyKHtJUgadxpQDJrlAuwiZxh0IqYLkp588IoyarB+0L3P9lBDcITM4cjXYfeMGmqxq6iqEgEgUTlJmM7NAiKONW/sDbyHie6Pjxdus8HCR1Ww7U4ObFQSj+mQtQsTEdTbF9AUXe3IGLRAbaqhvMCOGmdDm6avtLyJUuC86ttemAhGwlADeGoDObU+9e1Ai8gknRQbwjH2UIy8zaYux0URX62eg1oQDZBHS/tVCtjwjl1Z0j8OqaC2vSlMmnip7H4zVk8NRHj7Y/oovz74izHrDkgCRgUcuoa+LF39tMAK/C0jMok0wwL+7WTYCKCwDPGjesrteGOGjULOiPYQ05z8EZdc/cRoYSMA5ivG0rpfyHCrJoeHyeEIMi4pKIchhYm/ocAdRK2a+gyHYq+1CZZTgn1ojBLujTePHFjXn/BZk9fpa8n+Dx0b3qEJsb1kT+yb+NYRfUEGF/yfgNpb0/ovSwtZgVlKjZFGU9uPjywTLbMWp+i5tGNcnL435V3+qahSrBFRvcNLfzraQNdLGyesUuSWwWtxTmg4bSZlHLtpF0sVsLUQmLuswbHaGWar8uqPFuyDPMlchLe4IvsgMj6vdQ48kVq9q+MjjEVHALEEFTfJQwApieFjCeUD5o+bPmCuF/RAh2QfReKY25tVVSVRl7bWSYptM561dD7OYeNwyeM31rVKahy1K7svWNZ6ORIVGECKntE3cv627lF1dolA== X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9473b8d8-5a5e-43e2-f26a-08dc5218e123 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Apr 2024 06:56:39.2574 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: S3dBO+drkJqGGLW0O4Wb8ktZMFXDsRptPydG/CDp3PKgd1d8LHT9lBB5V6bPIyLP3h5klisV2XSVlIdrB6e+NivzhUTies5XCrs6tClupF4= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR13MB3984 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Long Wu This follows the mainline Linux kernel commit 0dcf7f500b0a (nfp: use TX ring pointer write back) by Jakub Kicinski. This will speed up Tx completions, because we avoid a read from device memory (replacing PCIe read with DMA read), and it works on CoreNIC firmware with NFDk. Signed-off-by: Long Wu Reviewed-by: Chaoyong He --- drivers/net/nfp/nfdk/nfp_nfdk_dp.c | 7 ++++++ drivers/net/nfp/nfp_ethdev.c | 16 +++++++++++++- drivers/net/nfp/nfp_net_common.c | 34 ++++++++++++++++++++++++++++++ drivers/net/nfp/nfp_net_common.h | 5 +++++ drivers/net/nfp/nfp_rxtx.c | 28 +++++++++++++++++++++++- drivers/net/nfp/nfp_rxtx.h | 6 ++++++ 6 files changed, 94 insertions(+), 2 deletions(-) diff --git a/drivers/net/nfp/nfdk/nfp_nfdk_dp.c b/drivers/net/nfp/nfdk/nfp_nfdk_dp.c index 1911736e2b..41cdfd3a40 100644 --- a/drivers/net/nfp/nfdk/nfp_nfdk_dp.c +++ b/drivers/net/nfp/nfdk/nfp_nfdk_dp.c @@ -544,6 +544,13 @@ nfp_net_nfdk_tx_queue_setup(struct rte_eth_dev *dev, return -ENOMEM; } + if (hw->txrwb_mz != NULL) { + txq->txrwb = (uint64_t *)hw->txrwb_mz->addr + queue_idx; + txq->txrwb_dma = (uint64_t)hw->txrwb_mz->iova + + queue_idx * sizeof(uint64_t); + nn_cfg_writeq(&hw->super, NFP_NET_CFG_TXR_WB_ADDR(queue_idx), txq->txrwb_dma); + } + nfp_net_reset_tx_queue(txq); dev->data->tx_queues[queue_idx] = txq; diff --git a/drivers/net/nfp/nfp_ethdev.c b/drivers/net/nfp/nfp_ethdev.c index 568de1d024..b711e15b9f 100644 --- a/drivers/net/nfp/nfp_ethdev.c +++ b/drivers/net/nfp/nfp_ethdev.c @@ -360,6 +360,9 @@ nfp_net_start(struct rte_eth_dev *dev) if ((hw->cap & NFP_NET_CFG_CTRL_RINGCFG) != 0) new_ctrl |= NFP_NET_CFG_CTRL_RINGCFG; + if ((hw->cap & NFP_NET_CFG_CTRL_TXRWB) != 0) + new_ctrl |= NFP_NET_CFG_CTRL_TXRWB; + if (nfp_reconfig(hw, new_ctrl, update) != 0) return -EIO; @@ -577,6 +580,8 @@ nfp_net_uninit(struct rte_eth_dev *eth_dev) nfp_net_flow_priv_uninit(net_hw->pf_dev, net_hw->idx); rte_free(net_hw->eth_xstats_base); + if ((net_hw->super.cap & NFP_NET_CFG_CTRL_TXRWB) != 0) + nfp_net_txrwb_free(eth_dev); nfp_ipsec_uninit(eth_dev); if (net_hw->mac_stats_area != NULL) nfp_cpp_area_release_free(net_hw->mac_stats_area); @@ -987,6 +992,12 @@ nfp_net_init(struct rte_eth_dev *eth_dev) goto xstats_free; } + if ((hw->cap & NFP_NET_CFG_CTRL_TXRWB) != 0) { + err = nfp_net_txrwb_alloc(eth_dev); + if (err != 0) + goto xstats_free; + } + nfp_net_pf_read_mac(app_fw_nic, port); nfp_write_mac(hw, &hw->mac_addr.addr_bytes[0]); @@ -1025,12 +1036,15 @@ nfp_net_init(struct rte_eth_dev *eth_dev) err = nfp_net_flow_priv_init(pf_dev, port); if (err != 0) { PMD_INIT_LOG(ERR, "Init net flow priv failed"); - goto xstats_free; + goto txrwb_free; } } return 0; +txrwb_free: + if ((hw->cap & NFP_NET_CFG_CTRL_TXRWB) != 0) + nfp_net_txrwb_free(eth_dev); xstats_free: rte_free(net_hw->eth_xstats_base); ipsec_exit: diff --git a/drivers/net/nfp/nfp_net_common.c b/drivers/net/nfp/nfp_net_common.c index c6889949ff..da1a7e7be1 100644 --- a/drivers/net/nfp/nfp_net_common.c +++ b/drivers/net/nfp/nfp_net_common.c @@ -2039,6 +2039,40 @@ nfp_net_check_dma_mask(struct nfp_net_hw *hw, return 0; } +int +nfp_net_txrwb_alloc(struct rte_eth_dev *eth_dev) +{ + struct nfp_net_hw *net_hw; + char mz_name[RTE_MEMZONE_NAMESIZE]; + + net_hw = nfp_net_get_hw(eth_dev); + snprintf(mz_name, sizeof(mz_name), "%s_TXRWB", eth_dev->data->name); + net_hw->txrwb_mz = rte_memzone_reserve_aligned(mz_name, + net_hw->max_tx_queues * sizeof(uint64_t), + rte_socket_id(), + RTE_MEMZONE_IOVA_CONTIG, RTE_CACHE_LINE_SIZE); + if (net_hw->txrwb_mz == NULL) { + PMD_INIT_LOG(ERR, "Failed to alloc %s for TX ring write back", + mz_name); + return -ENOMEM; + } + + return 0; +} + +void +nfp_net_txrwb_free(struct rte_eth_dev *eth_dev) +{ + struct nfp_net_hw *net_hw; + + net_hw = nfp_net_get_hw(eth_dev); + if (net_hw->txrwb_mz == NULL) + return; + + rte_memzone_free(net_hw->txrwb_mz); + net_hw->txrwb_mz = NULL; +} + void nfp_net_cfg_read_version(struct nfp_net_hw *hw) { diff --git a/drivers/net/nfp/nfp_net_common.h b/drivers/net/nfp/nfp_net_common.h index 49a5a84044..8066e77e6f 100644 --- a/drivers/net/nfp/nfp_net_common.h +++ b/drivers/net/nfp/nfp_net_common.h @@ -168,6 +168,9 @@ struct nfp_net_hw { /** Backpointer to the eth_dev of this port */ struct rte_eth_dev *eth_dev; + /** TX pointer ring write back memzone */ + const struct rte_memzone *txrwb_mz; + /** Info from the firmware */ struct nfp_net_fw_ver ver; uint32_t max_mtu; @@ -321,6 +324,8 @@ int nfp_net_fec_set(struct rte_eth_dev *dev, uint32_t fec_capa); void nfp_net_get_fw_version(struct nfp_net_hw *hw, uint32_t *fw_version); +int nfp_net_txrwb_alloc(struct rte_eth_dev *eth_dev); +void nfp_net_txrwb_free(struct rte_eth_dev *eth_dev); #define NFP_PRIV_TO_APP_FW_NIC(app_fw_priv)\ ((struct nfp_app_fw_nic *)app_fw_priv) diff --git a/drivers/net/nfp/nfp_rxtx.c b/drivers/net/nfp/nfp_rxtx.c index 1aee3ecb3f..f9c4636688 100644 --- a/drivers/net/nfp/nfp_rxtx.c +++ b/drivers/net/nfp/nfp_rxtx.c @@ -695,6 +695,26 @@ nfp_net_rx_queue_setup(struct rte_eth_dev *dev, return 0; } +static inline uint32_t +nfp_net_read_tx_free_qcp(struct nfp_net_txq *txq) +{ + /* + * If TX ring pointer write back is not supported, do a PCIe read. + * Otherwise read qcp value from write back dma address. + */ + if (txq->txrwb == NULL) + return nfp_qcp_read(txq->qcp_q, NFP_QCP_READ_PTR); + + /* + * In most cases the TX count is a power of two and the costly modulus + * operation can be substituted with a subtraction and an AND operation. + */ + if (rte_is_power_of_2(txq->tx_count) == 1) + return (*txq->txrwb) & (txq->tx_count - 1); + else + return (*txq->txrwb) % txq->tx_count; +} + /** * Check for descriptors with a complete status * @@ -714,7 +734,7 @@ nfp_net_tx_free_bufs(struct nfp_net_txq *txq) " status", txq->qidx); /* Work out how many packets have been sent */ - qcp_rd_p = nfp_qcp_read(txq->qcp_q, NFP_QCP_READ_PTR); + qcp_rd_p = nfp_net_read_tx_free_qcp(txq); if (qcp_rd_p == txq->rd_p) { PMD_TX_LOG(DEBUG, "queue %hu: It seems harrier is not sending " @@ -761,9 +781,13 @@ void nfp_net_tx_queue_release(struct rte_eth_dev *dev, uint16_t queue_idx) { + struct nfp_net_hw *net_hw; struct nfp_net_txq *txq = dev->data->tx_queues[queue_idx]; if (txq != NULL) { + net_hw = nfp_net_get_hw(dev); + if (net_hw->txrwb_mz != NULL) + nn_cfg_writeq(&net_hw->super, NFP_NET_CFG_TXR_WB_ADDR(queue_idx), 0); nfp_net_tx_queue_release_mbufs(txq); rte_eth_dma_zone_free(dev, "tx_ring", queue_idx); rte_free(txq->txbufs); @@ -777,6 +801,8 @@ nfp_net_reset_tx_queue(struct nfp_net_txq *txq) nfp_net_tx_queue_release_mbufs(txq); txq->wr_p = 0; txq->rd_p = 0; + if (txq->txrwb != NULL) + *txq->txrwb = 0; } int diff --git a/drivers/net/nfp/nfp_rxtx.h b/drivers/net/nfp/nfp_rxtx.h index 6ecabc232c..f463b9cf75 100644 --- a/drivers/net/nfp/nfp_rxtx.h +++ b/drivers/net/nfp/nfp_rxtx.h @@ -77,6 +77,12 @@ struct nfp_net_txq { * in a cache line. */ uint64_t dma; + + /** TX pointer ring write back area (indexed by queue id) */ + uint64_t *txrwb; + + /** TX pointer ring write back area DMA address */ + uint64_t txrwb_dma; } __rte_aligned(64); /* RX and freelist descriptor format */