From patchwork Mon Feb 14 08:56:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 107416 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D6A52A00C4; Mon, 14 Feb 2022 09:57:40 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1387641159; Mon, 14 Feb 2022 09:57:30 +0100 (CET) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2062.outbound.protection.outlook.com [40.107.243.62]) by mails.dpdk.org (Postfix) with ESMTP id BD6474114B for ; Mon, 14 Feb 2022 09:57:28 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dmBYHMa0GnZLwlmLSL3++gyTrIS/b/yOfkJYpJmvKR6hEhlSp3ArBmyrOwGIDqJM4z7CdFdNcxdPpiIrmFM/A4nfH2L/mDUMcCqWJOSmC/Qf8Wsc94461tUvKdJpompJCOyeHuU0hAQ4vXx81hE735knav3wpCf2Qod6lxfn6PE9HUgCZ0cgArogMHf7xKUYUISPwEw1SUBx2l5UqrAz8Yo5l/wP5nWlRBxmrpMD2ruaLEtpHi45Hptt5eXMC7M19W+49h9pHdIxprbGFplINXQb9rcaCqQvkpLbMtEafINciBinRRN5VvJD1oAfLpag+SmDYUaB4NgcLASHuZ2p0Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=r/mkmq4F1Ozg7XAK7Bufbq5B/O+przom0e94UuT1T/M=; b=d7gWtxohI8fuKCBWEVzh+PaVMpoYhoWwcUzO5vPYWCVtEitv14WSWPwPy+x3toG42CqHbAo7FWSye/NYj8n5AGPrEuXhQXriJxjEvFoG2TGnezyui6vE/ctEZa15LoQNYTiEV93EY5ggirNYtZDgObdp/npqO2uxmpF93bly0PWRHVwBP2QG+CorqLUtBiH9tdzDjhvRw1QrdTyaYjQ3ddw+HUWGvzehITIBx/CEA6IdXoSbtdhLIpaBcF285g98ow8BcyxAZDioUjfGOOBgPHpTXVv/LKVOUByDwFHZYI+KFLuLHUxEyCDKS9sKt8cfB2iYR7ooUsXk/kLRMQL61Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=r/mkmq4F1Ozg7XAK7Bufbq5B/O+przom0e94UuT1T/M=; b=aAL8lmNRj+vReZgnFZZs7kfstThlTJm22gVJcb4hoRQJEzngYnkygE8vOS5TcEoZguJf8klFVPiMEW65c/YJMeO4x/XrJPFZN+TloBBYOEB1ufRrTL8P7lupVWvQ2fVhRKpCbotYsXqSO2xMyj7dWXKkcREKuYkSBpl2HX3HJIDpZ2qpABE2V7GrgRA/QRlfD9Ewzauqc3ZrLKWpG1ac8iPzko9plVHdoA4vatAUtKeQK2jnqC/7hHkoL0KisuFv/0UuPwARgx8k08it+FwKkvxI6E/OvxRp0JSq57cG+HaGKFdWC7cVVZuNMrYHujZ3R1AgGqEy5PjKMg+/LIU9LQ== Received: from DM6PR03CA0019.namprd03.prod.outlook.com (2603:10b6:5:40::32) by BN6PR12MB1553.namprd12.prod.outlook.com (2603:10b6:405:6::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4975.11; Mon, 14 Feb 2022 08:57:27 +0000 Received: from DM6NAM11FT008.eop-nam11.prod.protection.outlook.com (2603:10b6:5:40:cafe::d9) by DM6PR03CA0019.outlook.office365.com (2603:10b6:5:40::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4975.11 via Frontend Transport; Mon, 14 Feb 2022 08:57:26 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.236) by DM6NAM11FT008.mail.protection.outlook.com (10.13.172.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4975.11 via Frontend Transport; Mon, 14 Feb 2022 08:57:26 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 14 Feb 2022 08:57:25 +0000 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Mon, 14 Feb 2022 00:57:23 -0800 From: Viacheslav Ovsiienko To: CC: , Subject: [PATCH 3/4] net/mlx5: add wait on time support in Tx datapath Date: Mon, 14 Feb 2022 10:56:54 +0200 Message-ID: <20220214085655.22648-4-viacheslavo@nvidia.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20220214085655.22648-1-viacheslavo@nvidia.com> References: <20220214085655.22648-1-viacheslavo@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 44c1313a-1789-4f4e-0981-08d9ef980621 X-MS-TrafficTypeDiagnostic: BN6PR12MB1553:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: WD8oN9IdgG7nL9KzKedKf1b3t2A1v1yHdZYJCcxkezL+PprRHtPicmxS7qtPBSFh2Ez1HXZrhgsKT+jCFzvfd7Xp+MfZ+PNs/VfrphkOOMlH1IiK2DIPjQktPAL/jls8LS64TM6I2OQU+ouNNyfvVYqaX0HO6fX3Y0IWne89CP3kOHku11E06f6ip/xHyKgRBvHZ2UaGWCwthAmEiIPT1Y4PfK2a4UqzoJOB0e/gcrfcP/D4yg9U7oGePRZ5uUaLe/0Otmma5pm72rFHd7wF2R1643xeGXM7MlSdU6vnFZe6zQjt+VUGvnJKSlm++KlAb9b2WjaqWH/zxuAHESxD7TFpEaN/F9QemjRPbpxzmeACs4eStLvMP24SuSSj8jXsH0v11UyKwJMS/kTsT1YdjHhX/jz6vUvlFgDnIhrr+OqfQn8fxyuKPC8WcQ5qaZKCU2DZtUrC3mAw6+v3mvUtiB0rEePC6JfFOxRzDfMc+97Q3AgaXMmo3Kq/dOCuzYJ//UjM+1UVjU/BrA00c0IahK4ekjIr2kxeJxtXcfKnPiUxQFIy4SZeFfmM9dLB+tIPQUrDp/1G8skqqQHKt35vvhY06/ojk2EkYTMiTG+F28t5bG7OH+5aZpL3xVtWzNzFupVAOImC94v0N1qIaXiHP9Uh0ji40FzRerhDyivspST84S9IaKvyOsX7enJjUxzUZbCNb1g5mv73nYbVbuTkzQ== X-Forefront-Antispam-Report: CIP:12.22.5.236; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(46966006)(36840700001)(40470700004)(6286002)(82310400004)(186003)(16526019)(70206006)(70586007)(4326008)(26005)(8676002)(1076003)(36860700001)(86362001)(7696005)(8936002)(107886003)(6666004)(5660300002)(40460700003)(2616005)(83380400001)(47076005)(6916009)(336012)(54906003)(426003)(81166007)(356005)(55016003)(36756003)(2906002)(508600001)(316002)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2022 08:57:26.5438 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 44c1313a-1789-4f4e-0981-08d9ef980621 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.236]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT008.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR12MB1553 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The hardware since ConnectX-7 supports waiting on specified moment of time with new introduced wait descriptor. A timestamp can be directrly placed into descriptor and pushed to sending queue. Once hardware encounter the wait descriptor the queue operation is suspended till specified moment of time. This patch update the Tx datapath to handle this new hardware wait capability. Signed-off-by: Viacheslav Ovsiienko --- drivers/net/mlx5/linux/mlx5_verbs.c | 4 +- drivers/net/mlx5/mlx5_tx.h | 72 +++++++++++++++++++++++++---- drivers/net/mlx5/mlx5_txq.c | 7 ++- 3 files changed, 72 insertions(+), 11 deletions(-) diff --git a/drivers/net/mlx5/linux/mlx5_verbs.c b/drivers/net/mlx5/linux/mlx5_verbs.c index 80dd0bb6c1..5f821c4645 100644 --- a/drivers/net/mlx5/linux/mlx5_verbs.c +++ b/drivers/net/mlx5/linux/mlx5_verbs.c @@ -1037,7 +1037,9 @@ mlx5_txq_ibv_obj_new(struct rte_eth_dev *dev, uint16_t idx) txq_data->wqe_comp = 0; txq_data->wqe_thres = txq_data->wqe_s / MLX5_TX_COMP_THRESH_INLINE_DIV; txq_data->wait_on_time = !!(!priv->config.tx_pp && - priv->config.hca_attr.wait_on_time); + priv->config.hca_attr.wait_on_time && + txq_data->offloads & + DEV_TX_OFFLOAD_SEND_ON_TIMESTAMP); #ifdef HAVE_IBV_FLOW_DV_SUPPORT /* * If using DevX need to query and store TIS transport domain value. diff --git a/drivers/net/mlx5/mlx5_tx.h b/drivers/net/mlx5/mlx5_tx.h index b50deb8b67..0adc3f4839 100644 --- a/drivers/net/mlx5/mlx5_tx.h +++ b/drivers/net/mlx5/mlx5_tx.h @@ -780,7 +780,7 @@ mlx5_tx_cseg_init(struct mlx5_txq_data *__rte_restrict txq, * compile time and may be used for optimization. */ static __rte_always_inline void -mlx5_tx_wseg_init(struct mlx5_txq_data *restrict txq, +mlx5_tx_qseg_init(struct mlx5_txq_data *restrict txq, struct mlx5_txq_local *restrict loc __rte_unused, struct mlx5_wqe *restrict wqe, unsigned int wci, @@ -795,6 +795,43 @@ mlx5_tx_wseg_init(struct mlx5_txq_data *restrict txq, qs->reserved1 = RTE_BE32(0); } +/** + * Build the Wait on Time Segment with specified timestamp value. + * + * @param txq + * Pointer to TX queue structure. + * @param loc + * Pointer to burst routine local context. + * @param wqe + * Pointer to WQE to fill with built Control Segment. + * @param ts + * Timesatmp value to wait. + * @param olx + * Configured Tx offloads mask. It is fully defined at + * compile time and may be used for optimization. + */ +static __rte_always_inline void +mlx5_tx_wseg_init(struct mlx5_txq_data *restrict txq, + struct mlx5_txq_local *restrict loc __rte_unused, + struct mlx5_wqe *restrict wqe, + uint64_t ts, + unsigned int olx __rte_unused) +{ + struct mlx5_wqe_wseg *ws; + + ws = RTE_PTR_ADD(wqe, MLX5_WSEG_SIZE); + ws->operation = rte_cpu_to_be_32(MLX5_WAIT_COND_CYCLIC_BIGGER); + ws->lkey = RTE_BE32(0); + ws->va_high = RTE_BE32(0); + ws->va_low = RTE_BE32(0); + if (txq->rt_timestamp) { + ts = ts % (uint64_t)NS_PER_S + | (ts / (uint64_t)NS_PER_S) << 32; + } + ws->value = rte_cpu_to_be_64(ts); + ws->mask = txq->rt_timemask; +} + /** * Build the Ethernet Segment without inlined data. * Supports Software Parser, Checksums and VLAN insertion Tx offload features. @@ -1626,9 +1663,9 @@ mlx5_tx_schedule_send(struct mlx5_txq_data *restrict txq, { if (MLX5_TXOFF_CONFIG(TXPP) && loc->mbuf->ol_flags & txq->ts_mask) { + struct mlx5_dev_ctx_shared *sh; struct mlx5_wqe *wqe; uint64_t ts; - int32_t wci; /* * Estimate the required space quickly and roughly. @@ -1640,13 +1677,32 @@ mlx5_tx_schedule_send(struct mlx5_txq_data *restrict txq, return MLX5_TXCMP_CODE_EXIT; /* Convert the timestamp into completion to wait. */ ts = *RTE_MBUF_DYNFIELD(loc->mbuf, txq->ts_offset, uint64_t *); - wci = mlx5_txpp_convert_tx_ts(txq->sh, ts); - if (unlikely(wci < 0)) - return MLX5_TXCMP_CODE_SINGLE; - /* Build the WAIT WQE with specified completion. */ wqe = txq->wqes + (txq->wqe_ci & txq->wqe_m); - mlx5_tx_cseg_init(txq, loc, wqe, 2, MLX5_OPCODE_WAIT, olx); - mlx5_tx_wseg_init(txq, loc, wqe, wci, olx); + sh = txq->sh; + if (txq->wait_on_time) { + /* The wait on time capability should be used. */ + ts -= sh->txpp.skew; + mlx5_tx_cseg_init(txq, loc, wqe, + 1 + sizeof(struct mlx5_wqe_wseg) / + MLX5_WSEG_SIZE, + MLX5_OPCODE_WAIT | + MLX5_OPC_MOD_WAIT_TIME << 24, olx); + mlx5_tx_wseg_init(txq, loc, wqe, ts, olx); + } else { + /* Legacy cross-channel operation should be used. */ + int32_t wci; + + wci = mlx5_txpp_convert_tx_ts(sh, ts); + if (unlikely(wci < 0)) + return MLX5_TXCMP_CODE_SINGLE; + /* Build the WAIT WQE with specified completion. */ + mlx5_tx_cseg_init(txq, loc, wqe, + 1 + sizeof(struct mlx5_wqe_qseg) / + MLX5_WSEG_SIZE, + MLX5_OPCODE_WAIT | + MLX5_OPC_MOD_WAIT_CQ_PI << 24, olx); + mlx5_tx_qseg_init(txq, loc, wqe, wci, olx); + } ++txq->wqe_ci; --loc->wqe_free; return MLX5_TXCMP_CODE_MULTI; diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c index 3585546628..90b71e380f 100644 --- a/drivers/net/mlx5/mlx5_txq.c +++ b/drivers/net/mlx5/mlx5_txq.c @@ -1302,7 +1302,8 @@ mlx5_txq_dynf_timestamp_set(struct rte_eth_dev *dev) (RTE_MBUF_DYNFLAG_TX_TIMESTAMP_NAME, NULL); off = rte_mbuf_dynfield_lookup (RTE_MBUF_DYNFIELD_TIMESTAMP_NAME, NULL); - if (nbit >= 0 && off >= 0 && sh->txpp.refcnt) + if (nbit >= 0 && off >= 0 && + (sh->txpp.refcnt || priv->config.hca_attr.wait_on_time)) mask = 1ULL << nbit; for (i = 0; i != priv->txqs_n; ++i) { data = (*priv->txqs)[i]; @@ -1312,6 +1313,8 @@ mlx5_txq_dynf_timestamp_set(struct rte_eth_dev *dev) data->ts_mask = mask; data->ts_offset = off; data->rt_timestamp = priv->config.rt_timestamp; - data->rt_timemask = ts_mask; + data->rt_timemask = (data->offloads & + RTE_ETH_TX_OFFLOAD_SEND_ON_TIMESTAMP) ? + ts_mask : 0; } }