From patchwork Mon Feb 14 08:56:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 107415 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 49E4FA00C4; Mon, 14 Feb 2022 09:57:33 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B9BB14114A; Mon, 14 Feb 2022 09:57:28 +0100 (CET) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2064.outbound.protection.outlook.com [40.107.237.64]) by mails.dpdk.org (Postfix) with ESMTP id EF06040C35 for ; Mon, 14 Feb 2022 09:57:26 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jTcV/IlNoTBixAnZZJV9m60glJvR8Jdttq3xk2HyQl6O7sRSaiYr7A7WmAsqpJ5r22rRULUwQt3E/Vb64ELSV55JpY6ds+fevCjSpnEKTR0jiGt0i7sy8QNTg7BiHi7vZYAYhmdVUgwtuZHVg+Q4cdafcyadoLvlFZXkoY6HTch1O8XlYIjqeltGMcNAi3UY53bTWZ9Ahwl4mo5uU/tC6ozODdjfZW8eDB8UVv1l5k+LvEM6D1DGd1ZwdSeo+ik7/4xqrpNDKaDt2f5lVK/Je0FF25s+U3rWPh4RZbA3STsMBwhuSgziAEioUfmcelGMKkyEjOHIFzfOfXAK3yZpAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=z2OeDAk4QG+tGKh24NcXlGhqfhohFuo8gys+NRzQYJU=; b=X6PK8UDe9o7Qug3BNSzrWIQlQL553UeDkS4cBBzA5p6kKhL4RJZEv3QpdhB+ZfLtFkeojXLZC0HrdvsSLKmFPwSRzQBUNVvQ6rtK0uOc4aAQbjcopmNO7UIK9wIsgEM+mu072YIExpB2vcTHxgQlkQU57VynaKzgy/7zyguBW5rRA8S5RaaZcD8g303/IQoNnGJvCOGLEEZWYYKqzMc8JWuGrr5kjQM7SlTmvxIdeX5CBfJSW6l7UXymuYnuoaRy6JZnVwUEH1lXFXyA9DnwNCZ5y5tBpW4T0aWqe+/Z3J9NBN4tcVkeltNSdp4yJV9P0zNGmRs1Pov2XC5QCcPegw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.234) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=z2OeDAk4QG+tGKh24NcXlGhqfhohFuo8gys+NRzQYJU=; b=oXm92h15jVhyRSI3pbqmM7X2Vxztg1qJUTOSglgEGad1tiCXzgVZo/ntkiu0Y5R9lojW5HuqFlwWiin5FkesTp8HXLOtbjrTe0DIQPuBSvNvPlpvGqJyrOx9LmulRAMPmDdbWH/w2qi7eKiO+gl5xVnCOGp0fVihPXtsqGzma7N6XBKxCTEA317iclUOEmraRusdYncRH/jRqmNbaVlDFbL8OK9D+HR0P8KPOLo5Zy5UMF4K5G/bbuoO4aHqxzydDWilsT6/cfXswR1AbWP2/FYEALQ6Z5foQ1XRrcyLE9gRdaz2dpozvTnaUdvfcMHfgMKy+PsvfTMzz/VVkC8x4A== Received: from BN6PR22CA0025.namprd22.prod.outlook.com (2603:10b6:404:37::11) by MWHPR1201MB0160.namprd12.prod.outlook.com (2603:10b6:301:50::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4975.11; Mon, 14 Feb 2022 08:57:25 +0000 Received: from BN8NAM11FT025.eop-nam11.prod.protection.outlook.com (2603:10b6:404:37:cafe::af) by BN6PR22CA0025.outlook.office365.com (2603:10b6:404:37::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4951.19 via Frontend Transport; Mon, 14 Feb 2022 08:57:24 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.234) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.234 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.234; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.234) by BN8NAM11FT025.mail.protection.outlook.com (10.13.177.136) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4975.11 via Frontend Transport; Mon, 14 Feb 2022 08:57:24 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL101.nvidia.com (10.27.9.10) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 14 Feb 2022 08:57:23 +0000 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Mon, 14 Feb 2022 00:57:21 -0800 From: Viacheslav Ovsiienko To: CC: , Subject: [PATCH 2/4] net/mlx5: configure Tx queue with send on time offload Date: Mon, 14 Feb 2022 10:56:53 +0200 Message-ID: <20220214085655.22648-3-viacheslavo@nvidia.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20220214085655.22648-1-viacheslavo@nvidia.com> References: <20220214085655.22648-1-viacheslavo@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 507e9b19-da90-4dce-5694-08d9ef9804f3 X-MS-TrafficTypeDiagnostic: MWHPR1201MB0160:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: F/wcvBwWZhaXYnBplE91QPTp9VE1rnDadK7pyHulMNwX5T4UEkx1xawvhemjBVrjMkh+tebeJO533x/Oxu8lFKnEPAHmg8IWHHUwnAHRPYCBkOctHH+DLpH6YLYkxnL4DtPYl81ybNeRDkL//RH3VcXOiGf7minGNuLB9YwTn+ZL9h1OdPqdv91jsXctInuwMoBuZToffZdimBOWq/BqXqsEKi0Z66D85vOVHkyGYfomojghrAbffZoT/DS1td2Bdfl90tSUcLyKWfrAk+hXim1bH7OmznK4vdw/Bwj0nI5WqdcQgLJJRGlZ9/Dq0ib2TXxjPR69EeAxbyXuFvlD6RydO3fCB0xCntycEPQdwbSlucdyQSgLpvgcD9DsRGgnNlkG3n7doITYPiX+rUxHMSEdT24BpEVvNuseY9jVwlvUyZx1aH+Lm2yWJ/zwlf1R37SjpdRi5ToFEnvXClVkjbQm8nRpAkI/Gm0JN46JIDSgjjt0kMiKBDO6GnKf47zVecV1nnP8MJdA6uKDVXYsuVfY47nyYi7F/YgDpQLybU+/sZEVsi8D/2ATMzqz5cI9noruGJ6oDgrQ+u6DhQbT6s4XJgb7AImuntf8ePwNzZRBQi0ej7/MMUqNAyeWvYpSA5jsItYanVjOFj5hZDxR+r1vAElN7puvIRvWffaa8TSLA+0lp5LIyAgKXxVpsiaLuB+2ULvBWB3jFYOyZyeh8A== X-Forefront-Antispam-Report: CIP:12.22.5.234; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(6286002)(356005)(186003)(54906003)(107886003)(8676002)(81166007)(26005)(508600001)(4326008)(70206006)(36860700001)(2616005)(16526019)(40460700003)(6916009)(55016003)(70586007)(82310400004)(86362001)(316002)(8936002)(47076005)(36756003)(83380400001)(6666004)(5660300002)(7696005)(336012)(426003)(1076003)(2906002)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2022 08:57:24.5040 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 507e9b19-da90-4dce-5694-08d9ef9804f3 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.234]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT025.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1201MB0160 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The wait on time configuration flag is copied to the Tx queue structure due to performance considerations. Timestamp mask is preparted and stored in queue structure as well. Signed-off-by: Viacheslav Ovsiienko --- drivers/net/mlx5/linux/mlx5_verbs.c | 2 ++ drivers/net/mlx5/mlx5.h | 3 +++ drivers/net/mlx5/mlx5_devx.c | 2 ++ drivers/net/mlx5/mlx5_tx.h | 3 +++ drivers/net/mlx5/mlx5_txq.c | 11 ++++++++++- 5 files changed, 20 insertions(+), 1 deletion(-) diff --git a/drivers/net/mlx5/linux/mlx5_verbs.c b/drivers/net/mlx5/linux/mlx5_verbs.c index 2b6eef44a7..80dd0bb6c1 100644 --- a/drivers/net/mlx5/linux/mlx5_verbs.c +++ b/drivers/net/mlx5/linux/mlx5_verbs.c @@ -1036,6 +1036,8 @@ mlx5_txq_ibv_obj_new(struct rte_eth_dev *dev, uint16_t idx) txq_data->wqe_pi = 0; txq_data->wqe_comp = 0; txq_data->wqe_thres = txq_data->wqe_s / MLX5_TX_COMP_THRESH_INLINE_DIV; + txq_data->wait_on_time = !!(!priv->config.tx_pp && + priv->config.hca_attr.wait_on_time); #ifdef HAVE_IBV_FLOW_DV_SUPPORT /* * If using DevX need to query and store TIS transport domain value. diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 737ad6895c..3983d3aa50 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -341,6 +341,9 @@ struct mlx5_lb_ctx { #define MLX5_CNT_ARRAY_IDX(pool, cnt) \ ((int)(((uint8_t *)(cnt) - (uint8_t *)((pool) + 1)) / \ MLX5_CNT_LEN(pool))) +#define MLX5_TS_MASK_SECS 8ull +/* timestamp wrapping in seconds, must be power of 2. */ + /* * The pool index and offset of counter in the pool array makes up the * counter index. In case the counter is from pool 0 and offset 0, it diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c index 91243f684f..c6994e4a75 100644 --- a/drivers/net/mlx5/mlx5_devx.c +++ b/drivers/net/mlx5/mlx5_devx.c @@ -1327,6 +1327,8 @@ mlx5_txq_devx_obj_new(struct rte_eth_dev *dev, uint16_t idx) txq_data->qp_num_8s = txq_obj->sq_obj.sq->id << 8; txq_data->db_heu = sh->cdev->config.dbnc == MLX5_TXDB_HEURISTIC; txq_data->db_nc = sh->tx_uar.dbnc; + txq_data->wait_on_time = !!(!priv->config.tx_pp && + priv->config.hca_attr.wait_on_time); /* Change Send Queue state to Ready-to-Send. */ ret = mlx5_txq_devx_modify(txq_obj, MLX5_TXQ_MOD_RST2RDY, 0); if (ret) { diff --git a/drivers/net/mlx5/mlx5_tx.h b/drivers/net/mlx5/mlx5_tx.h index c4b8271f6f..b50deb8b67 100644 --- a/drivers/net/mlx5/mlx5_tx.h +++ b/drivers/net/mlx5/mlx5_tx.h @@ -138,6 +138,8 @@ struct mlx5_txq_data { uint16_t vlan_en:1; /* VLAN insertion in WQE is supported. */ uint16_t db_nc:1; /* Doorbell mapped to non-cached region. */ uint16_t db_heu:1; /* Doorbell heuristic write barrier. */ + uint16_t rt_timestamp:1; /* Realtime timestamp format. */ + uint16_t wait_on_time:1; /* WQE with timestamp is supported. */ uint16_t fast_free:1; /* mbuf fast free on Tx is enabled. */ uint16_t inlen_send; /* Ordinary send data inline size. */ uint16_t inlen_empw; /* eMPW max packet size to inline. */ @@ -157,6 +159,7 @@ struct mlx5_txq_data { volatile uint32_t *cq_db; /* Completion queue doorbell. */ uint16_t port_id; /* Port ID of device. */ uint16_t idx; /* Queue index. */ + uint64_t rt_timemask; /* Scheduling timestamp mask. */ uint64_t ts_mask; /* Timestamp flag dynamic mask. */ int32_t ts_offset; /* Timestamp field dynamic offset. */ struct mlx5_dev_ctx_shared *sh; /* Shared context. */ diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c index 4e0bf7af9c..3585546628 100644 --- a/drivers/net/mlx5/mlx5_txq.c +++ b/drivers/net/mlx5/mlx5_txq.c @@ -108,7 +108,7 @@ mlx5_get_tx_port_offloads(struct rte_eth_dev *dev) RTE_ETH_TX_OFFLOAD_TCP_CKSUM); if (config->tso) offloads |= RTE_ETH_TX_OFFLOAD_TCP_TSO; - if (config->tx_pp) + if (config->tx_pp || config->hca_attr.wait_on_time) offloads |= RTE_ETH_TX_OFFLOAD_SEND_ON_TIMESTAMP; if (config->swp) { if (config->swp & MLX5_SW_PARSING_CSUM_CAP) @@ -1290,7 +1290,14 @@ mlx5_txq_dynf_timestamp_set(struct rte_eth_dev *dev) int off, nbit; unsigned int i; uint64_t mask = 0; + uint64_t ts_mask; + if (priv->config.rt_timestamp || !priv->config.hca_attr.dev_freq_khz) + ts_mask = MLX5_TS_MASK_SECS << 32; + else + ts_mask = rte_align64pow2(MLX5_TS_MASK_SECS * 1000ull * + priv->config.hca_attr.dev_freq_khz); + ts_mask = rte_cpu_to_be_64(ts_mask - 1ull); nbit = rte_mbuf_dynflag_lookup (RTE_MBUF_DYNFLAG_TX_TIMESTAMP_NAME, NULL); off = rte_mbuf_dynfield_lookup @@ -1304,5 +1311,7 @@ mlx5_txq_dynf_timestamp_set(struct rte_eth_dev *dev) data->sh = sh; data->ts_mask = mask; data->ts_offset = off; + data->rt_timestamp = priv->config.rt_timestamp; + data->rt_timemask = ts_mask; } }