From patchwork Mon Sep 19 16:37:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dariusz Sosnowski X-Patchwork-Id: 116441 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C7A45A00C3; Mon, 19 Sep 2022 18:39:21 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BA19F42829; Mon, 19 Sep 2022 18:39:21 +0200 (CEST) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2042.outbound.protection.outlook.com [40.107.243.42]) by mails.dpdk.org (Postfix) with ESMTP id 21915410E8 for ; Mon, 19 Sep 2022 18:39:21 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bOjCZdlmObpBnX7YsHItiiUpJrF2rUcflrlhJwQ7pc50jnV4/hSaO3hHr92bfou9R63zn9FUNtTijKmD1sc4ira5d5g+tT5A3HmYMtRZ8/FWDKqqvY77vsu4c9vagAhJAZwC708N0kGIjQWL8DRS50W5mWs41OF3H5JpdA3tJyaoeNDqGeTi/wPggK51dQIQ+mAMXJa4Z1pK5Ql5gYNWqQzwlvgmiA3phe22Yot/8boeEvjJRHnZsU4X6VKEgSXyAJlO1Q4W/Qv6YDh/lPFDCnEa/tr2GmnDa3pBGkrQwbb66f3J/iRvscb3Ur9bbdzWKamu3sDiOPdFjLF39WV9MA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=isDuX3nCtipDQi0v1lAF3NHpQ24MmNaxQOo6ocnz6Sc=; b=OVG6IJ5olcg9bwoi3UnlrgizUmiT5cJzg7hU26MZ8LUTASie7K8zadXd4F1R1x21rjf+3iNzB6wKgrqRWOQiwZmqqVsSPYQHe94AOA5y5jewDSDVMCNQcFo9lqUvlTsBTFVZOgXMUbL3a9CZ57uAJ9gq7dRDnsrtloEIfP57AG55icKORscXRmh5fYj4zjAGqKDlk7cZG3l9M0C0jd/Cda/IQOBUtUjFE6MWbmHMiRiyp3EBhvRm98+i7BWlU1/Ol57XBmb31IJOW8dEptiMe8ZZx7YpgzZj0+kGxuU8BH5hodW37IRRLdhscEjwmP30THclNL38iaUBsvvCxHlXQw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=isDuX3nCtipDQi0v1lAF3NHpQ24MmNaxQOo6ocnz6Sc=; b=G/6lSm3yGI7Q4mLmIt4RKRj+MR4rjsoK+YLSOaEriPSvOIgT2KBCZ/GOMPA3A8jjmPX6puv6lhyd6NYVvr23O4jZ9vowvu3rFX/aU1PcKkqxif1RP1zKdRUOKZlYprbdBGELcz2X4OB8X7hZ6xmeqzTX2Oui+6U7WLcx8mn0swQWML5s9hUAilrItjGObl/o2BgmCxzWbJ2wcyK6mXprYRpbxYc4Ycpwgmx3BKcyipJZ/wZV4xWsDT/Giuo4xNaiV0emKt2hh7OpZ2BNvy2ULXbc4UYVJEQCMVUu6rR0kliU+rJHGvZdeJZOT4V6GXj0q+mSls2ZWAHsH04mzhZkBg== Received: from DS7P222CA0013.NAMP222.PROD.OUTLOOK.COM (2603:10b6:8:2e::22) by CY5PR12MB6624.namprd12.prod.outlook.com (2603:10b6:930:40::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.22; Mon, 19 Sep 2022 16:39:19 +0000 Received: from DM6NAM11FT106.eop-nam11.prod.protection.outlook.com (2603:10b6:8:2e:cafe::ae) by DS7P222CA0013.outlook.office365.com (2603:10b6:8:2e::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5632.12 via Frontend Transport; Mon, 19 Sep 2022 16:39:19 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by DM6NAM11FT106.mail.protection.outlook.com (10.13.172.229) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5632.12 via Frontend Transport; Mon, 19 Sep 2022 16:39:19 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Mon, 19 Sep 2022 09:39:08 -0700 Received: from nvidia.com (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Mon, 19 Sep 2022 09:39:06 -0700 From: Dariusz Sosnowski To: Matan Azrad , Viacheslav Ovsiienko CC: Subject: [PATCH 3/7] common/mlx5: add hairpin RQ buffer type capabilities Date: Mon, 19 Sep 2022 16:37:26 +0000 Message-ID: <20220919163731.1540454-4-dsosnowski@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220919163731.1540454-1-dsosnowski@nvidia.com> References: <20220919163731.1540454-1-dsosnowski@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.230.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT106:EE_|CY5PR12MB6624:EE_ X-MS-Office365-Filtering-Correlation-Id: 81be9133-ad98-4366-61cd-08da9a5d7fff X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 2/MwdZ5MaQUyZHad4NfY2gEZqZa3SHp2Zf+raftFSw3/CZGUmjyTPJy+aDDreGW1jchY6dGcQIrd62iYHFyUQH0qiF/7x9YmFJgZLyGNENEQgkmiqKEaLu2FQRN1FeM+I0ud1yCFeEqpFWLeD9YxxNBQTfxzuZuNxb6fiVSQ5xzjpObA+noiPpfom4uRnLRsIW0cT1o9RExVH5ppG4JhSYZ0HCOLMDiP55Cum2pyHfxFXd6mJqi1CVeVydVp5sjHjdUuMUlIOVRIObhJZh9AXyYGdj4qv3jJbRwhC/U5MSCaMtv6JbbAUh1NUHQwtXnMMKXWU87eo/pK3VI1ew8I7JMWjQO59okK/PsRu7ifS2e//7pw89pTkGbQg0DR5Yg3s3/kp1hz/1cnI3SWa7bKD/oCmBHadl7INDSaxbEul0E/g7XzS9CNfRyxdSQ4ZBgjDaP9ioqIUiJOthLWogLN9PABD4mmDtuwSkv0RMV7CqHS8kj6njpqbFpycqGNP4zSl3R0XI23cTSSWeUXccnOn8DV0j8l5WC5ZBRBGQRP7S5vdJo1gWcu03z8LhCjbSpp94d2hj5jNjAlpQYjOmI5bnoC1cW+ncfRypPDvOTjnMboMVCwNqEIhGFou1MWecCr/v1/f+gKe9DEXvqqBgdnVqn5NzcNNQrR8NqXfx23VLLaXBHCE8n8MHlmkfLBbKgi9dGodkZ9QbXbRDPM3lUTWMHfsD2P+KXt7L99Vdyk5R3bgA2hT221XtogxpRPJbfB1Hj2CueCo36wjmcQTm3zTw== X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(396003)(346002)(136003)(39860400002)(376002)(451199015)(36840700001)(40470700004)(46966006)(47076005)(6636002)(478600001)(7636003)(186003)(7696005)(1076003)(55016003)(426003)(4326008)(16526019)(70206006)(26005)(70586007)(40460700003)(8676002)(2616005)(40480700001)(86362001)(356005)(41300700001)(5660300002)(316002)(36756003)(6666004)(336012)(82310400005)(2906002)(110136005)(6286002)(82740400003)(8936002)(36860700001)(83380400001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Sep 2022 16:39:19.5019 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 81be9133-ad98-4366-61cd-08da9a5d7fff X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT106.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6624 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds new HCA capability related to hairpin RQs. This new capability, hairpin_data_buffer_locked, indicates whether HCA supports locking data buffer of hairpin RQ in ICMC (Interconnect Context Memory Cache). Struct used to define RQ configuration (RQ context) is extended with hairpin_data_buffer_type field, which configures data buffer for hairpin RQ. It can take the following values: - MLX5_RQC_HAIRPIN_DATA_BUFFER_TYPE_UNLOCKED_INTERNAL_BUFFER - hairpin RQ's data buffer is stored in unlocked memory in ICMC. - MLX5_RQC_HAIRPIN_DATA_BUFFER_TYPE_LOCKED_INTERNAL_BUFFER - hairpin RQ's data buffer is stored in locked memory in ICMC. Signed-off-by: Dariusz Sosnowski Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/mlx5_devx_cmds.c | 3 +++ drivers/common/mlx5/mlx5_devx_cmds.h | 2 ++ drivers/common/mlx5/mlx5_prm.h | 12 ++++++++++-- 3 files changed, 15 insertions(+), 2 deletions(-) diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index 2b12ce0d4c..95b38783dc 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -985,6 +985,8 @@ mlx5_devx_cmd_query_hca_attr(void *ctx, hairpin_sq_wqe_bb_size); attr->hairpin_sq_wq_in_host_mem = MLX5_GET(cmd_hca_cap_2, hcattr, hairpin_sq_wq_in_host_mem); + attr->hairpin_data_buffer_locked = MLX5_GET(cmd_hca_cap_2, hcattr, + hairpin_data_buffer_locked); } if (attr->log_min_stride_wqe_sz == 0) attr->log_min_stride_wqe_sz = MLX5_MPRQ_LOG_MIN_STRIDE_WQE_SIZE; @@ -1285,6 +1287,7 @@ mlx5_devx_cmd_create_rq(void *ctx, MLX5_SET(rqc, rq_ctx, state, rq_attr->state); MLX5_SET(rqc, rq_ctx, flush_in_error_en, rq_attr->flush_in_error_en); MLX5_SET(rqc, rq_ctx, hairpin, rq_attr->hairpin); + MLX5_SET(rqc, rq_ctx, hairpin_data_buffer_type, rq_attr->hairpin_data_buffer_type); MLX5_SET(rqc, rq_ctx, user_index, rq_attr->user_index); MLX5_SET(rqc, rq_ctx, cqn, rq_attr->cqn); MLX5_SET(rqc, rq_ctx, counter_set_id, rq_attr->counter_set_id); diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index 9ac2d75df4..cceaf3411d 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -193,6 +193,7 @@ struct mlx5_hca_attr { uint32_t log_max_hairpin_num_packets:5; uint32_t hairpin_sq_wqe_bb_size:4; uint32_t hairpin_sq_wq_in_host_mem:1; + uint32_t hairpin_data_buffer_locked:1; uint32_t vhca_id:16; uint32_t relaxed_ordering_write:1; uint32_t relaxed_ordering_read:1; @@ -313,6 +314,7 @@ struct mlx5_devx_create_rq_attr { uint32_t state:4; uint32_t flush_in_error_en:1; uint32_t hairpin:1; + uint32_t hairpin_data_buffer_type:3; uint32_t ts_format:2; uint32_t user_index:24; uint32_t cqn:24; diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index 04d35ca845..9c1c93f916 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -2024,7 +2024,8 @@ struct mlx5_ifc_cmd_hca_cap_2_bits { u8 reserved_at_160[0x3]; u8 hairpin_sq_wqe_bb_size[0x5]; u8 hairpin_sq_wq_in_host_mem[0x1]; - u8 reserved_at_169[0x697]; + u8 hairpin_data_buffer_locked[0x1]; + u8 reserved_at_16a[0x696]; }; struct mlx5_ifc_esw_cap_bits { @@ -2304,7 +2305,9 @@ struct mlx5_ifc_rqc_bits { u8 reserved_at_c[0x1]; u8 flush_in_error_en[0x1]; u8 hairpin[0x1]; - u8 reserved_at_f[0xB]; + u8 reserved_at_f[0x6]; + u8 hairpin_data_buffer_type[0x3]; + u8 reserved_at_a8[0x2]; u8 ts_format[0x02]; u8 reserved_at_1c[0x4]; u8 reserved_at_20[0x8]; @@ -2813,6 +2816,11 @@ enum { MLX5_CQE_SIZE_128B = 0x1, }; +enum { + MLX5_RQC_HAIRPIN_DATA_BUFFER_TYPE_UNLOCKED_INTERNAL_BUFFER = 0x0, + MLX5_RQC_HAIRPIN_DATA_BUFFER_TYPE_LOCKED_INTERNAL_BUFFER = 0x1, +}; + struct mlx5_ifc_cqc_bits { u8 status[0x4]; u8 as_notify[0x1];