From patchwork Tue Feb 28 16:43:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Kozyrev X-Patchwork-Id: 124576 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1343C41D9E; Tue, 28 Feb 2023 17:43:54 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 48DBE42C4D; Tue, 28 Feb 2023 17:43:48 +0100 (CET) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2049.outbound.protection.outlook.com [40.107.237.49]) by mails.dpdk.org (Postfix) with ESMTP id 463E342BFE for ; Tue, 28 Feb 2023 17:43:47 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hzzJ3yxZ70CNKs6pGuxxHaq/ZmWKiNfNC+CS1E/qmO/e0bXdpzy1vkLGJG7RC2hD9Uis1Xj2VcANt17MHmulC7Spx6McUFgCuFU52F2I0iQbj5IaplKgkn9bLFtvgenvbxo9yHYpfE1MVMY22Ici4eqrkXLD0OXMUO8THee7Xj4Ar0gnitBih+97XOhSce67ZSQhiwC900uwoN+tKKdDsfO1Uecs6CEmOC8qVnnuhVXweyqCzh0HAzj5iDD9A0t3Lpo4NMEOpQRXnyhbQky4ecNEcqo+Z779m6SSIBKK6OBwAlrZQkZ1OdwwNQZWuBQSfWIFJT25dwsL49T/6K+6SA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=r1M7qO0R7oVB+xtud9Qr2m4+HeZ2WJyOrH8pnf3lTiM=; b=aKRBQwrYU1Q1DWdIdh2dymCEU8DAMNjwqowKFtidFp4KmTymmijJHf2yAdqgk2CHJIwjFMr6ui1Wh45puCL4JhICcbvf5fMgyPJ02au5hg9161qkzNzR5yA+Pc2ddnbsGdP3uWQqa3kH75NDiF24hqFs0BcrurLLVpstawo0bbTiLQH5nDVx8u1age/pkOVCeDcS+SISmT9SRZ0bHxNkkkK3NwbSfbRqwtisCn/udfYdcZfOOGTIwkCF4F9HDChIrpyhVgmMWZNacm5JzhWQBZdi1bC8gDYqmswSIBoIY5fLJSbi53QJwKbixxDtVcSZCrDqc8xsxKmI58t5NwOUOQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=r1M7qO0R7oVB+xtud9Qr2m4+HeZ2WJyOrH8pnf3lTiM=; b=Pe2GiRrFTh7un1FHtVcfrVVkncFkw9/PRFN+boJoeng796jhv8l0sQmJUhShLwI50sBelVFrx9C1h5Bis13K5ayRmjc3uof2a+IsZAytJQlHOXbOUDdRYGqyRsmiUYyssEDe5u+Zt9hUdwtO7T+ov8JV5bgRxrXH7cu7m/UsjctNPKpMpxCQ2F7Q8+OHQ1A2TlsqSAuZaiL8I3XD5o3ve0H13/m6tYYd3i8geiaL47XEhdu8wgdnIN8Qiwjedb+jsAE+gLlmR+RdOjH+sWroUi90Z93hO9vp79te194RD/K8+3+ER5+ELgjEovNWTOKIt0tvuUkVUJxZXIpIc+VeOw== Received: from BN8PR03CA0013.namprd03.prod.outlook.com (2603:10b6:408:94::26) by DS0PR12MB8017.namprd12.prod.outlook.com (2603:10b6:8:146::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6134.30; Tue, 28 Feb 2023 16:43:45 +0000 Received: from BN8NAM11FT043.eop-nam11.prod.protection.outlook.com (2603:10b6:408:94:cafe::78) by BN8PR03CA0013.outlook.office365.com (2603:10b6:408:94::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6134.29 via Frontend Transport; Tue, 28 Feb 2023 16:43:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN8NAM11FT043.mail.protection.outlook.com (10.13.177.218) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6156.14 via Frontend Transport; Tue, 28 Feb 2023 16:43:45 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Tue, 28 Feb 2023 08:43:28 -0800 Received: from pegasus01.mtr.labs.mlnx (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Tue, 28 Feb 2023 08:43:26 -0800 From: Alexander Kozyrev To: CC: , , Subject: [PATCH 1/5] common/mlx5: detect enhanced CQE compression capability Date: Tue, 28 Feb 2023 18:43:06 +0200 Message-ID: <20230228164310.807594-2-akozyrev@nvidia.com> X-Mailer: git-send-email 2.18.2 In-Reply-To: <20230228164310.807594-1-akozyrev@nvidia.com> References: <20230228164310.807594-1-akozyrev@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.230.37] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT043:EE_|DS0PR12MB8017:EE_ X-MS-Office365-Filtering-Correlation-Id: 09b6c986-29f6-4178-2700-08db19aaf580 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: jSXWBHs9MvbHYSJsepJfK5Gn0lUcJW1jdwy1oM6JaXiHjGFkoH13ZsYuKqpbZCPAkE9n2olOvYiSePmIBQo/sR9g7XbgIKDr7OtfGUqGETQlK+fbO1ZULtrCK5q2bSlLps8oepBEYNZNYTVTEeuPpz6kGSRcXbw8cddzXW5mfUOfPOOB8cBZVkg3KpTwyfLDfqNonZup5QkVQ7VZqabqHNWJv9yY8wXXvBgQlacK/dZfUWWBrFh1/AeyZ4hKFe8T6ACwbvXn4BnbaKsQ3waQRisAGxmHWufuKJLfgh+sIwFlPAIwKS9tLuqCc7gY4G193PRcE4vuW2vdEYU+DJtNSX+RVeSzwhrMiOE1qa7uGnXiJhBHxpLedYgsemDnlIq3S9LSni92NC51/nkN7cpm5WwfoGz9isO31LSAsZLnm0qBxrWhst4P3FA4ZiGqWFwHxUnISCMq71qsjk7IzNZHDre1d5i2LxqDktdSAd2cWbSVw/TOvb0/HYSL39+IQ132a4+fP4PtVfd4oXkCwRT8Snt59hIhjq6mQY/UXtSMxY2LyTo2P7ocNAj2sIM3dBtYt6n6ydgzE53jg+/sPTZQRhbvGQsCoGNaghWpk1JJhJmej7dnRPcG5Vra83LNmB7pBjANE07kDB4fs15aZUcxttnhss8k6prqiLRdibLYnigLfpXeLlKv45R9816XScjrIbeFwXpcKbCZWhCg8qH3Qf++Y3rcqFYZYdRMku6PjJYaywQL6Nai7Dw6dTvcRQQg X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230025)(4636009)(396003)(376002)(136003)(39860400002)(346002)(451199018)(40470700004)(36840700001)(46966006)(36756003)(86362001)(8676002)(6916009)(8936002)(4326008)(41300700001)(70586007)(70206006)(5660300002)(2906002)(34020700004)(356005)(36860700001)(7636003)(82740400003)(316002)(478600001)(107886003)(6666004)(336012)(40480700001)(40460700003)(82310400005)(426003)(54906003)(47076005)(16526019)(2616005)(186003)(83380400001)(1076003)(26005)(309714004); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2023 16:43:45.4821 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 09b6c986-29f6-4178-2700-08db19aaf580 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT043.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8017 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Enhanced CQE Compression is designed for better latency and SW utilization. Check the HCA capabilities to see if Enhanced CQE Compression is supported. Basic or Enhanced CQE Compression can be set as the CQE Compression Layout. Enhanced CQE Compression can be selected only if it is supported by the FW. Signed-off-by: Alexander Kozyrev Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/mlx5_devx_cmds.c | 3 +++ drivers/common/mlx5/mlx5_devx_cmds.h | 2 ++ drivers/common/mlx5/mlx5_prm.h | 30 +++++++++++++++++++++++----- 3 files changed, 30 insertions(+), 5 deletions(-) diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index bfc6e09eac..8b5582701f 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -1001,6 +1001,8 @@ mlx5_devx_cmd_query_hca_attr(void *ctx, mini_cqe_resp_flow_tag); attr->mini_cqe_resp_l3_l4_tag = MLX5_GET(cmd_hca_cap, hcattr, mini_cqe_resp_l3_l4_tag); + attr->enhanced_cqe_compression = MLX5_GET(cmd_hca_cap, hcattr, + enhanced_cqe_compression); attr->umr_indirect_mkey_disabled = MLX5_GET(cmd_hca_cap, hcattr, umr_indirect_mkey_disabled); attr->umr_modify_entity_size_disabled = @@ -2059,6 +2061,7 @@ mlx5_devx_cmd_create_cq(void *ctx, struct mlx5_devx_cq_attr *attr) MLX5_SET(cqc, cqctx, c_eqn, attr->eqn); MLX5_SET(cqc, cqctx, uar_page, attr->uar_page_id); MLX5_SET(cqc, cqctx, cqe_comp_en, !!attr->cqe_comp_en); + MLX5_SET(cqc, cqctx, cqe_comp_layout, !!attr->cqe_comp_layout); MLX5_SET(cqc, cqctx, mini_cqe_res_format, attr->mini_cqe_res_format); MLX5_SET(cqc, cqctx, mini_cqe_res_format_ext, attr->mini_cqe_res_format_ext); diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index 8e68eeaf37..44b627daf2 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -244,6 +244,7 @@ struct mlx5_hca_attr { uint32_t cqe_compression:1; uint32_t mini_cqe_resp_flow_tag:1; uint32_t mini_cqe_resp_l3_l4_tag:1; + uint32_t enhanced_cqe_compression:1; uint32_t pkt_integrity_match:1; /* 1 if HW supports integrity item */ struct mlx5_hca_qos_attr qos; struct mlx5_hca_vdpa_attr vdpa; @@ -468,6 +469,7 @@ struct mlx5_devx_cq_attr { uint32_t cqe_comp_en:1; uint32_t mini_cqe_res_format:2; uint32_t mini_cqe_res_format_ext:2; + uint32_t cqe_comp_layout:2; uint32_t log_cq_size:5; uint32_t log_page_size:5; uint32_t uar_page_id; diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index 54766c2d65..aa291f19a6 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -1715,10 +1715,28 @@ struct mlx5_ifc_cmd_hca_cap_bits { u8 max_geneve_tlv_options[0x8]; u8 reserved_at_568[0x3]; u8 max_geneve_tlv_option_data_len[0x5]; - u8 reserved_at_570[0x49]; + u8 flex_parser_header_modify[0x1]; + u8 reserved_at_571[0x2]; + u8 log_max_guaranteed_connections[0x5]; + u8 driver_version_before_init_hca[0x1]; + u8 adv_virtualization[0x1]; + u8 reserved_at_57a[0x1]; + u8 log_max_dct_connections[0x5]; + u8 log_max_atomic_size_qp[0x8]; + u8 reserved_at_587[0x3]; + u8 log_max_dci_stream_channels[0x5]; + u8 reserved_at_58f[0x3]; + u8 log_max_dci_errored_streams[0x5]; + u8 log_max_atomic_dize_dc[0x8]; + u8 max_multi_user_ggroup_size[0x10]; + u8 enhanced_cqe_compression[0x1]; + u8 reserved_at_5b0[0x1]; + u8 crossing_vhca_mkey[0x1]; + u8 log_max_dek[0x5]; + u8 reserved_at_5b7[0x1]; u8 mini_cqe_resp_l3_l4_tag[0x1]; u8 mini_cqe_resp_flow_tag[0x1]; - u8 enhanced_cqe_compression[0x1]; + u8 reserved_at_5ba[0x1]; u8 mini_cqe_resp_stride_index[0x1]; u8 cqe_128_always[0x1]; u8 cqe_compression_128[0x1]; @@ -3042,7 +3060,7 @@ struct mlx5_ifc_cqc_bits { u8 as_notify[0x1]; u8 initiator_src_dct[0x1]; u8 dbr_umem_valid[0x1]; - u8 reserved_at_7[0x1]; + u8 ext_element[0x1]; u8 cqe_sz[0x3]; u8 cc[0x1]; u8 reserved_at_c[0x1]; @@ -3052,8 +3070,10 @@ struct mlx5_ifc_cqc_bits { u8 cqe_comp_en[0x1]; u8 mini_cqe_res_format[0x2]; u8 st[0x4]; - u8 reserved_at_18[0x1]; - u8 cqe_comp_layout[0x7]; + u8 always_armed_cq[0x1]; + u8 ext_element_type[0x3]; + u8 reserved_at_1c[0x2]; + u8 cqe_comp_layout[0x2]; u8 dbr_umem_id[0x20]; u8 reserved_at_40[0x14]; u8 page_offset[0x6];