From patchwork Tue Nov 23 18:38:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 104623 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D1219A0C4C; Tue, 23 Nov 2021 19:38:39 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 73403410E5; Tue, 23 Nov 2021 19:38:34 +0100 (CET) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2045.outbound.protection.outlook.com [40.107.223.45]) by mails.dpdk.org (Postfix) with ESMTP id B487240040; Tue, 23 Nov 2021 19:38:32 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LN/TZ2r0wcXANoiBIV76d18DVBbF4vEfknBlMW2zME8hppQT/h8QXCs8wp2JQvedwjuT7z/QyLlyTwXNiLKJ27pPfjpWGWq+oaWls3n1/4Z+M7YFqu3mXFTQ3BsVDqJ7+PuwFjjpp08MH2CbWReseL6qb6wDSH41WBZeBQYltQzR7ehD0KyXcOmj/ENTpEqcYNJnuLkY8FVFoGQXQrKRIWakJa2XJsBiqJnRRMlrHsi3DjV8OM3WBj5J9JattvqRXnWPjkorKtWow9cggeAs5Ilzp70GGY6tMBpzwYyHKMxaYFeCZ2qYkpXW5j9/yeBidTd2/8oMUUm+JGRMD9stug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=j5UWnir1DK/ckQe9C/W6UehjuvtiqOYDOusMlwYdDAA=; b=DdXA9hEg24qiQd27xC4yv/9ra1E9RE1LUtDxw/YzkjoWEzx7xy69KjzC88ulJVZ/M91FiPZ0xetHitrQWEneroKFkmw4OSSW8O2wdXWknlsRz1oCWTmTMdcsanIRYRLZj3Ti3+zsB5xuQ4fO8jxvAwwhtYMVqZW6zcrmSqlMIWcx4ffUCNrPKmu7F6IfZpnfpeQp6QYSWk5LfnE0ujTsL4TssvfZnsuwk0IlBrrlmU240UmLVeqsD+GbsvXrXMd/8wUwtaMOSVfZZy6KzkQbBV0JQAdWw59SMhxtUaT4SBcWKVOyvztGWGvWEo5J73jWoGbOEdeLULSI/40jWQRkpA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=j5UWnir1DK/ckQe9C/W6UehjuvtiqOYDOusMlwYdDAA=; b=G5ooZGLgxcDBu+f47xZFXJ//45nptHbqPjnt8s2Ga6yfpYbJA9VMRoouGOgv/RZg0r2mEG0YJHUegxrJtQ2YUj3S5hPiTEKey5c+EFfmSmFcTdE/OdZFshfhpV/1jbLtLag+eaLGPNpeL3PkgT9w7baiA6ZmPmDfaTH8dcUBdL5epW3ipF+knjn1QBhMoOlfo65C/8qiaNI8QQewGz4PNUQ89oShFlstHY48xk093g7RBC3XL3TJqrBDz6vuyu1CeG+4wfhNDil8yUgZbGXUCCJy3FntCGzgiVP7X79hGY9apkFvzyjIG8BdCyFMty2QBjo7jq52qKwnULX222sIyw== Received: from DM5PR22CA0016.namprd22.prod.outlook.com (2603:10b6:3:101::26) by DM4PR12MB5325.namprd12.prod.outlook.com (2603:10b6:5:39e::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.13; Tue, 23 Nov 2021 18:38:30 +0000 Received: from DM6NAM11FT048.eop-nam11.prod.protection.outlook.com (2603:10b6:3:101:cafe::77) by DM5PR22CA0016.outlook.office365.com (2603:10b6:3:101::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4734.20 via Frontend Transport; Tue, 23 Nov 2021 18:38:30 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by DM6NAM11FT048.mail.protection.outlook.com (10.13.173.114) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4713.20 via Frontend Transport; Tue, 23 Nov 2021 18:38:29 +0000 Received: from nvidia.com (172.20.187.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 23 Nov 2021 18:38:27 +0000 From: To: CC: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko , Michael Baum , Subject: [PATCH 1/3] common/mlx5: add min WQE size for striding RQ Date: Tue, 23 Nov 2021 20:38:03 +0200 Message-ID: <20211123183805.2905792-2-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211123183805.2905792-1-michaelba@nvidia.com> References: <20211123183805.2905792-1-michaelba@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e30f956d-d5f9-4612-dcf1-08d9aeb071f9 X-MS-TrafficTypeDiagnostic: DM4PR12MB5325: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4714; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3pQZpvgDcsawpl9o9wvBvctMbpbMtuaYIgRNXqniRpCd3fWzO6W54e81lJFQF4aZs8pXBnV/vudz0Jbqs4J533aHRpPxgp+xUVSBRD9gVdJzcS4nUuUlmr+QrMGsC9MrUPO7WZaVn1wZCyRF9EXkYBRwDUTkAWunvkw2k7u/NU5+jva5mbQ7HJTJ/UyXzkIJ7dzPjQj8bqYJKA/3cVQ/Pid43qcohU5xlnqqmqKJbPV0IWoZjgK5FUMVpuFIvkMPbYUDSZmA+aONMqb57rHgrQVfcT5NXuB1fogKTv31v2Z+YxcOr8ZWdZ5GqPr9hE36JDq1Dvcz56psazZI/hdLVxyyDaSQrFNyqWTz5IlciUVupDEiFgRWZZaKhiaInnEAAfupc3IcN6rDidWzlGA5jl5fXdZhQgH6hlapvmERzCWHWl7g1zrqmHRqXXuRNeEUSw4+chLtEFbuC4MT/IbA4InVvDVeaCrU2c1axjrpFlxluVLxnDnisN5mACIuwAcpEt82WhsRHsB1mBJDHAGELr1LnraKgcMdo5CuGQCaZSDdwbcvPs3SGvwPKfl0HGVLoHKIGxR0bMTdmYtbskJp+4r4M4nUV1ajuC64jZFaKxWDAgwOTjKq+676RFncgBxt1DFflScnXNXWKMEuluiM77i3JUoZFyHjsz7OGx7EKby8In2tJ4BVgrhRcQWnBvI1KFcNoVLukHplY2YwjxVASA== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(508600001)(336012)(6916009)(70586007)(16526019)(36860700001)(70206006)(55016003)(5660300002)(356005)(6286002)(7696005)(2906002)(1076003)(47076005)(54906003)(186003)(8676002)(7636003)(26005)(426003)(6666004)(86362001)(450100002)(2616005)(316002)(36756003)(2876002)(83380400001)(8936002)(82310400004)(4326008); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Nov 2021 18:38:29.8323 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e30f956d-d5f9-4612-dcf1-08d9aeb071f9 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT048.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5325 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Michael Baum Some devices have a WQE size limit for striding RQ. On some newer devices, this limitation is smaller and information on its size is provided by the firmware. This patch adds the attribute query from firmware: the minimum required size of WQE in a strided RQ in granularity of Bytes. Cc: stable@dpdk.org Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/common/mlx5/mlx5_devx_cmds.c | 16 ++++++++++++++++ drivers/common/mlx5/mlx5_devx_cmds.h | 1 + drivers/common/mlx5/mlx5_prm.h | 11 +++++++++-- 3 files changed, 26 insertions(+), 2 deletions(-) diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index e52b995ee3..a8efdbe1ae 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -823,6 +823,7 @@ mlx5_devx_cmd_query_hca_attr(void *ctx, { uint32_t in[MLX5_ST_SZ_DW(query_hca_cap_in)] = {0}; uint32_t out[MLX5_ST_SZ_DW(query_hca_cap_out)] = {0}; + bool hca_cap_2_sup; uint64_t general_obj_types_supported = 0; void *hcattr; int rc, i; @@ -832,6 +833,7 @@ mlx5_devx_cmd_query_hca_attr(void *ctx, MLX5_HCA_CAP_OPMOD_GET_CUR); if (!hcattr) return rc; + hca_cap_2_sup = MLX5_GET(cmd_hca_cap, hcattr, hca_cap_2); attr->max_wqe_sz_sq = MLX5_GET(cmd_hca_cap, hcattr, max_wqe_sz_sq); attr->flow_counter_bulk_alloc_bitmap = MLX5_GET(cmd_hca_cap, hcattr, flow_counter_bulk_alloc); @@ -967,6 +969,20 @@ mlx5_devx_cmd_query_hca_attr(void *ctx, general_obj_types) & MLX5_GENERAL_OBJ_TYPES_CAP_CONN_TRACK_OFFLOAD); attr->rq_delay_drop = MLX5_GET(cmd_hca_cap, hcattr, rq_delay_drop); + if (hca_cap_2_sup) { + hcattr = mlx5_devx_get_hca_cap(ctx, in, out, &rc, + MLX5_GET_HCA_CAP_OP_MOD_GENERAL_DEVICE_2 | + MLX5_HCA_CAP_OPMOD_GET_CUR); + if (!hcattr) { + DRV_LOG(DEBUG, + "Failed to query DevX HCA capabilities 2."); + return rc; + } + attr->log_min_stride_wqe_sz = MLX5_GET(cmd_hca_cap_2, hcattr, + log_min_stride_wqe_sz); + } + if (attr->log_min_stride_wqe_sz == 0) + attr->log_min_stride_wqe_sz = MLX5_MPRQ_LOG_MIN_STRIDE_WQE_SIZE; if (attr->qos.sup) { hcattr = mlx5_devx_get_hca_cap(ctx, in, out, &rc, MLX5_GET_HCA_CAP_OP_MOD_QOS_CAP | diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index d7f71646a3..37821b493e 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -251,6 +251,7 @@ struct mlx5_hca_attr { uint32_t log_max_mmo_decompress:5; uint32_t umr_modify_entity_size_disabled:1; uint32_t umr_indirect_mkey_disabled:1; + uint32_t log_min_stride_wqe_sz:5; uint16_t max_wqe_sz_sq; }; diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index 2ded67e85e..8a7cb0e673 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -264,6 +264,9 @@ /* The maximum log value of segments per RQ WQE. */ #define MLX5_MAX_LOG_RQ_SEGS 5u +/* Log 2 of the default size of a WQE for Multi-Packet RQ. */ +#define MLX5_MPRQ_LOG_MIN_STRIDE_WQE_SIZE 14U + /* The alignment needed for WQ buffer. */ #define MLX5_WQE_BUF_ALIGNMENT rte_mem_page_size() @@ -1342,7 +1345,9 @@ enum { #define MLX5_STEERING_LOGIC_FORMAT_CONNECTX_6DX 0x1 struct mlx5_ifc_cmd_hca_cap_bits { - u8 reserved_at_0[0x30]; + u8 reserved_at_0[0x20]; + u8 hca_cap_2[0x1]; + u8 reserved_at_21[0xf]; u8 vhca_id[0x10]; u8 reserved_at_40[0x20]; u8 reserved_at_60[0x3]; @@ -1909,7 +1914,8 @@ struct mlx5_ifc_cmd_hca_cap_2_bits { u8 max_reformat_insert_offset[0x8]; u8 max_reformat_remove_size[0x8]; u8 max_reformat_remove_offset[0x8]; /* End of DW6. */ - u8 aso_conntrack_reg_id[0x8]; + u8 reserved_at_c0[0x3]; + u8 log_min_stride_wqe_sz[0x5]; u8 reserved_at_c8[0x3]; u8 log_conn_track_granularity[0x5]; u8 reserved_at_d0[0x3]; @@ -1922,6 +1928,7 @@ struct mlx5_ifc_cmd_hca_cap_2_bits { union mlx5_ifc_hca_cap_union_bits { struct mlx5_ifc_cmd_hca_cap_bits cmd_hca_cap; + struct mlx5_ifc_cmd_hca_cap_2_bits cmd_hca_cap_2; struct mlx5_ifc_per_protocol_networking_offload_caps_bits per_protocol_networking_offload_caps; struct mlx5_ifc_qos_cap_bits qos_cap;