From patchwork Tue Apr 12 14:59:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tal Shnaiderman X-Patchwork-Id: 109615 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 62803A050A; Tue, 12 Apr 2022 17:01:45 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0796B42806; Tue, 12 Apr 2022 17:01:27 +0200 (CEST) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2053.outbound.protection.outlook.com [40.107.92.53]) by mails.dpdk.org (Postfix) with ESMTP id 44553427ED for ; Tue, 12 Apr 2022 17:01:25 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eoxziAMTPWCL7X32vXO2gkpbJEZdh28Phc/qyAEA8u3ng/sKMLyYigo0KnWTqNoHZWbDEYy0hw9uXGHJhc3+x7wyXCwV9dJRHi40MEAZMRE8jcLNjrdjh4TMhaj94GI7/qfnc4FOn9RZVmJT2ADzeoU0KeDQgdyuZIhRgXPqrh1PBlsGtynZVwO2aPINKSTKx7tCTCuK8Zud7TU9U6P5CGK/EiGmn/dxV9+BKmBQfsWqIJbbkdWH82i/yrTSwFT3r2FqBKigDXhHDLaHifRtOrOhjzsso1+Kpgec5n/cMYsjr8aXT6NEWQmdgZPryAqLPK4bEW7dSviXM41FFERhnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nncNWP1rzhPAfiJ5aJKz8ItLkAbvKzjSpXo0IHEh74w=; b=fEUsuG4LBEyo4hsMTz6NH/kTdAJFHiASGlsKl6kvzOlsAIWTds2VOpl4exK1Idi5S71oleh6TkF81rPgHU64+bYs8ILj+hHBS+sbOEbsoAeYvCXV4WfZEyn3gQANKf9/khXx/QFG5gwBopxn9AvkWoXV7vVO1jFBYC1BNMVy96cwVMwCyZaYuzABOu8OiEKKP4eE5pGiJjgUsh/MjQr0WsnFTIbjexT0nQjvtVar8DQNpGF9D68TVtChAs/SVQLS+zSk+vGINnMaAvnVE6Ly1XFh4AkoycNwXM5/b8cCoOm27rGqubbpEaKWeA9pFzmAZuh8t3TAjooK+ZhBX6Fq4Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=temperror (sender ip is 12.22.5.236) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=temperror action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nncNWP1rzhPAfiJ5aJKz8ItLkAbvKzjSpXo0IHEh74w=; b=iKAS0dRnEuKuAHdTBMbiCotq69jI6lsODQM/y0mp70/nH/irIPnI/GOnfeRq2vxpNW+nWL4JggbptxDWT6algWIkT8Eu/FCNxpDIZN5Ayya6EtqIxG4N83iAUcwnNTpDJPsBAQrRvGhDCZ0YPXmXdi2++Rxn6V4skYqI3O5wyyMA/Sm9UCksDE7R4nWtEN/yw9+j/2M5A+BJVBSq78NJWsJbwKDDPmeLpTjgwMnVAemse0QhgkRFoqj5WKDSiiNnwntTwJAatc3U3VgMnADYMLrJ2sobxsa6qIMrRckDW4LaZvLf2ZAss2HQTLTNu/Jbr8Vky0SJaJGRseLpIS5icA== Received: from DS7PR03CA0107.namprd03.prod.outlook.com (2603:10b6:5:3b7::22) by BN6PR12MB1444.namprd12.prod.outlook.com (2603:10b6:405:10::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5144.29; Tue, 12 Apr 2022 15:01:22 +0000 Received: from DM6NAM11FT063.eop-nam11.prod.protection.outlook.com (2603:10b6:5:3b7:cafe::21) by DS7PR03CA0107.outlook.office365.com (2603:10b6:5:3b7::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5144.23 via Frontend Transport; Tue, 12 Apr 2022 15:01:22 +0000 X-MS-Exchange-Authentication-Results: spf=temperror (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=temperror action=none header.from=nvidia.com; Received-SPF: TempError (protection.outlook.com: error in processing during lookup of nvidia.com: DNS Timeout) Received: from mail.nvidia.com (12.22.5.236) by DM6NAM11FT063.mail.protection.outlook.com (10.13.172.219) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5144.20 via Frontend Transport; Tue, 12 Apr 2022 15:01:20 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Tue, 12 Apr 2022 15:01:09 +0000 Received: from nvidia.com (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Tue, 12 Apr 2022 08:01:07 -0700 From: Tal Shnaiderman To: CC: , Subject: [RFC PATCH 1/3] net/mlx5: support multi-packet RQ on Windows Date: Tue, 12 Apr 2022 17:59:58 +0300 Message-ID: <20220412150000.3412-2-talshn@nvidia.com> X-Mailer: git-send-email 2.16.1.windows.4 In-Reply-To: <20220412150000.3412-1-talshn@nvidia.com> References: <20220412150000.3412-1-talshn@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.230.35] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 31166148-ca94-4d0e-3b49-08da1c954dd7 X-MS-TrafficTypeDiagnostic: BN6PR12MB1444:EE_ X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: A2+PLvCd+QwO5PHrWn4Ftz14k5sDIUdvO4/4oWHF/E5MFeb0QziNLiw6gCuBzeTCwc47NHG3MPAoXkmKu93e6RsIth4gBXwLOaIgyk6xkY+uMwWsSprZvI12+uvSqNCnLPXT6esOcrJeTI+ur683VggPx+y8RbpSnE0YBW5JbIHYzoXNQF6QKU5mpWPLVSDLr1QEymsjBj5CJ9dXaJ3XwZNmJUOUqQ5s3PIfT08iwpQ8/dIr/1YbPx7dJfVwX+HUTTxyhBaM9lAWM++mKhIxNoiQwTyTGfwh+kpjnjxkKmtyALaABxLla8FREShDJnCS1iveEqxv3dmjZlq436Ea9B4G8W0MFQezqbhHnhdz3d0Ug2QLa2Qwqg0F1yvWtuUaXycUZRfCcY/3/pRhP7SB1EzUCjavkhdXKNt/n1H7CS273DH3KR9XWoRP/Bjg/YQiHwLaznsGLXTTQN/8nWn3XhoFDD0tvH13qrOmdE5ySgEo2jG0Q6GI7C9h01PI9M0WYMqPxhBRbfgrFF8UC363SmayQlvQ4J/vRKVXgWkP8apAkxBcmE28nRqHW8LzpdyxU3GOgpyMRuARNlEV34VqG4itMRz6E+nrHf6Je3s1AHKlrR/nm81rcaNwUAe6OAj4fWB+VzBR+5qHiKfuj5Tc60sC6/yRnfrumQTyCYBoCYRj7DrIEZQ33FEOr74jPsc2fkiEz/fKCMZAcZ6BS2Y6ZQ== X-Forefront-Antispam-Report: CIP:12.22.5.236; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(46966006)(40470700004)(36840700001)(26005)(1076003)(5660300002)(86362001)(83380400001)(316002)(63370400001)(40460700003)(186003)(508600001)(63350400001)(6916009)(36756003)(107886003)(6666004)(54906003)(8936002)(426003)(16526019)(2906002)(336012)(47076005)(82310400005)(2616005)(7696005)(55016003)(81166007)(70206006)(36860700001)(356005)(70586007)(6286002)(4326008)(8676002)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Apr 2022 15:01:20.6596 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 31166148-ca94-4d0e-3b49-08da1c954dd7 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.236]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT063.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR12MB1444 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Multi-Packet RQ can further save PCIe bandwidth by posting a single large buffer for multiple packets. Instead of posting a buffer per a packet, one large buffer is posted to receive multiple packets on the buffer. Add support for multi-packet RQ on Windows. The feature is disabled by default and can by enabled by setting mprq_en=1 in the PMD specific arguments. Signed-off-by: Tal Shnaiderman --- drivers/common/mlx5/mlx5_devx_cmds.c | 3 +++ drivers/common/mlx5/mlx5_devx_cmds.h | 2 ++ drivers/common/mlx5/windows/mlx5_win_defs.h | 7 +++++++ drivers/net/mlx5/windows/mlx5_os.c | 26 ++++++++++++++++++++++++++ 4 files changed, 38 insertions(+) diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index d02ac2a678..7900254287 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -970,6 +970,9 @@ mlx5_devx_cmd_query_hca_attr(void *ctx, general_obj_types) & MLX5_GENERAL_OBJ_TYPES_CAP_CONN_TRACK_OFFLOAD); attr->rq_delay_drop = MLX5_GET(cmd_hca_cap, hcattr, rq_delay_drop); + attr->striding_rq = MLX5_GET(cmd_hca_cap, hcattr, striding_rq); + attr->ext_stride_num_range = + MLX5_GET(cmd_hca_cap, hcattr, ext_stride_num_range); if (hca_cap_2_sup) { hcattr = mlx5_devx_get_hca_cap(ctx, in, out, &rc, MLX5_GET_HCA_CAP_OP_MOD_GENERAL_DEVICE_2 | diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index 1bac18c59d..2d813c0fdc 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -256,6 +256,8 @@ struct mlx5_hca_attr { uint32_t esw_mgr_vport_id_valid:1; /* E-Switch Mgr vport ID is valid. */ uint16_t esw_mgr_vport_id; /* E-Switch Mgr vport ID . */ uint16_t max_wqe_sz_sq; + uint32_t striding_rq:1; + uint32_t ext_stride_num_range:1; }; /* LAG Context. */ diff --git a/drivers/common/mlx5/windows/mlx5_win_defs.h b/drivers/common/mlx5/windows/mlx5_win_defs.h index 3554e4a7ff..822104a109 100644 --- a/drivers/common/mlx5/windows/mlx5_win_defs.h +++ b/drivers/common/mlx5/windows/mlx5_win_defs.h @@ -258,4 +258,11 @@ enum { MLX5_MATCH_MISC_PARAMETERS = 1 << 1, MLX5_MATCH_INNER_HEADERS = 1 << 2, }; + +#define MLX5_MIN_SINGLE_WQE_LOG_NUM_STRIDES 9 +#define MLX5_MAX_SINGLE_WQE_LOG_NUM_STRIDES 16 +#define MLX5_MIN_SINGLE_STRIDE_LOG_NUM_BYTES 6 +#define MLX5_MAX_SINGLE_STRIDE_LOG_NUM_BYTES 13 +#define MLX5_EXT_MIN_SINGLE_WQE_LOG_NUM_STRIDES 3 +#define IB_QPT_RAW_PACKET 8 #endif /* __MLX5_WIN_DEFS_H__ */ diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c index c7bb81549e..02438ece56 100644 --- a/drivers/net/mlx5/windows/mlx5_os.c +++ b/drivers/net/mlx5/windows/mlx5_os.c @@ -187,6 +187,32 @@ mlx5_os_capabilities_prepare(struct mlx5_dev_ctx_shared *sh) if (sh->dev_cap.tso) sh->dev_cap.tso_max_payload_sz = 1 << hca_attr->max_lso_cap; DRV_LOG(DEBUG, "Counters are not supported."); + if (hca_attr->striding_rq) { + sh->dev_cap.mprq.enabled = 1; + sh->dev_cap.mprq.log_min_stride_size = + MLX5_MIN_SINGLE_STRIDE_LOG_NUM_BYTES; + sh->dev_cap.mprq.log_max_stride_size = + MLX5_MAX_SINGLE_STRIDE_LOG_NUM_BYTES; + if (hca_attr->ext_stride_num_range) + sh->dev_cap.mprq.log_min_stride_num = + MLX5_EXT_MIN_SINGLE_WQE_LOG_NUM_STRIDES; + else + sh->dev_cap.mprq.log_min_stride_num = + MLX5_MIN_SINGLE_WQE_LOG_NUM_STRIDES; + sh->dev_cap.mprq.log_max_stride_num = + MLX5_MAX_SINGLE_WQE_LOG_NUM_STRIDES; + DRV_LOG(DEBUG, "\tmin_single_stride_log_num_of_bytes: %u", + sh->dev_cap.mprq.log_min_stride_size); + DRV_LOG(DEBUG, "\tmax_single_stride_log_num_of_bytes: %u", + sh->dev_cap.mprq.log_max_stride_size); + DRV_LOG(DEBUG, "\tmin_single_wqe_log_num_of_strides: %u", + sh->dev_cap.mprq.log_min_stride_num); + DRV_LOG(DEBUG, "\tmax_single_wqe_log_num_of_strides: %u", + sh->dev_cap.mprq.log_max_stride_num); + DRV_LOG(DEBUG, "\tmin_stride_wqe_log_size: %u", + sh->dev_cap.mprq.log_min_stride_wqe_size); + DRV_LOG(DEBUG, "Device supports Multi-Packet RQ."); + } if (hca_attr->rss_ind_tbl_cap) { /* * DPDK doesn't support larger/variable indirection tables.