From patchwork Tue Feb 28 16:43:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Kozyrev X-Patchwork-Id: 124577 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5415441D9E; Tue, 28 Feb 2023 17:44:00 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 50842427F5; Tue, 28 Feb 2023 17:43:52 +0100 (CET) Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam02on2043.outbound.protection.outlook.com [40.107.95.43]) by mails.dpdk.org (Postfix) with ESMTP id A904F427F5 for ; Tue, 28 Feb 2023 17:43:50 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KT6N7f+/muqd0R+Lth20VM8TIeWJIulFQv68WvG9agWVNuC9bUVnfWgQwe+fgJ7+QKpDxZDw370p/uQsyZC9ST/gKLkyaU0HKGetE0lsPI7jmjQCbnk75b7npOirC2uyxlnW/5jWBM2J55g5OahoAurDUB+Jknpl1baoYRgmceQfdQFLWnw66QFVPm9TTnK2e3HkqYDvufAdbhU15q/XZWo3mkwZ2zuL1yijop4QrGFXDccQCq2VU8ydXmqG0RZucebuLcfm9Wc4uxb5G5iUHWxnFvpjdDLzEDAve8nb9TjpGWvoko0gD7RjPEtKvdkD9a6nupJixdE4FRMDJRD1kQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qn+Dnyd8/h66tp/JukU+sScF4FxujZ57bSfBtM9PWH8=; b=Lv8z9PUINxRzLYelGY8tBfF7+XMxi8XttkssagdJBWvOqhE1uwcTi+X0O0I9HlzPF1yWutyJ0I/OeDCWfVHDutj4ZCCf9/Q8hUGEIfzULjN83SpNFq7o4eN6bay5/6QdA2gH1AdKxPl3TlWaALC5vU2ef0uVTAM99NE/4ZUhi639xI3wf2P/yBhrGqYKx/31+GzMtaXEqNVZ07YqwBYXLa69dk2Uqstqi2xoznMRxokE2ahjE3+kajaBoMfVn+LeJOWHYvPJC23TuA6+ueF0f9Kml0JD42XvTmYjlvHoDE3crjTn7LSbDvwC9wyR1+tjYtMMVlyaHhyQgtZWTRQnnA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qn+Dnyd8/h66tp/JukU+sScF4FxujZ57bSfBtM9PWH8=; b=iH/PYBSTn08wnbBMNcuWXZVbWK/PTWwKKShYrP6kdkyHK6HyhdXW4pLCfdmr+o1a7g6BsS37cCV6e9rcxLnGwAiZlV/KE7bL7GsK7f26pAIKj5dwKmKVtEbkX9VwkHzTJQifg/H8Us2XNVQYfEelTRjAy6gaY9i3wRNrHhh6wELO+Kfjv0O8lY4v3S8UjgoiQK2j2C6warxL+t+mG7ONQJI5Ya/5K+sfeTj80IOHZSASNHTiEZ+yb0UaGH92RZjSlLTkUSEjD6znL/GSwUyT4s/FB9x5C5P5nyainyYlpoENWdtZ+TnObgAFqa0lPdtjeD/lw6GaJxnAeknnfVs3dQ== Received: from BN8PR15CA0009.namprd15.prod.outlook.com (2603:10b6:408:c0::22) by PH0PR12MB8174.namprd12.prod.outlook.com (2603:10b6:510:298::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6134.30; Tue, 28 Feb 2023 16:43:48 +0000 Received: from BN8NAM11FT085.eop-nam11.prod.protection.outlook.com (2603:10b6:408:c0:cafe::2c) by BN8PR15CA0009.outlook.office365.com (2603:10b6:408:c0::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6134.30 via Frontend Transport; Tue, 28 Feb 2023 16:43:48 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN8NAM11FT085.mail.protection.outlook.com (10.13.176.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6156.12 via Frontend Transport; Tue, 28 Feb 2023 16:43:47 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Tue, 28 Feb 2023 08:43:29 -0800 Received: from pegasus01.mtr.labs.mlnx (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Tue, 28 Feb 2023 08:43:28 -0800 From: Alexander Kozyrev To: CC: , , Subject: [PATCH 2/5] common/mlx5: add CQE validity iteration count Date: Tue, 28 Feb 2023 18:43:07 +0200 Message-ID: <20230228164310.807594-3-akozyrev@nvidia.com> X-Mailer: git-send-email 2.18.2 In-Reply-To: <20230228164310.807594-1-akozyrev@nvidia.com> References: <20230228164310.807594-1-akozyrev@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.230.37] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT085:EE_|PH0PR12MB8174:EE_ X-MS-Office365-Filtering-Correlation-Id: 40b3d7eb-7e97-436e-4336-08db19aaf6da X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: SmMv7+aEFNxaYbr4/8G4mIm/IuATt0ag3pYe0RGA0qupX38Hnf2fIcvGS3OMvOULjOEgVbwG9hh5LZwtuUMcAD4CDw5+/QWw7btRMvO/RIhs2MhISKGq/i7tYIVMC03tqRHmCQOCSdco4d4owEAE+R9XBE296w5PiARXlY6nahi8ta95rmog8ci2NecoEC3mHq6IuB/+t91nEEnRJufkYbBzEhjva0JTGiQ3bVnOqtSAkHMX92xEvo1/qiMQ56hneuYpFK0dCYmJ30cQmME6hh83ze/I8/piBakjt7+rpF9FQL7EhQ58yR+jM31ONpZ9cBHScIaZxAVExPDRLG98UpYgW1WwGgRjHD0rvho3kf5vsDpL5MzcGcQtOBFSdb8dOiaum5vc8qPEJUdGyMyOWxSb1PnyhENNkXraLACK/IhF/X+leB2l4BnLDcizTPHVoFpJsEx8GHVEXNNvI10V6AZYOiix25oDUTz14SulVmCXXu7I+MyCD5x41DHkePYBoviDZezkinMex43ewEcyvFI345BRRG0wBeM0SoDCqChgFU7Vxq8iyJ0Oh0dpCBZIRXFynKHUFqfKrWbZ0Pa9/xxifW+GOO1CjhXEfaCVxN70oNVPiPGPBgy0JNjpV9jtrBB7lwLwnYQRVsg1hulBUbfmktv9zGOOX+AvI2t7CPdUupYBLIgkB3qLgzE/u3IzgNcPprPqI31+tDBD7KVjpChnDMc578QGx8+QAnlWzvw= X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230025)(4636009)(136003)(39860400002)(346002)(396003)(376002)(451199018)(40470700004)(46966006)(36840700001)(86362001)(8936002)(5660300002)(186003)(1076003)(316002)(8676002)(70586007)(70206006)(41300700001)(6916009)(26005)(4326008)(2906002)(36756003)(54906003)(83380400001)(40460700003)(478600001)(40480700001)(107886003)(6666004)(16526019)(336012)(47076005)(356005)(2616005)(82740400003)(426003)(34020700004)(7636003)(82310400005)(36860700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2023 16:43:47.7503 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 40b3d7eb-7e97-436e-4336-08db19aaf6da X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT085.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB8174 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The validity iteration count replaces the functionality of the owner bit in terms of indicating that a new CQE was written to buffer. On iteration=k on the CQ buffer, only entries with the iteration_count=k should be treated as new CQEs or mini CQE arrays. The validity iteration count is used when the Enhanced CQE compression is selected. Add this CQE field and the method to check it. Signed-off-by: Alexander Kozyrev Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/mlx5_common.h | 57 ++++++++++++++++++++++---- drivers/common/mlx5/mlx5_common_devx.c | 4 +- drivers/common/mlx5/mlx5_prm.h | 12 ++++-- 3 files changed, 62 insertions(+), 11 deletions(-) diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index f8d07d6c6b..9fb85ddefb 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -180,7 +180,26 @@ enum mlx5_cqe_status { }; /** - * Check whether CQE is valid. + * Check whether CQE has an error opcode. + * + * @param op_code + * Opcode to check. + * + * @return + * The CQE status. + */ +static __rte_always_inline enum mlx5_cqe_status +check_cqe_error(const uint8_t op_code) +{ + rte_io_rmb(); + if (unlikely(op_code == MLX5_CQE_RESP_ERR || + op_code == MLX5_CQE_REQ_ERR)) + return MLX5_CQE_STATUS_ERR; + return MLX5_CQE_STATUS_SW_OWN; +} + +/** + * Check whether CQE is valid using owner bit. * * @param cqe * Pointer to CQE. @@ -201,13 +220,37 @@ check_cqe(volatile struct mlx5_cqe *cqe, const uint16_t cqes_n, const uint8_t op_owner = MLX5_CQE_OWNER(op_own); const uint8_t op_code = MLX5_CQE_OPCODE(op_own); - if (unlikely((op_owner != (!!(idx))) || (op_code == MLX5_CQE_INVALID))) + if (unlikely((op_owner != (!!(idx))) || + (op_code == MLX5_CQE_INVALID))) return MLX5_CQE_STATUS_HW_OWN; - rte_io_rmb(); - if (unlikely(op_code == MLX5_CQE_RESP_ERR || - op_code == MLX5_CQE_REQ_ERR)) - return MLX5_CQE_STATUS_ERR; - return MLX5_CQE_STATUS_SW_OWN; + return check_cqe_error(op_code); +} + +/** + * Check whether CQE is valid using validity iteration count. + * + * @param cqe + * Pointer to CQE. + * @param cqes_n + * Log 2 of completion queue size. + * @param ci + * Consumer index. + * + * @return + * The CQE status. + */ +static __rte_always_inline enum mlx5_cqe_status +check_cqe_iteration(volatile struct mlx5_cqe *cqe, const uint16_t cqes_n, + const uint32_t ci) +{ + const uint8_t op_own = cqe->op_own; + const uint8_t op_code = MLX5_CQE_OPCODE(op_own); + const uint8_t vic = ci >> cqes_n; + + if (unlikely((cqe->validity_iteration_count != vic) || + (op_code == MLX5_CQE_INVALID))) + return MLX5_CQE_STATUS_HW_OWN; + return check_cqe_error(op_code); } /* diff --git a/drivers/common/mlx5/mlx5_common_devx.c b/drivers/common/mlx5/mlx5_common_devx.c index 5f53996b72..431d8361ce 100644 --- a/drivers/common/mlx5/mlx5_common_devx.c +++ b/drivers/common/mlx5/mlx5_common_devx.c @@ -41,8 +41,10 @@ mlx5_cq_init(struct mlx5_devx_cq *cq_obj, uint16_t cq_size) volatile struct mlx5_cqe *cqe = cq_obj->cqes; uint16_t i; - for (i = 0; i < cq_size; i++, cqe++) + for (i = 0; i < cq_size; i++, cqe++) { cqe->op_own = (MLX5_CQE_INVALID << 4) | MLX5_CQE_OWNER_MASK; + cqe->validity_iteration_count = MLX5_CQE_VIC_INIT; + } } /** diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index aa291f19a6..a52feba7e4 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -26,12 +26,18 @@ /* Get CQE opcode. */ #define MLX5_CQE_OPCODE(op_own) (((op_own) & 0xf0) >> 4) +/* Get CQE number of mini CQEs. */ +#define MLX5_CQE_NUM_MINIS(op_own) (((op_own) & 0xf0) >> 4) + /* Get CQE solicited event. */ #define MLX5_CQE_SE(op_own) (((op_own) >> 1) & 1) /* Invalidate a CQE. */ #define MLX5_CQE_INVALIDATE (MLX5_CQE_INVALID << 4) +/* Initialize CQE validity iteration count. */ +#define MLX5_CQE_VIC_INIT 0xffu + /* Hardware index widths. */ #define MLX5_CQ_INDEX_WIDTH 24 #define MLX5_WQ_INDEX_WIDTH 16 @@ -442,7 +448,7 @@ struct mlx5_cqe { uint64_t timestamp; uint32_t sop_drop_qpn; uint16_t wqe_counter; - uint8_t rsvd5; + uint8_t validity_iteration_count; uint8_t op_own; }; @@ -450,7 +456,7 @@ struct mlx5_cqe_ts { uint64_t timestamp; uint32_t sop_drop_qpn; uint16_t wqe_counter; - uint8_t rsvd5; + uint8_t validity_iteration_count; uint8_t op_own; }; @@ -5041,8 +5047,8 @@ struct mlx5_mini_cqe8 { }; struct { uint16_t wqe_counter; + uint8_t validity_iteration_count; uint8_t s_wqe_opcode; - uint8_t reserved; } s_wqe_info; }; union {