From patchwork Wed Feb 1 07:28:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Vesker X-Patchwork-Id: 122785 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B41B441B9B; Wed, 1 Feb 2023 08:29:47 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7B34041611; Wed, 1 Feb 2023 08:29:10 +0100 (CET) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2057.outbound.protection.outlook.com [40.107.244.57]) by mails.dpdk.org (Postfix) with ESMTP id 8F08042D3F for ; Wed, 1 Feb 2023 08:29:08 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Akl21FP7r/imHExsTaxN0GGMZhhU6TTQaIsiAqGOT6W/OuVCY4MR/v6d13aGqmD0ETaI4wbqxi/R5bffYlb5933kSPh3MZv2VbVn0Bd8cCEFqfGWLMZJjbMJVlmo1rjV/Hs7y2Lul3+5AwiPA2qj5n1QVn7a3wI+FITwF6CosW6+1MlUSSokK86co61VCY1L/h8ksReWFNMTMq1VR51i7hhgOSrtMAUOVP7KU4m1unbjxwAY035AxSvCyaxwSSblpl5ARJ1iqoesH/pDK7+S71IW4xCknzuGBna4pBbERwM/NMuK/7TCZwpPDcCK0L/xGzUnij02jA2dlx2OvwJ8qw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=A2D0P8B+eE93VbCo8qqLvTULa6Y8ekyxebk8V4wKVvo=; b=js0aQH13IyykYXtuwczCSV3MFxKmVadwNyyRvSV+YNbNPVoKRJfhOzCL0wbClh3qjurOjAdnrqb45b1l5QmRv1VXW9JWw9qQVrlj4givl0Wy0JbTtvgUBC3ta2HT2s/5u5UjehJD9ehCx5kKhpb3m2seYAaDmE8wQsZOiNu4qeQYEYSCyTfd9uTGMkztSissq78e/MzBlBKnh1L11pOywv+xry7r+DJdGHnzCWnkDuXvPqJBHu1RzSS/DzKD8mqNMA0CN0V+sNrTn46hvjxyKR8mrTHhqHJ3KjKxzjRtMdZry4Cq6aoHJIL1UGmo5bRIKVb9zOahJuQhafqB37twwg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=A2D0P8B+eE93VbCo8qqLvTULa6Y8ekyxebk8V4wKVvo=; b=tqwsNQ2o+VQXl1AlUimZBssbkklXFZkFto3GsUgStQlJoDiw6/D5+z913oyicr4/o9FjZmF3ZJL8oZAbk60iu8roTg2dvKNF2JH84SKXMkHTeAeeuvALVi1TJpmxd6IYiUTQY5Y3/ZI1J9fNfU4Iei0UOy4fpXV56PI8UTP+1XFFiVikSt+XMXQ/bFrvEyr7iaFA0Z911YwztdPtiUFnFohmmG9xrEQKRFHqpDhMqCH4OUdLE+v5gOGP3wN5mJLU9Rbu23fFfg+5wsUdyh0+VpeCW7OWhhvUxrGxOI0gDnQ7pfqQcPjRglkyOb3YTmdVeRMBJXjH950w7vNwB9YzGw== Received: from BN1PR10CA0013.namprd10.prod.outlook.com (2603:10b6:408:e0::18) by DM4PR12MB5216.namprd12.prod.outlook.com (2603:10b6:5:398::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.23; Wed, 1 Feb 2023 07:29:06 +0000 Received: from BN8NAM11FT107.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e0:cafe::43) by BN1PR10CA0013.outlook.office365.com (2603:10b6:408:e0::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6043.38 via Frontend Transport; Wed, 1 Feb 2023 07:29:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BN8NAM11FT107.mail.protection.outlook.com (10.13.176.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.24 via Frontend Transport; Wed, 1 Feb 2023 07:29:06 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Tue, 31 Jan 2023 23:28:54 -0800 Received: from nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Tue, 31 Jan 2023 23:28:52 -0800 From: Alex Vesker To: , , , "Matan Azrad" CC: , Subject: [v2 07/16] net/mlx5/hws: add send FW range STE WQE Date: Wed, 1 Feb 2023 09:28:06 +0200 Message-ID: <20230201072815.1329101-8-valex@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230201072815.1329101-1-valex@nvidia.com> References: <20230131093346.1261066-1-valex@nvidia.com> <20230201072815.1329101-1-valex@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.230.37] X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT107:EE_|DM4PR12MB5216:EE_ X-MS-Office365-Filtering-Correlation-Id: 71ab3337-6161-4471-3f8e-08db0426005a X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: KC7SB8HgP9pvjWzQI/G/KsUxC4wdzQg1R0zEbSpDY+AlSByVGqPBVYuJhJHPVTEdCrZ/aFkMrLTcJ92xABdrk432Bb0X56LMCeri4AlNudhU9Iyy8K9A0+ckpAKvjz4VX2xBSRT58+I7CIxJSYYFSJrwCJ6s9kjuxwH+mN/nIZgl88ofu1suhk2B2+G44PlHfvYaHPWWSC+jLwKTtpS9hN9DLTdV6XfTmPCwe/cGwxNsrC3HBPDvJicXs0hcXlM4r7SzSBAKHR4Sid7y8KpGQzLxQM8eCWLz+yCoucw7GLZ+Q0XaW3MF+zBxv63anBu+WKTPQxMatDvN5z2m1lbeOJZmd3VQfE8e73qzeXrKirBHTkkDL/riCd7KIYmr4O/Zrh3j0r4JBpdVTwjNGuRyxncqaEcKnTH4hz4tYf5913G5SE1pxjg8q4z5xqsFYpu735ryBOwA3P0+jF2avd/ogfqlW+AqNjAR2AJzN4FUAPT0AVLihudgKD3ZJIDxNYFtcWEtXMwZxIFt4Ap41einmFKpEIFiCL4XDC1UyRiIUlgwHfXmqUjjyWuVy2ChXcJSQsH1A1YUzOOWxEJ9+IA15Vdly/5jhs03M3ToaeEH6KgHb+NGhRnUjA9ltl/5DaPL1+CxRqFezId6uEJaOHvtPpCNdJYRi+HRL0lKGrf2xL7j5O86e9vbDCkKzYpy8Jk9wq7JlM8EvH8DwbFUHPS89A== X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230025)(4636009)(39860400002)(396003)(136003)(346002)(376002)(451199018)(46966006)(40470700004)(36840700001)(70206006)(70586007)(4326008)(47076005)(41300700001)(8936002)(36860700001)(8676002)(426003)(83380400001)(336012)(40460700003)(7636003)(2616005)(82740400003)(316002)(110136005)(54906003)(82310400005)(5660300002)(356005)(40480700001)(55016003)(1076003)(6636002)(36756003)(16526019)(186003)(26005)(6286002)(86362001)(478600001)(7696005)(107886003)(2906002)(6666004); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Feb 2023 07:29:06.2208 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 71ab3337-6161-4471-3f8e-08db0426005a X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT107.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5216 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org FW WQE supports complex rules, constructed from 2 STEs, for example: Hash(DefinerA) SteMatch(DefinerB) SteRange(DefinerC) DefinerA is a subset of DefinerB This complex rule is written using a single FW command which has a single WQE control, STE match data0 and STE range data1. FW manages STEs/ICM and coherency between deletion and creation. It is possible to also pass the definer value as part of the STE, this is not supported with current HW. Signed-off-by: Alex Vesker --- drivers/net/mlx5/hws/mlx5dr_send.c | 19 +++++++++++++++++++ drivers/net/mlx5/hws/mlx5dr_send.h | 3 +++ 2 files changed, 22 insertions(+) diff --git a/drivers/net/mlx5/hws/mlx5dr_send.c b/drivers/net/mlx5/hws/mlx5dr_send.c index a9958df4f2..51aaf5c8e2 100644 --- a/drivers/net/mlx5/hws/mlx5dr_send.c +++ b/drivers/net/mlx5/hws/mlx5dr_send.c @@ -242,11 +242,15 @@ int mlx5dr_send_wqe_fw(struct ibv_context *ibv_ctx, struct mlx5dr_wqe_gta_ctrl_seg *send_wqe_ctrl, void *send_wqe_match_data, void *send_wqe_match_tag, + void *send_wqe_range_data, + void *send_wqe_range_tag, bool is_jumbo, uint8_t gta_opcode) { + bool has_range = send_wqe_range_data || send_wqe_range_tag; bool has_match = send_wqe_match_data || send_wqe_match_tag; struct mlx5dr_wqe_gta_data_seg_ste gta_wqe_data0 = {0}; + struct mlx5dr_wqe_gta_data_seg_ste gta_wqe_data1 = {0}; struct mlx5dr_wqe_gta_ctrl_seg gta_wqe_ctrl = {0}; struct mlx5dr_cmd_generate_wqe_attr attr = {0}; struct mlx5dr_wqe_ctrl_seg wqe_ctrl = {0}; @@ -278,6 +282,17 @@ int mlx5dr_send_wqe_fw(struct ibv_context *ibv_ctx, attr.gta_data_0 = (uint8_t *)>a_wqe_data0; } + /* Set GTA range WQE DATA */ + if (has_range) { + if (send_wqe_range_data) + memcpy(>a_wqe_data1, send_wqe_range_data, sizeof(gta_wqe_data1)); + else + mlx5dr_send_wqe_set_tag(>a_wqe_data1, send_wqe_range_tag, false); + + gta_wqe_data1.rsvd1_definer = htobe32(send_attr->range_definer_id << 8); + attr.gta_data_1 = (uint8_t *)>a_wqe_data1; + } + attr.pdn = pd_num; attr.wqe_ctrl = (uint8_t *)&wqe_ctrl; attr.gta_ctrl = (uint8_t *)>a_wqe_ctrl; @@ -336,6 +351,8 @@ void mlx5dr_send_stes_fw(struct mlx5dr_send_engine *queue, ste_attr->wqe_ctrl, ste_attr->wqe_data, ste_attr->wqe_tag, + ste_attr->range_wqe_data, + ste_attr->range_wqe_tag, ste_attr->wqe_tag_is_jumbo, ste_attr->gta_opcode); if (ret) @@ -350,6 +367,8 @@ void mlx5dr_send_stes_fw(struct mlx5dr_send_engine *queue, ste_attr->wqe_ctrl, ste_attr->wqe_data, ste_attr->wqe_tag, + ste_attr->range_wqe_data, + ste_attr->range_wqe_tag, ste_attr->wqe_tag_is_jumbo, ste_attr->gta_opcode); if (ret) diff --git a/drivers/net/mlx5/hws/mlx5dr_send.h b/drivers/net/mlx5/hws/mlx5dr_send.h index 1e845b1c7a..47bb66b3c7 100644 --- a/drivers/net/mlx5/hws/mlx5dr_send.h +++ b/drivers/net/mlx5/hws/mlx5dr_send.h @@ -161,6 +161,7 @@ struct mlx5dr_send_engine_post_attr { uint8_t notify_hw; uint8_t fence; uint8_t match_definer_id; + uint8_t range_definer_id; size_t len; struct mlx5dr_rule *rule; uint32_t id; @@ -182,8 +183,10 @@ struct mlx5dr_send_ste_attr { uint32_t direct_index; struct mlx5dr_send_engine_post_attr send_attr; struct mlx5dr_rule_match_tag *wqe_tag; + struct mlx5dr_rule_match_tag *range_wqe_tag; struct mlx5dr_wqe_gta_ctrl_seg *wqe_ctrl; struct mlx5dr_wqe_gta_data_seg_ste *wqe_data; + struct mlx5dr_wqe_gta_data_seg_ste *range_wqe_data; }; /**