From patchwork Fri Feb 3 05:21:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiawei Wang X-Patchwork-Id: 122992 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 301E441BB7; Fri, 3 Feb 2023 06:21:42 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 48B68427E9; Fri, 3 Feb 2023 06:21:39 +0100 (CET) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2072.outbound.protection.outlook.com [40.107.93.72]) by mails.dpdk.org (Postfix) with ESMTP id 34C9F41611 for ; Fri, 3 Feb 2023 06:21:38 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mgvihclgDtDVljI/DUNoDv8ICtrl0oD4cIf3mzXRpTShhQ/QFKDfDX2LSrbwJQpkGRC9dlABE2iAVxw7INTj8GtLPPAp8ISeLuQl2Uf4ee4W3SmCnVO53NkCZ2Kt1wTeMRKtIV31S7eJzX6Qqy2amXeWtyyIi7O+bL/bmRIqGgABExO743GmCEyH75EJ7uFGuUAuSTR5rd4KnUApsMIt6Bbb5K+2q5gLccsqF3eccCgQXTlu/Nj6kc1gk4QJ5r+b514AXhHw9GtbDZzISyag/F7AknyftAsnCz1/vqWkKnpOeLDbLl13c1c9fVhqvvm/hcsHC+CLcXdK4AVif4wQVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=x24RmbQZ95AnkDovbDkmhG+18CWdeghOP5V6TQQHfZo=; b=TG7XZBGjTME3rtQ6B6aCQArbJx70bP2Dt62Rtt+jbrOvwDJWMMiK6OakpWdutI+KyL++igw//+M5Pbj6tgL7uRRn7s5Ts8vX1euNiY7Xsd0XK0Vyq7e/vI00st0FIx24ZeYVqzXo4aXmHnnnjBdEJ97WCgqaqujTZ4SzRw5eBtdMu7WOqpT0D8US0+0WsXjNkhXdL1ZHd+6IuSLOGnxjB5XOcLMwCGiu4mq2PGrKLfEYD7a5ifH/Jnbq46akwNj6xGcC15q5nzeV9qug9uqX7cP/W65b3btW+gPjpD9EmklEHbNyglGwIfUsEB3YEK9x9MsOdYU/ot1j04DH1I7ePA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=x24RmbQZ95AnkDovbDkmhG+18CWdeghOP5V6TQQHfZo=; b=kkRKqqIoHbC5XQgDDcwb2AVdmTuaMmTMXMLIcmhhSQuHGMkRqUUgp6XUsRDoJC6W94BJyamW0ac4786vAGTL5BzA2Vls158CdlVG2aAj1efh0mtcTSfBaWW2X0qphaF6NkUunSxwrLWFetwbuGCs65BqvmagOdDSmzrlTMCD12cQOuaHTDeCfEIQhnG5R6bSQfT+vWyeONT1+tUjSv2KudEas7Xh/rB4h64RThtDXDQ55diuPXjA8bLYu+ojAtfTK14jCv92i5kUz0YHcrYZeXzJaUdM4b2oQcGpUuiu36oDLM9x/xRt14PAY9hNTMhi9mKczuU2mIrCwIfB+l7zlg== Received: from DM5PR08CA0038.namprd08.prod.outlook.com (2603:10b6:4:60::27) by PH0PR12MB8149.namprd12.prod.outlook.com (2603:10b6:510:297::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.27; Fri, 3 Feb 2023 05:21:35 +0000 Received: from DM6NAM11FT102.eop-nam11.prod.protection.outlook.com (2603:10b6:4:60:cafe::bb) by DM5PR08CA0038.outlook.office365.com (2603:10b6:4:60::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.27 via Frontend Transport; Fri, 3 Feb 2023 05:21:35 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DM6NAM11FT102.mail.protection.outlook.com (10.13.173.172) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.31 via Frontend Transport; Fri, 3 Feb 2023 05:21:35 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 2 Feb 2023 21:21:25 -0800 Received: from nvidia.com (10.126.231.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 2 Feb 2023 21:21:23 -0800 From: Jiawei Wang To: , , Matan Azrad CC: , Subject: [PATCH 1/3] drivers: add lag Rx port affinity in PRM Date: Fri, 3 Feb 2023 07:21:05 +0200 Message-ID: <20230203052107.48293-2-jiaweiw@nvidia.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20230203052107.48293-1-jiaweiw@nvidia.com> References: <20230203052107.48293-1-jiaweiw@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.37] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT102:EE_|PH0PR12MB8149:EE_ X-MS-Office365-Filtering-Correlation-Id: d2f585c6-7f82-4dfa-97e3-08db05a684ca X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: g9Q/HFNKvgWkg1k2In0JxVXItvBELZmH7SR87q92t/DLIpCBhDx6wOfj36VGVYeqBXNbziaaPh/NgW+KHVgP7dTPhrDtn8mtlSkyFLBNL03w1IZpuHdUMu/cRkgOOfz45GGbEhqc/1+cu8M+8+ax8tLWyLAQZbzGG/0lXTac3XL6mE7h4/FomMxLZrSjTGg2y2im5qi5F0CLOLezkaTZDuYfdCjYT3fcLaZxGArqfW87Gq0Nu+P6tx2m7Z1Ux3Om264dx7vEVN77cuEvqUX/VhlTCo3pt1Dh30m9Zznos0c38dt+9QxJ3E00uCh+EGY364PVPCvlVqsXd/TcTWFaxcq/njq9TmucRSEgzkenDvTNineN1uorIkQ8FIRUAD6qGm2Rbi3geviIvyq6XvHd9ok3b0yl3HmnPvPlEe2Qj9OcQEJo6MA+qm7JKkGpyBH6usfMRIT+CTPUB7bEI32PEr4h7VmH4TzbAoEp6KUpN/UfF4s7TxRtQzpAst8AJz7hSo8w0K8S8V/+Zc5mX5mB2GuWaDiLwZk3u4nub2wnAgvHg77dnEWXIzIS7GAWQYRQK7GrA8WBKEIFnlRFmm5R/ByLNOLVbroF0iul4cB4kMItfKMT3BlRqMR8k/HP3ds6JaoAZPwqGIBX/Gw4wiWPTtYIoBPLtasovt+wHI8CrwXeLDU6rnQ+DcK5+oaRNQaz5sst4/u5rKVL0LYd+YknXA== X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230025)(4636009)(136003)(376002)(346002)(39860400002)(396003)(451199018)(46966006)(36840700001)(40470700004)(82740400003)(426003)(7636003)(8936002)(40460700003)(1076003)(478600001)(336012)(83380400001)(2616005)(186003)(26005)(16526019)(6286002)(70586007)(6636002)(4326008)(316002)(41300700001)(110136005)(8676002)(2906002)(70206006)(5660300002)(54906003)(107886003)(7696005)(6666004)(86362001)(55016003)(36756003)(40480700001)(47076005)(82310400005)(36860700001)(356005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Feb 2023 05:21:35.2074 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d2f585c6-7f82-4dfa-97e3-08db05a684ca X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT102.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB8149 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds function to query hca capability via Devx for lag_rx_port_affinity. Signed-off-by: Jiawei Wang --- drivers/common/mlx5/mlx5_devx_cmds.c | 3 +++ drivers/common/mlx5/mlx5_devx_cmds.h | 1 + drivers/common/mlx5/mlx5_prm.h | 7 +++++-- drivers/net/mlx5/linux/mlx5_os.c | 4 ++++ drivers/net/mlx5/mlx5.h | 2 ++ 5 files changed, 15 insertions(+), 2 deletions(-) diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index e3a4927d0f..a157ab4b6c 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -1141,6 +1141,9 @@ mlx5_devx_cmd_query_hca_attr(void *ctx, attr->outer_ipv4_ihl = MLX5_GET (flow_table_nic_cap, hcattr, ft_field_support_2_nic_receive.outer_ipv4_ihl); + attr->lag_rx_port_affinity = MLX5_GET + (flow_table_nic_cap, hcattr, + ft_field_support_2_nic_receive.lag_rx_port_affinity); /* Query HCA offloads for Ethernet protocol. */ hcattr = mlx5_devx_get_hca_cap(ctx, in, out, &rc, MLX5_GET_HCA_CAP_OP_MOD_ETHERNET_OFFLOAD_CAPS | diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index c94b9eac06..9cf045ccaa 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -288,6 +288,7 @@ struct mlx5_hca_attr { uint32_t alloc_flow_counter_pd:1; uint32_t flow_counter_access_aso:1; uint32_t flow_access_aso_opc_mod:8; + uint32_t lag_rx_port_affinity:1; }; /* LAG Context. */ diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index 9294f65e24..8bbb800206 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -905,7 +905,8 @@ struct mlx5_ifc_fte_match_set_misc_bits { u8 vxlan_vni[0x18]; u8 reserved_at_b8[0x8]; u8 geneve_vni[0x18]; - u8 reserved_at_e4[0x6]; + u8 lag_rx_port_affinity[0x4]; + u8 reserved_at_e8[0x2]; u8 geneve_tlv_option_0_exist[0x1]; u8 geneve_oam[0x1]; u8 reserved_at_e0[0xc]; @@ -2044,7 +2045,9 @@ struct mlx5_ifc_ft_fields_support_bits { * Table 1872 - Flow Table Fields Supported 2 Format */ struct mlx5_ifc_ft_fields_support_2_bits { - u8 reserved_at_0[0xd]; + u8 reserved_at_0[0xa]; + u8 lag_rx_port_affinity[0x1]; + u8 reserved_at_c[0x2]; u8 hash_result[0x1]; u8 reserved_at_e[0x1]; u8 tunnel_header_2_3[0x1]; diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index a71474c90a..60462da39d 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -1390,6 +1390,10 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, DRV_LOG(DEBUG, "DV flow is not supported!"); } #endif + if (hca_attr->lag_rx_port_affinity) { + sh->lag_rx_port_affinity_en = 1; + DRV_LOG(DEBUG, "LAG Rx Port Affinity enabled"); + } } /* Process parameters and store port configuration on priv structure. */ err = mlx5_port_args_config(priv, mkvlist, &priv->config); diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 16b33e1548..bbd7262a51 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -1374,6 +1374,8 @@ struct mlx5_dev_ctx_shared { uint32_t hws_tags:1; /* Check if tags info for HWS initialized. */ uint32_t shared_mark_enabled:1; /* If mark action is enabled on Rxqs (shared E-Switch domain). */ + uint32_t lag_rx_port_affinity_en:1; + /* lag_rx_port_affinity is supported. */ uint32_t hws_max_log_bulk_sz:5; /* Log of minimal HWS counters created hard coded. */ uint32_t hws_max_nb_counters; /* Maximal number for HWS counters. */ From patchwork Fri Feb 3 05:21:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiawei Wang X-Patchwork-Id: 122993 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E039B41BB7; Fri, 3 Feb 2023 06:21:47 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6845542BD9; Fri, 3 Feb 2023 06:21:42 +0100 (CET) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2089.outbound.protection.outlook.com [40.107.243.89]) by mails.dpdk.org (Postfix) with ESMTP id F053D42BD9 for ; Fri, 3 Feb 2023 06:21:40 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=T2sp8jNtOpqQEt+9J8VSn2nQ/WtPsNlO9WoT5T1gaFQfvX5Lxj9/TH4XuHYLdubf9Kw1QHzx27QvF+iMaT7LKvFmUrhsSV0Vk9JrcAJ5jy6zMSeDkKy8+5Z5p7PCht9i2DrO4icpPzWTX/S/OBh3T5YRh/UQfU0hI7V7HWXAl4Doi8nLhEEOJ4SfQnN14uE/uBxla6vXbSHSLdTsBZR7ZSvf4lm7u6iJcYfl7vD0ePOz/6rf5tmAMjaZkiaHMP4gDud4vH/eXfs/GGcsADekj56JQLDHgT/uOiNcdGMJJn9h2+OAtxykq2LN9uK9vkpWyELwQQSTDYG8kCC1R6f77w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jsPPHstNKnif66X9kVLIx4lqeWWNltejfrwk3l1bH7Y=; b=SU1JFQMEiyUSbEU/mtPCToqUENnYxNSynsFtLnHzaDn5qgYBl69ZiVeiNm7Yw9gm0psDFrb8AHe0PoP2McPcdR2kKJbwx3JcCy9NmLco3dRpuU7TZFjl2ak+QNKxBL58IiMhAFmQvBecV8+RbKE510m/IzNr8o71cDbDP/IU4mbGdibHgox7ZR+/CTdcOIOEyL0bQEBWNxPMXwtMylcwHJ1IiwjuBxC+dX3Jz0KLD9jioaNwfEtg/E8Jsk4lZ+j4gDB7ElCcOpQIwvsLPkK6upqO6zNF3UYWFqfozeEU9d/z/kHDmtYrWvw8bmcLC8YDK7Da9cizr+iAJ5xDc5xdMA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=amd.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jsPPHstNKnif66X9kVLIx4lqeWWNltejfrwk3l1bH7Y=; b=mNe4zcTu0AqF5j8VoKfLFWldEa6TPxNcZeDFyvbTzpOXODIY7J/4tebVfcwdeKpemmz83QKjvKv84Z2K27IdXfvG/BTfhYc5j6fw7qRfW5YQHcwiFIjWUXG3k/pLzGcCU2J5+YDSWugT7/UBLzA9bHRWEo7GWNEjyKwTbufRqJG+rUUJilNJCS8z9QYqGYwIUkLaHrJFFoM0hFJq1TL4eZFAk7iuVhNU0hy+mS+/2t1pfEhJk5PxoIlIadPDYhwueUkLW/0kqKEfC71uxlhzBJyq6F+m23zHsLROJfF1eRLf10drg+ZLvUbPVo5eL3zDCLeJlFdwahBk7R6/Hm5MOA== Received: from DM5PR08CA0060.namprd08.prod.outlook.com (2603:10b6:4:60::49) by CY5PR12MB6599.namprd12.prod.outlook.com (2603:10b6:930:41::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.25; Fri, 3 Feb 2023 05:21:38 +0000 Received: from DM6NAM11FT102.eop-nam11.prod.protection.outlook.com (2603:10b6:4:60::4) by DM5PR08CA0060.outlook.office365.com (2603:10b6:4:60::49) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.29 via Frontend Transport; Fri, 3 Feb 2023 05:21:38 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DM6NAM11FT102.mail.protection.outlook.com (10.13.173.172) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.31 via Frontend Transport; Fri, 3 Feb 2023 05:21:38 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 2 Feb 2023 21:21:29 -0800 Received: from nvidia.com (10.126.231.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 2 Feb 2023 21:21:26 -0800 From: Jiawei Wang To: , , Ferruh Yigit , Matan Azrad CC: , Subject: [PATCH 2/3] net/mlx5: add PHY affinity item support Date: Fri, 3 Feb 2023 07:21:06 +0200 Message-ID: <20230203052107.48293-3-jiaweiw@nvidia.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20230203052107.48293-1-jiaweiw@nvidia.com> References: <20230203052107.48293-1-jiaweiw@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.37] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT102:EE_|CY5PR12MB6599:EE_ X-MS-Office365-Filtering-Correlation-Id: 1133b9b3-ffe1-4df7-c9c6-08db05a6868a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: p4UmJ7eXl9j/bc2xa8QUXJc3pP0lA2oGb7rekQqR5DDcl1ToJ85SOssodzXEkkDGGV2rVYdo1NpfVctDuaC3t2xWH4kK3vUCnHk5GJBLgx84w2aEetTUocoaOsAAJ31ozqPOUbUykcDXlnXQ+6tsUS5B8EyC51im5meBtcYsUPN8z3fscZ5iTBZcBRg0mi0tvH/jJe/+wQhGfKFgCIqsRGH6ZGHBSAXIG1atH0hLYF+z7YvpGzjpgMl87XRhQHFf093y1PGtKYk4O4fIVP7V7NZ9Wf0OtK1bVZJ0TAvYcoFu+U0Df2xipIEGUMjzg8ooX0S/drfo+S9Xc+Bxgv0uI+r7Fi6XLgezM2XraERzdtQxLuspkCaiy49SjESzHwJppqmfnB2wdVRZKIR9PPWdbzqINM/w28UJXOVsttZCxRK3JIz8Nmdg4qncVW0t2badscdP8bhcVjplH8YTkn8C+sOzbOzLWS8FrmWZNS8xyQW8QszHVRYJuio2hxOP6MqtQcchdy8KPY8itvGDRplVIXIgk3G7WYrSREQkklyOKZOSwE5UOR139l1YmCJSSqwC6UP5dm4mhJkOXHYpgPfG1q/MLd9hupCMjs68Neh/95Ppds2bvvl1apYXVX4+ppHQg6uM3Ya7+6JJVhidKdGUCBmTgP8TLe9ORndxYF5/ersMfM/Cgo/4OBPNVTQ7jiYzQtYjWNgdy0gsPpu9pvFMpw== X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230025)(4636009)(376002)(39860400002)(396003)(136003)(346002)(451199018)(36840700001)(40470700004)(46966006)(6286002)(82310400005)(86362001)(1076003)(16526019)(186003)(26005)(107886003)(6666004)(2616005)(7636003)(7696005)(478600001)(356005)(70206006)(82740400003)(54906003)(6636002)(47076005)(426003)(2906002)(110136005)(36756003)(36860700001)(8676002)(55016003)(8936002)(336012)(316002)(5660300002)(4326008)(83380400001)(70586007)(40480700001)(41300700001)(40460700003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Feb 2023 05:21:38.1291 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1133b9b3-ffe1-4df7-c9c6-08db05a6868a X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT102.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6599 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds the new phy affinity item support in PMD: RTE_FLOW_ITEM_TYPE_PHY_AFFINITY. This patch adds the validation function for the new item, it works for NIC-RX and FDB rule on ROOT-table only. Signed-off-by: Jiawei Wang --- doc/guides/nics/features/default.ini | 1 + doc/guides/nics/features/mlx5.ini | 1 + doc/guides/nics/mlx5.rst | 8 ++- drivers/net/mlx5/linux/mlx5_os.c | 2 + drivers/net/mlx5/mlx5.h | 1 + drivers/net/mlx5/mlx5_flow.h | 3 + drivers/net/mlx5/mlx5_flow_dv.c | 100 ++++++++++++++++++++++++++- drivers/net/mlx5/mlx5_flow_hw.c | 14 ++++ 8 files changed, 127 insertions(+), 3 deletions(-) diff --git a/doc/guides/nics/features/default.ini b/doc/guides/nics/features/default.ini index 510cc6679d..fd5edd11c8 100644 --- a/doc/guides/nics/features/default.ini +++ b/doc/guides/nics/features/default.ini @@ -141,6 +141,7 @@ udp = vlan = vxlan = vxlan_gpe = +phy_affinity = [rte_flow actions] age = diff --git a/doc/guides/nics/features/mlx5.ini b/doc/guides/nics/features/mlx5.ini index 62fd330e2b..9142f04f93 100644 --- a/doc/guides/nics/features/mlx5.ini +++ b/doc/guides/nics/features/mlx5.ini @@ -87,6 +87,7 @@ vlan = Y vxlan = Y vxlan_gpe = Y represented_port = Y +phy_affinity = Y [rte_flow actions] age = I diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index f137f156f9..5569b045d7 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -106,6 +106,7 @@ Features - Sub-Function representors. - Sub-Function. - Matching on represented port. +- Matching on phy affinity. Limitations @@ -595,13 +596,18 @@ Limitations - key - sequence - Matching on checksum and sequence needs MLNX_OFED 5.6+. +- Matching on checksum and sequence needs MLNX_OFED 5.6+. - The NIC egress flow rules on representor port are not supported. - When using DV/verbs flow engine (``dv_flow_en`` = 1/0 respectively), Match on SPI field in ESP header for group 0 needs MLNX_OFED 5.6+. +- Match on phy affinity: + + - Supports NIC ingress flow in group 0. + - Supports E-Switch flow in group 0 and depends on + device-managed flow steering (DMFS) mode. Statistics ---------- diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 60462da39d..1c26b30702 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -1394,6 +1394,8 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, sh->lag_rx_port_affinity_en = 1; DRV_LOG(DEBUG, "LAG Rx Port Affinity enabled"); } + priv->num_lag_ports = hca_attr->num_lag_ports; + DRV_LOG(DEBUG, "The number of lag ports is %d", priv->num_lag_ports); } /* Process parameters and store port configuration on priv structure. */ err = mlx5_port_args_config(priv, mkvlist, &priv->config); diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index bbd7262a51..c7a7b176b8 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -1668,6 +1668,7 @@ struct mlx5_priv { unsigned int mtr_reg_share:1; /* Whether support meter REG_C share. */ unsigned int lb_used:1; /* Loopback queue is referred to. */ uint32_t mark_enabled:1; /* If mark action is enabled on rxqs. */ + uint32_t num_lag_ports:4; /* Number of ports can be bonded. */ uint16_t domain_id; /* Switch domain identifier. */ uint16_t vport_id; /* Associated VF vport index (if any). */ uint32_t vport_meta_tag; /* Used for vport index match ove VF LAG. */ diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index e376dcae93..64b1b7c37c 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -219,6 +219,9 @@ enum mlx5_feature_name { /* Meter color item */ #define MLX5_FLOW_ITEM_METER_COLOR (UINT64_C(1) << 44) +/* PHY affinity item */ +#define MLX5_FLOW_ITEM_PHY_AFFINITY (UINT64_C(1) << 49) + /* Outer Masks. */ #define MLX5_FLOW_LAYER_OUTER_L3 \ (MLX5_FLOW_LAYER_OUTER_L3_IPV4 | MLX5_FLOW_LAYER_OUTER_L3_IPV6) diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index 7ca909999b..994f184aaf 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -3764,6 +3764,75 @@ flow_dv_validate_item_meter_color(struct rte_eth_dev *dev, return 0; } +/** + * Validate Phy affinity item. + * + * @param[in] dev + * Pointer to the rte_eth_dev structure. + * @param[in] item + * Item specification. + * @param[in] attr + * Attributes of flow that includes this item. + * @param[out] error + * Pointer to error structure. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +flow_dv_validate_item_phy_affinity(struct rte_eth_dev *dev, + const struct rte_flow_item *item, + const struct rte_flow_attr *attr, + struct rte_flow_error *error) +{ + struct mlx5_priv *priv = dev->data->dev_private; + const struct rte_flow_item_phy_affinity *spec = item->spec; + const struct rte_flow_item_phy_affinity *mask = item->mask; + struct rte_flow_item_phy_affinity nic_mask = { + .affinity = UINT8_MAX + }; + int ret; + + if (!priv->sh->lag_rx_port_affinity_en) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ITEM, NULL, + "Unsupported phy affinity with Older FW"); + if ((attr->transfer && priv->fdb_def_rule) || + attr->egress || attr->group) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM_SPEC, + item->spec, + "phy affinity is not supported with egress or FDB on non root table"); + if (!spec) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ITEM_SPEC, + item->spec, + "data cannot be empty"); + if (spec->affinity == 0) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM_SPEC, + item->spec, + "zero affinity number not supported"); + if (spec->affinity > priv->num_lag_ports) + return rte_flow_error_set(error, ENOTSUP, + RTE_FLOW_ERROR_TYPE_ITEM_SPEC, + item->spec, + "exceed max affinity number in lag ports"); + if (!mask) + mask = &rte_flow_item_phy_affinity_mask; + if (!mask->affinity) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ITEM_SPEC, NULL, + "mask cannot be zero"); + ret = mlx5_flow_item_acceptable(item, (const uint8_t *)mask, + (const uint8_t *)&nic_mask, + sizeof(struct rte_flow_item_phy_affinity), + MLX5_ITEM_RANGE_NOT_ACCEPTED, error); + if (ret < 0) + return ret; + return 0; +} + int flow_dv_encap_decap_match_cb(void *tool_ctx __rte_unused, struct mlx5_list_entry *entry, void *cb_ctx) @@ -7443,6 +7512,13 @@ flow_dv_validate(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, return ret; last_item = MLX5_FLOW_ITEM_METER_COLOR; break; + case RTE_FLOW_ITEM_TYPE_PHY_AFFINITY: + ret = flow_dv_validate_item_phy_affinity(dev, items, + attr, error); + if (ret < 0) + return ret; + last_item = MLX5_FLOW_ITEM_PHY_AFFINITY; + break; default: return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_ITEM, @@ -9981,7 +10057,7 @@ flow_dv_translate_item_tag(struct rte_eth_dev *dev, void *key, const struct rte_flow_item_tag *tag_vv = item->spec; const struct rte_flow_item_tag *tag_v; const struct rte_flow_item_tag *tag_m; - enum modify_reg reg; + int reg; uint32_t index; if (MLX5_ITEM_VALID(item, key_type)) @@ -9996,7 +10072,7 @@ flow_dv_translate_item_tag(struct rte_eth_dev *dev, void *key, else reg = flow_hw_get_reg_id(RTE_FLOW_ITEM_TYPE_TAG, index); MLX5_ASSERT(reg > 0); - flow_dv_match_meta_reg(key, reg, tag_v->data, tag_m->data); + flow_dv_match_meta_reg(key, (enum modify_reg)reg, tag_v->data, tag_m->data); } /** @@ -10639,6 +10715,22 @@ flow_dv_translate_item_meter_color(struct rte_eth_dev *dev, void *key, flow_dv_match_meta_reg(key, (enum modify_reg)reg, value, mask); } +static void +flow_dv_translate_item_phy_affinity(void *key, + const struct rte_flow_item *item, + uint32_t key_type) +{ + const struct rte_flow_item_phy_affinity *affinity_v; + const struct rte_flow_item_phy_affinity *affinity_m; + void *misc_v; + + MLX5_ITEM_UPDATE(item, key_type, affinity_v, affinity_m, + &rte_flow_item_phy_affinity_mask); + misc_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters); + MLX5_SET(fte_match_set_misc, misc_v, lag_rx_port_affinity, + affinity_v->affinity & affinity_m->affinity); +} + static uint32_t matcher_zero[MLX5_ST_SZ_DW(fte_match_param)] = { 0 }; #define HEADER_IS_ZERO(match_criteria, headers) \ @@ -13430,6 +13522,10 @@ flow_dv_translate_items(struct rte_eth_dev *dev, last_item = flow_dv_translate_item_integrity(items, wks, key_type); break; + case RTE_FLOW_ITEM_TYPE_PHY_AFFINITY: + flow_dv_translate_item_phy_affinity(key, items, key_type); + last_item = MLX5_FLOW_ITEM_PHY_AFFINITY; + break; default: break; } diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c index 20c71ff7f0..e5ca86ca4b 100644 --- a/drivers/net/mlx5/mlx5_flow_hw.c +++ b/drivers/net/mlx5/mlx5_flow_hw.c @@ -4715,6 +4715,20 @@ flow_hw_pattern_validate(struct rte_eth_dev *dev, "Unsupported meter color register"); break; } + case RTE_FLOW_ITEM_TYPE_PHY_AFFINITY: + { + if (!priv->sh->lag_rx_port_affinity_en) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ITEM, NULL, + "Unsupported phy affinity with Older FW"); + if ((attr->transfer && priv->fdb_def_rule) || attr->egress) + return rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_ITEM, NULL, + "Phy affinity item not supported" + " with egress or transfer" + " attribute"); + break; + } case RTE_FLOW_ITEM_TYPE_VOID: case RTE_FLOW_ITEM_TYPE_ETH: case RTE_FLOW_ITEM_TYPE_VLAN: From patchwork Fri Feb 3 05:21:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiawei Wang X-Patchwork-Id: 122994 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9FBE041BB7; Fri, 3 Feb 2023 06:21:55 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E930A42D0D; Fri, 3 Feb 2023 06:21:47 +0100 (CET) Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2043.outbound.protection.outlook.com [40.107.96.43]) by mails.dpdk.org (Postfix) with ESMTP id 109CB41611 for ; Fri, 3 Feb 2023 06:21:47 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KXS/1L4t4sFq784UY/MJykTPBwwV44c7Y7WWKMI4n7mU8fZqhBR2V5cvX8iwXbOsgWb5sxUFe0YIEz8gj2aZ0B3+dI7DH7DRWiSQGZCEf/SGwHDtlv+bTAjS5O3Emc6lPQkAqIExMnPp5zP8y1dtku9EJ48KdBeJv1T2MvFDGuCh8POoE1B9jZKMOaTDiHvuO2LpAu3O4/f72MT9sLHYM1362sCUZP9HJAut18zlUpKtkpBzT+Gzv/ZzeBz69oYeQpxbEVA8Go1D30MvOKO0POsWd+9LHnrJsrncgiHW5BPokRUR6PijmAal+M83x9BSWKNg9CsNpTblal+QYIyqPQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pCfX3nPd141Nw54zS79fQO+QwsD0CpMbShWqb4ldNxk=; b=aDjPpM6JJj1wTDSpo9aof4XdS+F2WIikwWJPlk4RuoKVx95UYrqE+0sj8POEc90C+6WWThnzrGWfdE6lVZutwIjcNGQs2n178YUE+dU/P+eJca6O9rsT5lHtOYgb1v253fxM7eIT5zNihrx+6NY3qqgoKDAnrtGY/66oMTx4LOyMJq13GpgbXysN3SxlLrRqPJw3bTUgpFkrZlOZunCFVpU5F96gZAl4KqsmG3MVT6KgwmhS0TtMVIBh7K+wmdo95TT6iO+2qWzixrJ6io8cNMtatM+kkptgkCnuwSCcD5c+Lyz0rlkN9maSu9SYNoSvA5U0MzJd3qROeucliXXMsQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pCfX3nPd141Nw54zS79fQO+QwsD0CpMbShWqb4ldNxk=; b=o4EOCKkoIhLazKGJHY4emM8dmCfm5tsBGJBDpVZvfumT+K0qFkPM6WqSRKcph6yeBTw+yGLQe11GFrTEHXHg9TOYgP+AJFVXxR7AI1AEovyFBzkb2ZIntiROMivnJuwlP2oXCVfclmsMQXOWlX+FLa6bHT5zsGhoM1XgQo41aLEszkLekFV7JK0DvN9c3OrJQpFHK3Q6pFtmPNc7syVgbTBYMv1yyET4NpGLGCzBN4ozNWg/vl3eUY9iYKXqmv60ljmpeYKXkqBW2xz/N6cNjddMTLGfWjbPk0KvQU0C5GE9E3OK1NjN9kbcu/NljVORyASYtoEI9zkpk2LnavOAtg== Received: from BN9PR03CA0300.namprd03.prod.outlook.com (2603:10b6:408:f5::35) by MN2PR12MB4334.namprd12.prod.outlook.com (2603:10b6:208:1d1::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.29; Fri, 3 Feb 2023 05:21:45 +0000 Received: from BN8NAM11FT112.eop-nam11.prod.protection.outlook.com (2603:10b6:408:f5:cafe::53) by BN9PR03CA0300.outlook.office365.com (2603:10b6:408:f5::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.27 via Frontend Transport; Fri, 3 Feb 2023 05:21:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN8NAM11FT112.mail.protection.outlook.com (10.13.176.210) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.28 via Frontend Transport; Fri, 3 Feb 2023 05:21:44 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 2 Feb 2023 21:21:38 -0800 Received: from nvidia.com (10.126.231.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 2 Feb 2023 21:21:36 -0800 From: Jiawei Wang To: , , Matan Azrad CC: , Subject: [PATCH 3/3] drivers: enhance the Tx queue affinity Date: Fri, 3 Feb 2023 07:21:07 +0200 Message-ID: <20230203052107.48293-4-jiaweiw@nvidia.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20230203052107.48293-1-jiaweiw@nvidia.com> References: <20230203052107.48293-1-jiaweiw@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.37] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT112:EE_|MN2PR12MB4334:EE_ X-MS-Office365-Filtering-Correlation-Id: 1888cc48-2702-437f-1b4e-08db05a68a96 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: iFRm+bsi0zXUMxzWthKFQgVOHNVkTA/9+0ptpl8bIzBFrI8Ja4BVspLC3qEeb4FaU4FIzq8PE9Hv8R3esy0HqiwPRCqfT2XLzTrFyEtkI6bOOTSG01KSIQt6eqwlUseiAZhgUBIfP5+dS7ZwJMiXmK8s/qTZdVIFZYx97z7rp20xToPive5717Z8i64yOvMausMy8SmPs7klXHs2JONwYRcCpGredVWY84op+Z0sKIhColsF3335iSEE2FdUJhScsirsrUfBzM2w45zxKy5HWkAd2rJyUVuqHowLBlgmf49ARl3SUWGMEoSZqSfg4yZ+mQjORXOw1Vtrp2a7cM9qabaqlZ59+TFJQwuzlTluMJmiPGiA5okRjOf2s2tqH/+Flw12X9bD9DYXsFYDPihuqOFZ5ycQPiWjLf11U2ByfLO5K2mvpqE0okrFNvCHSMDdAOw9tVn5Awo7xe1o16+aYN/PfxmEbooolqOtwTtG2Sxs6QHG3e1uzAtxUVtlQ+WLAgmeoIJ9SbUXdl4W1ut+PDPhA5bt0pqBn4+H/jXe0QDz7Y7yv/lsmNtcKbSswmA6pjOb0RcOOThCR7Fs0sXTc4iYKsIDdmOgau5HNKLI4OPgqGNZnpr/ogR9xOJyq1aU4iMZYqKIIc2+d7cLkpLqj7ofBXoakYsG9g3stcsdvzpog4IVCAiQwB6cMxuKn1HCPSW3eYUuz9b0L5ffUZXG4w== X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230025)(4636009)(136003)(376002)(396003)(346002)(39860400002)(451199018)(46966006)(40470700004)(36840700001)(36860700001)(82310400005)(7636003)(316002)(5660300002)(54906003)(6636002)(110136005)(426003)(336012)(2616005)(47076005)(83380400001)(40480700001)(40460700003)(82740400003)(36756003)(55016003)(86362001)(356005)(16526019)(7696005)(186003)(2906002)(6286002)(26005)(478600001)(70206006)(4326008)(70586007)(1076003)(107886003)(6666004)(8676002)(41300700001)(8936002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Feb 2023 05:21:44.8829 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1888cc48-2702-437f-1b4e-08db05a68a96 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT112.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4334 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Previous patch supports the tx phy affinity configuration in the Tx queue API, it supports to set the affinity value per Queue. This patch updates TIS creation with tx_phy_affinity value of Tx queue, TIS index 1 goes to hardware port 0, TIS index 2 goes to hardware port 1, and TIS index 0 is reserved for default HWS hash mode. Signed-off-by: Jiawei Wang --- drivers/common/mlx5/mlx5_prm.h | 8 ------- drivers/net/mlx5/mlx5.c | 43 +++++++++++++++------------------- drivers/net/mlx5/mlx5_devx.c | 24 ++++++++++--------- drivers/net/mlx5/mlx5_ethdev.c | 1 + drivers/net/mlx5/mlx5_tx.h | 1 + drivers/net/mlx5/mlx5_txq.c | 8 +++++++ 6 files changed, 42 insertions(+), 43 deletions(-) diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index 8bbb800206..ded001d0b2 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -2331,14 +2331,6 @@ struct mlx5_ifc_query_nic_vport_context_in_bits { u8 reserved_at_68[0x18]; }; -/* - * lag_tx_port_affinity: 0 auto-selection, 1 PF1, 2 PF2 vice versa. - * Each TIS binds to one PF by setting lag_tx_port_affinity (>0). - * Once LAG enabled, we create multiple TISs and bind each one to - * different PFs, then TIS[i] gets affinity i+1 and goes to PF i+1. - */ -#define MLX5_IFC_LAG_MAP_TIS_AFFINITY(index, num) ((num) ? \ - (index) % (num) + 1 : 0) struct mlx5_ifc_tisc_bits { u8 strict_lag_tx_port_affinity[0x1]; u8 reserved_at_1[0x3]; diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index b8643cebdd..c75c98b8b0 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1162,9 +1162,9 @@ mlx5_dev_ctx_shared_mempool_subscribe(struct rte_eth_dev *dev) static int mlx5_setup_tis(struct mlx5_dev_ctx_shared *sh) { - int i; struct mlx5_devx_lag_context lag_ctx = { 0 }; struct mlx5_devx_tis_attr tis_attr = { 0 }; + int i; tis_attr.transport_domain = sh->td->id; if (sh->bond.n_port) { @@ -1178,35 +1178,30 @@ mlx5_setup_tis(struct mlx5_dev_ctx_shared *sh) DRV_LOG(ERR, "Failed to query lag affinity."); return -1; } - if (sh->lag.affinity_mode == MLX5_LAG_MODE_TIS) { - for (i = 0; i < sh->bond.n_port; i++) { - tis_attr.lag_tx_port_affinity = - MLX5_IFC_LAG_MAP_TIS_AFFINITY(i, - sh->bond.n_port); - sh->tis[i] = mlx5_devx_cmd_create_tis(sh->cdev->ctx, - &tis_attr); - if (!sh->tis[i]) { - DRV_LOG(ERR, "Failed to TIS %d/%d for bonding device" - " %s.", i, sh->bond.n_port, - sh->ibdev_name); - return -1; - } - } + if (sh->lag.affinity_mode == MLX5_LAG_MODE_TIS) DRV_LOG(DEBUG, "LAG number of ports : %d, affinity_1 & 2 : pf%d & %d.\n", sh->bond.n_port, lag_ctx.tx_remap_affinity_1, lag_ctx.tx_remap_affinity_2); - return 0; - } - if (sh->lag.affinity_mode == MLX5_LAG_MODE_HASH) + else if (sh->lag.affinity_mode == MLX5_LAG_MODE_HASH) DRV_LOG(INFO, "Device %s enabled HW hash based LAG.", sh->ibdev_name); } - tis_attr.lag_tx_port_affinity = 0; - sh->tis[0] = mlx5_devx_cmd_create_tis(sh->cdev->ctx, &tis_attr); - if (!sh->tis[0]) { - DRV_LOG(ERR, "Failed to TIS 0 for bonding device" - " %s.", sh->ibdev_name); - return -1; + for (i = 0; i <= sh->bond.n_port; i++) { + /* + * lag_tx_port_affinity: 0 auto-selection, 1 PF1, 2 PF2 vice versa. + * Each TIS binds to one PF by setting lag_tx_port_affinity (> 0). + * Once LAG enabled, we create multiple TISs and bind each one to + * different PFs, then TIS[i+1] gets affinity i+1 and goes to PF i+1. + * TIS[0] is reserved for HW Hash mode. + */ + tis_attr.lag_tx_port_affinity = i; + sh->tis[i] = mlx5_devx_cmd_create_tis(sh->cdev->ctx, &tis_attr); + if (!sh->tis[i]) { + DRV_LOG(ERR, "Failed to create TIS %d/%d for [bonding] device" + " %s.", i, sh->bond.n_port, + sh->ibdev_name); + return -1; + } } return 0; } diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c index f6e1943fd7..a3fe0b3b98 100644 --- a/drivers/net/mlx5/mlx5_devx.c +++ b/drivers/net/mlx5/mlx5_devx.c @@ -1190,17 +1190,19 @@ static uint32_t mlx5_get_txq_tis_num(struct rte_eth_dev *dev, uint16_t queue_idx) { struct mlx5_priv *priv = dev->data->dev_private; - int tis_idx; - - if (priv->sh->bond.n_port && priv->sh->lag.affinity_mode == - MLX5_LAG_MODE_TIS) { - tis_idx = (priv->lag_affinity_idx + queue_idx) % - priv->sh->bond.n_port; - DRV_LOG(INFO, "port %d txq %d gets affinity %d and maps to PF %d.", - dev->data->port_id, queue_idx, tis_idx + 1, - priv->sh->lag.tx_remap_affinity[tis_idx]); - } else { - tis_idx = 0; + struct mlx5_txq_data *txq_data = (*priv->txqs)[queue_idx]; + int tis_idx = 0; + + if (priv->sh->bond.n_port) { + if (txq_data->tx_phy_affinity) { + tis_idx = txq_data->tx_phy_affinity; + } else if (priv->sh->lag.affinity_mode == MLX5_LAG_MODE_TIS) { + tis_idx = (priv->lag_affinity_idx + queue_idx) % + priv->sh->bond.n_port + 1; + DRV_LOG(INFO, "port %d txq %d gets affinity %d and maps to PF %d.", + dev->data->port_id, queue_idx, tis_idx, + priv->sh->lag.tx_remap_affinity[tis_idx - 1]); + } } MLX5_ASSERT(priv->sh->tis[tis_idx]); return priv->sh->tis[tis_idx]->id; diff --git a/drivers/net/mlx5/mlx5_ethdev.c b/drivers/net/mlx5/mlx5_ethdev.c index 4a85415ff3..354bb4f022 100644 --- a/drivers/net/mlx5/mlx5_ethdev.c +++ b/drivers/net/mlx5/mlx5_ethdev.c @@ -352,6 +352,7 @@ mlx5_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *info) info->switch_info.domain_id = priv->domain_id; info->switch_info.port_id = priv->representor_id; info->switch_info.rx_domain = 0; /* No sub Rx domains. */ + info->nb_phy_ports = priv->sh->bond.n_port; if (priv->representor) { uint16_t port_id; diff --git a/drivers/net/mlx5/mlx5_tx.h b/drivers/net/mlx5/mlx5_tx.h index a056be7ca8..674c2aebe5 100644 --- a/drivers/net/mlx5/mlx5_tx.h +++ b/drivers/net/mlx5/mlx5_tx.h @@ -144,6 +144,7 @@ struct mlx5_txq_data { uint16_t inlen_send; /* Ordinary send data inline size. */ uint16_t inlen_empw; /* eMPW max packet size to inline. */ uint16_t inlen_mode; /* Minimal data length to inline. */ + uint8_t tx_phy_affinity; /* TxQ affinity configuration. */ uint32_t qp_num_8s; /* QP number shifted by 8. */ uint64_t offloads; /* Offloads for Tx Queue. */ struct mlx5_mr_ctrl mr_ctrl; /* MR control descriptor. */ diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c index 5543f2c570..4e53706807 100644 --- a/drivers/net/mlx5/mlx5_txq.c +++ b/drivers/net/mlx5/mlx5_txq.c @@ -392,6 +392,13 @@ mlx5_tx_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, container_of(txq, struct mlx5_txq_ctrl, txq); int res; + if (conf->tx_phy_affinity > priv->num_lag_ports) { + rte_errno = EINVAL; + DRV_LOG(ERR, "port %u unable to setup Tx queue index %u" + " affinity is %u exceeds the maximum %u", dev->data->port_id, + idx, conf->tx_phy_affinity, priv->num_lag_ports); + return -rte_errno; + } res = mlx5_tx_queue_pre_setup(dev, idx, &desc); if (res) return res; @@ -1095,6 +1102,7 @@ mlx5_txq_new(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, tmpl->txq.elts_m = desc - 1; tmpl->txq.port_id = dev->data->port_id; tmpl->txq.idx = idx; + tmpl->txq.tx_phy_affinity = conf->tx_phy_affinity; txq_set_params(tmpl); if (txq_adjust_params(tmpl)) goto error;