From patchwork Fri Jan 12 07:50:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Li X-Patchwork-Id: 135847 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5E4984389F; Fri, 12 Jan 2024 08:51:33 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4CCC84064E; Fri, 12 Jan 2024 08:51:33 +0100 (CET) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2064.outbound.protection.outlook.com [40.107.220.64]) by mails.dpdk.org (Postfix) with ESMTP id 5A3E44064A for ; Fri, 12 Jan 2024 08:51:32 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ChfugGX2xmQoHhgnYbKDe5wUYjeZBLMZ5Irh5mmsqjcXYbXNlTe917QYXgwD5VBlThcBvbPtj3+uK4CLFNBqWMf/tDBmFi2XRPDJX83Jo38jjgl5pqeZiy5eBGZedqeparn5Ic5i5tg6vM2sT1NsV5+oxSSZhe87Ngaa93w5dRcJWvr/pFoJHp0NZjMTKPPsSNpVT5vNpQ5RimT66cRwJN7uSEPUbzt+Vqw9nb/pwZw5ihVa3beapzNFQmWSyn1iwSh60WpvzZ6ALtpPlikZShHnVwJhGvnrNeakLyxngNRbxj84mVgTehmlpdbRm3rsQsB5YJKx7w0WuNPneOHwcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Xweu5PjNsV6taB0p7Bc3RlU3N6MTcmei58Lx2vln1us=; b=Fr7tPPeVqOP16cZ/JJobgI90y2LzBk+z8mQulsWzDmhrZ7H1zyX3X+7GVD4GjtnHYlzHhpAHkxHTQktFmlvXcsLjZzPmq/NErxvDHQ6G7AfY5l6NEm3/hP0uDmCBp3kSD7I75tSVExwcuRFowvv3l9/+QJGN7OXxibAZo4MteLzvffe5fXJfDTT2BZXNUNLW5L65kYg1YK3AlWNUU3dPushE1B9xHwgohRQnXe8RoIz6ZM5i2Syv9ceUeCsYVO5QjOuNPl/VFaILUZGxkRugQHGeK3Ri2W/74iXb2pPIbzdmwgOdTm/8GdqcujRQL37tEXCZDenyGOtLsisw8VJ2mQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Xweu5PjNsV6taB0p7Bc3RlU3N6MTcmei58Lx2vln1us=; b=jope8JXp9RN7b6JOs/+cOvaNR2i6qLQH8hgA2oNG2X7dIk4DJB1mOuSb7k4iIfQSoHqac9K/IWrZnZMIbhv5VYkoyFybPyNL/oqUfuQXYVt3Xu0KWFJqSMiXPJFsMoBLT6a1RUL9bPfMwZEaeGoS/GRCJPOpkoaTRjLLohRc5TLJOLlE91oKQFj2eBs/i+IKIlTtPe8xgQpV+JZQHJhRUZw0zDtrSYhDxmNOdVd7+FskZMVdAbdlgq84fX+cntgMal6iMRRnHwz70W+9nWCILOtMao8IauXL3yfmhH7trA8ZjXgB9pvnO+aRQ1/w/Vw5lXomDmobgMJKQ5K6+X2HXw== Received: from MW4PR03CA0280.namprd03.prod.outlook.com (2603:10b6:303:b5::15) by DM4PR12MB6256.namprd12.prod.outlook.com (2603:10b6:8:a3::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7181.17; Fri, 12 Jan 2024 07:51:29 +0000 Received: from CO1PEPF000044FA.namprd21.prod.outlook.com (2603:10b6:303:b5:cafe::b4) by MW4PR03CA0280.outlook.office365.com (2603:10b6:303:b5::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7181.21 via Frontend Transport; Fri, 12 Jan 2024 07:51:29 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by CO1PEPF000044FA.mail.protection.outlook.com (10.167.241.200) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7202.4 via Frontend Transport; Fri, 12 Jan 2024 07:51:29 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 11 Jan 2024 23:51:15 -0800 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 11 Jan 2024 23:51:13 -0800 From: Gavin Li To: , , , , , CC: , Subject: [V1 1/2] net/mlx5: discover IPv6 traffic class support in RDMA core Date: Fri, 12 Jan 2024 09:50:54 +0200 Message-ID: <20240112075055.1288263-2-gavinl@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240112075055.1288263-1-gavinl@nvidia.com> References: <20240112075055.1288263-1-gavinl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044FA:EE_|DM4PR12MB6256:EE_ X-MS-Office365-Filtering-Correlation-Id: 6cc726b9-edc0-4fe2-94e2-08dc13434950 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 6oy4ci3wuzXkn4569IGzdPDBeU/Zcz7fkC9J8gYUg6k7yvF2wWY24J8FoXLXaCSaD9/p2abssJ/c7mFq3j2CBmB1GrCSE3tod3kyot2WeoKirtpiCULRiFLY7S3AX58dusxtOKd8vqDYk2Ign6yB27EZPavNJ66JFXcj/XPBRonC4ToqE2mIXeCKxB4tCbzuTYsgEtgBfu10ChwwC+jNpQSvB/L8vyZh4r3fiUw6igFB1BqnP2CNmryOBhOHRH//N6NwYOuHHi5cYBkNitnSN/rLMOQX5I3Z9uu5dX04j8lR8VQ2sOzw9N7o24WzMZgvPsWksiMKgobukhzNEO/sjAPVhdQPj0CUbkkxRh+lJih/f8Y+I5HzQGudGIIFsFOa93bmbp6LZ2d0etm2lUesUYWjFgLWbqOXyAQ2KY3KHBXJfGtBZPHyZuiKVVUk3ytmvzD4nZ7rz4zeVPYyFT/0LxfIfrCu9xkunpIZjR9I6sa0n/n7/UEqSlm1FhYCwqVRwVTT1NfDhp4HvM2/XhmAhsfv/vE96ZzUpdOugaMak+g2XECJZvGeK/K9Wvx+V2OEQFCJ3DkJ5+Hc8fH0quQpFm7hHU0dNg7kAFO1kymTZODJJhADRVv7JeMOntVtnQRq40mFoJT2lN1qcpVXElMK7+lvNIOYY9TvZNmqwt9Ju4l1YWnuj0DjuZvZU5GA5OHyVKrN+5j+Bz9aFHwWj0/GHzOy/uXWwejJPESAnTdQDLtafV3KQsz9RFOyzer8FGnu/HUE6Ra0fiXGIbrgx4C4GA== X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230031)(4636009)(346002)(136003)(396003)(39860400002)(376002)(230922051799003)(1800799012)(186009)(64100799003)(82310400011)(451199024)(40470700004)(46966006)(36840700001)(2616005)(16526019)(83380400001)(336012)(26005)(1076003)(426003)(110136005)(70206006)(107886003)(6666004)(36860700001)(6286002)(5660300002)(8676002)(478600001)(7696005)(2906002)(54906003)(4326008)(8936002)(6636002)(316002)(41300700001)(70586007)(7636003)(47076005)(82740400003)(86362001)(36756003)(356005)(40480700001)(40460700003)(55016003)(2101003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Jan 2024 07:51:29.1959 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6cc726b9-edc0-4fe2-94e2-08dc13434950 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044FA.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6256 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Previously, IPv6 traffic class used the same ids of IPv4 DSCP and ECN by rdam core and firmware. New FW support new IPv6 traffic class id which is recommended to be used though the old way is still working. FW exposed a new cap bit to indicate the supporting of the new id while RDMA core does not have such mechanism. To fix the backward compatibility issue of combination of RDMA core and FW of different versions, a new function and a new flag were introduced to check if the new IPv6 traffic class id is supported by RDMA core. Signed-off-by: Gavin Li Acked-by: Suanming Mou --- drivers/net/mlx5/linux/mlx5_os.c | 4 +++ drivers/net/mlx5/mlx5.h | 1 + drivers/net/mlx5/mlx5_flow.c | 42 ++++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5_flow.h | 1 + 4 files changed, 48 insertions(+) diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index ae82e1e5d8..5ae31c88f4 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -1602,6 +1602,10 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, goto error; } rte_rwlock_init(&priv->ind_tbls_lock); + if (sh->config.dv_flow_en == 1 && + !priv->sh->ipv6_tc_fallback && + mlx5_flow_discover_ipv6_tc_support(eth_dev)) + priv->sh->ipv6_tc_fallback = 1; if (priv->sh->config.dv_flow_en == 2) { #ifdef HAVE_MLX5_HWS_SUPPORT if (priv->sh->config.dv_esw_en) { diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 263ebead7f..779805bcd8 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -1444,6 +1444,7 @@ struct mlx5_dev_ctx_shared { uint32_t lag_rx_port_affinity_en:1; /* lag_rx_port_affinity is supported. */ uint32_t hws_max_log_bulk_sz:5; + uint32_t ipv6_tc_fallback:1; /* Log of minimal HWS counters created hard coded. */ uint32_t hws_max_nb_counters; /* Maximal number for HWS counters. */ uint32_t max_port; /* Maximal IB device port index. */ diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index 85e8c77c81..90b72b7b0a 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -12476,3 +12476,45 @@ mlx5_flow_pick_transfer_proxy(struct rte_eth_dev *dev, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, "unable to find a proxy port"); } + +/** + * Discover ipv6 traffic class id support in rdma core and firmware. + * + * @param dev + * Ethernet device. + * + * @return + * 0, rdma core is good to work with firmware. + * -EOPNOTSUPP, rdma core could not work with new ipv6 tc id. + */ +int +mlx5_flow_discover_ipv6_tc_support(struct rte_eth_dev *dev) +{ + struct rte_flow_action_set_dscp set_dscp; + struct rte_flow_attr attr; + struct rte_flow_action actions[2]; + struct rte_flow_item items[3]; + struct rte_flow_error error; + uint32_t flow_idx; + + memset(&attr, 0, sizeof(attr)); + memset(actions, 0, sizeof(actions)); + memset(items, 0, sizeof(items)); + attr.group = 1; + attr.egress = 1; + items[0].type = RTE_FLOW_ITEM_TYPE_ETH; + items[1].type = RTE_FLOW_ITEM_TYPE_IPV6; + items[2].type = RTE_FLOW_ITEM_TYPE_END; + /* Random value */ + set_dscp.dscp = 9; + actions[0].type = RTE_FLOW_ACTION_TYPE_SET_IPV6_DSCP; + actions[0].conf = &set_dscp; + actions[1].type = RTE_FLOW_ACTION_TYPE_END; + + flow_idx = flow_list_create(dev, MLX5_FLOW_TYPE_GEN, &attr, items, actions, true, &error); + if (!flow_idx) + return -EOPNOTSUPP; + + flow_list_destroy(dev, MLX5_FLOW_TYPE_GEN, flow_idx); + return 0; +} diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index 120609c595..33d4a28077 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -2638,6 +2638,7 @@ void mlx5_flow_destroy_sub_policy_with_rxq(struct rte_eth_dev *dev, struct mlx5_flow_meter_policy *mtr_policy); int mlx5_flow_dv_discover_counter_offset_support(struct rte_eth_dev *dev); int mlx5_flow_discover_dr_action_support(struct rte_eth_dev *dev); +int mlx5_flow_discover_ipv6_tc_support(struct rte_eth_dev *dev); int mlx5_action_handle_attach(struct rte_eth_dev *dev); int mlx5_action_handle_detach(struct rte_eth_dev *dev); int mlx5_action_handle_flush(struct rte_eth_dev *dev); From patchwork Fri Jan 12 07:50:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Li X-Patchwork-Id: 135848 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 85F544389F; Fri, 12 Jan 2024 08:51:41 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DE8484069D; Fri, 12 Jan 2024 08:51:34 +0100 (CET) Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2040.outbound.protection.outlook.com [40.107.102.40]) by mails.dpdk.org (Postfix) with ESMTP id 584DE4067B for ; Fri, 12 Jan 2024 08:51:33 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ldCmkzF7fZI80nGY/qUi5WuSrCSbAYyssPeyXgw0YK3JbNQWTKQZ2wybfMPV22WPD5kbUkSdz+GppQHtgzgyiRz6/RDcAlD/OOVaG1GIirNSzgCXqw38QKjGYPd3SyClkC1oQ5m/1+V0nQ6d9nI9TYvjUjbwI6tSwEaOBxwyfV1OkfBvbN3gBFBYwD22jJXA61ll7P84WV/Nx0D9IK5F98FEdPRe0kiHR/pSm22dwirrnB1/LCv2+yGOMcHcAJiS3zTHB+5xoaS9w0nxgZdxGvxaU54HHCaNgrUfcbJXjsEh1Q+dL/5M8T/6uVPXVObtnf7O3j+KJ1vzCKaoF3PB3w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bltEg0QvRKkZQjxvr5+Orgxt0h2/ignATRzFqK1uD0A=; b=BO1hLBrFkXaaa8LDjAsuri6MqHE2e2+WxOZYgoRbhoMs2KNwXeTOWUdW7+7GbiY3U0TuwyBpv53lSuyDk/5ziUUt2svPpMTsZZPnSlw58U821USDdJidsKaEgJND7dsXTluXYG1H8sxDxpbcf1w3fZ33gKmWImp34o+XCKDiba6vjndhSVtxp0WaeGbxuYUaIxrSaHyiEd8WV+oMaV3R4eRbJ4oLaxCW+CdDZ/GxznKYGqmHRYm3MPr/c42MdNmWoMiXrJ0Nt+lXah9Bj3iRGHwdhqTjZgZHX95Ez96NRlIjkPMcvt/cJkLYT3mPj8bzsw88y+Lqf9Cdum2HwtuLBA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bltEg0QvRKkZQjxvr5+Orgxt0h2/ignATRzFqK1uD0A=; b=jY/ignUPCo7Auh2erZ6dfOGy2AHg4bqD7T5iuswTDA8UMmqzewL9KKFqjtqUeukCtAqsYe6TPNNON9Z/qYar5KnS+6lg9xlNNAMWX2TMZkHLg9kmEPa0WD8xYdizzbhyNflIel2QTYInX7jvtcWEuX0uaTeqhNd0FVhR2tU12CJm3K3OdzEqBdWe35gu2g8ywCq4Mz9SVMNtbvXvD5rO82g9y7kewrk2DlLJadqIpFK1HOtyo8nv4/mVxTyDAHDUP8MGiuwQx8rlXEY8rPj3MQpAQ8hT6ddDRUhA6ijtbFU6ejo60Gr/c/eWNbWWMVyMqyr1Z0XpB7kfM/lxiNb1wA== Received: from SN7PR18CA0022.namprd18.prod.outlook.com (2603:10b6:806:f3::21) by MW3PR12MB4474.namprd12.prod.outlook.com (2603:10b6:303:2e::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7181.18; Fri, 12 Jan 2024 07:51:31 +0000 Received: from SN1PEPF0002636A.namprd02.prod.outlook.com (2603:10b6:806:f3:cafe::78) by SN7PR18CA0022.outlook.office365.com (2603:10b6:806:f3::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7181.21 via Frontend Transport; Fri, 12 Jan 2024 07:51:30 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SN1PEPF0002636A.mail.protection.outlook.com (10.167.241.135) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7181.14 via Frontend Transport; Fri, 12 Jan 2024 07:51:30 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 11 Jan 2024 23:51:18 -0800 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Thu, 11 Jan 2024 23:51:15 -0800 From: Gavin Li To: , , , , , CC: , Subject: [V1 2/2] net/mlx5: use traffic class PRM field for IPv6 modification Date: Fri, 12 Jan 2024 09:50:55 +0200 Message-ID: <20240112075055.1288263-3-gavinl@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240112075055.1288263-1-gavinl@nvidia.com> References: <20240112075055.1288263-1-gavinl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002636A:EE_|MW3PR12MB4474:EE_ X-MS-Office365-Filtering-Correlation-Id: 3b772e86-d8fe-43e8-353f-08dc134349fc X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: jJ+v7O6BFVwURodkAnP9Ec88Ax03pCTHZtc2spKcm7Uto6wga+eAU4N/NGu2kmrAUL2V0U4di4lxZsePH73It321cg6SJ4U2iK0IBlLzaK309mXN5SHWyV07kz+6Zajw91viXBOjB8MWUk7Wx4GMsEaBHuTcmo+ABEPoQdoyI2emt4MQr9VweyddRmFTFmJMs5fYKIrlMPPKkxlDUf8C3D7vvEoPps+4E/5jOqXQnhC6kLSqXpWo+Qkz8XQxkteIvdVPwtiX6WGB364pGeR7QvLTMk3XWn9o5j+vTZuHIZUnF2sFMgH4hzZz6ZaGO5gaWfj0WwufaYJXped3w2orZNwjnIn6nvsCKE/vlulVjKR3/TGWLoiAdH5cF4YYuoVZvvYkP+DbttrMhhos4ZPp2YrUEbntN99Zh+2Ekf6ezqg9h0Aa1R89HaY0JnAMMuaczqu+rcAfHVJ4I3sDcBFkeOEnOi2dX4a1hRGFnbdC1XqPe2JW7gOcNjFkczcB5vPL90UhFL0R6+D/uJ4N0S7Bzgv34ypH+t2acK796zehybALj269BUtp2e5N/1S/AZEtqFDFRZ5/bd9Mh9vjKOvqV5I4sARFcchh+lfiUUrDDGKkQy8bDLr4y2JY1c3pw3sHSirLLUN84l4jdvhT7DeeWE0REHyPAgOGvsYKHuMc4TXm79CKMSC6ZJi3wJCpz5je0GTXyEKkKJDQOSiMAqLT5lM0/mGLR26dlB3uss0qEhc6NXJQKZiHnZVDe9kuAPRm5DNycnZRdR9Og+lB221QQQ== X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230031)(4636009)(346002)(376002)(136003)(396003)(39860400002)(230922051799003)(1800799012)(64100799003)(186009)(82310400011)(451199024)(36840700001)(40470700004)(46966006)(40480700001)(55016003)(40460700003)(83380400001)(336012)(70586007)(36756003)(356005)(86362001)(7636003)(41300700001)(70206006)(36860700001)(1076003)(110136005)(6286002)(2616005)(4326008)(26005)(47076005)(426003)(107886003)(16526019)(478600001)(6636002)(54906003)(316002)(2906002)(8936002)(7696005)(8676002)(5660300002)(6666004)(30864003)(82740400003)(2101003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Jan 2024 07:51:30.3363 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3b772e86-d8fe-43e8-353f-08dc134349fc X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002636A.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR12MB4474 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org New PRM defined new field OUT_IPV6_TRAFFIC_CLASS for IPv6 which will be used by both IPv6 ECN and DSCP. A new cap bit modify_out_ipv6_traffic_class is added. It can be used to check if the new field is supported by FW. However, IPv6 ECN and DSCP starts from different offset in the same byte. Update SWS and HWS to used the new filed and introduce extra offset for IPv6 DSCP data and mask to solve the issue. Signed-off-by: Gavin Li Acked-by: Suanming Mou --- drivers/common/mlx5/mlx5_devx_cmds.c | 3 ++ drivers/common/mlx5/mlx5_devx_cmds.h | 1 + drivers/common/mlx5/mlx5_prm.h | 8 ++- drivers/net/mlx5/linux/mlx5_os.c | 5 +- drivers/net/mlx5/mlx5_flow.h | 3 ++ drivers/net/mlx5/mlx5_flow_dv.c | 78 ++++++++++++++++++++++++---- drivers/net/mlx5/mlx5_flow_hw.c | 7 +++ 7 files changed, 92 insertions(+), 13 deletions(-) diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index 4d8818924a..3a894f894a 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -1229,6 +1229,9 @@ mlx5_devx_cmd_query_hca_attr(void *ctx, attr->modify_outer_ip_ecn = MLX5_GET (flow_table_nic_cap, hcattr, ft_header_modify_nic_receive.outer_ip_ecn); + attr->modify_outer_ipv6_traffic_class = MLX5_GET + (flow_table_nic_cap, hcattr, + ft_header_modify_nic_receive.outer_ipv6_traffic_class); attr->set_reg_c = 0xffff; if (attr->nic_flow_table) { #define GET_RX_REG_X_BITS \ diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index 7f23e925a5..4a6008dc1a 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -304,6 +304,7 @@ struct mlx5_hca_attr { uint32_t set_reg_c:16; uint32_t nic_flow_table:1; uint32_t modify_outer_ip_ecn:1; + uint32_t modify_outer_ipv6_traffic_class:1; union { uint32_t max_flow_counter; struct { diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index 0d46ba9c40..69404b5ed8 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -848,6 +848,7 @@ enum mlx5_modification_field { MLX5_MODI_META_REG_C_13 = 0x94, MLX5_MODI_META_REG_C_14 = 0x95, MLX5_MODI_META_REG_C_15 = 0x96, + MLX5_MODI_OUT_IPV6_TRAFFIC_CLASS = 0x11C, MLX5_MODI_OUT_IPV4_TOTAL_LEN = 0x11D, MLX5_MODI_OUT_IPV6_PAYLOAD_LEN = 0x11E, MLX5_MODI_OUT_IPV4_IHL = 0x11F, @@ -2202,7 +2203,9 @@ struct mlx5_ifc_ft_fields_support_bits { u8 metadata_reg_c_x[0x8]; }; /* end of DW3 */ /* set_action_field_support_2 */ - u8 reserved_at_80[0x80]; + u8 reserved_at_80[0x37]; + u8 outer_ipv6_traffic_class[0x1]; + u8 reserved_at_B8[0x48]; /* add_action_field_support */ u8 reserved_at_100[0x80]; /* add_action_field_support_2 */ @@ -2240,7 +2243,8 @@ struct mlx5_ifc_ft_fields_support_2_bits { u8 inner_l4_checksum_ok[0x1]; u8 outer_ipv4_checksum_ok[0x1]; u8 outer_l4_checksum_ok[0x1]; /* end of DW0 */ - u8 reserved_at_20[0x18]; + u8 reserved_at_20[0x17]; + u8 outer_ipv6_traffic_class[0x1]; union { struct { u8 metadata_reg_c_15[0x1]; diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 5ae31c88f4..6ea0296109 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -1602,9 +1602,10 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, goto error; } rte_rwlock_init(&priv->ind_tbls_lock); - if (sh->config.dv_flow_en == 1 && + if (!priv->sh->cdev->config.hca_attr.modify_outer_ipv6_traffic_class || + (sh->config.dv_flow_en == 1 && !priv->sh->ipv6_tc_fallback && - mlx5_flow_discover_ipv6_tc_support(eth_dev)) + mlx5_flow_discover_ipv6_tc_support(eth_dev))) priv->sh->ipv6_tc_fallback = 1; if (priv->sh->config.dv_flow_en == 2) { #ifdef HAVE_MLX5_HWS_SUPPORT diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index 33d4a28077..fe4f46724b 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -413,6 +413,9 @@ enum mlx5_feature_name { #define IPPROTO_MPLS 137 #endif +#define MLX5_IPV6_HDR_ECN_MASK 0x3 +#define MLX5_IPV6_HDR_DSCP_SHIFT 2 + /* UDP port number for MPLS */ #define MLX5_UDP_PORT_MPLS 6635 diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index 97f55003c3..ecf86d861d 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -253,6 +253,11 @@ struct field_modify_info modify_ipv6[] = { {0, 0, 0}, }; +struct field_modify_info modify_ipv6_traffic_class[] = { + {1, 0, MLX5_MODI_OUT_IPV6_TRAFFIC_CLASS}, + {0, 0, 0}, +}; + struct field_modify_info modify_udp[] = { {2, 0, MLX5_MODI_OUT_UDP_SPORT}, {2, 2, MLX5_MODI_OUT_UDP_DPORT}, @@ -1323,6 +1328,7 @@ static int flow_dv_convert_action_modify_ipv6_dscp (struct mlx5_flow_dv_modify_hdr_resource *resource, const struct rte_flow_action *action, + uint32_t ipv6_tc_off, struct rte_flow_error *error) { const struct rte_flow_action_set_dscp *conf = @@ -1330,6 +1336,7 @@ flow_dv_convert_action_modify_ipv6_dscp struct rte_flow_item item = { .type = RTE_FLOW_ITEM_TYPE_IPV6 }; struct rte_flow_item_ipv6 ipv6; struct rte_flow_item_ipv6 ipv6_mask; + struct field_modify_info *modify_info; memset(&ipv6, 0, sizeof(ipv6)); memset(&ipv6_mask, 0, sizeof(ipv6_mask)); @@ -1338,12 +1345,19 @@ flow_dv_convert_action_modify_ipv6_dscp * rdma-core only accept the DSCP bits byte aligned start from * bit 0 to 5 as to be compatible with IPv4. No need to shift the * bits in IPv6 case as rdma-core requires byte aligned value. + * IPV6 DSCP uses OUT_IPV6_TRAFFIC_CLASS as ID but it starts from 2 + * bits left. Shift the mask left for IPV6 DSCP. Do it here because + * it's needed to distinguish DSCP from ECN in data field construct */ - ipv6.hdr.vtc_flow = conf->dscp; - ipv6_mask.hdr.vtc_flow = RTE_IPV6_HDR_DSCP_MASK >> 22; + ipv6.hdr.vtc_flow = conf->dscp << ipv6_tc_off; + ipv6_mask.hdr.vtc_flow = RTE_IPV6_HDR_DSCP_MASK >> (22 - ipv6_tc_off); item.spec = &ipv6; item.mask = &ipv6_mask; - return flow_dv_convert_modify_action(&item, modify_ipv6, NULL, resource, + if (ipv6_tc_off) + modify_info = modify_ipv6_traffic_class; + else + modify_info = modify_ipv6; + return flow_dv_convert_modify_action(&item, modify_info, NULL, resource, MLX5_MODIFICATION_TYPE_SET, error); } @@ -1576,6 +1590,12 @@ mlx5_modify_flex_item(const struct rte_eth_dev *dev, } } +static inline bool +mlx5_dv_modify_ipv6_traffic_class_supported(struct mlx5_priv *priv) +{ + return !priv->sh->ipv6_tc_fallback; +} + void mlx5_flow_field_id_to_modify_info (const struct rte_flow_action_modify_data *data, @@ -1731,9 +1751,20 @@ mlx5_flow_field_id_to_modify_info break; case RTE_FLOW_FIELD_IPV6_DSCP: MLX5_ASSERT(data->offset + width <= 6); - off_be = 6 - (data->offset + width); - info[idx] = (struct field_modify_info){1, 0, - MLX5_MODI_OUT_IP_DSCP}; + /* + * IPV6 DSCP uses OUT_IPV6_TRAFFIC_CLASS as ID but it starts from 2 + * bits left. Shift the mask left for IPV6 DSCP. Do it here because + * it's needed to distinguish DSCP from ECN in data field construct + */ + if (mlx5_dv_modify_ipv6_traffic_class_supported(priv)) { + off_be = 6 - (data->offset + width) + MLX5_IPV6_HDR_DSCP_SHIFT; + info[idx] = (struct field_modify_info){1, 0, + MLX5_MODI_OUT_IPV6_TRAFFIC_CLASS}; + } else { + off_be = 6 - (data->offset + width); + info[idx] = (struct field_modify_info){1, 0, + MLX5_MODI_OUT_IP_DSCP}; + } if (mask) mask[idx] = flow_modify_info_mask_8(width, off_be); else @@ -2029,7 +2060,6 @@ mlx5_flow_field_id_to_modify_info } break; case RTE_FLOW_FIELD_IPV4_ECN: - case RTE_FLOW_FIELD_IPV6_ECN: MLX5_ASSERT(data->offset + width <= 2); off_be = 2 - (data->offset + width); info[idx] = (struct field_modify_info){1, 0, @@ -2039,6 +2069,20 @@ mlx5_flow_field_id_to_modify_info else info[idx].offset = off_be; break; + case RTE_FLOW_FIELD_IPV6_ECN: + MLX5_ASSERT(data->offset + width <= 2); + off_be = 2 - (data->offset + width); + if (mlx5_dv_modify_ipv6_traffic_class_supported(priv)) + info[idx] = (struct field_modify_info){1, 0, + MLX5_MODI_OUT_IPV6_TRAFFIC_CLASS}; + else + info[idx] = (struct field_modify_info){1, 0, + MLX5_MODI_OUT_IP_ECN}; + if (mask) + mask[idx] = flow_modify_info_mask_8(width, off_be); + else + info[idx].offset = off_be; + break; case RTE_FLOW_FIELD_GTP_PSC_QFI: MLX5_ASSERT(data->offset + width <= 8); off_be = data->offset + 8; @@ -2161,7 +2205,7 @@ flow_dv_convert_action_modify_field struct field_modify_info dcopy[MLX5_ACT_MAX_MOD_FIELDS] = { {0, 0, 0} }; uint32_t mask[MLX5_ACT_MAX_MOD_FIELDS] = {0, 0, 0, 0, 0}; - uint32_t type, meta = 0; + uint32_t type, meta = 0, dscp = 0; if (conf->src.field == RTE_FLOW_FIELD_POINTER || conf->src.field == RTE_FLOW_FIELD_VALUE) { @@ -2181,6 +2225,17 @@ flow_dv_convert_action_modify_field meta = rte_cpu_to_be_32(meta); item.spec = &meta; } + if (mlx5_dv_modify_ipv6_traffic_class_supported(dev->data->dev_private) && + conf->dst.field == RTE_FLOW_FIELD_IPV6_DSCP && + !(mask[0] & MLX5_IPV6_HDR_ECN_MASK)) { + dscp = *(const unaligned_uint32_t *)item.spec; + /* + * IPV6 DSCP uses OUT_IPV6_TRAFFIC_CLASS as ID but it starts from 2 + * bits left. Shift the data left for IPV6 DSCP + */ + dscp <<= MLX5_IPV6_HDR_DSCP_SHIFT; + item.spec = &dscp; + } } else { type = MLX5_MODIFICATION_TYPE_COPY; /** For COPY fill the destination field (dcopy) without mask. */ @@ -14385,6 +14440,7 @@ flow_dv_translate(struct rte_eth_dev *dev, struct mlx5_flow_sub_actions_list *sample_act; uint32_t sample_act_pos = UINT32_MAX; uint32_t age_act_pos = UINT32_MAX; + uint32_t ipv6_tc_off = 0; uint32_t num_of_dest = 0; int tmp_actions_n = 0; uint32_t table; @@ -14941,8 +14997,12 @@ flow_dv_translate(struct rte_eth_dev *dev, action_flags |= MLX5_FLOW_ACTION_SET_IPV4_DSCP; break; case RTE_FLOW_ACTION_TYPE_SET_IPV6_DSCP: + if (mlx5_dv_modify_ipv6_traffic_class_supported(priv)) + ipv6_tc_off = MLX5_IPV6_HDR_DSCP_SHIFT; + else + ipv6_tc_off = 0; if (flow_dv_convert_action_modify_ipv6_dscp(mhdr_res, - actions, error)) + actions, ipv6_tc_off, error)) return -rte_errno; action_flags |= MLX5_FLOW_ACTION_SET_IPV6_DSCP; break; diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c index c4a90a3690..504a250e44 100644 --- a/drivers/net/mlx5/mlx5_flow_hw.c +++ b/drivers/net/mlx5/mlx5_flow_hw.c @@ -2862,6 +2862,13 @@ flow_hw_modify_field_construct(struct mlx5_hw_q_job *job, } off_b = rte_bsf32(mask); data = flow_dv_fetch_field(values + field->offset, field->size); + /* + * IPV6 DSCP uses OUT_IPV6_TRAFFIC_CLASS as ID but it starts from 2 + * bits left. Shift the data left for IPV6 DSCP + */ + if (field->id == MLX5_MODI_OUT_IPV6_TRAFFIC_CLASS && + !(mask & MLX5_IPV6_HDR_ECN_MASK)) + data <<= MLX5_IPV6_HDR_DSCP_SHIFT; data = (data & mask) >> off_b; job->mhdr_cmd[i++].data1 = rte_cpu_to_be_32(data); ++field;