From patchwork Tue May 12 12:21:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiri Kuzin X-Patchwork-Id: 70105 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id EC3C9A04A2; Tue, 12 May 2020 14:22:20 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B54671BFE3; Tue, 12 May 2020 14:22:15 +0200 (CEST) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2052.outbound.protection.outlook.com [40.107.20.52]) by dpdk.org (Postfix) with ESMTP id B71FA1BF95 for ; Tue, 12 May 2020 14:22:13 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RJpVM8ak1V5tARcA3+DVoaWfJ62fFSAjGFHTqwmfOvRO23hEa1oYLUPFpW+GyKQwW0SwNfOtFNTwKLtVlb8uoW3S4S5TtSkTq5mxDRZUDAB75r1dOaM6AVp6GS9K+7dLHJKsj8CnUHyVF14RaCv3/FH9nIJdrip+93jIWN11lYmhelrJSD7s2fw0czg6sjunQMQp2SJqrV1PBpYXBP4Od7KKR9U30rgZACZ+WHquVskvmA8JyrYOKf+FyxTYQrHq9ZEYE5ok7qwG07dsR8S1BoPTMbWlrgF4G/OWRRBHv1OU1G8FO/J3nzjDZV9/+LCx2/41GP3nMEIbkjuNDppgmA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NaQpfBRqG2fzVV4Cwib4nn6vmFRqNSyfmyx32nCjwHY=; b=H8C4YekRD6K2lqZ5xUMtpjoA4FPtSfmg775DyH1UI9gszJ1aInq/uLwb/5FUAw5otmg3mOGTNVkYduHKzEiwyGSqZbFADhIa6RdjvC5oXXJLBf7g1KLeCvQD76Y2NrmEjq9j0pyUJSjfJnLLMW6VV70E5DIiqcoyAD9jDiJe5U8sdPKifTK9R6DRSqP5TI4T+G0CRSbpjJTlJE5NxjVPV13bjnhjJ4iVCHoEMxoffPljZh/OG8sKtbYDccvrGHIa8AoP4ebHbpvT1JtmluurXuCXJ4Cdv4JEUx+kSCcl78+q1i3l/6cQESkEbZ171ElJgHN4pNbZgZQcSVm3RNm+VQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NaQpfBRqG2fzVV4Cwib4nn6vmFRqNSyfmyx32nCjwHY=; b=dKkq9sPkN9vvgr0hPYtkXZ1qDAfzrn6Yj7V5wbSpO4kHZXHl+4rSmnBqae0fnYJs8/MvjdRk7LeyZgJv30HLL8BgEDya7VyAB/9/PqSnim+4CNjri+/V67phnbVM9/OkknSnefV8SUdSLsI8SPxcHv+3GIyyzbux85yWll28HoA= Authentication-Results: dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=none action=none header.from=mellanox.com; Received: from AM0PR0502MB3874.eurprd05.prod.outlook.com (2603:10a6:208:1a::29) by AM0PR0502MB3844.eurprd05.prod.outlook.com (2603:10a6:208:20::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2979.27; Tue, 12 May 2020 12:22:12 +0000 Received: from AM0PR0502MB3874.eurprd05.prod.outlook.com ([fe80::2cea:8f0a:cb2f:2a85]) by AM0PR0502MB3874.eurprd05.prod.outlook.com ([fe80::2cea:8f0a:cb2f:2a85%6]) with mapi id 15.20.2979.033; Tue, 12 May 2020 12:22:12 +0000 From: Shiri Kuzin To: dev@dpdk.org Cc: matan@mellanox.com, viacheslavo@mellanox.com Date: Tue, 12 May 2020 15:21:44 +0300 Message-Id: <1589286106-23411-2-git-send-email-shirik@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1589286106-23411-1-git-send-email-shirik@mellanox.com> References: <1589286106-23411-1-git-send-email-shirik@mellanox.com> X-ClientProxiedBy: AM3PR07CA0138.eurprd07.prod.outlook.com (2603:10a6:207:8::24) To AM0PR0502MB3874.eurprd05.prod.outlook.com (2603:10a6:208:1a::29) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mellanox.com (94.188.199.18) by AM3PR07CA0138.eurprd07.prod.outlook.com (2603:10a6:207:8::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3000.12 via Frontend Transport; Tue, 12 May 2020 12:22:12 +0000 X-Mailer: git-send-email 1.8.3.1 X-Originating-IP: [94.188.199.18] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: b7323913-8324-4f76-d055-08d7f66f196c X-MS-TrafficTypeDiagnostic: AM0PR0502MB3844:|AM0PR0502MB3844: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-Forefront-PRVS: 0401647B7F X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ZJXx9w80kimDFdV2DdG3VsAm1JNE0vh9ocCMeO5glYcAIFHLuIm9A3ZCHfPpkQP+BtaP3kgoP8uPboAwRPa7PMwsUTBhYVHQY5gxUDBW/xnbu3q0eEzDm/zCjx3zMSr6EwMgFQz9Idsh2IT23BLAASabDGfvnQYyIo127bVUP0Ud95wl4byIt6sVuLPCz9MZ2yP75dYNV7izpqFm77r6cWsjsl3T+orV7Z1+tjHWw3yXXTAc9KGbd6Kzgtugxuz1nW3W2Uke56XzQEAgdFISp93EEuA5/ETtoy2qx15NjvCttv+4sWeqSjyo5AgbMQrlZehxpyLPjY+jRy/5tydfwH3o/qlxDfAc4bMAtFcpIz1rjF8J/AHAyRnzNT6tTWiebbFBPg+8e7/7F4Ybh3LHJQ8iRS+KlcWZcLX7/fUw1AdHeAxfVfbGrQ/IDxBDohTQwLAA9VdRB6hx7zOkOslTy6n9bcsqQbKwSWF9Hu78kjRnXRHc4/U4yMkc6sCf38Lp2LLlGtWtFRCpcagfgn3yhg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR0502MB3874.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(346002)(39860400002)(376002)(136003)(366004)(396003)(33430700001)(8936002)(4326008)(55016002)(86362001)(107886003)(5660300002)(956004)(316002)(66946007)(66556008)(66476007)(2616005)(2906002)(52116002)(186003)(26005)(7696005)(16526019)(33440700001)(8886007)(6666004)(6916009)(8676002)(478600001)(36756003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: cUs3opevHPf86K54lFVt+uHvpm/+Cn9gIDwtp9ZT2C4F24rR6L2qsHxcw3O8IzCWBmfdANjwOrAyBsybdwyhGf1LH7SIbuUJGQiQohPDkkQFITddvlw/W95Wtew3vbd7mQu+PstHFZ7QWBdVkyv5lujU+p3/mcJ7nOkKq/FoUAcdwZsMs/zf/DgDtdsZXj2jwjP4sK2mI88k+lffP68QkiAiCOgu3DfNxV71lwc37QrrsKHAollQi6OpG2b5HKRnv2xI02lORZGuDeAHurK6bkHoYgg3d6+GvuPbtLlRWcOz4QjAmyhz/446CuWOWEd3ju7OHZcAJr4VSimjz3c6WUtyNhgNknvs2omXHpXwvH9F/hUxsoBQg3mBdRO1Us2M4uc7sA6qxKLnkmTRwRcBOq1VQSeF9WiBvHPvQQtaQlfBrytrdJOVtFau3CkOoYg6t0Tt8tpZm6l/jOPjcz+ScnlGaj6CLF8v5eARisVfrlI= X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: b7323913-8324-4f76-d055-08d7f66f196c X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2020 12:22:12.6717 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Y+Ctjj1o5UoQJC5JaN1b2N3TFLgnycZgwurbu2IlvTNz5bhqzSoZQyT4F4XyjCCFfMD3btJRHe0VlWUG3B2aIA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0502MB3844 Subject: [dpdk-dev] [PATCH 1/3] common/mlx5: fix relaxed ordering support detection X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Relaxed ordering is a PCI optimization that allows reordering of reads/writes in order to improve performance. In order to enable this optimization only when relaxed ordering is supported, it is checked if IBV_ACCESS_RELAXED_ORDERING is defined in verbs.h. Since IBV_ACCESS_RELAXED_ORDERING is an enum and not defined relaxed ordering wasn't enabled even when supported. This issue is fixed by using AUTOCONF to check if relaxed ordering is supported and disabling only if it isn't. Fixes: 53ac93f71ad1 ("net/mlx5: create relaxed ordering memory regions") Signed-off-by: Shiri Kuzin --- drivers/common/mlx5/Makefile | 5 +++++ drivers/common/mlx5/meson.build | 2 ++ drivers/common/mlx5/mlx5_glue.h | 2 +- 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/drivers/common/mlx5/Makefile b/drivers/common/mlx5/Makefile index 8b663ef..0d8cc1b 100644 --- a/drivers/common/mlx5/Makefile +++ b/drivers/common/mlx5/Makefile @@ -68,6 +68,11 @@ mlx5_autoconf.h.new: FORCE mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-h.sh $Q $(RM) -f -- '$@' $Q sh -- '$<' '$@' \ + HAVE_IBV_RELAXED_ORDERING \ + infiniband/verbs.h \ + enum IBV_ACCESS_RELAXED_ORDERING \ + $(AUTOCONF_OUTPUT) + $Q sh -- '$<' '$@' \ HAVE_IBV_DEVICE_STRIDING_RQ_SUPPORT \ infiniband/mlx5dv.h \ enum MLX5DV_CQE_RES_FORMAT_CSUM_STRIDX \ diff --git a/drivers/common/mlx5/meson.build b/drivers/common/mlx5/meson.build index 165aa25..5a802ba 100644 --- a/drivers/common/mlx5/meson.build +++ b/drivers/common/mlx5/meson.build @@ -94,6 +94,8 @@ has_member_args = [ # [ "MACRO to define if found", "header for the search", # "symbol to search" ] has_sym_args = [ + [ 'HAVE_IBV_RELAXED_ORDERING', 'infiniband/verbs.h', + 'IBV_ACCESS_RELAXED_ORDERING ' ], [ 'HAVE_IBV_DEVICE_STRIDING_RQ_SUPPORT', 'infiniband/mlx5dv.h', 'MLX5DV_CQE_RES_FORMAT_CSUM_STRIDX' ], [ 'HAVE_IBV_DEVICE_TUNNEL_SUPPORT', 'infiniband/mlx5dv.h', diff --git a/drivers/common/mlx5/mlx5_glue.h b/drivers/common/mlx5/mlx5_glue.h index 184c410..81d6a22 100644 --- a/drivers/common/mlx5/mlx5_glue.h +++ b/drivers/common/mlx5/mlx5_glue.h @@ -98,7 +98,7 @@ uint64_t comp_mask; }; #endif -#ifndef IBV_ACCESS_RELAXED_ORDERING +#ifndef HAVE_IBV_RELAXED_ORDERING #define IBV_ACCESS_RELAXED_ORDERING 0 #endif From patchwork Tue May 12 12:21:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiri Kuzin X-Patchwork-Id: 70106 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3966AA04A2; Tue, 12 May 2020 14:22:31 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 29C5D1BFEE; Tue, 12 May 2020 14:22:18 +0200 (CEST) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2049.outbound.protection.outlook.com [40.107.20.49]) by dpdk.org (Postfix) with ESMTP id 3E1901BFE1 for ; Tue, 12 May 2020 14:22:15 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dOKUvlhZXuX2iI+7gtq+/zZt3Pw5r+B0F4ii/KHQAEmDHXS7JgZXJ/Bc4aQx3mWUbAKPSyDE+z8ihm1+hebz65Hya5ICKGwPZ4X1OLa3NoQBHntIxfLRPgDvVb+BrThZeSqFeHmXTzEbo+ccaO5R2Jm0V1GcorXIL8lgqBsL4/PX5fogxG2Hh5WEjY8/mxTz8kDvn8ByQZqTWRxqVwsfFc6TVuJfp7Z9b1xAwoTkFx+CYI/cmFC5/FsAnwN0VTMOCLekzBy27aD1DHNIf1YQG3p+B4R+lMceSR3Zsy6QISLrWcRU0qO7SChc8llSe3EbvcFQfSb9MAKfZO0K1ZCZ7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Sy26Wi/CSECeB3WfUQ0/KnCaEMmYLibzuZAFXDWM+AQ=; b=b288e+lfXK1IOQT0jInTlmn387gTAGNi4pRtONHTTvZD1KkuM55NOYY32KLH3agzoCMvS8doUAA2uJBfpINB6YHTgTMgjSFReoY5i5Hfp3GvapR7RTDrQC0cA3gJoLF8bqns9FqscvkEo5TlHhJVUr0co9OKFZpqQXx3TuLHpvZA7Id0IIXfS/qjBfofyVinNYztsyGURpLlnr4KZblxfUATK1ClK8TtAd1Q1EY1TWk6nOg51kGVjco3x4XXDgRSw+1NdrPRhI/3wFP80Mnj7ggnKHcRQ4rg9qd5iKqhJb0ylf5tq1CV/8ozs77OYJEXjFCjG2s4+NmrLU9lrTAkwQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Sy26Wi/CSECeB3WfUQ0/KnCaEMmYLibzuZAFXDWM+AQ=; b=qEtu2scRr7tNGKZj7zKSFNnoSyVRju4Fke6zefan8rzxwDJxQMBxOm+uOi2tf6JZKOpBBmrGHTjjGdhGUBO4TfdOV79q8GTKZ9UCN3c7DFdS4v9Plb2SEg2YF3BtoTUbgoi5DF1FtCfsAur8kzdSYUxXnxPCtobOgf9/+t7V0tM= Authentication-Results: dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=none action=none header.from=mellanox.com; Received: from AM0PR0502MB3874.eurprd05.prod.outlook.com (2603:10a6:208:1a::29) by AM0PR0502MB3844.eurprd05.prod.outlook.com (2603:10a6:208:20::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2979.27; Tue, 12 May 2020 12:22:14 +0000 Received: from AM0PR0502MB3874.eurprd05.prod.outlook.com ([fe80::2cea:8f0a:cb2f:2a85]) by AM0PR0502MB3874.eurprd05.prod.outlook.com ([fe80::2cea:8f0a:cb2f:2a85%6]) with mapi id 15.20.2979.033; Tue, 12 May 2020 12:22:14 +0000 From: Shiri Kuzin To: dev@dpdk.org Cc: matan@mellanox.com, viacheslavo@mellanox.com Date: Tue, 12 May 2020 15:21:45 +0300 Message-Id: <1589286106-23411-3-git-send-email-shirik@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1589286106-23411-1-git-send-email-shirik@mellanox.com> References: <1589286106-23411-1-git-send-email-shirik@mellanox.com> X-ClientProxiedBy: AM3PR07CA0138.eurprd07.prod.outlook.com (2603:10a6:207:8::24) To AM0PR0502MB3874.eurprd05.prod.outlook.com (2603:10a6:208:1a::29) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mellanox.com (94.188.199.18) by AM3PR07CA0138.eurprd07.prod.outlook.com (2603:10a6:207:8::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3000.12 via Frontend Transport; Tue, 12 May 2020 12:22:13 +0000 X-Mailer: git-send-email 1.8.3.1 X-Originating-IP: [94.188.199.18] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 634ba28f-402e-4ec3-ffd3-08d7f66f1a65 X-MS-TrafficTypeDiagnostic: AM0PR0502MB3844:|AM0PR0502MB3844: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1079; X-Forefront-PRVS: 0401647B7F X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: M+w8KjZLswCxwB7Bxj4hCnBeF5qGS/x6/cIclXkGWmPiyWIu8zPhHb+Qxa1dIKk6v8mBN2Y/fohiSWCnulsh+lXEk/8dxw9nJRjfwuC/oJiO89mv+4y3tsLUhumF1ziPdFS7StJphCqh4aW5OFZzJNvvtSv9Hs8OBGr5NvdmWSuDdgxJFQadNbX2TgEgYvpgc4SuPyoyHTpLakgKS+zY3yhkVFbu+gZTxQ5zhat08O+IeyD5BIZYBXOHUFrVxluQ0pYP6jEXd/xVHR8mNwOFmYfOJMeEoeR2WYb3t65zxJvHOAOA9n4zLMOLxFOy6sPs5kui00E6TDGbWB6VbK5XB3h2znm+lZmAdVfTSJY1JCAsEok0XJkH7dcf0Q1v3qd8emHNDNNAAgW3fKx9k2xX4DQgUAEIe8451rkfBm6HsSgLuwQPwBj4IjxmACvVNoFKXteGF9ZMDAYtOHdaAfxpqO1BfIXG3wjrMsVJMV7Xnoix9YIivdMcCJ9KX30TlEx5GSPlW5Fulvma3eA/uuwF4A== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR0502MB3874.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(346002)(39860400002)(376002)(136003)(366004)(396003)(33430700001)(8936002)(4326008)(55016002)(86362001)(107886003)(5660300002)(956004)(316002)(66946007)(66556008)(66476007)(2616005)(2906002)(52116002)(186003)(26005)(7696005)(16526019)(33440700001)(8886007)(6666004)(6916009)(8676002)(478600001)(36756003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 1VHC+zYi58Rh1uElcgl1mIGBZT78v170K7DQ8GML1DFhrIG+Yq1ei8YSL7OEVwfpTn76I8wwQY9y/grD/Lx77ZgEXOJuiJau2ZbU3MD4SuSg98UMatgmOXwWlyGckNDXKjLAG25XuYggJgnusrCWKn+pBIoC8Pp5h2p4DwzpjuIyxEPFKiobfwh0uSEavjPP78Rq7wM9WzTkTpfTr6MM10Ilfql9icyQ1ZOHLYpltKz1QHvCg7TxI7t2pw2s5Xzg6kcG1FkF0YT2VdppTgS5XmViL+PTXUOBFgUy6SLmEQ/t87l3Z7pfGVJ1qZk7tL3Ihet2EGQMG6v3e/fpUmSvKiODLLhWRTcWMwCAkm8MA/ei2I3JMFFmI6zuD1DRopmmkJ0dCt1yGtqSKJPpB94+isZlQajfFAxQMk5K6xbxab3l9eEfhsJKGv3gnopkvGR05WCUYgjMZHPvvvvpjXN9am/MwIB4CslB3BWOtYHNDWM= X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 634ba28f-402e-4ec3-ffd3-08d7f66f1a65 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2020 12:22:14.2488 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: gQOjfMO5f9uc0WsnSepT5KemmcUtomMgpPRsW8uoTipMzKXddUU+fCYiwLVrzpw7lFU46Zq9Wu/O9gJDfiV//g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0502MB3844 Subject: [dpdk-dev] [PATCH 2/3] common/mlx5: fix relaxed ordering count object X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" In order to improve performance relaxed ordering was enabled when creating count object using Devx. Currently rte enables this optimization by default when using Devx. This causes an issue when using firmware that does not have this capability causing a count object failure. In order to fix this issue a check of firmware capabilities was added before enabling relaxed ordering. Fixes: 53ac93f71ad1 ("net/mlx5: create relaxed ordering memory regions") Signed-off-by: Shiri Kuzin --- drivers/common/mlx5/mlx5_devx_cmds.c | 4 ++++ drivers/common/mlx5/mlx5_devx_cmds.h | 2 ++ drivers/common/mlx5/mlx5_prm.h | 3 ++- drivers/net/mlx5/mlx5_flow_dv.c | 8 +++++--- 4 files changed, 13 insertions(+), 4 deletions(-) diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index 230ac58..fba485e 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -451,6 +451,10 @@ struct mlx5_devx_obj * attr->log_max_hairpin_num_packets = MLX5_GET (cmd_hca_cap, hcattr, log_min_hairpin_wq_data_sz); attr->vhca_id = MLX5_GET(cmd_hca_cap, hcattr, vhca_id); + attr->relaxed_ordering_write = MLX5_GET(cmd_hca_cap, hcattr, + relaxed_ordering_write); + attr->relaxed_ordering_read = MLX5_GET(cmd_hca_cap, hcattr, + relaxed_ordering_read); attr->eth_net_offloads = MLX5_GET(cmd_hca_cap, hcattr, eth_net_offloads); attr->eth_virt = MLX5_GET(cmd_hca_cap, hcattr, eth_virt); diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index ac10687..49b174a 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -100,6 +100,8 @@ struct mlx5_hca_attr { uint32_t log_max_hairpin_wq_data_sz:5; uint32_t log_max_hairpin_num_packets:5; uint32_t vhca_id:16; + uint32_t relaxed_ordering_write:1; + uint32_t relaxed_ordering_read:1; struct mlx5_hca_qos_attr qos; struct mlx5_hca_vdpa_attr vdpa; }; diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index 4ab1c75..6212085 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -999,7 +999,8 @@ struct mlx5_ifc_cmd_hca_cap_bits { u8 reserved_at_d0[0xb]; u8 log_max_cq[0x5]; u8 log_max_eq_sz[0x8]; - u8 reserved_at_e8[0x2]; + u8 relaxed_ordering_write[0x1]; + u8 relaxed_ordering_read[0x1]; u8 log_max_mkey[0x6]; u8 reserved_at_f0[0x8]; u8 dump_fill_mkey[0x1]; diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index 4ebb7ce..c7702c5 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -4100,8 +4100,8 @@ struct field_modify_info modify_tcp[] = { static struct mlx5_counter_stats_mem_mng * flow_dv_create_counter_stat_mem_mng(struct rte_eth_dev *dev, int raws_n) { - struct mlx5_ibv_shared *sh = ((struct mlx5_priv *) - (dev->data->dev_private))->sh; + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_ibv_shared *sh = priv->sh; struct mlx5_devx_mkey_attr mkey_attr; struct mlx5_counter_stats_mem_mng *mem_mng; volatile struct flow_counter_stats *raw_data; @@ -4133,7 +4133,9 @@ struct field_modify_info modify_tcp[] = { mkey_attr.pg_access = 0; mkey_attr.klm_array = NULL; mkey_attr.klm_num = 0; - mkey_attr.relaxed_ordering = 1; + if (priv->config.hca_attr.relaxed_ordering_write && + priv->config.hca_attr.relaxed_ordering_read) + mkey_attr.relaxed_ordering = 1; mem_mng->dm = mlx5_devx_cmd_mkey_create(sh->ctx, &mkey_attr); if (!mem_mng->dm) { mlx5_glue->devx_umem_dereg(mem_mng->umem); From patchwork Tue May 12 12:21:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiri Kuzin X-Patchwork-Id: 70107 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0EC1DA04A2; Tue, 12 May 2020 14:22:38 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6C8661C00D; Tue, 12 May 2020 14:22:19 +0200 (CEST) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2057.outbound.protection.outlook.com [40.107.20.57]) by dpdk.org (Postfix) with ESMTP id 0E0971BFEC for ; Tue, 12 May 2020 14:22:17 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CEtkoQ1AC9JUilMSvubV9UoiSvJMAk3wZ3m6cSdKrMyFuP4MtacTKzf+gLcOqywZbqkfUhr/+ZqX6V4uuWTtzgERIT/42stwaqJpgfFOA/42Y5kOei30k+3Ph+RLsqegFTyQceS03zEesVyy+oAq67sqV8A82WLpd21wmjBuMVVUTeHXWhBYJYO87sE2W/k/Oazca03pJYiEfD10UtkXN7UgZDv9V3dWtMs/4luGysCbwnYnsLXqh9AWeVzoQYw4X8b7oYXFMWS78QTpJmeV3MMHklmLuA+Y0lyxwnFDfIJqD3BkA94zQu6vlqEtBBi5fiOvjFCSVtlLvg6OG+Upfg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SfnQY21BEyHrM2kTJmHJmP4QWZYtNdpMsEdN6QpBQAU=; b=Z8slV1q2dZoeHIEChGYhhEGwI/Qmu89PL+5dHu+AeORuG1DSkrrF0kpyYe9aKTOWyKfQdWKtYysCfngPyW2O3UEQk+f15d+XsELBVhMEnDThssCjOB6EvHOJqMG2HqHzpLkzT3rrTIH9n2cz1lC/Pvv6zb/CDUUTX2xyeFfwgb0hoo3qso2ojmfdqCKRS5hz0RwL+2GvvnHNz9Ch8xLlfIdS8cCDpKDy582rw4956Zh3pHpDB9JLyxxdlHoPGGQ8/+itqpli0GNpA/IJF9AwZU7prnGBP1ojK0ctfi6PXcrsHtsiZhViGBUJ6PIhD158x9D2byxUoCTa2E9mSfbaiw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SfnQY21BEyHrM2kTJmHJmP4QWZYtNdpMsEdN6QpBQAU=; b=UXeXavDhIuw90sZOk7bDFGn8ss6A1WQRYoQx4qLyuNIVTg2HdHKVfwggNrzjXHsr1raAiAUZjVrvpvqyi3uioYZApkRKGwIZbz5eb6EfCv0weIdtLAHXIV4zIOFoibt8TKuS8zss1azpS4zsHc9b0wWEbub653JJGEOAW2Pp9Lg= Authentication-Results: dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=none action=none header.from=mellanox.com; Received: from AM0PR0502MB3874.eurprd05.prod.outlook.com (2603:10a6:208:1a::29) by AM0PR0502MB3844.eurprd05.prod.outlook.com (2603:10a6:208:20::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2979.27; Tue, 12 May 2020 12:22:16 +0000 Received: from AM0PR0502MB3874.eurprd05.prod.outlook.com ([fe80::2cea:8f0a:cb2f:2a85]) by AM0PR0502MB3874.eurprd05.prod.outlook.com ([fe80::2cea:8f0a:cb2f:2a85%6]) with mapi id 15.20.2979.033; Tue, 12 May 2020 12:22:16 +0000 From: Shiri Kuzin To: dev@dpdk.org Cc: matan@mellanox.com, viacheslavo@mellanox.com Date: Tue, 12 May 2020 15:21:46 +0300 Message-Id: <1589286106-23411-4-git-send-email-shirik@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1589286106-23411-1-git-send-email-shirik@mellanox.com> References: <1589286106-23411-1-git-send-email-shirik@mellanox.com> X-ClientProxiedBy: AM3PR07CA0138.eurprd07.prod.outlook.com (2603:10a6:207:8::24) To AM0PR0502MB3874.eurprd05.prod.outlook.com (2603:10a6:208:1a::29) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mellanox.com (94.188.199.18) by AM3PR07CA0138.eurprd07.prod.outlook.com (2603:10a6:207:8::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3000.12 via Frontend Transport; Tue, 12 May 2020 12:22:14 +0000 X-Mailer: git-send-email 1.8.3.1 X-Originating-IP: [94.188.199.18] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: fd8eab8e-039a-4302-5854-08d7f66f1b81 X-MS-TrafficTypeDiagnostic: AM0PR0502MB3844:|AM0PR0502MB3844: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:16; X-Forefront-PRVS: 0401647B7F X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: N561RmCxZT3UlyhSQcp0by0twlKzPrtBWfyKS+crgql/dES88+I99AB1ISDStGLhi5HfCtA8exRCFl/85ivlGXks1z3r6SwmUQCs3ModMqm5mlgbcosRen4WmV0Y6TtamzUbhhpprLOALyEsKkn+RscnrwzKfl74uO/WlR39ys1lsJzYGZVsRkNxUa56jAff//zCuuLab8t+HrdXaaGy4BHCs5LYJYXGLPpEsgOXejsbfbuC0F1uHsIHz2oedwNsMILqvXZrhqBxidc6ucK2Y3rLbVvHLcw8HS3KjpeIFShtVZBbJ0Jc+VE+MRg50Mhr7o4BYCzmo39SpvWn3nHVdvX58neLQ57+ZwvPEROBHW+Qe4jImQgXFlK1hMU8vjPxhzhse9NALeV9MNJ5EwxlmlurgSkAHN6baYMFu0YQVYqhVjZw0DMiMhujYo7q+idTq7VVXeHyM5LSe6MzSPzv2gmEQJ7fix8cnWPDpB2eIiNlr7YltkPUiJwWG1FqEa8pOycWMDINc4gZRcqpzxmenQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR0502MB3874.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(346002)(39860400002)(376002)(136003)(366004)(396003)(33430700001)(8936002)(4326008)(55016002)(86362001)(107886003)(5660300002)(956004)(316002)(66946007)(66556008)(66476007)(2616005)(2906002)(52116002)(186003)(26005)(7696005)(16526019)(33440700001)(8886007)(6666004)(6916009)(8676002)(478600001)(36756003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: yZZH+S1E33iWwUY6MM1vBct4QL1yP/gADfk+7aKWANHdPjyh6+cwX1u1uqITdC86/RKNhX3K0w5OzYjXA8TGBN3mG9ynll407ZtGWWPyuojHTlA4nd1cwnATeBRKnfVl+7ab1l4IIvfGuwXbnLxt3GMfJ6rXXGoBJ0t6SAVG7W4GdxchyWQge9TjqiTpu2kzMk7t6ri2pcIVXfHGVGASxW5FTIWXh3D64y+Qs7UMCMGdV66UbfZIuhwBYhI0i/9FSbz7dZQ0JJAbW25iSvH0FJtjDQs4eK46jSUI9V79vKvgcaVozEtbbP1Duxi2cKaHP/aQ6JzWaWOo72jJ3OUckYtCo1I8iUzIOGFuyWVcdy/6m50erM+5lt5NmU4IN9f/uwLoK93OVxBr83f4WW3Dh0g0LTWOASAnK0riz1W1CJu25BF8jbU5T1j8O75olCaffUadtmfQIVO/3YUsyPe/PRZe6u/znB3cIdp4YLo8QM4= X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: fd8eab8e-039a-4302-5854-08d7f66f1b81 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2020 12:22:16.0868 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: RfuMV4zSiBFSIU+Ce/rz7ItIGPvkAAex15HVz7HN4SplYma7g6fXOaxcINCETBOPJyPQdIexGyPFZeKtxzbAxQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0502MB3844 Subject: [dpdk-dev] [PATCH 3/3] common/mlx5: disable relaxed ordering in unsuitable cpus X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Relaxed ordering is a PCI optimization that enables reordering reads/writes in order to improve performance. Relaxed ordering was enabled for all processors causing a degradation in performance in Haswell and Broadwell processors that don't support this optimization. In order to avoid that we check if the processor is Haswell or Broadwell and if so we disable relaxed ordering. Signed-off-by: Shiri Kuzin --- drivers/common/mlx5/mlx5_common.c | 82 ++++++++++++++++++++++++++++++++++++ drivers/common/mlx5/mlx5_common.h | 2 + drivers/common/mlx5/mlx5_common_mr.c | 6 ++- drivers/net/mlx5/mlx5_flow_dv.c | 3 +- 4 files changed, 90 insertions(+), 3 deletions(-) diff --git a/drivers/common/mlx5/mlx5_common.c b/drivers/common/mlx5/mlx5_common.c index 4261045..1c77763 100644 --- a/drivers/common/mlx5/mlx5_common.c +++ b/drivers/common/mlx5/mlx5_common.c @@ -22,6 +22,8 @@ const struct mlx5_glue *mlx5_glue; #endif +uint8_t haswell_broadwell_cpu; + /** * Get PCI information by sysfs device path. * @@ -292,6 +294,29 @@ enum mlx5_class #endif +/* In case this is an x86_64 intel processor to check if + * we should use relaxed ordering. + */ +#ifdef RTE_ARCH_X86_64 +/** + * This function returns processor identification and feature information + * into the registers. + * + * @param eax, ebx, ecx, edx + * Pointers to the registers that will hold cpu information. + * @param level + * The main category of information returned. + */ +static inline void mlx5_cpu_id(unsigned int level, + unsigned int *eax, unsigned int *ebx, + unsigned int *ecx, unsigned int *edx) +{ + __asm__("cpuid\n\t" + : "=a" (*eax), "=b" (*ebx), "=c" (*ecx), "=d" (*edx) + : "0" (level)); +} +#endif + RTE_INIT_PRIO(mlx5_log_init, LOG) { mlx5_common_logtype = rte_log_register("pmd.common.mlx5"); @@ -350,3 +375,60 @@ enum mlx5_class mlx5_glue = NULL; return; } + +/** + * This function is responsible of initializing the variable + * haswell_broadwell_cpu by checking if the cpu is intel + * and reading the data returned from mlx5_cpu_id(). + * since haswell and broadwell cpus don't have improved performance + * when using relaxed ordering we want to check the cpu type before + * before deciding whether to enable RO or not. + * if the cpu is haswell or broadwell the variable will be set to 1 + * otherwise it will be 0. + */ +RTE_INIT_PRIO(mlx5_is_haswell_broadwell_cpu, LOG) +{ +#ifdef RTE_ARCH_X86_64 + unsigned int broadwell_models[4] = {0x3d, 0x47, 0x4F, 0x56}; + unsigned int haswell_models[4] = {0x3c, 0x3f, 0x45, 0x46}; + unsigned int i, model, family, brand_id, vendor; + unsigned int signature_intel_ebx = 0x756e6547; + unsigned int extended_model; + unsigned int eax = 0; + unsigned int ebx = 0; + unsigned int ecx = 0; + unsigned int edx = 0; + int max_level; + + mlx5_cpu_id(0, &eax, &ebx, &ecx, &edx); + vendor = ebx; + max_level = eax; + if (max_level < 1) { + haswell_broadwell_cpu = 0; + return; + } + mlx5_cpu_id(1, &eax, &ebx, &ecx, &edx); + model = (eax >> 4) & 0x0f; + family = (eax >> 8) & 0x0f; + brand_id = ebx & 0xff; + extended_model = (eax >> 12) & 0xf0; + /* Check if the processor is Haswell or Broadwell */ + if (vendor == signature_intel_ebx) { + if (family == 0x06) + model += extended_model; + if (brand_id == 0 && family == 0x6) { + for (i = 0; i < RTE_DIM(broadwell_models); i++) + if (model == broadwell_models[i]) { + haswell_broadwell_cpu = 1; + return; + } + for (i = 0; i < RTE_DIM(haswell_models); i++) + if (model == haswell_models[i]) { + haswell_broadwell_cpu = 1; + return; + } + } + } +#endif + haswell_broadwell_cpu = 0; +} diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index b37b820..8cd3ea5 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -213,4 +213,6 @@ enum mlx5_class { void mlx5_translate_port_name(const char *port_name_in, struct mlx5_switch_info *port_info_out); +extern uint8_t haswell_broadwell_cpu; + #endif /* RTE_PMD_MLX5_COMMON_H_ */ diff --git a/drivers/common/mlx5/mlx5_common_mr.c b/drivers/common/mlx5/mlx5_common_mr.c index 9d4a06d..3b46446 100644 --- a/drivers/common/mlx5/mlx5_common_mr.c +++ b/drivers/common/mlx5/mlx5_common_mr.c @@ -770,7 +770,8 @@ struct mlx5_mr * */ mr->ibv_mr = mlx5_glue->reg_mr(pd, (void *)data.start, len, IBV_ACCESS_LOCAL_WRITE | - IBV_ACCESS_RELAXED_ORDERING); + (haswell_broadwell_cpu ? 0 : + IBV_ACCESS_RELAXED_ORDERING)); if (mr->ibv_mr == NULL) { DEBUG("Fail to create a verbs MR for address (%p)", (void *)addr); @@ -1045,7 +1046,8 @@ struct mlx5_mr * return NULL; mr->ibv_mr = mlx5_glue->reg_mr(pd, (void *)addr, len, IBV_ACCESS_LOCAL_WRITE | - IBV_ACCESS_RELAXED_ORDERING); + (haswell_broadwell_cpu ? 0 : + IBV_ACCESS_RELAXED_ORDERING)); if (mr->ibv_mr == NULL) { DRV_LOG(WARNING, "Fail to create a verbs MR for address (%p)", diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c index c7702c5..8b018fb 100644 --- a/drivers/net/mlx5/mlx5_flow_dv.c +++ b/drivers/net/mlx5/mlx5_flow_dv.c @@ -4134,7 +4134,8 @@ struct field_modify_info modify_tcp[] = { mkey_attr.klm_array = NULL; mkey_attr.klm_num = 0; if (priv->config.hca_attr.relaxed_ordering_write && - priv->config.hca_attr.relaxed_ordering_read) + priv->config.hca_attr.relaxed_ordering_read && + !haswell_broadwell_cpu) mkey_attr.relaxed_ordering = 1; mem_mng->dm = mlx5_devx_cmd_mkey_create(sh->ctx, &mkey_attr); if (!mem_mng->dm) {