From patchwork Wed May 24 12:01:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: erezf@nvidia.com X-Patchwork-Id: 127325 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D12C842B8C; Wed, 24 May 2023 14:02:41 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1F5E241101; Wed, 24 May 2023 14:02:41 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2079.outbound.protection.outlook.com [40.107.93.79]) by mails.dpdk.org (Postfix) with ESMTP id BEF714067E; Wed, 24 May 2023 14:02:38 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CKoi+FQMqr9/mH5CYXWQl6VQHKSB2nFbe4LDu1P6FC4Fa2Bn2d/K6xhgUiT7pwBlG9kFHem1RY87PVlYnX/jsP7Pg4lYeQCR5izS8KUz5uSWOAiqf0u6EeZDRwn/kWJMjmWkZWOJgujyoMiBMS47dNc2Hn2YiK3GJa5noIy2/HS2Ix4MCmCWVWaOHT1iF9s7o7nD5qv25Y3vrv7LQzXASuSDc2cQoWKOkOSekLd6QXQfzxeYhd0cf6S1pQhmcdBOOkBJ03Q0Y1uwJZ8o/SBM0g4z5qt8RL4iqJrabgGqVjWh7UVN2oJjgVndHAjEv2NaEDMWT+WSKrzZ0Q/14Z3FpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ZM8XFqR5MCRIgWIMQL0UWe5CZyg3HPy/MlIOS9jj6IM=; b=KeRvYt5OLvkmbdurdjTrw+2FNhndQaroKuDGf5M8IXhXJmKQWUGmi3hM/K/okqgaLBKQtdVHl5aEOivDBKPsyFCyFnVdK+IKPmWNZxYyo3sQFi2s8gRz/7vHCt9yVAOlls4T48u4T7hkjVZbab489oVhOtgbCO7yMrOSvwBpPDDOTiKQer+1oewsRo1c5aKsTtVL0Sd+3yDKANucqWbfB7n5UtinfBh11wW844ZEBxaRj3eZBnwg5ccZ8oQ/Jxqn/wBLW5YNOw7AgMn1AqgeDwG/ISgQUSRh6vE6GBnYeGPSF3GdvCtybY8iay+gR9njSnZiUVKsdTjAUHyVrquKrA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZM8XFqR5MCRIgWIMQL0UWe5CZyg3HPy/MlIOS9jj6IM=; b=qW1frMxNphOSBp+Uvm7Wj+lekmrR6THGK0ZP8nH4tbU6v/WFTHLysZ2BrTW7yks/5X4git+wcuovPSQp++LW1DI40c34sJLZtMJM5FGjmiBsCS2fYB/y5PmoLoAED5ax9Mrj0HWON3SQaaHvKy4izjE8HeNke4fNq2nQPJyqu+vD0AEmQFRhJDSk0eB8GfvB9Kv8F4kDT90mlZmgi5XstsqXZclVe0+CeDp19S26Qjf25IOYd+2RYUXCWIJkWnVj9dTiVlJYsYLPGv7Ru6SLVJf5gFFSlfs4OwvRzzk1RqRW8wosGzlyhH9O4kJG1B+iA47oaMRz5rtDtm7J71aKJg== Received: from BN0PR03CA0045.namprd03.prod.outlook.com (2603:10b6:408:e7::20) by DM4PR12MB7549.namprd12.prod.outlook.com (2603:10b6:8:10f::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6433.15; Wed, 24 May 2023 12:02:35 +0000 Received: from BN8NAM11FT060.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e7:cafe::8d) by BN0PR03CA0045.outlook.office365.com (2603:10b6:408:e7::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6433.15 via Frontend Transport; Wed, 24 May 2023 12:02:35 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BN8NAM11FT060.mail.protection.outlook.com (10.13.177.211) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6411.30 via Frontend Transport; Wed, 24 May 2023 12:02:35 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Wed, 24 May 2023 05:02:15 -0700 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Wed, 24 May 2023 05:02:13 -0700 From: To: CC: , , , "Erez Ferber" , Subject: [PATCH] common/mlx5: adjust fork call with the new kernel API Date: Wed, 24 May 2023 15:01:40 +0300 Message-ID: <20230524120140.416144-1-erezf@nvidia.com> X-Mailer: git-send-email 2.14.1 MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT060:EE_|DM4PR12MB7549:EE_ X-MS-Office365-Filtering-Correlation-Id: 699a6e7d-99e6-4e98-45fe-08db5c4ec326 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: mEMTBkD/JlQghLosmavpqy+lYj+dSkmX4tLh/sA03a8fTD/2tIQKfPrFNLUii1uTlTCqiokjARm19+jbiiXKV6qUO97+MmLDSdefJbmk+ARhq2LJoj5rpqidgcPjyFFTCPGi7GO1z9ZybdzA7SBSN7/M2xezF2cN6iR3MaWW/4KKmUDydzsEJgXwXSAs009rZaKvim9QVvckU83rKrHAdTjBhCRfkJt/ri0Y66DQj/LGQXRHrdVF8BznWL9NRY2174dM2kDGKWH27thefzusLyb05LyNbzwjTmNcpQff2fB2wplsuBrym2T1kKb9jGM55gP+PnTjqT1ht1xgk8svyOJ/3z+Of4jT5qkXs6nl494dAcwL4wQmvRx4T+pyizdzz8avO/HNxlZk161Ssszqxm1xdUQaai83hdbosJOBI5D393iURxRRexLhZ4+zsEQSEhbIkZ3kjaXpjrwa1aj6tEPcW7hA0uZ4ZTHWGLnM0c7nVM1dLaD4c/JmIa9sPcORR5JR6hXxv3AheU3lEv3v/6HnFuyrRYm3Qs1qG34uHEe/UBJyMS/hmsU7CvgCag2clMbNO+z7dJ91/xvdFW3AlLgr79pXtGsU71yM1uXzhVXFO7MG0JfpppHj7qtnIVb+wIyTPrI0TjLJ68hmnVRrCPRytHSYZW0qilShnnx39OY98pDmhnpUR8zlUME89JMHUZgV5wM7HaNK9Gr9ueMMvAXd/2TmLmGTp+i37r6oinMJPwX5fvAi7ye3SRJIXrJo X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230028)(4636009)(396003)(376002)(136003)(39860400002)(346002)(451199021)(40470700004)(36840700001)(46966006)(6666004)(478600001)(40460700003)(83380400001)(47076005)(2616005)(336012)(426003)(36860700001)(40480700001)(86362001)(55016003)(356005)(7636003)(82740400003)(26005)(36756003)(1076003)(16526019)(186003)(6286002)(41300700001)(7696005)(8676002)(8936002)(316002)(82310400005)(5660300002)(2876002)(2906002)(54906003)(450100002)(70206006)(70586007)(6916009)(4326008); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2023 12:02:35.2608 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 699a6e7d-99e6-4e98-45fe-08db5c4ec326 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT060.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB7549 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Erez Ferber While doing process fork() the operating system remaps all the parent process's memory to the address space of the child process and activates the Copy-on-Write mechanics - it duplicates physical pages once memory writing happens in the child process. Sometimes memory duplication is not allowed - for example, if the page contains hardware queue descriptors. To handle similar issues the rdma-core library should be prepared for forking. The ibv_fork_init() prepares the library to track all the related memory and prevent it from forking using madvise() system API. This approach allows fork, but not all the memory is forked to the child process and, application should care not to touch pages where the parent application allocated the rdma-core objects. The newer kernels propose an option of copy-on-fork for DMA pages and tracking all the memory and disabling it for the forking is no longer needed. The new API routine ibv_is_fork_initialized() should be involved to decide if library initialization for forking is required. Fixes: 0e83b8e536 ("net/mlx5: move rdma-core calls to separate file") Cc: stable@dpdk.org Signed-off-by: Erez Ferber Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/linux/meson.build | 2 ++ drivers/common/mlx5/linux/mlx5_glue.c | 4 ++++ 2 files changed, 6 insertions(+) diff --git a/drivers/common/mlx5/linux/meson.build b/drivers/common/mlx5/linux/meson.build index 96a6c6c9be..c1fcb36fdd 100644 --- a/drivers/common/mlx5/linux/meson.build +++ b/drivers/common/mlx5/linux/meson.build @@ -219,6 +219,8 @@ has_sym_args = [ 'ibv_import_device' ], [ 'HAVE_MLX5DV_DR_ACTION_CREATE_DEST_ROOT_TABLE', 'infiniband/mlx5dv.h', 'mlx5dv_dr_action_create_dest_root_table' ], + [ 'HAVE_IBV_FORK_UNNEEDED', 'infiniband/verbs.h', + 'ibv_is_fork_initialized'], ] if libmtcr_ul_found has_sym_args += [ diff --git a/drivers/common/mlx5/linux/mlx5_glue.c b/drivers/common/mlx5/linux/mlx5_glue.c index 702eb36b62..88b99fe029 100644 --- a/drivers/common/mlx5/linux/mlx5_glue.c +++ b/drivers/common/mlx5/linux/mlx5_glue.c @@ -19,6 +19,10 @@ static int mlx5_glue_fork_init(void) { +#ifdef HAVE_IBV_FORK_UNNEEDED + if (ibv_is_fork_initialized() == IBV_FORK_UNNEEDED) + return 0; /* ibv_fork_init() not needed */ +#endif return ibv_fork_init(); }