From patchwork Tue Jan 4 01:47:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elena Agostini X-Patchwork-Id: 105592 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9E2CFA034D; Mon, 3 Jan 2022 18:37:41 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 048CA410E0; Mon, 3 Jan 2022 18:37:37 +0100 (CET) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2058.outbound.protection.outlook.com [40.107.92.58]) by mails.dpdk.org (Postfix) with ESMTP id B5AAE40042 for ; Mon, 3 Jan 2022 18:37:34 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hTtuigp6vk2xxcCHNzr5FT98+h+czD6Szt/Uhv/9ZiI4qWuVs+VF/1mCB8UlsooV9DUgy4zDgA6yQUkDZm7ScaoMJh4bfZ37vsDqp5zpsYT79O47Vy+YMxqKwDiERlrvJ2JR4BA9fvKAt/ZO70YYb7kee1pwjUO2Yma/Sw9szzl00XCZ9Op4gybBioXt9/E3nlL/PsLJZazEl9eQ4edXDSCr359pp6Ad7tEyu0K+cJItaexJ0DRWg+39eCf9WQpvY02qyFGBAIskriju6ALR22VrYMltnc7FB0M3sEHtY2VW5VZS7T4cVgbME7zTNKkVSm0qeg0X97N/GYTlnRO+Dg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=iMKfkRTPISRq5l34il6PHGJq6baX1p9oxit7OOtZeog=; b=GqexSQpaHIbDxY/2YXs0K0UN/K27XLG8cTvApXgUexRHM0uV2Xw8wqduIl3O/a0166t6BzrFIWxOAUH8S47JhTwJqUTwJcZXYD5tbitPNSyBQGBuVcf0Gy+Qf7YlhXBxD9puntv4PJMANHBTohAXhBT4phKEOnvqVQfQYKY2OTEG5kHING/ZBgyxIJqY1I2sq7w8UaTarFaqYp3jvhLQtDjg3zZHIE2juxsPwg2AseicNvrdhGw3E8GC4eyYmzvTp23lneGJlKv8Zme6kf7wTXf0uFQ0CgDiEOk5osEPnuVCgR02HiqSECceF0V17z6cfGeUo/wriBmn8rYf/NBh3w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iMKfkRTPISRq5l34il6PHGJq6baX1p9oxit7OOtZeog=; b=X+tYWIzDjJmJL1gc8bWo8D2Qy3kDB3iCloal37uw9ygUK+Jn0T2/OGlDiijoWlkxLmpYsT24UuDgCZNgcF5OgJgIMfGtu6OACbjOvaurOaOZWCe1HDK6OWoUk/FCV7OjIRXO+FvHzg8j1OIhGYFn+IpJ4ftKgWPZ2B3fW/smk/3k9FtfN/BGkO/3JKMIyQOgUXkkiBriHyvyrHubDmfz8vVyt5JBSZw/LiyuvWud/PMgdkaFHhpI035eYaMOsa1dpYDuunKqhJtua8TQEcago2QkkNRYY09LWqhFBMu3umxw0ngs2BXxhTAsGaRHgb+rxTmTATg3zHStJYY8YkUxQA== Received: from MW4PR04CA0149.namprd04.prod.outlook.com (2603:10b6:303:84::34) by DM6PR12MB3674.namprd12.prod.outlook.com (2603:10b6:5:1c4::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4844.15; Mon, 3 Jan 2022 17:37:33 +0000 Received: from CO1NAM11FT065.eop-nam11.prod.protection.outlook.com (2603:10b6:303:84:cafe::82) by MW4PR04CA0149.outlook.office365.com (2603:10b6:303:84::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4844.15 via Frontend Transport; Mon, 3 Jan 2022 17:37:33 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.236) by CO1NAM11FT065.mail.protection.outlook.com (10.13.174.62) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4844.14 via Frontend Transport; Mon, 3 Jan 2022 17:37:32 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 3 Jan 2022 17:37:32 +0000 Received: from nvidia.com (172.20.187.6) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.986.9; Mon, 3 Jan 2022 09:37:31 -0800 From: To: CC: Elena Agostini Subject: [PATCH v1 1/3] gpudev: mem alloc aligned memory Date: Tue, 4 Jan 2022 01:47:19 +0000 Message-ID: <20220104014721.1799-2-eagostini@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220104014721.1799-1-eagostini@nvidia.com> References: <20220104014721.1799-1-eagostini@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9e1bdfed-6d66-4980-7dc8-08d9cedfb911 X-MS-TrafficTypeDiagnostic: DM6PR12MB3674:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: a2lRiFi3mv7VG0OEzC+Ko1UArYdbN9uYFugkB8g/Mve6onCGBDB2oTE4pg9QT+Xqv9yxs9YXNO7DdnJiStBs29ZwQPHFg8KGz41cnyIwdeGt/nbg+APXzXS0+BRS7V96p/L/UQsoYIOJdc8XAEesxXhU4aMGUAdbfgXBpdP6WIikz8kU7uesv/swHDWz2xg/Cq8tULlmZAb2d9xvNKmBpGvp2pauqT9kT2XWAgttxtSgGlSaqNLpgT5vwHVSD2ynd5W0yOYnncxAqKCS5Tu4rL04Gi4CBv9R5kt3XUXEoIYQgxYf/0EpFNb30vafGQ/TaTnQfZ+/jlrsXmvE4SmtGerMFBy3iBuKEZbJ8erjXLNw+nXrlUvKaZbCuf9W62x7tGPntNG+2oofo5ZEGeB4enqYSM0xL6kaboSDiTyXwNvYk+2OjwRF0FB3izEKoVq7wqFjWj2TKucmWin07Q+Xux9VXW1QVPT0gDaj1MLh39nqKTU85CZ9Jz64MUaI7RDc+lNd/EUEvxVv+RRVVW7acKbNSHYkWUisB4KLG2V7LUeXbSPRO5guEWVLEVt8oMlVrcc9PMKLSQ6HVlWJn51nMTjCAmJjevtay+6WN6d2JuM6QKNKOssqGiFZdoq3rhnEZYZZ7jOWH852mzvDp3Sfs3/VJ5YspZPUe/02N5UfPhWl2hl/PedYZYPyQ6ezdVW9eGDpZinJVc0ZuowfBN4W9dc3lCT1VXZ/9irAXBIqO2qk+eRvVCwxjYfhRtt3lVBoJbmPJZh1/8EPu7Fnu3ud86vu1gOYyIgNLA2IovlJ1bI= X-Forefront-Antispam-Report: CIP:12.22.5.236; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(4636009)(46966006)(40470700002)(36840700001)(356005)(86362001)(2616005)(6286002)(40460700001)(508600001)(7696005)(316002)(336012)(16526019)(36756003)(26005)(6666004)(186003)(36860700001)(5660300002)(8676002)(70206006)(2906002)(70586007)(2876002)(83380400001)(426003)(107886003)(47076005)(4326008)(55016003)(81166007)(6916009)(8936002)(1076003)(82310400004)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jan 2022 17:37:32.6502 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9e1bdfed-6d66-4980-7dc8-08d9cedfb911 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.236]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT065.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB3674 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Elena Agostini Similarly to rte_malloc, rte_gpu_mem_alloc accept as input the memory alignment size. GPU driver should return GPU memory address aligned with the input value. Signed-off-by: Elena Agostini --- lib/gpudev/gpudev.c | 10 ++++++++-- lib/gpudev/gpudev_driver.h | 2 +- lib/gpudev/rte_gpudev.h | 10 +++++++--- 3 files changed, 16 insertions(+), 6 deletions(-) diff --git a/lib/gpudev/gpudev.c b/lib/gpudev/gpudev.c index 9ae36dbae9..dc8c3baefa 100644 --- a/lib/gpudev/gpudev.c +++ b/lib/gpudev/gpudev.c @@ -527,7 +527,7 @@ rte_gpu_info_get(int16_t dev_id, struct rte_gpu_info *info) } void * -rte_gpu_mem_alloc(int16_t dev_id, size_t size) +rte_gpu_mem_alloc(int16_t dev_id, size_t size, unsigned int align) { struct rte_gpu *dev; void *ptr; @@ -549,7 +549,13 @@ rte_gpu_mem_alloc(int16_t dev_id, size_t size) if (size == 0) /* dry-run */ return NULL; - ret = dev->ops.mem_alloc(dev, size, &ptr); + if (align && !rte_is_power_of_2(align)) { + GPU_LOG(ERR, "requested alignment is not a power of two %u", align); + rte_errno = EINVAL; + return NULL; + } + + ret = dev->ops.mem_alloc(dev, size, &ptr, align); switch (ret) { case 0: diff --git a/lib/gpudev/gpudev_driver.h b/lib/gpudev/gpudev_driver.h index cb7b101f2f..d06f465194 100644 --- a/lib/gpudev/gpudev_driver.h +++ b/lib/gpudev/gpudev_driver.h @@ -27,7 +27,7 @@ enum rte_gpu_state { struct rte_gpu; typedef int (rte_gpu_close_t)(struct rte_gpu *dev); typedef int (rte_gpu_info_get_t)(struct rte_gpu *dev, struct rte_gpu_info *info); -typedef int (rte_gpu_mem_alloc_t)(struct rte_gpu *dev, size_t size, void **ptr); +typedef int (rte_gpu_mem_alloc_t)(struct rte_gpu *dev, size_t size, void **ptr, unsigned int align); typedef int (rte_gpu_mem_free_t)(struct rte_gpu *dev, void *ptr); typedef int (rte_gpu_mem_register_t)(struct rte_gpu *dev, size_t size, void *ptr); typedef int (rte_gpu_mem_unregister_t)(struct rte_gpu *dev, void *ptr); diff --git a/lib/gpudev/rte_gpudev.h b/lib/gpudev/rte_gpudev.h index fa3f3aad4f..9e2e2c5dce 100644 --- a/lib/gpudev/rte_gpudev.h +++ b/lib/gpudev/rte_gpudev.h @@ -364,18 +364,22 @@ int rte_gpu_info_get(int16_t dev_id, struct rte_gpu_info *info); * @param size * Number of bytes to allocate. * Requesting 0 will do nothing. - * + * @param align + * If 0, the return is a pointer that is suitably aligned for any kind of + * variable (in the same manner as malloc()). + * Otherwise, the return is a pointer that is a multiple of *align*. In + * this case, it must obviously be a power of two. * @return * A pointer to the allocated memory, otherwise NULL and rte_errno is set: * - ENODEV if invalid dev_id - * - EINVAL if reserved flags + * - EINVAL if align is not a power of two * - ENOTSUP if operation not supported by the driver * - E2BIG if size is higher than limit * - ENOMEM if out of space * - EPERM if driver error */ __rte_experimental -void *rte_gpu_mem_alloc(int16_t dev_id, size_t size) +void *rte_gpu_mem_alloc(int16_t dev_id, size_t size, unsigned int align) __rte_alloc_size(2); /** From patchwork Tue Jan 4 01:47:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elena Agostini X-Patchwork-Id: 105593 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B0211A034D; Mon, 3 Jan 2022 18:37:46 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E4E8C41148; Mon, 3 Jan 2022 18:37:37 +0100 (CET) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2074.outbound.protection.outlook.com [40.107.243.74]) by mails.dpdk.org (Postfix) with ESMTP id 076D14003C for ; Mon, 3 Jan 2022 18:37:34 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=e6bZT/2hM64ovZldMZc6vN37qle723CH5GOtGqdpYnbxTT4Dy1JaJEn82RSouVibJ17e7qPnDXWTwVYPY3gu7eOcHEe7UsGmW8hLeTs5jMPSDCHzX9kei8MdymidYu0zbWi0UjiqK2WMbhdOvfzzIf2z/z3/wYAirx6ps2VTVdf9Rw0v+YM3Lcc2lSKy2wa5y98CpwTXECkXqPVxiqaoWdGZIIB/eRhGNn8QtvtbBiuGO+lyOsGOcfGCecifLVcLTjpMHGpMaR7AGlA1/evxTVUHYwuDiSknYAZfEGC0i1ga3kYPZiAlcvhQsadc8quhy0TAPT0KE0jE45ZEo7o+3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OkdTox4plqxUBYxrxXqv2iVwrfQvqv/v85nXlPQy2mQ=; b=bBiKf6DA6/8nNsRPB1RCGkITDX4B1bWKx4W2zFsY43PbFC4aLVQ9zm7WtcrFbBZCj3/NX4IZxKiHIJmYvNbXjrs4YgNieO2dP0m/4nLINfn4YY0ZXeZ4AwRyubbLG++fUKSnRoUOXTfEc9yqQmW3+vsVPxj2gVXqSI0S2hGV2eaedJgA+5+3uEbcrPbeZ0vYhGsqlbQNj9O+upJeBN2XkC2U5IFDjrA6u7ShXL4cW5iiNBswocV/eTM11kcEGQwtK4GBbXBn60hRUmhdJR8BOuBiOROgB5twYWaopY/apD3i6NEKeJedRfuHC12yWt8vRdbA07zsK4ifexIRvlyY9w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OkdTox4plqxUBYxrxXqv2iVwrfQvqv/v85nXlPQy2mQ=; b=cMlXT3YqO4e+0ncMKKVFxfcABgr5TEpws8CGe3hM8XKipOwRSFi7BaEfCvcHzkRyfpc6YIQO1X+kWH4NeFpcC6WxNUYSUKsxWTskkHHltUZDm9kaMxaOsthtwsvBEKWenw0ouCptkkn5B+XY5ns3gDBTFJ4G/q/6oVhuUoM4uTjoOWtthZJrCt6PoHFZz2eB4n3JkeWHuva0Fc3fvBqG6F0FH6ltZKBHp6cYD/2mz+clWTu59HCaIMY0zUcP7CyR6piN96Y7XDX5ZL58FcIB7nupTWa2c6UXW+MXZ+XA4qKUkwb2dpuFnAas20vY3y6+ZWnTbT2C/E9XkOpcT5blHw== Received: from MW4PR04CA0147.namprd04.prod.outlook.com (2603:10b6:303:84::32) by DM6PR12MB4281.namprd12.prod.outlook.com (2603:10b6:5:21e::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4844.14; Mon, 3 Jan 2022 17:37:33 +0000 Received: from CO1NAM11FT065.eop-nam11.prod.protection.outlook.com (2603:10b6:303:84:cafe::78) by MW4PR04CA0147.outlook.office365.com (2603:10b6:303:84::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4844.15 via Frontend Transport; Mon, 3 Jan 2022 17:37:33 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.236) by CO1NAM11FT065.mail.protection.outlook.com (10.13.174.62) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4844.14 via Frontend Transport; Mon, 3 Jan 2022 17:37:33 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 3 Jan 2022 17:37:32 +0000 Received: from nvidia.com (172.20.187.6) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.986.9; Mon, 3 Jan 2022 09:37:31 -0800 From: To: CC: Elena Agostini Subject: [PATCH v1 2/3] app/test-gpudev: test aligned memory allocation Date: Tue, 4 Jan 2022 01:47:20 +0000 Message-ID: <20220104014721.1799-3-eagostini@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220104014721.1799-1-eagostini@nvidia.com> References: <20220104014721.1799-1-eagostini@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 704c61ac-9a53-4920-ed31-08d9cedfb983 X-MS-TrafficTypeDiagnostic: DM6PR12MB4281:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:510; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lZmUpOijhheFYd6p+KD6u82nAbjxRVd9Q8k9Vqz9bJZ3R4o5H3kyNKzMnHkLjMM6UCN2uWYYUT2WJ1j29IilL1+pe0v3EsmoAXFcYnhWY87FuC8vvZI1L8ChUdrH3A+0AQZ5zouswNJ/K+jZlNazwrfR8vo1YaOpJyiwKvkCJxAC8vFTlhUcYhh0AbVJXVgem3S1Pm+3y0B8OzhWHWyuIc54stADOU/SohvE1ZNFedglKVnUbh5XpasaOblyEx8NJg0QtAJFKdRHdhhNtZGAeWlUFlpq1alVC0A61hwVvWcoQZVvz83ddXs/rUDJg8Y8kWGDil0/SDBpMsumOONjBjgv4SNvA9S482oeHZq06vcBwfNAdXz/QE+rn6wZTe3cpEUZ6plOEP+QV8/DWgx1TRjERtyZTZMb/UGs1FfFpwV6FbVEL2agg2f5sJtGyi/vGV/Re0WiIZSj43eIWf+udk914HvrpK+ROlw3pSpbQL8JxdTX50KsaHZz58Pj9OEta/RqC9VHziXgu5y2DGWjWrK31saJKvaVEQMM9W4zt1HifFxKHnu15vhPMfAs7fOU69/mfiOYrm2KHkmZFauEq1L1w+1g43sTJi9/wb86J8gTgDbTNWvDZzOOomm4QrFe8KuL9B7+TC6Tz4wYtwBJ21mbtQ09t5MNHYk89g3zVJnqY5aC5BhyD/nMrctzlHKq7fH4tRO/q7/BT94mfJuy/WG43g30R5f8MyeedEH0thVgKMd9X8wrWACDj8BhJbR18tP4iWYQuChqb51CJqCwUAEcYuwYhOMCN/wBQuQPIgo= X-Forefront-Antispam-Report: CIP:12.22.5.236; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(4636009)(36840700001)(40470700002)(46966006)(6666004)(8936002)(426003)(82310400004)(356005)(8676002)(86362001)(336012)(6286002)(40460700001)(36756003)(107886003)(2616005)(83380400001)(55016003)(7696005)(5660300002)(16526019)(508600001)(36860700001)(26005)(2876002)(186003)(47076005)(4326008)(70206006)(70586007)(81166007)(2906002)(316002)(1076003)(6916009)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jan 2022 17:37:33.3845 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 704c61ac-9a53-4920-ed31-08d9cedfb983 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.236]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT065.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4281 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Elena Agostini Update gpudev app to test GPU memory aligned allocation. Signed-off-by: Elena Agostini --- app/test-gpudev/main.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/app/test-gpudev/main.c b/app/test-gpudev/main.c index 5c1aa3d52f..f36f46cbca 100644 --- a/app/test-gpudev/main.c +++ b/app/test-gpudev/main.c @@ -69,11 +69,12 @@ alloc_gpu_memory(uint16_t gpu_id) void *ptr_2 = NULL; size_t buf_bytes = 1024; int ret; + unsigned align = 4096; printf("\n=======> TEST: Allocate GPU memory\n\n"); - /* Alloc memory on GPU 0 */ - ptr_1 = rte_gpu_mem_alloc(gpu_id, buf_bytes); + /* Alloc memory on GPU 0 without any specific alignment */ + ptr_1 = rte_gpu_mem_alloc(gpu_id, buf_bytes, 0); if (ptr_1 == NULL) { fprintf(stderr, "rte_gpu_mem_alloc GPU memory returned error\n"); goto error; @@ -81,7 +82,8 @@ alloc_gpu_memory(uint16_t gpu_id) printf("GPU memory allocated at 0x%p size is %zd bytes\n", ptr_1, buf_bytes); - ptr_2 = rte_gpu_mem_alloc(gpu_id, buf_bytes); + /* Alloc memory on GPU 0 with 4kB alignment */ + ptr_2 = rte_gpu_mem_alloc(gpu_id, buf_bytes, align); if (ptr_2 == NULL) { fprintf(stderr, "rte_gpu_mem_alloc GPU memory returned error\n"); goto error; @@ -89,6 +91,11 @@ alloc_gpu_memory(uint16_t gpu_id) printf("GPU memory allocated at 0x%p size is %zd bytes\n", ptr_2, buf_bytes); + if (((uintptr_t)ptr_2) % align) { + fprintf(stderr, "Memory address 0x%p is not aligned to %u\n", ptr_2, align); + goto error; + } + ret = rte_gpu_mem_free(gpu_id, (uint8_t *)(ptr_1)+0x700); if (ret < 0) { printf("GPU memory 0x%p NOT freed: GPU driver didn't find this memory address internally.\n", From patchwork Tue Jan 4 01:47:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elena Agostini X-Patchwork-Id: 105594 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 67020A034D; Mon, 3 Jan 2022 18:37:51 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DF1E041150; Mon, 3 Jan 2022 18:37:38 +0100 (CET) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2040.outbound.protection.outlook.com [40.107.93.40]) by mails.dpdk.org (Postfix) with ESMTP id 9F5684003C for ; Mon, 3 Jan 2022 18:37:35 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MfOp7xk3gJxyu3207hGF5DG3qfNVMfJHo1QMGq1MukLQhGdoX4WKYgi1hqb7tXFRMMhD2br7UaEbvNblDlCEgselccj8ALFviaCUnbpxAZ2kUBjvJSYV+/w6/ZWnejjsPOIgJK+7GSwSOt+efaL/Iq3sZ7nRTChN5Is9+dGMSvGstqj3XA7PjzbirLm3fSNegmb+DSq5+VSwD7uwxhpIGG0rt0LZ68PODswmtUlxxEd21zc0ndcr1PiqdSsKNB2HtnXGWjMdZ5CES2CJkSIcuc06SQACDK6tAkavrj6aVxLTk7rbt8LhFNTpV0jo+gFdq09OvdAMpkKXpAk0ZfKuiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=I3/MB5AUaWPQOcM+apTD1UYyUEeMQqye6OecUY1L60k=; b=aN2kV/qiRn8meEe7cQ2h8DwnHbRrgPNAjd+PRiibO2XV/9Jnl6nLLMEOHZIVQyznW6g8NsHRSlUwf6YkF/jYDJD9nx7gBWAAvwylqEAOndjHSFIfnwuBMWTraWbpM3AXd0LuYOAuDmgYOve2FgZkIlLDD6bMz4lT7VYFHTc4MZIYEyh0h4j0v3Xr4PwurSjMht2eiTgCR6Z2wyPzxiffayGHoBxERyE8YVMLuxX6nFWM+AFwX7+RbmXgtaN1rK93Mo865etBk5qZdvNtAoUooCq+IqwDGApTSCObzLx09I8JqltvMTztDBy9CMCmICXaZgfD583EbmbnSxsUvB1Ilw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=I3/MB5AUaWPQOcM+apTD1UYyUEeMQqye6OecUY1L60k=; b=qIvgeNNz78BsJubklCd++pJJCQEBWie5UKsdAlW/ysrF5K0g/4wr+F7eIH8a+QvjNzpm/wBfxMaVUguawjFZz/Gz2JQWcN0EPmOPB4Aw/m6fXqVFYusowWLRP6Mijggsf64Osj1P/OXIhKX+8fPyJo1sOkRi5oiTOwBXl360DeAM3xDJPQPW+sS3maTr1pbsoBfPf7MMEZly9Fbo5bmtkBVVi5tOfTiKGvY7Vo4SIsNxzrZfnI5jADpdFEbeyjvptqwxEgRGReLEBH1WRqhvgwOPuMrFEXP9JGKZ/fwsbKIBzRA0NUXEMCns3INYjT/NPsQ2Q8284StJWY1IXK/YZQ== Received: from MW4P223CA0006.NAMP223.PROD.OUTLOOK.COM (2603:10b6:303:80::11) by CH2PR12MB4264.namprd12.prod.outlook.com (2603:10b6:610:a4::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4801.19; Mon, 3 Jan 2022 17:37:33 +0000 Received: from CO1NAM11FT019.eop-nam11.prod.protection.outlook.com (2603:10b6:303:80:cafe::d1) by MW4P223CA0006.outlook.office365.com (2603:10b6:303:80::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4844.13 via Frontend Transport; Mon, 3 Jan 2022 17:37:33 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.235) by CO1NAM11FT019.mail.protection.outlook.com (10.13.175.57) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4844.14 via Frontend Transport; Mon, 3 Jan 2022 17:37:33 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 3 Jan 2022 17:37:32 +0000 Received: from nvidia.com (172.20.187.6) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.986.9; Mon, 3 Jan 2022 09:37:32 -0800 From: To: CC: Elena Agostini Subject: [PATCH v1 3/3] gpu/cuda: mem alloc aligned memory Date: Tue, 4 Jan 2022 01:47:21 +0000 Message-ID: <20220104014721.1799-4-eagostini@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220104014721.1799-1-eagostini@nvidia.com> References: <20220104014721.1799-1-eagostini@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0e42c60e-ef65-47e0-0a28-08d9cedfb97d X-MS-TrafficTypeDiagnostic: CH2PR12MB4264:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:428; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: LVOmw97Fe0s8i8koqVHo67GLBUI4JxbUwDlyxdiMzS+UjBUNog96aWPd6vxZTeoMw9CXG/6yecicyVYN7gr/KWNDIkTRGiaN5UBruo4LLozmkxd3Fi5mmu7+jWc9aK31xSfjXkq6r33D+I2Sv9H2XU2Nf4abVjv3EPL4pvrlBplX3h5Pjb/TxfeEsrfQuuCYH03dJW8OxJiOI6m5pzCsuU2y7O4v6zSB3NxbTb8NAv4Dw7caXmkg5aTL+9rwGI7iSPxaO10pECqs9yZNFPUq+1/GU6dME5TwXDxfg/8SKinmQk1Ot9lvy06JUwHxI6DAPb8W6jADWXDyVConpduKP+6WF9nwmUpMBtyjcfZYS7kKkjbPSiD79QOyepdKlOfQTmLo6pMBRUPu9oN6vP71+ArD2J6KgmiZAaOSZ93uspifExjdZeY491Yehfee1zNM2Us6OQWHtjahnpc033f4fW51cqFWb9ImeTy/xctjw6Ovy0wHdVh95Yv7VMiDa2DaJW2moO7Ndun6ZQCnPIkXQv4wUz9jkYUcgLZzS6UmATVhJ+uj/f7/OVPHrFqlMSEbaC+PKXekMcTUIUYIFrr8DiWVNgk8ffHudFOYZKAaWAYFqZovpfM5xD7fumoYjHbTV213ePgmfFo59VX148P8galWKtHp5XDzBeRF4fKKQGAeJls4zoVEfvdn+ebkXPcxp5o+OAFCtFMVsa/YIskBb+wR5eGH7UHAvPTH3pKTWEyWpSJLNYn+b7pW4jqNbIjk9TW8FQNBet9QdmeQqYzmbSq6vmoF4ytA8V++kzBq1TM= X-Forefront-Antispam-Report: CIP:12.22.5.235; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(4636009)(40470700002)(36840700001)(46966006)(47076005)(8676002)(186003)(36756003)(83380400001)(6666004)(36860700001)(81166007)(86362001)(26005)(7696005)(336012)(2876002)(70206006)(1076003)(5660300002)(4326008)(40460700001)(356005)(107886003)(8936002)(6286002)(16526019)(2616005)(55016003)(70586007)(82310400004)(2906002)(508600001)(426003)(316002)(6916009)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jan 2022 17:37:33.3550 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0e42c60e-ef65-47e0-0a28-08d9cedfb97d X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.235]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT019.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4264 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Elena Agostini Implement aligned GPU memory allocation in GPU CUDA driver. Signed-off-by: Elena Agostini --- drivers/gpu/cuda/cuda.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/drivers/gpu/cuda/cuda.c b/drivers/gpu/cuda/cuda.c index 882df08e56..4ad3f5fc90 100644 --- a/drivers/gpu/cuda/cuda.c +++ b/drivers/gpu/cuda/cuda.c @@ -139,8 +139,10 @@ typedef uintptr_t cuda_ptr_key; /* Single entry of the memory list */ struct mem_entry { CUdeviceptr ptr_d; + CUdeviceptr ptr_orig_d; void *ptr_h; size_t size; + size_t size_orig; struct rte_gpu *dev; CUcontext ctx; cuda_ptr_key pkey; @@ -569,7 +571,7 @@ cuda_dev_info_get(struct rte_gpu *dev, struct rte_gpu_info *info) */ static int -cuda_mem_alloc(struct rte_gpu *dev, size_t size, void **ptr) +cuda_mem_alloc(struct rte_gpu *dev, size_t size, void **ptr, unsigned int align) { CUresult res; const char *err_string; @@ -610,8 +612,10 @@ cuda_mem_alloc(struct rte_gpu *dev, size_t size, void **ptr) /* Allocate memory */ mem_alloc_list_tail->size = size; - res = pfn_cuMemAlloc(&(mem_alloc_list_tail->ptr_d), - mem_alloc_list_tail->size); + mem_alloc_list_tail->size_orig = size + align; + + res = pfn_cuMemAlloc(&(mem_alloc_list_tail->ptr_orig_d), + mem_alloc_list_tail->size_orig); if (res != 0) { pfn_cuGetErrorString(res, &(err_string)); rte_cuda_log(ERR, "cuCtxSetCurrent current failed with %s", @@ -620,6 +624,13 @@ cuda_mem_alloc(struct rte_gpu *dev, size_t size, void **ptr) return -rte_errno; } + + /* Align memory address */ + mem_alloc_list_tail->ptr_d = mem_alloc_list_tail->ptr_orig_d; + if (align && ((uintptr_t)mem_alloc_list_tail->ptr_d) % align) + mem_alloc_list_tail->ptr_d += (align - + (((uintptr_t)mem_alloc_list_tail->ptr_d) % align)); + /* GPUDirect RDMA attribute required */ res = pfn_cuPointerSetAttribute(&flag, CU_POINTER_ATTRIBUTE_SYNC_MEMOPS, @@ -634,7 +645,6 @@ cuda_mem_alloc(struct rte_gpu *dev, size_t size, void **ptr) mem_alloc_list_tail->pkey = get_hash_from_ptr((void *)mem_alloc_list_tail->ptr_d); mem_alloc_list_tail->ptr_h = NULL; - mem_alloc_list_tail->size = size; mem_alloc_list_tail->dev = dev; mem_alloc_list_tail->ctx = (CUcontext)((uintptr_t)dev->mpshared->info.context); mem_alloc_list_tail->mtype = GPU_MEM; @@ -761,6 +771,7 @@ cuda_mem_register(struct rte_gpu *dev, size_t size, void *ptr) mem_alloc_list_tail->dev = dev; mem_alloc_list_tail->ctx = (CUcontext)((uintptr_t)dev->mpshared->info.context); mem_alloc_list_tail->mtype = CPU_REGISTERED; + mem_alloc_list_tail->ptr_orig_d = mem_alloc_list_tail->ptr_d; /* Restore original ctx as current ctx */ res = pfn_cuCtxSetCurrent(current_ctx); @@ -796,7 +807,7 @@ cuda_mem_free(struct rte_gpu *dev, void *ptr) } if (mem_item->mtype == GPU_MEM) { - res = pfn_cuMemFree(mem_item->ptr_d); + res = pfn_cuMemFree(mem_item->ptr_orig_d); if (res != 0) { pfn_cuGetErrorString(res, &(err_string)); rte_cuda_log(ERR, "cuMemFree current failed with %s",