From patchwork Fri Apr 29 13:52:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elena Agostini X-Patchwork-Id: 110515 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 041E7A04FF; Fri, 29 Apr 2022 15:53:26 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AA4E5415D7; Fri, 29 Apr 2022 15:53:26 +0200 (CEST) Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1anam02on2087.outbound.protection.outlook.com [40.107.96.87]) by mails.dpdk.org (Postfix) with ESMTP id 4CA69410E3 for ; Fri, 29 Apr 2022 15:53:25 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=S/vog6uoD5qi1u/kXCdAobfgsrFigNw1fUnzTk1cg1hN25fAeQ9nGPnoRPYn+4iILexEeZ3RX4z4FCNe7xZe5IYLdvd8tMtyVYaqyFxLAiUJDaY8eT9X6rOc+ejeyQVAzVd6FCUM8IK/CqHQ8SizR8MNnSAFPqqgTzWGO+yoO+8lbZ6x4/j040gM6UUzSkri7CjXDtll5C4LwtpGk7p6rEzwUVRtl9H4kYf6/X8QDmookl/wv2kXSmAOJ4zoyOKD12vIKMXZ9s91mOmjLtkrnhezty7nsZQUvguV1I0GQaSLd22GQgVN2+IkVIzat6OK+9nGHcREKw4+hEyT9MOh4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=fUvB1QL95JMy9dv5B2kHUqtsKMS11cJoYrFRLNULOmE=; b=MZENTI2rZ3nuKqzWARfrcLca31mCRlW0d6EXTf+QDXEM+3edku66dQlvEwKCjgrdRPRV1+obMdklF7aCrez+YY/yxMgh/fhyJ/ll0Bcfi0K/t7VoHJA3NhOeGrB6ELaHkfD48sVk47RxkTlL/WFUMM1VzdxIQULLiAtr2Sb8l62qVqaHGK9I2dytBhTAu7+mxjjth5MU1Iglgy6u6sajVo2fZzVBmNtu3jJZAGcq7ygVDRTbP7MR6dw3pqXnNqFvUKNcnh8KqbQltmuJPUUrDPm128OXs/rhxaGPHuQ97uuZJS0Q8o3SQBvZE2FkzNaG4oIuwa950vkmh2hRMdOtwA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.238) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fUvB1QL95JMy9dv5B2kHUqtsKMS11cJoYrFRLNULOmE=; b=G/gRPR+pF0hkuVdw1a6JJTa7sVZVfha75x4IcDOS5oobKzcbSauNgr48xTCayyxz0LOgDrkRPhma+OPeyDLaM9Y1uxbL62P/HDUACAMQQ9LvlgCr2Mv/nVBSZsrKbTOnhu3uA+O7mAAYcHLElKkprxvhdJJSveXC/DIvrHG+FqyiqW83tYj11iuxjAvvw1lzbzC1XmJwLHSxCsKtJPFWVidXQdiYQuDgzzGMHMipavu7KgneveiKT/tnn+eiuyRVotAVfowq7bIAB0sdlDDMpGxeHrEkkvYlYHZ2dlTQR14b2ZdZy9e7mbwaPec1hiyzSsYFmY1L2FNn4enH3vN0Mg== Received: from DM6PR03CA0030.namprd03.prod.outlook.com (2603:10b6:5:40::43) by PH7PR12MB5927.namprd12.prod.outlook.com (2603:10b6:510:1da::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.20; Fri, 29 Apr 2022 13:53:22 +0000 Received: from DM6NAM11FT058.eop-nam11.prod.protection.outlook.com (2603:10b6:5:40:cafe::d) by DM6PR03CA0030.outlook.office365.com (2603:10b6:5:40::43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.14 via Frontend Transport; Fri, 29 Apr 2022 13:53:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.238) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.238 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.238; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.238) by DM6NAM11FT058.mail.protection.outlook.com (10.13.172.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5206.12 via Frontend Transport; Fri, 29 Apr 2022 13:53:21 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL105.nvidia.com (10.27.9.14) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Fri, 29 Apr 2022 13:53:20 +0000 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Fri, 29 Apr 2022 06:53:20 -0700 From: To: CC: Elena Agostini Subject: [PATCH] gpu/cuda: GPU_REGISTERED to distinguish GPU memory CPU mapped Date: Fri, 29 Apr 2022 13:52:51 +0000 Message-ID: <20220429135251.1373996-1-eagostini@nvidia.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d22b6c28-82e5-419c-07bb-08da29e79f7f X-MS-TrafficTypeDiagnostic: PH7PR12MB5927:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: QsMopyPRc6UZnjYnWaeGd2zhWwUm1kxxscICYyFXsP0JKqcsa0MKnOsvy9JcYkKuDlnn3Gf6TTHP9QMnk514yeBmuBBphXdgkayo9I/Jwges2xU9Wdlv245m7B6kG1Gh2EpWqnDbk30fpN9GSEkbmHAV7R4bkt1+gJowm+RnAtmNRSTNoALLS+wl4OyDnXSFlsJFmTHZkqy+awXKtRA7rMgcpHpueHyfH5e4aXY21IxrZJ4rc4WfA9XaVqh0l3omfUrmgwG5zBPVroEmUftcGuKOjevYpGVSqB+lEWLen/4QJbuPhB8J24K42ct7veY/ozrJzpGU23QSo74L8blWMO9Fc6kK/PAtJ91+b2uqFgTO2PnDDeYzLD9AtucbNyXiIqcuZE4wdcCXh1IRAKWImnhUPMut7hY0RtBFYT0jiy0/dFs0mZAC3d45bonLUyPRsDO4HghqlM/eytr0qe4XIhItGAxZNtO4Jy8CvPp/IyjUiHlGsKegGG0VXB4xfgceQB+p6jOwtcqnO2pFgYOQNyXCQmR9JpEReYZcBady31MflOEKV96vS0SjC2MJJ14ym2DPrMGYTV4tCwwAm+KgSvxG2XAF1sXWBE8Ot5wAX3lGtyGwprJgwRzox9+GKipC7weaJQR81SsP1lBBy23hXaDaCPW+IipYnyB2GnkiGoCvVJ9t8kQGyQpa28Y1x8If6uakAQ2EAU6ps2kIZhbpmg== X-Forefront-Antispam-Report: CIP:12.22.5.238; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(46966006)(36840700001)(40470700004)(40460700003)(83380400001)(2616005)(336012)(426003)(36860700001)(55016003)(81166007)(5660300002)(36756003)(316002)(4326008)(6916009)(70586007)(70206006)(8676002)(86362001)(47076005)(508600001)(8936002)(356005)(107886003)(6666004)(16526019)(186003)(1076003)(6286002)(82310400005)(2906002)(26005)(2876002)(7696005)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Apr 2022 13:53:21.5339 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d22b6c28-82e5-419c-07bb-08da29e79f7f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.238]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT058.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB5927 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Elena Agostini Enable GPU_REGISTERED flag in gpu/cuda driver in the memory list. If a GPU memory address CPU mapped is freed before being unmapped, CUDA driver unmaps it before freeing the memory. Signed-off-by: Elena Agostini --- drivers/gpu/cuda/cuda.c | 78 +++++++++++++++++++++++++---------------- 1 file changed, 47 insertions(+), 31 deletions(-) diff --git a/drivers/gpu/cuda/cuda.c b/drivers/gpu/cuda/cuda.c index 5b8476ac20..c6bf54c130 100644 --- a/drivers/gpu/cuda/cuda.c +++ b/drivers/gpu/cuda/cuda.c @@ -257,7 +257,7 @@ struct cuda_info { enum mem_type { GPU_MEM = 0, CPU_REGISTERED, - GPU_REGISTERED /* Not used yet */ + GPU_REGISTERED }; /* key associated to a memory address */ @@ -953,13 +953,14 @@ cuda_mem_cpu_map(struct rte_gpu *dev, __rte_unused size_t size, void *ptr_in, vo return -rte_errno; } + mem_item->mtype = GPU_REGISTERED; *ptr_out = mem_item->ptr_h; return 0; } static int -cuda_mem_free(struct rte_gpu *dev, void *ptr) +cuda_mem_unregister(struct rte_gpu *dev, void *ptr) { CUresult res; struct mem_entry *mem_item; @@ -978,11 +979,11 @@ cuda_mem_free(struct rte_gpu *dev, void *ptr) return -rte_errno; } - if (mem_item->mtype == GPU_MEM) { - res = pfn_cuMemFree(mem_item->ptr_orig_d); + if (mem_item->mtype == CPU_REGISTERED) { + res = pfn_cuMemHostUnregister(ptr); if (res != 0) { pfn_cuGetErrorString(res, &(err_string)); - rte_cuda_log(ERR, "cuMemFree current failed with %s", + rte_cuda_log(ERR, "cuMemHostUnregister current failed with %s", err_string); rte_errno = EPERM; return -rte_errno; @@ -993,74 +994,89 @@ cuda_mem_free(struct rte_gpu *dev, void *ptr) rte_cuda_log(ERR, "Memory type %d not supported", mem_item->mtype); - return -EPERM; + rte_errno = EPERM; + return -rte_errno; } static int -cuda_mem_unregister(struct rte_gpu *dev, void *ptr) +cuda_mem_cpu_unmap(struct rte_gpu *dev, void *ptr_in) { - CUresult res; struct mem_entry *mem_item; - const char *err_string; cuda_ptr_key hk; if (dev == NULL) return -ENODEV; - hk = get_hash_from_ptr((void *)ptr); + hk = get_hash_from_ptr((void *)ptr_in); mem_item = mem_list_find_item(hk); if (mem_item == NULL) { - rte_cuda_log(ERR, "Memory address 0x%p not found in driver memory", ptr); + rte_cuda_log(ERR, "Memory address 0x%p not found in driver memory.", ptr_in); rte_errno = EPERM; return -rte_errno; } - if (mem_item->mtype == CPU_REGISTERED) { - res = pfn_cuMemHostUnregister(ptr); - if (res != 0) { - pfn_cuGetErrorString(res, &(err_string)); - rte_cuda_log(ERR, "cuMemHostUnregister current failed with %s", - err_string); + if (mem_item->mtype == GPU_REGISTERED) { + if (gdrcopy_unpin(gdrc_h, mem_item->mh, (void *)mem_item->ptr_d, + mem_item->size)) { + rte_cuda_log(ERR, "Error unexposing GPU memory address 0x%p.", ptr_in); rte_errno = EPERM; return -rte_errno; } - return mem_list_del_item(hk); + mem_item->mtype = GPU_MEM; + } else { + rte_errno = EPERM; + return -rte_errno; } - rte_cuda_log(ERR, "Memory type %d not supported", mem_item->mtype); - - rte_errno = EPERM; - return -rte_errno; + return 0; } static int -cuda_mem_cpu_unmap(struct rte_gpu *dev, void *ptr_in) +cuda_mem_free(struct rte_gpu *dev, void *ptr) { + CUresult res; struct mem_entry *mem_item; + const char *err_string; cuda_ptr_key hk; if (dev == NULL) return -ENODEV; - hk = get_hash_from_ptr((void *)ptr_in); + hk = get_hash_from_ptr((void *)ptr); mem_item = mem_list_find_item(hk); if (mem_item == NULL) { - rte_cuda_log(ERR, "Memory address 0x%p not found in driver memory.", ptr_in); + rte_cuda_log(ERR, "Memory address 0x%p not found in driver memory", ptr); rte_errno = EPERM; return -rte_errno; } - if (gdrcopy_unpin(gdrc_h, mem_item->mh, (void *)mem_item->ptr_d, - mem_item->size)) { - rte_cuda_log(ERR, "Error unexposing GPU memory address 0x%p.", ptr_in); - rte_errno = EPERM; - return -rte_errno; + /* + * If a GPU memory area that's CPU mapped is being freed + * without calling cpu_unmap, force the unmapping. + */ + if (mem_item->mtype == GPU_REGISTERED) { + cuda_mem_cpu_unmap(dev, ptr); } - return 0; + if (mem_item->mtype == GPU_MEM) { + res = pfn_cuMemFree(mem_item->ptr_orig_d); + if (res != 0) { + pfn_cuGetErrorString(res, &(err_string)); + rte_cuda_log(ERR, "cuMemFree current failed with %s", + err_string); + rte_errno = EPERM; + return -rte_errno; + } + + return mem_list_del_item(hk); + } + + rte_cuda_log(ERR, "Memory type %d not supported", mem_item->mtype); + + return -EPERM; } static int