[v2] gpudev: pin GPU memory
Checks
Commit Message
From: Elena Agostini <eagostini@nvidia.com>
Enable the possibility to make a GPU memory area accessible from
the CPU.
GPU memory has to be allocated via rte_gpu_mem_alloc().
This patch allows the gpudev library to pin, through the GPU driver,
a chunk of GPU memory and to return a memory pointer usable
by the CPU to access the GPU memory area.
Signed-off-by: Elena Agostini <eagostini@nvidia.com>
---
lib/gpudev/gpudev.c | 47 +++++++++++++++++++++++++++++++++++
lib/gpudev/gpudev_driver.h | 6 +++++
lib/gpudev/rte_gpudev.h | 50 ++++++++++++++++++++++++++++++++++++++
lib/gpudev/version.map | 2 ++
4 files changed, 105 insertions(+)
Comments
04/01/2022 03:41, eagostini@nvidia.com:
> From: Elena Agostini <eagostini@nvidia.com>
>
> Enable the possibility to make a GPU memory area accessible from
> the CPU.
>
> GPU memory has to be allocated via rte_gpu_mem_alloc().
>
> This patch allows the gpudev library to pin, through the GPU driver,
> a chunk of GPU memory and to return a memory pointer usable
> by the CPU to access the GPU memory area.
>
> Signed-off-by: Elena Agostini <eagostini@nvidia.com>
[...]
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Pin a chunk of GPU memory to make it accessible from the CPU
You should define what means "pin" exactly.
Which properties should we expect?
> + * using the memory pointer returned by the function.
Which function should return the pointer?
rte_gpu_mem_pin is returning an int.
> + * GPU memory has to be allocated via rte_gpu_mem_alloc().
Why pinning is not done by rte_gpu_mem_alloc()?
Should it be a flag?
> + *
> + * @param dev_id
> + * Device ID requiring pinned memory.
> + * @param size
> + * Number of bytes to pin.
> + * Requesting 0 will do nothing.
> + * @param ptr
> + * Pointer to the GPU memory area to be pinned.
> + * NULL is a no-op accepted value.
> +
> + * @return
> + * A pointer to the pinned GPU memory usable by the CPU, otherwise NULL and rte_errno is set:
> + * - ENODEV if invalid dev_id
> + * - EINVAL if reserved flags
Which reserved flags?
> + * - ENOTSUP if operation not supported by the driver
> + * - E2BIG if size is higher than limit
> + * - ENOMEM if out of space
Is out of space relevant for pinning?
> + * - EPERM if driver error
> + */
> +__rte_experimental
> +int rte_gpu_mem_pin(int16_t dev_id, size_t size, void *ptr);
> 04/01/2022 03:41, eagostini@nvidia.com:
> > From: Elena Agostini <eagostini@nvidia.com>
> >
> > Enable the possibility to make a GPU memory area accessible from
> > the CPU.
> >
> > GPU memory has to be allocated via rte_gpu_mem_alloc().
> >
> > This patch allows the gpudev library to pin, through the GPU driver,
> > a chunk of GPU memory and to return a memory pointer usable
> > by the CPU to access the GPU memory area.
> >
> > Signed-off-by: Elena Agostini <eagostini@nvidia.com>
> [...]
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Pin a chunk of GPU memory to make it accessible from the CPU
>
> You should define what means "pin" exactly.
> Which properties should we expect?
>
Thanks for reviewing, this is the kind of discussion I wanted to have.
Maybe "pin" is too GDRCopy specific oriented.
Here I want to make a GPU memory buffer visible from the CPU. In case
of NVIDIA, this means the GPU memory address has to be pinned (virtual address
doesn't change) and dma-mapped.
Maybe the name should be more like rte_gpu_mem_to_cpu() that's more
explicative and generic.
> > + * using the memory pointer returned by the function.
>
> Which function should return the pointer?
> rte_gpu_mem_pin is returning an int.
Oversight, will fix it.
>
>
> > + * GPU memory has to be allocated via rte_gpu_mem_alloc().
>
> Why pinning is not done by rte_gpu_mem_alloc()?
> Should it be a flag?
rte_gpu_mem_alloc() allocate virtual memory on the GPU that doesn't have
to be necessarily shared (pinned) to make it visible from CPU.
>
> > + *
> > + * @param dev_id
> > + * Device ID requiring pinned memory.
> > + * @param size
> > + * Number of bytes to pin.
> > + * Requesting 0 will do nothing.
> > + * @param ptr
> > + * Pointer to the GPU memory area to be pinned.
> > + * NULL is a no-op accepted value.
> > +
> > + * @return
> > + * A pointer to the pinned GPU memory usable by the CPU, otherwise NULL and rte_errno is set:
> > + * - ENODEV if invalid dev_id
> > + * - EINVAL if reserved flags
>
> Which reserved flags?
>
> > + * - ENOTSUP if operation not supported by the driver
> > + * - E2BIG if size is higher than limit
> > + * - ENOMEM if out of space
>
> Is out of space relevant for pinning?
Yes, let me add it
>
> > + * - EPERM if driver error
> > + */
> > +__rte_experimental
> > +int rte_gpu_mem_pin(int16_t dev_id, size_t size, void *ptr);
What happens when the Nvidia GPU driver kernel callback occurs to invalidate the pinned GPU memory region? Doesn't the NIC need to cease all DMA transfers to/from that region before the kernel callback can complete?
From: Elena Agostini <eagostini@nvidia.com>
Sent: 04 January 2022 13:55
To: NBU-Contact-Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>
Cc: dev@dpdk.org
Subject: Re: [PATCH v2] gpudev: pin GPU memory
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> 04/01/2022 03:41, eagostini@nvidia.com<mailto:eagostini@nvidia.com>:
> > From: Elena Agostini <eagostini@nvidia.com<mailto:eagostini@nvidia.com>>
> >
> > Enable the possibility to make a GPU memory area accessible from
> > the CPU.
> >
> > GPU memory has to be allocated via rte_gpu_mem_alloc().
> >
> > This patch allows the gpudev library to pin, through the GPU driver,
> > a chunk of GPU memory and to return a memory pointer usable
> > by the CPU to access the GPU memory area.
> >
> > Signed-off-by: Elena Agostini <eagostini@nvidia.com<mailto:eagostini@nvidia.com>>
> [...]
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Pin a chunk of GPU memory to make it accessible from the CPU
>
> You should define what means "pin" exactly.
> Which properties should we expect?
>
Thanks for reviewing, this is the kind of discussion I wanted to have.
Maybe "pin" is too GDRCopy specific oriented.
Here I want to make a GPU memory buffer visible from the CPU. In case
of NVIDIA, this means the GPU memory address has to be pinned (virtual address
doesn't change) and dma-mapped.
Maybe the name should be more like rte_gpu_mem_to_cpu() that's more
explicative and generic.
> > + * using the memory pointer returned by the function.
>
> Which function should return the pointer?
> rte_gpu_mem_pin is returning an int.
Oversight, will fix it.
>
>
> > + * GPU memory has to be allocated via rte_gpu_mem_alloc().
>
> Why pinning is not done by rte_gpu_mem_alloc()?
> Should it be a flag?
rte_gpu_mem_alloc() allocate virtual memory on the GPU that doesn't have
to be necessarily shared (pinned) to make it visible from CPU.
>
> > + *
> > + * @param dev_id
> > + * Device ID requiring pinned memory.
> > + * @param size
> > + * Number of bytes to pin.
> > + * Requesting 0 will do nothing.
> > + * @param ptr
> > + * Pointer to the GPU memory area to be pinned.
> > + * NULL is a no-op accepted value.
> > +
> > + * @return
> > + * A pointer to the pinned GPU memory usable by the CPU, otherwise NULL and rte_errno is set:
> > + * - ENODEV if invalid dev_id
> > + * - EINVAL if reserved flags
>
> Which reserved flags?
>
> > + * - ENOTSUP if operation not supported by the driver
> > + * - E2BIG if size is higher than limit
> > + * - ENOMEM if out of space
>
> Is out of space relevant for pinning?
Yes, let me add it
>
> > + * - EPERM if driver error
> > + */
> > +__rte_experimental
> > +int rte_gpu_mem_pin(int16_t dev_id, size_t size, void *ptr);
This specific feature is not about NIC activity, it’s about make GPU memory visible and accessible to the CPU so there isn’t an asynchronous hidden invalidation procedure.
That’s the reason for my comment to change the name from rte_gpu_mem_pin() to rte_gpu_mem_to_cpu() (or similar).
The scenario you are proposing can happen but in a different context, this is not the case.
Thanks
EA
From: John Alexander <John.Alexander@datapath.co.uk>
Date: Tuesday, 4 January 2022 at 18:31
To: Elena Agostini <eagostini@nvidia.com>, NBU-Contact-Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>
Cc: dev@dpdk.org <dev@dpdk.org>
Subject: RE: [PATCH v2] gpudev: pin GPU memory
External email: Use caution opening links or attachments
What happens when the Nvidia GPU driver kernel callback occurs to invalidate the pinned GPU memory region? Doesn’t the NIC need to cease all DMA transfers to/from that region before the kernel callback can complete?
John Alexander
Senior Software Engineer
[cid:image152483.jpg@5CD90415.402D82AB]<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.datapath.co.uk%2F&data=04%7C01%7Ceagostini%40nvidia.com%7Ce2fbb83b86ac4dc3571308d9cfa7b653%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637769142848844684%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=9b3%2FEpEiezz%2B0nLDmm2pE5ZcLkEzNbW%2F%2BJ1o61q2J3k%3D&reserved=0>
Bemrose House, Bemrose Park, Wayzgoose Drive
,
Derby
,
DE21 6XQ
[cid:image839434.png@FD4260AC.D2B1B11C]
+44 (0)1332 294 441<tel:+44%20(0)1332%20294%20441>
|
www.datapath.co.uk<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.datapath.co.uk%2F&data=04%7C01%7Ceagostini%40nvidia.com%7Ce2fbb83b86ac4dc3571308d9cfa7b653%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637769142848844684%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=9b3%2FEpEiezz%2B0nLDmm2pE5ZcLkEzNbW%2F%2BJ1o61q2J3k%3D&reserved=0>
[LinkedIn]<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fdatapath-ltd&data=04%7C01%7Ceagostini%40nvidia.com%7Ce2fbb83b86ac4dc3571308d9cfa7b653%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637769142848844684%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=0PTscJhiHEK27PKDjq2nNgH5oRe499xubLh4S705u4A%3D&reserved=0>
[Twitter]<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.twitter.com%2Fdatapathltd&data=04%7C01%7Ceagostini%40nvidia.com%7Ce2fbb83b86ac4dc3571308d9cfa7b653%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637769142848844684%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=CVia5Sq%2Fi2yOAGx5d8PSsF7wJsA2oVM%2BgySwg5E7p2U%3D&reserved=0>
[YouTube]<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.youtube.com%2Fdatapathderby&data=04%7C01%7Ceagostini%40nvidia.com%7Ce2fbb83b86ac4dc3571308d9cfa7b653%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637769142848844684%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Obz00eF3OgTdp3YI5ux2YFRdx6Q65keqlMXVY6QNXCM%3D&reserved=0>
[cid:image974625.png@E5A3F546.B8FD2EA9]<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.datapath.co.uk%2Fevents-and-webinars%2F&data=04%7C01%7Ceagostini%40nvidia.com%7Ce2fbb83b86ac4dc3571308d9cfa7b653%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637769142848844684%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=MO6M5IE0mAymLgzIZwGqPPxqftcmzH7Gy43ZOGYHwAs%3D&reserved=0>
[Vote for Datapath]<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.datapath.co.uk%2Fdatapath-shortlisted-for-coveted-industry-award%2F&data=04%7C01%7Ceagostini%40nvidia.com%7Ce2fbb83b86ac4dc3571308d9cfa7b653%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637769142848844684%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=D0wKDJVR%2B9kSzyIvtxt5yOdKZPCb%2Fd48biairPlYqt4%3D&reserved=0>
Datapath Ltd. Registered Number: 1609392. Registered in England at Bemrose House, Bemrose Park, Wayzgoose Drive, Derby. DE21 6XQ.
From: Elena Agostini <eagostini@nvidia.com>
Sent: 04 January 2022 13:55
To: NBU-Contact-Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>
Cc: dev@dpdk.org
Subject: Re: [PATCH v2] gpudev: pin GPU memory
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> 04/01/2022 03:41, eagostini@nvidia.com<mailto:eagostini@nvidia.com>:
> > From: Elena Agostini <eagostini@nvidia.com<mailto:eagostini@nvidia.com>>
> >
> > Enable the possibility to make a GPU memory area accessible from
> > the CPU.
> >
> > GPU memory has to be allocated via rte_gpu_mem_alloc().
> >
> > This patch allows the gpudev library to pin, through the GPU driver,
> > a chunk of GPU memory and to return a memory pointer usable
> > by the CPU to access the GPU memory area.
> >
> > Signed-off-by: Elena Agostini <eagostini@nvidia.com<mailto:eagostini@nvidia.com>>
> [...]
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Pin a chunk of GPU memory to make it accessible from the CPU
>
> You should define what means "pin" exactly.
> Which properties should we expect?
>
Thanks for reviewing, this is the kind of discussion I wanted to have.
Maybe "pin" is too GDRCopy specific oriented.
Here I want to make a GPU memory buffer visible from the CPU. In case
of NVIDIA, this means the GPU memory address has to be pinned (virtual address
doesn't change) and dma-mapped.
Maybe the name should be more like rte_gpu_mem_to_cpu() that's more
explicative and generic.
> > + * using the memory pointer returned by the function.
>
> Which function should return the pointer?
> rte_gpu_mem_pin is returning an int.
Oversight, will fix it.
>
>
> > + * GPU memory has to be allocated via rte_gpu_mem_alloc().
>
> Why pinning is not done by rte_gpu_mem_alloc()?
> Should it be a flag?
rte_gpu_mem_alloc() allocate virtual memory on the GPU that doesn't have
to be necessarily shared (pinned) to make it visible from CPU.
>
> > + *
> > + * @param dev_id
> > + * Device ID requiring pinned memory.
> > + * @param size
> > + * Number of bytes to pin.
> > + * Requesting 0 will do nothing.
> > + * @param ptr
> > + * Pointer to the GPU memory area to be pinned.
> > + * NULL is a no-op accepted value.
> > +
> > + * @return
> > + * A pointer to the pinned GPU memory usable by the CPU, otherwise NULL and rte_errno is set:
> > + * - ENODEV if invalid dev_id
> > + * - EINVAL if reserved flags
>
> Which reserved flags?
>
> > + * - ENOTSUP if operation not supported by the driver
> > + * - E2BIG if size is higher than limit
> > + * - ENOMEM if out of space
>
> Is out of space relevant for pinning?
Yes, let me add it
>
> > + * - EPERM if driver error
> > + */
> > +__rte_experimental
> > +int rte_gpu_mem_pin(int16_t dev_id, size_t size, void *ptr);
@@ -634,6 +634,53 @@ rte_gpu_mem_unregister(int16_t dev_id, void *ptr)
return GPU_DRV_RET(dev->ops.mem_unregister(dev, ptr));
}
+int
+rte_gpu_mem_pin(int16_t dev_id, size_t size, void *ptr)
+{
+ struct rte_gpu *dev;
+
+ dev = gpu_get_by_id(dev_id);
+ if (dev == NULL) {
+ GPU_LOG(ERR, "pin mem for invalid device ID %d", dev_id);
+ rte_errno = ENODEV;
+ return -rte_errno;
+ }
+
+ if (dev->ops.mem_pin == NULL) {
+ GPU_LOG(ERR, "mem pinning not supported");
+ rte_errno = ENOTSUP;
+ return -rte_errno;
+ }
+
+ if (ptr == NULL || size == 0) /* dry-run */
+ return 0;
+
+ return GPU_DRV_RET(dev->ops.mem_pin(dev, size, ptr));
+}
+
+int
+rte_gpu_mem_unpin(int16_t dev_id, void *ptr)
+{
+ struct rte_gpu *dev;
+
+ dev = gpu_get_by_id(dev_id);
+ if (dev == NULL) {
+ GPU_LOG(ERR, "unpin mem for invalid device ID %d", dev_id);
+ rte_errno = ENODEV;
+ return -rte_errno;
+ }
+
+ if (dev->ops.mem_unpin == NULL) {
+ rte_errno = ENOTSUP;
+ return -rte_errno;
+ }
+
+ if (ptr == NULL) /* dry-run */
+ return 0;
+
+ return GPU_DRV_RET(dev->ops.mem_unpin(dev, ptr));
+}
+
int
rte_gpu_wmb(int16_t dev_id)
{
@@ -31,6 +31,8 @@ typedef int (rte_gpu_mem_alloc_t)(struct rte_gpu *dev, size_t size, void **ptr);
typedef int (rte_gpu_mem_free_t)(struct rte_gpu *dev, void *ptr);
typedef int (rte_gpu_mem_register_t)(struct rte_gpu *dev, size_t size, void *ptr);
typedef int (rte_gpu_mem_unregister_t)(struct rte_gpu *dev, void *ptr);
+typedef int (rte_gpu_mem_pin_t)(struct rte_gpu *dev, size_t size, void *ptr);
+typedef int (rte_gpu_mem_unpin_t)(struct rte_gpu *dev, void *ptr);
typedef int (rte_gpu_wmb_t)(struct rte_gpu *dev);
struct rte_gpu_ops {
@@ -46,6 +48,10 @@ struct rte_gpu_ops {
rte_gpu_mem_register_t *mem_register;
/* Unregister CPU memory from device. */
rte_gpu_mem_unregister_t *mem_unregister;
+ /* Pin GPU memory. */
+ rte_gpu_mem_pin_t *mem_pin;
+ /* Unpin GPU memory. */
+ rte_gpu_mem_unpin_t *mem_unpin;
/* Enforce GPU write memory barrier. */
rte_gpu_wmb_t *wmb;
};
@@ -447,6 +447,56 @@ int rte_gpu_mem_register(int16_t dev_id, size_t size, void *ptr);
__rte_experimental
int rte_gpu_mem_unregister(int16_t dev_id, void *ptr);
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Pin a chunk of GPU memory to make it accessible from the CPU
+ * using the memory pointer returned by the function.
+ * GPU memory has to be allocated via rte_gpu_mem_alloc().
+ *
+ * @param dev_id
+ * Device ID requiring pinned memory.
+ * @param size
+ * Number of bytes to pin.
+ * Requesting 0 will do nothing.
+ * @param ptr
+ * Pointer to the GPU memory area to be pinned.
+ * NULL is a no-op accepted value.
+
+ * @return
+ * A pointer to the pinned GPU memory usable by the CPU, otherwise NULL and rte_errno is set:
+ * - ENODEV if invalid dev_id
+ * - EINVAL if reserved flags
+ * - ENOTSUP if operation not supported by the driver
+ * - E2BIG if size is higher than limit
+ * - ENOMEM if out of space
+ * - EPERM if driver error
+ */
+__rte_experimental
+int rte_gpu_mem_pin(int16_t dev_id, size_t size, void *ptr);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Unpin a chunk of GPU memory previously pinned with rte_gpu_mem_pin()
+ *
+ * @param dev_id
+ * Reference device ID.
+ * @param ptr
+ * Pointer to the memory area to be unpinned.
+ * NULL is a no-op accepted value.
+ *
+ * @return
+ * 0 on success, -rte_errno otherwise:
+ * - ENODEV if invalid dev_id
+ * - ENOTSUP if operation not supported by the driver
+ * - EPERM if driver error
+ */
+__rte_experimental
+int rte_gpu_mem_unpin(int16_t dev_id, void *ptr);
+
/**
* @warning
* @b EXPERIMENTAL: this API may change without prior notice.
@@ -21,7 +21,9 @@ EXPERIMENTAL {
rte_gpu_is_valid;
rte_gpu_mem_alloc;
rte_gpu_mem_free;
+ rte_gpu_mem_pin;
rte_gpu_mem_register;
+ rte_gpu_mem_unpin;
rte_gpu_mem_unregister;
rte_gpu_wmb;
};