Message ID | cover.1543495935.git.anatoly.burakov@intel.com (mailing list archive) |
---|---|
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 363661B4E7; Thu, 29 Nov 2018 14:48:49 +0100 (CET) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id EDEDF322C for <dev@dpdk.org>; Thu, 29 Nov 2018 14:48:39 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Nov 2018 05:48:37 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,294,1539673200"; d="scan'208";a="116397784" Received: from irvmail001.ir.intel.com ([163.33.26.43]) by fmsmga002.fm.intel.com with ESMTP; 29 Nov 2018 05:48:36 -0800 Received: from sivswdev01.ir.intel.com (sivswdev01.ir.intel.com [10.237.217.45]) by irvmail001.ir.intel.com (8.14.3/8.13.6/MailSET/Hub) with ESMTP id wATDmZID012495; Thu, 29 Nov 2018 13:48:35 GMT Received: from sivswdev01.ir.intel.com (localhost [127.0.0.1]) by sivswdev01.ir.intel.com with ESMTP id wATDmZdU019964; Thu, 29 Nov 2018 13:48:35 GMT Received: (from aburakov@localhost) by sivswdev01.ir.intel.com with LOCAL id wATDmZwe019960; Thu, 29 Nov 2018 13:48:35 GMT From: Anatoly Burakov <anatoly.burakov@intel.com> To: dev@dpdk.org Cc: shahafs@mellanox.com, yskoh@mellanox.com, thomas@monjalon.net, shreyansh.jain@nxp.com Date: Thu, 29 Nov 2018 13:48:31 +0000 Message-Id: <cover.1543495935.git.anatoly.burakov@intel.com> X-Mailer: git-send-email 1.7.0.7 Subject: [dpdk-dev] [PATCH 0/4] Allow using external memory without malloc X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> |
Series | Allow using external memory without malloc | |
Message
Anatoly Burakov
Nov. 29, 2018, 1:48 p.m. UTC
Currently, the only way to use externally allocated memory is through rte_malloc API's. While this is fine for a lot of use cases, it may not be suitable for certain other use cases like manual memory management, etc. This patchset adds another API to register memory segments with DPDK (so that API's like ``rte_mem_virt2memseg`` could be relied on by PMD's and such), but not create a malloc heap out of them. Aside from the obvious (not adding memory to a heap), the other major difference between this API and the ``rte_malloc_heap_*`` external memory functions is the fact that no DMA mapping is performed automatically. This really draws a line in the sand, and there are now two ways of doing things - do everything automatically (using the ``rte_malloc_heap_*`` API's), or do everything manually (``rte_extmem_*`` and future DMA mapping API [1] that would replace ``rte_vfio_dma_map``). This way, the consistency of API is kept, and flexibility is also allowed. [1] https://mails.dpdk.org/archives/dev/2018-November/118175.html Note: at the time of this writing, there's no release notes template, so no release notes updates in this patchset. They will be added in later revisions. Anatoly Burakov (4): malloc: separate creating memseg list and malloc heap malloc: separate destroying memseg list and heap data mem: allow registering external memory areas mem: allow usage of non-heap external memory in multiprocess .../prog_guide/env_abstraction_layer.rst | 63 +++++++-- lib/librte_eal/common/eal_common_memory.c | 116 +++++++++++++++++ lib/librte_eal/common/include/rte_memory.h | 122 ++++++++++++++++++ lib/librte_eal/common/malloc_heap.c | 104 +++++++++++---- lib/librte_eal/common/malloc_heap.h | 15 ++- lib/librte_eal/common/rte_malloc.c | 115 +++++++---------- lib/librte_eal/rte_eal_version.map | 4 + 7 files changed, 434 insertions(+), 105 deletions(-)
Comments
Hi Anatoly, Thursday, November 29, 2018 3:49 PM, Anatoly Burakov: > Subject: [PATCH 0/4] Allow using external memory without malloc > > Currently, the only way to use externally allocated memory is through > rte_malloc API's. While this is fine for a lot of use cases, it may not be suitable > for certain other use cases like manual memory management, etc. > > This patchset adds another API to register memory segments with DPDK (so > that API's like ``rte_mem_virt2memseg`` could be relied on by PMD's and > such), but not create a malloc heap out of them. > > Aside from the obvious (not adding memory to a heap), the other major > difference between this API and the ``rte_malloc_heap_*`` external memory > functions is the fact that no DMA mapping is performed automatically. > > This really draws a line in the sand, and there are now two ways of doing > things - do everything automatically (using the ``rte_malloc_heap_*`` API's), > or do everything manually (``rte_extmem_*`` and future DMA mapping API > [1] that would replace ``rte_vfio_dma_map``). This way, the consistency of > API is kept, and flexibility is also allowed. > As you know I like the idea. One question though, do you see a use case for application to have externally allocated memory which needs to be registered to the DPDK subsystem however not being used for DMA? My only guess would be so helper libraries which requires the memory allocation from user (however it doesn't seems like a good API). If no use case, maybe it is better to merge between the two (rte_extmem_* and rte_dma_map) to have a single call for app to register and DMA map the memory. The rte_mem_virt2memseg is not something application needs to understand, it is used internally by PMDs or other libs. > [1] > https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fma > ils.dpdk.org%2Farchives%2Fdev%2F2018- > November%2F118175.html&data=02%7C01%7Cshahafs%40mellanox.co > m%7C007a9234feaf42c82f6508d656015eb1%7Ca652971c7d2e4d9ba6a4d1492 > 56f461b%7C0%7C0%7C636790961244424277&sdata=YqwcPEEhJM3I7Toe > Ne%2BGcbeo%2FmPbYEnNFckoA7ES2Hg%3D&reserved=0 > > Note: at the time of this writing, there's no release notes > template, so no release notes updates in this patchset. > They will be added in later revisions. > > Anatoly Burakov (4): > malloc: separate creating memseg list and malloc heap > malloc: separate destroying memseg list and heap data > mem: allow registering external memory areas > mem: allow usage of non-heap external memory in multiprocess > > .../prog_guide/env_abstraction_layer.rst | 63 +++++++-- > lib/librte_eal/common/eal_common_memory.c | 116 > +++++++++++++++++ > lib/librte_eal/common/include/rte_memory.h | 122 > ++++++++++++++++++ > lib/librte_eal/common/malloc_heap.c | 104 +++++++++++---- > lib/librte_eal/common/malloc_heap.h | 15 ++- > lib/librte_eal/common/rte_malloc.c | 115 +++++++---------- > lib/librte_eal/rte_eal_version.map | 4 + > 7 files changed, 434 insertions(+), 105 deletions(-) > > -- > 2.17.1
> On Dec 1, 2018, at 9:48 PM, Shahaf Shuler <shahafs@mellanox.com> wrote: > > Hi Anatoly, > > Thursday, November 29, 2018 3:49 PM, Anatoly Burakov: >> Subject: [PATCH 0/4] Allow using external memory without malloc >> >> Currently, the only way to use externally allocated memory is through >> rte_malloc API's. While this is fine for a lot of use cases, it may not be suitable >> for certain other use cases like manual memory management, etc. >> >> This patchset adds another API to register memory segments with DPDK (so >> that API's like ``rte_mem_virt2memseg`` could be relied on by PMD's and >> such), but not create a malloc heap out of them. >> >> Aside from the obvious (not adding memory to a heap), the other major >> difference between this API and the ``rte_malloc_heap_*`` external memory >> functions is the fact that no DMA mapping is performed automatically. >> >> This really draws a line in the sand, and there are now two ways of doing >> things - do everything automatically (using the ``rte_malloc_heap_*`` API's), >> or do everything manually (``rte_extmem_*`` and future DMA mapping API >> [1] that would replace ``rte_vfio_dma_map``). This way, the consistency of >> API is kept, and flexibility is also allowed. >> > > As you know I like the idea. > One question though, do you see a use case for application to have externally allocated memory which needs to be registered to the DPDK subsystem however not being used for DMA? > My only guess would be so helper libraries which requires the memory allocation from user (however it doesn't seems like a good API). > > If no use case, maybe it is better to merge between the two (rte_extmem_* and rte_dma_map) to have a single call for app to register and DMA map the memory. The rte_mem_virt2memseg is not something application needs to understand, it is used internally by PMDs or other libs. Just FYI. My implementation for mlx4/5 doesn't need to have a separate registration for DMA by rte_dma_map() as long as it is included in the memseg list. Registration is done only if Lkey lookup misses and only mem free event is taken to release it. From my end, the reason why we wanted to have a generic DMA registration was because some people doesn't want to use the new API to make the ext mem included in the memseg list but want to simply call the API for DMA registration. In a nutshell, mlx4/5 needs users use either rte_extmem_register() or rte_dma_map(). However, it is no problem to call both. Thanks, Yongseok >> [1] >> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fma >> ils.dpdk.org%2Farchives%2Fdev%2F2018- >> November%2F118175.html&data=02%7C01%7Cshahafs%40mellanox.co >> m%7C007a9234feaf42c82f6508d656015eb1%7Ca652971c7d2e4d9ba6a4d1492 >> 56f461b%7C0%7C0%7C636790961244424277&sdata=YqwcPEEhJM3I7Toe >> Ne%2BGcbeo%2FmPbYEnNFckoA7ES2Hg%3D&reserved=0 >> >> Note: at the time of this writing, there's no release notes >> template, so no release notes updates in this patchset. >> They will be added in later revisions. >> >> Anatoly Burakov (4): >> malloc: separate creating memseg list and malloc heap >> malloc: separate destroying memseg list and heap data >> mem: allow registering external memory areas >> mem: allow usage of non-heap external memory in multiprocess >> >> .../prog_guide/env_abstraction_layer.rst | 63 +++++++-- >> lib/librte_eal/common/eal_common_memory.c | 116 >> +++++++++++++++++ >> lib/librte_eal/common/include/rte_memory.h | 122 >> ++++++++++++++++++ >> lib/librte_eal/common/malloc_heap.c | 104 +++++++++++---- >> lib/librte_eal/common/malloc_heap.h | 15 ++- >> lib/librte_eal/common/rte_malloc.c | 115 +++++++---------- >> lib/librte_eal/rte_eal_version.map | 4 + >> 7 files changed, 434 insertions(+), 105 deletions(-) >> >> -- >> 2.17.1
On 02-Dec-18 11:28 PM, Yongseok Koh wrote: > >> On Dec 1, 2018, at 9:48 PM, Shahaf Shuler <shahafs@mellanox.com> wrote: >> >> Hi Anatoly, >> >> Thursday, November 29, 2018 3:49 PM, Anatoly Burakov: >>> Subject: [PATCH 0/4] Allow using external memory without malloc >>> >>> Currently, the only way to use externally allocated memory is through >>> rte_malloc API's. While this is fine for a lot of use cases, it may not be suitable >>> for certain other use cases like manual memory management, etc. >>> >>> This patchset adds another API to register memory segments with DPDK (so >>> that API's like ``rte_mem_virt2memseg`` could be relied on by PMD's and >>> such), but not create a malloc heap out of them. >>> >>> Aside from the obvious (not adding memory to a heap), the other major >>> difference between this API and the ``rte_malloc_heap_*`` external memory >>> functions is the fact that no DMA mapping is performed automatically. >>> >>> This really draws a line in the sand, and there are now two ways of doing >>> things - do everything automatically (using the ``rte_malloc_heap_*`` API's), >>> or do everything manually (``rte_extmem_*`` and future DMA mapping API >>> [1] that would replace ``rte_vfio_dma_map``). This way, the consistency of >>> API is kept, and flexibility is also allowed. >>> >> >> As you know I like the idea. >> One question though, do you see a use case for application to have externally allocated memory which needs to be registered to the DPDK subsystem however not being used for DMA? >> My only guess would be so helper libraries which requires the memory allocation from user (however it doesn't seems like a good API). >> >> If no use case, maybe it is better to merge between the two (rte_extmem_* and rte_dma_map) to have a single call for app to register and DMA map the memory. The rte_mem_virt2memseg is not something application needs to understand, it is used internally by PMDs or other libs. > > Just FYI. > My implementation for mlx4/5 doesn't need to have a separate registration for > DMA by rte_dma_map() as long as it is included in the memseg list. Registration > is done only if Lkey lookup misses and only mem free event is taken to release > it. From my end, the reason why we wanted to have a generic DMA registration was > because some people doesn't want to use the new API to make the ext mem included > in the memseg list but want to simply call the API for DMA registration. > > In a nutshell, mlx4/5 needs users use either rte_extmem_register() or > rte_dma_map(). However, it is no problem to call both. It would be good to create a segment when using rte_dma_map(). Unfortunately, that's not realistic :) Registering memory within DPDK does not necessarily have to be performed by the primary process - whichever process that wants to create the table, can do so, and later processes have to attach to the memory area. There's also no way to know if memory segment can be attached to - this is a question only application can answer. In other words, there's no way to combine rte_dma_map() and rte_extmem_register() into one call and keep multiprocess support. > > > Thanks, > Yongseok > >>> [1] >>> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fma >>> ils.dpdk.org%2Farchives%2Fdev%2F2018- >>> November%2F118175.html&data=02%7C01%7Cshahafs%40mellanox.co >>> m%7C007a9234feaf42c82f6508d656015eb1%7Ca652971c7d2e4d9ba6a4d1492 >>> 56f461b%7C0%7C0%7C636790961244424277&sdata=YqwcPEEhJM3I7Toe >>> Ne%2BGcbeo%2FmPbYEnNFckoA7ES2Hg%3D&reserved=0 >>> >>> Note: at the time of this writing, there's no release notes >>> template, so no release notes updates in this patchset. >>> They will be added in later revisions. >>> >>> Anatoly Burakov (4): >>> malloc: separate creating memseg list and malloc heap >>> malloc: separate destroying memseg list and heap data >>> mem: allow registering external memory areas >>> mem: allow usage of non-heap external memory in multiprocess >>> >>> .../prog_guide/env_abstraction_layer.rst | 63 +++++++-- >>> lib/librte_eal/common/eal_common_memory.c | 116 >>> +++++++++++++++++ >>> lib/librte_eal/common/include/rte_memory.h | 122 >>> ++++++++++++++++++ >>> lib/librte_eal/common/malloc_heap.c | 104 +++++++++++---- >>> lib/librte_eal/common/malloc_heap.h | 15 ++- >>> lib/librte_eal/common/rte_malloc.c | 115 +++++++---------- >>> lib/librte_eal/rte_eal_version.map | 4 + >>> 7 files changed, 434 insertions(+), 105 deletions(-) >>> >>> -- >>> 2.17.1 > >
On Mon, Dec 03, 2018 at 10:23:03AM +0000, Burakov, Anatoly wrote: > On 02-Dec-18 11:28 PM, Yongseok Koh wrote: > > > > > On Dec 1, 2018, at 9:48 PM, Shahaf Shuler <shahafs@mellanox.com> wrote: > > > > > > Hi Anatoly, > > > > > > Thursday, November 29, 2018 3:49 PM, Anatoly Burakov: > > > > Subject: [PATCH 0/4] Allow using external memory without malloc > > > > > > > > Currently, the only way to use externally allocated memory is through > > > > rte_malloc API's. While this is fine for a lot of use cases, it may not be suitable > > > > for certain other use cases like manual memory management, etc. > > > > > > > > This patchset adds another API to register memory segments with DPDK (so > > > > that API's like ``rte_mem_virt2memseg`` could be relied on by PMD's and > > > > such), but not create a malloc heap out of them. > > > > > > > > Aside from the obvious (not adding memory to a heap), the other major > > > > difference between this API and the ``rte_malloc_heap_*`` external memory > > > > functions is the fact that no DMA mapping is performed automatically. > > > > > > > > This really draws a line in the sand, and there are now two ways of doing > > > > things - do everything automatically (using the ``rte_malloc_heap_*`` API's), > > > > or do everything manually (``rte_extmem_*`` and future DMA mapping API > > > > [1] that would replace ``rte_vfio_dma_map``). This way, the consistency of > > > > API is kept, and flexibility is also allowed. > > > > > > > > > > As you know I like the idea. > > > One question though, do you see a use case for application to have externally allocated memory which needs to be registered to the DPDK subsystem however not being used for DMA? > > > My only guess would be so helper libraries which requires the memory allocation from user (however it doesn't seems like a good API). > > > > > > If no use case, maybe it is better to merge between the two (rte_extmem_* and rte_dma_map) to have a single call for app to register and DMA map the memory. The rte_mem_virt2memseg is not something application needs to understand, it is used internally by PMDs or other libs. > > > > Just FYI. > > My implementation for mlx4/5 doesn't need to have a separate registration for > > DMA by rte_dma_map() as long as it is included in the memseg list. Registration > > is done only if Lkey lookup misses and only mem free event is taken to release > > it. From my end, the reason why we wanted to have a generic DMA registration was > > because some people doesn't want to use the new API to make the ext mem included > > in the memseg list but want to simply call the API for DMA registration. > > > > In a nutshell, mlx4/5 needs users use either rte_extmem_register() or > > rte_dma_map(). However, it is no problem to call both. > > It would be good to create a segment when using rte_dma_map(). > Unfortunately, that's not realistic :) > > Registering memory within DPDK does not necessarily have to be performed by > the primary process - whichever process that wants to create the table, can > do so, and later processes have to attach to the memory area. There's also > no way to know if memory segment can be attached to - this is a question > only application can answer. > > In other words, there's no way to combine rte_dma_map() and > rte_extmem_register() into one call and keep multiprocess support. Sorry for late reply. I was away for a while. I understood your point that rte_dma_map() can't create a segment but isn't the opposite possible? I still have a question about rte_extmem_register/unregister/attach/detach(). Why don't these APIs generate memory events? Do you define the memory events are limited to memories for malloc? What if some app wants to know the events even if it is extmem? What makes difference between two types of extmem (one for malloc heap and the other for just memseg) in generating the events? I've reviewed your patches and all look good :-) But it is still unclear to me. Thanks, Yongseok > > > > [1] > > > > https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fma > > > > ils.dpdk.org%2Farchives%2Fdev%2F2018- > > > > November%2F118175.html&data=02%7C01%7Cshahafs%40mellanox.co > > > > m%7C007a9234feaf42c82f6508d656015eb1%7Ca652971c7d2e4d9ba6a4d1492 > > > > 56f461b%7C0%7C0%7C636790961244424277&sdata=YqwcPEEhJM3I7Toe > > > > Ne%2BGcbeo%2FmPbYEnNFckoA7ES2Hg%3D&reserved=0 > > > > > > > > Note: at the time of this writing, there's no release notes > > > > template, so no release notes updates in this patchset. > > > > They will be added in later revisions. > > > > > > > > Anatoly Burakov (4): > > > > malloc: separate creating memseg list and malloc heap > > > > malloc: separate destroying memseg list and heap data > > > > mem: allow registering external memory areas > > > > mem: allow usage of non-heap external memory in multiprocess > > > > > > > > .../prog_guide/env_abstraction_layer.rst | 63 +++++++-- > > > > lib/librte_eal/common/eal_common_memory.c | 116 > > > > +++++++++++++++++ > > > > lib/librte_eal/common/include/rte_memory.h | 122 > > > > ++++++++++++++++++ > > > > lib/librte_eal/common/malloc_heap.c | 104 +++++++++++---- > > > > lib/librte_eal/common/malloc_heap.h | 15 ++- > > > > lib/librte_eal/common/rte_malloc.c | 115 +++++++---------- > > > > lib/librte_eal/rte_eal_version.map | 4 + > > > > 7 files changed, 434 insertions(+), 105 deletions(-) > > > > > > > > -- > > > > 2.17.1 > > > > > > > -- > Thanks, > Anatoly
On 12-Dec-18 12:55 PM, Yongseok Koh wrote: > On Mon, Dec 03, 2018 at 10:23:03AM +0000, Burakov, Anatoly wrote: >> On 02-Dec-18 11:28 PM, Yongseok Koh wrote: >>> >>>> On Dec 1, 2018, at 9:48 PM, Shahaf Shuler <shahafs@mellanox.com> wrote: >>>> >>>> Hi Anatoly, >>>> >>>> Thursday, November 29, 2018 3:49 PM, Anatoly Burakov: >>>>> Subject: [PATCH 0/4] Allow using external memory without malloc >>>>> >>>>> Currently, the only way to use externally allocated memory is through >>>>> rte_malloc API's. While this is fine for a lot of use cases, it may not be suitable >>>>> for certain other use cases like manual memory management, etc. >>>>> >>>>> This patchset adds another API to register memory segments with DPDK (so >>>>> that API's like ``rte_mem_virt2memseg`` could be relied on by PMD's and >>>>> such), but not create a malloc heap out of them. >>>>> >>>>> Aside from the obvious (not adding memory to a heap), the other major >>>>> difference between this API and the ``rte_malloc_heap_*`` external memory >>>>> functions is the fact that no DMA mapping is performed automatically. >>>>> >>>>> This really draws a line in the sand, and there are now two ways of doing >>>>> things - do everything automatically (using the ``rte_malloc_heap_*`` API's), >>>>> or do everything manually (``rte_extmem_*`` and future DMA mapping API >>>>> [1] that would replace ``rte_vfio_dma_map``). This way, the consistency of >>>>> API is kept, and flexibility is also allowed. >>>>> >>>> >>>> As you know I like the idea. >>>> One question though, do you see a use case for application to have externally allocated memory which needs to be registered to the DPDK subsystem however not being used for DMA? >>>> My only guess would be so helper libraries which requires the memory allocation from user (however it doesn't seems like a good API). >>>> >>>> If no use case, maybe it is better to merge between the two (rte_extmem_* and rte_dma_map) to have a single call for app to register and DMA map the memory. The rte_mem_virt2memseg is not something application needs to understand, it is used internally by PMDs or other libs. >>> >>> Just FYI. >>> My implementation for mlx4/5 doesn't need to have a separate registration for >>> DMA by rte_dma_map() as long as it is included in the memseg list. Registration >>> is done only if Lkey lookup misses and only mem free event is taken to release >>> it. From my end, the reason why we wanted to have a generic DMA registration was >>> because some people doesn't want to use the new API to make the ext mem included >>> in the memseg list but want to simply call the API for DMA registration. >>> >>> In a nutshell, mlx4/5 needs users use either rte_extmem_register() or >>> rte_dma_map(). However, it is no problem to call both. >> >> It would be good to create a segment when using rte_dma_map(). >> Unfortunately, that's not realistic :) >> >> Registering memory within DPDK does not necessarily have to be performed by >> the primary process - whichever process that wants to create the table, can >> do so, and later processes have to attach to the memory area. There's also >> no way to know if memory segment can be attached to - this is a question >> only application can answer. >> >> In other words, there's no way to combine rte_dma_map() and >> rte_extmem_register() into one call and keep multiprocess support. > > Sorry for late reply. I was away for a while. > > I understood your point that rte_dma_map() can't create a segment but isn't the > opposite possible? I still have a question about > rte_extmem_register/unregister/attach/detach(). Why don't these APIs generate > memory events? Do you define the memory events are limited to memories for > malloc? What if some app wants to know the events even if it is extmem? What > makes difference between two types of extmem (one for malloc heap and the other > for just memseg) in generating the events? > > I've reviewed your patches and all look good :-) But it is still unclear to me. Hi Yongseok, Idealistically speaking, my view is, if you want the luxury of DPDK doing everything for you, use malloc heaps. If you don't - you're on your own :) Any callbacks etc. that you might want to get if you're *not* using malloc, is not really my problem - it's yours, because *you* don't want to use built-in DPDK facilities, for whatever reason. Pragmatically speaking, right now the reason to not do so is to avoid that memory being automatically mapped for DMA due to VFIO currently using callbacks mechanism to subscribe to notifications. When they are decoupled - we can talk about making it so that registered external memory still triggers callbacks (although i do not see why, to be honest). > > > Thanks, > Yongseok > >>>>> [1] >>>>> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fma >>>>> ils.dpdk.org%2Farchives%2Fdev%2F2018- >>>>> November%2F118175.html&data=02%7C01%7Cshahafs%40mellanox.co >>>>> m%7C007a9234feaf42c82f6508d656015eb1%7Ca652971c7d2e4d9ba6a4d1492 >>>>> 56f461b%7C0%7C0%7C636790961244424277&sdata=YqwcPEEhJM3I7Toe >>>>> Ne%2BGcbeo%2FmPbYEnNFckoA7ES2Hg%3D&reserved=0 >>>>> >>>>> Note: at the time of this writing, there's no release notes >>>>> template, so no release notes updates in this patchset. >>>>> They will be added in later revisions. >>>>> >>>>> Anatoly Burakov (4): >>>>> malloc: separate creating memseg list and malloc heap >>>>> malloc: separate destroying memseg list and heap data >>>>> mem: allow registering external memory areas >>>>> mem: allow usage of non-heap external memory in multiprocess >>>>> >>>>> .../prog_guide/env_abstraction_layer.rst | 63 +++++++-- >>>>> lib/librte_eal/common/eal_common_memory.c | 116 >>>>> +++++++++++++++++ >>>>> lib/librte_eal/common/include/rte_memory.h | 122 >>>>> ++++++++++++++++++ >>>>> lib/librte_eal/common/malloc_heap.c | 104 +++++++++++---- >>>>> lib/librte_eal/common/malloc_heap.h | 15 ++- >>>>> lib/librte_eal/common/rte_malloc.c | 115 +++++++---------- >>>>> lib/librte_eal/rte_eal_version.map | 4 + >>>>> 7 files changed, 434 insertions(+), 105 deletions(-) >>>>> >>>>> -- >>>>> 2.17.1 >>> >>> >> >> >> -- >> Thanks, >> Anatoly >