Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/48852/?format=api
{ "id": 48852, "url": "http://patchwork.dpdk.org/api/patches/48852/?format=api", "web_url": "http://patchwork.dpdk.org/project/dpdk/patch/7dd880b42a204c1e90433bd0f102fdba67baace9.1544788118.git.anatoly.burakov@intel.com/", "project": { "id": 1, "url": "http://patchwork.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<7dd880b42a204c1e90433bd0f102fdba67baace9.1544788118.git.anatoly.burakov@intel.com>", "list_archive_url": "https://inbox.dpdk.org/dev/7dd880b42a204c1e90433bd0f102fdba67baace9.1544788118.git.anatoly.burakov@intel.com", "date": "2018-12-14T11:50:58", "name": "[v2,3/4] mem: allow registering external memory areas", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": true, "hash": "45e6f1a642cf31f8e967c45c83bece8b95bc58ce", "submitter": { "id": 4, "url": "http://patchwork.dpdk.org/api/people/4/?format=api", "name": "Anatoly Burakov", "email": "anatoly.burakov@intel.com" }, "delegate": { "id": 1, "url": "http://patchwork.dpdk.org/api/users/1/?format=api", "username": "tmonjalo", "first_name": "Thomas", "last_name": "Monjalon", "email": "thomas@monjalon.net" }, "mbox": "http://patchwork.dpdk.org/project/dpdk/patch/7dd880b42a204c1e90433bd0f102fdba67baace9.1544788118.git.anatoly.burakov@intel.com/mbox/", "series": [ { "id": 2778, "url": "http://patchwork.dpdk.org/api/series/2778/?format=api", "web_url": "http://patchwork.dpdk.org/project/dpdk/list/?series=2778", "date": "2018-12-14T11:50:55", "name": "Allow using external memory without malloc", "version": 2, "mbox": "http://patchwork.dpdk.org/series/2778/mbox/" } ], "comments": "http://patchwork.dpdk.org/api/patches/48852/comments/", "check": "success", "checks": "http://patchwork.dpdk.org/api/patches/48852/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@dpdk.org", "Delivered-To": "patchwork@dpdk.org", "Received": [ "from [92.243.14.124] (localhost [127.0.0.1])\n\tby dpdk.org (Postfix) with ESMTP id 0B4F81B9F3;\n\tFri, 14 Dec 2018 12:51:14 +0100 (CET)", "from mga12.intel.com (mga12.intel.com [192.55.52.136])\n\tby dpdk.org (Postfix) with ESMTP id 797721B99B\n\tfor <dev@dpdk.org>; Fri, 14 Dec 2018 12:51:08 +0100 (CET)", "from orsmga003.jf.intel.com ([10.7.209.27])\n\tby fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;\n\t14 Dec 2018 03:51:07 -0800", "from irvmail001.ir.intel.com ([163.33.26.43])\n\tby orsmga003.jf.intel.com with ESMTP; 14 Dec 2018 03:51:05 -0800", "from sivswdev05.ir.intel.com (sivswdev05.ir.intel.com\n\t[10.243.17.64])\n\tby irvmail001.ir.intel.com (8.14.3/8.13.6/MailSET/Hub) with ESMTP id\n\twBEBp4aj005265; Fri, 14 Dec 2018 11:51:05 GMT", "from sivswdev05.ir.intel.com (localhost [127.0.0.1])\n\tby sivswdev05.ir.intel.com with ESMTP id wBEBp4ne007515;\n\tFri, 14 Dec 2018 11:51:04 GMT", "(from aburakov@localhost)\n\tby sivswdev05.ir.intel.com with LOCAL id wBEBp4tV007511;\n\tFri, 14 Dec 2018 11:51:04 GMT" ], "X-Amp-Result": "SKIPPED(no attachment in message)", "X-Amp-File-Uploaded": "False", "X-ExtLoop1": "1", "X-IronPort-AV": "E=Sophos;i=\"5.56,352,1539673200\"; d=\"scan'208\";a=\"110367475\"", "From": "Anatoly Burakov <anatoly.burakov@intel.com>", "To": "dev@dpdk.org", "Cc": "John McNamara <john.mcnamara@intel.com>,\n\tMarko Kovacevic <marko.kovacevic@intel.com>, shahafs@mellanox.com,\n\tyskoh@mellanox.com, thomas@monjalon.net, shreyansh.jain@nxp.com", "Date": "Fri, 14 Dec 2018 11:50:58 +0000", "Message-Id": "<7dd880b42a204c1e90433bd0f102fdba67baace9.1544788118.git.anatoly.burakov@intel.com>", "X-Mailer": "git-send-email 1.7.0.7", "In-Reply-To": [ "<cover.1544788118.git.anatoly.burakov@intel.com>", "<cover.1544788118.git.anatoly.burakov@intel.com>" ], "References": [ "<cover.1544788118.git.anatoly.burakov@intel.com>", "<cover.1543495935.git.anatoly.burakov@intel.com>\n\t<cover.1544788118.git.anatoly.burakov@intel.com>" ], "Subject": "[dpdk-dev] [PATCH v2 3/4] mem: allow registering external memory\n\tareas", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n\t<mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n\t<mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "content": "The general use-case of using external memory is well covered by\nexisting external memory API's. However, certain use cases require\nmanual management of externally allocated memory areas, so this\nmemory should not be added to the heap. It should, however, be\nadded to DPDK's internal structures, so that API's like\n``rte_virt2memseg`` would work on such external memory segments.\n\nThis commit adds such an API to DPDK. The new functions will allow\nto register and unregister externally allocated memory areas, as\nwell as documentation for them.\n\nSigned-off-by: Anatoly Burakov <anatoly.burakov@intel.com>\nAcked-by: Yongseok Koh <yskoh@mellanox.com>\n---\n\nNotes:\n v2:\n - Do more stringent alignment checks\n - Fix a bug where n_pages was used as is without\n parameter checking\n\n .../prog_guide/env_abstraction_layer.rst | 60 ++++++++++++---\n doc/guides/rel_notes/release_19_02.rst | 6 ++\n lib/librte_eal/common/eal_common_memory.c | 77 +++++++++++++++++++\n lib/librte_eal/common/include/rte_memory.h | 63 +++++++++++++++\n lib/librte_eal/rte_eal_version.map | 2 +\n 5 files changed, 198 insertions(+), 10 deletions(-)", "diff": "diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst\nindex 8b5d050c7..d7799b626 100644\n--- a/doc/guides/prog_guide/env_abstraction_layer.rst\n+++ b/doc/guides/prog_guide/env_abstraction_layer.rst\n@@ -212,17 +212,26 @@ Normally, these options do not need to be changed.\n Support for Externally Allocated Memory\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n \n-It is possible to use externally allocated memory in DPDK, using a set of malloc\n-heap API's. Support for externally allocated memory is implemented through\n-overloading the socket ID - externally allocated heaps will have socket ID's\n-that would be considered invalid under normal circumstances. Requesting an\n-allocation to take place from a specified externally allocated memory is a\n-matter of supplying the correct socket ID to DPDK allocator, either directly\n-(e.g. through a call to ``rte_malloc``) or indirectly (through data\n-structure-specific allocation API's such as ``rte_ring_create``).\n+It is possible to use externally allocated memory in DPDK. There are two ways in\n+which using externally allocated memory can work: the malloc heap API's, and\n+manual memory management.\n \n-Since there is no way DPDK can verify whether memory are is available or valid,\n-this responsibility falls on the shoulders of the user. All multiprocess\n++ Using heap API's for externally allocated memory\n+\n+Using using a set of malloc heap API's is the recommended way to use externally\n+allocated memory in DPDK. In this way, support for externally allocated memory\n+is implemented through overloading the socket ID - externally allocated heaps\n+will have socket ID's that would be considered invalid under normal\n+circumstances. Requesting an allocation to take place from a specified\n+externally allocated memory is a matter of supplying the correct socket ID to\n+DPDK allocator, either directly (e.g. through a call to ``rte_malloc``) or\n+indirectly (through data structure-specific allocation API's such as\n+``rte_ring_create``). Using these API's also ensures that mapping of externally\n+allocated memory for DMA is also performed on any memory segment that is added\n+to a DPDK malloc heap.\n+\n+Since there is no way DPDK can verify whether memory is available or valid, this\n+responsibility falls on the shoulders of the user. All multiprocess\n synchronization is also user's responsibility, as well as ensuring that all\n calls to add/attach/detach/remove memory are done in the correct order. It is\n not required to attach to a memory area in all processes - only attach to memory\n@@ -246,6 +255,37 @@ The expected workflow is as follows:\n For more information, please refer to ``rte_malloc`` API documentation,\n specifically the ``rte_malloc_heap_*`` family of function calls.\n \n++ Using externally allocated memory without DPDK API's\n+\n+While using heap API's is the recommended method of using externally allocated\n+memory in DPDK, there are certain use cases where the overhead of DPDK heap API\n+is undesirable - for example, when manual memory management is performed on an\n+externally allocated area. To support use cases where externally allocated\n+memory will not be used as part of normal DPDK workflow, there is also another\n+set of API's under the ``rte_extmem_*`` namespace.\n+\n+These API's are (as their name implies) intended to allow registering or\n+unregistering externally allocated memory to/from DPDK's internal page table, to\n+allow API's like ``rte_virt2memseg`` etc. to work with externally allocated\n+memory. Memory added this way will not be available for any regular DPDK\n+allocators; DPDK will leave this memory for the user application to manage.\n+\n+The expected workflow is as follows:\n+\n+* Get a pointer to memory area\n+* Register memory within DPDK\n+ - If IOVA table is not specified, IOVA addresses will be assumed to be\n+ unavailable\n+* Perform DMA mapping with ``rte_vfio_dma_map`` if needed\n+* Use the memory area in your application\n+* If memory area is no longer needed, it can be unregistered\n+ - If the area was mapped for DMA, unmapping must be performed before\n+ unregistering memory\n+\n+Since these externally allocated memory areas will not be managed by DPDK, it is\n+therefore up to the user application to decide how to use them and what to do\n+with them once they're registered.\n+\n Per-lcore and Shared Variables\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n \ndiff --git a/doc/guides/rel_notes/release_19_02.rst b/doc/guides/rel_notes/release_19_02.rst\nindex e86ef9511..0b79918a9 100644\n--- a/doc/guides/rel_notes/release_19_02.rst\n+++ b/doc/guides/rel_notes/release_19_02.rst\n@@ -54,6 +54,12 @@ New Features\n Also, make sure to start the actual text at the margin.\n =========================================================\n \n+* **Added API to register external memory in DPDK.**\n+\n+ A new ``rte_extmem_register``/``rte_extmem_unregister`` API was added to allow\n+ chunks of external memory to be registered with DPDK without adding them to\n+ the malloc heap.\n+\n * **Updated the enic driver.**\n \n * Added support for ``RTE_ETH_DEV_CLOSE_REMOVE`` flag.\ndiff --git a/lib/librte_eal/common/eal_common_memory.c b/lib/librte_eal/common/eal_common_memory.c\nindex d47ea4938..ea43c1362 100644\n--- a/lib/librte_eal/common/eal_common_memory.c\n+++ b/lib/librte_eal/common/eal_common_memory.c\n@@ -24,6 +24,7 @@\n #include \"eal_memalloc.h\"\n #include \"eal_private.h\"\n #include \"eal_internal_cfg.h\"\n+#include \"malloc_heap.h\"\n \n /*\n * Try to mmap *size bytes in /dev/zero. If it is successful, return the\n@@ -775,6 +776,82 @@ rte_memseg_get_fd_offset(const struct rte_memseg *ms, size_t *offset)\n \treturn ret;\n }\n \n+int __rte_experimental\n+rte_extmem_register(void *va_addr, size_t len, rte_iova_t iova_addrs[],\n+\t\tunsigned int n_pages, size_t page_sz)\n+{\n+\tstruct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config;\n+\tunsigned int socket_id, n;\n+\tint ret = 0;\n+\n+\tif (va_addr == NULL || page_sz == 0 || len == 0 ||\n+\t\t\t!rte_is_power_of_2(page_sz) ||\n+\t\t\tRTE_ALIGN(len, page_sz) != len ||\n+\t\t\t((len / page_sz) != n_pages && iova_addrs != NULL) ||\n+\t\t\t!rte_is_aligned(va_addr, page_sz)) {\n+\t\trte_errno = EINVAL;\n+\t\treturn -1;\n+\t}\n+\trte_rwlock_write_lock(&mcfg->memory_hotplug_lock);\n+\n+\t/* make sure the segment doesn't already exist */\n+\tif (malloc_heap_find_external_seg(va_addr, len) != NULL) {\n+\t\trte_errno = EEXIST;\n+\t\tret = -1;\n+\t\tgoto unlock;\n+\t}\n+\n+\t/* get next available socket ID */\n+\tsocket_id = mcfg->next_socket_id;\n+\tif (socket_id > INT32_MAX) {\n+\t\tRTE_LOG(ERR, EAL, \"Cannot assign new socket ID's\\n\");\n+\t\trte_errno = ENOSPC;\n+\t\tret = -1;\n+\t\tgoto unlock;\n+\t}\n+\n+\t/* we can create a new memseg */\n+\tn = len / page_sz;\n+\tif (malloc_heap_create_external_seg(va_addr, iova_addrs, n,\n+\t\t\tpage_sz, \"extmem\", socket_id) == NULL) {\n+\t\tret = -1;\n+\t\tgoto unlock;\n+\t}\n+\n+\t/* memseg list successfully created - increment next socket ID */\n+\tmcfg->next_socket_id++;\n+unlock:\n+\trte_rwlock_write_unlock(&mcfg->memory_hotplug_lock);\n+\treturn ret;\n+}\n+\n+int __rte_experimental\n+rte_extmem_unregister(void *va_addr, size_t len)\n+{\n+\tstruct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config;\n+\tstruct rte_memseg_list *msl;\n+\tint ret = 0;\n+\n+\tif (va_addr == NULL || len == 0) {\n+\t\trte_errno = EINVAL;\n+\t\treturn -1;\n+\t}\n+\trte_rwlock_write_lock(&mcfg->memory_hotplug_lock);\n+\n+\t/* find our segment */\n+\tmsl = malloc_heap_find_external_seg(va_addr, len);\n+\tif (msl == NULL) {\n+\t\trte_errno = ENOENT;\n+\t\tret = -1;\n+\t\tgoto unlock;\n+\t}\n+\n+\tret = malloc_heap_destroy_external_seg(msl);\n+unlock:\n+\trte_rwlock_write_unlock(&mcfg->memory_hotplug_lock);\n+\treturn ret;\n+}\n+\n /* init memory subsystem */\n int\n rte_eal_memory_init(void)\ndiff --git a/lib/librte_eal/common/include/rte_memory.h b/lib/librte_eal/common/include/rte_memory.h\nindex d970825df..ff23fc2c1 100644\n--- a/lib/librte_eal/common/include/rte_memory.h\n+++ b/lib/librte_eal/common/include/rte_memory.h\n@@ -423,6 +423,69 @@ int __rte_experimental\n rte_memseg_get_fd_offset_thread_unsafe(const struct rte_memseg *ms,\n \t\tsize_t *offset);\n \n+/**\n+ * @warning\n+ * @b EXPERIMENTAL: this API may change without prior notice\n+ *\n+ * Register external memory chunk with DPDK.\n+ *\n+ * @note Using this API is mutually exclusive with ``rte_malloc`` family of\n+ * API's.\n+ *\n+ * @note This API will not perform any DMA mapping. It is expected that user\n+ * will do that themselves.\n+ *\n+ * @param va_addr\n+ * Start of virtual area to register. Must be aligned by ``page_sz``.\n+ * @param len\n+ * Length of virtual area to register. Must be aligned by ``page_sz``.\n+ * @param iova_addrs\n+ * Array of page IOVA addresses corresponding to each page in this memory\n+ * area. Can be NULL, in which case page IOVA addresses will be set to\n+ * RTE_BAD_IOVA.\n+ * @param n_pages\n+ * Number of elements in the iova_addrs array. Ignored if ``iova_addrs``\n+ * is NULL.\n+ * @param page_sz\n+ * Page size of the underlying memory\n+ *\n+ * @return\n+ * - 0 on success\n+ * - -1 in case of error, with rte_errno set to one of the following:\n+ * EINVAL - one of the parameters was invalid\n+ * EEXIST - memory chunk is already registered\n+ * ENOSPC - no more space in internal config to store a new memory chunk\n+ */\n+int __rte_experimental\n+rte_extmem_register(void *va_addr, size_t len, rte_iova_t iova_addrs[],\n+\t\tunsigned int n_pages, size_t page_sz);\n+\n+/**\n+ * @warning\n+ * @b EXPERIMENTAL: this API may change without prior notice\n+ *\n+ * Unregister external memory chunk with DPDK.\n+ *\n+ * @note Using this API is mutually exclusive with ``rte_malloc`` family of\n+ * API's.\n+ *\n+ * @note This API will not perform any DMA unmapping. It is expected that user\n+ * will do that themselves.\n+ *\n+ * @param va_addr\n+ * Start of virtual area to unregister\n+ * @param len\n+ * Length of virtual area to unregister\n+ *\n+ * @return\n+ * - 0 on success\n+ * - -1 in case of error, with rte_errno set to one of the following:\n+ * EINVAL - one of the parameters was invalid\n+ * ENOENT - memory chunk was not found\n+ */\n+int __rte_experimental\n+rte_extmem_unregister(void *va_addr, size_t len);\n+\n /**\n * Dump the physical memory layout to a file.\n *\ndiff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map\nindex 3fe78260d..593691a14 100644\n--- a/lib/librte_eal/rte_eal_version.map\n+++ b/lib/librte_eal/rte_eal_version.map\n@@ -296,6 +296,8 @@ EXPERIMENTAL {\n \trte_devargs_remove;\n \trte_devargs_type_count;\n \trte_eal_cleanup;\n+\trte_extmem_register;\n+\trte_extmem_unregister;\n \trte_fbarray_attach;\n \trte_fbarray_destroy;\n \trte_fbarray_detach;\n", "prefixes": [ "v2", "3/4" ] }