Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/77607/?format=api
{ "id": 77607, "url": "http://patchwork.dpdk.org/api/patches/77607/?format=api", "web_url": "http://patchwork.dpdk.org/project/dpdk/patch/ccc9efa8619cb4a1f852e1ca8801a17266fc0d02.1600088628.git.anatoly.burakov@intel.com/", "project": { "id": 1, "url": "http://patchwork.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<ccc9efa8619cb4a1f852e1ca8801a17266fc0d02.1600088628.git.anatoly.burakov@intel.com>", "list_archive_url": "https://inbox.dpdk.org/dev/ccc9efa8619cb4a1f852e1ca8801a17266fc0d02.1600088628.git.anatoly.burakov@intel.com", "date": "2020-09-14T13:04:05", "name": "[v4] eal: detach memsegs on cleanup", "commit_ref": null, "pull_url": null, "state": "accepted", "archived": true, "hash": "91d3ca8b42f4ec0b87f55a65cca9ff00f61d6f7e", "submitter": { "id": 4, "url": "http://patchwork.dpdk.org/api/people/4/?format=api", "name": "Burakov, Anatoly", "email": "anatoly.burakov@intel.com" }, "delegate": { "id": 24651, "url": "http://patchwork.dpdk.org/api/users/24651/?format=api", "username": "dmarchand", "first_name": "David", "last_name": "Marchand", "email": "david.marchand@redhat.com" }, "mbox": "http://patchwork.dpdk.org/project/dpdk/patch/ccc9efa8619cb4a1f852e1ca8801a17266fc0d02.1600088628.git.anatoly.burakov@intel.com/mbox/", "series": [ { "id": 12191, "url": "http://patchwork.dpdk.org/api/series/12191/?format=api", "web_url": "http://patchwork.dpdk.org/project/dpdk/list/?series=12191", "date": "2020-09-14T13:04:05", "name": "[v4] eal: detach memsegs on cleanup", "version": 4, "mbox": "http://patchwork.dpdk.org/series/12191/mbox/" } ], "comments": "http://patchwork.dpdk.org/api/patches/77607/comments/", "check": "success", "checks": "http://patchwork.dpdk.org/api/patches/77607/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from dpdk.org (dpdk.org [92.243.14.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id 6CAD1A04C7;\n\tMon, 14 Sep 2020 15:06:31 +0200 (CEST)", "from [92.243.14.124] (localhost [127.0.0.1])\n\tby dpdk.org (Postfix) with ESMTP id 101E02BAB;\n\tMon, 14 Sep 2020 15:06:31 +0200 (CEST)", "from mga18.intel.com (mga18.intel.com [134.134.136.126])\n by dpdk.org (Postfix) with ESMTP id A8A51FFA\n for <dev@dpdk.org>; Mon, 14 Sep 2020 15:06:28 +0200 (CEST)", "from orsmga001.jf.intel.com ([10.7.209.18])\n by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;\n 14 Sep 2020 06:04:08 -0700", "from silpixa00399498.ir.intel.com (HELO\n silpixa00399498.ger.corp.intel.com) ([10.237.222.52])\n by orsmga001.jf.intel.com with ESMTP; 14 Sep 2020 06:04:05 -0700" ], "IronPort-SDR": [ "\n EQSm6kVCye7/SoAcUJqLqfIUxJ67xt1jj3Wrh5eCoSJbYmb2ig4DZOuoj2cuvj2bOqASK9RI01\n zCdWWZr0kvZw==", "\n 5NTLGF1ZLSt4OHKu5cuZTGqrZkyw2/mNsWtPaOzWGHY303UQJ83tfxkrwoXPRKsmEfQfogKkgl\n R+jl4pUyJ/7A==" ], "X-IronPort-AV": [ "E=McAfee;i=\"6000,8403,9743\"; a=\"146812067\"", "E=Sophos;i=\"5.76,426,1592895600\"; d=\"scan'208\";a=\"146812067\"", "E=Sophos;i=\"5.76,426,1592895600\"; d=\"scan'208\";a=\"379380587\"" ], "X-Amp-Result": "SKIPPED(no attachment in message)", "X-Amp-File-Uploaded": "False", "X-ExtLoop1": "1", "From": "Anatoly Burakov <anatoly.burakov@intel.com>", "To": "dev@dpdk.org", "Cc": "Bruce Richardson <bruce.richardson@intel.com>,\n Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>,\n Narcisa Ana Maria Vasile <navasile@linux.microsoft.com>,\n Dmitry Malloy <dmitrym@microsoft.com>,\n Pallavi Kadam <pallavi.kadam@intel.com>, stephen@networkplumber.org", "Date": "Mon, 14 Sep 2020 14:04:05 +0100", "Message-Id": "\n <ccc9efa8619cb4a1f852e1ca8801a17266fc0d02.1600088628.git.anatoly.burakov@intel.com>", "X-Mailer": "git-send-email 2.17.1", "In-Reply-To": "\n <b6ea726288d6e74c07bf4526cfcdb77eff443495.1600083548.git.anatoly.burakov@intel.com>", "References": "\n <b6ea726288d6e74c07bf4526cfcdb77eff443495.1600083548.git.anatoly.burakov@intel.com>", "Subject": "[dpdk-dev] [PATCH v4] eal: detach memsegs on cleanup", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "content": "Currently, we don't detach the shared memory on EAL cleanup, which\nleaves the page table descriptors still holding on to the file\ndescriptors as well as memory space occupied by them. Fix it by adding\nanother detach stage that closes the internal memory allocator resource\nreferences, detaches shared fbarrays and unmaps the shared mem config.\n\nBugzilla ID: 380\nBugzilla ID: 381\n\nSigned-off-by: Anatoly Burakov <anatoly.burakov@intel.com>\n---\n\nNotes:\n v4:\n - Fix Windows build\n \n v3:\n - Added missing Bugzilla ID for similar bug\n - Fixed deadlock on exit\n \n v2:\n - Fixed checkpatch warnings\n \n Not backporting to stable because this fix isn't critical but is rather\n \"nice to have\".\n\n lib/librte_eal/common/eal_common_memory.c | 54 ++++++++++++++\n lib/librte_eal/common/eal_memalloc.h | 3 +\n lib/librte_eal/common/eal_private.h | 7 ++\n lib/librte_eal/freebsd/eal.c | 2 +\n lib/librte_eal/freebsd/eal_memalloc.c | 5 ++\n lib/librte_eal/linux/eal.c | 2 +\n lib/librte_eal/linux/eal_memalloc.c | 85 +++++++++++++++++++++++\n lib/librte_eal/windows/eal.c | 3 +-\n lib/librte_eal/windows/eal_memalloc.c | 7 ++\n 9 files changed, 167 insertions(+), 1 deletion(-)", "diff": "diff --git a/lib/librte_eal/common/eal_common_memory.c b/lib/librte_eal/common/eal_common_memory.c\nindex 33917fa835..0e99986d3d 100644\n--- a/lib/librte_eal/common/eal_common_memory.c\n+++ b/lib/librte_eal/common/eal_common_memory.c\n@@ -1002,6 +1002,60 @@ rte_extmem_detach(void *va_addr, size_t len)\n \treturn sync_memory(va_addr, len, false);\n }\n \n+/* detach all EAL memory */\n+int\n+rte_eal_memory_detach(void)\n+{\n+\tstruct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config;\n+\tsize_t page_sz = rte_mem_page_size();\n+\tunsigned int i;\n+\n+\trte_rwlock_write_lock(&mcfg->memory_hotplug_lock);\n+\n+\t/* detach internal memory subsystem data first */\n+\tif (eal_memalloc_cleanup())\n+\t\tRTE_LOG(ERR, EAL, \"Could not release memory subsystem data\\n\");\n+\n+\tfor (i = 0; i < RTE_DIM(mcfg->memsegs); i++) {\n+\t\tstruct rte_memseg_list *msl = &mcfg->memsegs[i];\n+\n+\t\t/* skip uninitialized segments */\n+\t\tif (msl->base_va == NULL)\n+\t\t\tcontinue;\n+\t\t/*\n+\t\t * external segments are supposed to be detached at this point,\n+\t\t * but if they aren't, we can't really do anything about it,\n+\t\t * because if we skip them here, they'll become invalid after\n+\t\t * we unmap the memconfig anyway. however, if this is externally\n+\t\t * referenced memory, we have no business unmapping it.\n+\t\t */\n+\t\tif (!msl->external)\n+\t\t\tif (rte_mem_unmap(msl->base_va, msl->len) != 0)\n+\t\t\t\tRTE_LOG(ERR, EAL, \"Could not unmap memory: %s\\n\",\n+\t\t\t\t\t\tstrerror(errno));\n+\n+\t\t/*\n+\t\t * we are detaching the fbarray rather than destroying because\n+\t\t * other processes might still reference this fbarray, and we\n+\t\t * have no way of knowing if they still do.\n+\t\t */\n+\t\tif (rte_fbarray_detach(&msl->memseg_arr))\n+\t\t\tRTE_LOG(ERR, EAL, \"Could not detach fbarray: %s\\n\",\n+\t\t\t\t\trte_strerror(rte_errno));\n+\t}\n+\trte_rwlock_write_unlock(&mcfg->memory_hotplug_lock);\n+\n+\t/*\n+\t * we've detached the memseg lists, so we can unmap the shared mem\n+\t * config - we can't zero it out because it might still be referenced\n+\t * by other processes.\n+\t */\n+\trte_mem_unmap(mcfg, RTE_ALIGN(sizeof(*mcfg), page_sz));\n+\trte_eal_get_configuration()->mem_config = NULL;\n+\n+\treturn 0;\n+}\n+\n /* init memory subsystem */\n int\n rte_eal_memory_init(void)\ndiff --git a/lib/librte_eal/common/eal_memalloc.h b/lib/librte_eal/common/eal_memalloc.h\nindex e953cd84e6..ebc3a6f6c1 100644\n--- a/lib/librte_eal/common/eal_memalloc.h\n+++ b/lib/librte_eal/common/eal_memalloc.h\n@@ -93,4 +93,7 @@ eal_memalloc_get_seg_fd_offset(int list_idx, int seg_idx, size_t *offset);\n int\n eal_memalloc_init(void);\n \n+int\n+eal_memalloc_cleanup(void);\n+\n #endif /* EAL_MEMALLOC_H */\ndiff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h\nindex a6a6381567..84c4621768 100644\n--- a/lib/librte_eal/common/eal_private.h\n+++ b/lib/librte_eal/common/eal_private.h\n@@ -433,6 +433,13 @@ int rte_eal_hugepage_init(void);\n */\n int rte_eal_hugepage_attach(void);\n \n+/**\n+ * Detaches all memory mappings from a process.\n+ *\n+ * This function is private to the EAL.\n+ */\n+int rte_eal_memory_detach(void);\n+\n /**\n * Find a bus capable of identifying a device.\n *\ndiff --git a/lib/librte_eal/freebsd/eal.c b/lib/librte_eal/freebsd/eal.c\nindex 798add0b59..361593a005 100644\n--- a/lib/librte_eal/freebsd/eal.c\n+++ b/lib/librte_eal/freebsd/eal.c\n@@ -963,6 +963,8 @@ rte_eal_cleanup(void)\n \t\teal_get_internal_configuration();\n \trte_service_finalize();\n \trte_mp_channel_cleanup();\n+\t/* after this point, any DPDK pointers will become dangling */\n+\trte_eal_memory_detach();\n \trte_trace_save();\n \teal_trace_fini();\n \teal_cleanup_config(internal_conf);\ndiff --git a/lib/librte_eal/freebsd/eal_memalloc.c b/lib/librte_eal/freebsd/eal_memalloc.c\nindex 6893448db7..00ab02cb63 100644\n--- a/lib/librte_eal/freebsd/eal_memalloc.c\n+++ b/lib/librte_eal/freebsd/eal_memalloc.c\n@@ -74,6 +74,11 @@ eal_memalloc_get_seg_fd_offset(int list_idx __rte_unused,\n \treturn -ENOTSUP;\n }\n \n+int eal_memalloc_cleanup(void)\n+{\n+\treturn 0;\n+}\n+\n int\n eal_memalloc_init(void)\n {\ndiff --git a/lib/librte_eal/linux/eal.c b/lib/librte_eal/linux/eal.c\nindex 0960f01d05..eb69f4aedf 100644\n--- a/lib/librte_eal/linux/eal.c\n+++ b/lib/librte_eal/linux/eal.c\n@@ -1359,6 +1359,8 @@ rte_eal_cleanup(void)\n \t\trte_memseg_walk(mark_freeable, NULL);\n \trte_service_finalize();\n \trte_mp_channel_cleanup();\n+\t/* after this point, any DPDK pointers will become dangling */\n+\trte_eal_memory_detach();\n \trte_trace_save();\n \teal_trace_fini();\n \teal_cleanup_config(internal_conf);\ndiff --git a/lib/librte_eal/linux/eal_memalloc.c b/lib/librte_eal/linux/eal_memalloc.c\nindex db60e79975..e782734c98 100644\n--- a/lib/librte_eal/linux/eal_memalloc.c\n+++ b/lib/librte_eal/linux/eal_memalloc.c\n@@ -1418,6 +1418,31 @@ secondary_msl_create_walk(const struct rte_memseg_list *msl,\n \treturn 0;\n }\n \n+static int\n+secondary_msl_destroy_walk(const struct rte_memseg_list *msl,\n+\t\tvoid *arg __rte_unused)\n+{\n+\tstruct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config;\n+\tstruct rte_memseg_list *local_msl;\n+\tint msl_idx, ret;\n+\n+\tif (msl->external)\n+\t\treturn 0;\n+\n+\tmsl_idx = msl - mcfg->memsegs;\n+\tlocal_msl = &local_memsegs[msl_idx];\n+\n+\tret = rte_fbarray_destroy(&local_msl->memseg_arr);\n+\tif (ret < 0) {\n+\t\tRTE_LOG(ERR, EAL, \"Cannot destroy local memory map\\n\");\n+\t\treturn -1;\n+\t}\n+\tlocal_msl->base_va = NULL;\n+\tlocal_msl->len = 0;\n+\n+\treturn 0;\n+}\n+\n static int\n alloc_list(int list_idx, int len)\n {\n@@ -1450,6 +1475,34 @@ alloc_list(int list_idx, int len)\n \treturn 0;\n }\n \n+static int\n+destroy_list(int list_idx)\n+{\n+\tconst struct internal_config *internal_conf =\n+\t\t\teal_get_internal_configuration();\n+\n+\t/* single-file segments mode does not need fd list */\n+\tif (!internal_conf->single_file_segments) {\n+\t\tint *fds = fd_list[list_idx].fds;\n+\t\tint i;\n+\t\t/* go through each fd and ensure it's closed */\n+\t\tfor (i = 0; i < fd_list[list_idx].len; i++) {\n+\t\t\tif (fds[i] >= 0) {\n+\t\t\t\tclose(fds[i]);\n+\t\t\t\tfds[i] = -1;\n+\t\t\t}\n+\t\t}\n+\t\tfree(fds);\n+\t\tfd_list[list_idx].fds = NULL;\n+\t\tfd_list[list_idx].len = 0;\n+\t} else if (fd_list[list_idx].memseg_list_fd >= 0) {\n+\t\tclose(fd_list[list_idx].memseg_list_fd);\n+\t\tfd_list[list_idx].count = 0;\n+\t\tfd_list[list_idx].memseg_list_fd = -1;\n+\t}\n+\treturn 0;\n+}\n+\n static int\n fd_list_create_walk(const struct rte_memseg_list *msl,\n \t\tvoid *arg __rte_unused)\n@@ -1467,6 +1520,20 @@ fd_list_create_walk(const struct rte_memseg_list *msl,\n \treturn alloc_list(msl_idx, len);\n }\n \n+static int\n+fd_list_destroy_walk(const struct rte_memseg_list *msl, void *arg __rte_unused)\n+{\n+\tstruct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config;\n+\tint msl_idx;\n+\n+\tif (msl->external)\n+\t\treturn 0;\n+\n+\tmsl_idx = msl - mcfg->memsegs;\n+\n+\treturn destroy_list(msl_idx);\n+}\n+\n int\n eal_memalloc_set_seg_fd(int list_idx, int seg_idx, int fd)\n {\n@@ -1603,6 +1670,24 @@ eal_memalloc_get_seg_fd_offset(int list_idx, int seg_idx, size_t *offset)\n \treturn 0;\n }\n \n+int\n+eal_memalloc_cleanup(void)\n+{\n+\t/* close all remaining fd's - these are per-process, so it's safe */\n+\tif (rte_memseg_list_walk_thread_unsafe(fd_list_destroy_walk, NULL))\n+\t\treturn -1;\n+\n+\t/* destroy the shadow page table if we're a secondary process */\n+\tif (rte_eal_process_type() == RTE_PROC_PRIMARY)\n+\t\treturn 0;\n+\n+\tif (rte_memseg_list_walk_thread_unsafe(secondary_msl_destroy_walk,\n+\t\t\tNULL))\n+\t\treturn -1;\n+\n+\treturn 0;\n+}\n+\n int\n eal_memalloc_init(void)\n {\ndiff --git a/lib/librte_eal/windows/eal.c b/lib/librte_eal/windows/eal.c\nindex e50601dd36..d5db007717 100644\n--- a/lib/librte_eal/windows/eal.c\n+++ b/lib/librte_eal/windows/eal.c\n@@ -250,7 +250,8 @@ rte_eal_cleanup(void)\n {\n \tstruct internal_config *internal_conf =\n \t\teal_get_internal_configuration();\n-\n+\t/* after this point, any DPDK pointers will become dangling */\n+\trte_eal_memory_detach();\n \teal_cleanup_config(internal_conf);\n \treturn 0;\n }\ndiff --git a/lib/librte_eal/windows/eal_memalloc.c b/lib/librte_eal/windows/eal_memalloc.c\nindex d8cae3ebc1..85a9712cea 100644\n--- a/lib/librte_eal/windows/eal_memalloc.c\n+++ b/lib/librte_eal/windows/eal_memalloc.c\n@@ -437,6 +437,13 @@ eal_memalloc_sync_with_primary(void)\n \treturn -1;\n }\n \n+int\n+eal_memalloc_cleanup(void)\n+{\n+\t/* not implemented */\n+\treturn 0;\n+}\n+\n int\n eal_memalloc_init(void)\n {\n", "prefixes": [ "v4" ] }