Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/136106/?format=api
{ "id": 136106, "url": "http://patchwork.dpdk.org/api/patches/136106/?format=api", "web_url": "http://patchwork.dpdk.org/project/dpdk/patch/1706103911-6907-1-git-send-email-rahulgupt@linux.microsoft.com/", "project": { "id": 1, "url": "http://patchwork.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<1706103911-6907-1-git-send-email-rahulgupt@linux.microsoft.com>", "list_archive_url": "https://inbox.dpdk.org/dev/1706103911-6907-1-git-send-email-rahulgupt@linux.microsoft.com", "date": "2024-01-24T13:45:11", "name": "[v4] eal: refactor rte_eal_init into sub-functions", "commit_ref": null, "pull_url": null, "state": "changes-requested", "archived": true, "hash": "af58c156b8bae4112046f76326e4dc8a57a36d0c", "submitter": { "id": 3218, "url": "http://patchwork.dpdk.org/api/people/3218/?format=api", "name": "Rahul Gupta", "email": "rahulgupt@linux.microsoft.com" }, "delegate": { "id": 24651, "url": "http://patchwork.dpdk.org/api/users/24651/?format=api", "username": "dmarchand", "first_name": "David", "last_name": "Marchand", "email": "david.marchand@redhat.com" }, "mbox": "http://patchwork.dpdk.org/project/dpdk/patch/1706103911-6907-1-git-send-email-rahulgupt@linux.microsoft.com/mbox/", "series": [ { "id": 30903, "url": "http://patchwork.dpdk.org/api/series/30903/?format=api", "web_url": "http://patchwork.dpdk.org/project/dpdk/list/?series=30903", "date": "2024-01-24T13:45:11", "name": "[v4] eal: refactor rte_eal_init into sub-functions", "version": 4, "mbox": "http://patchwork.dpdk.org/series/30903/mbox/" } ], "comments": "http://patchwork.dpdk.org/api/patches/136106/comments/", "check": "fail", "checks": "http://patchwork.dpdk.org/api/patches/136106/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from mails.dpdk.org (mails.dpdk.org [217.70.189.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id 2EC53439B6;\n\tWed, 24 Jan 2024 14:45:17 +0100 (CET)", "from mails.dpdk.org (localhost [127.0.0.1])\n\tby mails.dpdk.org (Postfix) with ESMTP id 1524742DE5;\n\tWed, 24 Jan 2024 14:45:17 +0100 (CET)", "from linux.microsoft.com (linux.microsoft.com [13.77.154.182])\n by mails.dpdk.org (Postfix) with ESMTP id 9787441133\n for <dev@dpdk.org>; Wed, 24 Jan 2024 14:45:15 +0100 (CET)", "by linux.microsoft.com (Postfix, from userid 1179)\n id EE0F020E34F4; Wed, 24 Jan 2024 05:45:14 -0800 (PST)" ], "DKIM-Filter": "OpenDKIM Filter v2.11.0 linux.microsoft.com EE0F020E34F4", "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com;\n s=default; t=1706103914;\n bh=t/I+vW7N2IDGYO9GwHh/nnx8gHFwcrTJ6NVgZ/Z2KkA=;\n h=From:To:Cc:Subject:Date:From;\n b=McG1O1+ywefdG4QMSDc72D3Nc8jWi7io1DdLpQUX/4G5PSJq0cl6B/yaNLig6XiPT\n GZHDn1/q6lzk06GsN0O8tDCuv0Pdk+jFdCzrVPHdgJAu0rfchIaG+UAsu/sebYyKVy\n V61ODNJuyecRgRcu7gOKQl1Q6NpfdphkUHg0PSSU=", "From": "Rahul Gupta <rahulgupt@linux.microsoft.com>", "To": "dev@dpdk.org, thomas@monjalon.net, bruce.richardson@intel.com,\n dmitry.kozliuk@gmail.com, stephen@networkplumber.org", "Cc": "sovaradh@linux.microsoft.com, okaya@kernel.org,\n sujithsankar@microsoft.com,\n sowmini.varadhan@microsoft.com, krathinavel@microsoft.com,\n rahulrgupta27@gmail.com, Rahul Gupta <rahulgupt@microsoft.com>", "Subject": "[dpdk-dev] [PATCH v4] eal: refactor rte_eal_init into sub-functions", "Date": "Wed, 24 Jan 2024 05:45:11 -0800", "Message-Id": "<1706103911-6907-1-git-send-email-rahulgupt@linux.microsoft.com>", "X-Mailer": "git-send-email 1.8.3.1", "MIME-Version": "1.0", "Content-Type": "text/plain; charset=UTF-8", "Content-Transfer-Encoding": "8bit", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org" }, "content": "From: Rahul Gupta <rahulgupt@microsoft.com>\n\nIn continuation to the following email, I am sending this patch.\n(https://inbox.dpdk.org/dev/20231110172523.GA17466@microsoft.com/)\n\nInitialization requires rte_eal_init + rte_pktmbuf_pool_create which\ncan consume a total time of 500-600 ms:\na) For many devices FLR may take a significant chunk of time\n (200-250 ms in our use-case), this FLR is triggered during device\n probe in rte_eal_init().\nb) rte_pktmbuf_pool_create() can consume up to 300-350 ms for\napplications that require huge memory.\n\nThis cost is incurred on each restart (which happens in our use-case\nduring binary updates for servicing).\nThis patch provides an optimization using pthreads that applications\ncan use and which can save 200-230ms.\n\nIn this patch, rte_eal_init() is refactored into two parts-\na) 1st part is dependent code ie- it’s a perquisite of the FLR and\n mempool creation. So this code needs to be executed before any\n pthreads. Its named as rte_eal_init_setup()\nb) 2nd part of code is independent code ie- it can execute in parallel\n to mempool creation in a pthread. Its named as rte_eal_init_async_setup().\n\nIn existing applications no changes are required unless they wish to leverage\nthe optimization.\n\nIf the application wants to leverage this optimization, then it needs to call\nrte_eal_init_async() (instead of call rte_eal_init()), then it can create a\nthread using rte_eal_remote_launch() to schedule a task it would like todo in\nparallel rte_eal_init_async_setup(), this task can be a mbuf pool creation\nusing- rte_pktmbuf_pool_create()\n\nAfter this, if next operations require completion of above task, then\nuser can use rte_eal_init_wait_async_setup_complete(),\nor if user wants to just check status of that thread, then use-\nrte_eal_init_async_setup_done()\n\n---\nv2: Address Stephen Hemminger's comment\n---\nv3: address support for single lcore\n---\nv4: address Brue Richardson and Stephen Hemminger comment\nExisting application need not do any changes if bootup optimization is not needed.\n\n app/test-pmd/testpmd.c | 24 ++++++++-\n lib/eal/include/rte_eal.h | 107 ++++++++++++++++++++++++++++++++++++++\n lib/eal/linux/eal.c | 62 ++++++++++++++++++++--\n lib/eal/version.map | 7 +++\n 4 files changed, 196 insertions(+), 4 deletions(-)", "diff": "diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c\nindex 9e4e99e53b..c8eb194f64 100644\n--- a/app/test-pmd/testpmd.c\n+++ b/app/test-pmd/testpmd.c\n@@ -4531,6 +4531,8 @@ main(int argc, char** argv)\n \tportid_t port_id;\n \tuint16_t count;\n \tint ret;\n+\tint lcore_id;\n+\tint main_lcore_id;\n \n #ifdef RTE_EXEC_ENV_WINDOWS\n \tsignal(SIGINT, signal_handler);\n@@ -4550,11 +4552,31 @@ main(int argc, char** argv)\n \t\trte_exit(EXIT_FAILURE, \"Cannot register log type\");\n \trte_log_set_level(testpmd_logtype, RTE_LOG_DEBUG);\n \n-\tdiag = rte_eal_init(argc, argv);\n+\tdiag = rte_eal_init_async(argc, argv);\n \tif (diag < 0)\n \t\trte_exit(EXIT_FAILURE, \"Cannot init EAL: %s\\n\",\n \t\t\t rte_strerror(rte_errno));\n \n+\tmain_lcore_id = rte_get_main_lcore();\n+\tlcore_id = rte_get_next_lcore(main_lcore_id, 0, 1);\n+\t/* Gives status of rte_eal_init_async() */\n+\tif (main_lcore_id != lcore_id)\n+\t\twhile (rte_eal_init_async_setup_done(lcore_id) == 0)\n+\t\t\t;\n+\n+\t/*\n+\t * Use rte_eal_init_wait_async_setup_complete() to get return value of\n+\t * rte_eal_init_async().\n+\t * Or\n+\t * if testpmd application don't want to know progress/status of\n+\t * rte_eal_init_async() and just want to wait till it finishes\n+\t * then use following function.\n+\t */\n+\tret = rte_eal_init_wait_async_setup_complete();\n+\tif (ret < 0)\n+\t\trte_exit(EXIT_FAILURE, \"Cannot init EAL: \"\n+\t\t\t \"rte_eal_init_async() failed: %s\\n\",\n+\t\t\t strerror(ret));\n \t/* allocate port structures, and init them */\n \tinit_port();\n \ndiff --git a/lib/eal/include/rte_eal.h b/lib/eal/include/rte_eal.h\nindex c2256f832e..6d7044b632 100644\n--- a/lib/eal/include/rte_eal.h\n+++ b/lib/eal/include/rte_eal.h\n@@ -111,6 +111,113 @@ int rte_eal_iopl_init(void);\n */\n int rte_eal_init(int argc, char **argv);\n \n+/**\n+ * Initialize the Environment Abstraction Layer (EAL).\n+ *\n+ * This function is to be executed on the MAIN lcore only, as soon\n+ * as possible in the application's main() function.\n+ * It puts the WORKER lcores in the WAIT state.\n+ *\n+ * @param argc\n+ * A non-negative value. If it is greater than 0, the array members\n+ * for argv[0] through argv[argc] (non-inclusive) shall contain pointers\n+ * to strings.\n+ * @param argv\n+ * An array of strings. The contents of the array, as well as the strings\n+ * which are pointed to by the array, may be modified by this function.\n+ * The program name pointer argv[0] is copied into the last parsed argv\n+ * so that argv[0] is still the same after deducing the parsed arguments.\n+ * @return\n+ * - On success, the number of parsed arguments, which is greater or\n+ * equal to zero. After the call to rte_eal_init_async(),\n+ * all arguments argv[x] with x < ret may have been modified by this\n+ * function call and should not be further interpreted by the\n+ * application. The EAL does not take any ownership of the memory used\n+ * for either the argv array, or its members.\n+ * - On failure, -1 and rte_errno is set to a value indicating the cause\n+ * for failure. In some instances, the application will need to be\n+ * restarted as part of clearing the issue.\n+ *\n+ * Error codes returned via rte_errno:\n+ * EACCES indicates a permissions issue.\n+ *\n+ * EAGAIN indicates either a bus or system resource was not available,\n+ * setup may be attempted again.\n+ *\n+ * EALREADY indicates that the rte_eal_init_async function has already been\n+ * called, and cannot be called again.\n+ *\n+ * EFAULT indicates the tailq configuration name was not found in\n+ * memory configuration.\n+ *\n+ * EINVAL indicates invalid parameters were passed as argv/argc.\n+ *\n+ * ENOMEM indicates failure likely caused by an out-of-memory condition.\n+ *\n+ * ENODEV indicates memory setup issues.\n+ *\n+ * ENOTSUP indicates that the EAL cannot initialize on this system.\n+ *\n+ * EPROTO indicates that the PCI bus is either not present, or is not\n+ * readable by the eal.\n+ *\n+ * ENOEXEC indicates that a service core failed to launch successfully.\n+ */\n+int rte_eal_init_async(int argc, char **argv);\n+\n+/**\n+ * Initialize the Environment Abstraction Layer (EAL): Initial setup\n+ *\n+ * Its called from rte_eal_init() on MAIN lcore only and must NOT be directly\n+ * called by user application.\n+ * The driver dependent code is present in this function, ie before calling any other\n+ * function eal library function this function must be complete successfully.\n+ *\n+ * return value is same as rte_eal_init().\n+ */\n+\n+__rte_experimental\n+int rte_eal_init_setup(int argc, char **argv);\n+\n+/**\n+ * Initialize the Environment Abstraction Layer (EAL): FLR and probe device\n+ *\n+ * Its thread is forked by rte_eal_init() and must NOT be directly called by user application.\n+ * Launched on next available worker lcore.\n+ * In this function initialisation needed for memory pool creation is completed,\n+ * so this code can be executed in parallel to non device related operations\n+ * like mbuf pool creation.\n+ *\n+ * return value is same as rte_eal_init().\n+ */\n+__rte_experimental\n+int rte_eal_init_async_setup(__attribute__((unused)) void *arg);\n+\n+/**\n+ * Initialize the Environment Abstraction Layer (EAL): Indication of rte_eal_init() completion\n+ *\n+ * It waits for rte_eal_init_async() to finish. It MUST be called from application,\n+ * when a thread join is needed. Typically application will call this function after\n+ * it performs all device independent operation (like mbuf pool creation) on initial lcore.\n+ *\n+ * return value is same as rte_eal_init().\n+ */\n+__rte_experimental\n+int rte_eal_init_wait_async_setup_complete(void);\n+\n+/**\n+ * Initialize the Environment Abstraction Layer (EAL): Indication of rte_eal_init() completion\n+ *\n+ * It shows status of rte_eal_init_async() ie the function is executing or completed.\n+ * It MUST be called from application,\n+ * Typically an application will call this function when it wants to know status of\n+ * rte_eal_init_async() (ie FLR and probe thread).\n+ *\n+ * return value is same as rte_eal_init().\n+ */\n+__rte_experimental\n+int rte_eal_init_async_setup_done(int lcore_id);\n+\n /**\n * Clean up the Environment Abstraction Layer (EAL)\n *\ndiff --git a/lib/eal/linux/eal.c b/lib/eal/linux/eal.c\nindex fd422f1f62..e86aac5980 100644\n--- a/lib/eal/linux/eal.c\n+++ b/lib/eal/linux/eal.c\n@@ -962,9 +962,8 @@ eal_worker_thread_create(unsigned int lcore_id)\n \treturn ret;\n }\n \n-/* Launch threads, called at application init(). */\n-int\n-rte_eal_init(int argc, char **argv)\n+__rte_experimental\n+int rte_eal_init_setup(int argc, char **argv)\n {\n \tint i, fctret, ret;\n \tstatic RTE_ATOMIC(uint32_t) run_once;\n@@ -1268,7 +1267,15 @@ rte_eal_init(int argc, char **argv)\n \t */\n \trte_eal_mp_remote_launch(sync_func, NULL, SKIP_MAIN);\n \trte_eal_mp_wait_lcore();\n+\treturn fctret;\n+}\n \n+__rte_experimental int\n+rte_eal_init_async_setup(__attribute__((unused)) void *arg)\n+{\n+\tint ret = 0;\n+\tstruct internal_config *internal_conf =\n+\t\teal_get_internal_configuration();\n \t/* initialize services so vdevs register service during bus_probe. */\n \tret = rte_service_init();\n \tif (ret) {\n@@ -1322,6 +1329,55 @@ rte_eal_init(int argc, char **argv)\n \n \teal_mcfg_complete();\n \n+\treturn 0;\n+}\n+\n+/*\n+ * waits until function executing on given lcore finishes.\n+ * returns value returned by the function executing on that lcore.\n+ */\n+__rte_experimental int\n+rte_eal_init_wait_async_setup_complete(void)\n+{\n+\tint lcore_id = -1;\n+\tlcore_id = rte_lcore_id();\n+\tlcore_id = rte_get_next_lcore(lcore_id, 0, 1);\n+\tint ret = rte_eal_wait_lcore(lcore_id);\n+\treturn ret;\n+}\n+\n+/*\n+ * returns current status of execution on a given lcore\n+ */\n+__rte_experimental int\n+rte_eal_init_async_setup_done(int lcore_id)\n+{\n+\tint ret = (lcore_config[lcore_id].state);\n+\treturn (ret == WAIT);\n+}\n+\n+/* Launch threads, called at application init(). */\n+int\n+rte_eal_init(int argc, char **argv)\n+{\n+\tint fctret = rte_eal_init_setup(argc, argv);\n+\tif (fctret < 0)\n+\t\treturn fctret;\n+\treturn rte_eal_init_async_setup(NULL);\n+}\n+\n+/* Launch threads, called at application init(). */\n+__rte_experimental int\n+rte_eal_init_async(int argc, char **argv)\n+{\n+\tint lcore_id;\n+\tint fctret = rte_eal_init_setup(argc, argv);\t/* initial lcore*/\n+\tif (fctret < 0)\n+\t\treturn fctret;\n+\tlcore_id = rte_lcore_id();\n+\tlcore_id = rte_get_next_lcore(lcore_id, 0, 1);\n+\t/* running on a worker lcore */\n+\trte_eal_remote_launch(rte_eal_init_async_setup, NULL, lcore_id);\n \treturn fctret;\n }\n \ndiff --git a/lib/eal/version.map b/lib/eal/version.map\nindex 5e0cd47c82..5e7ccb67c4 100644\n--- a/lib/eal/version.map\n+++ b/lib/eal/version.map\n@@ -393,6 +393,13 @@ EXPERIMENTAL {\n \t# added in 23.07\n \trte_memzone_max_get;\n \trte_memzone_max_set;\n+\n+\t# added in 24.01\n+\trte_eal_init_async;\n+\trte_eal_init_setup;\n+\trte_eal_init_async_setup;\n+\trte_eal_init_async_setup_done;\n+\trte_eal_init_wait_async_setup_complete;\n };\n \n INTERNAL {\n", "prefixes": [ "v4" ] }