From patchwork Fri Apr 1 08:44:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 109064 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9A747A0503; Fri, 1 Apr 2022 10:44:41 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3C3D94067E; Fri, 1 Apr 2022 10:44:41 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 1DDEA4014F for ; Fri, 1 Apr 2022 10:44:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1648802678; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2hYKblygkmv+J96E2yaKOonj8z+apdTI1+tKw3h58AY=; b=iClUztND5CXdCW0fYZGwPfNbiPCXtBAYbDI1G0zKUzyyzknyjuwjSLcQ+Xtj1556sDj05s QrisltJ7jBifbSSJ3EyS0PFASo7ax/7tdf5S47nQ+z0pdLEBAa0HOKg795cXpSBbuvIjJ1 qDwkVyUev3PRkDrOp8SxW6p+PScqIjc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-176--7ut9uddP9elGEOSSu5p3A-1; Fri, 01 Apr 2022 04:44:33 -0400 X-MC-Unique: -7ut9uddP9elGEOSSu5p3A-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F0AEB185A794; Fri, 1 Apr 2022 08:44:32 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.195.27]) by smtp.corp.redhat.com (Postfix) with ESMTP id 77AEA401DBC; Fri, 1 Apr 2022 08:44:31 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: thomas@monjalon.net, Bruce Richardson , Dmitry Kozlyuk , Narcisa Ana Maria Vasile , Dmitry Malloy , Pallavi Kadam Subject: [PATCH v2 1/2] eal: cleanup lcore ID hand-over Date: Fri, 1 Apr 2022 10:44:18 +0200 Message-Id: <20220401084419.21150-1-david.marchand@redhat.com> In-Reply-To: <20220323093001.20618-1-david.marchand@redhat.com> References: <20220323093001.20618-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org So far, a worker thread has been using its thread_id to discover which lcore has been assigned to it. On the other hand, as noted by Tyler, the pthread API does not strictly guarantee that a new thread won't start running eal_thread_loop before pthread_create writes to &lcore_config[xx].thread_id. Though all OS implementations supported in DPDK (recently) ensure this property, it is more robust to have the main thread directly pass the worker thread lcore. Signed-off-by: David Marchand --- lib/eal/common/eal_thread.h | 4 ++-- lib/eal/freebsd/eal.c | 2 +- lib/eal/freebsd/eal_thread.c | 16 +++------------- lib/eal/linux/eal.c | 2 +- lib/eal/linux/eal_thread.c | 17 ++++------------- lib/eal/windows/eal.c | 2 +- lib/eal/windows/eal_thread.c | 23 +++++++---------------- 7 files changed, 19 insertions(+), 47 deletions(-) diff --git a/lib/eal/common/eal_thread.h b/lib/eal/common/eal_thread.h index 4a49117be8..3c84efd553 100644 --- a/lib/eal/common/eal_thread.h +++ b/lib/eal/common/eal_thread.h @@ -8,10 +8,10 @@ #include /** - * basic loop of thread, called for each thread by eal_init(). + * Basic loop of EAL thread, called for each worker thread by rte_eal_init(). * * @param arg - * opaque pointer + * The lcore_id (passed as an integer) of this worker thread. */ __rte_noreturn void *eal_thread_loop(void *arg); diff --git a/lib/eal/freebsd/eal.c b/lib/eal/freebsd/eal.c index 71993fe25b..e233e57184 100644 --- a/lib/eal/freebsd/eal.c +++ b/lib/eal/freebsd/eal.c @@ -813,7 +813,7 @@ rte_eal_init(int argc, char **argv) /* create a thread for each lcore */ ret = pthread_create(&lcore_config[i].thread_id, NULL, - eal_thread_loop, NULL); + eal_thread_loop, (void *)(uintptr_t)i); if (ret != 0) rte_panic("Cannot create thread\n"); diff --git a/lib/eal/freebsd/eal_thread.c b/lib/eal/freebsd/eal_thread.c index 3b18030d73..9044d70ef7 100644 --- a/lib/eal/freebsd/eal_thread.c +++ b/lib/eal/freebsd/eal_thread.c @@ -76,25 +76,15 @@ rte_eal_remote_launch(int (*f)(void *), void *arg, unsigned worker_id) /* main loop of threads */ __rte_noreturn void * -eal_thread_loop(__rte_unused void *arg) +eal_thread_loop(void *arg) { + unsigned int lcore_id = (uintptr_t)arg; char c; int n, ret; unsigned lcore_id; - pthread_t thread_id; int m2w, w2m; char cpuset[RTE_CPU_AFFINITY_STR_LEN]; - thread_id = pthread_self(); - - /* retrieve our lcore_id from the configuration structure */ - RTE_LCORE_FOREACH_WORKER(lcore_id) { - if (thread_id == lcore_config[lcore_id].thread_id) - break; - } - if (lcore_id == RTE_MAX_LCORE) - rte_panic("cannot retrieve lcore id\n"); - m2w = lcore_config[lcore_id].pipe_main2worker[0]; w2m = lcore_config[lcore_id].pipe_worker2main[1]; @@ -102,7 +92,7 @@ eal_thread_loop(__rte_unused void *arg) ret = eal_thread_dump_current_affinity(cpuset, sizeof(cpuset)); RTE_LOG(DEBUG, EAL, "lcore %u is ready (tid=%p;cpuset=[%s%s])\n", - lcore_id, thread_id, cpuset, ret == 0 ? "" : "..."); + lcore_id, pthread_self(), cpuset, ret == 0 ? "" : "..."); rte_eal_trace_thread_lcore_ready(lcore_id, cpuset); diff --git a/lib/eal/linux/eal.c b/lib/eal/linux/eal.c index 025e5cc10d..98c6838026 100644 --- a/lib/eal/linux/eal.c +++ b/lib/eal/linux/eal.c @@ -1147,7 +1147,7 @@ rte_eal_init(int argc, char **argv) /* create a thread for each lcore */ ret = pthread_create(&lcore_config[i].thread_id, NULL, - eal_thread_loop, NULL); + eal_thread_loop, (void *)(uintptr_t)i); if (ret != 0) rte_panic("Cannot create thread\n"); diff --git a/lib/eal/linux/eal_thread.c b/lib/eal/linux/eal_thread.c index fa6cd7e2c4..26b0a7d48a 100644 --- a/lib/eal/linux/eal_thread.c +++ b/lib/eal/linux/eal_thread.c @@ -70,24 +70,14 @@ rte_eal_remote_launch(int (*f)(void *), void *arg, unsigned int worker_id) /* main loop of threads */ __rte_noreturn void * -eal_thread_loop(__rte_unused void *arg) +eal_thread_loop(void *arg) { + unsigned int lcore_id = (uintptr_t)arg; char c; int n, ret; - unsigned lcore_id; - pthread_t thread_id; int m2w, w2m; char cpuset[RTE_CPU_AFFINITY_STR_LEN]; - thread_id = pthread_self(); - - /* retrieve our lcore_id from the configuration structure */ - RTE_LCORE_FOREACH_WORKER(lcore_id) { - if (thread_id == lcore_config[lcore_id].thread_id) - break; - } - if (lcore_id == RTE_MAX_LCORE) - rte_panic("cannot retrieve lcore id\n"); m2w = lcore_config[lcore_id].pipe_main2worker[0]; w2m = lcore_config[lcore_id].pipe_worker2main[1]; @@ -96,7 +86,8 @@ eal_thread_loop(__rte_unused void *arg) ret = eal_thread_dump_current_affinity(cpuset, sizeof(cpuset)); RTE_LOG(DEBUG, EAL, "lcore %u is ready (tid=%zx;cpuset=[%s%s])\n", - lcore_id, (uintptr_t)thread_id, cpuset, ret == 0 ? "" : "..."); + lcore_id, (uintptr_t)pthread_self(), cpuset, + ret == 0 ? "" : "..."); rte_eal_trace_thread_lcore_ready(lcore_id, cpuset); diff --git a/lib/eal/windows/eal.c b/lib/eal/windows/eal.c index ca3c41aaa7..1874f9f6d7 100644 --- a/lib/eal/windows/eal.c +++ b/lib/eal/windows/eal.c @@ -420,7 +420,7 @@ rte_eal_init(int argc, char **argv) lcore_config[i].state = WAIT; /* create a thread for each lcore */ - if (eal_thread_create(&lcore_config[i].thread_id) != 0) + if (eal_thread_create(&lcore_config[i].thread_id, i) != 0) rte_panic("Cannot create thread\n"); ret = pthread_setaffinity_np(lcore_config[i].thread_id, sizeof(rte_cpuset_t), &lcore_config[i].cpuset); diff --git a/lib/eal/windows/eal_thread.c b/lib/eal/windows/eal_thread.c index ff84cb42af..95e96548de 100644 --- a/lib/eal/windows/eal_thread.c +++ b/lib/eal/windows/eal_thread.c @@ -63,32 +63,21 @@ rte_eal_remote_launch(lcore_function_t *f, void *arg, unsigned int worker_id) /* main loop of threads */ void * -eal_thread_loop(void *arg __rte_unused) +eal_thread_loop(void *arg) { + unsigned int lcore_id = (uintptr_t)arg; char c; int n, ret; - unsigned int lcore_id; - pthread_t thread_id; int m2w, w2m; char cpuset[RTE_CPU_AFFINITY_STR_LEN]; - thread_id = pthread_self(); - - /* retrieve our lcore_id from the configuration structure */ - RTE_LCORE_FOREACH_WORKER(lcore_id) { - if (thread_id == lcore_config[lcore_id].thread_id) - break; - } - if (lcore_id == RTE_MAX_LCORE) - rte_panic("cannot retrieve lcore id\n"); - m2w = lcore_config[lcore_id].pipe_main2worker[0]; w2m = lcore_config[lcore_id].pipe_worker2main[1]; __rte_thread_init(lcore_id, &lcore_config[lcore_id].cpuset); RTE_LOG(DEBUG, EAL, "lcore %u is ready (tid=%zx;cpuset=[%s])\n", - lcore_id, (uintptr_t)thread_id, cpuset); + lcore_id, (uintptr_t)pthread_self(), cpuset); /* read on our pipe to get commands */ while (1) { @@ -144,13 +133,15 @@ eal_thread_loop(void *arg __rte_unused) /* function to create threads */ int -eal_thread_create(pthread_t *thread) +eal_thread_create(pthread_t *thread, unsigned int lcore_id) { HANDLE th; th = CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)(ULONG_PTR)eal_thread_loop, - NULL, CREATE_SUSPENDED, (LPDWORD)thread); + (LPVOID)(uintptr_t)lcore_id, + CREATE_SUSPENDED, + (LPDWORD)thread); if (!th) return -1; From patchwork Fri Apr 1 08:44:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 109065 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 252B8A0503; Fri, 1 Apr 2022 10:44:46 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1324C42913; Fri, 1 Apr 2022 10:44:46 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 3EEF64014F for ; Fri, 1 Apr 2022 10:44:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1648802683; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CkqMpHhtYJ4LA4AESJyu6TPdJ323s2JUXixitgZtYVM=; b=HW8F2REF2NnjsJnI0m0XhU7CQQy79fEfG0JwWNNwqDkazocVVpn0EG3I6nWdrPwfbicsvh cmUnJSqzREuyh2RIG+csbaXW18iQfCMWCeAUM2tMN2tmEzVl+0qjPFEs0cVLtApdyLsDSB rlssCrypNnRdmq22zKc4J9cT+81cbMk= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-529-xRQ6RSjXOnGDNArFtAO9pw-1; Fri, 01 Apr 2022 04:44:39 -0400 X-MC-Unique: xRQ6RSjXOnGDNArFtAO9pw-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2A7B42A2AD40; Fri, 1 Apr 2022 08:44:39 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.195.27]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3170B401DB0; Fri, 1 Apr 2022 08:44:37 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: thomas@monjalon.net, =?utf-8?q?Morten_Br=C3=B8rup?= , Tyler Retzlaff , Bruce Richardson , Dmitry Kozlyuk , Narcisa Ana Maria Vasile , Dmitry Malloy , Pallavi Kadam Subject: [PATCH v2 2/2] eal: factorize lcore main loop Date: Fri, 1 Apr 2022 10:44:19 +0200 Message-Id: <20220401084419.21150-2-david.marchand@redhat.com> In-Reply-To: <20220401084419.21150-1-david.marchand@redhat.com> References: <20220323093001.20618-1-david.marchand@redhat.com> <20220401084419.21150-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org All OS implementations provide the same main loop. Introduce helpers (shared for Linux and FreeBSD) to handle synchronisation between main and threads and factorize the rest as common code. Thread id are now logged as string in a common format across OS. Signed-off-by: David Marchand Acked-by: Morten Brørup Acked-by: Tyler Retzlaff --- Changes since v1: - rebased on top of Tyler fix for Windows and new cleanup, --- lib/eal/common/eal_common_launch.c | 36 +++++++- lib/eal/common/eal_common_thread.c | 63 ++++++++++++++ lib/eal/common/eal_thread.h | 27 ++++++ lib/eal/freebsd/eal.c | 7 +- lib/eal/freebsd/eal_thread.c | 128 ---------------------------- lib/eal/linux/eal.c | 4 +- lib/eal/linux/eal_thread.c | 130 +---------------------------- lib/eal/unix/eal_unix_thread.c | 63 ++++++++++++++ lib/eal/unix/meson.build | 5 +- lib/eal/windows/eal_thread.c | 126 +++++++--------------------- 10 files changed, 227 insertions(+), 362 deletions(-) create mode 100644 lib/eal/unix/eal_unix_thread.c diff --git a/lib/eal/common/eal_common_launch.c b/lib/eal/common/eal_common_launch.c index 9f393b9bda..5770803172 100644 --- a/lib/eal/common/eal_common_launch.c +++ b/lib/eal/common/eal_common_launch.c @@ -5,11 +5,13 @@ #include #include +#include #include #include #include #include "eal_private.h" +#include "eal_thread.h" /* * Wait until a lcore finished its job. @@ -18,12 +20,44 @@ int rte_eal_wait_lcore(unsigned worker_id) { while (__atomic_load_n(&lcore_config[worker_id].state, - __ATOMIC_ACQUIRE) != WAIT) + __ATOMIC_ACQUIRE) != WAIT) rte_pause(); return lcore_config[worker_id].ret; } +/* + * Send a message to a worker lcore identified by worker_id to call a + * function f with argument arg. Once the execution is done, the + * remote lcore switches to WAIT state. + */ +int +rte_eal_remote_launch(int (*f)(void *), void *arg, unsigned int worker_id) +{ + int rc = -EBUSY; + + /* Check if the worker is in 'WAIT' state. Use acquire order + * since 'state' variable is used as the guard variable. + */ + if (__atomic_load_n(&lcore_config[worker_id].state, + __ATOMIC_ACQUIRE) != WAIT) + goto finish; + + lcore_config[worker_id].arg = arg; + /* Ensure that all the memory operations are completed + * before the worker thread starts running the function. + * Use worker thread function as the guard variable. + */ + __atomic_store_n(&lcore_config[worker_id].f, f, __ATOMIC_RELEASE); + + eal_thread_wake_worker(worker_id); + rc = 0; + +finish: + rte_eal_trace_thread_remote_launch(f, arg, worker_id, rc); + return rc; +} + /* * Check that every WORKER lcores are in WAIT state, then call * rte_eal_remote_launch() for all of them. If call_main is true diff --git a/lib/eal/common/eal_common_thread.c b/lib/eal/common/eal_common_thread.c index 684bea166c..962b7e9ac4 100644 --- a/lib/eal/common/eal_common_thread.c +++ b/lib/eal/common/eal_common_thread.c @@ -9,6 +9,7 @@ #include #include +#include #include #include #include @@ -163,6 +164,68 @@ __rte_thread_uninit(void) RTE_PER_LCORE(_lcore_id) = LCORE_ID_ANY; } +/* main loop of threads */ +__rte_noreturn void * +eal_thread_loop(void *arg) +{ + unsigned int lcore_id = (uintptr_t)arg; + char cpuset[RTE_CPU_AFFINITY_STR_LEN]; + int ret; + + __rte_thread_init(lcore_id, &lcore_config[lcore_id].cpuset); + + ret = eal_thread_dump_current_affinity(cpuset, sizeof(cpuset)); + RTE_LOG(DEBUG, EAL, "lcore %u is ready (tid=%zx;cpuset=[%s%s])\n", + lcore_id, (uintptr_t)pthread_self(), cpuset, + ret == 0 ? "" : "..."); + + rte_eal_trace_thread_lcore_ready(lcore_id, cpuset); + + /* read on our pipe to get commands */ + while (1) { + lcore_function_t *f; + void *fct_arg; + + eal_thread_wait_command(); + + /* Set the state to 'RUNNING'. Use release order + * since 'state' variable is used as the guard variable. + */ + __atomic_store_n(&lcore_config[lcore_id].state, RUNNING, + __ATOMIC_RELEASE); + + eal_thread_ack_command(); + + /* Load 'f' with acquire order to ensure that + * the memory operations from the main thread + * are accessed only after update to 'f' is visible. + * Wait till the update to 'f' is visible to the worker. + */ + while ((f = __atomic_load_n(&lcore_config[lcore_id].f, + __ATOMIC_ACQUIRE)) == NULL) + rte_pause(); + + /* call the function and store the return value */ + fct_arg = lcore_config[lcore_id].arg; + ret = f(fct_arg); + lcore_config[lcore_id].ret = ret; + lcore_config[lcore_id].f = NULL; + lcore_config[lcore_id].arg = NULL; + + /* Store the state with release order to ensure that + * the memory operations from the worker thread + * are completed before the state is updated. + * Use 'state' as the guard variable. + */ + __atomic_store_n(&lcore_config[lcore_id].state, WAIT, + __ATOMIC_RELEASE); + } + + /* never reached */ + /* pthread_exit(NULL); */ + /* return NULL; */ +} + enum __rte_ctrl_thread_status { CTRL_THREAD_LAUNCHING, /* Yet to call pthread_create function */ CTRL_THREAD_RUNNING, /* Control thread is running successfully */ diff --git a/lib/eal/common/eal_thread.h b/lib/eal/common/eal_thread.h index 3c84efd553..ca3378d463 100644 --- a/lib/eal/common/eal_thread.h +++ b/lib/eal/common/eal_thread.h @@ -58,4 +58,31 @@ eal_thread_dump_affinity(rte_cpuset_t *cpuset, char *str, unsigned int size); int eal_thread_dump_current_affinity(char *str, unsigned int size); +/** + * Called by the main thread to wake up a worker in 'WAIT' state. + * This function blocks until the worker acknowledge it started processing a + * new command. + * This function is private to EAL. + * + * @param worker_id + * The lcore_id of a worker thread. + */ +void +eal_thread_wake_worker(unsigned int worker_id); + +/** + * Called by a worker thread to sleep after entering 'WAIT' state. + * This function is private to EAL. + */ +void +eal_thread_wait_command(void); + +/** + * Called by a worker thread to acknowledge new command after leaving 'WAIT' + * state. + * This function is private to EAL. + */ +void +eal_thread_ack_command(void); + #endif /* EAL_THREAD_H */ diff --git a/lib/eal/freebsd/eal.c b/lib/eal/freebsd/eal.c index e233e57184..a6b20960f2 100644 --- a/lib/eal/freebsd/eal.c +++ b/lib/eal/freebsd/eal.c @@ -579,7 +579,6 @@ int rte_eal_init(int argc, char **argv) { int i, fctret, ret; - pthread_t thread_id; static uint32_t run_once; uint32_t has_run = 0; char cpuset[RTE_CPU_AFFINITY_STR_LEN]; @@ -604,8 +603,6 @@ rte_eal_init(int argc, char **argv) return -1; } - thread_id = pthread_self(); - eal_reset_internal_config(internal_conf); /* clone argv to report out later in telemetry */ @@ -794,8 +791,8 @@ rte_eal_init(int argc, char **argv) ret = eal_thread_dump_current_affinity(cpuset, sizeof(cpuset)); - RTE_LOG(DEBUG, EAL, "Main lcore %u is ready (tid=%p;cpuset=[%s%s])\n", - config->main_lcore, thread_id, cpuset, + RTE_LOG(DEBUG, EAL, "Main lcore %u is ready (tid=%zx;cpuset=[%s%s])\n", + config->main_lcore, (uintptr_t)pthread_self(), cpuset, ret == 0 ? "" : "..."); RTE_LCORE_FOREACH_WORKER(i) { diff --git a/lib/eal/freebsd/eal_thread.c b/lib/eal/freebsd/eal_thread.c index 9044d70ef7..ab81b527bc 100644 --- a/lib/eal/freebsd/eal_thread.c +++ b/lib/eal/freebsd/eal_thread.c @@ -20,138 +20,10 @@ #include #include #include -#include #include "eal_private.h" #include "eal_thread.h" -/* - * Send a message to a worker lcore identified by worker_id to call a - * function f with argument arg. Once the execution is done, the - * remote lcore switches to WAIT state. - */ -int -rte_eal_remote_launch(int (*f)(void *), void *arg, unsigned worker_id) -{ - int n; - char c = 0; - int m2w = lcore_config[worker_id].pipe_main2worker[1]; - int w2m = lcore_config[worker_id].pipe_worker2main[0]; - int rc = -EBUSY; - - /* Check if the worker is in 'WAIT' state. Use acquire order - * since 'state' variable is used as the guard variable. - */ - if (__atomic_load_n(&lcore_config[worker_id].state, - __ATOMIC_ACQUIRE) != WAIT) - goto finish; - - lcore_config[worker_id].arg = arg; - /* Ensure that all the memory operations are completed - * before the worker thread starts running the function. - * Use worker thread function as the guard variable. - */ - __atomic_store_n(&lcore_config[worker_id].f, f, __ATOMIC_RELEASE); - - /* send message */ - n = 0; - while (n == 0 || (n < 0 && errno == EINTR)) - n = write(m2w, &c, 1); - if (n < 0) - rte_panic("cannot write on configuration pipe\n"); - - /* wait ack */ - do { - n = read(w2m, &c, 1); - } while (n < 0 && errno == EINTR); - - if (n <= 0) - rte_panic("cannot read on configuration pipe\n"); - - rc = 0; -finish: - rte_eal_trace_thread_remote_launch(f, arg, worker_id, rc); - return rc; -} - -/* main loop of threads */ -__rte_noreturn void * -eal_thread_loop(void *arg) -{ - unsigned int lcore_id = (uintptr_t)arg; - char c; - int n, ret; - unsigned lcore_id; - int m2w, w2m; - char cpuset[RTE_CPU_AFFINITY_STR_LEN]; - - m2w = lcore_config[lcore_id].pipe_main2worker[0]; - w2m = lcore_config[lcore_id].pipe_worker2main[1]; - - __rte_thread_init(lcore_id, &lcore_config[lcore_id].cpuset); - - ret = eal_thread_dump_current_affinity(cpuset, sizeof(cpuset)); - RTE_LOG(DEBUG, EAL, "lcore %u is ready (tid=%p;cpuset=[%s%s])\n", - lcore_id, pthread_self(), cpuset, ret == 0 ? "" : "..."); - - rte_eal_trace_thread_lcore_ready(lcore_id, cpuset); - - /* read on our pipe to get commands */ - while (1) { - lcore_function_t *f; - void *fct_arg; - - /* wait command */ - do { - n = read(m2w, &c, 1); - } while (n < 0 && errno == EINTR); - - if (n <= 0) - rte_panic("cannot read on configuration pipe\n"); - - /* Set the state to 'RUNNING'. Use release order - * since 'state' variable is used as the guard variable. - */ - __atomic_store_n(&lcore_config[lcore_id].state, RUNNING, - __ATOMIC_RELEASE); - - /* send ack */ - n = 0; - while (n == 0 || (n < 0 && errno == EINTR)) - n = write(w2m, &c, 1); - if (n < 0) - rte_panic("cannot write on configuration pipe\n"); - - /* Load 'f' with acquire order to ensure that - * the memory operations from the main thread - * are accessed only after update to 'f' is visible. - * Wait till the update to 'f' is visible to the worker. - */ - while ((f = __atomic_load_n(&lcore_config[lcore_id].f, - __ATOMIC_ACQUIRE)) == NULL) - rte_pause(); - - /* call the function and store the return value */ - fct_arg = lcore_config[lcore_id].arg; - ret = f(fct_arg); - lcore_config[lcore_id].ret = ret; - lcore_config[lcore_id].f = NULL; - lcore_config[lcore_id].arg = NULL; - - /* Store the state with release order to ensure that - * the memory operations from the worker thread - * are completed before the state is updated. - * Use 'state' as the guard variable. - */ - __atomic_store_n(&lcore_config[lcore_id].state, WAIT, - __ATOMIC_RELEASE); - } - - /* never reached */ - /* pthread_exit(NULL); */ - /* return NULL; */ -} - /* require calling thread tid by gettid() */ int rte_sys_gettid(void) { diff --git a/lib/eal/linux/eal.c b/lib/eal/linux/eal.c index 98c6838026..1ef263434a 100644 --- a/lib/eal/linux/eal.c +++ b/lib/eal/linux/eal.c @@ -862,7 +862,6 @@ int rte_eal_init(int argc, char **argv) { int i, fctret, ret; - pthread_t thread_id; static uint32_t run_once; uint32_t has_run = 0; const char *p; @@ -890,7 +889,6 @@ rte_eal_init(int argc, char **argv) p = strrchr(argv[0], '/'); strlcpy(logid, p ? p + 1 : argv[0], sizeof(logid)); - thread_id = pthread_self(); eal_reset_internal_config(internal_conf); @@ -1129,7 +1127,7 @@ rte_eal_init(int argc, char **argv) ret = eal_thread_dump_current_affinity(cpuset, sizeof(cpuset)); RTE_LOG(DEBUG, EAL, "Main lcore %u is ready (tid=%zx;cpuset=[%s%s])\n", - config->main_lcore, (uintptr_t)thread_id, cpuset, + config->main_lcore, (uintptr_t)pthread_self(), cpuset, ret == 0 ? "" : "..."); RTE_LCORE_FOREACH_WORKER(i) { diff --git a/lib/eal/linux/eal_thread.c b/lib/eal/linux/eal_thread.c index 26b0a7d48a..820cc905e0 100644 --- a/lib/eal/linux/eal_thread.c +++ b/lib/eal/linux/eal_thread.c @@ -14,139 +14,11 @@ #include #include #include -#include +#include #include "eal_private.h" #include "eal_thread.h" -/* - * Send a message to a worker lcore identified by worker_id to call a - * function f with argument arg. Once the execution is done, the - * remote lcore switches to WAIT state. - */ -int -rte_eal_remote_launch(int (*f)(void *), void *arg, unsigned int worker_id) -{ - int n; - char c = 0; - int m2w = lcore_config[worker_id].pipe_main2worker[1]; - int w2m = lcore_config[worker_id].pipe_worker2main[0]; - int rc = -EBUSY; - - /* Check if the worker is in 'WAIT' state. Use acquire order - * since 'state' variable is used as the guard variable. - */ - if (__atomic_load_n(&lcore_config[worker_id].state, - __ATOMIC_ACQUIRE) != WAIT) - goto finish; - - lcore_config[worker_id].arg = arg; - /* Ensure that all the memory operations are completed - * before the worker thread starts running the function. - * Use worker thread function pointer as the guard variable. - */ - __atomic_store_n(&lcore_config[worker_id].f, f, __ATOMIC_RELEASE); - - /* send message */ - n = 0; - while (n == 0 || (n < 0 && errno == EINTR)) - n = write(m2w, &c, 1); - if (n < 0) - rte_panic("cannot write on configuration pipe\n"); - - /* wait ack */ - do { - n = read(w2m, &c, 1); - } while (n < 0 && errno == EINTR); - - if (n <= 0) - rte_panic("cannot read on configuration pipe\n"); - - rc = 0; -finish: - rte_eal_trace_thread_remote_launch(f, arg, worker_id, rc); - return rc; -} - -/* main loop of threads */ -__rte_noreturn void * -eal_thread_loop(void *arg) -{ - unsigned int lcore_id = (uintptr_t)arg; - char c; - int n, ret; - int m2w, w2m; - char cpuset[RTE_CPU_AFFINITY_STR_LEN]; - - - m2w = lcore_config[lcore_id].pipe_main2worker[0]; - w2m = lcore_config[lcore_id].pipe_worker2main[1]; - - __rte_thread_init(lcore_id, &lcore_config[lcore_id].cpuset); - - ret = eal_thread_dump_current_affinity(cpuset, sizeof(cpuset)); - RTE_LOG(DEBUG, EAL, "lcore %u is ready (tid=%zx;cpuset=[%s%s])\n", - lcore_id, (uintptr_t)pthread_self(), cpuset, - ret == 0 ? "" : "..."); - - rte_eal_trace_thread_lcore_ready(lcore_id, cpuset); - - /* read on our pipe to get commands */ - while (1) { - lcore_function_t *f; - void *fct_arg; - - /* wait command */ - do { - n = read(m2w, &c, 1); - } while (n < 0 && errno == EINTR); - - if (n <= 0) - rte_panic("cannot read on configuration pipe\n"); - - /* Set the state to 'RUNNING'. Use release order - * since 'state' variable is used as the guard variable. - */ - __atomic_store_n(&lcore_config[lcore_id].state, RUNNING, - __ATOMIC_RELEASE); - - /* send ack */ - n = 0; - while (n == 0 || (n < 0 && errno == EINTR)) - n = write(w2m, &c, 1); - if (n < 0) - rte_panic("cannot write on configuration pipe\n"); - - /* Load 'f' with acquire order to ensure that - * the memory operations from the main thread - * are accessed only after update to 'f' is visible. - * Wait till the update to 'f' is visible to the worker. - */ - while ((f = __atomic_load_n(&lcore_config[lcore_id].f, - __ATOMIC_ACQUIRE)) == NULL) - rte_pause(); - - /* call the function and store the return value */ - fct_arg = lcore_config[lcore_id].arg; - ret = f(fct_arg); - lcore_config[lcore_id].ret = ret; - lcore_config[lcore_id].f = NULL; - lcore_config[lcore_id].arg = NULL; - - /* Store the state with release order to ensure that - * the memory operations from the worker thread - * are completed before the state is updated. - * Use 'state' as the guard variable. - */ - __atomic_store_n(&lcore_config[lcore_id].state, WAIT, - __ATOMIC_RELEASE); - } - - /* never reached */ - /* pthread_exit(NULL); */ - /* return NULL; */ -} - /* require calling thread tid by gettid() */ int rte_sys_gettid(void) { diff --git a/lib/eal/unix/eal_unix_thread.c b/lib/eal/unix/eal_unix_thread.c new file mode 100644 index 0000000000..70b5ba6b98 --- /dev/null +++ b/lib/eal/unix/eal_unix_thread.c @@ -0,0 +1,63 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2022 Red Hat, Inc. + */ + +#include +#include + +#include + +#include "eal_private.h" + +void +eal_thread_wake_worker(unsigned int worker_id) +{ + int m2w = lcore_config[worker_id].pipe_main2worker[1]; + int w2m = lcore_config[worker_id].pipe_worker2main[0]; + char c = 0; + int n; + + do { + n = write(m2w, &c, 1); + } while (n == 0 || (n < 0 && errno == EINTR)); + if (n < 0) + rte_panic("cannot write on configuration pipe\n"); + + do { + n = read(w2m, &c, 1); + } while (n < 0 && errno == EINTR); + if (n <= 0) + rte_panic("cannot read on configuration pipe\n"); +} + +void +eal_thread_wait_command(void) +{ + unsigned int lcore_id = rte_lcore_id(); + int m2w; + char c; + int n; + + m2w = lcore_config[lcore_id].pipe_main2worker[0]; + do { + n = read(m2w, &c, 1); + } while (n < 0 && errno == EINTR); + if (n <= 0) + rte_panic("cannot read on configuration pipe\n"); +} + +void +eal_thread_ack_command(void) +{ + unsigned int lcore_id = rte_lcore_id(); + char c = 0; + int w2m; + int n; + + w2m = lcore_config[lcore_id].pipe_worker2main[1]; + do { + n = write(w2m, &c, 1); + } while (n == 0 || (n < 0 && errno == EINTR)); + if (n < 0) + rte_panic("cannot write on configuration pipe\n"); +} diff --git a/lib/eal/unix/meson.build b/lib/eal/unix/meson.build index a22ea7cabc..781505ca90 100644 --- a/lib/eal/unix/meson.build +++ b/lib/eal/unix/meson.build @@ -3,9 +3,10 @@ sources += files( 'eal_file.c', + 'eal_filesystem.c', + 'eal_firmware.c', 'eal_unix_memory.c', + 'eal_unix_thread.c', 'eal_unix_timer.c', - 'eal_firmware.c', - 'eal_filesystem.c', 'rte_thread.c', ) diff --git a/lib/eal/windows/eal_thread.c b/lib/eal/windows/eal_thread.c index 95e96548de..f3c61b4456 100644 --- a/lib/eal/windows/eal_thread.c +++ b/lib/eal/windows/eal_thread.c @@ -11,124 +11,62 @@ #include #include #include -#include #include "eal_private.h" +#include "eal_thread.h" #include "eal_windows.h" -/* - * Send a message to a worker lcore identified by worker_id to call a - * function f with argument arg. Once the execution is done, the - * remote lcore switches to WAIT state. - */ -int -rte_eal_remote_launch(lcore_function_t *f, void *arg, unsigned int worker_id) +void +eal_thread_wake_worker(unsigned int worker_id) { - int n; - char c = 0; int m2w = lcore_config[worker_id].pipe_main2worker[1]; int w2m = lcore_config[worker_id].pipe_worker2main[0]; + char c = 0; + int n; - /* Check if the worker is in 'WAIT' state. Use acquire order - * since 'state' variable is used as the guard variable. - */ - if (__atomic_load_n(&lcore_config[worker_id].state, - __ATOMIC_ACQUIRE) != WAIT) - return -EBUSY; - - lcore_config[worker_id].arg = arg; - /* Ensure that all the memory operations are completed - * before the worker thread starts running the function. - * Use worker thread function as the guard variable. - */ - __atomic_store_n(&lcore_config[worker_id].f, f, __ATOMIC_RELEASE); - - /* send message */ - n = 0; - while (n == 0 || (n < 0 && errno == EINTR)) + do { n = _write(m2w, &c, 1); + } while (n == 0 || (n < 0 && errno == EINTR)); if (n < 0) rte_panic("cannot write on configuration pipe\n"); - /* wait ack */ do { n = _read(w2m, &c, 1); } while (n < 0 && errno == EINTR); - if (n <= 0) rte_panic("cannot read on configuration pipe\n"); - - return 0; } -/* main loop of threads */ -void * -eal_thread_loop(void *arg) +void +eal_thread_wait_command(void) { - unsigned int lcore_id = (uintptr_t)arg; + unsigned int lcore_id = rte_lcore_id(); + int m2w; char c; - int n, ret; - int m2w, w2m; - char cpuset[RTE_CPU_AFFINITY_STR_LEN]; + int n; m2w = lcore_config[lcore_id].pipe_main2worker[0]; - w2m = lcore_config[lcore_id].pipe_worker2main[1]; + do { + n = _read(m2w, &c, 1); + } while (n < 0 && errno == EINTR); + if (n <= 0) + rte_panic("cannot read on configuration pipe\n"); +} - __rte_thread_init(lcore_id, &lcore_config[lcore_id].cpuset); - - RTE_LOG(DEBUG, EAL, "lcore %u is ready (tid=%zx;cpuset=[%s])\n", - lcore_id, (uintptr_t)pthread_self(), cpuset); - - /* read on our pipe to get commands */ - while (1) { - lcore_function_t *f; - void *fct_arg; - - /* wait command */ - do { - n = _read(m2w, &c, 1); - } while (n < 0 && errno == EINTR); - - if (n <= 0) - rte_panic("cannot read on configuration pipe\n"); - - /* Set the state to 'RUNNING'. Use release order - * since 'state' variable is used as the guard variable. - */ - __atomic_store_n(&lcore_config[lcore_id].state, RUNNING, - __ATOMIC_RELEASE); - - /* send ack */ - n = 0; - while (n == 0 || (n < 0 && errno == EINTR)) - n = _write(w2m, &c, 1); - if (n < 0) - rte_panic("cannot write on configuration pipe\n"); - - /* Load 'f' with acquire order to ensure that - * the memory operations from the main thread - * are accessed only after update to 'f' is visible. - * Wait till the update to 'f' is visible to the worker. - */ - while ((f = __atomic_load_n(&lcore_config[lcore_id].f, - __ATOMIC_ACQUIRE)) == NULL) - rte_pause(); - - /* call the function and store the return value */ - fct_arg = lcore_config[lcore_id].arg; - ret = f(fct_arg); - lcore_config[lcore_id].ret = ret; - lcore_config[lcore_id].f = NULL; - lcore_config[lcore_id].arg = NULL; - - /* Store the state with release order to ensure that - * the memory operations from the worker thread - * are completed before the state is updated. - * Use 'state' as the guard variable. - */ - __atomic_store_n(&lcore_config[lcore_id].state, WAIT, - __ATOMIC_RELEASE); - } +void +eal_thread_ack_command(void) +{ + unsigned int lcore_id = rte_lcore_id(); + char c = 0; + int w2m; + int n; + + w2m = lcore_config[lcore_id].pipe_worker2main[1]; + do { + n = _write(w2m, &c, 1); + } while (n == 0 || (n < 0 && errno == EINTR)); + if (n < 0) + rte_panic("cannot write on configuration pipe\n"); } /* function to create threads */