From patchwork Tue Sep 6 02:49:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhichao Zeng X-Patchwork-Id: 115946 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5C701A054A; Tue, 6 Sep 2022 04:48:41 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 00798400D6; Tue, 6 Sep 2022 04:48:41 +0200 (CEST) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by mails.dpdk.org (Postfix) with ESMTP id 280BB40041; Tue, 6 Sep 2022 04:48:38 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1662432519; x=1693968519; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=uGJgAVHRst1D+4TzMtMOWicRNvMZL3OTIDZcr4E13PI=; b=DxjdlAJojFF6k8RByCA+Op1wM0KNIMHfsQ56uUSH1Zf+VsnApp6tPOun o/3EmhTy9vnACLgJwYxbbSlBlZ++s2e9baGplHM//NGyciSZ8ioACR7f8 dy01KXFVTAEY3peEYtoodFTE572inVCNW15gXTo4odh3yZIV4fJhBCWTb pmC4GZ4gfVqQgZNVYLCVvX1v6cK4eTFXx0bn4WAXKXw8ef8U1YoTKYdD2 qqqQ600iO1qLL/X4h/8U+KJrljtKbo6slVR788qvDt0UsTyLquudnxlF+ p1e7A+/l6aavWW3BdFZ9Pg2zow5ZnT8YZqAti9on71rWF12DoGnW7vF2l w==; X-IronPort-AV: E=McAfee;i="6500,9779,10461"; a="276881057" X-IronPort-AV: E=Sophos;i="5.93,292,1654585200"; d="scan'208";a="276881057" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Sep 2022 19:48:38 -0700 X-IronPort-AV: E=Sophos;i="5.93,292,1654585200"; d="scan'208";a="942285101" Received: from unknown (HELO localhost.localdomain) ([10.239.252.103]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Sep 2022 19:48:35 -0700 From: Zhichao Zeng To: dev@dpdk.org Cc: david.marchand@redhat.com, qiming.yang@intel.com, yidingx.zhou@intel.com, Zhichao Zeng , stable@dpdk.org, Bruce Richardson , Harman Kalra Subject: [PATCH v5] lib/eal: fix segfaults in exiting Date: Tue, 6 Sep 2022 10:49:50 +0800 Message-Id: <20220906024950.160740-1-zhichaox.zeng@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The 'eal-intr-thread' is not closed before memory cleanup in the process of exiting. There is a small chance that the 'eal-intr-thread' is about to use some pointers, the memory was just cleaned, which causes segfaults caught by ASan. This patch closes the 'eal-intr-thread' before memory cleanup in 'rte_eal_cleanup' to avoid segfaults, and adds a flag to avoid executing 'rte_eal_cleanup' in the child process which is forked to execut some test cases(e.g. debug_autotest of dpdk-test). Bugzilla ID: 1006 Cc: stable@dpdk.org Signed-off-by: Zhichao Zeng --- v2: add same API for FreeBSD --- v3: fix rte_eal_cleanup crash in debug_autotest --- v4: shorten the prompt message and optimize the commit log --- v5: simplify patch --- lib/eal/common/eal_private.h | 7 +++++++ lib/eal/freebsd/eal.c | 13 +++++++++++++ lib/eal/freebsd/eal_interrupts.c | 12 ++++++++++++ lib/eal/linux/eal.c | 13 +++++++++++++ lib/eal/linux/eal_interrupts.c | 12 ++++++++++++ 5 files changed, 57 insertions(+) diff --git a/lib/eal/common/eal_private.h b/lib/eal/common/eal_private.h index 44d14241f0..7adf41b7d7 100644 --- a/lib/eal/common/eal_private.h +++ b/lib/eal/common/eal_private.h @@ -152,6 +152,13 @@ int rte_eal_tailqs_init(void); */ int rte_eal_intr_init(void); +/** + * Destroy interrupt handling thread. + * + * This function is private to EAL. + */ +void rte_eal_intr_destroy(void); + /** * Close the default log stream * diff --git a/lib/eal/freebsd/eal.c b/lib/eal/freebsd/eal.c index 26fbc91b26..f27c1d9f97 100644 --- a/lib/eal/freebsd/eal.c +++ b/lib/eal/freebsd/eal.c @@ -72,6 +72,8 @@ struct lcore_config lcore_config[RTE_MAX_LCORE]; /* used by rte_rdtsc() */ int rte_cycles_vmware_tsc_map; +/* used to judge if is forked */ +static int is_forked; int eal_clean_runtime_dir(void) @@ -574,6 +576,11 @@ static void rte_eal_init_alert(const char *msg) RTE_LOG(ERR, EAL, "%s\n", msg); } +static void mark_forked(void) +{ + is_forked++; +} + /* Launch threads, called at application init(). */ int rte_eal_init(int argc, char **argv) @@ -883,16 +890,22 @@ rte_eal_init(int argc, char **argv) eal_mcfg_complete(); + pthread_atfork(NULL, NULL, mark_forked); + return fctret; } int rte_eal_cleanup(void) { + if (is_forked) + return 0; + struct internal_config *internal_conf = eal_get_internal_configuration(); rte_service_finalize(); rte_mp_channel_cleanup(); + rte_eal_intr_destroy(); rte_trace_save(); eal_trace_fini(); /* after this point, any DPDK pointers will become dangling */ diff --git a/lib/eal/freebsd/eal_interrupts.c b/lib/eal/freebsd/eal_interrupts.c index 9f720bdc8f..cac3859b06 100644 --- a/lib/eal/freebsd/eal_interrupts.c +++ b/lib/eal/freebsd/eal_interrupts.c @@ -648,6 +648,18 @@ rte_eal_intr_init(void) return ret; } +void +rte_eal_intr_destroy(void) +{ + /* cancel the host thread to wait/handle the interrupt */ + pthread_cancel(intr_thread); + pthread_join(intr_thread, NULL); + + /* close kqueue */ + close(kq); + kq = -1; +} + int rte_intr_rx_ctl(struct rte_intr_handle *intr_handle, int epfd, int op, unsigned int vec, void *data) diff --git a/lib/eal/linux/eal.c b/lib/eal/linux/eal.c index 37d29643a5..ea38ec183d 100644 --- a/lib/eal/linux/eal.c +++ b/lib/eal/linux/eal.c @@ -76,6 +76,8 @@ struct lcore_config lcore_config[RTE_MAX_LCORE]; /* used by rte_rdtsc() */ int rte_cycles_vmware_tsc_map; +/* used to judge if is forked */ +static int is_forked; int eal_clean_runtime_dir(void) @@ -954,6 +956,11 @@ eal_worker_thread_create(unsigned int lcore_id) return ret; } +static void mark_forked(void) +{ + is_forked++; +} + /* Launch threads, called at application init(). */ int rte_eal_init(int argc, char **argv) @@ -1324,6 +1331,8 @@ rte_eal_init(int argc, char **argv) eal_mcfg_complete(); + pthread_atfork(NULL, NULL, mark_forked); + return fctret; } @@ -1347,6 +1356,9 @@ mark_freeable(const struct rte_memseg_list *msl, const struct rte_memseg *ms, int rte_eal_cleanup(void) { + if (is_forked) + return 0; + /* if we're in a primary process, we need to mark hugepages as freeable * so that finalization can release them back to the system. */ @@ -1362,6 +1374,7 @@ rte_eal_cleanup(void) vfio_mp_sync_cleanup(); #endif rte_mp_channel_cleanup(); + rte_eal_intr_destroy(); rte_trace_save(); eal_trace_fini(); /* after this point, any DPDK pointers will become dangling */ diff --git a/lib/eal/linux/eal_interrupts.c b/lib/eal/linux/eal_interrupts.c index d52ec8eb4c..7e9853e8e7 100644 --- a/lib/eal/linux/eal_interrupts.c +++ b/lib/eal/linux/eal_interrupts.c @@ -1199,6 +1199,18 @@ rte_eal_intr_init(void) return ret; } +void +rte_eal_intr_destroy(void) +{ + /* cancel the host thread to wait/handle the interrupt */ + pthread_cancel(intr_thread); + pthread_join(intr_thread, NULL); + + /* close the pipe used by epoll */ + close(intr_pipe.writefd); + close(intr_pipe.readfd); +} + static void eal_intr_proc_rxtx_intr(int fd, const struct rte_intr_handle *intr_handle) {