From patchwork Sun May 2 02:33:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Kozlyuk X-Patchwork-Id: 92619 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EAF54A0524; Sun, 2 May 2021 04:33:43 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 62ADF40140; Sun, 2 May 2021 04:33:43 +0200 (CEST) Received: from mail-lf1-f45.google.com (mail-lf1-f45.google.com [209.85.167.45]) by mails.dpdk.org (Postfix) with ESMTP id DDE4E4013F; Sun, 2 May 2021 04:33:42 +0200 (CEST) Received: by mail-lf1-f45.google.com with SMTP id j4so3037695lfp.0; Sat, 01 May 2021 19:33:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=l51WRyRaTEw6Cj0PNk6Znpc6GyMd3/NSANfzzQTezwo=; b=C4w9EYhtj6mnact1R5ZFgWnLoUQpHLKQzzFX+0QUkzDLznQzqnZ3nA1iAJqhEN7JRG c1JIiH9aqN0VFu31fBchPgSnc2Hn1XJ8P8kVxBW3LYpT8xpS7NaP6HwVmL4GL2yO9klW HcwjREwG2mImdFagKUYiLyagcjIidZhhUXSRYTGNnBKVfCj2CAiPCH/R5o7Gi1C/lk4o XcaDx6vJBHP/XxYb5Viaez+6PlvF7cmuR97AWg1ZEHueonzTqHpaxRR0oevq2TtWqCTA U3Kc6bhcbuiHt3anxwZnFfSmX9Y0Y5ex3bADlAltg3SOCzrJspVPC0CzGuCHVMfNFsEh kUhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=l51WRyRaTEw6Cj0PNk6Znpc6GyMd3/NSANfzzQTezwo=; b=FgmkpUbxqmR30DoXqM0qjQZlRSs37Nu9sNoY90bDBqPxIf9rwyxw+9VAQFGtwrWHwV TJWjKeANfPh8ZIiUSz2Q1gd9C704d0Zp5RClVtr6Ec0T83GefhmtdVY47IlmQ2YQJPHY 5p++OF+pw3D8WwytmP0temsmYn+nwBWpfqpfwnEaIlj/sxP5e/o1hZd2iaw7UrmckqYv L2FkP2nfGw7fK+08jk8HNGg7bQl5OZ2NeFx5FCsXR+fGJ3PfVJrNsdyzfCeByulVdZqI NOqjcGUcaVjCxeNgb7xgUD0lCeuAhQNp0TyUf6+F5PPzwzLJpnAqRFAipTP+DUalB1wH 1/SA== X-Gm-Message-State: AOAM531GdocSw4eukFL57PjUX0ImOBXAuf5hdKkuTqUiARKEEQwVgjwt 6NmiqZGp8duPz4Txv3O+kz+41VmLQFEiJw== X-Google-Smtp-Source: ABdhPJwCVeyjqXM18ZkRZc06IylivUYcn4aDpXOg5Q+TkSccQGSa4Biz8gB0qHI7wIuCSGxxoLzklA== X-Received: by 2002:a05:6512:1295:: with SMTP id u21mr805189lfs.584.1619922822138; Sat, 01 May 2021 19:33:42 -0700 (PDT) Received: from localhost.localdomain (broadband-37-110-65-23.ip.moscow.rt.ru. [37.110.65.23]) by smtp.gmail.com with ESMTPSA id w26sm716970lfn.82.2021.05.01.19.33.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 01 May 2021 19:33:41 -0700 (PDT) From: Dmitry Kozlyuk To: dev@dpdk.org Cc: Dmitry Kozlyuk , stable@dpdk.org, Harman Kalra , Narcisa Ana Maria Vasile , Dmitry Malloy , Pallavi Kadam Date: Sun, 2 May 2021 05:33:31 +0300 Message-Id: <20210502023333.30351-1-dmitry.kozliuk@gmail.com> X-Mailer: git-send-email 2.29.3 MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 1/3] eal/windows: fix use of incorrect thread ID X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Interrupt thread ID retained its value after interrupt thread finish. Other interrupt routines could then operate on the wrong thread. Clear interrupt thread ID before thread termination. Fixes: 5c016fc0205a ("eal/windows: add interrupt thread skeleton") Cc: stable@dpdk.org Signed-off-by: Dmitry Kozlyuk Acked-by: Tyler Retzlaff --- lib/eal/windows/eal_interrupts.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/lib/eal/windows/eal_interrupts.c b/lib/eal/windows/eal_interrupts.c index 1d4cf794df..9cde02b003 100644 --- a/lib/eal/windows/eal_interrupts.c +++ b/lib/eal/windows/eal_interrupts.c @@ -46,8 +46,11 @@ eal_intr_thread_main(LPVOID arg __rte_unused) eal_intr_process(&events[i]); } + intr_thread = 0; + CloseHandle(intr_iocp); intr_iocp = NULL; + return NULL; } From patchwork Sun May 2 02:33:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Kozlyuk X-Patchwork-Id: 92620 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DABF6A0524; Sun, 2 May 2021 04:33:50 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 945D941110; Sun, 2 May 2021 04:33:44 +0200 (CEST) Received: from mail-lf1-f52.google.com (mail-lf1-f52.google.com [209.85.167.52]) by mails.dpdk.org (Postfix) with ESMTP id 9F59540692; Sun, 2 May 2021 04:33:43 +0200 (CEST) Received: by mail-lf1-f52.google.com with SMTP id 124so2997206lff.5; Sat, 01 May 2021 19:33:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OIkV9lF5P4FL0Qog325tf+flJAw1LR3JAkiHJ8u2jeI=; b=MrWk/nqp2NPjTUWB38mp9JacRmBi1ygp3CMhfQNraBmt940+6/TgWxuqIG6vbL5SDx 5ZS+ynMxBjOQ7ihfaUogjyYhqQNcko31OEljfe7dH6u7CAP6IL46sYqMNvsoZJKenz/8 v9wu4tj7yMpWVKSecvjZif2wr7+y29dAhE4fWk5BCjY96eiTsPPiRL6spe83s3cYrYLF X9+Bv0PZgF/h5oAxOMOERkFo89bFePShrPK+yBy7OZIUplmb7CdRVOA15DQDTHzFSGcD cJJA1pOlSsJCNF+jUnSinlUH0i2kZL+aa6KSomygKaNmDTZ7C7D3420+jlKqX12zENxf 6P+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OIkV9lF5P4FL0Qog325tf+flJAw1LR3JAkiHJ8u2jeI=; b=ay7SZUByn2oZ6B8JibXYIgotyP8Fbt5F1GB9i8tHTFM2VI0l/UtqvF2K4QYa7OMY3s TBGX8NbgZHMQw48r7hLqX7RmSINJytN4XWwzG9dwcwECpWK0cAim4tJLxs3DvagwP+BU eGXzrIwaElZk9Q+mHnq1a3sXBAr3oldhmjBUPizPQV3TV1VuSQFuSh6qYSAfZ4zkHtgQ ZaZCcpmRFvm/tSijtKCzoXS9IepmcgWAFVfOymddCzxMjvsYsBzCZ526z6h9pJx/2bY2 Coe6du5eozRHw4On9gMwZitDxJ++NNFi7nqgq1k70LCrrZUn1xIVaj75QwBqQa7+AX0w J5fw== X-Gm-Message-State: AOAM531ohL5LSPYclQK2BgX2qR2MfXIkeFTCWn0+0i/2m0YwMl+DcBH3 jBKePgl1Fi/Ben/Le/BVHGm6JoJCgpuNAg== X-Google-Smtp-Source: ABdhPJyg8x3cee63iCO9vf7epQ3P6toQpZ3QaatIMcD+KQt3NwrkyUMGquuDnurw5+qbiikTKxfBSg== X-Received: by 2002:a19:c218:: with SMTP id l24mr8738559lfc.529.1619922823018; Sat, 01 May 2021 19:33:43 -0700 (PDT) Received: from localhost.localdomain (broadband-37-110-65-23.ip.moscow.rt.ru. [37.110.65.23]) by smtp.gmail.com with ESMTPSA id w26sm716970lfn.82.2021.05.01.19.33.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 01 May 2021 19:33:42 -0700 (PDT) From: Dmitry Kozlyuk To: dev@dpdk.org Cc: Dmitry Kozlyuk , stable@dpdk.org, Harman Kalra , Narcisa Ana Maria Vasile , Dmitry Malloy , Pallavi Kadam Date: Sun, 2 May 2021 05:33:32 +0300 Message-Id: <20210502023333.30351-2-dmitry.kozliuk@gmail.com> X-Mailer: git-send-email 2.29.3 In-Reply-To: <20210502023333.30351-1-dmitry.kozliuk@gmail.com> References: <20210502023333.30351-1-dmitry.kozliuk@gmail.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 2/3] eal/windows: fix interrupt thread handle leakage X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Each time a work was scheduled in the interrupt thread, usually an alarm, a handle was opened but not closed. Opening a handle is a system call, which harms alarm precision. Instead of opening and closing a handle each time, open it when interrupt thread starts and close it when the thread finishes. Fixes: 5c016fc0205a ("eal/windows: add interrupt thread skeleton") Cc: stable@dpdk.org Signed-off-by: Dmitry Kozlyuk Tested-by: Pallavi Kadam --- lib/eal/windows/eal_interrupts.c | 34 +++++++++++++++++++++++--------- 1 file changed, 25 insertions(+), 9 deletions(-) diff --git a/lib/eal/windows/eal_interrupts.c b/lib/eal/windows/eal_interrupts.c index 9cde02b003..f24ed6e54e 100644 --- a/lib/eal/windows/eal_interrupts.c +++ b/lib/eal/windows/eal_interrupts.c @@ -10,6 +10,7 @@ static pthread_t intr_thread; static HANDLE intr_iocp; +static HANDLE intr_thread_handle; static void eal_intr_process(const OVERLAPPED_ENTRY *event) @@ -17,9 +18,27 @@ eal_intr_process(const OVERLAPPED_ENTRY *event) RTE_SET_USED(event); } +static int +eal_intr_thread_handle_init(void) +{ + DWORD thread_id = GetCurrentThreadId(); + + intr_thread_handle = OpenThread(THREAD_ALL_ACCESS, FALSE, thread_id); + if (intr_thread_handle == NULL) { + RTE_LOG_WIN32_ERR("OpenThread(%lu)", thread_id); + return -1; + } + return 0; +} + static void * eal_intr_thread_main(LPVOID arg __rte_unused) { + if (eal_intr_thread_handle_init() < 0) { + RTE_LOG(ERR, EAL, "Cannot open interrupt thread handle\n"); + goto cleanup; + } + while (1) { OVERLAPPED_ENTRY events[16]; ULONG event_count, i; @@ -46,6 +65,10 @@ eal_intr_thread_main(LPVOID arg __rte_unused) eal_intr_process(&events[i]); } + CloseHandle(intr_thread_handle); + intr_thread_handle = NULL; + +cleanup: intr_thread = 0; CloseHandle(intr_iocp); @@ -93,15 +116,8 @@ rte_intr_rx_ctl(__rte_unused struct rte_intr_handle *intr_handle, int eal_intr_thread_schedule(void (*func)(void *arg), void *arg) { - HANDLE handle; - - handle = OpenThread(THREAD_ALL_ACCESS, FALSE, intr_thread); - if (handle == NULL) { - RTE_LOG_WIN32_ERR("OpenThread(%llu)", intr_thread); - return -ENOENT; - } - - if (!QueueUserAPC((PAPCFUNC)(ULONG_PTR)func, handle, (ULONG_PTR)arg)) { + if (!QueueUserAPC((PAPCFUNC)(ULONG_PTR)func, + intr_thread_handle, (ULONG_PTR)arg)) { RTE_LOG_WIN32_ERR("QueueUserAPC()"); return -EINVAL; } From patchwork Sun May 2 02:33:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Kozlyuk X-Patchwork-Id: 92621 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8498FA0524; Sun, 2 May 2021 04:33:57 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E0DC84114C; Sun, 2 May 2021 04:33:45 +0200 (CEST) Received: from mail-lj1-f173.google.com (mail-lj1-f173.google.com [209.85.208.173]) by mails.dpdk.org (Postfix) with ESMTP id 8E1CB4110B; Sun, 2 May 2021 04:33:44 +0200 (CEST) Received: by mail-lj1-f173.google.com with SMTP id p12so2786118ljg.1; Sat, 01 May 2021 19:33:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HObbVr9jgvknTt2fY6BcYddgNt9hTercuLIdk6hNN7Y=; b=X06i5q7UG/J9kxTNEr2/BWWkdxHe7Wq8EbmSuJ3/RUDL1l3Gerwqde74483X74rNa3 J7VhG9g7Nm0jiAcW7J/SwO1Pj8DJKSwdGP1JXXry/p5ZAqK2PvmDF2KfGIzcieDV0pzQ vnSrZIjQKTBdipfulI/E1BXNWyl6R02BGnb8CIuJj9uXt83Zsr2z8lLCdYH2XvbsOzoz dcU+h2Xp/wZHmlsHmolFkr87fQM7f6uJb8MwQxTBA+rSTTTunaMl4XXbRLvYqWOYD4Xw tyWMR9ctgEdc5cIdAeUASn8mLsYFdrs5CrI2XqIubcRjZKfyCcksYB5dh0zKy9cufru/ RKKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HObbVr9jgvknTt2fY6BcYddgNt9hTercuLIdk6hNN7Y=; b=nbYRtSlmTcZbFz1ec8CCGxo9yL4uzgZvOFYilYDUCOksg0vzwmr8VABkVUxNU0yALq PkFerHb4N5F21Hsnn1UlzUM8ekBbX9xZQkgdYtDoRo5Q8b9w1mLdMy62PF2F8eJ/4Z54 /+Ts7V6zVsFwHuDBwpLooHjYiUUCthmBOdssSoUpBYEIc+Pn7KA3E+VjnpQcqr89pw02 4l2x7d65ji4JqvXJjCn6mlq11+qgJbAc6m1gQ3WLx6rb02oqG3Ql8meFuERbTCiE9nM6 rVUDO/19sKRgcTgGSID4BP/PO8TLWNlILNnuEsPGM2iQTmMIFC0TwsdnhsEAq/I2LoBE AN7A== X-Gm-Message-State: AOAM5332FaaMzJmcW73nGRUvUYzrCLmkOVR+BGfFO9J31W90DwpCN+HZ HNdaNeghs09zklkZ7u+U4SHwG0PkebN0PA== X-Google-Smtp-Source: ABdhPJy42cYLzXdW/EO+NVz2PcbeYEaagEMXS976jdMxxfKSVdNoGfsb2cwh0B3/ZyLqPkJOVB3eng== X-Received: by 2002:a2e:1409:: with SMTP id u9mr8644128ljd.155.1619922823897; Sat, 01 May 2021 19:33:43 -0700 (PDT) Received: from localhost.localdomain (broadband-37-110-65-23.ip.moscow.rt.ru. [37.110.65.23]) by smtp.gmail.com with ESMTPSA id w26sm716970lfn.82.2021.05.01.19.33.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 01 May 2021 19:33:43 -0700 (PDT) From: Dmitry Kozlyuk To: dev@dpdk.org Cc: Dmitry Kozlyuk , stable@dpdk.org, Narcisa Ana Maria Vasile , Dmitry Malloy , Pallavi Kadam , Harman Kalra Date: Sun, 2 May 2021 05:33:33 +0300 Message-Id: <20210502023333.30351-3-dmitry.kozliuk@gmail.com> X-Mailer: git-send-email 2.29.3 In-Reply-To: <20210502023333.30351-1-dmitry.kozliuk@gmail.com> References: <20210502023333.30351-1-dmitry.kozliuk@gmail.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH 3/3] eal/windows: cleanup interrupt resources X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Interrupt manager in Windows EAL allocates on IOCP and starts a control thread that runs indefinitely. At DPDK cleanup this thread was not stopped and IOCP handle was not closed. Gracefully stop interrupt-handling in rte_eal_cleanup(). The thread already closes IOCP handle before exiting. Fixes: 5c016fc0205a ("eal/windows: add interrupt thread skeleton") Cc: stable@dpdk.org Signed-off-by: Dmitry Kozlyuk Acked-by: Ranjit Menon Acked-by: Jie Zhou Tested-by: Jie Zhou --- lib/eal/windows/eal.c | 1 + lib/eal/windows/eal_interrupts.c | 26 ++++++++++++++++++++++++-- lib/eal/windows/eal_windows.h | 5 +++++ 3 files changed, 30 insertions(+), 2 deletions(-) diff --git a/lib/eal/windows/eal.c b/lib/eal/windows/eal.c index 28c787c0b0..25afc42f8e 100644 --- a/lib/eal/windows/eal.c +++ b/lib/eal/windows/eal.c @@ -258,6 +258,7 @@ rte_eal_cleanup(void) { struct internal_config *internal_conf = eal_get_internal_configuration(); + eal_intr_thread_cancel(); /* after this point, any DPDK pointers will become dangling */ rte_eal_memory_detach(); eal_cleanup_config(internal_conf); diff --git a/lib/eal/windows/eal_interrupts.c b/lib/eal/windows/eal_interrupts.c index f24ed6e54e..bb0585cb34 100644 --- a/lib/eal/windows/eal_interrupts.c +++ b/lib/eal/windows/eal_interrupts.c @@ -7,6 +7,8 @@ #include "eal_private.h" #include "eal_windows.h" +#define IOCP_KEY_SHUTDOWN UINT32_MAX + static pthread_t intr_thread; static HANDLE intr_iocp; @@ -34,12 +36,14 @@ eal_intr_thread_handle_init(void) static void * eal_intr_thread_main(LPVOID arg __rte_unused) { + bool finished = false; + if (eal_intr_thread_handle_init() < 0) { RTE_LOG(ERR, EAL, "Cannot open interrupt thread handle\n"); goto cleanup; } - while (1) { + while (!finished) { OVERLAPPED_ENTRY events[16]; ULONG event_count, i; BOOL result; @@ -61,8 +65,13 @@ eal_intr_thread_main(LPVOID arg __rte_unused) continue; } - for (i = 0; i < event_count; i++) + for (i = 0; i < event_count; i++) { + if (events[i].lpCompletionKey == IOCP_KEY_SHUTDOWN) { + finished = true; + break; + } eal_intr_process(&events[i]); + } } CloseHandle(intr_thread_handle); @@ -125,6 +134,19 @@ eal_intr_thread_schedule(void (*func)(void *arg), void *arg) return 0; } +void +eal_intr_thread_cancel(void) +{ + if (!PostQueuedCompletionStatus( + intr_iocp, 0, IOCP_KEY_SHUTDOWN, NULL)) { + RTE_LOG_WIN32_ERR("PostQueuedCompletionStatus()"); + RTE_LOG(ERR, EAL, "Cannot cancel interrupt thread\n"); + return; + } + + WaitForSingleObject(intr_thread_handle, INFINITE); +} + int rte_intr_callback_register( __rte_unused const struct rte_intr_handle *intr_handle, diff --git a/lib/eal/windows/eal_windows.h b/lib/eal/windows/eal_windows.h index 478accc1b9..7cc811485d 100644 --- a/lib/eal/windows/eal_windows.h +++ b/lib/eal/windows/eal_windows.h @@ -67,6 +67,11 @@ unsigned int eal_socket_numa_node(unsigned int socket_id); */ int eal_intr_thread_schedule(void (*func)(void *arg), void *arg); +/** + * Request interrupt thread to stop and wait its termination. + */ +void eal_intr_thread_cancel(void); + /** * Open virt2phys driver interface device. *