From patchwork Fri Mar 6 14:48:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 66347 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1310AA056A; Fri, 6 Mar 2020 15:49:16 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 597492BA8; Fri, 6 Mar 2020 15:49:15 +0100 (CET) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by dpdk.org (Postfix) with ESMTP id 9F61FFEB for ; Fri, 6 Mar 2020 15:49:13 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1583506152; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=kWNYYAiRd/5K3urRlYO8EyqNmY07OaiaFQhKTLAfUc8=; b=P10dRiZ5/RYrk5cfDFBnEDrMwzGU5nP0qI5yjt+lH1jZoRi3mmVGtQOOQKpOZV58KEMJFC cgqjk0+Yz2nxDemSXuwOcmtOZr6B0b4hjtFOjbZGbL2YCHj4Gq/9idOyEnGQHhCXiYMecE My8dfJdfwP9PZHrAoUQ/xcD7NqNPvSU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-381--TnG40rDMNamL7KHQmNg_Q-1; Fri, 06 Mar 2020 09:49:09 -0500 X-MC-Unique: -TnG40rDMNamL7KHQmNg_Q-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E40CC73476; Fri, 6 Mar 2020 14:49:07 +0000 (UTC) Received: from dmarchan.remote.csb (ovpn-204-36.brq.redhat.com [10.40.204.36]) by smtp.corp.redhat.com (Postfix) with ESMTP id B5F4F60BEC; Fri, 6 Mar 2020 14:49:04 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: echaudro@redhat.com, aconole@redhat.com, maxime.coquelin@redhat.com, anatoly.burakov@intel.com, Wenzhuo Lu , Jingjing Wu , Bernard Iremonger Date: Fri, 6 Mar 2020 15:48:44 +0100 Message-Id: <20200306144844.27255-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [dpdk-dev] [RFC PATCH] app/testpmd: only lock text pages X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Since 18.05 and the memory subsystem rework, EAL reserves some big (unused) mappings. In testpmd, we have been locking all pages to avoid page faults during benchmark/performance regression tests [1]. However, asking for locking all the pages triggers issues on FreeBSD [2] and becomes really heavy in some Linux configurations (see [3], [4]). This patch changes the behavior so that testpmd only lock pages containing .text by default. 1: https://git.dpdk.org/dpdk/commit/?id=1c036b16c284 2: https://git.dpdk.org/dpdk/commit/?id=fb7b8b32cd95 3: https://bugzilla.redhat.com/show_bug.cgi?id=1786923 4: http://mails.dpdk.org/archives/dev/2020-February/158477.html Signed-off-by: David Marchand --- app/test-pmd/parameters.c | 4 +-- app/test-pmd/testpmd.c | 53 +++++++++++++++++++++++++++++++-------- app/test-pmd/testpmd.h | 5 +++- 3 files changed, 49 insertions(+), 13 deletions(-) diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c index 404dba2b2..952371e6c 100644 --- a/app/test-pmd/parameters.c +++ b/app/test-pmd/parameters.c @@ -1304,9 +1304,9 @@ launch_args_parse(int argc, char** argv) if (!strcmp(lgopts[opt_idx].name, "hot-plug")) hot_plug = 1; if (!strcmp(lgopts[opt_idx].name, "mlockall")) - do_mlockall = 1; + do_mlock = TESTPMD_MLOCK_ALL; if (!strcmp(lgopts[opt_idx].name, "no-mlockall")) - do_mlockall = 0; + do_mlock = TESTPMD_MLOCK_NONE; if (!strcmp(lgopts[opt_idx].name, "noisy-tx-sw-buffer-size")) { n = atoi(optarg); diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 035836adf..9a1716321 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include @@ -390,9 +391,9 @@ uint32_t event_print_mask = (UINT32_C(1) << RTE_ETH_EVENT_UNKNOWN) | (UINT32_C(1) << RTE_ETH_EVENT_MACSEC) | (UINT32_C(1) << RTE_ETH_EVENT_INTR_RMV); /* - * Decide if all memory are locked for performance. + * Decide which part of memory is locked for performance. */ -int do_mlockall = 0; +int do_mlock = TESTPMD_MLOCK_TEXT; /* * NIC bypass mode configuration options. @@ -3455,6 +3456,42 @@ signal_handler(int signum) } } +static void +lock_pages(const void *_addr, size_t _len, const char *prefix) +{ + const void *addr; + size_t pagesize; + size_t len; + + /* While Linux does not care, FreeBSD mlock expects page aligned + * address (according to the man). + */ + pagesize = sysconf(_SC_PAGESIZE); + addr = RTE_PTR_ALIGN_FLOOR(_addr, pagesize); + len = _len + ((uintptr_t)_addr & (pagesize - 1)); + if (mlock(addr, len)) { + TESTPMD_LOG(NOTICE, "%s: mlock %p (0x%zx) aligned to %p (0x%zx) failed with error \"%s\"\n", + prefix, _addr, _len, addr, len, strerror(errno)); + } +} + +static int +lock_text_cb(struct dl_phdr_info *info, __rte_unused size_t size, + __rte_unused void *data) +{ + int i; + + for (i = 0; i < info->dlpi_phnum; i++) { + void *addr; + + if (info->dlpi_phdr[i].p_memsz == 0) + continue; + addr = (void *)(info->dlpi_addr + info->dlpi_phdr[i].p_vaddr); + lock_pages(addr, info->dlpi_phdr[i].p_memsz, info->dlpi_name); + } + return 0; +} + int main(int argc, char** argv) { @@ -3514,19 +3551,15 @@ main(int argc, char** argv) latencystats_enabled = 0; #endif - /* on FreeBSD, mlockall() is disabled by default */ -#ifdef RTE_EXEC_ENV_FREEBSD - do_mlockall = 0; -#else - do_mlockall = 1; -#endif - argc -= diag; argv += diag; if (argc > 1) launch_args_parse(argc, argv); - if (do_mlockall && mlockall(MCL_CURRENT | MCL_FUTURE)) { + if (do_mlock == TESTPMD_MLOCK_TEXT ) { + dl_iterate_phdr(lock_text_cb, NULL); + } else if (do_mlock == TESTPMD_MLOCK_ALL && + mlockall(MCL_CURRENT | MCL_FUTURE)) { TESTPMD_LOG(NOTICE, "mlockall() failed with error \"%s\"\n", strerror(errno)); } diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index 7a7c73f79..a38e5a1f5 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -343,7 +343,10 @@ extern uint32_t event_print_mask; /**< set by "--print-event xxxx" and "--mask-event xxxx parameters */ extern bool setup_on_probe_event; /**< disabled by port setup-on iterator */ extern uint8_t hot_plug; /**< enable by "--hot-plug" parameter */ -extern int do_mlockall; /**< set by "--mlockall" or "--no-mlockall" parameter */ +#define TESTPMD_MLOCK_NONE 0 +#define TESTPMD_MLOCK_TEXT 1 +#define TESTPMD_MLOCK_ALL 2 +extern int do_mlock; /**< set by "--mlockall" or "--no-mlockall" parameter */ extern uint8_t clear_ptypes; /**< disabled by set ptype cmd */ #ifdef RTE_LIBRTE_IXGBE_BYPASS