From patchwork Fri Sep 27 05:41:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Hu X-Patchwork-Id: 59948 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0491C2C23; Fri, 27 Sep 2019 07:42:28 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id 299502BE6; Fri, 27 Sep 2019 07:42:26 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7179E142F; Thu, 26 Sep 2019 22:42:25 -0700 (PDT) Received: from net-arm-thunderx2-01.test.ast.arm.com (net-arm-thunderx2-01.shanghai.arm.com [10.169.40.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 9F0463F67D; Thu, 26 Sep 2019 22:44:58 -0700 (PDT) From: Gavin Hu To: dev@dpdk.org Cc: nd@arm.com, thomas@monjalon.net, stephen@networkplumber.org, hemant.agrawal@nxp.com, jerinj@marvell.com, pbhagavatula@marvell.com, Honnappa.Nagarahalli@arm.com, ruifeng.wang@arm.com, phil.yang@arm.com, steve.capper@arm.com, stable@dpdk.org Date: Fri, 27 Sep 2019 13:41:38 +0800 Message-Id: <1569562904-43950-2-git-send-email-gavin.hu@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1569562904-43950-1-git-send-email-gavin.hu@arm.com> References: <1569562904-43950-1-git-send-email-gavin.hu@arm.com> In-Reply-To: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> Subject: [dpdk-dev] [PATCH v7 1/7] bus/fslmc: fix the conflicting dmb function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" There are two definitions conflicting each other, for more details, refer to [1]. include/rte_atomic_64.h:19: error: "dmb" redefined [-Werror] drivers/bus/fslmc/mc/fsl_mc_sys.h:36: note: this is the location of the previous definition #define dmb() {__asm__ __volatile__("" : : : "memory"); } The fix is to include the spinlock.h file before the other header files, this is inline with the coding style[2] about the "header includes". The fix changes the function to take the argument for arm to be meaningful. [1] http://inbox.dpdk.org/users/VI1PR08MB537631AB25F41B8880DCCA988FDF0@i VI1PR08MB5376.eurprd08.prod.outlook.com/T/#u [2] https://doc.dpdk.org/guides/contributing/coding_style.html Fixes: 3af733ba8da8 ("bus/fslmc: introduce MC object functions") Cc: stable@dpdk.org Signed-off-by: Gavin Hu Reviewed-by: Phil Yang Acked-by: Hemant Agrawal --- drivers/bus/fslmc/mc/fsl_mc_sys.h | 10 +++++++--- drivers/bus/fslmc/mc/mc_sys.c | 3 +-- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/drivers/bus/fslmc/mc/fsl_mc_sys.h b/drivers/bus/fslmc/mc/fsl_mc_sys.h index d0c7b39..fe9dc95 100644 --- a/drivers/bus/fslmc/mc/fsl_mc_sys.h +++ b/drivers/bus/fslmc/mc/fsl_mc_sys.h @@ -33,10 +33,14 @@ struct fsl_mc_io { #include #ifndef dmb -#define dmb() {__asm__ __volatile__("" : : : "memory"); } +#ifdef RTE_ARCH_ARM64 +#define dmb(opt) {asm volatile("dmb " #opt : : : "memory"); } +#else +#define dmb(opt) #endif -#define __iormb() dmb() -#define __iowmb() dmb() +#endif +#define __iormb() dmb(ld) +#define __iowmb() dmb(st) #define __arch_getq(a) (*(volatile uint64_t *)(a)) #define __arch_putq(v, a) (*(volatile uint64_t *)(a) = (v)) #define __arch_putq32(v, a) (*(volatile uint32_t *)(a) = (v)) diff --git a/drivers/bus/fslmc/mc/mc_sys.c b/drivers/bus/fslmc/mc/mc_sys.c index efafdc3..22143ef 100644 --- a/drivers/bus/fslmc/mc/mc_sys.c +++ b/drivers/bus/fslmc/mc/mc_sys.c @@ -4,11 +4,10 @@ * Copyright 2017 NXP * */ +#include #include #include -#include - /** User space framework uses MC Portal in shared mode. Following change * introduces lock in MC FLIB */ From patchwork Fri Sep 27 05:41:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Hu X-Patchwork-Id: 59949 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 222AC2C52; Fri, 27 Sep 2019 07:42:31 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id A211E2C2F for ; Fri, 27 Sep 2019 07:42:28 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2042B337; Thu, 26 Sep 2019 22:42:28 -0700 (PDT) Received: from net-arm-thunderx2-01.test.ast.arm.com (net-arm-thunderx2-01.shanghai.arm.com [10.169.40.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 7D9A83F67D; Thu, 26 Sep 2019 22:45:01 -0700 (PDT) From: Gavin Hu To: dev@dpdk.org Cc: nd@arm.com, thomas@monjalon.net, stephen@networkplumber.org, hemant.agrawal@nxp.com, jerinj@marvell.com, pbhagavatula@marvell.com, Honnappa.Nagarahalli@arm.com, ruifeng.wang@arm.com, phil.yang@arm.com, steve.capper@arm.com Date: Fri, 27 Sep 2019 13:41:39 +0800 Message-Id: <1569562904-43950-3-git-send-email-gavin.hu@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1569562904-43950-1-git-send-email-gavin.hu@arm.com> References: <1569562904-43950-1-git-send-email-gavin.hu@arm.com> In-Reply-To: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> Subject: [dpdk-dev] [PATCH v7 2/7] eal: add the APIs to wait until equal X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The rte_wait_until_equal_xx APIs abstract the functionality of 'polling for a memory location to become equal to a given value'. Add the RTE_ARM_USE_WFE configuration entry for aarch64, disabled by default. When it is enabled, the above APIs will call WFE instruction to save CPU cycles and power. Signed-off-by: Gavin Hu Reviewed-by: Ruifeng Wang Reviewed-by: Steve Capper Reviewed-by: Ola Liljedahl Reviewed-by: Honnappa Nagarahalli Reviewed-by: Phil Yang Acked-by: Pavan Nikhilesh Acked-by: Jerin Jacob --- config/arm/meson.build | 1 + config/common_base | 5 + .../common/include/arch/arm/rte_pause_64.h | 30 ++++++ lib/librte_eal/common/include/generic/rte_pause.h | 106 +++++++++++++++++++++ 4 files changed, 142 insertions(+) diff --git a/config/arm/meson.build b/config/arm/meson.build index 979018e..b4b4cac 100644 --- a/config/arm/meson.build +++ b/config/arm/meson.build @@ -26,6 +26,7 @@ flags_common_default = [ ['RTE_LIBRTE_AVP_PMD', false], ['RTE_SCHED_VECTOR', false], + ['RTE_ARM_USE_WFE', false], ] flags_generic = [ diff --git a/config/common_base b/config/common_base index 8ef75c2..8861713 100644 --- a/config/common_base +++ b/config/common_base @@ -111,6 +111,11 @@ CONFIG_RTE_MAX_VFIO_CONTAINERS=64 CONFIG_RTE_MALLOC_DEBUG=n CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=n CONFIG_RTE_USE_LIBBSD=n +# Use WFE instructions to implement the rte_wait_for_equal_xxx APIs, +# calling these APIs put the cores in low power state while waiting +# for the memory address to become equal to the expected value. +# This is supported only by aarch64. +CONFIG_RTE_ARM_USE_WFE=n # # Recognize/ignore the AVX/AVX512 CPU flags for performance/power testing. diff --git a/lib/librte_eal/common/include/arch/arm/rte_pause_64.h b/lib/librte_eal/common/include/arch/arm/rte_pause_64.h index 93895d3..dabde17 100644 --- a/lib/librte_eal/common/include/arch/arm/rte_pause_64.h +++ b/lib/librte_eal/common/include/arch/arm/rte_pause_64.h @@ -1,5 +1,6 @@ /* SPDX-License-Identifier: BSD-3-Clause * Copyright(c) 2017 Cavium, Inc + * Copyright(c) 2019 Arm Limited */ #ifndef _RTE_PAUSE_ARM64_H_ @@ -17,6 +18,35 @@ static inline void rte_pause(void) asm volatile("yield" ::: "memory"); } +#ifdef RTE_ARM_USE_WFE +#define __WAIT_UNTIL_EQUAL(name, asm_op, wide, type) \ +static __rte_always_inline void \ +rte_wait_until_equal_##name(volatile type * addr, type expected) \ +{ \ + type tmp; \ + asm volatile( \ + #asm_op " %" #wide "[tmp], %[addr]\n" \ + "cmp %" #wide "[tmp], %" #wide "[expected]\n" \ + "b.eq 2f\n" \ + "sevl\n" \ + "1: wfe\n" \ + #asm_op " %" #wide "[tmp], %[addr]\n" \ + "cmp %" #wide "[tmp], %" #wide "[expected]\n" \ + "bne 1b\n" \ + "2:\n" \ + : [tmp] "=&r" (tmp) \ + : [addr] "Q"(*addr), [expected] "r"(expected) \ + : "cc", "memory"); \ +} +/* Wait for *addr to be updated with expected value */ +__WAIT_UNTIL_EQUAL(relaxed_16, ldxrh, w, uint16_t) +__WAIT_UNTIL_EQUAL(acquire_16, ldaxrh, w, uint16_t) +__WAIT_UNTIL_EQUAL(relaxed_32, ldxr, w, uint32_t) +__WAIT_UNTIL_EQUAL(acquire_32, ldaxr, w, uint32_t) +__WAIT_UNTIL_EQUAL(relaxed_64, ldxr, x, uint64_t) +__WAIT_UNTIL_EQUAL(acquire_64, ldaxr, x, uint64_t) +#endif + #ifdef __cplusplus } #endif diff --git a/lib/librte_eal/common/include/generic/rte_pause.h b/lib/librte_eal/common/include/generic/rte_pause.h index 52bd4db..8906473 100644 --- a/lib/librte_eal/common/include/generic/rte_pause.h +++ b/lib/librte_eal/common/include/generic/rte_pause.h @@ -1,5 +1,6 @@ /* SPDX-License-Identifier: BSD-3-Clause * Copyright(c) 2017 Cavium, Inc + * Copyright(c) 2019 Arm Limited */ #ifndef _RTE_PAUSE_H_ @@ -12,6 +13,10 @@ * */ +#include +#include +#include + /** * Pause CPU execution for a short while * @@ -20,4 +25,105 @@ */ static inline void rte_pause(void); +/** + * Wait for *addr to be updated with a 16-bit expected value, with a relaxed + * memory ordering model meaning the loads around this API can be reordered. + * + * @param addr + * A pointer to the memory location. + * @param expected + * A 16-bit expected value to be in the memory location. + */ +__rte_always_inline +static void +rte_wait_until_equal_relaxed_16(volatile uint16_t *addr, uint16_t expected); + +/** + * Wait for *addr to be updated with a 32-bit expected value, with a relaxed + * memory ordering model meaning the loads around this API can be reordered. + * + * @param addr + * A pointer to the memory location. + * @param expected + * A 32-bit expected value to be in the memory location. + */ +__rte_always_inline +static void +rte_wait_until_equal_relaxed_32(volatile uint32_t *addr, uint32_t expected); + +/** + * Wait for *addr to be updated with a 64-bit expected value, with a relaxed + * memory ordering model meaning the loads around this API can be reordered. + * + * @param addr + * A pointer to the memory location. + * @param expected + * A 64-bit expected value to be in the memory location. + */ +__rte_always_inline +static void +rte_wait_until_equal_relaxed_64(volatile uint64_t *addr, uint64_t expected); + +/** + * Wait for *addr to be updated with a 16-bit expected value, with an acquire + * memory ordering model meaning the loads after this API can't be observed + * before this API. + * + * @param addr + * A pointer to the memory location. + * @param expected + * A 16-bit expected value to be in the memory location. + */ +__rte_always_inline +static void +rte_wait_until_equal_acquire_16(volatile uint16_t *addr, uint16_t expected); + +/** + * Wait for *addr to be updated with a 32-bit expected value, with an acquire + * memory ordering model meaning the loads after this API can't be observed + * before this API. + * + * @param addr + * A pointer to the memory location. + * @param expected + * A 32-bit expected value to be in the memory location. + */ +__rte_always_inline +static void +rte_wait_until_equal_acquire_32(volatile uint32_t *addr, uint32_t expected); + +/** + * Wait for *addr to be updated with a 64-bit expected value, with an acquire + * memory ordering model meaning the loads after this API can't be observed + * before this API. + * + * @param addr + * A pointer to the memory location. + * @param expected + * A 64-bit expected value to be in the memory location. + */ +__rte_always_inline +static void +rte_wait_until_equal_acquire_64(volatile uint64_t *addr, uint64_t expected); + +#if !defined(RTE_ARM_USE_WFE) +#define __WAIT_UNTIL_EQUAL(op_name, size, type, memorder) \ +__rte_always_inline \ +static void \ +rte_wait_until_equal_##op_name##_##size(volatile type *addr, \ + type expected) \ +{ \ + while (__atomic_load_n(addr, memorder) != expected) \ + rte_pause(); \ +} + +/* Wait for *addr to be updated with expected value */ +__WAIT_UNTIL_EQUAL(relaxed, 16, uint16_t, __ATOMIC_RELAXED) +__WAIT_UNTIL_EQUAL(acquire, 16, uint16_t, __ATOMIC_ACQUIRE) +__WAIT_UNTIL_EQUAL(relaxed, 32, uint32_t, __ATOMIC_RELAXED) +__WAIT_UNTIL_EQUAL(acquire, 32, uint32_t, __ATOMIC_ACQUIRE) +__WAIT_UNTIL_EQUAL(relaxed, 64, uint64_t, __ATOMIC_RELAXED) +__WAIT_UNTIL_EQUAL(acquire, 64, uint64_t, __ATOMIC_ACQUIRE) +#endif /* RTE_ARM_USE_WFE */ + #endif /* _RTE_PAUSE_H_ */ From patchwork Fri Sep 27 05:41:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Hu X-Patchwork-Id: 59950 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id AB2A32C58; Fri, 27 Sep 2019 07:42:33 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id 3F75A2C58 for ; Fri, 27 Sep 2019 07:42:31 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C3531142F; Thu, 26 Sep 2019 22:42:30 -0700 (PDT) Received: from net-arm-thunderx2-01.test.ast.arm.com (net-arm-thunderx2-01.shanghai.arm.com [10.169.40.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 2BD0F3F67D; Thu, 26 Sep 2019 22:45:04 -0700 (PDT) From: Gavin Hu To: dev@dpdk.org Cc: nd@arm.com, thomas@monjalon.net, stephen@networkplumber.org, hemant.agrawal@nxp.com, jerinj@marvell.com, pbhagavatula@marvell.com, Honnappa.Nagarahalli@arm.com, ruifeng.wang@arm.com, phil.yang@arm.com, steve.capper@arm.com Date: Fri, 27 Sep 2019 13:41:40 +0800 Message-Id: <1569562904-43950-4-git-send-email-gavin.hu@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1569562904-43950-1-git-send-email-gavin.hu@arm.com> References: <1569562904-43950-1-git-send-email-gavin.hu@arm.com> In-Reply-To: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> Subject: [dpdk-dev] [PATCH v7 3/7] spinlock: use wfe to reduce contention on aarch64 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" In acquiring a spinlock, cores repeatedly poll the lock variable. This is replaced by rte_wait_until_equal API. Running the micro benchmarking and the testpmd and l3fwd traffic tests on ThunderX2, Ampere eMAG80 and Arm N1SDP, everything went well and no notable performance gain nor degradation was measured. Signed-off-by: Gavin Hu Reviewed-by: Ruifeng Wang Reviewed-by: Phil Yang Reviewed-by: Steve Capper Reviewed-by: Ola Liljedahl Reviewed-by: Honnappa Nagarahalli Tested-by: Pavan Nikhilesh --- .../common/include/arch/arm/rte_spinlock.h | 26 ++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/lib/librte_eal/common/include/arch/arm/rte_spinlock.h b/lib/librte_eal/common/include/arch/arm/rte_spinlock.h index 1a6916b..b61c055 100644 --- a/lib/librte_eal/common/include/arch/arm/rte_spinlock.h +++ b/lib/librte_eal/common/include/arch/arm/rte_spinlock.h @@ -16,6 +16,32 @@ extern "C" { #include #include "generic/rte_spinlock.h" +/* armv7a does support WFE, but an explicit wake-up signal using SEV is + * required (must be preceded by DSB to drain the store buffer) and + * this is less performant, so keep armv7a implementation unchanged. + */ +#ifndef RTE_FORCE_INTRINSICS +static inline void +rte_spinlock_lock(rte_spinlock_t *sl) +{ + unsigned int tmp; + /* http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc. + * faqs/ka16809.html + */ + asm volatile( + "1: ldaxr %w[tmp], %w[locked]\n" + "cbnz %w[tmp], 2f\n" + "stxr %w[tmp], %w[one], %w[locked]\n" + "cbnz %w[tmp], 1b\n" + "ret\n" + "2: sevl\n" + "wfe\n" + "jmp 1b\n" + : [tmp] "=&r" (tmp), [locked] "+Q"(sl->locked) + : [one] "r" (1) +} +#endif + static inline int rte_tm_supported(void) { return 0; From patchwork Fri Sep 27 05:41:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Hu X-Patchwork-Id: 59951 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8867D2E83; Fri, 27 Sep 2019 07:42:37 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id DF6132C6A for ; Fri, 27 Sep 2019 07:42:33 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7288E337; Thu, 26 Sep 2019 22:42:33 -0700 (PDT) Received: from net-arm-thunderx2-01.test.ast.arm.com (net-arm-thunderx2-01.shanghai.arm.com [10.169.40.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id CF65B3F67D; Thu, 26 Sep 2019 22:45:06 -0700 (PDT) From: Gavin Hu To: dev@dpdk.org Cc: nd@arm.com, thomas@monjalon.net, stephen@networkplumber.org, hemant.agrawal@nxp.com, jerinj@marvell.com, pbhagavatula@marvell.com, Honnappa.Nagarahalli@arm.com, ruifeng.wang@arm.com, phil.yang@arm.com, steve.capper@arm.com Date: Fri, 27 Sep 2019 13:41:41 +0800 Message-Id: <1569562904-43950-5-git-send-email-gavin.hu@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1569562904-43950-1-git-send-email-gavin.hu@arm.com> References: <1569562904-43950-1-git-send-email-gavin.hu@arm.com> In-Reply-To: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> Subject: [dpdk-dev] [PATCH v7 4/7] ticketlock: use new API to reduce contention on aarch64 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" While using ticket lock, cores repeatedly poll the lock variable. This is replaced by rte_wait_until_equal API. Running ticketlock_autotest on ThunderX2, Ampere eMAG80, and Arm N1SDP[1], there were variances between runs, but no notable performance gain or degradation were seen with and without this patch. [1] https://community.arm.com/developer/tools-software/oss-platforms/w/\ docs/440/neoverse-n1-sdp Signed-off-by: Gavin Hu Reviewed-by: Honnappa Nagarahalli Tested-by: Phil Yang Tested-by: Pavan Nikhilesh Reviewed-by: Jerin Jacob --- lib/librte_eal/common/include/generic/rte_ticketlock.h | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/lib/librte_eal/common/include/generic/rte_ticketlock.h b/lib/librte_eal/common/include/generic/rte_ticketlock.h index d9bec87..232bbe9 100644 --- a/lib/librte_eal/common/include/generic/rte_ticketlock.h +++ b/lib/librte_eal/common/include/generic/rte_ticketlock.h @@ -66,8 +66,7 @@ static inline void rte_ticketlock_lock(rte_ticketlock_t *tl) { uint16_t me = __atomic_fetch_add(&tl->s.next, 1, __ATOMIC_RELAXED); - while (__atomic_load_n(&tl->s.current, __ATOMIC_ACQUIRE) != me) - rte_pause(); + rte_wait_until_equal_acquire_16(&tl->s.current, me); } /** From patchwork Fri Sep 27 05:41:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Hu X-Patchwork-Id: 59952 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BF1253423; Fri, 27 Sep 2019 07:42:39 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id 85F672E83 for ; Fri, 27 Sep 2019 07:42:36 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 214E6142F; Thu, 26 Sep 2019 22:42:36 -0700 (PDT) Received: from net-arm-thunderx2-01.test.ast.arm.com (net-arm-thunderx2-01.shanghai.arm.com [10.169.40.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 7E1D83F67D; Thu, 26 Sep 2019 22:45:09 -0700 (PDT) From: Gavin Hu To: dev@dpdk.org Cc: nd@arm.com, thomas@monjalon.net, stephen@networkplumber.org, hemant.agrawal@nxp.com, jerinj@marvell.com, pbhagavatula@marvell.com, Honnappa.Nagarahalli@arm.com, ruifeng.wang@arm.com, phil.yang@arm.com, steve.capper@arm.com Date: Fri, 27 Sep 2019 13:41:42 +0800 Message-Id: <1569562904-43950-6-git-send-email-gavin.hu@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1569562904-43950-1-git-send-email-gavin.hu@arm.com> References: <1569562904-43950-1-git-send-email-gavin.hu@arm.com> In-Reply-To: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> Subject: [dpdk-dev] [PATCH v7 5/7] ring: use wfe to wait for ring tail update on aarch64 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Instead of polling for tail to be updated, use wfe instruction. Signed-off-by: Gavin Hu Reviewed-by: Ruifeng Wang Reviewed-by: Steve Capper Reviewed-by: Ola Liljedahl Reviewed-by: Honnappa Nagarahalli --- lib/librte_ring/rte_ring_c11_mem.h | 4 ++-- lib/librte_ring/rte_ring_generic.h | 3 +-- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/lib/librte_ring/rte_ring_c11_mem.h b/lib/librte_ring/rte_ring_c11_mem.h index 0fb73a3..764d8f1 100644 --- a/lib/librte_ring/rte_ring_c11_mem.h +++ b/lib/librte_ring/rte_ring_c11_mem.h @@ -2,6 +2,7 @@ * * Copyright (c) 2017,2018 HXT-semitech Corporation. * Copyright (c) 2007-2009 Kip Macy kmacy@freebsd.org + * Copyright (c) 2019 Arm Limited * All rights reserved. * Derived from FreeBSD's bufring.h * Used as BSD-3 Licensed with permission from Kip Macy. @@ -21,8 +22,7 @@ update_tail(struct rte_ring_headtail *ht, uint32_t old_val, uint32_t new_val, * we need to wait for them to complete */ if (!single) - while (unlikely(ht->tail != old_val)) - rte_pause(); + rte_wait_until_equal_relaxed_32(&ht->tail, old_val); __atomic_store_n(&ht->tail, new_val, __ATOMIC_RELEASE); } diff --git a/lib/librte_ring/rte_ring_generic.h b/lib/librte_ring/rte_ring_generic.h index 953cdbb..6828527 100644 --- a/lib/librte_ring/rte_ring_generic.h +++ b/lib/librte_ring/rte_ring_generic.h @@ -23,8 +23,7 @@ update_tail(struct rte_ring_headtail *ht, uint32_t old_val, uint32_t new_val, * we need to wait for them to complete */ if (!single) - while (unlikely(ht->tail != old_val)) - rte_pause(); + rte_wait_until_equal_relaxed_32(&ht->tail, old_val); ht->tail = new_val; } From patchwork Fri Sep 27 05:41:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Hu X-Patchwork-Id: 59953 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 26DCE34F3; Fri, 27 Sep 2019 07:42:43 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id 4AF4C3237 for ; Fri, 27 Sep 2019 07:42:39 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CD853337; Thu, 26 Sep 2019 22:42:38 -0700 (PDT) Received: from net-arm-thunderx2-01.test.ast.arm.com (net-arm-thunderx2-01.shanghai.arm.com [10.169.40.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 2D1643F67D; Thu, 26 Sep 2019 22:45:12 -0700 (PDT) From: Gavin Hu To: dev@dpdk.org Cc: nd@arm.com, thomas@monjalon.net, stephen@networkplumber.org, hemant.agrawal@nxp.com, jerinj@marvell.com, pbhagavatula@marvell.com, Honnappa.Nagarahalli@arm.com, ruifeng.wang@arm.com, phil.yang@arm.com, steve.capper@arm.com Date: Fri, 27 Sep 2019 13:41:43 +0800 Message-Id: <1569562904-43950-7-git-send-email-gavin.hu@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1569562904-43950-1-git-send-email-gavin.hu@arm.com> References: <1569562904-43950-1-git-send-email-gavin.hu@arm.com> In-Reply-To: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> Subject: [dpdk-dev] [PATCH v7 6/7] net/thunderx: use new API to save cycles on aarch64 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Use the new API to wait in low power state instead of continuous polling to save CPU cycles and power. Signed-off-by: Gavin Hu Reviewed-by: Ruifeng Wang Acked-by: Jerin Jacob --- drivers/net/thunderx/nicvf_rxtx.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/net/thunderx/nicvf_rxtx.c b/drivers/net/thunderx/nicvf_rxtx.c index 1c42874..16f34c6 100644 --- a/drivers/net/thunderx/nicvf_rxtx.c +++ b/drivers/net/thunderx/nicvf_rxtx.c @@ -385,8 +385,7 @@ nicvf_fill_rbdr(struct nicvf_rxq *rxq, int to_fill) ltail++; } - while (__atomic_load_n(&rbdr->tail, __ATOMIC_RELAXED) != next_tail) - rte_pause(); + rte_wait_until_equal_relaxed_32(&rbdr->tail, next_tail); __atomic_store_n(&rbdr->tail, ltail, __ATOMIC_RELEASE); nicvf_addr_write(door, to_fill); From patchwork Fri Sep 27 05:41:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Hu X-Patchwork-Id: 59954 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id F31B63977; Fri, 27 Sep 2019 07:42:45 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id 008922C39 for ; Fri, 27 Sep 2019 07:42:41 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8650E142F; Thu, 26 Sep 2019 22:42:41 -0700 (PDT) Received: from net-arm-thunderx2-01.test.ast.arm.com (net-arm-thunderx2-01.shanghai.arm.com [10.169.40.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D90EC3F67D; Thu, 26 Sep 2019 22:45:14 -0700 (PDT) From: Gavin Hu To: dev@dpdk.org Cc: nd@arm.com, thomas@monjalon.net, stephen@networkplumber.org, hemant.agrawal@nxp.com, jerinj@marvell.com, pbhagavatula@marvell.com, Honnappa.Nagarahalli@arm.com, ruifeng.wang@arm.com, phil.yang@arm.com, steve.capper@arm.com Date: Fri, 27 Sep 2019 13:41:44 +0800 Message-Id: <1569562904-43950-8-git-send-email-gavin.hu@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1569562904-43950-1-git-send-email-gavin.hu@arm.com> References: <1569562904-43950-1-git-send-email-gavin.hu@arm.com> In-Reply-To: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> Subject: [dpdk-dev] [PATCH v7 7/7] event/opdl: use new API to save cycles on aarch64 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Use the new API to wait in low power state instead of continuous polling to save CPU cycles and power. Signed-off-by: Gavin Hu Reviewed-by: Ruifeng Wang Reviewed-by: Jerin Jacob --- drivers/event/opdl/opdl_ring.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/drivers/event/opdl/opdl_ring.c b/drivers/event/opdl/opdl_ring.c index e8b29e2..f446fa3 100644 --- a/drivers/event/opdl/opdl_ring.c +++ b/drivers/event/opdl/opdl_ring.c @@ -17,6 +17,7 @@ #include #include #include +#include #include "opdl_ring.h" #include "opdl_log.h" @@ -475,9 +476,7 @@ opdl_ring_input_multithread(struct opdl_ring *t, const void *entries, /* If another thread started inputting before this one, but hasn't * finished, we need to wait for it to complete to update the tail. */ - while (unlikely(__atomic_load_n(&s->shared.tail, __ATOMIC_ACQUIRE) != - old_head)) - rte_pause(); + rte_wait_until_equal_acquire_32(&s->shared.tail, old_head); __atomic_store_n(&s->shared.tail, old_head + num_entries, __ATOMIC_RELEASE);