From patchwork Mon Jan 28 17:29:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eads, Gage" X-Patchwork-Id: 50071 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 57EEE6C9B; Mon, 28 Jan 2019 18:29:08 +0100 (CET) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 390A15F62 for ; Mon, 28 Jan 2019 18:29:06 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jan 2019 09:29:06 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,534,1539673200"; d="scan'208";a="315547971" Received: from txasoft-yocto.an.intel.com ([10.123.72.192]) by fmsmga005.fm.intel.com with ESMTP; 28 Jan 2019 09:29:05 -0800 From: Gage Eads To: dev@dpdk.org Cc: olivier.matz@6wind.com, arybchenko@solarflare.com, bruce.richardson@intel.com, konstantin.ananyev@intel.com, gavin.hu@arm.com, Honnappa.Nagarahalli@arm.com, nd@arm.com, chaozhu@linux.vnet.ibm.com, jerinj@marvell.com, hemant.agrawal@nxp.com Date: Mon, 28 Jan 2019 11:29:45 -0600 Message-Id: <20190128172945.27251-2-gage.eads@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20190128172945.27251-1-gage.eads@intel.com> References: <20190128172945.27251-1-gage.eads@intel.com> Subject: [dpdk-dev] [PATCH 1/1] eal: add 128-bit cmpset (x86-64 only) X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This operation can be used for non-blocking algorithms, such as a non-blocking stack or ring. Signed-off-by: Gage Eads --- .../common/include/arch/x86/rte_atomic_64.h | 31 +++++++++++ lib/librte_eal/common/include/generic/rte_atomic.h | 65 ++++++++++++++++++++++ 2 files changed, 96 insertions(+) diff --git a/lib/librte_eal/common/include/arch/x86/rte_atomic_64.h b/lib/librte_eal/common/include/arch/x86/rte_atomic_64.h index fd2ec9c53..b7b90b83e 100644 --- a/lib/librte_eal/common/include/arch/x86/rte_atomic_64.h +++ b/lib/librte_eal/common/include/arch/x86/rte_atomic_64.h @@ -34,6 +34,7 @@ /* * Inspired from FreeBSD src/sys/amd64/include/atomic.h * Copyright (c) 1998 Doug Rabson + * Copyright (c) 2019 Intel Corporation * All rights reserved. */ @@ -46,6 +47,7 @@ #include #include +#include #include /*------------------------- 64 bit atomic operations -------------------------*/ @@ -208,4 +210,33 @@ static inline void rte_atomic64_clear(rte_atomic64_t *v) } #endif +static inline int __rte_experimental +rte_atomic128_cmpset(volatile rte_int128_t *dst, + rte_int128_t *exp, rte_int128_t *src, + unsigned int weak, + enum rte_atomic_memmodel_t success, + enum rte_atomic_memmodel_t failure) +{ + RTE_SET_USED(weak); + RTE_SET_USED(success); + RTE_SET_USED(failure); + uint8_t res; + + asm volatile ( + MPLOCKED + "cmpxchg16b %[dst];" + " sete %[res]" + : [dst] "=m" (dst->val[0]), + "=A" (exp->val[0]), + [res] "=r" (res) + : "c" (src->val[1]), + "b" (src->val[0]), + "m" (dst->val[0]), + "d" (exp->val[1]), + "a" (exp->val[0]) + : "memory"); + + return res; +} + #endif /* _RTE_ATOMIC_X86_64_H_ */ diff --git a/lib/librte_eal/common/include/generic/rte_atomic.h b/lib/librte_eal/common/include/generic/rte_atomic.h index b99ba4688..8d612d566 100644 --- a/lib/librte_eal/common/include/generic/rte_atomic.h +++ b/lib/librte_eal/common/include/generic/rte_atomic.h @@ -14,6 +14,7 @@ #include #include +#include #ifdef __DOXYGEN__ @@ -1082,4 +1083,68 @@ static inline void rte_atomic64_clear(rte_atomic64_t *v) } #endif +/*------------------------ 128 bit atomic operations -------------------------*/ + +/** + * 128-bit integer structure. + */ +typedef struct { + uint64_t val[2]; +} __rte_aligned(16) rte_int128_t; + +/** + * Memory consistency models used in atomic operations. These control the + * behavior of the operation with respect to memory barriers and + * thread synchronization. + * + * These directly match those in the C++11 standard; for details on their + * behavior, refer to the standard. + */ +enum rte_atomic_memmodel_t { + RTE_ATOMIC_RELAXED, + RTE_ATOMIC_CONSUME, + RTE_ATOMIC_ACQUIRE, + RTE_ATOMIC_RELEASE, + RTE_ATOMIC_ACQ_REL, + RTE_ATOMIC_SEQ_CST, +}; + +/* Only implemented on x86-64 currently. The ifdef prevents compilation from + * failing for architectures without a definition of this function. + */ +#if defined(RTE_ARCH_X86_64) +/** + * An atomic compare and set function used by the mutex functions. + * (atomic) equivalent to: + * if (*dst == exp) + * *dst = src (all 128-bit words) + * + * @param dst + * The destination into which the value will be written. + * @param exp + * Pointer to the expected value. If the operation fails, this memory is + * updated with the actual value. + * @param src + * Pointer to the new value. + * @param weak + * A value of true allows the comparison to spuriously fail. Implementations + * may ignore this argument and only implement the strong variant. + * @param success + * If successful, the operation's memory behavior conforms to this (or a + * stronger) model. + * @param failure + * If unsuccessful, the operation's memory behavior conforms to this (or a + * stronger) model. This argument cannot be RTE_ATOMIC_RELEASE, + * RTE_ATOMIC_ACQ_REL, or a stronger model than success. + * @return + * Non-zero on success; 0 on failure. + */ +static inline int __rte_experimental +rte_atomic128_cmpset(volatile rte_int128_t *dst, + rte_int128_t *exp, rte_int128_t *src, + unsigned int weak, + enum rte_atomic_memmodel_t success, + enum rte_atomic_memmodel_t failure); +#endif + #endif /* _RTE_ATOMIC_H_ */