From patchwork Mon Aug 12 12:49:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 143071 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 23912457A1; Mon, 12 Aug 2024 14:59:20 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8583340687; Mon, 12 Aug 2024 14:59:09 +0200 (CEST) Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2067.outbound.protection.outlook.com [40.107.21.67]) by mails.dpdk.org (Postfix) with ESMTP id 998B84066C for ; Mon, 12 Aug 2024 14:59:06 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=aR1rb9mcJmOGjLHeXZbD4BZM5X/zYkaL765ob1yoPESn2LwpjC1BqxELxgFmr71VJKO8YC/WRJ7ADl9K9c5XpEVTJQBT6ZYxRoEKFEjE5+OqpS1nGLZbhFeNDIKbczL4aMba6GJXm06c7Uu0VdcKlTA/iMvIXMA7IqpbcClUXOYU/lI8/R/gnpGeqB3qUlXoDb4g1Y6Mx+SnILkbSCVKb0W8aUi7e8DJRR3P4I4dap/ZkR53eUz3USYd56otBbwLQ6vpUdb/yo5YcRg7RzV8/axX8hLkZnKBhNlOwdfJyT629upGn+stute6CmOJrYfWgThCEbK+1nQDeKthRNwiMw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LC4TQ3a7do+0jo/eKoqAaxOsgy9jxcb/CtIn2kIWEIE=; b=pswlkyHkUv5OqU+11iIoP9qOxTYRk8hXnOahL8P9jZCtsnRdfLP2+Zhbn4No6Z0ZPbxfeGI0SUeSKbHZQ7KlhFsK3FvVNXFDnw0r3FZHwNcZpFxTvoEkzrBMsjzI58xEdceDJcJZTcGOzJ5s0cXiyYcZHOob4eAMPvcJEy6P1yW6lzANj5gPyqC4idovsbL6L46ngLlnpHIjbsT4c4Xie7eWBzbtcoSJvnQgdjBWuD8y3FoFTAhqcyaz6MGDmh3j66UJe5tc58DmtuLljLnERcaotQzsXrhpMtzYZSCvxxkZWtBZGbdkrgZYmkuMVyQIDWJfVwVJcTZJq6TSRRkkyw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LC4TQ3a7do+0jo/eKoqAaxOsgy9jxcb/CtIn2kIWEIE=; b=xqC9R7Wzcp1lf/+x+Z306uoV9tt/q0luAp23ziR+p1+hBYJyjdIoCxnoxdE5NZ+n1nfkErxxME41AsZAJCVV00vlPkVpzCBSCXonrnJZvMlxCurNJHOTXbolrpMx0FNWM+Tgpbk+C2BILC7wqmNdovRVWtF/9LQhRGjeFjiWNxd/9eztmHavILRS7r6aD/uq8T6Q+C8Or17kGM1fLlFP8UJH7ejeHnDhA8sx6qKaTnOfdxmMoWHxpXFnU7HqlBf/ndGiMKsi66WLnhzR0f0gNUH2f8oIkLr5PZ3FJyULsXx5G7yuKANtMu2HMVSZoROIIExPZRhK01uFKzR8z1utxg== Received: from DBBPR09CA0019.eurprd09.prod.outlook.com (2603:10a6:10:c0::31) by AS8PR07MB7976.eurprd07.prod.outlook.com (2603:10a6:20b:352::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.22; Mon, 12 Aug 2024 12:59:03 +0000 Received: from DB1PEPF0003922E.eurprd03.prod.outlook.com (2603:10a6:10:c0:cafe::f9) by DBBPR09CA0019.outlook.office365.com (2603:10a6:10:c0::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.20 via Frontend Transport; Mon, 12 Aug 2024 12:59:03 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DB1PEPF0003922E.mail.protection.outlook.com (10.167.8.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.8 via Frontend Transport; Mon, 12 Aug 2024 12:59:03 +0000 Received: from seliicinfr00049.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.65) with Microsoft SMTP Server id 15.2.1544.11; Mon, 12 Aug 2024 14:59:03 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00049.seli.gic.ericsson.se (Postfix) with ESMTP id E547D380070; Mon, 12 Aug 2024 14:59:02 +0200 (CEST) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , Heng Wang , "Stephen Hemminger" , Tyler Retzlaff , =?utf-8?q?Morten_Br=C3=B8rup?= , Jack Bond-Preston , =?utf-8?q?Mattias_R?= =?utf-8?q?=C3=B6nnblom?= Subject: [PATCH v3 1/5] eal: extend bit manipulation functionality Date: Mon, 12 Aug 2024 14:49:26 +0200 Message-ID: <20240812124930.604796-2-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240812124930.604796-1-mattias.ronnblom@ericsson.com> References: <20240809095829.589396-2-mattias.ronnblom@ericsson.com> <20240812124930.604796-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB1PEPF0003922E:EE_|AS8PR07MB7976:EE_ X-MS-Office365-Filtering-Correlation-Id: a9735aed-8122-4182-fa4d-08dcbace8ae7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|1800799024|36860700013|82310400026; X-Microsoft-Antispam-Message-Info: =?utf-8?q?stcfgaCtwQ//0O4m7B1HTs9clGWtzSr?= =?utf-8?q?tMHiLVU7vZqCqumWVyEIDmjPlk3fbOuNdUtygkckfIai9HGaauMEIxOfTDF4SLcEy?= =?utf-8?q?C8Fjk5DA7NNyMAnGzuxdR7qmS2v9GlAw4IzppmsfcyPujFrTaNxhzrmEPp8zIn2ag?= =?utf-8?q?ga1UP2se7PvypZPQgK+vNWYIR+yFOV2Od9td+4LXiiUMa/oxUtdbQkg5lep6xNCMQ?= =?utf-8?q?1Ghykz7U9ECKMi2j05UPgCOHxdsHCqKPotlsXNtKaAoc56kWKAsil0UXtafshUl5d?= =?utf-8?q?yrIEm859w1NrDhjIfSkp0uRi41eZcdewhtGPO3rH8DPXO4voDCUL3AllkS/qFagl2?= =?utf-8?q?cVC4EWQ9lxOWhTRN4JpWBkhAzP5Cpq5ANAUHe9vSJjgSvOZoU+jxcjxp/I35AShX/?= =?utf-8?q?0oVongkgB2f4DBgHFte1YbHqkwvH95DsQ8Xe1CNTKxvEH+13OTMRFNTuMscGi/LyW?= =?utf-8?q?50jtdN9Jy0KFYVRZexpwDyPkOKKwtr9VSxEmq4HCSGEwhza3+5vZe09mQhwj4Gka2?= =?utf-8?q?t9seiMpOeQNRQPKLsGxHRpB6BiUngMNMpgNM6XUWPmNFb7GYBHd6HSPQwwu8QAmyk?= =?utf-8?q?LXaOo1jAMzzHiS2QcE6nz+gVgycSCCWsdhZCUZ6jbBlPf1DJkNt/J3zLMShTsRx1c?= =?utf-8?q?45/WpjktVQjXbeMX9460zKmzNW0nZpD+CXOWHs6ooZubkzamBfcMIHxiPIZ3sSzIc?= =?utf-8?q?B2xDuICmeIfqx7GWUIcYbA55ukpaQDUGs2mEsCRSzazg6wlm9+bBHRsB92qF1ahFN?= =?utf-8?q?0+1ePFO9rCMFy6NA+YVWOKpWqgu+Rb8ofAcWyr3sEcHe6s/PYrQmYpTjxMzZh8FE1?= =?utf-8?q?4zEqFOixkY29l1FD28Cf5h5Fwsgc4nHFtMzOZ8q/lgcTAR+l7kZnJ/fi0qNXq4HdW?= =?utf-8?q?emI8otwIuY2uy/ipHHmVo8ktxj8ZrGj4YWbK/DynKJ4/fgM/pe4xmmmsynzBQYeav?= =?utf-8?q?d8N9C9FtEHYAeSz8ysfyNhgAWA5icr7AIM4IAE3RGxfj3iYLX+2/btK0hQaxC1lHP?= =?utf-8?q?fF4T/XFcBiTj3nr4fPUW0uyVwsWbKzyaGOptVDDMQRd3BuIp4gbo/XeN9rbOShZJD?= =?utf-8?q?iPU944tMHBmBEDoptUJBfHNgLEzvmuorYeRcsxUz2IauwK+RKHulq7lFCK6Q55fDw?= =?utf-8?q?j8o2JuBlpOQXwfvI2oYMuOMK03Ehr6ZIkbD6a4lVQ3OgSS/VkhUBzTO6jqvM5tkwV?= =?utf-8?q?cecduWPZTZv7PEhKY/vd49QaQj6QJrf+7n/a0vjdkvaK1RjfgGWbvpEMy0tRH04rF?= =?utf-8?q?R8939qsWoLiRk7piVKRt4mmaw4Ye9G371psXUIuhlOhEummvsYFLk8O4hYd1hFg4K?= =?utf-8?q?HfD7Mk/O6/t4QlF9RD9GzXzFv3E/heYlXA=3D=3D?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230040)(376014)(1800799024)(36860700013)(82310400026); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2024 12:59:03.5474 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a9735aed-8122-4182-fa4d-08dcbace8ae7 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DB1PEPF0003922E.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR07MB7976 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add functionality to test and modify the value of individual bits in 32-bit or 64-bit words. These functions have no implications on memory ordering, atomicity and does not use volatile and thus does not prevent any compiler optimizations. Signed-off-by: Mattias Rönnblom Acked-by: Morten Brørup Acked-by: Tyler Retzlaff Acked-by: Jack Bond-Preston --- PATCH v3: * Remove unnecessary include. * Remove redundant 'fun' parameter from the __RTE_GEN_BIT_*() macros (Jack Bond-Preston). * Introduce __RTE_BIT_BIT_OPS() macro, consistent with how things are done when generating the atomic bit operations. * Refer to volatile bit op functions as variants instead of families (macro parameter naming). RFC v6: * Have rte_bit_test() accept const-marked bitsets. RFC v4: * Add rte_bit_flip() which, believe it or not, flips the value of a bit. * Mark macro-generated private functions as experimental. * Use macros to generate *assign*() functions. RFC v3: * Work around lack of C++ support for _Generic (Tyler Retzlaff). * Fix ','-related checkpatch warnings. --- lib/eal/include/rte_bitops.h | 260 ++++++++++++++++++++++++++++++++++- 1 file changed, 258 insertions(+), 2 deletions(-) diff --git a/lib/eal/include/rte_bitops.h b/lib/eal/include/rte_bitops.h index 449565eeae..6915b945ba 100644 --- a/lib/eal/include/rte_bitops.h +++ b/lib/eal/include/rte_bitops.h @@ -2,6 +2,7 @@ * Copyright(c) 2020 Arm Limited * Copyright(c) 2010-2019 Intel Corporation * Copyright(c) 2023 Microsoft Corporation + * Copyright(c) 2024 Ericsson AB */ #ifndef _RTE_BITOPS_H_ @@ -11,12 +12,14 @@ * @file * Bit Operations * - * This file defines a family of APIs for bit operations - * without enforcing memory ordering. + * This file provides functionality for low-level, single-word + * arithmetic and bit-level operations, such as counting or + * setting individual bits. */ #include +#include #include #ifdef __cplusplus @@ -105,6 +108,197 @@ extern "C" { #define RTE_FIELD_GET64(mask, reg) \ ((typeof(mask))(((reg) & (mask)) >> rte_ctz64(mask))) +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Test bit in word. + * + * Generic selection macro to test the value of a bit in a 32-bit or + * 64-bit word. The type of operation depends on the type of the @c + * addr parameter. + * + * This macro does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + */ +#define rte_bit_test(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_test32, \ + const uint32_t *: __rte_bit_test32, \ + uint64_t *: __rte_bit_test64, \ + const uint64_t *: __rte_bit_test64)(addr, nr) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Set bit in word. + * + * Generic selection macro to set a bit in a 32-bit or 64-bit + * word. The type of operation depends on the type of the @c addr + * parameter. + * + * This macro does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + */ +#define rte_bit_set(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_set32, \ + uint64_t *: __rte_bit_set64)(addr, nr) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Clear bit in word. + * + * Generic selection macro to clear a bit in a 32-bit or 64-bit + * word. The type of operation depends on the type of the @c addr + * parameter. + * + * This macro does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + */ +#define rte_bit_clear(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_clear32, \ + uint64_t *: __rte_bit_clear64)(addr, nr) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Assign a value to a bit in word. + * + * Generic selection macro to assign a value to a bit in a 32-bit or 64-bit + * word. The type of operation depends on the type of the @c addr parameter. + * + * This macro does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param value + * The new value of the bit - true for '1', or false for '0'. + */ +#define rte_bit_assign(addr, nr, value) \ + _Generic((addr), \ + uint32_t *: __rte_bit_assign32, \ + uint64_t *: __rte_bit_assign64)(addr, nr, value) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Flip a bit in word. + * + * Generic selection macro to change the value of a bit to '0' if '1' + * or '1' if '0' in a 32-bit or 64-bit word. The type of operation + * depends on the type of the @c addr parameter. + * + * This macro does not give any guarantees in regards to memory + * ordering or atomicity. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + */ +#define rte_bit_flip(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_flip32, \ + uint64_t *: __rte_bit_flip64)(addr, nr) + +#define __RTE_GEN_BIT_TEST(variant, qualifier, size) \ + __rte_experimental \ + static inline bool \ + __rte_bit_ ## variant ## test ## size(const qualifier uint ## size ## _t *addr, \ + unsigned int nr) \ + { \ + RTE_ASSERT(nr < size); \ + \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + return *addr & mask; \ + } + +#define __RTE_GEN_BIT_SET(variant, qualifier, size) \ + __rte_experimental \ + static inline void \ + __rte_bit_ ## variant ## set ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr) \ + { \ + RTE_ASSERT(nr < size); \ + \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + *addr |= mask; \ + } \ + +#define __RTE_GEN_BIT_CLEAR(variant, qualifier, size) \ + __rte_experimental \ + static inline void \ + __rte_bit_ ## variant ## clear ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr) \ + { \ + RTE_ASSERT(nr < size); \ + \ + uint ## size ## _t mask = ~((uint ## size ## _t)1 << nr); \ + (*addr) &= mask; \ + } \ + +#define __RTE_GEN_BIT_ASSIGN(variant, qualifier, size) \ + __rte_experimental \ + static inline void \ + __rte_bit_ ## variant ## assign ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr, bool value) \ + { \ + if (value) \ + __rte_bit_ ## variant ## set ## size(addr, nr); \ + else \ + __rte_bit_ ## variant ## clear ## size(addr, nr); \ + } + +#define __RTE_GEN_BIT_FLIP(variant, qualifier, size) \ + __rte_experimental \ + static inline void \ + __rte_bit_ ## variant ## flip ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr) \ + { \ + bool value; \ + \ + value = __rte_bit_ ## variant ## test ## size(addr, nr); \ + __rte_bit_ ## variant ## assign ## size(addr, nr, !value); \ + } + +#define __RTE_GEN_BIT_OPS(v, qualifier, size) \ + __RTE_GEN_BIT_TEST(v, qualifier, size) \ + __RTE_GEN_BIT_SET(v, qualifier, size) \ + __RTE_GEN_BIT_CLEAR(v, qualifier, size) \ + __RTE_GEN_BIT_ASSIGN(v, qualifier, size) \ + __RTE_GEN_BIT_FLIP(v, qualifier, size) + +#define __RTE_GEN_BIT_OPS_SIZE(size) \ + __RTE_GEN_BIT_OPS(,, size) + +__RTE_GEN_BIT_OPS_SIZE(32) +__RTE_GEN_BIT_OPS_SIZE(64) + /*------------------------ 32-bit relaxed operations ------------------------*/ /** @@ -787,6 +981,68 @@ rte_log2_u64(uint64_t v) #ifdef __cplusplus } + +/* + * Since C++ doesn't support generic selection (i.e., _Generic), + * function overloading is used instead. Such functions must be + * defined outside 'extern "C"' to be accepted by the compiler. + */ + +#undef rte_bit_test +#undef rte_bit_set +#undef rte_bit_clear +#undef rte_bit_assign +#undef rte_bit_flip + +#define __RTE_BIT_OVERLOAD_SZ_2(fun, qualifier, size, arg1_type, arg1_name) \ + static inline void \ + rte_bit_ ## fun(qualifier uint ## size ## _t *addr, \ + arg1_type arg1_name) \ + { \ + __rte_bit_ ## fun ## size(addr, arg1_name); \ + } + +#define __RTE_BIT_OVERLOAD_2(fun, qualifier, arg1_type, arg1_name) \ + __RTE_BIT_OVERLOAD_SZ_2(fun, qualifier, 32, arg1_type, arg1_name) \ + __RTE_BIT_OVERLOAD_SZ_2(fun, qualifier, 64, arg1_type, arg1_name) + +#define __RTE_BIT_OVERLOAD_SZ_2R(fun, qualifier, size, ret_type, arg1_type, \ + arg1_name) \ + static inline ret_type \ + rte_bit_ ## fun(qualifier uint ## size ## _t *addr, \ + arg1_type arg1_name) \ + { \ + return __rte_bit_ ## fun ## size(addr, arg1_name); \ + } + +#define __RTE_BIT_OVERLOAD_2R(fun, qualifier, ret_type, arg1_type, arg1_name) \ + __RTE_BIT_OVERLOAD_SZ_2R(fun, qualifier, 32, ret_type, arg1_type, \ + arg1_name) \ + __RTE_BIT_OVERLOAD_SZ_2R(fun, qualifier, 64, ret_type, arg1_type, \ + arg1_name) + +#define __RTE_BIT_OVERLOAD_SZ_3(fun, qualifier, size, arg1_type, arg1_name, \ + arg2_type, arg2_name) \ + static inline void \ + rte_bit_ ## fun(uint ## size ## _t *addr, arg1_type arg1_name, \ + arg2_type arg2_name) \ + { \ + __rte_bit_ ## fun ## size(addr, arg1_name, arg2_name); \ + } + +#define __RTE_BIT_OVERLOAD_3(fun, qualifier, arg1_type, arg1_name, arg2_type, \ + arg2_name) \ + __RTE_BIT_OVERLOAD_SZ_3(fun, qualifier, 32, arg1_type, arg1_name, \ + arg2_type, arg2_name) \ + __RTE_BIT_OVERLOAD_SZ_3(fun, qualifier, 64, arg1_type, arg1_name, \ + arg2_type, arg2_name) + +__RTE_BIT_OVERLOAD_2R(test, const, bool, unsigned int, nr) +__RTE_BIT_OVERLOAD_2(set,, unsigned int, nr) +__RTE_BIT_OVERLOAD_2(clear,, unsigned int, nr) +__RTE_BIT_OVERLOAD_3(assign,, unsigned int, nr, bool, value) +__RTE_BIT_OVERLOAD_2(flip,, unsigned int, nr) + #endif #endif /* _RTE_BITOPS_H_ */ From patchwork Mon Aug 12 12:49:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 143070 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 28ACC457A1; Mon, 12 Aug 2024 14:59:11 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CCA1240654; Mon, 12 Aug 2024 14:59:07 +0200 (CEST) Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2054.outbound.protection.outlook.com [40.107.21.54]) by mails.dpdk.org (Postfix) with ESMTP id B84224066A for ; Mon, 12 Aug 2024 14:59:05 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=wHfImwp1jg+u7r/EV49sv4UYe6aRwlcdfQTZzaBzysEk8Ni2i4iwnptWaml8zpXjMRxJWVWAyEaQFxCz45fOfVjHxTWSe98q6mxz3HLypyBXHs7Q47wgbLr/iA6ba/NDUI3VbGh9cz8aLZh+FMEH2/TYCJ5JwU/IV/jtWQ2DifIpfk2oRQb9SFhpHW/xTE5wORqhLfrnMBVkUky8GrMDnWvRkAowJdh6+DSXNpfuVcTVNuiTCec4GNwZNm/JAyN7NDqN8u60non1AZDMCL1X+QnlpAreIVtiqKb5hhrSHjBJPCxUyLouCmx7c5X9MibvDtGb73weLL/Ulu4X3rv2qQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rlj6TbwktGuphxsLT7E1iv4WZ/+LLJlVUku+K2BR9LA=; b=vWKS0hu9E8XknSD16EEBS+xL5lZUp4AYR/nSvErDwk7yegFhUgzH1oyTxScHAc0o+zFYUCiPNjbV97QKq78xJrghe3qVJjwT88Ebp45+owZOOzjh6jPjoV92kELhOStah3gFzcj6qRpCwOZir4Ue2/Ev9f9qUcxuT5WCGLPVbLYjUEsH4zFhIYyOItlGrqmPQSZC8kPDfk9YDxDIHQ5cpHWzwlqaSChBGjJAM+QmKYgQ/yGgAzq/5BwmGXUOxklGt6Rx/qs9fCC2873kxXF6hmCDOz3apJMK+Efdy9+eOpHDczl00sjlWdULfDk1OsZ4PcXsU/2WDFCGLROl2IMQBQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rlj6TbwktGuphxsLT7E1iv4WZ/+LLJlVUku+K2BR9LA=; b=cT//4uHMtaZRxzNQBYvoDfFwweIgWDP8TF01QA3cFLzrzJ92EvYPfgnC10ROQjr/5KxJXaDDaTlkALlHzdsRPn+/2hw258zwOccvsgIZZGH4qxQ2ZGXwiIZr8XA4uCTr8rl3UBB4zGhqH0OoAf9yylN4uSdTn4BcIK7o0OpKiTauiVaysG+yw0nrCjLbrZ1oGgA7O0uKoFIbrwiR7QPGMWICdqauJw7c15uTTdVT3N/5NCbOJ6HCpgcgDcGJTzefVVGog7T9bOzH+hPWnsQmhEOPB0IOuKWGR8uFsXyPckV4QFqjuz1cd/yEDV3GWiX5iqc1GrwuKfFsZmJNkiMh6A== Received: from AM0PR06CA0144.eurprd06.prod.outlook.com (2603:10a6:208:ab::49) by VI1PR07MB6431.eurprd07.prod.outlook.com (2603:10a6:800:13f::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.22; Mon, 12 Aug 2024 12:59:03 +0000 Received: from AM3PEPF00009BA1.eurprd04.prod.outlook.com (2603:10a6:208:ab:cafe::b1) by AM0PR06CA0144.outlook.office365.com (2603:10a6:208:ab::49) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.20 via Frontend Transport; Mon, 12 Aug 2024 12:59:03 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by AM3PEPF00009BA1.mail.protection.outlook.com (10.167.16.26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.8 via Frontend Transport; Mon, 12 Aug 2024 12:59:03 +0000 Received: from seliicinfr00049.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.61) with Microsoft SMTP Server id 15.2.1544.11; Mon, 12 Aug 2024 14:59:03 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00049.seli.gic.ericsson.se (Postfix) with ESMTP id 0357038007A; Mon, 12 Aug 2024 14:59:03 +0200 (CEST) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , Heng Wang , "Stephen Hemminger" , Tyler Retzlaff , =?utf-8?q?Morten_Br=C3=B8rup?= , Jack Bond-Preston , =?utf-8?q?Mattias_R?= =?utf-8?q?=C3=B6nnblom?= Subject: [PATCH v3 2/5] eal: add unit tests for bit operations Date: Mon, 12 Aug 2024 14:49:27 +0200 Message-ID: <20240812124930.604796-3-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240812124930.604796-1-mattias.ronnblom@ericsson.com> References: <20240809095829.589396-2-mattias.ronnblom@ericsson.com> <20240812124930.604796-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM3PEPF00009BA1:EE_|VI1PR07MB6431:EE_ X-MS-Office365-Filtering-Correlation-Id: d1a75e42-5396-42e4-4a4e-08dcbace8adc X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|82310400026|36860700013|376014; X-Microsoft-Antispam-Message-Info: =?utf-8?q?7FYBP2IHU0FVQs5WysoYM/TuEbAZen7?= =?utf-8?q?YB44/oo4SCfV7SenX/VJTTaTJ6W1XQCvxMs3GlkabZMjfwoK3ztUfgORAtQxrhthz?= =?utf-8?q?JdADA9EAguhDCCMRr/uLJZYQfUPZB8BoxAxTW3Z4RtG6ijANB6Ei6SL7HkwxlUdOW?= =?utf-8?q?VTh7zoc/wU7kpxaOU6r7FWDvWNR7SBlw806nMY6REg5kbJ2yok0D8Q3tTOTEtFQAx?= =?utf-8?q?qnYPYjRMZ0puM8fpJm1g4LOldDmWjWkgxb/Gl6Ivv08EgVzglA4prpGLosrt/6VlE?= =?utf-8?q?9RmoJKj3ZQtx6ROI/wQFyODTcMbwUovl+4XLnZYI64XcI+h5WqcPIS37NY9SI13uO?= =?utf-8?q?RENZ0GZmhZe4ND3DGwrsXLeVDMbHo6cHNETXm9DAJdmEUoqSAsnM85vt5FA9KN6td?= =?utf-8?q?a6KFV6p/CGfeoqQbzXdgXA9hBCDGCeO2QeOxe1masQleV/ByDNuDrYSd2zu5KsGxW?= =?utf-8?q?dRdblqBpd6rzJ6rNIiZyKwPpFLZoNbNOEWUjjvXUFakrBdhdPVkfg9VqeA/rt0Sd+?= =?utf-8?q?qQO2wsllopB+Px/V02yGhWCCFSnFTBeQEWa3HS26iTt3Zn3lXsqjgX9xBG7QkQ3Ao?= =?utf-8?q?oNT9QALSNHTvMGW7kJQ3lgWcDUXcnIZlA5neYU3gzaltpPMZKAyvb2uV4f8+qunBp?= =?utf-8?q?MpwsmL/fNbQDoKATv3RIqRpXAG/n1wi/26iJrqclulvywYoxTvbnBG1ZzhaXY4ttY?= =?utf-8?q?Yaj/yxQ2fpVZjSgGYNPRSgmLPIiNnnebRmqr8RoHnJq+4VEu+g6we2wKCvOugJhWY?= =?utf-8?q?wC2puOHCf8znLYj0+pBf72gULyGsWxxqOEa4G4xezHdPG1pM3s0e7wAJH7n4TPd+S?= =?utf-8?q?LUF1lwd4k6Z9NieCOM5gRiLya48sg86XqEUIK3lGsDo/JgqQzscJbM64O1HKaChmH?= =?utf-8?q?NWu3WpaYww3kca+NB9uEnqELDyPj88w3iX59/ChinVGirycF6rNeHMlP/nNnNYXpS?= =?utf-8?q?LaaUj6lrMsbIqrOYIUlAn/DOQM3gnKn5QOS9FlgmCFU/BKUsxq03zp8CPWuQJQwCt?= =?utf-8?q?9/NmBjmqiMHn4GcxPgy7xl9mlNYDaV5Hr15rTBbzlJj6YiUsRlUn+w+pfh6lNNU5o?= =?utf-8?q?4mfNvmE2Xs5361tOXBOp3gRwT2GMpGg+aNhGZnQ+AyABK/DsReGiXEdJPlnTLsHIU?= =?utf-8?q?qf7S+4N5QwJXCoX2MlTkOOOfM8sDnPAPK1uiPLSJ1XDmxPOvH6t4/BfxRay6lORKI?= =?utf-8?q?Y0ErR0ubEmro1Ok3/49/GWbeAJouw3xL8cH0LVZ+vKENwKIgP10ys8vh4VOvfjhzn?= =?utf-8?q?VNHn39wY0DbDNWW/3rxa70U/vIhM2o4wqj82zHqoV05uYJkx608rECp7K/M6Kqr+2?= =?utf-8?q?WJVrkwk/klHs4dtrBok4K0pRSIuuvEXqDA=3D=3D?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230040)(1800799024)(82310400026)(36860700013)(376014); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2024 12:59:03.5021 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d1a75e42-5396-42e4-4a4e-08dcbace8adc X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: AM3PEPF00009BA1.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR07MB6431 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Extend bitops tests to cover the rte_bit_[test|set|clear|assign|flip]() functions. The tests are converted to use the test suite runner framework. Signed-off-by: Mattias Rönnblom Acked-by: Morten Brørup Acked-by: Tyler Retzlaff Acked-by: Jack Bond-Preston --- RFC v6: * Test rte_bit_*test() usage through const pointers. RFC v4: * Remove redundant line continuations. --- app/test/test_bitops.c | 85 ++++++++++++++++++++++++++++++++++-------- 1 file changed, 70 insertions(+), 15 deletions(-) diff --git a/app/test/test_bitops.c b/app/test/test_bitops.c index 0d4ccfb468..322f58c066 100644 --- a/app/test/test_bitops.c +++ b/app/test/test_bitops.c @@ -1,13 +1,68 @@ /* SPDX-License-Identifier: BSD-3-Clause * Copyright(c) 2019 Arm Limited + * Copyright(c) 2024 Ericsson AB */ +#include + #include #include +#include #include "test.h" -uint32_t val32; -uint64_t val64; +#define GEN_TEST_BIT_ACCESS(test_name, set_fun, clear_fun, assign_fun, \ + flip_fun, test_fun, size) \ + static int \ + test_name(void) \ + { \ + uint ## size ## _t reference = (uint ## size ## _t)rte_rand(); \ + unsigned int bit_nr; \ + uint ## size ## _t word = (uint ## size ## _t)rte_rand(); \ + \ + for (bit_nr = 0; bit_nr < size; bit_nr++) { \ + bool reference_bit = (reference >> bit_nr) & 1; \ + bool assign = rte_rand() & 1; \ + if (assign) \ + assign_fun(&word, bit_nr, reference_bit); \ + else { \ + if (reference_bit) \ + set_fun(&word, bit_nr); \ + else \ + clear_fun(&word, bit_nr); \ + \ + } \ + TEST_ASSERT(test_fun(&word, bit_nr) == reference_bit, \ + "Bit %d had unexpected value", bit_nr); \ + flip_fun(&word, bit_nr); \ + TEST_ASSERT(test_fun(&word, bit_nr) != reference_bit, \ + "Bit %d had unflipped value", bit_nr); \ + flip_fun(&word, bit_nr); \ + \ + const uint ## size ## _t *const_ptr = &word; \ + TEST_ASSERT(test_fun(const_ptr, bit_nr) == \ + reference_bit, \ + "Bit %d had unexpected value", bit_nr); \ + } \ + \ + for (bit_nr = 0; bit_nr < size; bit_nr++) { \ + bool reference_bit = (reference >> bit_nr) & 1; \ + TEST_ASSERT(test_fun(&word, bit_nr) == reference_bit, \ + "Bit %d had unexpected value", bit_nr); \ + } \ + \ + TEST_ASSERT(reference == word, "Word had unexpected value"); \ + \ + return TEST_SUCCESS; \ + } + +GEN_TEST_BIT_ACCESS(test_bit_access32, rte_bit_set, rte_bit_clear, + rte_bit_assign, rte_bit_flip, rte_bit_test, 32) + +GEN_TEST_BIT_ACCESS(test_bit_access64, rte_bit_set, rte_bit_clear, + rte_bit_assign, rte_bit_flip, rte_bit_test, 64) + +static uint32_t val32; +static uint64_t val64; #define MAX_BITS_32 32 #define MAX_BITS_64 64 @@ -117,22 +172,22 @@ test_bit_relaxed_test_set_clear(void) return TEST_SUCCESS; } +static struct unit_test_suite test_suite = { + .suite_name = "Bitops test suite", + .unit_test_cases = { + TEST_CASE(test_bit_access32), + TEST_CASE(test_bit_access64), + TEST_CASE(test_bit_relaxed_set), + TEST_CASE(test_bit_relaxed_clear), + TEST_CASE(test_bit_relaxed_test_set_clear), + TEST_CASES_END() + } +}; + static int test_bitops(void) { - val32 = 0; - val64 = 0; - - if (test_bit_relaxed_set() < 0) - return TEST_FAILED; - - if (test_bit_relaxed_clear() < 0) - return TEST_FAILED; - - if (test_bit_relaxed_test_set_clear() < 0) - return TEST_FAILED; - - return TEST_SUCCESS; + return unit_test_suite_runner(&test_suite); } REGISTER_FAST_TEST(bitops_autotest, true, true, test_bitops); From patchwork Mon Aug 12 12:49:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 143073 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 11BCA457A1; Mon, 12 Aug 2024 14:59:34 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1F53E40695; Mon, 12 Aug 2024 14:59:13 +0200 (CEST) Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2086.outbound.protection.outlook.com [40.107.22.86]) by mails.dpdk.org (Postfix) with ESMTP id 7604840676 for ; Mon, 12 Aug 2024 14:59:08 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=DQ/vEb3YASSwKMiTsvBIwPErsO1gSgWYFKHDuiS0Nz5Tp0qaa2Y5BhxpXUeHRXBOcE5EPqmeYpt7U3M+3wI7/hhLPcggs866uhCBC0HCAGcVvIMtW83+ltDq58NsHSKrMmxZEoTfiKzC6CWWY2M+58HdZFNRe7S72+HBSj8ffsVmLD2R7DbwhFQA32WLMFXTehgP4wSc7oeVw8+Dv4FJPoerubfPHC0nirDjYkczX0aA7w9c7PPdGN1XpWEoDP7XToxU/buT0OJNDAaIxhGKxXfdrc0dYPF7ISmrse/lWteEv2O1Errc0iL+PlBYd3eWgchVqv8hTNyMsul7WMo4Gg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=p2bopKrgX6j+qUbGtKzogv0UOwOffVwUw76s5SnN30o=; b=felrAzafSAIJRlPHeQenzwo1lGaRZO+gv14A8AFN54bTrw/dk/IUHzKv/t7Zz8JnuyP+AwRE8uq1E8MUFWT40YN6YSOCzfZxOfmKYSDtz8rVQdUFD/YE/8FMAbWLg/3PRyT9q1+mKJFRKP+HbVkMLBgUTmx6qeY4LvLhpk0bblbT9ngnWozf/WUWZAyCnN9rsze3nA2XPH3MUqzWHDbGqVJfBLd2+JfgT9GgpU0MyYwLP4Boi/cv/PhO5O6IIhfE/A/On+wJpdcxERqlH5aqZgmSkg3nGbYsYb2Aa5iu+sI9NJmujEGwXRRpvt949lJqWdO7wTx9wGh1ymr7QVU/2Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=p2bopKrgX6j+qUbGtKzogv0UOwOffVwUw76s5SnN30o=; b=BZ9Yg2ynyXMUCrRYlwL+oVZBgdlwmhGcVOaI7waYeN7cnjST71siH7qopty0YyY5vBZRX8aPtPzReskbTlqiVPlx2kQ/Ih+AjdD1LBoidHYP/W4cF55dDs3CRAP2ufDc0MFK1hsEr1XvQk4/uh3AeiT4ED0yw+T/yhyju20/H/eR8jwsqwUDYiR1VM371Tg8EwYFeDj7TXzHulNuV+qiqCjCHksyyZ5R0dXaYEjOZaxaLTyB0E3jIjhVOfjXmUy/nzdy8XpUqShhmMYGsd48IdYSF6BeBYcRZQSr+se1ZbLD9r4QghHI+pXvA6pq1dcEOx5scNIFYK2rmDWMFs44tw== Received: from DU7P194CA0008.EURP194.PROD.OUTLOOK.COM (2603:10a6:10:553::25) by DU0PR07MB9218.eurprd07.prod.outlook.com (2603:10a6:10:42e::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.19; Mon, 12 Aug 2024 12:59:04 +0000 Received: from DB1PEPF000509EA.eurprd03.prod.outlook.com (2603:10a6:10:553:cafe::cb) by DU7P194CA0008.outlook.office365.com (2603:10a6:10:553::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.20 via Frontend Transport; Mon, 12 Aug 2024 12:59:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by DB1PEPF000509EA.mail.protection.outlook.com (10.167.242.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.8 via Frontend Transport; Mon, 12 Aug 2024 12:59:04 +0000 Received: from seliicinfr00049.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.67) with Microsoft SMTP Server id 15.2.1544.11; Mon, 12 Aug 2024 14:59:03 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00049.seli.gic.ericsson.se (Postfix) with ESMTP id 13B3C380061; Mon, 12 Aug 2024 14:59:03 +0200 (CEST) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , Heng Wang , "Stephen Hemminger" , Tyler Retzlaff , =?utf-8?q?Morten_Br=C3=B8rup?= , Jack Bond-Preston , =?utf-8?q?Mattias_R?= =?utf-8?q?=C3=B6nnblom?= Subject: [PATCH v3 3/5] eal: add atomic bit operations Date: Mon, 12 Aug 2024 14:49:28 +0200 Message-ID: <20240812124930.604796-4-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240812124930.604796-1-mattias.ronnblom@ericsson.com> References: <20240809095829.589396-2-mattias.ronnblom@ericsson.com> <20240812124930.604796-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB1PEPF000509EA:EE_|DU0PR07MB9218:EE_ X-MS-Office365-Filtering-Correlation-Id: 6170513a-56ec-4c99-a8ca-08dcbace8b6f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|36860700013|82310400026|1800799024|376014; X-Microsoft-Antispam-Message-Info: =?utf-8?q?uca3gNJXtsg5oP+XEuqlOnLAY5oCbBu?= =?utf-8?q?WHKnUiAPP7O6lKuBzoYfTscIA9DfuUU9V4ghtmKvtrFRgxiZ00NdYQ3przO44X588?= =?utf-8?q?YA3LlYKzf7cU7DRK8EacgmLFoWY2Fj0So3e7ZKpCpHBvum0DFWPnQ3R5zm328+H0G?= =?utf-8?q?RG4buEjkeIfCT/g2NIqcIbpQZxgF5m2fy+sUhOPBylbVFP7Q2cbG67Ys3uSWACO2N?= =?utf-8?q?9+LEcxYcdU7b6kkBMs8qzM9ZMyId8T8gsVsKYAMwYPJfOLQwc157lX1noZ96vttFj?= =?utf-8?q?NN61ZmRljXZntOlD+2eolr6swiqmjORx9TUH1vWrSB7OfaBAX5+cmGTEf5VAvWqfu?= =?utf-8?q?N9BBKODByayrZpnQqnBxCW6nDNDTq6qHMPyGSmUZFY9UIsc1jv7fb2Hm2XCb0MNDP?= =?utf-8?q?lriDfsfXTNloqVV6iUXCrRfodt1jt2iP5nivfRemqLnsACbeR8eqyzdxffwB9iU00?= =?utf-8?q?CGsPx7slgSFUVKbzMfY7ym5C6kLGszIwUwreth9BeU8NGx47ii9+Z/c9Rq2ARcP1X?= =?utf-8?q?6BYi0LXUC76Mx37HoucBpsmx+QzLobVBXbk9iXxPtKKa8hy89b70YsaDu0hO9kpEP?= =?utf-8?q?dlVIyojdZSc0294rfqwEFRLLRyqYG1iWT71ORvLyJj+FqKR+FtmoQg4Lx9vjMNBYr?= =?utf-8?q?LPRKh9e9HqTqqq5o5Va4lBItFKKEutzF5ztlHn6tmBdjtYBYr7RU2OfOH2mMCOC3E?= =?utf-8?q?EshgWuoCqgZ3wPmb7D5LJ6nQBBPMNsN5jYILst4sABjslzIVe3B05ZmxIS3SgTUft?= =?utf-8?q?3ZxEZqUTTfEpz/Rx8U2KxtHg3Nll+x6z5YNj7Gu/rSren0CQ4qx+o0BY93cuKInY5?= =?utf-8?q?EwahI8Au2SsVH2uunXYwY1FL39T8EmY2XW7FJO8TdB+nLOvMMyg1ZAa1pQ+0WrHAC?= =?utf-8?q?wFcnkr172PGyBlmXnAiPPspqYsXKxsP5Y6JV9rdIfJASqW9e45LwHbB4loEgDJeYs?= =?utf-8?q?CM4wCk7FXPORTsPJzpl3znTLHqx/m+LEKXb1hUpXzjiQW5HPkqgD3+VIXTT8eBDAW?= =?utf-8?q?E5d+Y7JQsb2tHyPTCLfM/7ntjP7GjZnyRZfYBsYbxLJ08rc5w1h6orKvsiAymLGLs?= =?utf-8?q?OtFAMuJLMvWHYzoygQZDA2F/tChIhm1Xx3MtQ7kfaj8mm6BPjkPcNsB+eCTkYZDmV?= =?utf-8?q?iJ8W9JBIZzVsW5DFgNDKWgzgsGkN/fN3H4hP88d3ZKDfC0n+RyddjcOr9DEiE/6Ph?= =?utf-8?q?/Zg9aTy5/eqQOhjbRAmC6oJU9d9M5oW0W+r+vlvp1tT35EjB1LM5TzxUQAvdZIqDi?= =?utf-8?q?OutGJnVbSjdBbAZde88z30KGdY8dfTC8/p5koSKnQ9J/BltGHZhX4PA0/yMPyx/HO?= =?utf-8?q?t8+YZX7kyciM9b66meqJz2NFBaf1vtiJEQ=3D=3D?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230040)(36860700013)(82310400026)(1800799024)(376014); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2024 12:59:04.4366 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6170513a-56ec-4c99-a8ca-08dcbace8b6f X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: DB1PEPF000509EA.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR07MB9218 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add atomic bit test/set/clear/assign/flip and test-and-set/clear/assign/flip functions. All atomic bit functions allow (and indeed, require) the caller to specify a memory order. Signed-off-by: Mattias Rönnblom Acked-by: Morten Brørup Acked-by: Tyler Retzlaff Acked-by: Jack Bond-Preston --- PATCH v3: * Introduce __RTE_GEN_BIT_ATOMIC_*() 'qualifier' argument already in this patch (Jack Bond-Preston). * Refer to volatile bit op functions as variants instead of families (macro parameter naming). * Update release notes. PATCH: * Add missing macro #undef for C++ version of atomic bit flip. RFC v7: * Replace compare-exchange-based rte_bitset_atomic_test_and_*() and flip() with implementations that use the previous value as returned by the atomic fetch function. * Reword documentation to match the non-atomic macro variants. * Remove pointer to for memory model documentation, since there is no documentation for that API. RFC v6: * Have rte_bit_atomic_test() accept const-marked bitsets. RFC v4: * Add atomic bit flip. * Mark macro-generated private functions experimental. RFC v3: * Work around lack of C++ support for _Generic (Tyler Retzlaff). RFC v2: o Add rte_bit_atomic_test_and_assign() (for consistency). o Fix bugs in rte_bit_atomic_test_and_[set|clear](). o Use to support MSVC. --- doc/guides/rel_notes/release_24_11.rst | 17 + lib/eal/include/rte_bitops.h | 415 +++++++++++++++++++++++++ 2 files changed, 432 insertions(+) diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst index 0ff70d9057..3111b1e4c0 100644 --- a/doc/guides/rel_notes/release_24_11.rst +++ b/doc/guides/rel_notes/release_24_11.rst @@ -56,6 +56,23 @@ New Features ======================================================= +* **Extended bit operations API.** + + The support for bit-level operations on single 32- and 64-bit words + in has been extended with two families of + semantically well-defined functions. + + rte_bit_[test|set|clear|assign|flip]() functions provide excellent + performance (by avoiding restricting the compiler and CPU), but give + no guarantees in regards to memory ordering or atomicity. + + rte_bit_atomic_*() provides atomic bit-level operations, including + the possibility to specifying memory ordering constraints. + + The new public API elements are polymorphic, using the _Generic- + based macros (for C) and function overloading (in C++ translation + units). + Removed Items ------------- diff --git a/lib/eal/include/rte_bitops.h b/lib/eal/include/rte_bitops.h index 6915b945ba..3ad6795fd1 100644 --- a/lib/eal/include/rte_bitops.h +++ b/lib/eal/include/rte_bitops.h @@ -21,6 +21,7 @@ #include #include +#include #ifdef __cplusplus extern "C" { @@ -226,6 +227,204 @@ extern "C" { uint32_t *: __rte_bit_flip32, \ uint64_t *: __rte_bit_flip64)(addr, nr) +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Test if a particular bit in a word is set with a particular memory + * order. + * + * Test a bit with the resulting memory load ordered as per the + * specified memory order. + * + * @param addr + * A pointer to the word to query. + * @param nr + * The index of the bit. + * @param memory_order + * The memory order to use. + * @return + * Returns true if the bit is set, and false otherwise. + */ +#define rte_bit_atomic_test(addr, nr, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_test32, \ + const uint32_t *: __rte_bit_atomic_test32, \ + uint64_t *: __rte_bit_atomic_test64, \ + const uint64_t *: __rte_bit_atomic_test64)(addr, nr, \ + memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically set bit in word. + * + * Generic selection macro to atomically set bit specified by @c nr in + * the word pointed to by @c addr to '1', with the memory ordering as + * specified by @c memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param memory_order + * The memory order to use. + */ +#define rte_bit_atomic_set(addr, nr, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_set32, \ + uint64_t *: __rte_bit_atomic_set64)(addr, nr, memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically clear bit in word. + * + * Generic selection macro to atomically set bit specified by @c nr in + * the word pointed to by @c addr to '0', with the memory ordering as + * specified by @c memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param memory_order + * The memory order to use. + */ +#define rte_bit_atomic_clear(addr, nr, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_clear32, \ + uint64_t *: __rte_bit_atomic_clear64)(addr, nr, memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically assign a value to bit in word. + * + * Generic selection macro to atomically set bit specified by @c nr in the + * word pointed to by @c addr to the value indicated by @c value, with + * the memory ordering as specified with @c memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param value + * The new value of the bit - true for '1', or false for '0'. + * @param memory_order + * The memory order to use. + */ +#define rte_bit_atomic_assign(addr, nr, value, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_assign32, \ + uint64_t *: __rte_bit_atomic_assign64)(addr, nr, value, \ + memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically flip bit in word. + * + * Generic selection macro to atomically negate the value of the bit + * specified by @c nr in the word pointed to by @c addr to the value + * indicated by @c value, with the memory ordering as specified with + * @c memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param memory_order + * The memory order to use. + */ +#define rte_bit_atomic_flip(addr, nr, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_flip32, \ + uint64_t *: __rte_bit_atomic_flip64)(addr, nr, memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically test and set a bit in word. + * + * Generic selection macro to atomically test and set bit specified by + * @c nr in the word pointed to by @c addr to '1', with the memory + * ordering as specified with @c memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param memory_order + * The memory order to use. + * @return + * Returns true if the bit was set, and false otherwise. + */ +#define rte_bit_atomic_test_and_set(addr, nr, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_test_and_set32, \ + uint64_t *: __rte_bit_atomic_test_and_set64)(addr, nr, \ + memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically test and clear a bit in word. + * + * Generic selection macro to atomically test and clear bit specified + * by @c nr in the word pointed to by @c addr to '0', with the memory + * ordering as specified with @c memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param memory_order + * The memory order to use. + * @return + * Returns true if the bit was set, and false otherwise. + */ +#define rte_bit_atomic_test_and_clear(addr, nr, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_test_and_clear32, \ + uint64_t *: __rte_bit_atomic_test_and_clear64)(addr, nr, \ + memory_order) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Atomically test and assign a bit in word. + * + * Generic selection macro to atomically test and assign bit specified + * by @c nr in the word pointed to by @c addr the value specified by + * @c value, with the memory ordering as specified with @c + * memory_order. + * + * @param addr + * A pointer to the word to modify. + * @param nr + * The index of the bit. + * @param value + * The new value of the bit - true for '1', or false for '0'. + * @param memory_order + * The memory order to use. + * @return + * Returns true if the bit was set, and false otherwise. + */ +#define rte_bit_atomic_test_and_assign(addr, nr, value, memory_order) \ + _Generic((addr), \ + uint32_t *: __rte_bit_atomic_test_and_assign32, \ + uint64_t *: __rte_bit_atomic_test_and_assign64)(addr, nr, \ + value, \ + memory_order) + #define __RTE_GEN_BIT_TEST(variant, qualifier, size) \ __rte_experimental \ static inline bool \ @@ -299,6 +498,146 @@ extern "C" { __RTE_GEN_BIT_OPS_SIZE(32) __RTE_GEN_BIT_OPS_SIZE(64) +#define __RTE_GEN_BIT_ATOMIC_TEST(variant, qualifier, size) \ + __rte_experimental \ + static inline bool \ + __rte_bit_atomic_ ## variant ## test ## size(const qualifier uint ## size ## _t *addr, \ + unsigned int nr, int memory_order) \ + { \ + RTE_ASSERT(nr < size); \ + \ + const qualifier RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (const qualifier RTE_ATOMIC(uint ## size ## _t) *)addr; \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + return rte_atomic_load_explicit(a_addr, memory_order) & mask; \ + } + +#define __RTE_GEN_BIT_ATOMIC_SET(variant, qualifier, size) \ + __rte_experimental \ + static inline void \ + __rte_bit_atomic_ ## variant ## set ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr, int memory_order) \ + { \ + RTE_ASSERT(nr < size); \ + \ + qualifier RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (qualifier RTE_ATOMIC(uint ## size ## _t) *)addr; \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + rte_atomic_fetch_or_explicit(a_addr, mask, memory_order); \ + } + +#define __RTE_GEN_BIT_ATOMIC_CLEAR(variant, qualifier, size) \ + __rte_experimental \ + static inline void \ + __rte_bit_atomic_ ## variant ## clear ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr, int memory_order) \ + { \ + RTE_ASSERT(nr < size); \ + \ + qualifier RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (qualifier RTE_ATOMIC(uint ## size ## _t) *)addr; \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + rte_atomic_fetch_and_explicit(a_addr, ~mask, memory_order); \ + } + +#define __RTE_GEN_BIT_ATOMIC_FLIP(variant, qualifier, size) \ + __rte_experimental \ + static inline void \ + __rte_bit_atomic_ ## variant ## flip ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr, int memory_order) \ + { \ + RTE_ASSERT(nr < size); \ + \ + qualifier RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (qualifier RTE_ATOMIC(uint ## size ## _t) *)addr; \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + rte_atomic_fetch_xor_explicit(a_addr, mask, memory_order); \ + } + +#define __RTE_GEN_BIT_ATOMIC_ASSIGN(variant, qualifier, size) \ + __rte_experimental \ + static inline void \ + __rte_bit_atomic_## variant ## assign ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr, bool value, \ + int memory_order) \ + { \ + if (value) \ + __rte_bit_atomic_ ## variant ## set ## size(addr, nr, memory_order); \ + else \ + __rte_bit_atomic_ ## variant ## clear ## size(addr, nr, \ + memory_order); \ + } + +#define __RTE_GEN_BIT_ATOMIC_TEST_AND_SET(variant, qualifier, size) \ + __rte_experimental \ + static inline bool \ + __rte_bit_atomic_ ## variant ## test_and_set ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr, \ + int memory_order) \ + { \ + RTE_ASSERT(nr < size); \ + \ + qualifier RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (qualifier RTE_ATOMIC(uint ## size ## _t) *)addr; \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + uint ## size ## _t prev; \ + \ + prev = rte_atomic_fetch_or_explicit(a_addr, mask, \ + memory_order); \ + \ + return prev & mask; \ + } + +#define __RTE_GEN_BIT_ATOMIC_TEST_AND_CLEAR(variant, qualifier, size) \ + __rte_experimental \ + static inline bool \ + __rte_bit_atomic_ ## variant ## test_and_clear ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr, \ + int memory_order) \ + { \ + RTE_ASSERT(nr < size); \ + \ + qualifier RTE_ATOMIC(uint ## size ## _t) *a_addr = \ + (qualifier RTE_ATOMIC(uint ## size ## _t) *)addr; \ + uint ## size ## _t mask = (uint ## size ## _t)1 << nr; \ + uint ## size ## _t prev; \ + \ + prev = rte_atomic_fetch_and_explicit(a_addr, ~mask, \ + memory_order); \ + \ + return prev & mask; \ + } + +#define __RTE_GEN_BIT_ATOMIC_TEST_AND_ASSIGN(variant, qualifier, size) \ + __rte_experimental \ + static inline bool \ + __rte_bit_atomic_ ## variant ## test_and_assign ## size(qualifier uint ## size ## _t *addr, \ + unsigned int nr, \ + bool value, \ + int memory_order) \ + { \ + if (value) \ + return __rte_bit_atomic_ ## variant ## test_and_set ## size(addr, nr, memory_order); \ + else \ + return __rte_bit_atomic_ ## variant ## test_and_clear ## size(addr, nr, memory_order); \ + } + +#define __RTE_GEN_BIT_ATOMIC_OPS(variant, qualifier, size) \ + __RTE_GEN_BIT_ATOMIC_TEST(variant, qualifier, size) \ + __RTE_GEN_BIT_ATOMIC_SET(variant, qualifier, size) \ + __RTE_GEN_BIT_ATOMIC_CLEAR(variant, qualifier, size) \ + __RTE_GEN_BIT_ATOMIC_ASSIGN(variant, qualifier, size) \ + __RTE_GEN_BIT_ATOMIC_TEST_AND_SET(variant, qualifier, size) \ + __RTE_GEN_BIT_ATOMIC_TEST_AND_CLEAR(variant, qualifier, size) \ + __RTE_GEN_BIT_ATOMIC_TEST_AND_ASSIGN(variant, qualifier, size) \ + __RTE_GEN_BIT_ATOMIC_FLIP(variant, qualifier, size) + +#define __RTE_GEN_BIT_ATOMIC_OPS_SIZE(size) \ + __RTE_GEN_BIT_ATOMIC_OPS(,, size) + +__RTE_GEN_BIT_ATOMIC_OPS_SIZE(32) +__RTE_GEN_BIT_ATOMIC_OPS_SIZE(64) + /*------------------------ 32-bit relaxed operations ------------------------*/ /** @@ -994,6 +1333,15 @@ rte_log2_u64(uint64_t v) #undef rte_bit_assign #undef rte_bit_flip +#undef rte_bit_atomic_test +#undef rte_bit_atomic_set +#undef rte_bit_atomic_clear +#undef rte_bit_atomic_assign +#undef rte_bit_atomic_flip +#undef rte_bit_atomic_test_and_set +#undef rte_bit_atomic_test_and_clear +#undef rte_bit_atomic_test_and_assign + #define __RTE_BIT_OVERLOAD_SZ_2(fun, qualifier, size, arg1_type, arg1_name) \ static inline void \ rte_bit_ ## fun(qualifier uint ## size ## _t *addr, \ @@ -1037,12 +1385,79 @@ rte_log2_u64(uint64_t v) __RTE_BIT_OVERLOAD_SZ_3(fun, qualifier, 64, arg1_type, arg1_name, \ arg2_type, arg2_name) +#define __RTE_BIT_OVERLOAD_SZ_3R(fun, qualifier, size, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name) \ + static inline ret_type \ + rte_bit_ ## fun(uint ## size ## _t *addr, arg1_type arg1_name, \ + arg2_type arg2_name) \ + { \ + return __rte_bit_ ## fun ## size(addr, arg1_name, arg2_name); \ + } + +#define __RTE_BIT_OVERLOAD_3R(fun, qualifier, ret_type, arg1_type, arg1_name, \ + arg2_type, arg2_name) \ + __RTE_BIT_OVERLOAD_SZ_3R(fun, qualifier, 32, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name) \ + __RTE_BIT_OVERLOAD_SZ_3R(fun, qualifier, 64, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name) + +#define __RTE_BIT_OVERLOAD_SZ_4(fun, qualifier, size, arg1_type, arg1_name, \ + arg2_type, arg2_name, arg3_type, arg3_name) \ + static inline void \ + rte_bit_ ## fun(uint ## size ## _t *addr, arg1_type arg1_name, \ + arg2_type arg2_name, arg3_type arg3_name) \ + { \ + __rte_bit_ ## fun ## size(addr, arg1_name, arg2_name, \ + arg3_name); \ + } + +#define __RTE_BIT_OVERLOAD_4(fun, qualifier, arg1_type, arg1_name, arg2_type, \ + arg2_name, arg3_type, arg3_name) \ + __RTE_BIT_OVERLOAD_SZ_4(fun, qualifier, 32, arg1_type, arg1_name, \ + arg2_type, arg2_name, arg3_type, arg3_name) \ + __RTE_BIT_OVERLOAD_SZ_4(fun, qualifier, 64, arg1_type, arg1_name, \ + arg2_type, arg2_name, arg3_type, arg3_name) + +#define __RTE_BIT_OVERLOAD_SZ_4R(fun, qualifier, size, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name, arg3_type, \ + arg3_name) \ + static inline ret_type \ + rte_bit_ ## fun(uint ## size ## _t *addr, arg1_type arg1_name, \ + arg2_type arg2_name, arg3_type arg3_name) \ + { \ + return __rte_bit_ ## fun ## size(addr, arg1_name, arg2_name, \ + arg3_name); \ + } + +#define __RTE_BIT_OVERLOAD_4R(fun, qualifier, ret_type, arg1_type, arg1_name, \ + arg2_type, arg2_name, arg3_type, arg3_name) \ + __RTE_BIT_OVERLOAD_SZ_4R(fun, qualifier, 32, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name, arg3_type, \ + arg3_name) \ + __RTE_BIT_OVERLOAD_SZ_4R(fun, qualifier, 64, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name, arg3_type, \ + arg3_name) + __RTE_BIT_OVERLOAD_2R(test, const, bool, unsigned int, nr) __RTE_BIT_OVERLOAD_2(set,, unsigned int, nr) __RTE_BIT_OVERLOAD_2(clear,, unsigned int, nr) __RTE_BIT_OVERLOAD_3(assign,, unsigned int, nr, bool, value) __RTE_BIT_OVERLOAD_2(flip,, unsigned int, nr) +__RTE_BIT_OVERLOAD_3R(atomic_test, const, bool, unsigned int, nr, + int, memory_order) +__RTE_BIT_OVERLOAD_3(atomic_set,, unsigned int, nr, int, memory_order) +__RTE_BIT_OVERLOAD_3(atomic_clear,, unsigned int, nr, int, memory_order) +__RTE_BIT_OVERLOAD_4(atomic_assign,, unsigned int, nr, bool, value, + int, memory_order) +__RTE_BIT_OVERLOAD_3(atomic_flip,, unsigned int, nr, int, memory_order) +__RTE_BIT_OVERLOAD_3R(atomic_test_and_set,, bool, unsigned int, nr, + int, memory_order) +__RTE_BIT_OVERLOAD_3R(atomic_test_and_clear,, bool, unsigned int, nr, + int, memory_order) +__RTE_BIT_OVERLOAD_4R(atomic_test_and_assign,, bool, unsigned int, nr, + bool, value, int, memory_order) + #endif #endif /* _RTE_BITOPS_H_ */ From patchwork Mon Aug 12 12:49:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 143074 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5F7BD457A1; Mon, 12 Aug 2024 14:59:44 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CB71B40698; Mon, 12 Aug 2024 14:59:14 +0200 (CEST) Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03on2084.outbound.protection.outlook.com [40.107.104.84]) by mails.dpdk.org (Postfix) with ESMTP id C214840695 for ; Mon, 12 Aug 2024 14:59:10 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=P1nRFG3Uf88sPcf/WkaNH84tf4pDw18m1UlCxQg7RrpP1Z2MLv5jbE3NV2LJyYCdOwdAuWXHjQiwHKaVWkknzzWpCXZWMZuRYXnuVl50G7kYNL/NFzhRccLZu92H/meve63V5ZBsSScQGNXg9qj1fAD7Rn/WXIy1s9ZnSnq1qGWg0HAy9+EH8HDAPDHt3tCQ+fJLJ3/W6rigkE9ZhpGFpFPBet6n/I2OfRTL1Ge0EKWCKHQQ36v6fcgxwSVXcz7DIE9N0jWnR0ASXRJtg8hKnHlwfG3EHs2au+RaGAatCHPvdqDctnxQpmEVfm3fRMV47svZAfhPBr0r7ANwCJKZmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=a5WMo7gTsm2Qr9ZTxjtFLLnl+2HwDjGCyFIPlASvPhI=; b=F2Lczouat2I08c9sbfL4Wom7Xe2O3M4islFyIm44vYDBOb58FqVPJjm2Ujh8dEtwvfdT8JJixrawpM9pded9lE54MrpgMWcY+x6UpEhvh3MG+M/xg5YA0KClTPlMuvYt6EZKKTkjwBpJrYqexXWC6O7jU3kSQ5ENE95o+gTlXcjX6AZgh7VxQS6AiTwoNFYN8rRbLaN/h1WkTzcB55+GT3QQ0uBvUErhFr9Sxp97lxVybcnjm629vnjvVpt0tZvTmFnNDJ788x9XpEyqtP8VuYcw7EazR2AAjzq2fpc82pfDK9+wC92V7pLLA9U1v+1ThXbe8Qks7Jh0eGOqsL47YA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=a5WMo7gTsm2Qr9ZTxjtFLLnl+2HwDjGCyFIPlASvPhI=; b=KiRtyBZZ/eNbea9zOAnrjAmQSiAk7M58E3EpGswLf3E2PJDEZzZB/FqPs9oZsmbhVzt4bja6EI8ysjc3h5LUf+Axx+1MVf283Na2ORmi1VpsiKEcHm1StTYFDlKgS+OKTI1Vm5GOpSsPFUAM7TR0rF2CrHgNFv22oLPp+XhxE4ep4DrwEHTMDBuyPnXm7ZrU20pnVz8SffdVsvqTMKzShasIVSc4zB/TnyZtDQLyJ/+/9EZVPjXosAekMGhxzSeJ2o9WNiIoFFkgV6Dumk2NYCYmobpQ9LsTpyAaLzf+KBQc3vzOUdTm3b5EGEB5RDNImloB2QArLpLqQ7Gy5ph1XA== Received: from AS4P189CA0015.EURP189.PROD.OUTLOOK.COM (2603:10a6:20b:5d7::19) by PA4PR07MB7293.eurprd07.prod.outlook.com (2603:10a6:102:f6::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.20; Mon, 12 Aug 2024 12:59:05 +0000 Received: from AM3PEPF0000A796.eurprd04.prod.outlook.com (2603:10a6:20b:5d7:cafe::6) by AS4P189CA0015.outlook.office365.com (2603:10a6:20b:5d7::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.20 via Frontend Transport; Mon, 12 Aug 2024 12:59:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by AM3PEPF0000A796.mail.protection.outlook.com (10.167.16.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.8 via Frontend Transport; Mon, 12 Aug 2024 12:59:05 +0000 Received: from seliicinfr00049.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.62) with Microsoft SMTP Server id 15.2.1544.11; Mon, 12 Aug 2024 14:59:03 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00049.seli.gic.ericsson.se (Postfix) with ESMTP id 26F9338007D; Mon, 12 Aug 2024 14:59:03 +0200 (CEST) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , Heng Wang , "Stephen Hemminger" , Tyler Retzlaff , =?utf-8?q?Morten_Br=C3=B8rup?= , Jack Bond-Preston , =?utf-8?q?Mattias_R?= =?utf-8?q?=C3=B6nnblom?= Subject: [PATCH v3 4/5] eal: add unit tests for atomic bit access functions Date: Mon, 12 Aug 2024 14:49:29 +0200 Message-ID: <20240812124930.604796-5-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240812124930.604796-1-mattias.ronnblom@ericsson.com> References: <20240809095829.589396-2-mattias.ronnblom@ericsson.com> <20240812124930.604796-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM3PEPF0000A796:EE_|PA4PR07MB7293:EE_ X-MS-Office365-Filtering-Correlation-Id: 60315ba9-8964-422b-134a-08dcbace8bd8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|36860700013|1800799024|376014|82310400026; X-Microsoft-Antispam-Message-Info: =?utf-8?q?kxnWA8IX+P2bniCeSgwJmSHotm+DtZA?= =?utf-8?q?syDE6nPSHi/LtYuTn1+BxN+spIal7R2T0opVDolyJKqa3e1WOFFBZlanJycwSHLT1?= =?utf-8?q?0Meq4VEvN/StqDGD9pfbI+Rylpel7bPeJSma+NRRivvg/wzPvK6g3zY3suu5a/tT2?= =?utf-8?q?JL3nOi5u95AxcO4m51V2Lz+2sCH+n5eCapo4oQmeCnr6Tg7aUml2uWkAIJJfaIn1M?= =?utf-8?q?dQ2antQExPDeEebibLUbiy1+M7x6h7rD9ZDR9G2v918duUWk8nJEnW06TLAVvZc5j?= =?utf-8?q?NqHZaNEUUrli66n6/6NDfA9YbyfOucCjHrTtNAfYCnqFUMUviaqLAGpX59AnKFHhS?= =?utf-8?q?JqTQytQTK5v1VYGcqk6RBIwvVrpUVLb7+yr5h6q5UkQMikCDC9vEvDxXc2qMOz9IG?= =?utf-8?q?qS9rsYBUed1BsnVSbSkwOuvT9EI0PbvkoF2UiziCHhBy2+IYlxlsIDMNWBhRvF8wI?= =?utf-8?q?/aaah8agzgoVCzjoWxmCO/+DafMb38beI5MOnO2m7MtbdFMSLIK+DKAyMKFhVOntl?= =?utf-8?q?yTu7m7HkbAFtkLgOEHsFsvttZQQOgl8SjwhsLWfw7BmCpf/0l3F5xXFCZ3VaaSpiq?= =?utf-8?q?wqUEyxpywIE/0qH5UOqwjVWcT8iAlO8k5OSjDiLXUpopYYOAkPT+/PsNjtbUh+Mri?= =?utf-8?q?7lhoO0ufUqa6FZhom/yGwVyTtOgjulIAtezCqMlzdP9MVIDRVkuFtY0xYLLaacrF0?= =?utf-8?q?vJAv24dR3n2P1RRup8AhmHD4foWo4qg1ODKKs4wbTvzFGFE/S0E5VKm2G38ZevAy2?= =?utf-8?q?VBMvAYI55pv77NbAm/zrNjX2mF2PNndmuqZ/KQZEbght4vwH86RA4aIjzqAuRxZEg?= =?utf-8?q?RoeD6Nj4gr5ecCB3QJCYw6MSMcOtzS5PZOnAviWq4ZhWLAur9uhqtC9HsLtLlPdY4?= =?utf-8?q?tM+yoyz9jsvzivXsiT4X2mX+AhNXO95rUFqvmx19GRaRN9r4wBa0AYYQjg6YxwzZe?= =?utf-8?q?5QDiMjuUvdhPgbt3TAYEVPZpuVhVgqkOSxVf5BxGTlIG6NcAYCJv1c0UIO/BiweKw?= =?utf-8?q?H2J+xwXaKE2I5lp7lEqUvs8+uUQXoX5uXLPUA93yL/4ugKHaygMcJxlkRuZx+LTqX?= =?utf-8?q?5+D/5LOja/oT1bw7uBjY5wxjDA6JqsrmT2UI+sIEswiFNLtH8HOSM576dh5vWveMR?= =?utf-8?q?b4dR6xfvxwYVapH2zGFAm+EMRYg5FT/0y8dILSBeD6rpOkszORlBnc0Qtk6N+gZbB?= =?utf-8?q?1LYyToAkx1KE+R3pZQ9mzGYmzOgFN2lplSaRwY7Awloc0FZ7ilqmM2FhQ3BoIHR3K?= =?utf-8?q?pUMCnHxoGs732kCNgLVqxDwAPkjmUWYEZHLCXCTq6p8d4rbhmgRQC3cHheg9IY+VI?= =?utf-8?q?yIcoHjUdlIJLEbmXvVzPXMR7xDu77KcJWg=3D=3D?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230040)(36860700013)(1800799024)(376014)(82310400026); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2024 12:59:05.1522 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 60315ba9-8964-422b-134a-08dcbace8bd8 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: AM3PEPF0000A796.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR07MB7293 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Extend bitops tests to cover the rte_bit_atomic_*() family of functions. Signed-off-by: Mattias Rönnblom Acked-by: Morten Brørup Acked-by: Tyler Retzlaff Acked-by: Jack Bond-Preston --- RFC v4: * Add atomicity test for atomic bit flip. RFC v3: * Rename variable 'main' to make ICC happy. --- app/test/test_bitops.c | 313 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 312 insertions(+), 1 deletion(-) diff --git a/app/test/test_bitops.c b/app/test/test_bitops.c index 322f58c066..b80216a0a1 100644 --- a/app/test/test_bitops.c +++ b/app/test/test_bitops.c @@ -3,10 +3,13 @@ * Copyright(c) 2024 Ericsson AB */ +#include #include -#include #include +#include +#include +#include #include #include "test.h" @@ -61,6 +64,304 @@ GEN_TEST_BIT_ACCESS(test_bit_access32, rte_bit_set, rte_bit_clear, GEN_TEST_BIT_ACCESS(test_bit_access64, rte_bit_set, rte_bit_clear, rte_bit_assign, rte_bit_flip, rte_bit_test, 64) +#define bit_atomic_set(addr, nr) \ + rte_bit_atomic_set(addr, nr, rte_memory_order_relaxed) + +#define bit_atomic_clear(addr, nr) \ + rte_bit_atomic_clear(addr, nr, rte_memory_order_relaxed) + +#define bit_atomic_assign(addr, nr, value) \ + rte_bit_atomic_assign(addr, nr, value, rte_memory_order_relaxed) + +#define bit_atomic_flip(addr, nr) \ + rte_bit_atomic_flip(addr, nr, rte_memory_order_relaxed) + +#define bit_atomic_test(addr, nr) \ + rte_bit_atomic_test(addr, nr, rte_memory_order_relaxed) + +GEN_TEST_BIT_ACCESS(test_bit_atomic_access32, bit_atomic_set, + bit_atomic_clear, bit_atomic_assign, + bit_atomic_flip, bit_atomic_test, 32) + +GEN_TEST_BIT_ACCESS(test_bit_atomic_access64, bit_atomic_set, + bit_atomic_clear, bit_atomic_assign, + bit_atomic_flip, bit_atomic_test, 64) + +#define PARALLEL_TEST_RUNTIME 0.25 + +#define GEN_TEST_BIT_PARALLEL_ASSIGN(size) \ + \ + struct parallel_access_lcore ## size \ + { \ + unsigned int bit; \ + uint ## size ##_t *word; \ + bool failed; \ + }; \ + \ + static int \ + run_parallel_assign ## size(void *arg) \ + { \ + struct parallel_access_lcore ## size *lcore = arg; \ + uint64_t deadline = rte_get_timer_cycles() + \ + PARALLEL_TEST_RUNTIME * rte_get_timer_hz(); \ + bool value = false; \ + \ + do { \ + bool new_value = rte_rand() & 1; \ + bool use_test_and_modify = rte_rand() & 1; \ + bool use_assign = rte_rand() & 1; \ + \ + if (rte_bit_atomic_test(lcore->word, lcore->bit, \ + rte_memory_order_relaxed) != value) { \ + lcore->failed = true; \ + break; \ + } \ + \ + if (use_test_and_modify) { \ + bool old_value; \ + if (use_assign) \ + old_value = rte_bit_atomic_test_and_assign( \ + lcore->word, lcore->bit, new_value, \ + rte_memory_order_relaxed); \ + else { \ + old_value = new_value ? \ + rte_bit_atomic_test_and_set( \ + lcore->word, lcore->bit, \ + rte_memory_order_relaxed) : \ + rte_bit_atomic_test_and_clear( \ + lcore->word, lcore->bit, \ + rte_memory_order_relaxed); \ + } \ + if (old_value != value) { \ + lcore->failed = true; \ + break; \ + } \ + } else { \ + if (use_assign) \ + rte_bit_atomic_assign(lcore->word, lcore->bit, \ + new_value, \ + rte_memory_order_relaxed); \ + else { \ + if (new_value) \ + rte_bit_atomic_set( \ + lcore->word, lcore->bit, \ + rte_memory_order_relaxed); \ + else \ + rte_bit_atomic_clear( \ + lcore->word, lcore->bit, \ + rte_memory_order_relaxed); \ + } \ + } \ + \ + value = new_value; \ + } while (rte_get_timer_cycles() < deadline); \ + \ + return 0; \ + } \ + \ + static int \ + test_bit_atomic_parallel_assign ## size(void) \ + { \ + unsigned int worker_lcore_id; \ + uint ## size ## _t word = 0; \ + struct parallel_access_lcore ## size lmain = { \ + .word = &word \ + }; \ + struct parallel_access_lcore ## size lworker = { \ + .word = &word \ + }; \ + \ + if (rte_lcore_count() < 2) { \ + printf("Need multiple cores to run parallel test.\n"); \ + return TEST_SKIPPED; \ + } \ + \ + worker_lcore_id = rte_get_next_lcore(-1, 1, 0); \ + \ + lmain.bit = rte_rand_max(size); \ + do { \ + lworker.bit = rte_rand_max(size); \ + } while (lworker.bit == lmain.bit); \ + \ + int rc = rte_eal_remote_launch(run_parallel_assign ## size, \ + &lworker, worker_lcore_id); \ + TEST_ASSERT(rc == 0, "Worker thread launch failed"); \ + \ + run_parallel_assign ## size(&lmain); \ + \ + rte_eal_mp_wait_lcore(); \ + \ + TEST_ASSERT(!lmain.failed, "Main lcore atomic access failed"); \ + TEST_ASSERT(!lworker.failed, "Worker lcore atomic access " \ + "failed"); \ + \ + return TEST_SUCCESS; \ + } + +GEN_TEST_BIT_PARALLEL_ASSIGN(32) +GEN_TEST_BIT_PARALLEL_ASSIGN(64) + +#define GEN_TEST_BIT_PARALLEL_TEST_AND_MODIFY(size) \ + \ + struct parallel_test_and_set_lcore ## size \ + { \ + uint ## size ##_t *word; \ + unsigned int bit; \ + uint64_t flips; \ + }; \ + \ + static int \ + run_parallel_test_and_modify ## size(void *arg) \ + { \ + struct parallel_test_and_set_lcore ## size *lcore = arg; \ + uint64_t deadline = rte_get_timer_cycles() + \ + PARALLEL_TEST_RUNTIME * rte_get_timer_hz(); \ + do { \ + bool old_value; \ + bool new_value = rte_rand() & 1; \ + bool use_assign = rte_rand() & 1; \ + \ + if (use_assign) \ + old_value = rte_bit_atomic_test_and_assign( \ + lcore->word, lcore->bit, new_value, \ + rte_memory_order_relaxed); \ + else \ + old_value = new_value ? \ + rte_bit_atomic_test_and_set( \ + lcore->word, lcore->bit, \ + rte_memory_order_relaxed) : \ + rte_bit_atomic_test_and_clear( \ + lcore->word, lcore->bit, \ + rte_memory_order_relaxed); \ + if (old_value != new_value) \ + lcore->flips++; \ + } while (rte_get_timer_cycles() < deadline); \ + \ + return 0; \ + } \ + \ + static int \ + test_bit_atomic_parallel_test_and_modify ## size(void) \ + { \ + unsigned int worker_lcore_id; \ + uint ## size ## _t word = 0; \ + unsigned int bit = rte_rand_max(size); \ + struct parallel_test_and_set_lcore ## size lmain = { \ + .word = &word, \ + .bit = bit \ + }; \ + struct parallel_test_and_set_lcore ## size lworker = { \ + .word = &word, \ + .bit = bit \ + }; \ + \ + if (rte_lcore_count() < 2) { \ + printf("Need multiple cores to run parallel test.\n"); \ + return TEST_SKIPPED; \ + } \ + \ + worker_lcore_id = rte_get_next_lcore(-1, 1, 0); \ + \ + int rc = rte_eal_remote_launch(run_parallel_test_and_modify ## size, \ + &lworker, worker_lcore_id); \ + TEST_ASSERT(rc == 0, "Worker thread launch failed"); \ + \ + run_parallel_test_and_modify ## size(&lmain); \ + \ + rte_eal_mp_wait_lcore(); \ + \ + uint64_t total_flips = lmain.flips + lworker.flips; \ + bool expected_value = total_flips % 2; \ + \ + TEST_ASSERT(expected_value == rte_bit_test(&word, bit), \ + "After %"PRId64" flips, the bit value " \ + "should be %d", total_flips, expected_value); \ + \ + uint64_t expected_word = 0; \ + rte_bit_assign(&expected_word, bit, expected_value); \ + \ + TEST_ASSERT(expected_word == word, "Untouched bits have " \ + "changed value"); \ + \ + return TEST_SUCCESS; \ + } + +GEN_TEST_BIT_PARALLEL_TEST_AND_MODIFY(32) +GEN_TEST_BIT_PARALLEL_TEST_AND_MODIFY(64) + +#define GEN_TEST_BIT_PARALLEL_FLIP(size) \ + \ + struct parallel_flip_lcore ## size \ + { \ + uint ## size ##_t *word; \ + unsigned int bit; \ + uint64_t flips; \ + }; \ + \ + static int \ + run_parallel_flip ## size(void *arg) \ + { \ + struct parallel_flip_lcore ## size *lcore = arg; \ + uint64_t deadline = rte_get_timer_cycles() + \ + PARALLEL_TEST_RUNTIME * rte_get_timer_hz(); \ + do { \ + rte_bit_atomic_flip(lcore->word, lcore->bit, \ + rte_memory_order_relaxed); \ + lcore->flips++; \ + } while (rte_get_timer_cycles() < deadline); \ + \ + return 0; \ + } \ + \ + static int \ + test_bit_atomic_parallel_flip ## size(void) \ + { \ + unsigned int worker_lcore_id; \ + uint ## size ## _t word = 0; \ + unsigned int bit = rte_rand_max(size); \ + struct parallel_flip_lcore ## size lmain = { \ + .word = &word, \ + .bit = bit \ + }; \ + struct parallel_flip_lcore ## size lworker = { \ + .word = &word, \ + .bit = bit \ + }; \ + \ + if (rte_lcore_count() < 2) { \ + printf("Need multiple cores to run parallel test.\n"); \ + return TEST_SKIPPED; \ + } \ + \ + worker_lcore_id = rte_get_next_lcore(-1, 1, 0); \ + \ + int rc = rte_eal_remote_launch(run_parallel_flip ## size, \ + &lworker, worker_lcore_id); \ + TEST_ASSERT(rc == 0, "Worker thread launch failed"); \ + \ + run_parallel_flip ## size(&lmain); \ + \ + rte_eal_mp_wait_lcore(); \ + \ + uint64_t total_flips = lmain.flips + lworker.flips; \ + bool expected_value = total_flips % 2; \ + \ + TEST_ASSERT(expected_value == rte_bit_test(&word, bit), \ + "After %"PRId64" flips, the bit value " \ + "should be %d", total_flips, expected_value); \ + \ + uint64_t expected_word = 0; \ + rte_bit_assign(&expected_word, bit, expected_value); \ + \ + TEST_ASSERT(expected_word == word, "Untouched bits have " \ + "changed value"); \ + \ + return TEST_SUCCESS; \ + } + +GEN_TEST_BIT_PARALLEL_FLIP(32) +GEN_TEST_BIT_PARALLEL_FLIP(64) + static uint32_t val32; static uint64_t val64; @@ -177,6 +478,16 @@ static struct unit_test_suite test_suite = { .unit_test_cases = { TEST_CASE(test_bit_access32), TEST_CASE(test_bit_access64), + TEST_CASE(test_bit_access32), + TEST_CASE(test_bit_access64), + TEST_CASE(test_bit_atomic_access32), + TEST_CASE(test_bit_atomic_access64), + TEST_CASE(test_bit_atomic_parallel_assign32), + TEST_CASE(test_bit_atomic_parallel_assign64), + TEST_CASE(test_bit_atomic_parallel_test_and_modify32), + TEST_CASE(test_bit_atomic_parallel_test_and_modify64), + TEST_CASE(test_bit_atomic_parallel_flip32), + TEST_CASE(test_bit_atomic_parallel_flip64), TEST_CASE(test_bit_relaxed_set), TEST_CASE(test_bit_relaxed_clear), TEST_CASE(test_bit_relaxed_test_set_clear), From patchwork Mon Aug 12 12:49:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 143072 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CE2A6457A1; Mon, 12 Aug 2024 14:59:26 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DCFD240664; Mon, 12 Aug 2024 14:59:11 +0200 (CEST) Received: from EUR03-VI1-obe.outbound.protection.outlook.com (mail-vi1eur03on2054.outbound.protection.outlook.com [40.107.103.54]) by mails.dpdk.org (Postfix) with ESMTP id 3DDB14029C for ; Mon, 12 Aug 2024 14:59:08 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ca9TIM0LY+lbIv0WLOOBdh76T1CXReuNupq54bNDbGIK6+noze7l1vhvzFb+bRhN8iGZbiAbCbTNgl6Zw2Bin8ljbdVzPcBTlklWwJKshuTPTAAzlO5giOZg3LyRVZ0TM8GcJr/5wO0uBZMXdYUbRicr4wiN7I4ipYDwP/KnAoIVh1fx0cngcWYnAzyf/I1mHwbkuE5hzuZq4j/AnH9OBudafsOZ9CXfniXpQkzTCGZw92S9GUYpvOojLEXE256LJ8k8P9yVS/XOHudAZhcOBNUaDInkBKnUhhQCf+PuNUMhDk48sLjJsdqTLJhOBS5f/wuvjrsfSjFrucg2JxwBqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4gu713Q5k/Sa8NLSQz3/cXIfhY4v+9X1yoQVKEQwYuU=; b=mWmHcbOSgiEXi1NldDpZUZu6DGytW4mGJcRMoKlNFzDZbMv+2njB3xlxJDqsxyF+o+enuI8gqHSoJ8h5pfbEIw2T9JfpsFX3OiKsV17S/kGolva3kDXSPsQBg9rrOVN7Wzj8LPJI0h3+3xX9+IQ0bvdXXD5ksjChr0KGl1RkDfHiaGHGfl8PvdYj5b4W4sLdiDFCj59sp4lQCo/YtlfKebczhUPs9CcDe1q3dnuENGH06Msu8yTaV/jwYt8zwiXQVYm5ddAP2Bd4bTntGJffHYKeVASemigpgHxEUlhDIkv3CJpBRmxh4MaRHKXg7dsCvzV6/7krPGPqHFaWw6Z4aA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4gu713Q5k/Sa8NLSQz3/cXIfhY4v+9X1yoQVKEQwYuU=; b=Fr59BewhDWzFYuv/f70f0U9xTzuthwhxYyH/GdksILoY+wBuIWzx/RFVdJYNBVDTqJGF0Is0vYmBdwH1USp8tADvaLPmklv3B2kWvwlTRptRZjIuDReZ6KNEihu35uX7Y1vQyZeJv6KZddufKyK/h9XUwAVNm+5JfDcQPeRG6q4VxiMjMp7Xax5dUl7zcV9cFPxLyWvE2MbGtdQEz4nZ1su2dOnFHTq4tSrOE4WlyHbknNWRDDtM7cfeQ2n/5+62Tbth/mLrTqsw6q+p8xAcqK5nJUm0ceNUzW2cd3+v2apJiFX7pDte2IAua8KvvzNAOEchfpGMx6gpd171O+9kaA== Received: from AM0PR06CA0112.eurprd06.prod.outlook.com (2603:10a6:208:ab::17) by PA4PR07MB7680.eurprd07.prod.outlook.com (2603:10a6:102:c6::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.20; Mon, 12 Aug 2024 12:59:04 +0000 Received: from AM3PEPF00009BA1.eurprd04.prod.outlook.com (2603:10a6:208:ab:cafe::27) by AM0PR06CA0112.outlook.office365.com (2603:10a6:208:ab::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.20 via Frontend Transport; Mon, 12 Aug 2024 12:59:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by AM3PEPF00009BA1.mail.protection.outlook.com (10.167.16.26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.8 via Frontend Transport; Mon, 12 Aug 2024 12:59:03 +0000 Received: from seliicinfr00049.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.61) with Microsoft SMTP Server id 15.2.1544.11; Mon, 12 Aug 2024 14:59:03 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00049.seli.gic.ericsson.se (Postfix) with ESMTP id 3B3E0380070; Mon, 12 Aug 2024 14:59:03 +0200 (CEST) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , Heng Wang , "Stephen Hemminger" , Tyler Retzlaff , =?utf-8?q?Morten_Br=C3=B8rup?= , Jack Bond-Preston , =?utf-8?q?Mattias_R?= =?utf-8?q?=C3=B6nnblom?= Subject: [PATCH v3 5/5] eal: extend bitops to handle volatile pointers Date: Mon, 12 Aug 2024 14:49:30 +0200 Message-ID: <20240812124930.604796-6-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240812124930.604796-1-mattias.ronnblom@ericsson.com> References: <20240809095829.589396-2-mattias.ronnblom@ericsson.com> <20240812124930.604796-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM3PEPF00009BA1:EE_|PA4PR07MB7680:EE_ X-MS-Office365-Filtering-Correlation-Id: 02dbb63d-3bd7-4d14-3389-08dcbace8b26 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|36860700013|82310400026|1800799024|376014; X-Microsoft-Antispam-Message-Info: =?utf-8?q?VcSP7oR68JxQlFLaQ53+Y45tKHtObpH?= =?utf-8?q?YwQOwaPGCWbS7/2xVxn8Lz9k/NpCu3QGCPcL2obMmpZIfIJNTAb/lVh1rP8IBU5Fm?= =?utf-8?q?ZCTfQl+FSmli0qgg3Bw1ej8oYPgrgg+dxcXdEBsuU333SL28Q+IgtjMt/OjIfXgCU?= =?utf-8?q?zF6RzxBPPF6B1Mww9qs6vF+N88CDVeJ1Ak5YCusTKEhdKeFJZ8EF00S2NNQWjYnl4?= =?utf-8?q?+BIWuot0nKn929jPaAQcD1131ywKs0lOTFltEgAIE3EulUQvReLqVc6XdzZPyUpxr?= =?utf-8?q?2tyRrVyVN3kPFRhp4sqc9eMZ9us5GWO3Fc9eJzBA+HSg/zXW6+JasB/T0gkKARlu/?= =?utf-8?q?PuUhONqzIPsjsECtfA8AhxTGuKCiO53XQwT58P0VF6CiAGHta2tr2+6bbF1uz5C+B?= =?utf-8?q?og2jyZuWJ2qZZi9gBdoHfGZBB44Szpmsp0/wu7VSuu0dHFeDolxpS8Zunm+oPYFWk?= =?utf-8?q?fj1V5rTo/7gSbglkpuYzoLtQ70hw1kB34YyfA1FToTdf6ZAGWt+mtS7yppaDRd88J?= =?utf-8?q?RdJWbR62TCrbCoYp+KkOwW8xBx8JGlBTyUe+2RxUDA4dz5yoP9mQfFNEky2nLBr2t?= =?utf-8?q?5PLTUhGuHfIIOUfqPOgU0SZb351tWXcOW0Co0eJwdz4m2OLLLUziAqDcKYaprnSFY?= =?utf-8?q?xm23h1EkfpCXpQ51BjJsyrXgrv9unGS//ysDpAc/0dgn9g/KBdTH+7RIIfRDOneNT?= =?utf-8?q?FvphfVLjYrGoVZ0x/FU6x9occgScwmVRw0IWN9PA0H1Guws/jg14ic2lgxtoiSrby?= =?utf-8?q?vUMYle4gZFnWMmqMBg+/ejq3JHEv+/w8tWm9JzPbaejKKxIjUsPxd4eYg6gV4mkJ3?= =?utf-8?q?UJW4Q5aS2oQt3J98NAV7V7bvuZlUm6IPLFaYPOtfEQRJuyAkREKw+UtgJDhsczto7?= =?utf-8?q?OgPdKuj8bFhEFi8ASPcJtILaLX2RFI4mOVglBgYhLZKSKFQbX4U2vkjLo4qEJ+AGm?= =?utf-8?q?kxs8zgW+XPvANPXyf5rh/Kphu7D3ObFJavMcPBDPwdTyCJq4v2CA6jygYKIOY1TXT?= =?utf-8?q?R2KgwRx8KLMPMGS41eOHPLALD2r2S/ErQgEOeN7THZWJTJyYUMclxeOtnsdCc1uSn?= =?utf-8?q?eXxgkMl15zuxZu7nJhUWT6mdTZrYaqSY/dxKOV7rimTBNrN38gGsihZ8SuOR3Tcmo?= =?utf-8?q?fevYese4LkIjofnkKkw4a9zoW96B/bxGuPJRHhPD8zhLGUy4C37Ycm1zLug1lDb/o?= =?utf-8?q?1QjYuIpEe5cwbDTp6wgCuissWgj/kyRomNJiqDl4DD8n+wRAa4/PuB+kzfhMTwwib?= =?utf-8?q?1NiZ6uuLpKa7wTAJWsdxoLFsYBxMJuc/O3Wa/WSFP7eGD4OXDhbvfaoIi7OrhFDX9?= =?utf-8?q?CJspleJdruuKQ+kG1p4eGg6Baj8y0b7Crw=3D=3D?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230040)(36860700013)(82310400026)(1800799024)(376014); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2024 12:59:03.9864 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 02dbb63d-3bd7-4d14-3389-08dcbace8b26 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: AM3PEPF00009BA1.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR07MB7680 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Have rte_bit_[test|set|clear|assign|flip]() and rte_bit_atomic_*() handle volatile-marked pointers. Signed-off-by: Mattias Rönnblom Acked-by: Morten Brørup Acked-by: Jack Bond-Preston --- PATCH v3: * Updated to reflect removed 'fun' parameter in __RTE_GEN_BIT_*() (Jack Bond-Preston). PATCH v2: * Actually run the test_bit_atomic_v_access*() test functions. --- app/test/test_bitops.c | 32 +++- lib/eal/include/rte_bitops.h | 301 +++++++++++++++++++++++------------ 2 files changed, 222 insertions(+), 111 deletions(-) diff --git a/app/test/test_bitops.c b/app/test/test_bitops.c index b80216a0a1..10e87f6776 100644 --- a/app/test/test_bitops.c +++ b/app/test/test_bitops.c @@ -14,13 +14,13 @@ #include "test.h" #define GEN_TEST_BIT_ACCESS(test_name, set_fun, clear_fun, assign_fun, \ - flip_fun, test_fun, size) \ + flip_fun, test_fun, size, mod) \ static int \ test_name(void) \ { \ uint ## size ## _t reference = (uint ## size ## _t)rte_rand(); \ unsigned int bit_nr; \ - uint ## size ## _t word = (uint ## size ## _t)rte_rand(); \ + mod uint ## size ## _t word = (uint ## size ## _t)rte_rand(); \ \ for (bit_nr = 0; bit_nr < size; bit_nr++) { \ bool reference_bit = (reference >> bit_nr) & 1; \ @@ -41,7 +41,7 @@ "Bit %d had unflipped value", bit_nr); \ flip_fun(&word, bit_nr); \ \ - const uint ## size ## _t *const_ptr = &word; \ + const mod uint ## size ## _t *const_ptr = &word; \ TEST_ASSERT(test_fun(const_ptr, bit_nr) == \ reference_bit, \ "Bit %d had unexpected value", bit_nr); \ @@ -59,10 +59,16 @@ } GEN_TEST_BIT_ACCESS(test_bit_access32, rte_bit_set, rte_bit_clear, - rte_bit_assign, rte_bit_flip, rte_bit_test, 32) + rte_bit_assign, rte_bit_flip, rte_bit_test, 32,) GEN_TEST_BIT_ACCESS(test_bit_access64, rte_bit_set, rte_bit_clear, - rte_bit_assign, rte_bit_flip, rte_bit_test, 64) + rte_bit_assign, rte_bit_flip, rte_bit_test, 64,) + +GEN_TEST_BIT_ACCESS(test_bit_v_access32, rte_bit_set, rte_bit_clear, + rte_bit_assign, rte_bit_flip, rte_bit_test, 32, volatile) + +GEN_TEST_BIT_ACCESS(test_bit_v_access64, rte_bit_set, rte_bit_clear, + rte_bit_assign, rte_bit_flip, rte_bit_test, 64, volatile) #define bit_atomic_set(addr, nr) \ rte_bit_atomic_set(addr, nr, rte_memory_order_relaxed) @@ -81,11 +87,19 @@ GEN_TEST_BIT_ACCESS(test_bit_access64, rte_bit_set, rte_bit_clear, GEN_TEST_BIT_ACCESS(test_bit_atomic_access32, bit_atomic_set, bit_atomic_clear, bit_atomic_assign, - bit_atomic_flip, bit_atomic_test, 32) + bit_atomic_flip, bit_atomic_test, 32,) GEN_TEST_BIT_ACCESS(test_bit_atomic_access64, bit_atomic_set, bit_atomic_clear, bit_atomic_assign, - bit_atomic_flip, bit_atomic_test, 64) + bit_atomic_flip, bit_atomic_test, 64,) + +GEN_TEST_BIT_ACCESS(test_bit_atomic_v_access32, bit_atomic_set, + bit_atomic_clear, bit_atomic_assign, + bit_atomic_flip, bit_atomic_test, 32, volatile) + +GEN_TEST_BIT_ACCESS(test_bit_atomic_v_access64, bit_atomic_set, + bit_atomic_clear, bit_atomic_assign, + bit_atomic_flip, bit_atomic_test, 64, volatile) #define PARALLEL_TEST_RUNTIME 0.25 @@ -480,8 +494,12 @@ static struct unit_test_suite test_suite = { TEST_CASE(test_bit_access64), TEST_CASE(test_bit_access32), TEST_CASE(test_bit_access64), + TEST_CASE(test_bit_v_access32), + TEST_CASE(test_bit_v_access64), TEST_CASE(test_bit_atomic_access32), TEST_CASE(test_bit_atomic_access64), + TEST_CASE(test_bit_atomic_v_access32), + TEST_CASE(test_bit_atomic_v_access64), TEST_CASE(test_bit_atomic_parallel_assign32), TEST_CASE(test_bit_atomic_parallel_assign64), TEST_CASE(test_bit_atomic_parallel_test_and_modify32), diff --git a/lib/eal/include/rte_bitops.h b/lib/eal/include/rte_bitops.h index 3ad6795fd1..d7a07c4099 100644 --- a/lib/eal/include/rte_bitops.h +++ b/lib/eal/include/rte_bitops.h @@ -127,12 +127,16 @@ extern "C" { * @param nr * The index of the bit. */ -#define rte_bit_test(addr, nr) \ - _Generic((addr), \ - uint32_t *: __rte_bit_test32, \ - const uint32_t *: __rte_bit_test32, \ - uint64_t *: __rte_bit_test64, \ - const uint64_t *: __rte_bit_test64)(addr, nr) +#define rte_bit_test(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_test32, \ + const uint32_t *: __rte_bit_test32, \ + volatile uint32_t *: __rte_bit_v_test32, \ + const volatile uint32_t *: __rte_bit_v_test32, \ + uint64_t *: __rte_bit_test64, \ + const uint64_t *: __rte_bit_test64, \ + volatile uint64_t *: __rte_bit_v_test64, \ + const volatile uint64_t *: __rte_bit_v_test64)(addr, nr) /** * @warning @@ -152,10 +156,12 @@ extern "C" { * @param nr * The index of the bit. */ -#define rte_bit_set(addr, nr) \ - _Generic((addr), \ - uint32_t *: __rte_bit_set32, \ - uint64_t *: __rte_bit_set64)(addr, nr) +#define rte_bit_set(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_set32, \ + volatile uint32_t *: __rte_bit_v_set32, \ + uint64_t *: __rte_bit_set64, \ + volatile uint64_t *: __rte_bit_v_set64)(addr, nr) /** * @warning @@ -175,10 +181,12 @@ extern "C" { * @param nr * The index of the bit. */ -#define rte_bit_clear(addr, nr) \ - _Generic((addr), \ - uint32_t *: __rte_bit_clear32, \ - uint64_t *: __rte_bit_clear64)(addr, nr) +#define rte_bit_clear(addr, nr) \ + _Generic((addr), \ + uint32_t *: __rte_bit_clear32, \ + volatile uint32_t *: __rte_bit_v_clear32, \ + uint64_t *: __rte_bit_clear64, \ + volatile uint64_t *: __rte_bit_v_clear64)(addr, nr) /** * @warning @@ -202,7 +210,9 @@ extern "C" { #define rte_bit_assign(addr, nr, value) \ _Generic((addr), \ uint32_t *: __rte_bit_assign32, \ - uint64_t *: __rte_bit_assign64)(addr, nr, value) + volatile uint32_t *: __rte_bit_v_assign32, \ + uint64_t *: __rte_bit_assign64, \ + volatile uint64_t *: __rte_bit_v_assign64)(addr, nr, value) /** * @warning @@ -225,7 +235,9 @@ extern "C" { #define rte_bit_flip(addr, nr) \ _Generic((addr), \ uint32_t *: __rte_bit_flip32, \ - uint64_t *: __rte_bit_flip64)(addr, nr) + volatile uint32_t *: __rte_bit_v_flip32, \ + uint64_t *: __rte_bit_flip64, \ + volatile uint64_t *: __rte_bit_v_flip64)(addr, nr) /** * @warning @@ -250,9 +262,13 @@ extern "C" { _Generic((addr), \ uint32_t *: __rte_bit_atomic_test32, \ const uint32_t *: __rte_bit_atomic_test32, \ + volatile uint32_t *: __rte_bit_atomic_v_test32, \ + const volatile uint32_t *: __rte_bit_atomic_v_test32, \ uint64_t *: __rte_bit_atomic_test64, \ - const uint64_t *: __rte_bit_atomic_test64)(addr, nr, \ - memory_order) + const uint64_t *: __rte_bit_atomic_test64, \ + volatile uint64_t *: __rte_bit_atomic_v_test64, \ + const volatile uint64_t *: __rte_bit_atomic_v_test64) \ + (addr, nr, memory_order) /** * @warning @@ -274,7 +290,10 @@ extern "C" { #define rte_bit_atomic_set(addr, nr, memory_order) \ _Generic((addr), \ uint32_t *: __rte_bit_atomic_set32, \ - uint64_t *: __rte_bit_atomic_set64)(addr, nr, memory_order) + volatile uint32_t *: __rte_bit_atomic_v_set32, \ + uint64_t *: __rte_bit_atomic_set64, \ + volatile uint64_t *: __rte_bit_atomic_v_set64)(addr, nr, \ + memory_order) /** * @warning @@ -296,7 +315,10 @@ extern "C" { #define rte_bit_atomic_clear(addr, nr, memory_order) \ _Generic((addr), \ uint32_t *: __rte_bit_atomic_clear32, \ - uint64_t *: __rte_bit_atomic_clear64)(addr, nr, memory_order) + volatile uint32_t *: __rte_bit_atomic_v_clear32, \ + uint64_t *: __rte_bit_atomic_clear64, \ + volatile uint64_t *: __rte_bit_atomic_v_clear64)(addr, nr, \ + memory_order) /** * @warning @@ -320,8 +342,11 @@ extern "C" { #define rte_bit_atomic_assign(addr, nr, value, memory_order) \ _Generic((addr), \ uint32_t *: __rte_bit_atomic_assign32, \ - uint64_t *: __rte_bit_atomic_assign64)(addr, nr, value, \ - memory_order) + volatile uint32_t *: __rte_bit_atomic_v_assign32, \ + uint64_t *: __rte_bit_atomic_assign64, \ + volatile uint64_t *: __rte_bit_atomic_v_assign64)(addr, nr, \ + value, \ + memory_order) /** * @warning @@ -344,7 +369,10 @@ extern "C" { #define rte_bit_atomic_flip(addr, nr, memory_order) \ _Generic((addr), \ uint32_t *: __rte_bit_atomic_flip32, \ - uint64_t *: __rte_bit_atomic_flip64)(addr, nr, memory_order) + volatile uint32_t *: __rte_bit_atomic_v_flip32, \ + uint64_t *: __rte_bit_atomic_flip64, \ + volatile uint64_t *: __rte_bit_atomic_v_flip64)(addr, nr, \ + memory_order) /** * @warning @@ -368,8 +396,10 @@ extern "C" { #define rte_bit_atomic_test_and_set(addr, nr, memory_order) \ _Generic((addr), \ uint32_t *: __rte_bit_atomic_test_and_set32, \ - uint64_t *: __rte_bit_atomic_test_and_set64)(addr, nr, \ - memory_order) + volatile uint32_t *: __rte_bit_atomic_v_test_and_set32, \ + uint64_t *: __rte_bit_atomic_test_and_set64, \ + volatile uint64_t *: __rte_bit_atomic_v_test_and_set64) \ + (addr, nr, memory_order) /** * @warning @@ -393,8 +423,10 @@ extern "C" { #define rte_bit_atomic_test_and_clear(addr, nr, memory_order) \ _Generic((addr), \ uint32_t *: __rte_bit_atomic_test_and_clear32, \ - uint64_t *: __rte_bit_atomic_test_and_clear64)(addr, nr, \ - memory_order) + volatile uint32_t *: __rte_bit_atomic_v_test_and_clear32, \ + uint64_t *: __rte_bit_atomic_test_and_clear64, \ + volatile uint64_t *: __rte_bit_atomic_v_test_and_clear64) \ + (addr, nr, memory_order) /** * @warning @@ -421,9 +453,10 @@ extern "C" { #define rte_bit_atomic_test_and_assign(addr, nr, value, memory_order) \ _Generic((addr), \ uint32_t *: __rte_bit_atomic_test_and_assign32, \ - uint64_t *: __rte_bit_atomic_test_and_assign64)(addr, nr, \ - value, \ - memory_order) + volatile uint32_t *: __rte_bit_atomic_v_test_and_assign32, \ + uint64_t *: __rte_bit_atomic_test_and_assign64, \ + volatile uint64_t *: __rte_bit_atomic_v_test_and_assign64) \ + (addr, nr, value, memory_order) #define __RTE_GEN_BIT_TEST(variant, qualifier, size) \ __rte_experimental \ @@ -493,7 +526,8 @@ extern "C" { __RTE_GEN_BIT_FLIP(v, qualifier, size) #define __RTE_GEN_BIT_OPS_SIZE(size) \ - __RTE_GEN_BIT_OPS(,, size) + __RTE_GEN_BIT_OPS(,, size) \ + __RTE_GEN_BIT_OPS(v_, volatile, size) __RTE_GEN_BIT_OPS_SIZE(32) __RTE_GEN_BIT_OPS_SIZE(64) @@ -633,7 +667,8 @@ __RTE_GEN_BIT_OPS_SIZE(64) __RTE_GEN_BIT_ATOMIC_FLIP(variant, qualifier, size) #define __RTE_GEN_BIT_ATOMIC_OPS_SIZE(size) \ - __RTE_GEN_BIT_ATOMIC_OPS(,, size) + __RTE_GEN_BIT_ATOMIC_OPS(,, size) \ + __RTE_GEN_BIT_ATOMIC_OPS(v_, volatile, size) __RTE_GEN_BIT_ATOMIC_OPS_SIZE(32) __RTE_GEN_BIT_ATOMIC_OPS_SIZE(64) @@ -1342,120 +1377,178 @@ rte_log2_u64(uint64_t v) #undef rte_bit_atomic_test_and_clear #undef rte_bit_atomic_test_and_assign -#define __RTE_BIT_OVERLOAD_SZ_2(fun, qualifier, size, arg1_type, arg1_name) \ +#define __RTE_BIT_OVERLOAD_V_2(family, v, fun, c, size, arg1_type, arg1_name) \ static inline void \ - rte_bit_ ## fun(qualifier uint ## size ## _t *addr, \ - arg1_type arg1_name) \ + rte_bit_ ## family ## fun(c uint ## size ## _t *addr, \ + arg1_type arg1_name) \ { \ - __rte_bit_ ## fun ## size(addr, arg1_name); \ + __rte_bit_ ## family ## v ## fun ## size(addr, arg1_name); \ } -#define __RTE_BIT_OVERLOAD_2(fun, qualifier, arg1_type, arg1_name) \ - __RTE_BIT_OVERLOAD_SZ_2(fun, qualifier, 32, arg1_type, arg1_name) \ - __RTE_BIT_OVERLOAD_SZ_2(fun, qualifier, 64, arg1_type, arg1_name) +#define __RTE_BIT_OVERLOAD_SZ_2(family, fun, c, size, arg1_type, arg1_name) \ + __RTE_BIT_OVERLOAD_V_2(family,, fun, c, size, arg1_type, \ + arg1_name) \ + __RTE_BIT_OVERLOAD_V_2(family, v_, fun, c volatile, size, \ + arg1_type, arg1_name) -#define __RTE_BIT_OVERLOAD_SZ_2R(fun, qualifier, size, ret_type, arg1_type, \ - arg1_name) \ +#define __RTE_BIT_OVERLOAD_2(family, fun, c, arg1_type, arg1_name) \ + __RTE_BIT_OVERLOAD_SZ_2(family, fun, c, 32, arg1_type, arg1_name) \ + __RTE_BIT_OVERLOAD_SZ_2(family, fun, c, 64, arg1_type, arg1_name) + +#define __RTE_BIT_OVERLOAD_V_2R(family, v, fun, c, size, ret_type, arg1_type, \ + arg1_name) \ static inline ret_type \ - rte_bit_ ## fun(qualifier uint ## size ## _t *addr, \ + rte_bit_ ## family ## fun(c uint ## size ## _t *addr, \ arg1_type arg1_name) \ { \ - return __rte_bit_ ## fun ## size(addr, arg1_name); \ + return __rte_bit_ ## family ## v ## fun ## size(addr, \ + arg1_name); \ } -#define __RTE_BIT_OVERLOAD_2R(fun, qualifier, ret_type, arg1_type, arg1_name) \ - __RTE_BIT_OVERLOAD_SZ_2R(fun, qualifier, 32, ret_type, arg1_type, \ +#define __RTE_BIT_OVERLOAD_SZ_2R(family, fun, c, size, ret_type, arg1_type, \ + arg1_name) \ + __RTE_BIT_OVERLOAD_V_2R(family,, fun, c, size, ret_type, arg1_type, \ + arg1_name) \ + __RTE_BIT_OVERLOAD_V_2R(family, v_, fun, c volatile, \ + size, ret_type, arg1_type, arg1_name) + +#define __RTE_BIT_OVERLOAD_2R(family, fun, c, ret_type, arg1_type, arg1_name) \ + __RTE_BIT_OVERLOAD_SZ_2R(family, fun, c, 32, ret_type, arg1_type, \ arg1_name) \ - __RTE_BIT_OVERLOAD_SZ_2R(fun, qualifier, 64, ret_type, arg1_type, \ + __RTE_BIT_OVERLOAD_SZ_2R(family, fun, c, 64, ret_type, arg1_type, \ arg1_name) -#define __RTE_BIT_OVERLOAD_SZ_3(fun, qualifier, size, arg1_type, arg1_name, \ - arg2_type, arg2_name) \ +#define __RTE_BIT_OVERLOAD_V_3(family, v, fun, c, size, arg1_type, arg1_name, \ + arg2_type, arg2_name) \ static inline void \ - rte_bit_ ## fun(uint ## size ## _t *addr, arg1_type arg1_name, \ - arg2_type arg2_name) \ + rte_bit_ ## family ## fun(c uint ## size ## _t *addr, \ + arg1_type arg1_name, arg2_type arg2_name) \ { \ - __rte_bit_ ## fun ## size(addr, arg1_name, arg2_name); \ + __rte_bit_ ## family ## v ## fun ## size(addr, arg1_name, \ + arg2_name); \ } -#define __RTE_BIT_OVERLOAD_3(fun, qualifier, arg1_type, arg1_name, arg2_type, \ +#define __RTE_BIT_OVERLOAD_SZ_3(family, fun, c, size, arg1_type, arg1_name, \ + arg2_type, arg2_name) \ + __RTE_BIT_OVERLOAD_V_3(family,, fun, c, size, arg1_type, arg1_name, \ + arg2_type, arg2_name) \ + __RTE_BIT_OVERLOAD_V_3(family, v_, fun, c volatile, size, arg1_type, \ + arg1_name, arg2_type, arg2_name) + +#define __RTE_BIT_OVERLOAD_3(family, fun, c, arg1_type, arg1_name, arg2_type, \ arg2_name) \ - __RTE_BIT_OVERLOAD_SZ_3(fun, qualifier, 32, arg1_type, arg1_name, \ + __RTE_BIT_OVERLOAD_SZ_3(family, fun, c, 32, arg1_type, arg1_name, \ arg2_type, arg2_name) \ - __RTE_BIT_OVERLOAD_SZ_3(fun, qualifier, 64, arg1_type, arg1_name, \ + __RTE_BIT_OVERLOAD_SZ_3(family, fun, c, 64, arg1_type, arg1_name, \ arg2_type, arg2_name) -#define __RTE_BIT_OVERLOAD_SZ_3R(fun, qualifier, size, ret_type, arg1_type, \ - arg1_name, arg2_type, arg2_name) \ +#define __RTE_BIT_OVERLOAD_V_3R(family, v, fun, c, size, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name) \ static inline ret_type \ - rte_bit_ ## fun(uint ## size ## _t *addr, arg1_type arg1_name, \ - arg2_type arg2_name) \ + rte_bit_ ## family ## fun(c uint ## size ## _t *addr, \ + arg1_type arg1_name, arg2_type arg2_name) \ { \ - return __rte_bit_ ## fun ## size(addr, arg1_name, arg2_name); \ + return __rte_bit_ ## family ## v ## fun ## size(addr, \ + arg1_name, \ + arg2_name); \ } -#define __RTE_BIT_OVERLOAD_3R(fun, qualifier, ret_type, arg1_type, arg1_name, \ - arg2_type, arg2_name) \ - __RTE_BIT_OVERLOAD_SZ_3R(fun, qualifier, 32, ret_type, arg1_type, \ +#define __RTE_BIT_OVERLOAD_SZ_3R(family, fun, c, size, ret_type, arg1_type, \ arg1_name, arg2_type, arg2_name) \ - __RTE_BIT_OVERLOAD_SZ_3R(fun, qualifier, 64, ret_type, arg1_type, \ - arg1_name, arg2_type, arg2_name) + __RTE_BIT_OVERLOAD_V_3R(family,, fun, c, size, ret_type, \ + arg1_type, arg1_name, arg2_type, arg2_name) \ + __RTE_BIT_OVERLOAD_V_3R(family, v_, fun, c volatile, size, \ + ret_type, arg1_type, arg1_name, arg2_type, \ + arg2_name) -#define __RTE_BIT_OVERLOAD_SZ_4(fun, qualifier, size, arg1_type, arg1_name, \ - arg2_type, arg2_name, arg3_type, arg3_name) \ +#define __RTE_BIT_OVERLOAD_3R(family, fun, c, ret_type, arg1_type, arg1_name, \ + arg2_type, arg2_name) \ + __RTE_BIT_OVERLOAD_SZ_3R(family, fun, c, 32, ret_type, \ + arg1_type, arg1_name, arg2_type, arg2_name) \ + __RTE_BIT_OVERLOAD_SZ_3R(family, fun, c, 64, ret_type, \ + arg1_type, arg1_name, arg2_type, arg2_name) + +#define __RTE_BIT_OVERLOAD_V_4(family, v, fun, c, size, arg1_type, arg1_name, \ + arg2_type, arg2_name, arg3_type, arg3_name) \ static inline void \ - rte_bit_ ## fun(uint ## size ## _t *addr, arg1_type arg1_name, \ - arg2_type arg2_name, arg3_type arg3_name) \ + rte_bit_ ## family ## fun(c uint ## size ## _t *addr, \ + arg1_type arg1_name, arg2_type arg2_name, \ + arg3_type arg3_name) \ { \ - __rte_bit_ ## fun ## size(addr, arg1_name, arg2_name, \ - arg3_name); \ + __rte_bit_ ## family ## v ## fun ## size(addr, arg1_name, \ + arg2_name, \ + arg3_name); \ } -#define __RTE_BIT_OVERLOAD_4(fun, qualifier, arg1_type, arg1_name, arg2_type, \ - arg2_name, arg3_type, arg3_name) \ - __RTE_BIT_OVERLOAD_SZ_4(fun, qualifier, 32, arg1_type, arg1_name, \ +#define __RTE_BIT_OVERLOAD_SZ_4(family, fun, c, size, arg1_type, arg1_name, \ arg2_type, arg2_name, arg3_type, arg3_name) \ - __RTE_BIT_OVERLOAD_SZ_4(fun, qualifier, 64, arg1_type, arg1_name, \ - arg2_type, arg2_name, arg3_type, arg3_name) - -#define __RTE_BIT_OVERLOAD_SZ_4R(fun, qualifier, size, ret_type, arg1_type, \ - arg1_name, arg2_type, arg2_name, arg3_type, \ - arg3_name) \ + __RTE_BIT_OVERLOAD_V_4(family,, fun, c, size, arg1_type, \ + arg1_name, arg2_type, arg2_name, arg3_type, \ + arg3_name) \ + __RTE_BIT_OVERLOAD_V_4(family, v_, fun, c volatile, size, \ + arg1_type, arg1_name, arg2_type, arg2_name, \ + arg3_type, arg3_name) + +#define __RTE_BIT_OVERLOAD_4(family, fun, c, arg1_type, arg1_name, arg2_type, \ + arg2_name, arg3_type, arg3_name) \ + __RTE_BIT_OVERLOAD_SZ_4(family, fun, c, 32, arg1_type, \ + arg1_name, arg2_type, arg2_name, arg3_type, \ + arg3_name) \ + __RTE_BIT_OVERLOAD_SZ_4(family, fun, c, 64, arg1_type, \ + arg1_name, arg2_type, arg2_name, arg3_type, \ + arg3_name) + +#define __RTE_BIT_OVERLOAD_V_4R(family, v, fun, c, size, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name, arg3_type, \ + arg3_name) \ static inline ret_type \ - rte_bit_ ## fun(uint ## size ## _t *addr, arg1_type arg1_name, \ - arg2_type arg2_name, arg3_type arg3_name) \ + rte_bit_ ## family ## fun(c uint ## size ## _t *addr, \ + arg1_type arg1_name, arg2_type arg2_name, \ + arg3_type arg3_name) \ { \ - return __rte_bit_ ## fun ## size(addr, arg1_name, arg2_name, \ - arg3_name); \ + return __rte_bit_ ## family ## v ## fun ## size(addr, \ + arg1_name, \ + arg2_name, \ + arg3_name); \ } -#define __RTE_BIT_OVERLOAD_4R(fun, qualifier, ret_type, arg1_type, arg1_name, \ - arg2_type, arg2_name, arg3_type, arg3_name) \ - __RTE_BIT_OVERLOAD_SZ_4R(fun, qualifier, 32, ret_type, arg1_type, \ +#define __RTE_BIT_OVERLOAD_SZ_4R(family, fun, c, size, ret_type, arg1_type, \ arg1_name, arg2_type, arg2_name, arg3_type, \ arg3_name) \ - __RTE_BIT_OVERLOAD_SZ_4R(fun, qualifier, 64, ret_type, arg1_type, \ - arg1_name, arg2_type, arg2_name, arg3_type, \ - arg3_name) - -__RTE_BIT_OVERLOAD_2R(test, const, bool, unsigned int, nr) -__RTE_BIT_OVERLOAD_2(set,, unsigned int, nr) -__RTE_BIT_OVERLOAD_2(clear,, unsigned int, nr) -__RTE_BIT_OVERLOAD_3(assign,, unsigned int, nr, bool, value) -__RTE_BIT_OVERLOAD_2(flip,, unsigned int, nr) - -__RTE_BIT_OVERLOAD_3R(atomic_test, const, bool, unsigned int, nr, + __RTE_BIT_OVERLOAD_V_4R(family,, fun, c, size, ret_type, arg1_type, \ + arg1_name, arg2_type, arg2_name, arg3_type, \ + arg3_name) \ + __RTE_BIT_OVERLOAD_V_4R(family, v_, fun, c volatile, size, \ + ret_type, arg1_type, arg1_name, arg2_type, \ + arg2_name, arg3_type, arg3_name) + +#define __RTE_BIT_OVERLOAD_4R(family, fun, c, ret_type, arg1_type, arg1_name, \ + arg2_type, arg2_name, arg3_type, arg3_name) \ + __RTE_BIT_OVERLOAD_SZ_4R(family, fun, c, 32, ret_type, \ + arg1_type, arg1_name, arg2_type, arg2_name, \ + arg3_type, arg3_name) \ + __RTE_BIT_OVERLOAD_SZ_4R(family, fun, c, 64, ret_type, \ + arg1_type, arg1_name, arg2_type, arg2_name, \ + arg3_type, arg3_name) + +__RTE_BIT_OVERLOAD_2R(, test, const, bool, unsigned int, nr) +__RTE_BIT_OVERLOAD_2(, set,, unsigned int, nr) +__RTE_BIT_OVERLOAD_2(, clear,, unsigned int, nr) +__RTE_BIT_OVERLOAD_3(, assign,, unsigned int, nr, bool, value) +__RTE_BIT_OVERLOAD_2(, flip,, unsigned int, nr) + +__RTE_BIT_OVERLOAD_3R(atomic_, test, const, bool, unsigned int, nr, int, memory_order) -__RTE_BIT_OVERLOAD_3(atomic_set,, unsigned int, nr, int, memory_order) -__RTE_BIT_OVERLOAD_3(atomic_clear,, unsigned int, nr, int, memory_order) -__RTE_BIT_OVERLOAD_4(atomic_assign,, unsigned int, nr, bool, value, +__RTE_BIT_OVERLOAD_3(atomic_, set,, unsigned int, nr, int, memory_order) +__RTE_BIT_OVERLOAD_3(atomic_, clear,, unsigned int, nr, int, memory_order) +__RTE_BIT_OVERLOAD_4(atomic_, assign,, unsigned int, nr, bool, value, int, memory_order) -__RTE_BIT_OVERLOAD_3(atomic_flip,, unsigned int, nr, int, memory_order) -__RTE_BIT_OVERLOAD_3R(atomic_test_and_set,, bool, unsigned int, nr, +__RTE_BIT_OVERLOAD_3(atomic_, flip,, unsigned int, nr, int, memory_order) +__RTE_BIT_OVERLOAD_3R(atomic_, test_and_set,, bool, unsigned int, nr, int, memory_order) -__RTE_BIT_OVERLOAD_3R(atomic_test_and_clear,, bool, unsigned int, nr, +__RTE_BIT_OVERLOAD_3R(atomic_, test_and_clear,, bool, unsigned int, nr, int, memory_order) -__RTE_BIT_OVERLOAD_4R(atomic_test_and_assign,, bool, unsigned int, nr, +__RTE_BIT_OVERLOAD_4R(atomic_, test_and_assign,, bool, unsigned int, nr, bool, value, int, memory_order) #endif