From patchwork Sun May 5 07:33:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 139864 Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 40C7943FAC; Sun, 5 May 2024 09:43:59 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1B6A340647; Sun, 5 May 2024 09:43:42 +0200 (CEST) Received: from EUR02-AM0-obe.outbound.protection.outlook.com (mail-am0eur02on2058.outbound.protection.outlook.com [40.107.247.58]) by mails.dpdk.org (Postfix) with ESMTP id 669AC402DE for ; Sun, 5 May 2024 09:43:37 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CHV2nx0q8BD6nhpVv6sTu9Ua5Kpprn2oqvG+afk1vt8CyZPzsGT2zDPC9IrrLvqRDOnt/rOsu2Cu9gC1WrPyIu+7R14Jlo+38sgfGZa7CcPGTyGNuQMj+I7/vIUMSLAqbVVsk+3R70GD4V9GnGkJ7JoVkXHkkbIy4RcTHDdAhWvuIfnuYmquVWWqiRuanO26Lfr5Fyu8l3BvDEY0Z5/W6ZF9azK1wfn48PYLyrlkVpJJ3nlJ3hw9FcGgt6v2fMvxVFl4HXOs9RbxMw9gjlFuTtb5w4r9e9QS4I6aKGBB98sunoZJFPAOFjJMNiXklGoktsa6UtXTxZMKK6aNLymDPQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=25zFeH5qAZJpWPxBUYFWpgHUkxk41okgz7mGFvGIbL8=; b=D89qWK2n0MqFDHXfuTSZhbQQzkACWckd03uUuiZUtXXeVNZc+weXbySx1ibBQkNk2pA6yImycLLe2BVNPRThIuhXn5DRdqMuBuNYcKxp0uA848NWwIVMalaHfqHAkQnwqJG6ADK8CG0eLEuQibhnCYCzLVz/4HGUjBjNsWgeXXT/7FZkTIlA7unOWkQt+v8wdsh/oVT80pmm2G9BA/qYrSCmNEsVvl9BtpJE2Ykeu4A27Y/cQmOKb51P3wI4DyKxCw9X7+MddVT7kaNu+TZ1YJ5KZvaK2JjB24JEBQ8stY4dfVnYVG95wKa7AZ30Kt8xgcMJzkpbeJiLU1OKhmYPNQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=25zFeH5qAZJpWPxBUYFWpgHUkxk41okgz7mGFvGIbL8=; b=L4gitdVprvdajJp1A4DqSRo9o7TMoPM4Zk8+xmSenhyVZmDCY6tYu988sC//wtgGZWVDQhCDAjoQmJ9opmHu8yMELTrEzruLWSJNjT24k8+DlS22kOr4eLkqAoCVZOGVY1l3SkbcGzOgd+Iw/PX0DxNdWE595iIYE2pt4yptWDqSrOIWXnD3pJZblpuG7bj2lyBZUR2gD8CvBOW03TyxQfnBsKDwUsEI6mMadp7zPyG0JnSqOl9MooaJsB5XoMQJE6AEXIEArMteZs2LnrzigLDwJJIjjXYObjwBdMuXH76tTdY+aR4+SJmnDp3N3FB6/ni6rWecNrfgHwCDIjbF5Q== Received: from AM5PR0101CA0024.eurprd01.prod.exchangelabs.com (2603:10a6:206:16::37) by AS8PR07MB7384.eurprd07.prod.outlook.com (2603:10a6:20b:289::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.39; Sun, 5 May 2024 07:43:34 +0000 Received: from AM3PEPF00009B9C.eurprd04.prod.outlook.com (2603:10a6:206:16:cafe::fe) by AM5PR0101CA0024.outlook.office365.com (2603:10a6:206:16::37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.40 via Frontend Transport; Sun, 5 May 2024 07:43:34 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by AM3PEPF00009B9C.mail.protection.outlook.com (10.167.16.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.18 via Frontend Transport; Sun, 5 May 2024 07:43:34 +0000 Received: from seliicinfr00049.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.66) with Microsoft SMTP Server id 15.2.1544.9; Sun, 5 May 2024 09:43:32 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00049.seli.gic.ericsson.se (Postfix) with ESMTP id 079CC38007A; Sun, 5 May 2024 09:43:32 +0200 (CEST) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , =?utf-8?q?Morten_Br=C3=B8rup?= , Tyler Retzlaff , Stephen Hemminger , Harry van Haaren , =?utf-8?q?Mattias_R=C3=B6nnb?= =?utf-8?q?lom?= Subject: [RFC v5 1/6] eal: add bitset type Date: Sun, 5 May 2024 09:33:08 +0200 Message-ID: <20240505073313.118515-1-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240216102348.480407-1-mattias.ronnblom@ericsson.com> References: <20240216102348.480407-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM3PEPF00009B9C:EE_|AS8PR07MB7384:EE_ X-MS-Office365-Filtering-Correlation-Id: f16aadfc-658f-4bf8-d32f-08dc6cd71169 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230031|1800799015|376005|36860700004|82310400017; X-Microsoft-Antispam-Message-Info: =?utf-8?q?IFT6MyWC32wiCK0y6psvc6hdUzXVOWF?= =?utf-8?q?q/OrItqlIAxKAfcLdx14xl4F14ithWy8XO/aVyCbHe0OlQV3ja0N82oHPI0ylJXO1?= =?utf-8?q?BadKeZxUHJ/LxzRmVvkzRiMT2C7QGm1HaB0AGjSdgGWWmj1s8JxkXKye+KUPZjtVe?= =?utf-8?q?Y+cyvHX9OPmACR93JrwZl57lki3Ng9W8RrylNqZorXjKh6mfbywomJ6K+ieLguSvV?= =?utf-8?q?ljxDZP7rwMV9nH0uQ3rPKM+UKpCRTb9MdacZOKFLb7Kyr5JWJoQtyQzKQBh5gAUZq?= =?utf-8?q?NwDba/B8lpLXr0ns8X9TXrhE3QL17beq9rgytP/z+kSEeE/4lbkEe9ql0ts3b6e6I?= =?utf-8?q?oVoUW4fARsxUyRH0bQYjj9LnOFv35ro7dUf+WtpXDvHyYjXGs2mMQ7+WfESxe+Wz6?= =?utf-8?q?+uz2kN5GRSqQBTEcT9+wfBsuZkJaiTxo3vUKYv/GhvcqwdE6VXiS3Jjehr130Vwl7?= =?utf-8?q?5tCyPCDvXSzzXeme0FwakbGKKEGUFjkqjTM58WDVdg6fY1QJbP6QQ1tnQ981gJfgz?= =?utf-8?q?WZZKhbaS7IkEGhBrCOmrV+B6YZl1is7DqHZoO1f98HeR5amAk77dvja0+eq6Y7N4e?= =?utf-8?q?OvJePuZiXdDoYdDUI5Y3t7LTL/AvZFghwK6wATXlvHC+HhXB31rISXjDuvHuPCegr?= =?utf-8?q?mODezP+XTxmc6nDxNpecCfdtTMFnLANR8PNYMp4NnwwkZ4ZjDbUvKF0bjUt8y3YFB?= =?utf-8?q?gNeicTcO/4Q3oiyxt2Vk4Tzckd8tUiIyc3MqbIePAQxc9rFIIOTfWUlSTJLFxcH9u?= =?utf-8?q?eSPb+7WMBpPhfoX4LYS+4AQ18KNlxpBQIfsefKs1cnetr5fMtYufp/grV2czAp1EC?= =?utf-8?q?Saz4XR4SGTGr2ACB7KPiF8krgL5HOdk8dqMHU8633gY+pOjgz3LBCbxPbWdYSkbmJ?= =?utf-8?q?cHHWr0vUD0gCCr1dlvLj3b7pLTXMkGjokjAQfpyKzcwNNintQgSi/T5l0CEjH7tp2?= =?utf-8?q?nQq3ZB4xJCNQ0H5Gtx+75UtXCHe2D9zv07Yki3nmpTS/tks6/5txnq8WtpaoN7Hly?= =?utf-8?q?Pqt2RBTxr1oRTpPfxHoL6JiJ/s0UAPh+Okq7hEu00Ro/WkE29CoATTE8Jna81x1sI?= =?utf-8?q?4EmwIt65kKsuXYlMLKvsOsXmI87DoMSMy6GZrp0FOK/aWVXt7/vYuNFLTk3iG+cNN?= =?utf-8?q?+7sjDg9Q1Za0lb1U3ng1UMPXVTCQw2XmfKxT65s+1GOcH0mw2ew0XyTZ5KVXtJpgU?= =?utf-8?q?+oNxCg1BniFHYhRTxmv5p2fCCNb7dTE3wvvkTdGjcu3SfAddb3Bq4oYwNjSTqqAY6?= =?utf-8?q?qvpX9Y0zEqEa/sfxay7UwTkNrZ0vid3PkBS0nK/0ZkfE39U0QtB7iTsE=3D?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230031)(1800799015)(376005)(36860700004)(82310400017); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 May 2024 07:43:34.5059 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f16aadfc-658f-4bf8-d32f-08dc6cd71169 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: AM3PEPF00009B9C.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR07MB7384 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Introduce a set of functions and macros that operate on sets of bits, kept in arrays of 64-bit words. RTE bitset is designed for bitsets which are larger than what fits in a single machine word (i.e., 64 bits). For very large bitsets, the API may be a more appropriate choice. Depends-on: series-31863 ("Improve EAL bit operations API") RFC v5: * Delegate bit test/set/clear/assign/flip to RTE bitops. * Note in the documentation that set/clear/assign/flip are not atomic. RFC v4: * Add function rte_bitset_flip() to change the value of a bit. * Add function rte_bitset_complement(), flipping the value of all bits. * Add function rte_bitset_assign(), setting the value of a bit based on a 'bool' parameter. * Add functions to perform logical shift the bitset left or right. * Add explicit destination bitset to logic operation type functions (e.g., rte_bitset_and()), to increase flexibility. * Split implementation and test suite into distinct commits. RFC v3: * Split the bitset from the htimer patchset, where it was originally hosted. * Rebase to current DPDK main. * Add note that rte_bitset_init() need not be called if bitset words have already been zeroed. * Use REGISTER_FAST_TEST instead of REGISTER_TEST_COMMAND. * Use rte_popcount64() instead of compiler builtin. RFC v2: * Replaced with include, to properly get size_t typedef. * Add to get __rte_experimental in . Signed-off-by: Mattias Rönnblom --- doc/api/doxy-api-index.md | 1 + lib/eal/common/meson.build | 1 + lib/eal/common/rte_bitset.c | 29 + lib/eal/include/meson.build | 1 + lib/eal/include/rte_bitset.h | 1061 ++++++++++++++++++++++++++++++++++ lib/eal/version.map | 2 + 6 files changed, 1095 insertions(+) create mode 100644 lib/eal/common/rte_bitset.c create mode 100644 lib/eal/include/rte_bitset.h diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index 8c1eb8fafa..1ce04a8edf 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -173,6 +173,7 @@ The public API headers are grouped by topics: [ring](@ref rte_ring.h), [stack](@ref rte_stack.h), [tailq](@ref rte_tailq.h), + [bitset](@ref rte_bitset.h), [bitmap](@ref rte_bitmap.h) - **packet framework**: diff --git a/lib/eal/common/meson.build b/lib/eal/common/meson.build index 22a626ba6f..c1bbf26654 100644 --- a/lib/eal/common/meson.build +++ b/lib/eal/common/meson.build @@ -31,6 +31,7 @@ sources += files( 'eal_common_uuid.c', 'malloc_elem.c', 'malloc_heap.c', + 'rte_bitset.c', 'rte_malloc.c', 'rte_random.c', 'rte_reciprocal.c', diff --git a/lib/eal/common/rte_bitset.c b/lib/eal/common/rte_bitset.c new file mode 100644 index 0000000000..35e55a64db --- /dev/null +++ b/lib/eal/common/rte_bitset.c @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2023 Ericsson AB + */ + +#include + +#include "rte_bitset.h" + +ssize_t +rte_bitset_to_str(const uint64_t *bitset, size_t num_bits, char *buf, + size_t capacity) +{ + size_t i; + + if (capacity < (num_bits + 1)) + return -EINVAL; + + for (i = 0; i < num_bits; i++) { + bool value; + + value = rte_bitset_test(bitset, num_bits - 1 - i); + + buf[i] = value ? '1' : '0'; + } + + buf[num_bits] = '\0'; + + return num_bits + 1; +} diff --git a/lib/eal/include/meson.build b/lib/eal/include/meson.build index e94b056d46..4b5f120a66 100644 --- a/lib/eal/include/meson.build +++ b/lib/eal/include/meson.build @@ -5,6 +5,7 @@ includes += include_directories('.') headers += files( 'rte_alarm.h', + 'rte_bitset.h', 'rte_bitmap.h', 'rte_bitops.h', 'rte_branch_prediction.h', diff --git a/lib/eal/include/rte_bitset.h b/lib/eal/include/rte_bitset.h new file mode 100644 index 0000000000..49a07c77b8 --- /dev/null +++ b/lib/eal/include/rte_bitset.h @@ -0,0 +1,1061 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2023 Ericsson AB + */ + +#ifndef _RTE_BITSET_H_ +#define _RTE_BITSET_H_ + +/** + * @file + * RTE Bitset + * + * This file provides functions and macros for querying and + * manipulating sets of bits kept in arrays of @c uint64_t-sized + * elements. + * + * The bits in a bitset are numbered from 0 to @c size - 1, with the + * lowest index being the least significant bit. + * + * The bitset array must be properly aligned. + * + * For optimal performance, the @c size parameter, required by + * many of the API's functions, should be a compile-time constant. + * + * For large bitsets, the rte_bitmap.h API may be more appropriate. + * + * @warning + * All functions modifying a bitset may overwrite any unused bits of + * the last word. Such unused bits are ignored by all functions reading + * bits. + * + */ + +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * The size (in bytes) of each element in the array used to represent + * a bitset. + */ +#define RTE_BITSET_WORD_SIZE (sizeof(uint64_t)) + +/** + * The size (in bits) of each element in the array used to represent + * a bitset. + */ +#define RTE_BITSET_WORD_BITS (RTE_BITSET_WORD_SIZE * CHAR_BIT) + +/** + * Computes the number of words required to store @c size bits. + */ +#define RTE_BITSET_NUM_WORDS(size) \ + ((size + RTE_BITSET_WORD_BITS - 1) / RTE_BITSET_WORD_BITS) + +/** + * Computes the amount of memory (in bytes) required to fit a bitset + * holding @c size bits. + */ +#define RTE_BITSET_SIZE(size) \ + ((size_t)(RTE_BITSET_NUM_WORDS(size) * RTE_BITSET_WORD_SIZE)) + +#define __RTE_BITSET_WORD_IDX(bit_num) ((bit_num) / RTE_BITSET_WORD_BITS) +#define __RTE_BITSET_BIT_OFFSET(bit_num) ((bit_num) % RTE_BITSET_WORD_BITS) +#define __RTE_BITSET_UNUSED(size) \ + ((RTE_BITSET_NUM_WORDS(size) * RTE_BITSET_WORD_BITS) \ + - (size)) +#define __RTE_BITSET_USED_MASK(size) \ + (UINT64_MAX >> __RTE_BITSET_UNUSED(size)) + +#define __RTE_BITSET_DELEGATE_N(fun, bitset, bit_num, ...) \ + fun(&(bitset)[__RTE_BITSET_WORD_IDX(bit_num)], \ + __RTE_BITSET_BIT_OFFSET(bit_num), __VA_ARGS__) + +/* MSVC doesn't have ##__VA_ARGS__, so argument-less -> special case */ +#define __RTE_BITSET_DELEGATE(fun, bitset, bit_num) \ + fun(&(bitset)[__RTE_BITSET_WORD_IDX(bit_num)], \ + __RTE_BITSET_BIT_OFFSET(bit_num)) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Declare a bitset. + * + * Declare (e.g., as a struct field) or define (e.g., as a stack + * variable) a bitset of the specified size. + * + * @param size + * The number of bits the bitset must be able to represent. Must be + * a compile-time constant. + * @param name + * The field or variable name of the resulting definition. + */ +#define RTE_BITSET_DECLARE(name, size) \ + uint64_t name[RTE_BITSET_NUM_WORDS(size)] + +#define __RTE_BITSET_FOREACH_LEFT(var, size, start_bit, len) \ + ((len) - 1 - ((var) >= (start_bit) ? (var) - (start_bit) : \ + (size) - (start_bit) + (var))) + +#define __RTE_BITSET_FOREACH(var, bitset, size, start_bit, len, flags) \ + for ((var) = __rte_bitset_find(bitset, size, start_bit, len, \ + flags); \ + (var) != -1; \ + (var) = __RTE_BITSET_FOREACH_LEFT(var, size, start_bit, \ + len) > 0 ? \ + __rte_bitset_find(bitset, size, \ + ((var) + 1) % (size), \ + __RTE_BITSET_FOREACH_LEFT(var, \ + size, \ + start_bit, \ + len), \ + flags) : -1) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Iterate over all bits set. + * + * This macro iterates over all bits set (i.e., all ones) in the + * bitset, in the forward direction (i.e., starting with the least + * significant '1'). + * + * @param var + * An iterator variable of type @c ssize_t. For each successive + * iteration, this variable will hold the bit index of a set bit. + * @param bitset + * A const uint64_t * pointer to the bitset array. + * @param size + * The size of the bitset (in bits). + */ + +#define RTE_BITSET_FOREACH_SET(var, bitset, size) \ + __RTE_BITSET_FOREACH(var, bitset, size, 0, size, 0) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Iterate over all bits cleared. + * + * This macro iterates over all bits cleared in the bitset, in the + * forward direction (i.e., starting with the lowest-indexed set bit). + * + * @param var + * An iterator variable of type @c ssize_t. For each successive iteration, + * this variable will hold the bit index of a cleared bit. + * @param bitset + * A const uint64_t * pointer to the bitset array. + * @param size + * The size of the bitset (in bits). + */ + +#define RTE_BITSET_FOREACH_CLEAR(var, bitset, size) \ + __RTE_BITSET_FOREACH(var, bitset, size, 0, size, \ + __RTE_BITSET_FIND_FLAG_FIND_CLEAR) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Iterate over all bits set within a range. + * + * This macro iterates over all bits set (i.e., all ones) in the + * specified range, in the forward direction (i.e., starting with the + * least significant '1'). + * + * @param var + * An iterator variable of type @c ssize_t. For each successive iteration, + * this variable will hold the bit index of a set bit. + * @param bitset + * A const uint64_t * pointer to the bitset array. + * @param size + * The size of the bitset (in bits). + * @param start_bit + * The index of the first bit to check. Must be less than @c size. + * @param len + * The length (in bits) of the range. @c start_bit + @c len must be less + * than or equal to @c size. + */ + +#define RTE_BITSET_FOREACH_SET_RANGE(var, bitset, size, start_bit, \ + len) \ + __RTE_BITSET_FOREACH(var, bitset, size, start_bit, len, 0) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Iterate over all cleared bits within a range. + * + * This macro iterates over all bits cleared (i.e., all zeroes) in the + * specified range, in the forward direction (i.e., starting with the + * least significant '0'). + * + * @param var + * An iterator variable of type @c ssize_t. For each successive iteration, + * this variable will hold the bit index of a set bit. + * @param bitset + * A const uint64_t * pointer to the bitset array. + * @param size + * The size of the bitset (in bits). + * @param start_bit + * The index of the first bit to check. Must be less than @c size. + * @param len + * The length (in bits) of the range. @c start_bit + @c len must be less + * than or equal to @c size. + */ + +#define RTE_BITSET_FOREACH_CLEAR_RANGE(var, bitset, size, start_bit, \ + len) \ + __RTE_BITSET_FOREACH(var, bitset, size, start_bit, len, \ + __RTE_BITSET_FIND_FLAG_FIND_CLEAR) + +#define RTE_BITSET_FOREACH_SET_WRAP(var, bitset, size, start_bit, \ + len) \ + __RTE_BITSET_FOREACH(var, bitset, size, start_bit, len, \ + __RTE_BITSET_FIND_FLAG_WRAP) + +#define RTE_BITSET_FOREACH_CLEAR_WRAP(var, bitset, size, start_bit, \ + len) \ + __RTE_BITSET_FOREACH(var, bitset, size, start_bit, len, \ + __RTE_BITSET_FIND_FLAG_WRAP | \ + __RTE_BITSET_FIND_FLAG_FIND_CLEAR) + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Initializes a bitset. + * + * All bits are cleared. + * + * In case all words in the bitset array are already set to zero by + * other means (e.g., at the time of memory allocation), this function + * need not be called. + * + * @param bitset + * A pointer to the array of bitset 64-bit words. + * @param size + * The size of the bitset (in bits). + */ + +__rte_experimental +static inline void +rte_bitset_init(uint64_t *bitset, size_t size) +{ + memset(bitset, 0, RTE_BITSET_SIZE(size)); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Test if a bit is set. + * + * @param bitset + * A pointer to the array of words making up the bitset. + * @param bit_num + * Index of the bit to test. Index 0 is the least significant bit. + * @return + * Returns true if the bit is '1', and false if the bit is '0'. + */ + +__rte_experimental +static inline bool +rte_bitset_test(const uint64_t *bitset, size_t bit_num) +{ + return __RTE_BITSET_DELEGATE(rte_bit_test, bitset, bit_num); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Set a bit in the bitset. + * + * Bits are numbered from 0 to (size - 1) (inclusive). + * + * The operation is not guaranteed to be atomic. + * + * @param bitset + * A pointer to the array of words making up the bitset. + * @param bit_num + * The index of the bit to be set. + */ + +__rte_experimental +static inline void +rte_bitset_set(uint64_t *bitset, size_t bit_num) +{ + __RTE_BITSET_DELEGATE(rte_bit_set, bitset, bit_num); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Clear a bit in the bitset. + * + * Bits are numbered 0 to (size - 1) (inclusive). + * + * The operation is not guaranteed to be atomic. + * + * @param bitset + * A pointer to the array of words making up the bitset. + * @param bit_num + * The index of the bit to be cleared. + */ + +__rte_experimental +static inline void +rte_bitset_clear(uint64_t *bitset, size_t bit_num) +{ + __RTE_BITSET_DELEGATE(rte_bit_clear, bitset, bit_num); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Set or clear a bit in the bitset. + * + * Bits are numbered 0 to (size - 1) (inclusive). + * + * The operation is not guaranteed to be atomic. + * + * @param bitset + * A pointer to the array of words making up the bitset. + * @param bit_num + * The index of the bit to be set or cleared. + * @param bit_value + * Control if the bit should be set or cleared. + */ + +__rte_experimental +static inline void +rte_bitset_assign(uint64_t *bitset, size_t bit_num, bool bit_value) +{ + __RTE_BITSET_DELEGATE_N(rte_bit_assign, bitset, bit_num, bit_value); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Change the value of a bit in the bitset. + * + * Bits are numbered 0 to (size - 1) (inclusive). + * + * The operation is not guaranteed to be atomic. + * + * @param bitset + * A pointer to the array of words making up the bitset. + * @param bit_num + * The index of the bit to be flipped. + */ + +__rte_experimental +static inline void +rte_bitset_flip(uint64_t *bitset, size_t bit_num) +{ + __RTE_BITSET_DELEGATE(rte_bit_flip, bitset, bit_num); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Set all bits in the bitset. + * + * @param bitset + * A pointer to the array of words making up the bitset. + * @param size + * The size of the bitset (in bits). + */ + +__rte_experimental +static inline void +rte_bitset_set_all(uint64_t *bitset, size_t size) +{ + memset(bitset, 0xFF, RTE_BITSET_SIZE(size)); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Clear all bits in the bitset. + * + * @param bitset + * A pointer to the array of words making up the bitset. + * @param size + * The size of the bitset (in bits). + */ + +__rte_experimental +static inline void +rte_bitset_clear_all(uint64_t *bitset, size_t size) +{ + rte_bitset_init(bitset, size); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Count all set bits (also known as the @e weight). + * + * @param bitset + * A pointer to the array of words making up the bitset. + * @param size + * The size of the bitset (in bits). + * @return + * Returns the number of '1' bits in the bitset. + */ + +__rte_experimental +static inline size_t +rte_bitset_count_set(const uint64_t *bitset, size_t size) +{ + size_t i; + size_t total = 0; + + /* + * Unused bits in a rte_bitset are always '0', and thus are + * not included in this count. + */ + for (i = 0; i < RTE_BITSET_NUM_WORDS(size) - 1; i++) + total += rte_popcount64(bitset[i]); + + total += rte_popcount64(bitset[i] & __RTE_BITSET_USED_MASK(size)); + + return total; +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Count all cleared bits. + * + * @param bitset + * A pointer to the array of words making up the bitset. + * @param size + * The size of the bitset (in bits). + * @return + * Returns the number of '0' bits in the bitset. + */ + +__rte_experimental +static inline size_t +rte_bitset_count_clear(const uint64_t *bitset, size_t size) +{ + return size - rte_bitset_count_set(bitset, size); +} + +#define __RTE_BITSET_FIND_FLAG_FIND_CLEAR (1U << 0) +#define __RTE_BITSET_FIND_FLAG_WRAP (1U << 1) + +__rte_experimental +static inline ssize_t +__rte_bitset_find_nowrap(const uint64_t *bitset, size_t __rte_unused size, + size_t start_bit, size_t len, bool find_clear) +{ + size_t word_idx; + size_t offset; + size_t end_bit = start_bit + len; + + RTE_ASSERT(end_bit <= size); + + if (unlikely(len == 0)) + return -1; + + word_idx = __RTE_BITSET_WORD_IDX(start_bit); + offset = __RTE_BITSET_BIT_OFFSET(start_bit); + + while (word_idx <= __RTE_BITSET_WORD_IDX(end_bit - 1)) { + uint64_t word; + int word_ffs; + + word = bitset[word_idx]; + if (find_clear) + word = ~word; + + word >>= offset; + + word_ffs = __builtin_ffsll(word); + + if (word_ffs != 0) { + ssize_t ffs = start_bit + word_ffs - 1; + + /* + * Check if set bit were among the last, + * unused bits, in the last word. + */ + if (unlikely(ffs >= (ssize_t)end_bit)) + return -1; + + return ffs; + } + + start_bit += (RTE_BITSET_WORD_BITS - offset); + word_idx++; + offset = 0; + } + + return -1; + +} + +__rte_experimental +static inline ssize_t +__rte_bitset_find(const uint64_t *bitset, size_t size, size_t start_bit, + size_t len, unsigned int flags) +{ + bool find_clear = flags & __RTE_BITSET_FIND_FLAG_FIND_CLEAR; + bool may_wrap = flags & __RTE_BITSET_FIND_FLAG_WRAP; + bool does_wrap = (start_bit + len) > size; + ssize_t rc; + + RTE_ASSERT(len <= size); + if (!may_wrap) + RTE_ASSERT(!does_wrap); + + if (may_wrap && does_wrap) { + size_t len0 = size - start_bit; + size_t len1 = len - len0; + + rc = __rte_bitset_find_nowrap(bitset, size, start_bit, len0, + find_clear); + if (rc < 0) + rc = __rte_bitset_find_nowrap(bitset, size, + 0, len1, find_clear); + } else + rc = __rte_bitset_find_nowrap(bitset, size, start_bit, + len, find_clear); + + return rc; +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Find first bit set. + * + * Scans the bitset in the forward direction (i.e., starting at the + * least significant bit), and returns the index of the first '1'. + * + * @param bitset + * A pointer to the array of words making up the bitset. + * @param size + * The size of the bitset (in bits). + * @return + * Returns the index of the least significant '1', or -1 if all + * bits are '0'. + */ + +__rte_experimental +static inline ssize_t +rte_bitset_find_first_set(const uint64_t *bitset, size_t size) +{ + return __rte_bitset_find(bitset, size, 0, size, 0); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Find first bit set at offset. + * + * Scans the bitset in the forward direction (i.e., starting at the + * least significant bit), starting at an offset @c start_bit into the + * bitset, and returns the index of the first '1' encountered. + * + * @param bitset + * A pointer to the array of words making up the bitset. + * @param size + * The size of the bitset (in bits). + * @param start_bit + * The index of the first bit to check. Must be less than @c size. + * @param len + * The number of bits to scan. @c start_bit + @c len must be less + * than or equal to @c size. + * @return + * Returns the index of the least significant '1', or -1 if all + * bits are '0'. + */ + +__rte_experimental +static inline ssize_t +rte_bitset_find_set(const uint64_t *bitset, size_t size, + size_t start_bit, size_t len) +{ + return __rte_bitset_find(bitset, size, start_bit, len, 0); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Find first bit set at offset, with wrap-around. + * + * Scans the bitset in the forward direction (i.e., starting at the + * least significant bit), starting at an offset @c start_bit into the + * bitset. If no '1' is encountered before the end of the bitset, the search + * will continue at index 0. + * + * @param bitset + * A pointer to the array of words making up the bitset. + * @param size + * The size of the bitset (in bits). + * @param start_bit + * The index of the first bit to check. Must be less than @c size. + * @param len + * The number of bits to scan. @c start_bit + @c len must be less + * than or equal to @c size. + * @return + * Returns the index of the least significant '1', or -1 if all + * bits are '0'. + */ + +__rte_experimental +static inline ssize_t +rte_bitset_find_set_wrap(const uint64_t *bitset, size_t size, + size_t start_bit, size_t len) +{ + return __rte_bitset_find(bitset, size, start_bit, len, + __RTE_BITSET_FIND_FLAG_WRAP); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Find first cleared bit. + * + * Scans the bitset in the forward direction (i.e., starting at the + * least significant bit), and returns the index of the first '0'. + * + * @param bitset + * A pointer to the array of words making up the bitset. + * @param size + * The size of the bitset (in bits). + * @return + * Returns the index of the least significant '0', or -1 if all + * bits are '1'. + */ + +__rte_experimental +static inline ssize_t +rte_bitset_find_first_clear(const uint64_t *bitset, size_t size) +{ + return __rte_bitset_find(bitset, size, 0, size, + __RTE_BITSET_FIND_FLAG_FIND_CLEAR); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Find first cleared bit at offset. + * + * Scans the bitset in the forward direction (i.e., starting at the + * least significant bit), starting at an offset @c start_bit into the + * bitset, and returns the index of the first '0' encountered. + * + * @param bitset + * A pointer to the array of words making up the bitset. + * @param size + * The size of the bitset (in bits). + * @param start_bit + * The index of the first bit to check. Must be less than @c size. + * @param len + * The number of bits to scan. @c start_bit + @c len must be less + * than or equal to @c size. + * @return + * Returns the index of the least significant '0', or -1 if all + * bits are '1'. + */ + +__rte_experimental +static inline ssize_t +rte_bitset_find_clear(const uint64_t *bitset, size_t size, + size_t start_bit, size_t len) +{ + return __rte_bitset_find(bitset, size, start_bit, len, + __RTE_BITSET_FIND_FLAG_FIND_CLEAR); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Find first cleared bit at offset, with wrap-around. + * + * Scans the bitset in the forward direction (i.e., starting at the + * least significant bit), starting at an offset @c start_bit into the + * bitset. If no '0' is encountered before the end of the bitset, the + * search will continue at index 0. + * + * @param bitset + * A pointer to the array of words making up the bitset. + * @param size + * The size of the bitset (in bits). + * @param start_bit + * The index of the first bit to check. Must be less than @c size. + * @param len + * The number of bits to scan. @c start_bit + @c len must be less + * than or equal to @c size. + * @return + * Returns the index of the least significant '0', or -1 if all + * bits are '1'. + */ + +__rte_experimental +static inline ssize_t +rte_bitset_find_clear_wrap(const uint64_t *bitset, size_t size, + size_t start_bit, size_t len) +{ + return __rte_bitset_find(bitset, size, start_bit, len, + __RTE_BITSET_FIND_FLAG_FIND_CLEAR | + __RTE_BITSET_FIND_FLAG_WRAP); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Copy bitset. + * + * Copy the bits of the @c src_bitset to the @c dst_bitset. + * + * The bitsets may not overlap and must be of equal size. + * + * @param dst_bitset + * A pointer to the array of words making up the bitset. + * @param src_bitset + * A pointer to the array of words making up the bitset. + * @param size + * The size of the bitsets (in bits). + */ + +__rte_experimental +static inline void +rte_bitset_copy(uint64_t *__rte_restrict dst_bitset, + const uint64_t *__rte_restrict src_bitset, + size_t size) +{ + rte_memcpy(dst_bitset, src_bitset, RTE_BITSET_SIZE(size)); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Bitwise or two bitsets. + * + * Perform a bitwise OR operation on all bits in the two equal-size + * bitsets @c src_bitset0 and @c src_bitset1, and store the results in + * @c dst_bitset. + * + * @param dst_bitset + * A pointer to the destination bitset. + * @param src_bitset0 + * A pointer to the first source bitset. + * @param src_bitset1 + * A pointer to the second source bitset. + * @param size + * The size of the bitsets (in bits). + */ + +__rte_experimental +static inline void +rte_bitset_or(uint64_t *dst_bitset, const uint64_t *src_bitset0, + const uint64_t *src_bitset1, size_t size) +{ + size_t i; + + for (i = 0; i < RTE_BITSET_NUM_WORDS(size); i++) + dst_bitset[i] = src_bitset0[i] | src_bitset1[i]; +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Bitwise and two bitsets. + * + * Perform a bitwise AND operation on all bits in the two equal-size + * bitsets @c src_bitset0 and @c src_bitset1, and store the result in + * @c dst_bitset. + * + * @param dst_bitset + * A pointer to the destination bitset. + * @param src_bitset0 + * A pointer to the first source bitset. + * @param src_bitset1 + * A pointer to the second source bitset. + * @param size + * The size of the bitsets (in bits). + */ + +__rte_experimental +static inline void +rte_bitset_and(uint64_t *dst_bitset, const uint64_t *src_bitset0, + const uint64_t *src_bitset1, size_t size) +{ + size_t i; + + for (i = 0; i < RTE_BITSET_NUM_WORDS(size); i++) + dst_bitset[i] = src_bitset0[i] & src_bitset1[i]; +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Bitwise xor two bitsets. + * + * Perform a bitwise XOR operation on all bits in the two equal-size + * bitsets @c src_bitset0 and @c src_bitset1, and store the result in + * @c dst_bitset. + * + * @param dst_bitset + * A pointer to the destination bitset. + * @param src_bitset0 + * A pointer to the first source bitset. + * @param src_bitset1 + * A pointer to the second source bitset. + * @param size + * The size of the bitsets (in bits). + */ + +__rte_experimental +static inline void +rte_bitset_xor(uint64_t *dst_bitset, const uint64_t *src_bitset0, + const uint64_t *src_bitset1, size_t size) +{ + size_t i; + + for (i = 0; i < RTE_BITSET_NUM_WORDS(size); i++) + dst_bitset[i] = src_bitset0[i] ^ src_bitset1[i]; +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Compute the bitwise complement of a bitset. + * + * Flip every bit in the @c src_bitset, and store the result in @c + * dst_bitset. + * + * @param dst_bitset + * A pointer to the destination bitset. + * @param src_bitset + * A pointer to the source bitset. + * @param size + * The size of the bitsets (in bits). + */ + +__rte_experimental +static inline void +rte_bitset_complement(uint64_t *dst_bitset, const uint64_t *src_bitset, + size_t size) +{ + size_t i; + + for (i = 0; i < RTE_BITSET_NUM_WORDS(size); i++) + dst_bitset[i] = ~src_bitset[i]; +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Shift bitset left. + * + * Perform a logical shift left of (multiply) @c src_bitset, and store + * the result in @c dst_bitset. + * + * @param dst_bitset + * A pointer to the destination bitset. + * @param src_bitset + * A pointer to the source bitset. + * @param size + * The size of the bitsets (in bits). + * @param shift_bits + * The number of bits to shift the bitset. + */ + +__rte_experimental +static inline void +rte_bitset_shift_left(uint64_t *dst_bitset, const uint64_t *src_bitset, + size_t size, size_t shift_bits) +{ + const int src_word_offset = shift_bits / RTE_BITSET_WORD_BITS; + const int src_bit_offset = shift_bits % RTE_BITSET_WORD_BITS; + unsigned int dst_idx; + + for (dst_idx = 0; dst_idx < RTE_BITSET_NUM_WORDS(size); dst_idx++) { + int src_high_idx = dst_idx - src_word_offset; + uint64_t low_bits = 0; + uint64_t high_bits = 0; + + if (src_high_idx >= 0) { + int src_low_idx = src_high_idx - 1; + + high_bits = src_bitset[src_high_idx] << src_bit_offset; + + if (src_bit_offset > 0 && src_low_idx >= 0) + low_bits = src_bitset[src_low_idx] >> + (RTE_BITSET_WORD_BITS - src_bit_offset); + } + dst_bitset[dst_idx] = low_bits | high_bits; + } +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Shift bitset right. + * + * Perform a logical shift right of (divide) @c src_bitset, and store + * the result in @c dst_bitset. + * + * @param dst_bitset + * A pointer to the destination bitset. + * @param src_bitset + * A pointer to the source bitset. + * @param size + * The size of the bitsets (in bits). + * @param shift_bits + * The number of bits to shift the bitset. + */ + +__rte_experimental +static inline void +rte_bitset_shift_right(uint64_t *dst_bitset, const uint64_t *src_bitset, + size_t size, size_t shift_bits) +{ + const int num_words = RTE_BITSET_NUM_WORDS(size); + const uint64_t used_mask = __RTE_BITSET_USED_MASK(size); + const int src_word_offset = shift_bits / RTE_BITSET_WORD_BITS; + const int src_bit_offset = shift_bits % RTE_BITSET_WORD_BITS; + int dst_idx; + + for (dst_idx = 0; dst_idx < num_words; dst_idx++) { + int src_low_idx = src_word_offset + dst_idx; + int src_high_idx = src_low_idx + 1; + uint64_t src_low_word_bits = 0; + uint64_t src_high_word_bits = 0; + + if (src_low_idx < num_words) { + src_low_word_bits = src_bitset[src_low_idx]; + + if (src_low_idx == (num_words - 1)) + src_low_word_bits &= used_mask; + + src_low_word_bits >>= src_bit_offset; + + if (src_bit_offset > 0 && src_high_idx < num_words) { + src_high_word_bits = src_bitset[src_high_idx]; + + if (src_high_idx == (num_words - 1)) + src_high_word_bits &= used_mask; + + src_high_word_bits <<= + (RTE_BITSET_WORD_BITS - src_bit_offset); + } + } + dst_bitset[dst_idx] = src_low_word_bits | src_high_word_bits; + } +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Compare two bitsets. + * + * Compare two bitsets for equality. + * + * @param bitset_a + * A pointer to the destination bitset. + * @param bitset_b + * A pointer to the source bitset. + * @param size + * The size of the bitsets (in bits). + */ + +__rte_experimental +static inline bool +rte_bitset_equal(const uint64_t *bitset_a, const uint64_t *bitset_b, + size_t size) +{ + size_t i; + uint64_t last_a, last_b; + + for (i = 0; i < RTE_BITSET_NUM_WORDS(size) - 1; i++) + if (bitset_a[i] != bitset_b[i]) + return false; + + last_a = bitset_a[i] << __RTE_BITSET_UNUSED(size); + last_b = bitset_b[i] << __RTE_BITSET_UNUSED(size); + + return last_a == last_b; +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Converts a bitset to a string. + * + * This function prints a string representation of the bitstring to + * the supplied buffer. + * + * Each bit is represented either by '0' or '1' in the output, with + * the first (left-most) character in the output being the most + * significant bit. The resulting string is NUL terminated. + * + * @param bitset + * A pointer to the array of bitset 64-bit words. + * @param size + * The number of bits the bitset represent. + * @param buf + * A buffer to hold the output. + * @param capacity + * The size of the buffer. Must be @c size + 1 or larger. + * @return + * Returns the number of bytes written (i.e., @c size + 1), or -EINVAL + * in case the buffer capacity was too small. + */ + +__rte_experimental +ssize_t +rte_bitset_to_str(const uint64_t *bitset, size_t size, char *buf, + size_t capacity); + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_BITSET_H_ */ diff --git a/lib/eal/version.map b/lib/eal/version.map index 3df50c3fbb..254d3fd4b2 100644 --- a/lib/eal/version.map +++ b/lib/eal/version.map @@ -396,6 +396,8 @@ EXPERIMENTAL { # added in 24.03 rte_vfio_get_device_info; # WINDOWS_NO_EXPORT + + rte_bitset_to_str; }; INTERNAL {