From patchwork Fri Aug 9 20:14:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 143056 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4E9F24577D; Fri, 9 Aug 2024 22:24:26 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A9E8842E58; Fri, 9 Aug 2024 22:24:20 +0200 (CEST) Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2056.outbound.protection.outlook.com [40.107.21.56]) by mails.dpdk.org (Postfix) with ESMTP id DC5C042DC1 for ; Fri, 9 Aug 2024 22:24:18 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=JK96aCr6jdwgzJs47jUWe/RF0UXSiFk7zY0OA+TIXWzR2/XSgv9yIZpySNnLhgrLydyvu8JsA2+KgkcdCB5tv5wygqKnpFOGaXHJDv8QjOwngmweD5r1ETfp10ioj+SDMpKHzYNEEDDdfChNMZ1FXZgx30LaAhvBh7t+g+pJ6N0RhgDCg9z0tiO8bkwR8/CH0Vj2020LCFHjfNg/pi7T/a6brTu1fmXHWhJh5kvtJjun7pq2fXHAKSq0IH3zS87L2a1G2P+i4NzGic78Ok5stbPTpxduCWmRcOIKEVPvmAjMAroIHNSKaDxPCC8cKVMuIfSPdRpzk5qYXyMUEjNnOA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Mx2k72mHtMfvaE5BD+qrppgugwcEHxL9T3OLZibFkgE=; b=YCyz/MDHIOcqSORyiw56HlV3fI6v9vBrvQnkDHktoLD1O7IJ3PpEUfC8Eas+Z4yM85tqea1Zb3CiGi5OOLNBEQFzswxrsKtvAwrXSGQaCLADVlXGLShBMBQ9thnq4squm8H7WSuee63hBKxQ5JwspeDoyB4zTykwMhznx8flroGZwfGf5+HCfL9Cib+/EQpND6jS+ZVgrT58NHJctuzg88VUVMRt5xRyQR0lW4K5MmkjjCsGreoiot7ssjy26c1usxe3LNgYuWXx1KYt2d5ktcYxs4JfgbU6TvmjI8lQFQZunuuKyH030VhVPBzGKa5z2qZ7szdiwAGta7yisnLwpw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Mx2k72mHtMfvaE5BD+qrppgugwcEHxL9T3OLZibFkgE=; b=dcgp5Lvzh8n9n6NHCTlSMg8sTvp4Xxv2rjdWmwOJuZmLhn37f08Zl3Th32yA2ZtndMrnSN22GnMsT5FR6pfw9lWZbpOjmhZUXwUt7oQMP+qiJx2EAmuVnF7+xej1Tb1PXRRTJcORYsyAMl5MjdInXyT9+47jxTmie0WLHR7uh0Tc76Tz82pcfMTEJhxZdmy6D0J55EaQ+CYRh6jt3HY+SmpUQXDCT7PrSzGzifg/WBSKFznnRdz3143rUegmnhtUPHI3E/V90JpNDyXDsEz9DWP35Ri9BPDDFTuPxTyO3c9rm8X20Uaox1XjY9MOS5M/VE4PGjXpZ8+vIc9UyXgzzg== Received: from AM4PR07CA0028.eurprd07.prod.outlook.com (2603:10a6:205:1::41) by DBAPR07MB6583.eurprd07.prod.outlook.com (2603:10a6:10:184::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.15; Fri, 9 Aug 2024 20:24:16 +0000 Received: from AM4PEPF00027A6C.eurprd04.prod.outlook.com (2603:10a6:205:1:cafe::4f) by AM4PR07CA0028.outlook.office365.com (2603:10a6:205:1::41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.10 via Frontend Transport; Fri, 9 Aug 2024 20:24:16 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by AM4PEPF00027A6C.mail.protection.outlook.com (10.167.16.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.8 via Frontend Transport; Fri, 9 Aug 2024 20:24:16 +0000 Received: from seliicinfr00050.seli.gic.ericsson.se (153.88.142.248) by smtp-central.internal.ericsson.com (100.87.178.65) with Microsoft SMTP Server id 15.2.1544.11; Fri, 9 Aug 2024 22:24:15 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00050.seli.gic.ericsson.se (Postfix) with ESMTP id BD8EF1C006B; Fri, 9 Aug 2024 22:24:15 +0200 (CEST) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: CC: , =?utf-8?q?Morten_Br=C3=B8rup?= , Tyler Retzlaff , Stephen Hemminger , Harry van Haaren , =?utf-8?q?Mattias_R=C3=B6nnb?= =?utf-8?q?lom?= Subject: [PATCH 5/6] service: use multi-word bitset to represent service flags Date: Fri, 9 Aug 2024 22:14:39 +0200 Message-ID: <20240809201440.590464-5-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240809201440.590464-1-mattias.ronnblom@ericsson.com> References: <20240505073313.118515-1-mattias.ronnblom@ericsson.com> <20240809201440.590464-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM4PEPF00027A6C:EE_|DBAPR07MB6583:EE_ X-MS-Office365-Filtering-Correlation-Id: 89e920d8-050b-4bdd-eaf2-08dcb8b13d9d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|36860700013|1800799024|376014|82310400026; X-Microsoft-Antispam-Message-Info: =?utf-8?q?2+rXoIcJzQF17/RlhQlnFn/an+TBews?= =?utf-8?q?1bRijVbZtO+kp1pBRN8a3Me0t8Im/1HmKbecFImk5GrffqcVFf564hpFlflpPHdD1?= =?utf-8?q?/69s978YKYyzz3YfO9ws4uIntBzT+2QXAoVnH/DhaqwvgRVWVpD4pfiqnW7O3wy2t?= =?utf-8?q?fvhnuf9UpNiFIrYjP0ATjYuhvmB4QIAPPfAfA5QQzyR91BelojWHQ7FrcywPhUPMs?= =?utf-8?q?Cm6qOiStg0Okr2286McntIYizeiQcvJfoAKqPhIGgeQ5oUXvevHhFcLt5rWl3bSXw?= =?utf-8?q?eGachUATOJO/AbkcnTA+c2R7QILgSyUh0aCy5LHYf+4MeRs5d6PcKTw/WhkU/wpiG?= =?utf-8?q?M2NYNE+2WdindbkFz1YiEIKRzYHP7y8g2cbioCg9TYkYtl/sq/8NQsW5inAR5zoN/?= =?utf-8?q?14muqBvoeDWaZcQok4eceIRHEQCIQaBsa7geyr/odfQw76NauktJkp29za1kXPdjE?= =?utf-8?q?PJZudK6QAkhTyQPQuFR4aXvk7hu4CR7g2pzn1tjaBl+Azh9EDBhCk42lsWWTbocvR?= =?utf-8?q?4uEAcn636fJSA3ytaku+DO2Bt8U5rZBizCEkqnd1KKKAJDeG9FSPON/cHBJq/mp6B?= =?utf-8?q?AbhEZyfIkNqVkhZSb6QqsfkMJSt00jXC0Yy9giLENqc4rxvWOzfRRuNd8CTIi/fAv?= =?utf-8?q?PxUSK4mDrQQUoJ7vXrwSkusw2rlgi3LqSIhy26ySEwHL1JagOWQHjCsoqN8silIIF?= =?utf-8?q?Z6MbdmWS7msRphthcyGuBH0B9UO6nvY343ZHuqVRiWhgZ/Gefwp3VwV3LZagg4QlG?= =?utf-8?q?EZLIfkYqT85SEl57h3WacQd8ZDE67dOVtxZ1Do2KVDdeQc1wvPDlOGtgxUbww0Ckj?= =?utf-8?q?TL45uF1Oibp4WLTuRI+UtHvfF98zUlxG4R6jFyFue26lH31SF9ZsUUDKvh0ZsU6ie?= =?utf-8?q?S5p4s5IZQVDYS3oOxEs6CTrnmSSwBQ7RwVObfomNMY/saemixschkPI4R5RO0+rOu?= =?utf-8?q?WMAIGay0/5QztxzAkyOMaCj67sqyI95AP7ZArdakaElorXtm3LovwkGtmyOt9lnZ9?= =?utf-8?q?Cf4nWZK1DuoLAmJkiHOlDSeNS0wPXpwt0KP6KGoWhORu9JL1NJyGDFjLy1N39fuLy?= =?utf-8?q?Z1hoBuWRLDAwhLsxJU7wNVwFBghqVpomJBNMlGzT/r+m9r9tdpQlw2zG6yu2pKWIt?= =?utf-8?q?TkDBh+vINe3v9Y4N94WPQkx8TfGUkq5qJR/tEx+5XirCv2VhH5aRB+kgntFmawfFD?= =?utf-8?q?LjDcT7ewQWN2DoqIRyvjpK8wJ2EEoSTVcamH9fY2obLj3bd3YfRJ8B1Ui8KQFhKuZ?= =?utf-8?q?cu9xLkjWrcR52tze0g2i0V3uHZSri5fXllC+FZBOJEAVqGMYaGL7T7i7IYlZwdDqM?= =?utf-8?q?/8/BMMd7/afVQzzAcJv5lu901RbpHaOA0w=3D=3D?= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230040)(36860700013)(1800799024)(376014)(82310400026); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Aug 2024 20:24:16.1780 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 89e920d8-050b-4bdd-eaf2-08dcb8b13d9d X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: AM4PEPF00027A6C.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR07MB6583 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Use a multi-word bitset to track which services are mapped to which lcores, allowing the RTE_SERVICE_NUM_MAX compile-time constant to be > 64. Replace array-of-bytes service-currently-active flags with a more compact multi-word bitset-based representation, reducing memory footprint somewhat. Signed-off-by: Mattias Rönnblom Acked-by: Tyler Retzlaff --- lib/eal/common/rte_service.c | 70 ++++++++++++++---------------------- 1 file changed, 27 insertions(+), 43 deletions(-) diff --git a/lib/eal/common/rte_service.c b/lib/eal/common/rte_service.c index 56379930b6..ec0f47e141 100644 --- a/lib/eal/common/rte_service.c +++ b/lib/eal/common/rte_service.c @@ -11,6 +11,7 @@ #include #include +#include #include #include #include @@ -63,11 +64,11 @@ struct service_stats { /* the internal values of a service core */ struct __rte_cache_aligned core_state { /* map of services IDs are run on this core */ - uint64_t service_mask; + RTE_BITSET_DECLARE(mapped_services, RTE_SERVICE_NUM_MAX); RTE_ATOMIC(uint8_t) runstate; /* running or stopped */ RTE_ATOMIC(uint8_t) thread_active; /* indicates when thread is in service_run() */ uint8_t is_service_core; /* set if core is currently a service core */ - uint8_t service_active_on_lcore[RTE_SERVICE_NUM_MAX]; + RTE_BITSET_DECLARE(service_active_on_lcore, RTE_SERVICE_NUM_MAX); RTE_ATOMIC(uint64_t) loops; RTE_ATOMIC(uint64_t) cycles; struct service_stats service_stats[RTE_SERVICE_NUM_MAX]; @@ -81,11 +82,6 @@ static uint32_t rte_service_library_initialized; int32_t rte_service_init(void) { - /* Hard limit due to the use of an uint64_t-based bitmask (and the - * clzl intrinsic). - */ - RTE_BUILD_BUG_ON(RTE_SERVICE_NUM_MAX > 64); - if (rte_service_library_initialized) { EAL_LOG(NOTICE, "service library init() called, init flag %d", @@ -296,7 +292,7 @@ rte_service_component_unregister(uint32_t id) /* clear the run-bit in all cores */ for (i = 0; i < RTE_MAX_LCORE; i++) - lcore_states[i].service_mask &= ~(UINT64_C(1) << id); + rte_bitset_clear(lcore_states[i].mapped_services, id); memset(&rte_services[id], 0, sizeof(struct rte_service_spec_impl)); @@ -410,7 +406,7 @@ service_runner_do_callback(struct rte_service_spec_impl *s, /* Expects the service 's' is valid. */ static int32_t -service_run(uint32_t i, struct core_state *cs, uint64_t service_mask, +service_run(uint32_t i, struct core_state *cs, const uint64_t *mapped_services, struct rte_service_spec_impl *s, uint32_t serialize_mt_unsafe) { if (!s) @@ -424,12 +420,12 @@ service_run(uint32_t i, struct core_state *cs, uint64_t service_mask, RUNSTATE_RUNNING || rte_atomic_load_explicit(&s->app_runstate, rte_memory_order_acquire) != RUNSTATE_RUNNING || - !(service_mask & (UINT64_C(1) << i))) { - cs->service_active_on_lcore[i] = 0; + !rte_bitset_test(mapped_services, i)) { + rte_bitset_clear(cs->service_active_on_lcore, i); return -ENOEXEC; } - cs->service_active_on_lcore[i] = 1; + rte_bitset_set(cs->service_active_on_lcore, i); if ((service_mt_safe(s) == 0) && (serialize_mt_unsafe == 1)) { if (!rte_spinlock_trylock(&s->execute_lock)) @@ -454,7 +450,7 @@ rte_service_may_be_active(uint32_t id) return -EINVAL; for (i = 0; i < lcore_count; i++) { - if (lcore_states[ids[i]].service_active_on_lcore[id]) + if (rte_bitset_test(lcore_states[ids[i]].service_active_on_lcore, id)) return 1; } @@ -474,7 +470,9 @@ rte_service_run_iter_on_app_lcore(uint32_t id, uint32_t serialize_mt_unsafe) */ rte_atomic_fetch_add_explicit(&s->num_mapped_cores, 1, rte_memory_order_relaxed); - int ret = service_run(id, cs, UINT64_MAX, s, serialize_mt_unsafe); + RTE_BITSET_DECLARE(all_services, RTE_SERVICE_NUM_MAX); + rte_bitset_set_all(all_services, RTE_SERVICE_NUM_MAX); + int ret = service_run(id, cs, all_services, s, serialize_mt_unsafe); rte_atomic_fetch_sub_explicit(&s->num_mapped_cores, 1, rte_memory_order_relaxed); @@ -485,7 +483,6 @@ static int32_t service_runner_func(void *arg) { RTE_SET_USED(arg); - uint8_t i; const int lcore = rte_lcore_id(); struct core_state *cs = &lcore_states[lcore]; @@ -497,20 +494,11 @@ service_runner_func(void *arg) */ while (rte_atomic_load_explicit(&cs->runstate, rte_memory_order_acquire) == RUNSTATE_RUNNING) { + ssize_t id; - const uint64_t service_mask = cs->service_mask; - uint8_t start_id; - uint8_t end_id; - - if (service_mask == 0) - continue; - - start_id = rte_ctz64(service_mask); - end_id = 64 - rte_clz64(service_mask); - - for (i = start_id; i < end_id; i++) { + RTE_BITSET_FOREACH_SET(id, cs->mapped_services, RTE_SERVICE_NUM_MAX) { /* return value ignored as no change to code flow */ - service_run(i, cs, service_mask, service_get(i), 1); + service_run(id, cs, cs->mapped_services, service_get(id), 1); } rte_atomic_store_explicit(&cs->loops, cs->loops + 1, rte_memory_order_relaxed); @@ -519,8 +507,7 @@ service_runner_func(void *arg) /* Switch off this core for all services, to ensure that future * calls to may_be_active() know this core is switched off. */ - for (i = 0; i < RTE_SERVICE_NUM_MAX; i++) - cs->service_active_on_lcore[i] = 0; + rte_bitset_clear_all(cs->service_active_on_lcore, RTE_SERVICE_NUM_MAX); /* Use SEQ CST memory ordering to avoid any re-ordering around * this store, ensuring that once this store is visible, the service @@ -586,7 +573,7 @@ rte_service_lcore_count_services(uint32_t lcore) if (!cs->is_service_core) return -ENOTSUP; - return rte_popcount64(cs->service_mask); + return rte_bitset_count_set(cs->mapped_services, RTE_SERVICE_NUM_MAX); } int32_t @@ -639,25 +626,23 @@ service_update(uint32_t sid, uint32_t lcore, uint32_t *set, uint32_t *enabled) !lcore_states[lcore].is_service_core) return -EINVAL; - uint64_t sid_mask = UINT64_C(1) << sid; if (set) { - uint64_t lcore_mapped = lcore_states[lcore].service_mask & - sid_mask; + uint64_t lcore_mapped = rte_bitset_test(lcore_states[lcore].mapped_services, sid); if (*set && !lcore_mapped) { - lcore_states[lcore].service_mask |= sid_mask; + rte_bitset_set(lcore_states[lcore].mapped_services, sid); rte_atomic_fetch_add_explicit(&rte_services[sid].num_mapped_cores, 1, rte_memory_order_relaxed); } if (!*set && lcore_mapped) { - lcore_states[lcore].service_mask &= ~(sid_mask); + rte_bitset_clear(lcore_states[lcore].mapped_services, sid); rte_atomic_fetch_sub_explicit(&rte_services[sid].num_mapped_cores, 1, rte_memory_order_relaxed); } } if (enabled) - *enabled = !!(lcore_states[lcore].service_mask & (sid_mask)); + *enabled = rte_bitset_test(lcore_states[lcore].mapped_services, sid); return 0; } @@ -699,11 +684,11 @@ set_lcore_state(uint32_t lcore, int32_t state) int32_t rte_service_lcore_reset_all(void) { - /* loop over cores, reset all to mask 0 */ + /* loop over cores, reset all mapped services */ uint32_t i; for (i = 0; i < RTE_MAX_LCORE; i++) { if (lcore_states[i].is_service_core) { - lcore_states[i].service_mask = 0; + rte_bitset_clear_all(lcore_states[i].mapped_services, RTE_SERVICE_NUM_MAX); set_lcore_state(i, ROLE_RTE); /* runstate act as guard variable Use * store-release memory order here to synchronize @@ -731,7 +716,7 @@ rte_service_lcore_add(uint32_t lcore) set_lcore_state(lcore, ROLE_SERVICE); /* ensure that after adding a core the mask and state are defaults */ - lcore_states[lcore].service_mask = 0; + rte_bitset_clear_all(lcore_states[lcore].mapped_services, RTE_SERVICE_NUM_MAX); /* Use store-release memory order here to synchronize with * load-acquire in runstate read functions. */ @@ -814,12 +799,11 @@ rte_service_lcore_stop(uint32_t lcore) uint32_t i; struct core_state *cs = &lcore_states[lcore]; - uint64_t service_mask = cs->service_mask; for (i = 0; i < RTE_SERVICE_NUM_MAX; i++) { - int32_t enabled = service_mask & (UINT64_C(1) << i); - int32_t service_running = rte_service_runstate_get(i); - int32_t only_core = (1 == + bool enabled = rte_bitset_test(cs->mapped_services, i); + bool service_running = rte_service_runstate_get(i); + bool only_core = (1 == rte_atomic_load_explicit(&rte_services[i].num_mapped_cores, rte_memory_order_relaxed));