From patchwork Wed Oct 5 09:16:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 117371 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3DB2CA0542; Wed, 5 Oct 2022 11:20:25 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7BFF9427F3; Wed, 5 Oct 2022 11:20:10 +0200 (CEST) Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2068.outbound.protection.outlook.com [40.107.21.68]) by mails.dpdk.org (Postfix) with ESMTP id 271D840E2D for ; Wed, 5 Oct 2022 11:20:08 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=oFa+e28QgJXafwXdiAg6hjS++Ju66lYkrVUc6L90ad/ZJNxDhtrNoWftKhBVAMCHKJqkOU4xswhH9aeKh2HMrUJMpPVmiKDurB4Fw8ya4pAarty6hCZKwrA9poNMOhRl7KB4Ihu3kvw7YKKqBmWH7g2Dmvw1/8A3pFnEIwI8wHxOS9p62JpHz0IEPdxEPFJGjzsFdXOCpU/ZNY9ptGBed+54qUok9ZKTQwuFjM1xNAZW2K6gT3ppurKCX69eZnPM1ImRsnFlTP+GHalbutQLd+pwexIk0dyuOUXta3O8f4QOWx79MHmB+JCIre6rVpuLB8VNbQa8gwj5R1fCeVCbUw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=58eC5StAEVExrYz9sZYsgCwhm28+XmKmgk6yaCVnjDo=; b=Oq4OgH5qS17clGgbtnRnZLzHgr5YYB3iq/AO8OxpOkTef2YrFpEGq8pWjZ9WfSZDzNac58fC35ujtDIQkSgIeNGY7LB9pv1XoaP0nypaYpD4lZjawIT+WadRrBnCXJ9xUjFYYW9CMQM1zlpLq4xzdc1l5L5pUTYlidlqFx0juWXfaizrg5kXUTe0bap9dRiKqk0OYKgETPJL4oWLhKLSXFXtHKT6hioaO8nqE6R33StAQDjr7C+3GfvnmE3ZLUVKHGXqfvEpEzl5N5+IN/4feRqEkwL2vXHksYxTTLJjRxDXH+ZKz64ysOmu/JEYRum5gk9iW2fs/OFtKDM0XYNScQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=arm.com smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=58eC5StAEVExrYz9sZYsgCwhm28+XmKmgk6yaCVnjDo=; b=AaEnc0E3IyG+8c2VWotvvnGXQRaMgT5/U12CQ8mjr0tK47xMeZwfuTbL6hM8z/LPI8nnVdcsRNc8jETNMR2IMwGT9orY2m3ZUbAIEbue1QB1f7srFezNG6l/QB6IS6rAu/kTBICAWgWRZ0trKcDcrjzTIo0xSMXP4f5KIErDz4c= Received: from AM6P192CA0066.EURP192.PROD.OUTLOOK.COM (2603:10a6:209:82::43) by DB8PR07MB6283.eurprd07.prod.outlook.com (2603:10a6:10:140::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5709.9; Wed, 5 Oct 2022 09:20:07 +0000 Received: from AM5EUR02FT049.eop-EUR02.prod.protection.outlook.com (2603:10a6:209:82:cafe::e0) by AM6P192CA0066.outlook.office365.com (2603:10a6:209:82::43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.32 via Frontend Transport; Wed, 5 Oct 2022 09:20:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by AM5EUR02FT049.mail.protection.outlook.com (10.152.9.233) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.5676.17 via Frontend Transport; Wed, 5 Oct 2022 09:20:06 +0000 Received: from ESESSMB502.ericsson.se (153.88.183.163) by ESESSMR504.ericsson.se (153.88.183.126) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2375.31; Wed, 5 Oct 2022 11:20:04 +0200 Received: from seliicinfr00050.seli.gic.ericsson.se (153.88.183.153) by smtp.internal.ericsson.com (153.88.183.190) with Microsoft SMTP Server id 15.1.2375.31 via Frontend Transport; Wed, 5 Oct 2022 11:20:03 +0200 Received: from localhost.localdomain (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00050.seli.gic.ericsson.se (Postfix) with ESMTP id 1E8FB1C0069; Wed, 5 Oct 2022 11:20:04 +0200 (CEST) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: Van@dpdk.org, Haaren@dpdk.org, Harry CC: , Honnappa Nagarahalli , =?utf-8?q?Morten_Br=C3=B8rup?= , nd , , =?utf-8?q?Mattias_R=C3=B6nnblom?= Subject: [PATCH v2 3/6] service: reduce average case service core overhead Date: Wed, 5 Oct 2022 11:16:12 +0200 Message-ID: <20221005091615.94652-4-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221005091615.94652-1-mattias.ronnblom@ericsson.com> References: <20220906161352.296110-1-mattias.ronnblom@ericsson.com> <20221005091615.94652-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM5EUR02FT049:EE_|DB8PR07MB6283:EE_ X-MS-Office365-Filtering-Correlation-Id: de657893-2cc4-43e2-5c78-08daa6b2cb1a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zQAlUAgNdPpSca/HxRyzRt18SxNdXZ17CKjZmuRtWRiB/Z5eHw35EJdC2l0KnLE7ZnP7JKfHeFbTrlbqjx/ijRaUTFCC9hvzAoTzahHSwpmVahg5gbxIYNmKr/GeOSq4qTBmIN5QpdSOPRar3D4fNlkhZy8DHQ9/57JTgr6TBQZISGi7g5xgt+UM42g09BBAAg5jitsvJJpOBgQ2tR+wlKNOFL9UbSQeXWP00ae0SmTyTNcDW9g/2PbsOOU+CKiHoMmFu5Rp0q5Ldvry2iW/KVX+vOkL4iYInaQM2ml8mSm3kTevmM0qD9kfsmusDhcKqJ1o/aeTfOOxl+hsuka6f7IJN7qndF0FHjsMZT+5ybzzqsfDW2okWNDO9JGwEsVtMz8WaUxomDPfFVHNeLP1m0k+imNX2s0pV6p4n0vDlX+x/OlpZfTJ6a1SJWv1Qq6AJUmHDjXOM+Myvnh3CayuD5y9AFlWVw4JfEa9heJwgDuhbS/aEeN77KAsjN4BntoKuUGy7ueyuURW/eNAiquUm6ch8BrsTW0UZZax6RSN5Q08beOs4E5XpbzaQpF79zGS3PK3gePqL99ApOPMdYhTaSPX1ojiwUPyvUMGRnA6p4dyD1SbPeHhlZY2oG5Mfv+gI16ag4b15HdDyC3g3qr6iqgrYHhe89vl81Ptv9hDOJqrnwt9QIZpcTZtvghf/HpH4ta4HQ8ixvYJVdNxFruaDZt0s9VO0S6dxOBM2WMuzgOm+ZGRO/T5eAaXwBtPMdRKkMQDIay/aCnQovmpjbzTGQ== X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230022)(4636009)(136003)(346002)(39860400002)(396003)(376002)(451199015)(36840700001)(40470700004)(46966006)(82310400005)(478600001)(6266002)(2616005)(70206006)(186003)(36756003)(70586007)(2906002)(8936002)(5660300002)(66574015)(47076005)(336012)(4326008)(8676002)(6666004)(26005)(316002)(6916009)(83380400001)(40480700001)(86362001)(107886003)(1076003)(36860700001)(40460700003)(356005)(54906003)(82960400001)(41300700001)(7636003)(82740400003); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Oct 2022 09:20:06.7613 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: de657893-2cc4-43e2-5c78-08daa6b2cb1a X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: AM5EUR02FT049.eop-EUR02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR07MB6283 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Optimize service loop so that the starting point is the lowest-indexed service mapped to the lcore in question, and terminate the loop at the highest-indexed service. While the worst case latency remains the same, this patch significantly reduces the service framework overhead for the average case. In particular, scenarios where an lcore only runs a single service, or multiple services which id values are close (e.g., three services with ids 17, 18 and 22), show significant improvements. The worse case is a where the lcore two services mapped to it; one with service id 0 and the other with id 63. On a service lcore serving a single service, the service loop overhead is reduced from ~190 core clock cycles to ~46, on an Intel Cascade Lake generation Xeon. On weakly ordered CPUs, the gain is larger, since the loop included load-acquire atomic operations. Signed-off-by: Mattias Rönnblom --- v2: Added build-time assertion to prevent the maximum number of services to accidentally be changed to a higher value than the implementation supports. (Harry van Haaren) --- lib/eal/common/rte_service.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/lib/eal/common/rte_service.c b/lib/eal/common/rte_service.c index 4d51de638d..035c36b8bb 100644 --- a/lib/eal/common/rte_service.c +++ b/lib/eal/common/rte_service.c @@ -78,6 +78,11 @@ static uint32_t rte_service_library_initialized; int32_t rte_service_init(void) { + /* Hard limit due to the use of an uint64_t-based bitmask (and the + * clzl intrinsic). + */ + RTE_BUILD_BUG_ON(RTE_SERVICE_NUM_MAX > 64); + if (rte_service_library_initialized) { RTE_LOG(NOTICE, EAL, "service library init() called, init flag %d\n", @@ -472,7 +477,6 @@ static int32_t service_runner_func(void *arg) { RTE_SET_USED(arg); - uint32_t i; const int lcore = rte_lcore_id(); struct core_state *cs = &lcore_states[lcore]; @@ -486,10 +490,17 @@ service_runner_func(void *arg) RUNSTATE_RUNNING) { const uint64_t service_mask = cs->service_mask; + uint8_t start_id; + uint8_t end_id; + uint8_t i; - for (i = 0; i < RTE_SERVICE_NUM_MAX; i++) { - if (!service_registered(i)) - continue; + if (service_mask == 0) + continue; + + start_id = __builtin_ctzl(service_mask); + end_id = 64 - __builtin_clzl(service_mask); + + for (i = start_id; i < end_id; i++) { /* return value ignored as no change to code flow */ service_run(i, cs, service_mask, service_get(i), 1); }