From patchwork Wed Oct 5 09:16:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Mattias_R=C3=B6nnblom?= X-Patchwork-Id: 117369 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2093CA0542; Wed, 5 Oct 2022 11:20:12 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5F21642670; Wed, 5 Oct 2022 11:20:08 +0200 (CEST) Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2076.outbound.protection.outlook.com [40.107.21.76]) by mails.dpdk.org (Postfix) with ESMTP id D340A40694 for ; Wed, 5 Oct 2022 11:20:06 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=N6ha+29ivm1E+0xRBPcgnp/b0qb7/QVsZMa3/k/iUOuiH/rQYJr8vbzWaGsremPN663WrXocZHjPk2tSTpidrEO2ERCtsHGdSq0zJXlpeCvNqtJD/WTzomeGsUwD6c6TUcUvUhL+kqH3h62sDbLGv1IRuKyH9DyZCSgAaAXTYvJx7qqsyxuNGwvorbVI01R/9DaSDdHgnIU4b62A9EG/oE1mQr9wrKW2ncUmz+WtCFq4IIhuveQ1/tyuA7cWA5svb2KfA7p8iq4fzqoG4G9FcIQzchQFrBtbSOAhS+kWJ5j0I0bsaVRarrpLQagn3d3+oA2yxobuKrUenTxauqK8lA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=RnQBfxfiG4j25vKsnQz4g87/cQnd0+c6alEQTabV+5Q=; b=Yv2VZMSnCAQyedZ4aqPVfcdt+AHlUkU/jce4MeL+HMb98MX3AiE1hFzCUCKvoO0M3EW69HmKyLDBUMeqM3xB3AnxmlczsaYcO8Bd+SK+9PCVkTA9niOCf88tpRAJAvh6rJGqWAG84/71JICBR8oozoXSwLl2KGRq0UdLTBlZ31L0RgleLwhFCAuhQ7nAr33fhh0i9sQfBBysLbl2UkoZVFM2sixrVZ6OIjUE7QWNwYGDPAcsyKLJaEXjLwD0WhxHgl6/NcvWyORLT0nEnO5cRVViI98DWDXhratjOEYvdmQ3nTgJBXfaYMHMFwAEJE7TL9DygYaM7Z6xTo10x5IAVg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=arm.com smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RnQBfxfiG4j25vKsnQz4g87/cQnd0+c6alEQTabV+5Q=; b=X2SmPZDxCWMdQD+A0AihwOsxfxbEHsoBIHw9SLRGnT7bpsP6QZ7W8W/NtxvCLYLGt1UcX8m8CDFp06R5xSrWpUBx8uiahyakQRSjAk4ts27WEXGT+dxaOpMbHpreEMS3GdcrtHLZ/r4yGjx9Ti79wjrDYcK8HJdadw0gpcfs1JM= Received: from AS9PR06CA0353.eurprd06.prod.outlook.com (2603:10a6:20b:466::30) by PR3PR07MB6875.eurprd07.prod.outlook.com (2603:10a6:102:7e::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5709.10; Wed, 5 Oct 2022 09:20:05 +0000 Received: from AM5EUR02FT099.eop-EUR02.prod.protection.outlook.com (2603:10a6:20b:466:cafe::5) by AS9PR06CA0353.outlook.office365.com (2603:10a6:20b:466::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.24 via Frontend Transport; Wed, 5 Oct 2022 09:20:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by AM5EUR02FT099.mail.protection.outlook.com (10.152.9.29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.5709.10 via Frontend Transport; Wed, 5 Oct 2022 09:20:05 +0000 Received: from ESESBMB503.ericsson.se (153.88.183.170) by ESESBMR503.ericsson.se (153.88.183.135) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2375.31; Wed, 5 Oct 2022 11:20:05 +0200 Received: from ESESSMB502.ericsson.se (153.88.183.163) by ESESBMB503.ericsson.se (153.88.183.170) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2375.31; Wed, 5 Oct 2022 11:20:04 +0200 Received: from seliicinfr00050.seli.gic.ericsson.se (153.88.183.153) by smtp.internal.ericsson.com (153.88.183.190) with Microsoft SMTP Server id 15.1.2375.31 via Frontend Transport; Wed, 5 Oct 2022 11:20:04 +0200 Received: from localhost.localdomain (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00050.seli.gic.ericsson.se (Postfix) with ESMTP id CA9C41C0069; Wed, 5 Oct 2022 11:20:04 +0200 (CEST) From: =?utf-8?q?Mattias_R=C3=B6nnblom?= To: Van@dpdk.org, Haaren@dpdk.org, Harry CC: , Honnappa Nagarahalli , =?utf-8?q?Morten_Br=C3=B8rup?= , nd , , =?utf-8?q?Mattias_R=C3=B6nnblom?= Subject: [PATCH v2 4/6] service: tweak cycle statistics semantics Date: Wed, 5 Oct 2022 11:16:13 +0200 Message-ID: <20221005091615.94652-5-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221005091615.94652-1-mattias.ronnblom@ericsson.com> References: <20220906161352.296110-1-mattias.ronnblom@ericsson.com> <20221005091615.94652-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM5EUR02FT099:EE_|PR3PR07MB6875:EE_ X-MS-Office365-Filtering-Correlation-Id: a8a5f44a-1e00-4551-272f-08daa6b2ca60 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: BRqHfXUdZnuGAr03wbYFMng5bQA7Co5VjtlJZq7A+YokO7Z+l8H55bPXG7hbJ0oPiQyP3JtBP26bzHToV7Uv5fmfo9Z1ReONLRX9bbpX+Qpnew/3bXd07NmNBldlVaRuvj24eZ9Tby+jYOHFMCThyxrLsaX48KCjDw84g32t3StYGi/LKwNG8Yf7J4odUmuiWkDo8+BZcGHvnJP3Dqcll8y/2R+hQS7HpOag3P0VbE2epeR3vwyqkGL5jHrcbzpNfHS8PmHTwZqeBoUA9obi8wktcuAo6uPiRSe0wTr2rxlfRpIbTMogFGqmihDyLutwtB7Unox038tXRv0zFSV/ehpCYkgGjXRjMLHRN5N7l1FfJs08k+TZ+qeiHtD6heN5Tu3PZ5mxgX0eAmzQElEXR+u6wshoJWomAlu/7ChiYT5C2m2Ffe1RZeJHIVmClv7Vh7LgmK+lNVGwukcj7f9UNi4kYq5Am4Nlrftw1n+kD8Om2yj85fnXuTSP9Ml696Uh6EASRixjHZ9AhZmJjY6KymVoAau+rINsOvSn2wz7QkflrjAita+tAqXeAD2HkOR9gy2y0MiW8EE8mgznBacFy0YAvh8CwvPS21XdKfFBIdJBjXuBDkreGeDgdY20IXOELxqnJJCW2zG4jXhVwW4252zJ60C0QExbFqLaIeM5T6srtmEgOO0MLlFkX9hDr1nNPB0JHIuKddSQNukD4rtQ0nsNZTF7jSkrXTrGdNhGoUREFFhEoMPkxPboczVKUzSV5FdYVx2j9ObrMpyqR/6UJg== X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230022)(4636009)(346002)(376002)(136003)(39860400002)(396003)(451199015)(40470700004)(36840700001)(46966006)(6666004)(107886003)(54906003)(6916009)(478600001)(86362001)(82310400005)(41300700001)(36756003)(8936002)(5660300002)(316002)(70586007)(8676002)(4326008)(70206006)(36860700001)(83380400001)(82740400003)(82960400001)(40480700001)(7636003)(356005)(40460700003)(6266002)(26005)(186003)(1076003)(2616005)(66574015)(336012)(47076005)(2906002); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Oct 2022 09:20:05.5566 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a8a5f44a-1e00-4551-272f-08daa6b2ca60 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: AM5EUR02FT099.eop-EUR02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR07MB6875 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org As a part of its service function, a service usually polls some kind of source (e.g., an RX queue, a ring, an eventdev port, or a timer wheel) to retrieve one or more items of work. In low-load situations, the service framework reports a significant amount of cycles spent for all running services, despite the fact they have performed little or no actual work. The per-call cycle expenditure for an idle service (i.e., a service currently without pending jobs) is typically very low. Polling an empty ring or RX queue is inexpensive. However, since the service function call frequency on an idle or lightly loaded lcore is going to be very high indeed, the service function calls' cycles adds up to a significant amount. The only thing preventing the idle services' cycles counters to make up 100% of the available CPU cycles is the overhead of the service framework itself. If the RTE_SERVICE_ATTR_CYCLES or RTE_SERVICE_LCORE_ATTR_CYCLES are used to estimate service core load, the cores may look very busy when the system is mostly doing nothing useful at all. This patch allows for an idle service to indicate that no actual work was performed during a particular service function call (by returning -EAGAIN). In such cases the RTE_SERVICE_ATTR_CYCLES and RTE_SERVICE_LCORE_ATTR_CYCLES values are not incremented. The convention of returning -EAGAIN for idle services may in the future also be used to have the lcore enter a short sleep, or reduce its operating frequency, in case all services are currently idle. This change is backward-compatible. Signed-off-by: Mattias Rönnblom --- lib/eal/common/rte_service.c | 26 +++++++++++++------------ lib/eal/include/rte_service_component.h | 5 +++++ 2 files changed, 19 insertions(+), 12 deletions(-) diff --git a/lib/eal/common/rte_service.c b/lib/eal/common/rte_service.c index 035c36b8bb..718371814f 100644 --- a/lib/eal/common/rte_service.c +++ b/lib/eal/common/rte_service.c @@ -10,6 +10,7 @@ #include #include +#include #include #include #include @@ -373,28 +374,29 @@ service_runner_do_callback(struct rte_service_spec_impl *s, if (service_stats_enabled(s)) { uint64_t start = rte_rdtsc(); - s->spec.callback(userdata); - uint64_t end = rte_rdtsc(); - uint64_t cycles = end - start; + int rc = s->spec.callback(userdata); /* The lcore service worker thread is the only writer, * and thus only a non-atomic load and an atomic store * is needed, and not the more expensive atomic * add. */ - __atomic_store_n(&cs->cycles, cs->cycles + cycles, - __ATOMIC_RELAXED); - struct service_stats *service_stats = &cs->service_stats[service_idx]; - __atomic_store_n(&service_stats->calls, - service_stats->calls + 1, - __ATOMIC_RELAXED); + if (likely(rc != -EAGAIN)) { + uint64_t end = rte_rdtsc(); + uint64_t cycles = end - start; - __atomic_store_n(&service_stats->cycles, - service_stats->cycles + cycles, - __ATOMIC_RELAXED); + __atomic_store_n(&cs->cycles, cs->cycles + cycles, + __ATOMIC_RELAXED); + __atomic_store_n(&service_stats->cycles, + service_stats->cycles + cycles, + __ATOMIC_RELAXED); + } + + __atomic_store_n(&service_stats->calls, + service_stats->calls + 1, __ATOMIC_RELAXED); } else s->spec.callback(userdata); } diff --git a/lib/eal/include/rte_service_component.h b/lib/eal/include/rte_service_component.h index 9e66ee7e29..9be49d698a 100644 --- a/lib/eal/include/rte_service_component.h +++ b/lib/eal/include/rte_service_component.h @@ -19,6 +19,11 @@ extern "C" { /** * Signature of callback function to run a service. + * + * A service function call resulting in no actual work being + * performed, should return -EAGAIN. In that case, the (presumbly few) + * cycles spent will not be counted toward the lcore or service-level + * cycles attributes. */ typedef int32_t (*rte_service_func)(void *args);