Message ID | 20240910083117.4281-1-pbhagavatula@marvell.com (mailing list archive) |
---|---|
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5FFAE45954; Tue, 10 Sep 2024 10:31:34 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E3D504021F; Tue, 10 Sep 2024 10:31:33 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 9516440151 for <dev@dpdk.org>; Tue, 10 Sep 2024 10:31:31 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 48A4fnSO029408; Tue, 10 Sep 2024 01:31:30 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:message-id :mime-version:subject:to; s=pfpt0220; bh=N/RisD4FpLu19xJ8AMrqTZf O9GI6f7WOvr40EVB0HCs=; b=WjTJa/xif6xL2H7Jr/FOkNX9cOwSnmmU4bYxNli 2xtbgWRaoCJGOI5EOvOmO00w6TmXPpyonQcoCbVNhrcCkAmpZioXb7gQjarw3zzG 6gjYB1FTBDPpC1BE+Q1/0z6+JCzRwBAsY92J+GDWQ9F4mqnpin1Z2K0iARgoYZM+ 8XXFLiANiOTH/q4DgwnZmkCWsVUYKglNbuLrN9JYUcXpr8iTINLtRrn3xyNZNkIM P74L8GN4uGC4fkU6YVBwLMo/WKusPSFoFerlm5tmUDohW+9nemCTsq+5zheKt9pb DaZxH3fFTXHVLITWYW3T9YkEA74r+EpjiYQoQfnRN3PVO0w== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 41gygtfxk0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 10 Sep 2024 01:31:30 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 10 Sep 2024 01:31:29 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Tue, 10 Sep 2024 01:31:29 -0700 Received: from MININT-80QBFE8.corp.innovium.com (MININT-80QBFE8.marvell.com [10.28.164.106]) by maili.marvell.com (Postfix) with ESMTP id EE6F13F709E; Tue, 10 Sep 2024 01:31:25 -0700 (PDT) From: <pbhagavatula@marvell.com> To: <jerinj@marvell.com>, <sthotton@marvell.com>, <abdullah.sevincer@intel.com>, <hemant.agrawal@nxp.com>, <sachin.saxena@oss.nxp.com>, <harry.van.haaren@intel.com>, <mattias.ronnblom@ericsson.com>, <liangma@liangbit.com>, <peter.mccarthy@intel.com> CC: <dev@dpdk.org>, Pavan Nikhilesh <pbhagavatula@marvell.com> Subject: [RFC 0/3] Introduce event prefetching Date: Tue, 10 Sep 2024 14:01:14 +0530 Message-ID: <20240910083117.4281-1-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-ORIG-GUID: KDXzXOia7sHiBDnfDl_NhRlG_fo-O5Cp X-Proofpoint-GUID: KDXzXOia7sHiBDnfDl_NhRlG_fo-O5Cp X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-06_09,2024-09-06_01,2024-09-02_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org |
Series |
Introduce event prefetching
|
|
Message
Pavan Nikhilesh Bhagavatula
Sept. 10, 2024, 8:31 a.m. UTC
From: Pavan Nikhilesh <pbhagavatula@marvell.com>
Event prefetching improves scheduling performance by pre-scheduling events to
event ports when dequeues are issued. This series introduces various types and
levels of prefetching to the eventdev library.
Prefetching Types:
* RTE_EVENT_DEV_PREFETCH_NONE: No prefetching.
* RTE_EVENT_DEV_PREFETCH: Always issue a prefetch when dequeue is issued.
* RTE_EVENT_DEV_PREFETCH_INTELLIGENT: Delay issuing prefetch until there
are no forward progress constraints with the held flow contexts.
Prefetching Levels:
* Event Device Level Prefetch: Prefetching can be enabled or disabled at the
event device during configuration. Event devices can indicate prefetching
capabilities using `RTE_EVENT_DEV_CAP_EVENT_PREFETCH` and
`RTE_EVENT_DEV_CAP_EVENT_INTELLIGENT_PREFETCH` via the event device info
function `info.event_dev_cap`.
* Event Port Level Prefetch: Prefetching can be selectively enabled or disabled
at the event port during runtime. Event devices can indicate this capability
using `RTE_EVENT_PORT_CAP_EVENT_PER_PORT_PREFETCH` via the event device info
function `info.event_port_cap`.
* Application Controlled Prefetch Hint: Applications can provide hints to the
event device to start prefetching events using the new API
`rte_event_port_prefetch`. Event devices can indicate this capabilities using
`RTE_EVENT_DEV_CAP_SW_PREFETCH` via the event device info function
`info.event_dev_cap`.
The rte_event_dequeue_burst operation initiates the prefetch operation, which
completes in parallel without affecting the dequeued event flow context and
dequeue latency. On the next dequeue operation, the prefetched events are
dequeued, and prefetch is initiated again.
In the case of application-controlled prefetch hints, the currently held flow
contexts, if any, are not affected by the prefetch operation. On the next
dequeue operation, the prefetched events are returned, but prefetch is not
initiated again until the application provides the hint again. If prefetching
is already enabled at the event device level or event port level,
the hint is ignored.
Pavan Nikhilesh (3):
eventdev: introduce event prefetching
eventdev: allow event ports to modified prefetches
eventdev: add SW event prefetch hint
app/test/test_eventdev.c | 113 +++++++++++++++
doc/guides/prog_guide/eventdev/eventdev.rst | 42 ++++++
lib/eventdev/eventdev_pmd.h | 4 +
lib/eventdev/eventdev_private.c | 19 +++
lib/eventdev/eventdev_trace_points.c | 6 +
lib/eventdev/rte_eventdev.h | 152 ++++++++++++++++++++
lib/eventdev/rte_eventdev_core.h | 9 ++
lib/eventdev/rte_eventdev_trace_fp.h | 19 ++-
lib/eventdev/version.map | 6 +
9 files changed, 369 insertions(+), 1 deletion(-)
--
2.46.0
Comments
On 2024-09-10 10:31, pbhagavatula@marvell.com wrote: > From: Pavan Nikhilesh <pbhagavatula@marvell.com> > > Event prefetching improves scheduling performance by pre-scheduling events to > event ports when dequeues are issued. This series introduces various types and > levels of prefetching to the eventdev library. > I would avoid to use the term "prefetch", since that word is used for manual (software) prefetching (preloading in ARM speak). In particular, since having the event device prefetch future event-related data may potentially also be interesting to do (i.e., to have an option to prefetch user-configurable lines in the buffer pointed to by a future event). I suggest RTE_EVENT_DEV_PRESCHEDULE*. If you make prescheduling a hint, you don't need the capability flags, and no error handling. Much smaller API, less fuzz for the application. > Prefetching Types: > * RTE_EVENT_DEV_PREFETCH_NONE: No prefetching. > * RTE_EVENT_DEV_PREFETCH: Always issue a prefetch when dequeue is issued. > * RTE_EVENT_DEV_PREFETCH_INTELLIGENT: Delay issuing prefetch until there > are no forward progress constraints with the held flow contexts. > Maybe RTE_EVENT_DEV_PREFETCH_ADAPTIVE is better. I suggest that being the default. In that case, for event devices which can't preschedule or which always (by design) have to preschedule, can continue doing what they are doing and still calling this "adaptive". RTE_EVENT_DEV_PREFETCH_AUTO would also work. Setting aside the complexity issue, prescheduling hints or configuration seems like a valuable feature to include in the Eventdev API. > Prefetching Levels: > * Event Device Level Prefetch: Prefetching can be enabled or disabled at the > event device during configuration. Event devices can indicate prefetching > capabilities using `RTE_EVENT_DEV_CAP_EVENT_PREFETCH` and > `RTE_EVENT_DEV_CAP_EVENT_INTELLIGENT_PREFETCH` via the event device info > function `info.event_dev_cap`. > * Event Port Level Prefetch: Prefetching can be selectively enabled or disabled > at the event port during runtime. Event devices can indicate this capability > using `RTE_EVENT_PORT_CAP_EVENT_PER_PORT_PREFETCH` via the event device info > function `info.event_port_cap`. > * Application Controlled Prefetch Hint: Applications can provide hints to the > event device to start prefetching events using the new API > `rte_event_port_prefetch`. Event devices can indicate this capabilities using > `RTE_EVENT_DEV_CAP_SW_PREFETCH` via the event device info function > `info.event_dev_cap`. > > The rte_event_dequeue_burst operation initiates the prefetch operation, which > completes in parallel without affecting the dequeued event flow context and > dequeue latency. On the next dequeue operation, the prefetched events are > dequeued, and prefetch is initiated again. > > In the case of application-controlled prefetch hints, the currently held flow > contexts, if any, are not affected by the prefetch operation. On the next > dequeue operation, the prefetched events are returned, but prefetch is not > initiated again until the application provides the hint again. If prefetching > is already enabled at the event device level or event port level, > the hint is ignored. > > Pavan Nikhilesh (3): > eventdev: introduce event prefetching > eventdev: allow event ports to modified prefetches > eventdev: add SW event prefetch hint > > app/test/test_eventdev.c | 113 +++++++++++++++ > doc/guides/prog_guide/eventdev/eventdev.rst | 42 ++++++ > lib/eventdev/eventdev_pmd.h | 4 + > lib/eventdev/eventdev_private.c | 19 +++ > lib/eventdev/eventdev_trace_points.c | 6 + > lib/eventdev/rte_eventdev.h | 152 ++++++++++++++++++++ > lib/eventdev/rte_eventdev_core.h | 9 ++ > lib/eventdev/rte_eventdev_trace_fp.h | 19 ++- > lib/eventdev/version.map | 6 + > 9 files changed, 369 insertions(+), 1 deletion(-) > > -- > 2.46.0
> On 2024-09-10 10:31, pbhagavatula@marvell.com wrote: > > From: Pavan Nikhilesh <pbhagavatula@marvell.com> > > > > Event prefetching improves scheduling performance by pre-scheduling > events to > > event ports when dequeues are issued. This series introduces various types > and > > levels of prefetching to the eventdev library. > > > > I would avoid to use the term "prefetch", since that word is used for > manual (software) prefetching (preloading in ARM speak). In particular, > since having the event device prefetch future event-related data may > potentially also be interesting to do (i.e., to have an option to > prefetch user-configurable lines in the buffer pointed to by a future > event). > > I suggest RTE_EVENT_DEV_PRESCHEDULE*. Yeah, preschedule makes more sense in this context, I was too far into the RFC to make the change. I will make the change in the next version. In fact OCTEON 10 already supports event data prefetching, I have a RFC for that in the pipeline. > > If you make prescheduling a hint, you don't need the capability flags, > and no error handling. Much smaller API, less fuzz for the application. > The hint part is in 3/3, it still has a capability flag attached to it so that application don’t crash if driver doesn’t implement the API i.e., ops is NULL. We could add a NULL check for ops but since its fast path I am not sure. > > Prefetching Types: > > * RTE_EVENT_DEV_PREFETCH_NONE: No prefetching. > > * RTE_EVENT_DEV_PREFETCH: Always issue a prefetch when dequeue is > issued. > > * RTE_EVENT_DEV_PREFETCH_INTELLIGENT: Delay issuing prefetch until > there > > are no forward progress constraints with the held flow contexts. > > > > Maybe RTE_EVENT_DEV_PREFETCH_ADAPTIVE is better. I suggest that being > the default. In that case, for event devices which can't preschedule or > which always (by design) have to preschedule, can continue doing what > they are doing and still calling this "adaptive". > Adaptive sounds good. I will make the change next version. I think the driver layer can fix up if application requests to disable a preschedule type that It's already doing and can't be disabled. But claiming prefetch capability when its not supported might not be a good idea. In the end the idea is to give the application fine grained control over prescheduling. As from what we have seen customers have wanted these fine grained controls over pre-scheduling. > RTE_EVENT_DEV_PREFETCH_AUTO would also work. > > Setting aside the complexity issue, prescheduling hints or configuration > seems like a valuable feature to include in the Eventdev API. > > > Prefetching Levels: > > * Event Device Level Prefetch: Prefetching can be enabled or disabled at the > > event device during configuration. Event devices can indicate prefetching > > capabilities using `RTE_EVENT_DEV_CAP_EVENT_PREFETCH` and > > `RTE_EVENT_DEV_CAP_EVENT_INTELLIGENT_PREFETCH` via the event > device info > > function `info.event_dev_cap`. > > * Event Port Level Prefetch: Prefetching can be selectively enabled or > disabled > > at the event port during runtime. Event devices can indicate this capability > > using `RTE_EVENT_PORT_CAP_EVENT_PER_PORT_PREFETCH` via the > event device info > > function `info.event_port_cap`. > > * Application Controlled Prefetch Hint: Applications can provide hints to the > > event device to start prefetching events using the new API > > `rte_event_port_prefetch`. Event devices can indicate this capabilities using > > `RTE_EVENT_DEV_CAP_SW_PREFETCH` via the event device info function > > `info.event_dev_cap`. > > > > The rte_event_dequeue_burst operation initiates the prefetch operation, > which > > completes in parallel without affecting the dequeued event flow context and > > dequeue latency. On the next dequeue operation, the prefetched events are > > dequeued, and prefetch is initiated again. > > > > In the case of application-controlled prefetch hints, the currently held flow > > contexts, if any, are not affected by the prefetch operation. On the next > > dequeue operation, the prefetched events are returned, but prefetch is not > > initiated again until the application provides the hint again. If prefetching > > is already enabled at the event device level or event port level, > > the hint is ignored. > > > > Pavan Nikhilesh (3): > > eventdev: introduce event prefetching > > eventdev: allow event ports to modified prefetches > > eventdev: add SW event prefetch hint > > > > app/test/test_eventdev.c | 113 +++++++++++++++ > > doc/guides/prog_guide/eventdev/eventdev.rst | 42 ++++++ > > lib/eventdev/eventdev_pmd.h | 4 + > > lib/eventdev/eventdev_private.c | 19 +++ > > lib/eventdev/eventdev_trace_points.c | 6 + > > lib/eventdev/rte_eventdev.h | 152 ++++++++++++++++++++ > > lib/eventdev/rte_eventdev_core.h | 9 ++ > > lib/eventdev/rte_eventdev_trace_fp.h | 19 ++- > > lib/eventdev/version.map | 6 + > > 9 files changed, 369 insertions(+), 1 deletion(-) > > > > -- > > 2.46.0