From patchwork Thu Jun 8 15:18:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yan, Zhirun" X-Patchwork-Id: 128425 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5B7CF42C61; Thu, 8 Jun 2023 17:25:57 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 216E742D0B; Thu, 8 Jun 2023 17:25:51 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id 8CB5040A84 for ; Thu, 8 Jun 2023 17:25:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686237948; x=1717773948; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ngMEITcYEpTuH4YD6aF+DwhRjSrETW2Mnw/nJ3I03qM=; b=bVdENjPdRL+eyX2obTLztj6dCZTnprQltRWz78LKYF2DqBqlrI86+e8F e9QTKtFLgxGUDI60fPGYSNX1GzaakeWD97Yn223PB9+Mwqql0zkXQ7ASs KxzyRxfHhLM6nZaPAZ9xUvDWfKuTcfmcbnBlcK+cbOlSrY2Pyj2lm/2zi 5q17h3+et9zCMPDgSa5edRwBk085Lvt650SxruDMJKzm+GcbTq+O5Zg6O 1omc3d4ddONj4dZOnO603yXV52np1ThPk2qCkHb34h2ttR8zoQY86bM8l uDFRhUuetaTQr1kMDcEsnlspZ34kU0BEXl4G2Wz8LFLoi/JrE8ostfWzX w==; X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="385681298" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="385681298" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Jun 2023 08:25:48 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="834232860" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="834232860" Received: from dpdk-zhirun-lmm.sh.intel.com ([10.67.119.94]) by orsmga004.jf.intel.com with ESMTP; 08 Jun 2023 08:25:45 -0700 From: Zhirun Yan To: dev@dpdk.org, jerinj@marvell.com, kirankumark@marvell.com, ndabilpuram@marvell.com, stephen@networkplumber.org, pbhagavatula@marvell.com, jerinjacobk@gmail.com Cc: cunming.liang@intel.com, haiyue.wang@intel.com, mattias.ronnblom@ericsson.com, Zhirun Yan Subject: [PATCH v11 01/16] graph: rename rte_graph_work as common Date: Thu, 8 Jun 2023 23:18:29 +0800 Message-Id: <20230608151844.1823783-2-zhirun.yan@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230608151844.1823783-1-zhirun.yan@intel.com> References: <20230608095759.1800617-1-zhirun.yan@intel.com> <20230608151844.1823783-1-zhirun.yan@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Rename rte_graph_work.h to rte_graph_work_common.h for supporting multiple graph worker model. Signed-off-by: Haiyue Wang Signed-off-by: Cunming Liang Signed-off-by: Zhirun Yan Acked-by: Jerin Jacob --- MAINTAINERS | 3 ++- lib/graph/graph_pcap.c | 2 +- lib/graph/graph_private.h | 2 +- lib/graph/meson.build | 2 +- lib/graph/{rte_graph_worker.h => rte_graph_worker_common.h} | 6 +++--- 5 files changed, 8 insertions(+), 7 deletions(-) rename lib/graph/{rte_graph_worker.h => rte_graph_worker_common.h} (99%) diff --git a/MAINTAINERS b/MAINTAINERS index 48830ae571..34ac499c14 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1716,10 +1716,11 @@ F: doc/guides/prog_guide/bpf_lib.rst Graph - EXPERIMENTAL M: Jerin Jacob M: Kiran Kumar K +M: Nithin Dabilpuram +M: Zhirun Yan F: lib/graph/ F: doc/guides/prog_guide/graph_lib.rst F: app/test/test_graph* -M: Nithin Dabilpuram F: examples/l3fwd-graph/ F: doc/guides/sample_app_ug/l3_forward_graph.rst diff --git a/lib/graph/graph_pcap.c b/lib/graph/graph_pcap.c index 6c43330029..8a220370fa 100644 --- a/lib/graph/graph_pcap.c +++ b/lib/graph/graph_pcap.c @@ -10,7 +10,7 @@ #include #include -#include "rte_graph_worker.h" +#include "rte_graph_worker_common.h" #include "graph_pcap_private.h" diff --git a/lib/graph/graph_private.h b/lib/graph/graph_private.h index eacdef45f0..307e5f70bc 100644 --- a/lib/graph/graph_private.h +++ b/lib/graph/graph_private.h @@ -13,7 +13,7 @@ #include #include "rte_graph.h" -#include "rte_graph_worker.h" +#include "rte_graph_worker_common.h" extern int rte_graph_logtype; diff --git a/lib/graph/meson.build b/lib/graph/meson.build index 3526d1b5d4..4e2b612ad3 100644 --- a/lib/graph/meson.build +++ b/lib/graph/meson.build @@ -16,6 +16,6 @@ sources = files( 'graph_populate.c', 'graph_pcap.c', ) -headers = files('rte_graph.h', 'rte_graph_worker.h') +headers = files('rte_graph.h', 'rte_graph_worker_common.h') deps += ['eal', 'pcapng'] diff --git a/lib/graph/rte_graph_worker.h b/lib/graph/rte_graph_worker_common.h similarity index 99% rename from lib/graph/rte_graph_worker.h rename to lib/graph/rte_graph_worker_common.h index 438595b15c..0bad2938f3 100644 --- a/lib/graph/rte_graph_worker.h +++ b/lib/graph/rte_graph_worker_common.h @@ -2,8 +2,8 @@ * Copyright(C) 2020 Marvell International Ltd. */ -#ifndef _RTE_GRAPH_WORKER_H_ -#define _RTE_GRAPH_WORKER_H_ +#ifndef _RTE_GRAPH_WORKER_COMMON_H_ +#define _RTE_GRAPH_WORKER_COMMON_H_ /** * @file rte_graph_worker.h @@ -518,4 +518,4 @@ rte_node_next_stream_move(struct rte_graph *graph, struct rte_node *src, } #endif -#endif /* _RTE_GRAPH_WORKER_H_ */ +#endif /* _RTE_GRAPH_WORKER_COIMMON_H_ */ From patchwork Thu Jun 8 15:18:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yan, Zhirun" X-Patchwork-Id: 128426 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9628642C61; Thu, 8 Jun 2023 17:26:03 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 286EA41611; Thu, 8 Jun 2023 17:25:54 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id CE65442D13 for ; Thu, 8 Jun 2023 17:25:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686237952; x=1717773952; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=3ceSnfkmlNF3RPqL9CvWCIfdJAdUn9+QAOiFuLZnH1c=; b=bmpEhcorKwGVnZRqL1ynpLsGC4Wa4haOrl+D4QL2Lwu3gndWM//ux9Ux Xw79NqgBRWcrXpYYV1I0riBqHnpBhimw8nVoDg2jtSMQZG1gQDnXqY2/u h9SGBT2LhpvaOSPrRvK4K/FYRzBZGjHHp/mb4ZvR5N/UOxUyyHhGKW1Q5 p+SMA8qr7AsoZdVzMMqiQIdSa7lQd5yXWjcbkyTCorc7LXorNIiPKw8fn /7jjsPX9oqU9Wb2b2Sd9CTIB7Kv0LppVX6CetxdQYVwKawbG2ForDL2sm OR1jrtaULuBh6np6Izks3rjb26FbC2Fil0ucgvSftixPL/cxbmbEZA8iM g==; X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="385681320" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="385681320" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Jun 2023 08:25:51 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="834232879" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="834232879" Received: from dpdk-zhirun-lmm.sh.intel.com ([10.67.119.94]) by orsmga004.jf.intel.com with ESMTP; 08 Jun 2023 08:25:48 -0700 From: Zhirun Yan To: dev@dpdk.org, jerinj@marvell.com, kirankumark@marvell.com, ndabilpuram@marvell.com, stephen@networkplumber.org, pbhagavatula@marvell.com, jerinjacobk@gmail.com Cc: cunming.liang@intel.com, haiyue.wang@intel.com, mattias.ronnblom@ericsson.com, Zhirun Yan Subject: [PATCH v11 02/16] graph: split graph worker into common and default model Date: Thu, 8 Jun 2023 23:18:30 +0800 Message-Id: <20230608151844.1823783-3-zhirun.yan@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230608151844.1823783-1-zhirun.yan@intel.com> References: <20230608095759.1800617-1-zhirun.yan@intel.com> <20230608151844.1823783-1-zhirun.yan@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org To support multiple graph worker model, split graph into common and default. Naming the current walk function as rte_graph_model_rtc cause the default model is RTC(Run-to-completion). Signed-off-by: Haiyue Wang Signed-off-by: Cunming Liang Signed-off-by: Zhirun Yan Acked-by: Jerin Jacob --- lib/graph/graph_pcap.c | 2 +- lib/graph/graph_private.h | 2 +- lib/graph/meson.build | 2 +- lib/graph/rte_graph_model_rtc.h | 62 +++++++++++++++++++++++++++++ lib/graph/rte_graph_worker.h | 35 ++++++++++++++++ lib/graph/rte_graph_worker_common.h | 57 -------------------------- 6 files changed, 100 insertions(+), 60 deletions(-) create mode 100644 lib/graph/rte_graph_model_rtc.h create mode 100644 lib/graph/rte_graph_worker.h diff --git a/lib/graph/graph_pcap.c b/lib/graph/graph_pcap.c index 8a220370fa..6c43330029 100644 --- a/lib/graph/graph_pcap.c +++ b/lib/graph/graph_pcap.c @@ -10,7 +10,7 @@ #include #include -#include "rte_graph_worker_common.h" +#include "rte_graph_worker.h" #include "graph_pcap_private.h" diff --git a/lib/graph/graph_private.h b/lib/graph/graph_private.h index 307e5f70bc..eacdef45f0 100644 --- a/lib/graph/graph_private.h +++ b/lib/graph/graph_private.h @@ -13,7 +13,7 @@ #include #include "rte_graph.h" -#include "rte_graph_worker_common.h" +#include "rte_graph_worker.h" extern int rte_graph_logtype; diff --git a/lib/graph/meson.build b/lib/graph/meson.build index 4e2b612ad3..3526d1b5d4 100644 --- a/lib/graph/meson.build +++ b/lib/graph/meson.build @@ -16,6 +16,6 @@ sources = files( 'graph_populate.c', 'graph_pcap.c', ) -headers = files('rte_graph.h', 'rte_graph_worker_common.h') +headers = files('rte_graph.h', 'rte_graph_worker.h') deps += ['eal', 'pcapng'] diff --git a/lib/graph/rte_graph_model_rtc.h b/lib/graph/rte_graph_model_rtc.h new file mode 100644 index 0000000000..10b359772f --- /dev/null +++ b/lib/graph/rte_graph_model_rtc.h @@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2020 Marvell International Ltd. + * Copyright(C) 2023 Intel Corporation + */ + +#include "rte_graph_worker_common.h" + +/** + * Perform graph walk on the circular buffer and invoke the process function + * of the nodes and collect the stats. + * + * @param graph + * Graph pointer returned from rte_graph_lookup function. + * + * @see rte_graph_lookup() + */ +static inline void +rte_graph_walk_rtc(struct rte_graph *graph) +{ + const rte_graph_off_t *cir_start = graph->cir_start; + const rte_node_t mask = graph->cir_mask; + uint32_t head = graph->head; + struct rte_node *node; + uint64_t start; + uint16_t rc; + void **objs; + + /* + * Walk on the source node(s) ((cir_start - head) -> cir_start) and then + * on the pending streams (cir_start -> (cir_start + mask) -> cir_start) + * in a circular buffer fashion. + * + * +-----+ <= cir_start - head [number of source nodes] + * | | + * | ... | <= source nodes + * | | + * +-----+ <= cir_start [head = 0] [tail = 0] + * | | + * | ... | <= pending streams + * | | + * +-----+ <= cir_start + mask + */ + while (likely(head != graph->tail)) { + node = (struct rte_node *)RTE_PTR_ADD(graph, cir_start[(int32_t)head++]); + RTE_ASSERT(node->fence == RTE_GRAPH_FENCE); + objs = node->objs; + rte_prefetch0(objs); + + if (rte_graph_has_stats_feature()) { + start = rte_rdtsc(); + rc = node->process(graph, node, objs, node->idx); + node->total_cycles += rte_rdtsc() - start; + node->total_calls++; + node->total_objs += rc; + } else { + node->process(graph, node, objs, node->idx); + } + node->idx = 0; + head = likely((int32_t)head > 0) ? head & mask : head; + } + graph->tail = 0; +} diff --git a/lib/graph/rte_graph_worker.h b/lib/graph/rte_graph_worker.h new file mode 100644 index 0000000000..5b58f7bda9 --- /dev/null +++ b/lib/graph/rte_graph_worker.h @@ -0,0 +1,35 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2020 Marvell International Ltd. + * Copyright(C) 2023 Intel Corporation + */ + +#ifndef _RTE_GRAPH_WORKER_H_ +#define _RTE_GRAPH_WORKER_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include "rte_graph_model_rtc.h" + +/** + * Perform graph walk on the circular buffer and invoke the process function + * of the nodes and collect the stats. + * + * @param graph + * Graph pointer returned from rte_graph_lookup function. + * + * @see rte_graph_lookup() + */ +__rte_experimental +static inline void +rte_graph_walk(struct rte_graph *graph) +{ + rte_graph_walk_rtc(graph); +} + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_GRAPH_WORKER_H_ */ diff --git a/lib/graph/rte_graph_worker_common.h b/lib/graph/rte_graph_worker_common.h index 0bad2938f3..b58f8f6947 100644 --- a/lib/graph/rte_graph_worker_common.h +++ b/lib/graph/rte_graph_worker_common.h @@ -128,63 +128,6 @@ __rte_experimental void __rte_node_stream_alloc_size(struct rte_graph *graph, struct rte_node *node, uint16_t req_size); -/** - * Perform graph walk on the circular buffer and invoke the process function - * of the nodes and collect the stats. - * - * @param graph - * Graph pointer returned from rte_graph_lookup function. - * - * @see rte_graph_lookup() - */ -__rte_experimental -static inline void -rte_graph_walk(struct rte_graph *graph) -{ - const rte_graph_off_t *cir_start = graph->cir_start; - const rte_node_t mask = graph->cir_mask; - uint32_t head = graph->head; - struct rte_node *node; - uint64_t start; - uint16_t rc; - void **objs; - - /* - * Walk on the source node(s) ((cir_start - head) -> cir_start) and then - * on the pending streams (cir_start -> (cir_start + mask) -> cir_start) - * in a circular buffer fashion. - * - * +-----+ <= cir_start - head [number of source nodes] - * | | - * | ... | <= source nodes - * | | - * +-----+ <= cir_start [head = 0] [tail = 0] - * | | - * | ... | <= pending streams - * | | - * +-----+ <= cir_start + mask - */ - while (likely(head != graph->tail)) { - node = (struct rte_node *)RTE_PTR_ADD(graph, cir_start[(int32_t)head++]); - RTE_ASSERT(node->fence == RTE_GRAPH_FENCE); - objs = node->objs; - rte_prefetch0(objs); - - if (rte_graph_has_stats_feature()) { - start = rte_rdtsc(); - rc = node->process(graph, node, objs, node->idx); - node->total_cycles += rte_rdtsc() - start; - node->total_calls++; - node->total_objs += rc; - } else { - node->process(graph, node, objs, node->idx); - } - node->idx = 0; - head = likely((int32_t)head > 0) ? head & mask : head; - } - graph->tail = 0; -} - /* Fast path helper functions */ /** From patchwork Thu Jun 8 15:18:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yan, Zhirun" X-Patchwork-Id: 128427 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3012842C61; Thu, 8 Jun 2023 17:26:10 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3007142D29; Thu, 8 Jun 2023 17:25:57 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id 150EF42D20 for ; Thu, 8 Jun 2023 17:25:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686237955; x=1717773955; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+z9i5Mz91r76eAu3VeqVc0fe7ZBoJIjw/A4PwNLYR5M=; b=f0OtykuoyFPPVzZeCPYc6MTQ/asLeEcuGjAsyGur9AFVBc0WQqr02piD ttowZakThG5yUTDRW68YbmbIZlV+l7Lswb2rxjok+IDWTOj+aYpk8Fx+W 69f01ClZInd9ca1WCaYizLuvVw+3VlQSscOiIZru5DIFmCjQxNmHLGtOv k9gi+5pR6Xrn7GNJUKVawP4qDuBU3e2rSV+WaQ9FOhLiBsH/VvRzzDB3g w5cN5Nfh02952CPvoLE38VB9iAOwBeF/hZVM+eYdnlXAYMsgSeCbrgW2u 4i1O/KMg3mE1KVR5+u9T8fq4gT5p77ZVko/EMr1+d3vsTqucpGCfMWJlr Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="385681335" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="385681335" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Jun 2023 08:25:54 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="834232907" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="834232907" Received: from dpdk-zhirun-lmm.sh.intel.com ([10.67.119.94]) by orsmga004.jf.intel.com with ESMTP; 08 Jun 2023 08:25:51 -0700 From: Zhirun Yan To: dev@dpdk.org, jerinj@marvell.com, kirankumark@marvell.com, ndabilpuram@marvell.com, stephen@networkplumber.org, pbhagavatula@marvell.com, jerinjacobk@gmail.com Cc: cunming.liang@intel.com, haiyue.wang@intel.com, mattias.ronnblom@ericsson.com, Zhirun Yan Subject: [PATCH v11 03/16] graph: move node process into inline function Date: Thu, 8 Jun 2023 23:18:31 +0800 Message-Id: <20230608151844.1823783-4-zhirun.yan@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230608151844.1823783-1-zhirun.yan@intel.com> References: <20230608095759.1800617-1-zhirun.yan@intel.com> <20230608151844.1823783-1-zhirun.yan@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Node process is a single and reusable block, move the code into an inline function. Signed-off-by: Haiyue Wang Signed-off-by: Cunming Liang Signed-off-by: Zhirun Yan Acked-by: Jerin Jacob --- lib/graph/rte_graph_model_rtc.h | 20 ++--------------- lib/graph/rte_graph_worker_common.h | 33 +++++++++++++++++++++++++++++ 2 files changed, 35 insertions(+), 18 deletions(-) diff --git a/lib/graph/rte_graph_model_rtc.h b/lib/graph/rte_graph_model_rtc.h index 10b359772f..4b6236e301 100644 --- a/lib/graph/rte_graph_model_rtc.h +++ b/lib/graph/rte_graph_model_rtc.h @@ -21,9 +21,6 @@ rte_graph_walk_rtc(struct rte_graph *graph) const rte_node_t mask = graph->cir_mask; uint32_t head = graph->head; struct rte_node *node; - uint64_t start; - uint16_t rc; - void **objs; /* * Walk on the source node(s) ((cir_start - head) -> cir_start) and then @@ -42,21 +39,8 @@ rte_graph_walk_rtc(struct rte_graph *graph) */ while (likely(head != graph->tail)) { node = (struct rte_node *)RTE_PTR_ADD(graph, cir_start[(int32_t)head++]); - RTE_ASSERT(node->fence == RTE_GRAPH_FENCE); - objs = node->objs; - rte_prefetch0(objs); - - if (rte_graph_has_stats_feature()) { - start = rte_rdtsc(); - rc = node->process(graph, node, objs, node->idx); - node->total_cycles += rte_rdtsc() - start; - node->total_calls++; - node->total_objs += rc; - } else { - node->process(graph, node, objs, node->idx); - } - node->idx = 0; - head = likely((int32_t)head > 0) ? head & mask : head; + __rte_node_process(graph, node); + head = likely((int32_t)head > 0) ? head & mask : head; } graph->tail = 0; } diff --git a/lib/graph/rte_graph_worker_common.h b/lib/graph/rte_graph_worker_common.h index b58f8f6947..41428974db 100644 --- a/lib/graph/rte_graph_worker_common.h +++ b/lib/graph/rte_graph_worker_common.h @@ -130,6 +130,39 @@ void __rte_node_stream_alloc_size(struct rte_graph *graph, /* Fast path helper functions */ +/** + * @internal + * + * Enqueue a given node to the tail of the graph reel. + * + * @param graph + * Pointer Graph object. + * @param node + * Pointer to node object to be enqueued. + */ +static __rte_always_inline void +__rte_node_process(struct rte_graph *graph, struct rte_node *node) +{ + uint64_t start; + uint16_t rc; + void **objs; + + RTE_ASSERT(node->fence == RTE_GRAPH_FENCE); + objs = node->objs; + rte_prefetch0(objs); + + if (rte_graph_has_stats_feature()) { + start = rte_rdtsc(); + rc = node->process(graph, node, objs, node->idx); + node->total_cycles += rte_rdtsc() - start; + node->total_calls++; + node->total_objs += rc; + } else { + node->process(graph, node, objs, node->idx); + } + node->idx = 0; +} + /** * @internal * From patchwork Thu Jun 8 15:18:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yan, Zhirun" X-Patchwork-Id: 128428 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2DF8342C61; Thu, 8 Jun 2023 17:26:16 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 429D242D33; Thu, 8 Jun 2023 17:25:59 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id 48C6542D31 for ; Thu, 8 Jun 2023 17:25:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686237958; x=1717773958; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=BjlgFzBftHsjc37XmHRrgFiFU0c8bgbCA2l+rEEZqDQ=; b=HZAY3GOsdPX4OOA2I/dflyiAzWuY+xU1j6viAXVdXvsf6U866vPw2y9K Sh1+/3daMwK4pB3c0ogJi4iDNXvdAUr02B/02E21f2qXb5YAHvsxYYQ+M mr2LPh+WfiB+3m6N99l/D7BYDvcK7QowRa3h71DK5Q8atQKPP3jthlZ3p Fob2iQ7t1ewtG7y+Fm6OypuKemxLEkTEBk11xKJfGiSUD3gwCskfF30Pf o5HNj/3L3C3fdCAGsVttRLH5xC8dNj6lu8lvS1H8uM8aE1S9nHcFjhfjE uwzJKgsCJh6vCIKo5HlLjRjCAoRVX2zGw/KGZm/YmkK3yj4QnxHJ3lzTQ Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="385681353" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="385681353" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Jun 2023 08:25:57 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="834232925" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="834232925" Received: from dpdk-zhirun-lmm.sh.intel.com ([10.67.119.94]) by orsmga004.jf.intel.com with ESMTP; 08 Jun 2023 08:25:54 -0700 From: Zhirun Yan To: dev@dpdk.org, jerinj@marvell.com, kirankumark@marvell.com, ndabilpuram@marvell.com, stephen@networkplumber.org, pbhagavatula@marvell.com, jerinjacobk@gmail.com Cc: cunming.liang@intel.com, haiyue.wang@intel.com, mattias.ronnblom@ericsson.com, Zhirun Yan Subject: [PATCH v11 04/16] graph: add get/set graph worker model APIs Date: Thu, 8 Jun 2023 23:18:32 +0800 Message-Id: <20230608151844.1823783-5-zhirun.yan@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230608151844.1823783-1-zhirun.yan@intel.com> References: <20230608095759.1800617-1-zhirun.yan@intel.com> <20230608151844.1823783-1-zhirun.yan@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add new get/set APIs to configure graph worker model which is used to determine which model will be chosen. Signed-off-by: Haiyue Wang Signed-off-by: Cunming Liang Signed-off-by: Zhirun Yan Acked-by: Jerin Jacob --- lib/graph/meson.build | 1 + lib/graph/rte_graph_worker.c | 39 ++++++++++++++++ lib/graph/rte_graph_worker_common.h | 70 +++++++++++++++++++++++++++++ lib/graph/version.map | 5 +++ 4 files changed, 115 insertions(+) create mode 100644 lib/graph/rte_graph_worker.c diff --git a/lib/graph/meson.build b/lib/graph/meson.build index 3526d1b5d4..9fab8243da 100644 --- a/lib/graph/meson.build +++ b/lib/graph/meson.build @@ -15,6 +15,7 @@ sources = files( 'graph_stats.c', 'graph_populate.c', 'graph_pcap.c', + 'rte_graph_worker.c', ) headers = files('rte_graph.h', 'rte_graph_worker.h') diff --git a/lib/graph/rte_graph_worker.c b/lib/graph/rte_graph_worker.c new file mode 100644 index 0000000000..7e2a918fae --- /dev/null +++ b/lib/graph/rte_graph_worker.c @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Intel Corporation + */ + +#include "rte_graph_worker_common.h" +#include "graph_private.h" + +bool +rte_graph_model_is_valid(uint8_t model) +{ + if (model > RTE_GRAPH_MODEL_MCORE_DISPATCH) + return false; + + return true; +} + +int +rte_graph_worker_model_set(uint8_t model) +{ + struct graph_head *graph_head = graph_list_head_get(); + struct graph *graph; + + if (!rte_graph_model_is_valid(model)) + return -EINVAL; + + STAILQ_FOREACH(graph, graph_head, next) + graph->graph->model = model; + + return 0; +} + +uint8_t +rte_graph_worker_model_get(struct rte_graph *graph) +{ + if (!rte_graph_model_is_valid(graph->model)) + return -EINVAL; + + return graph->model; +} diff --git a/lib/graph/rte_graph_worker_common.h b/lib/graph/rte_graph_worker_common.h index 41428974db..3a32001e35 100644 --- a/lib/graph/rte_graph_worker_common.h +++ b/lib/graph/rte_graph_worker_common.h @@ -29,6 +29,13 @@ extern "C" { #endif +/** Graph worker models */ +/* When adding a new graph model entry, update rte_graph_model_is_valid() implementation. */ +#define RTE_GRAPH_MODEL_RTC 0 /**< Run-To-Completion model. It is the default model. */ +#define RTE_GRAPH_MODEL_MCORE_DISPATCH 1 +/**< Dispatch model to support cross-core dispatching within core affinity. */ +#define RTE_GRAPH_MODEL_DEFAULT RTE_GRAPH_MODEL_RTC /**< Default graph model. */ + /** * @internal * @@ -41,6 +48,9 @@ struct rte_graph { rte_node_t nb_nodes; /**< Number of nodes in the graph. */ rte_graph_off_t *cir_start; /**< Pointer to circular buffer. */ rte_graph_off_t nodes_start; /**< Offset at which node memory starts. */ + uint8_t model; /**< graph model */ + uint8_t reserved1; /**< Reserved for future use. */ + uint16_t reserved2; /**< Reserved for future use. */ rte_graph_t id; /**< Graph identifier. */ int socket; /**< Socket ID where memory is allocated. */ char name[RTE_GRAPH_NAMESIZE]; /**< Name of the graph. */ @@ -490,6 +500,66 @@ rte_node_next_stream_move(struct rte_graph *graph, struct rte_node *src, } } +/** + * Test the validity of model. + * + * @param model + * Model to check. + * + * @return + * True if graph model is valid, false otherwise. + */ +__rte_experimental +bool +rte_graph_model_is_valid(uint8_t model); + +/** + * @note This function does not perform any locking, and is only safe to call + * before graph running. It will set all graphs the same model. + * + * @param model + * Name of the graph worker model. + * + * @return + * 0 on success, -1 otherwise. + */ +__rte_experimental +int rte_graph_worker_model_set(uint8_t model); + +/** + * Get the graph worker model + * + * @note All graph will use the same model and this function will get model from the first one. + * Used for slow path. + * + * @param graph + * Graph pointer. + * + * @return + * Graph worker model on success. + */ +__rte_experimental +uint8_t rte_graph_worker_model_get(struct rte_graph *graph); + +/** + * Get the graph worker model without check + * + * @note All graph will use the same model and this function will get model from the first one. + * Used for fast path. + * + * @param graph + * Graph pointer. + * + * @return + * Graph worker model on success. + */ +__rte_experimental +static __rte_always_inline +uint8_t rte_graph_worker_model_no_check_get(struct rte_graph *graph) +{ + return graph->model; +} + #ifdef __cplusplus } #endif diff --git a/lib/graph/version.map b/lib/graph/version.map index 13b838752d..e9a680a45e 100644 --- a/lib/graph/version.map +++ b/lib/graph/version.map @@ -14,10 +14,15 @@ EXPERIMENTAL { rte_graph_lookup; rte_graph_list_dump; rte_graph_max_count; + rte_graph_model_is_valid; rte_graph_node_get; rte_graph_node_get_by_name; rte_graph_obj_dump; rte_graph_walk; + rte_graph_worker_model_get; + rte_graph_worker_model_no_check_get; + rte_graph_worker_model_set; + rte_graph_cluster_stats_create; rte_graph_cluster_stats_destroy; From patchwork Thu Jun 8 15:18:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yan, Zhirun" X-Patchwork-Id: 128429 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 170B842C61; Thu, 8 Jun 2023 17:26:22 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 583C442D38; Thu, 8 Jun 2023 17:26:03 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id 8C62142D38 for ; Thu, 8 Jun 2023 17:26:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686237961; x=1717773961; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=wwk9YjWPqE+c69oBnWcC+ZEqDVW6V0zHRTgOvsSXDR8=; b=fjoQei6bzGaI/VNRL1CoAb/QKWvACA7x88gdP5cNB4cOPhwaHde5M34F is3MVBFMtWI70YSAboOGhZPGVSlGwjsYsRmY8FjYIABL0ZcbUQm4WKIeJ 2jP0iPcDBPvzRtKZ4k+g62qjxn0Vw84cwGvl5tZXCZVXSw7ulkFmbUZMV 5DgKIZPUAnVDpFtdlMZvqa4yjTSceEEaibS/MazmcSal6lNfjckv8EBCB 0zOjYuKfPz7ZzVmnqGaNEwowPPdnpRGx+qpk7SErdOzrUYHVq4o1Bv5sU SSAnWvB+lg3QDsblAlTFNzkguwJrjh/2XgrSp02bD8mq9ZjpjP+g7JtQO w==; X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="385681381" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="385681381" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Jun 2023 08:26:01 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="834232948" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="834232948" Received: from dpdk-zhirun-lmm.sh.intel.com ([10.67.119.94]) by orsmga004.jf.intel.com with ESMTP; 08 Jun 2023 08:25:58 -0700 From: Zhirun Yan To: dev@dpdk.org, jerinj@marvell.com, kirankumark@marvell.com, ndabilpuram@marvell.com, stephen@networkplumber.org, pbhagavatula@marvell.com, jerinjacobk@gmail.com Cc: cunming.liang@intel.com, haiyue.wang@intel.com, mattias.ronnblom@ericsson.com, Zhirun Yan Subject: [PATCH v11 05/16] graph: introduce graph node core affinity API Date: Thu, 8 Jun 2023 23:18:33 +0800 Message-Id: <20230608151844.1823783-6-zhirun.yan@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230608151844.1823783-1-zhirun.yan@intel.com> References: <20230608095759.1800617-1-zhirun.yan@intel.com> <20230608151844.1823783-1-zhirun.yan@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add lcore_id for node to hold affinity core id and impl rte_graph_model_mcore_dispatch_lcore_affinity_set to set node affinity with specific lcore. Signed-off-by: Haiyue Wang Signed-off-by: Cunming Liang Signed-off-by: Zhirun Yan Acked-by: Jerin Jacob --- lib/graph/graph_private.h | 2 + lib/graph/meson.build | 1 + lib/graph/node.c | 1 + lib/graph/rte_graph_model_mcore_dispatch.c | 30 +++++++++++++++ lib/graph/rte_graph_model_mcore_dispatch.h | 45 ++++++++++++++++++++++ lib/graph/version.map | 1 + 6 files changed, 80 insertions(+) create mode 100644 lib/graph/rte_graph_model_mcore_dispatch.c create mode 100644 lib/graph/rte_graph_model_mcore_dispatch.h diff --git a/lib/graph/graph_private.h b/lib/graph/graph_private.h index eacdef45f0..ea4409448d 100644 --- a/lib/graph/graph_private.h +++ b/lib/graph/graph_private.h @@ -51,6 +51,8 @@ struct node { STAILQ_ENTRY(node) next; /**< Next node in the list. */ char name[RTE_NODE_NAMESIZE]; /**< Name of the node. */ uint64_t flags; /**< Node configuration flag. */ + unsigned int lcore_id; + /**< Node runs on the Lcore ID used for mcore dispatch model. */ rte_node_process_t process; /**< Node process function. */ rte_node_init_t init; /**< Node init function. */ rte_node_fini_t fini; /**< Node fini function. */ diff --git a/lib/graph/meson.build b/lib/graph/meson.build index 9fab8243da..0685cf9e72 100644 --- a/lib/graph/meson.build +++ b/lib/graph/meson.build @@ -16,6 +16,7 @@ sources = files( 'graph_populate.c', 'graph_pcap.c', 'rte_graph_worker.c', + 'rte_graph_model_mcore_dispatch.c', ) headers = files('rte_graph.h', 'rte_graph_worker.h') diff --git a/lib/graph/node.c b/lib/graph/node.c index 149414dcd9..339b4a0da5 100644 --- a/lib/graph/node.c +++ b/lib/graph/node.c @@ -100,6 +100,7 @@ __rte_node_register(const struct rte_node_register *reg) goto free; } + node->lcore_id = RTE_MAX_LCORE; node->id = node_id++; /* Add the node at tail */ diff --git a/lib/graph/rte_graph_model_mcore_dispatch.c b/lib/graph/rte_graph_model_mcore_dispatch.c new file mode 100644 index 0000000000..9df2479a10 --- /dev/null +++ b/lib/graph/rte_graph_model_mcore_dispatch.c @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Intel Corporation + */ + +#include "graph_private.h" +#include "rte_graph_model_mcore_dispatch.h" + +int +rte_graph_model_mcore_dispatch_node_lcore_affinity_set(const char *name, unsigned int lcore_id) +{ + struct node *node; + int ret = -EINVAL; + + if (lcore_id >= RTE_MAX_LCORE) + return ret; + + graph_spinlock_lock(); + + STAILQ_FOREACH(node, node_list_head_get(), next) { + if (strncmp(node->name, name, RTE_NODE_NAMESIZE) == 0) { + node->lcore_id = lcore_id; + ret = 0; + break; + } + } + + graph_spinlock_unlock(); + + return ret; +} diff --git a/lib/graph/rte_graph_model_mcore_dispatch.h b/lib/graph/rte_graph_model_mcore_dispatch.h new file mode 100644 index 0000000000..7da0483d13 --- /dev/null +++ b/lib/graph/rte_graph_model_mcore_dispatch.h @@ -0,0 +1,45 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Intel Corporation + */ + +#ifndef _RTE_GRAPH_MODEL_MCORE_DISPATCH_H_ +#define _RTE_GRAPH_MODEL_MCORE_DISPATCH_H_ + +/** + * @file rte_graph_model_mcore_dispatch.h + * + * @warning + * @b EXPERIMENTAL: + * All functions in this file may be changed or removed without prior notice. + * + * These APIs allow to set core affinity with the node and only used for mcore + * dispatch model. + */ + +#ifdef __cplusplus +extern "C" { +#endif + +#include "rte_graph_worker_common.h" + +/** + * Set lcore affinity with the node used for mcore dispatch model. + * + * @param name + * Valid node name. In the case of the cloned node, the name will be + * "parent node name" + "-" + name. + * @param lcore_id + * The lcore ID value. + * + * @return + * 0 on success, error otherwise. + */ +__rte_experimental +int rte_graph_model_mcore_dispatch_node_lcore_affinity_set(const char *name, + unsigned int lcore_id); + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_GRAPH_MODEL_MCORE_DISPATCH_H_ */ diff --git a/lib/graph/version.map b/lib/graph/version.map index e9a680a45e..6ae19b0d6e 100644 --- a/lib/graph/version.map +++ b/lib/graph/version.map @@ -15,6 +15,7 @@ EXPERIMENTAL { rte_graph_list_dump; rte_graph_max_count; rte_graph_model_is_valid; + rte_graph_model_mcore_dispatch_node_lcore_affinity_set; rte_graph_node_get; rte_graph_node_get_by_name; rte_graph_obj_dump; From patchwork Thu Jun 8 15:18:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yan, Zhirun" X-Patchwork-Id: 128430 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DF1A742C61; Thu, 8 Jun 2023 17:26:29 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C9A4D42D46; Thu, 8 Jun 2023 17:26:06 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id DF9C342D3D for ; Thu, 8 Jun 2023 17:26:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686237965; x=1717773965; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=z7g+FJRvryMqEjie9HFv3RgOudicqZCoX3cX2fjlvPk=; b=mDeAgj9bqvQcdcUG2o6SGqsY+qdn7nJecU91WWMsANStuDalx7/Ef1Ya ko3HdD7wVNN9bmcdVVxOZhWXkPIipcN0eNZXpbEcZZQjl6V9lcNGZtHgp 6vJG0CiAf/vxZGukmYp2q1sDiYRpmn6eg4xK+HNy5Dg67uZz709v1stUs tJXWRQZ6bFDoFHpwBUOr0MpFP+0YC2CG8d19wmSWKiMYLjZOGGUaU85+x SICxEb8qUCq5x8s9cmfK+yMcgtocUefshPhsdPnNNmM4CsFuo54aLLbxJ dTiqoXuHMuxo9iMko5BNgAP0rIlKHwcE4B9wk5nKAT7k3AKeK/IXfbPTn g==; X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="385681405" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="385681405" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Jun 2023 08:26:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="834232972" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="834232972" Received: from dpdk-zhirun-lmm.sh.intel.com ([10.67.119.94]) by orsmga004.jf.intel.com with ESMTP; 08 Jun 2023 08:26:01 -0700 From: Zhirun Yan To: dev@dpdk.org, jerinj@marvell.com, kirankumark@marvell.com, ndabilpuram@marvell.com, stephen@networkplumber.org, pbhagavatula@marvell.com, jerinjacobk@gmail.com Cc: cunming.liang@intel.com, haiyue.wang@intel.com, mattias.ronnblom@ericsson.com, Zhirun Yan Subject: [PATCH v11 06/16] graph: introduce graph bind unbind API Date: Thu, 8 Jun 2023 23:18:34 +0800 Message-Id: <20230608151844.1823783-7-zhirun.yan@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230608151844.1823783-1-zhirun.yan@intel.com> References: <20230608095759.1800617-1-zhirun.yan@intel.com> <20230608151844.1823783-1-zhirun.yan@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add lcore_id for graph to hold affinity core id where graph would run on. Add bind/unbind API to set/unset graph affinity attribute. lcore_id will be set as MAX by default, it means not enable this attribute. Signed-off-by: Haiyue Wang Signed-off-by: Cunming Liang Signed-off-by: Zhirun Yan Acked-by: Jerin Jacob --- lib/graph/graph.c | 60 +++++++++++++++++++++++++++++++++++++++ lib/graph/graph_private.h | 2 ++ lib/graph/rte_graph.h | 22 ++++++++++++++ lib/graph/version.map | 3 +- 4 files changed, 86 insertions(+), 1 deletion(-) diff --git a/lib/graph/graph.c b/lib/graph/graph.c index 5582631b53..8d5bd8b9ae 100644 --- a/lib/graph/graph.c +++ b/lib/graph/graph.c @@ -260,6 +260,65 @@ graph_mem_fixup_secondary(struct rte_graph *graph) return graph_mem_fixup_node_ctx(graph); } +static bool +graph_src_node_avail(struct graph *graph) +{ + struct graph_node *graph_node; + + STAILQ_FOREACH(graph_node, &graph->node_list, next) + if ((graph_node->node->flags & RTE_NODE_SOURCE_F) && + (graph_node->node->lcore_id == RTE_MAX_LCORE || + graph->lcore_id == graph_node->node->lcore_id)) + return true; + + return false; +} + +int +rte_graph_model_mcore_dispatch_core_bind(rte_graph_t id, int lcore) +{ + struct graph *graph; + + GRAPH_ID_CHECK(id); + if (!rte_lcore_is_enabled(lcore)) + SET_ERR_JMP(ENOLINK, fail, "lcore %d not enabled", lcore); + + STAILQ_FOREACH(graph, &graph_list, next) + if (graph->id == id) + break; + + if (graph->graph->model == RTE_GRAPH_MODEL_MCORE_DISPATCH) + goto fail; + + graph->lcore_id = lcore; + graph->socket = rte_lcore_to_socket_id(lcore); + + /* check the availability of source node */ + if (!graph_src_node_avail(graph)) + graph->graph->head = 0; + + return 0; + +fail: + return -rte_errno; +} + +void +rte_graph_model_mcore_dispatch_core_unbind(rte_graph_t id) +{ + struct graph *graph; + + GRAPH_ID_CHECK(id); + STAILQ_FOREACH(graph, &graph_list, next) + if (graph->id == id) + break; + + graph->lcore_id = RTE_MAX_LCORE; + +fail: + return; +} + struct rte_graph * rte_graph_lookup(const char *name) { @@ -346,6 +405,7 @@ rte_graph_create(const char *name, struct rte_graph_param *prm) graph->src_node_count = src_node_count; graph->node_count = graph_nodes_count(graph); graph->id = graph_id; + graph->lcore_id = RTE_MAX_LCORE; graph->num_pkt_to_capture = prm->num_pkt_to_capture; if (prm->pcap_filename) rte_strscpy(graph->pcap_filename, prm->pcap_filename, RTE_GRAPH_PCAP_FILE_SZ); diff --git a/lib/graph/graph_private.h b/lib/graph/graph_private.h index ea4409448d..6d2137c81b 100644 --- a/lib/graph/graph_private.h +++ b/lib/graph/graph_private.h @@ -100,6 +100,8 @@ struct graph { /**< Circular buffer mask for wrap around. */ rte_graph_t id; /**< Graph identifier. */ + unsigned int lcore_id; + /**< Lcore identifier where the graph prefer to run on. Used for mcore dispatch model. */ size_t mem_sz; /**< Memory size of the graph. */ int socket; diff --git a/lib/graph/rte_graph.h b/lib/graph/rte_graph.h index c9a77297fc..f70c694e77 100644 --- a/lib/graph/rte_graph.h +++ b/lib/graph/rte_graph.h @@ -285,6 +285,28 @@ char *rte_graph_id_to_name(rte_graph_t id); __rte_experimental int rte_graph_export(const char *name, FILE *f); +/** + * Bind graph with specific lcore for mcore dispatch model. + * + * @param id + * Graph id to get the pointer of graph object + * @param lcore + * The lcore where the graph will run on + * @return + * 0 on success, error otherwise. + */ +__rte_experimental +int rte_graph_model_mcore_dispatch_core_bind(rte_graph_t id, int lcore); + +/** + * Unbind graph with lcore for mcore dispatch model + * + * @param id + * Graph id to get the pointer of graph object + */ +__rte_experimental +void rte_graph_model_mcore_dispatch_core_unbind(rte_graph_t id); + /** * Get graph object from its name. * diff --git a/lib/graph/version.map b/lib/graph/version.map index 6ae19b0d6e..9a20dba5e7 100644 --- a/lib/graph/version.map +++ b/lib/graph/version.map @@ -15,6 +15,8 @@ EXPERIMENTAL { rte_graph_list_dump; rte_graph_max_count; rte_graph_model_is_valid; + rte_graph_model_mcore_dispatch_core_bind; + rte_graph_model_mcore_dispatch_core_unbind; rte_graph_model_mcore_dispatch_node_lcore_affinity_set; rte_graph_node_get; rte_graph_node_get_by_name; @@ -24,7 +26,6 @@ EXPERIMENTAL { rte_graph_worker_model_no_check_get; rte_graph_worker_model_set; - rte_graph_cluster_stats_create; rte_graph_cluster_stats_destroy; rte_graph_cluster_stats_get; From patchwork Thu Jun 8 15:18:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yan, Zhirun" X-Patchwork-Id: 128431 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A8C4B42C61; Thu, 8 Jun 2023 17:26:35 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CDA0F42D3D; Thu, 8 Jun 2023 17:26:11 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id D34D042D40 for ; Thu, 8 Jun 2023 17:26:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686237970; x=1717773970; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ScSpOVMjpO+88DyUFzdGE1NUCj47I2Su3wngTqH4gb0=; b=CEA+xfjuzn6iINgpJnvw+rEUOl3wNWqZ4iVq0n8nzjt2jcxu3YEMbKcf U9Iq8QtEk+tq/b2Z9hCL/+KopJhRHbxAyEX5gyYLczE5yrgpM0dMT2PA3 IqEa06vn8QqQ6/s15BtWdU9YgH72nXCBHynK33SM+eGsRFxWvM0JhYfFU jkHeAZdfsteiG3M+dQGATtTHLFwuovpDCxryS5/CrLaFihcm7aJ7wrqtj FpYnjQeMGaMKgCBu+Sg3odxJpOKqp+9sBzjt5TkoilKKAmAa1Xh9eC8dJ k/euWrRnEuBS4pZQLV/kmsqvKy1Lfv9QJrme/m5rJtUFHco49Dgfv/Fe0 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="385681433" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="385681433" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Jun 2023 08:26:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="834233000" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="834233000" Received: from dpdk-zhirun-lmm.sh.intel.com ([10.67.119.94]) by orsmga004.jf.intel.com with ESMTP; 08 Jun 2023 08:26:04 -0700 From: Zhirun Yan To: dev@dpdk.org, jerinj@marvell.com, kirankumark@marvell.com, ndabilpuram@marvell.com, stephen@networkplumber.org, pbhagavatula@marvell.com, jerinjacobk@gmail.com Cc: cunming.liang@intel.com, haiyue.wang@intel.com, mattias.ronnblom@ericsson.com, Zhirun Yan Subject: [PATCH v11 07/16] graph: move node clone name func into private as common Date: Thu, 8 Jun 2023 23:18:35 +0800 Message-Id: <20230608151844.1823783-8-zhirun.yan@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230608151844.1823783-1-zhirun.yan@intel.com> References: <20230608095759.1800617-1-zhirun.yan@intel.com> <20230608151844.1823783-1-zhirun.yan@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Move clone_name() into graph_private.h as a common function for both node and graph to naming a new cloned object. Signed-off-by: Haiyue Wang Signed-off-by: Cunming Liang Signed-off-by: Zhirun Yan Acked-by: Jerin Jacob --- lib/graph/graph_private.h | 41 +++++++++++++++++++++++++++++++++++++++ lib/graph/node.c | 26 +------------------------ 2 files changed, 42 insertions(+), 25 deletions(-) diff --git a/lib/graph/graph_private.h b/lib/graph/graph_private.h index 6d2137c81b..a6d8c6e98b 100644 --- a/lib/graph/graph_private.h +++ b/lib/graph/graph_private.h @@ -11,6 +11,8 @@ #include #include #include +#include +#include #include "rte_graph.h" #include "rte_graph_worker.h" @@ -114,6 +116,45 @@ struct graph { /**< Nodes in a graph. */ }; +/* Node and graph common functions */ +/** + * @internal + * + * Naming a cloned graph or node by appending a string to base name. + * + * @param new_name + * Pointer to the name of the cloned object. + * @param base_name + * Pointer to the name of original object. + * @param append_str + * Pointer to the appended string. + * + * @return + * 0 on success, negative errno value otherwise. + */ +static inline int clone_name(char *new_name, char *base_name, const char *append_str) +{ + ssize_t sz, rc; + +#define SZ RTE_MIN(RTE_NODE_NAMESIZE, RTE_GRAPH_NAMESIZE) + rc = rte_strscpy(new_name, base_name, SZ); + if (rc < 0) + goto fail; + sz = rc; + rc = rte_strscpy(new_name + sz, "-", RTE_MAX((int16_t)(SZ - sz), 0)); + if (rc < 0) + goto fail; + sz += rc; + sz = rte_strscpy(new_name + sz, append_str, RTE_MAX((int16_t)(SZ - sz), 0)); + if (sz < 0) + goto fail; + + return 0; +fail: + rte_errno = E2BIG; + return -rte_errno; +} + /* Node functions */ STAILQ_HEAD(node_head, node); diff --git a/lib/graph/node.c b/lib/graph/node.c index 339b4a0da5..99a9622779 100644 --- a/lib/graph/node.c +++ b/lib/graph/node.c @@ -115,30 +115,6 @@ __rte_node_register(const struct rte_node_register *reg) return RTE_NODE_ID_INVALID; } -static int -clone_name(struct rte_node_register *reg, struct node *node, const char *name) -{ - ssize_t sz, rc; - -#define SZ RTE_NODE_NAMESIZE - rc = rte_strscpy(reg->name, node->name, SZ); - if (rc < 0) - goto fail; - sz = rc; - rc = rte_strscpy(reg->name + sz, "-", RTE_MAX((int16_t)(SZ - sz), 0)); - if (rc < 0) - goto fail; - sz += rc; - sz = rte_strscpy(reg->name + sz, name, RTE_MAX((int16_t)(SZ - sz), 0)); - if (sz < 0) - goto fail; - - return 0; -fail: - rte_errno = E2BIG; - return -rte_errno; -} - static rte_node_t node_clone(struct node *node, const char *name) { @@ -170,7 +146,7 @@ node_clone(struct node *node, const char *name) reg->next_nodes[i] = node->next_nodes[i]; /* Naming ceremony of the new node. name is node->name + "-" + name */ - if (clone_name(reg, node, name)) + if (clone_name(reg->name, node->name, name)) goto free; rc = __rte_node_register(reg); From patchwork Thu Jun 8 15:18:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yan, Zhirun" X-Patchwork-Id: 128432 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C20FC42C61; Thu, 8 Jun 2023 17:26:41 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DF03342D40; Thu, 8 Jun 2023 17:26:15 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id 69BD442D40 for ; Thu, 8 Jun 2023 17:26:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686237974; x=1717773974; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ugiSbSmNm/W3sfN8yPSwlWYf2h565KfAoikIH2mKWVY=; b=jiUnLginr9u+6fcwingE6mjLx2WBKiEBrl1o9cCxtVw8yyqgTNbvwikC /3BFmTe8yVI18buIbvS+C/aJUO5S8NryyqFZCOwwuHerQTv27FJxNyXwU tuQ35+eKhE9yEbfco93XTkrT8eFrZg0VJ7qvkDtWrZHPFwCkukZ9mwYQR QkCBtuhiC2XrPFYXOijT3jdqxUlX5p/5mpWPsG3XaSxG4owwGvarJs81X 0LVKnnVZR9/SURXcORb9PC6xrywtOI/+0Ex9IY/V10rfBfzl6YGRSnGs+ sVT//4exmZiPO1zCvPpClr3y5FmgiL7UkNeOZZO8Xh6NqaFDywmJeml42 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="385681448" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="385681448" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Jun 2023 08:26:10 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="834233024" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="834233024" Received: from dpdk-zhirun-lmm.sh.intel.com ([10.67.119.94]) by orsmga004.jf.intel.com with ESMTP; 08 Jun 2023 08:26:07 -0700 From: Zhirun Yan To: dev@dpdk.org, jerinj@marvell.com, kirankumark@marvell.com, ndabilpuram@marvell.com, stephen@networkplumber.org, pbhagavatula@marvell.com, jerinjacobk@gmail.com Cc: cunming.liang@intel.com, haiyue.wang@intel.com, mattias.ronnblom@ericsson.com, Zhirun Yan Subject: [PATCH v11 08/16] graph: introduce graph clone API for other worker core Date: Thu, 8 Jun 2023 23:18:36 +0800 Message-Id: <20230608151844.1823783-9-zhirun.yan@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230608151844.1823783-1-zhirun.yan@intel.com> References: <20230608095759.1800617-1-zhirun.yan@intel.com> <20230608151844.1823783-1-zhirun.yan@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds graph API for supporting to clone the graph object for a specified worker core. The new graph will also clone all nodes. Signed-off-by: Haiyue Wang Signed-off-by: Cunming Liang Signed-off-by: Zhirun Yan Acked-by: Jerin Jacob --- lib/graph/graph.c | 89 +++++++++++++++++++++++++++++++++++++++ lib/graph/graph_private.h | 2 + lib/graph/rte_graph.h | 20 +++++++++ lib/graph/version.map | 1 + 4 files changed, 112 insertions(+) diff --git a/lib/graph/graph.c b/lib/graph/graph.c index 8d5bd8b9ae..1b34f0e543 100644 --- a/lib/graph/graph.c +++ b/lib/graph/graph.c @@ -405,6 +405,7 @@ rte_graph_create(const char *name, struct rte_graph_param *prm) graph->src_node_count = src_node_count; graph->node_count = graph_nodes_count(graph); graph->id = graph_id; + graph->parent_id = RTE_GRAPH_ID_INVALID; graph->lcore_id = RTE_MAX_LCORE; graph->num_pkt_to_capture = prm->num_pkt_to_capture; if (prm->pcap_filename) @@ -469,6 +470,94 @@ rte_graph_destroy(rte_graph_t id) return rc; } +static rte_graph_t +graph_clone(struct graph *parent_graph, const char *name) +{ + struct graph_node *graph_node; + struct graph *graph; + + graph_spinlock_lock(); + + /* Don't allow to clone a node from a cloned graph */ + if (parent_graph->parent_id != RTE_GRAPH_ID_INVALID) + SET_ERR_JMP(EEXIST, fail, "A cloned graph is not allowed to be cloned"); + + /* Create graph object */ + graph = calloc(1, sizeof(*graph)); + if (graph == NULL) + SET_ERR_JMP(ENOMEM, fail, "Failed to calloc cloned graph object"); + + /* Naming ceremony of the new graph. name is node->name + "-" + name */ + if (clone_name(graph->name, parent_graph->name, name)) + goto free; + + /* Check for existence of duplicate graph */ + if (rte_graph_from_name(graph->name) != RTE_GRAPH_ID_INVALID) + SET_ERR_JMP(EEXIST, free, "Found duplicate graph %s", + graph->name); + + /* Clone nodes from parent graph firstly */ + STAILQ_INIT(&graph->node_list); + STAILQ_FOREACH(graph_node, &parent_graph->node_list, next) { + if (graph_node_add(graph, graph_node->node)) + goto graph_cleanup; + } + + /* Just update adjacency list of all nodes in the graph */ + if (graph_adjacency_list_update(graph)) + goto graph_cleanup; + + /* Initialize the graph object */ + graph->src_node_count = parent_graph->src_node_count; + graph->node_count = parent_graph->node_count; + graph->parent_id = parent_graph->id; + graph->lcore_id = parent_graph->lcore_id; + graph->socket = parent_graph->socket; + graph->id = graph_id; + + /* Allocate the Graph fast path memory and populate the data */ + if (graph_fp_mem_create(graph)) + goto graph_cleanup; + + /* Clone the graph model */ + graph->graph->model = parent_graph->graph->model; + + /* Call init() of the all the nodes in the graph */ + if (graph_node_init(graph)) + goto graph_mem_destroy; + + /* All good, Lets add the graph to the list */ + graph_id++; + STAILQ_INSERT_TAIL(&graph_list, graph, next); + + graph_spinlock_unlock(); + return graph->id; + +graph_mem_destroy: + graph_fp_mem_destroy(graph); +graph_cleanup: + graph_cleanup(graph); +free: + free(graph); +fail: + graph_spinlock_unlock(); + return RTE_GRAPH_ID_INVALID; +} + +rte_graph_t +rte_graph_clone(rte_graph_t id, const char *name) +{ + struct graph *graph; + + GRAPH_ID_CHECK(id); + STAILQ_FOREACH(graph, &graph_list, next) + if (graph->id == id) + return graph_clone(graph, name); + +fail: + return RTE_GRAPH_ID_INVALID; +} + rte_graph_t rte_graph_from_name(const char *name) { diff --git a/lib/graph/graph_private.h b/lib/graph/graph_private.h index a6d8c6e98b..354dc8ac0a 100644 --- a/lib/graph/graph_private.h +++ b/lib/graph/graph_private.h @@ -102,6 +102,8 @@ struct graph { /**< Circular buffer mask for wrap around. */ rte_graph_t id; /**< Graph identifier. */ + rte_graph_t parent_id; + /**< Parent graph identifier. */ unsigned int lcore_id; /**< Lcore identifier where the graph prefer to run on. Used for mcore dispatch model. */ size_t mem_sz; diff --git a/lib/graph/rte_graph.h b/lib/graph/rte_graph.h index f70c694e77..998cade200 100644 --- a/lib/graph/rte_graph.h +++ b/lib/graph/rte_graph.h @@ -247,6 +247,26 @@ rte_graph_t rte_graph_create(const char *name, struct rte_graph_param *prm); __rte_experimental int rte_graph_destroy(rte_graph_t id); +/** + * Clone Graph. + * + * Clone a graph from static graph (graph created from rte_graph_create()). And + * all cloned graphs attached to the parent graph MUST be destroyed together + * for fast schedule design limitation (stop ALL graph walk firstly). + * + * @param id + * Static graph id to clone from. + * @param name + * Name of the new graph. The library prepends the parent graph name to the + * user-specified name. The final graph name will be, + * "parent graph name" + "-" + name. + * + * @return + * Valid graph id on success, RTE_GRAPH_ID_INVALID otherwise. + */ +__rte_experimental +rte_graph_t rte_graph_clone(rte_graph_t id, const char *name); + /** * Get graph id from graph name. * diff --git a/lib/graph/version.map b/lib/graph/version.map index 9a20dba5e7..9e92b54ffa 100644 --- a/lib/graph/version.map +++ b/lib/graph/version.map @@ -5,6 +5,7 @@ EXPERIMENTAL { __rte_node_stream_alloc; __rte_node_stream_alloc_size; + rte_graph_clone; rte_graph_create; rte_graph_destroy; rte_graph_dump; From patchwork Thu Jun 8 15:18:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yan, Zhirun" X-Patchwork-Id: 128433 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 403F242C61; Thu, 8 Jun 2023 17:26:47 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 022B742D53; Thu, 8 Jun 2023 17:26:18 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id C46C842D4D for ; Thu, 8 Jun 2023 17:26:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686237976; x=1717773976; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GJ06n0LTszgKL5Nztkhdan2IuWBp88fFaxq7Yu+WH6s=; b=DV/mlhWN6EZI7QjkFbxss183Z3JjzkXXtViazhQ8dYjyyhwcTwi3GBKW K9/zJ1XQNM8VPeADXtjwwAszRy0C7HD7H+0/bgQikR4GmfOJXm4Y+80x0 hAIP1FkyKeMb2kWNiA3QtTNsP9dHRSdldsT1imKHmZJxUjzyL/o4PxXHj PuGaJQoPEdKN/KvjNRkIkqvadDezPqMPCSDLtNU+FAxK3Gt9tlrcFJMAr GZ9B+TbPj2R6YwD5KqbiBDn7wyoNqOsbz2U6gjNXrk5eOau7+vYkYuXtw B/e2bbcuI32yUwWwUoma2e5grSWIph8HtWrWXMvp4EuMlknbYz+vw3pPF w==; X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="385681472" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="385681472" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Jun 2023 08:26:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="834233047" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="834233047" Received: from dpdk-zhirun-lmm.sh.intel.com ([10.67.119.94]) by orsmga004.jf.intel.com with ESMTP; 08 Jun 2023 08:26:10 -0700 From: Zhirun Yan To: dev@dpdk.org, jerinj@marvell.com, kirankumark@marvell.com, ndabilpuram@marvell.com, stephen@networkplumber.org, pbhagavatula@marvell.com, jerinjacobk@gmail.com Cc: cunming.liang@intel.com, haiyue.wang@intel.com, mattias.ronnblom@ericsson.com, Zhirun Yan Subject: [PATCH v11 09/16] graph: add structure for stream moving between cores Date: Thu, 8 Jun 2023 23:18:37 +0800 Message-Id: <20230608151844.1823783-10-zhirun.yan@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230608151844.1823783-1-zhirun.yan@intel.com> References: <20230608095759.1800617-1-zhirun.yan@intel.com> <20230608151844.1823783-1-zhirun.yan@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add graph_mcore_dispatch_wq_node to hold graph scheduling workqueue node. Signed-off-by: Haiyue Wang Signed-off-by: Cunming Liang Signed-off-by: Zhirun Yan --- lib/graph/graph.c | 2 ++ lib/graph/graph_populate.c | 1 + lib/graph/graph_private.h | 12 ++++++++++++ lib/graph/rte_graph_worker_common.h | 29 +++++++++++++++++++++++++++++ 4 files changed, 44 insertions(+) diff --git a/lib/graph/graph.c b/lib/graph/graph.c index 1b34f0e543..968cbbf86c 100644 --- a/lib/graph/graph.c +++ b/lib/graph/graph.c @@ -291,6 +291,7 @@ rte_graph_model_mcore_dispatch_core_bind(rte_graph_t id, int lcore) goto fail; graph->lcore_id = lcore; + graph->graph->dispatch.lcore_id = graph->lcore_id; graph->socket = rte_lcore_to_socket_id(lcore); /* check the availability of source node */ @@ -314,6 +315,7 @@ rte_graph_model_mcore_dispatch_core_unbind(rte_graph_t id) break; graph->lcore_id = RTE_MAX_LCORE; + graph->graph->dispatch.lcore_id = RTE_MAX_LCORE; fail: return; diff --git a/lib/graph/graph_populate.c b/lib/graph/graph_populate.c index 2c0844ce92..ed596a7711 100644 --- a/lib/graph/graph_populate.c +++ b/lib/graph/graph_populate.c @@ -89,6 +89,7 @@ graph_nodes_populate(struct graph *_graph) } node->id = graph_node->node->id; node->parent_id = pid; + node->dispatch.lcore_id = graph_node->node->lcore_id; nb_edges = graph_node->node->nb_edges; node->nb_edges = nb_edges; off += sizeof(struct rte_node); diff --git a/lib/graph/graph_private.h b/lib/graph/graph_private.h index 354dc8ac0a..d84174b667 100644 --- a/lib/graph/graph_private.h +++ b/lib/graph/graph_private.h @@ -64,6 +64,18 @@ struct node { char next_nodes[][RTE_NODE_NAMESIZE]; /**< Names of next nodes. */ }; +/** + * @internal + * + * Structure that holds the graph scheduling workqueue node stream. + * Used for mcore dispatch model. + */ +struct graph_mcore_dispatch_wq_node { + rte_graph_off_t node_off; + uint16_t nb_objs; + void *objs[RTE_GRAPH_BURST_SIZE]; +} __rte_cache_aligned; + /** * @internal * diff --git a/lib/graph/rte_graph_worker_common.h b/lib/graph/rte_graph_worker_common.h index 3a32001e35..9e519b4d9d 100644 --- a/lib/graph/rte_graph_worker_common.h +++ b/lib/graph/rte_graph_worker_common.h @@ -36,12 +36,20 @@ extern "C" { /**< Dispatch model to support cross-core dispatching within core affinity. */ #define RTE_GRAPH_MODEL_DEFAULT RTE_GRAPH_MODEL_RTC /**< Default graph model. */ +/** + * @internal + * + * Singly-linked list head for graph schedule run-queue. + */ +SLIST_HEAD(rte_graph_rq_head, rte_graph); + /** * @internal * * Data structure to hold graph data. */ struct rte_graph { + /* Fast path area. */ uint32_t tail; /**< Tail of circular buffer. */ uint32_t head; /**< Head of circular buffer. */ uint32_t cir_mask; /**< Circular buffer wrap around mask. */ @@ -51,6 +59,20 @@ struct rte_graph { uint8_t model; /**< graph model */ uint8_t reserved1; /**< Reserved for future use. */ uint16_t reserved2; /**< Reserved for future use. */ + RTE_STD_C11 + union { + /* Fast schedule area for mcore dispatch model */ + struct { + struct rte_graph_rq_head *rq __rte_cache_aligned; /* The run-queue */ + struct rte_graph_rq_head rq_head; /* The head for run-queue list */ + + unsigned int lcore_id; /**< The graph running Lcore. */ + struct rte_ring *wq; /**< The work-queue for pending streams. */ + struct rte_mempool *mp; /**< The mempool for scheduling streams. */ + } dispatch; /** Only used by dispatch model */ + }; + SLIST_ENTRY(rte_graph) next; /* The next for rte_graph list */ + /* End of Fast path area.*/ rte_graph_t id; /**< Graph identifier. */ int socket; /**< Socket ID where memory is allocated. */ char name[RTE_GRAPH_NAMESIZE]; /**< Name of the graph. */ @@ -83,6 +105,13 @@ struct rte_node { /** Original process function when pcap is enabled. */ rte_node_process_t original_process; + RTE_STD_C11 + union { + /* Fast schedule area for mcore dispatch model */ + struct { + unsigned int lcore_id; /**< Node running lcore. */ + } dispatch; + }; /* Fast path area */ #define RTE_NODE_CTX_SZ 16 uint8_t ctx[RTE_NODE_CTX_SZ] __rte_cache_aligned; /**< Node Context. */ From patchwork Thu Jun 8 15:18:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yan, Zhirun" X-Patchwork-Id: 128434 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2BC1A42C61; Thu, 8 Jun 2023 17:26:53 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 104EA42D20; Thu, 8 Jun 2023 17:26:20 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id 3118942D51 for ; Thu, 8 Jun 2023 17:26:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686237978; x=1717773978; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=LEUCKUP7up/Uc/vD00veRF3W1Nw0GJubk4khQlc5XI4=; b=E8M5zloBBm5FrtQa4uXyJErLTaaL8raawVhaVgaFTdo2eSZFbK8r2Nmx 1gwQnnCvAgfKQ8ccZiRuUfYyD3a528yGWFjHsXcbGTshVANj2H/uR5sdY raY5n0lg+oFbbAvcmjwzSNBa93pKLZL5FhHFsaGePCehAIEtsXKMXVBQi lTiW7m9CiH4LVqZ7MSEQtt7I1XLNv0BwpT2sZagA/v95vXFRl844dxFQB 9pR9/nno7QEKSyO4gEKFYX8Krs4S6rLTtbIYghZQnrWJ4kXajAd1dxQiC 6zBknBL+/rI3DT3oraCWjNicn3cbqh77u/diqWM/WPf+Gxt3VpRiiyq/7 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="385681496" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="385681496" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Jun 2023 08:26:17 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="834233064" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="834233064" Received: from dpdk-zhirun-lmm.sh.intel.com ([10.67.119.94]) by orsmga004.jf.intel.com with ESMTP; 08 Jun 2023 08:26:13 -0700 From: Zhirun Yan To: dev@dpdk.org, jerinj@marvell.com, kirankumark@marvell.com, ndabilpuram@marvell.com, stephen@networkplumber.org, pbhagavatula@marvell.com, jerinjacobk@gmail.com Cc: cunming.liang@intel.com, haiyue.wang@intel.com, mattias.ronnblom@ericsson.com, Zhirun Yan Subject: [PATCH v11 10/16] graph: introduce stream moving cross cores Date: Thu, 8 Jun 2023 23:18:38 +0800 Message-Id: <20230608151844.1823783-11-zhirun.yan@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230608151844.1823783-1-zhirun.yan@intel.com> References: <20230608095759.1800617-1-zhirun.yan@intel.com> <20230608151844.1823783-1-zhirun.yan@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch introduces key functions to allow a worker thread to enable enqueue and move streams of objects to the next nodes over different cores for mcore dispatch model. Signed-off-by: Haiyue Wang Signed-off-by: Cunming Liang Signed-off-by: Zhirun Yan Acked-by: Pavan Nikhilesh --- lib/graph/graph.c | 6 +- lib/graph/graph_private.h | 31 ++++ lib/graph/meson.build | 2 +- lib/graph/rte_graph.h | 15 +- lib/graph/rte_graph_model_mcore_dispatch.c | 158 +++++++++++++++++++++ lib/graph/rte_graph_model_mcore_dispatch.h | 45 ++++++ lib/graph/version.map | 3 + 7 files changed, 255 insertions(+), 5 deletions(-) diff --git a/lib/graph/graph.c b/lib/graph/graph.c index 968cbbf86c..41251e3435 100644 --- a/lib/graph/graph.c +++ b/lib/graph/graph.c @@ -473,7 +473,7 @@ rte_graph_destroy(rte_graph_t id) } static rte_graph_t -graph_clone(struct graph *parent_graph, const char *name) +graph_clone(struct graph *parent_graph, const char *name, struct rte_graph_param *prm) { struct graph_node *graph_node; struct graph *graph; @@ -547,14 +547,14 @@ graph_clone(struct graph *parent_graph, const char *name) } rte_graph_t -rte_graph_clone(rte_graph_t id, const char *name) +rte_graph_clone(rte_graph_t id, const char *name, struct rte_graph_param *prm) { struct graph *graph; GRAPH_ID_CHECK(id); STAILQ_FOREACH(graph, &graph_list, next) if (graph->id == id) - return graph_clone(graph, name); + return graph_clone(graph, name, prm); fail: return RTE_GRAPH_ID_INVALID; diff --git a/lib/graph/graph_private.h b/lib/graph/graph_private.h index d84174b667..d0ef13b205 100644 --- a/lib/graph/graph_private.h +++ b/lib/graph/graph_private.h @@ -414,4 +414,35 @@ void graph_dump(FILE *f, struct graph *g); */ void node_dump(FILE *f, struct node *n); +/** + * @internal + * + * Create the graph schedule work queue for mcore dispatch model. + * All cloned graphs attached to the parent graph MUST be destroyed together + * for fast schedule design limitation. + * + * @param _graph + * The graph object + * @param _parent_graph + * The parent graph object which holds the run-queue head. + * @param prm + * Graph parameter, includes model-specific parameters in this graph. + * + * @return + * - 0: Success. + * - <0: Graph schedule work queue related error. + */ +int graph_sched_wq_create(struct graph *_graph, struct graph *_parent_graph, + struct rte_graph_param *prm); + +/** + * @internal + * + * Destroy the graph schedule work queue for mcore dispatch model. + * + * @param _graph + * The graph object + */ +void graph_sched_wq_destroy(struct graph *_graph); + #endif /* _RTE_GRAPH_PRIVATE_H_ */ diff --git a/lib/graph/meson.build b/lib/graph/meson.build index 0685cf9e72..9d51eabe33 100644 --- a/lib/graph/meson.build +++ b/lib/graph/meson.build @@ -20,4 +20,4 @@ sources = files( ) headers = files('rte_graph.h', 'rte_graph_worker.h') -deps += ['eal', 'pcapng'] +deps += ['eal', 'pcapng', 'mempool', 'ring'] diff --git a/lib/graph/rte_graph.h b/lib/graph/rte_graph.h index 998cade200..2ffee520b1 100644 --- a/lib/graph/rte_graph.h +++ b/lib/graph/rte_graph.h @@ -169,6 +169,17 @@ struct rte_graph_param { bool pcap_enable; /**< Pcap enable. */ uint64_t num_pkt_to_capture; /**< Number of packets to capture. */ char *pcap_filename; /**< Filename in which packets to be captured.*/ + + RTE_STD_C11 + union { + struct { + uint64_t rsvd; /**< Reserved for rtc model. */ + } rtc; + struct { + uint32_t wq_size_max; /**< Maximum size of workqueue for dispatch model. */ + uint32_t mp_capacity; /**< Capacity of memory pool for dispatch model. */ + } dispatch; + }; }; /** @@ -260,12 +271,14 @@ int rte_graph_destroy(rte_graph_t id); * Name of the new graph. The library prepends the parent graph name to the * user-specified name. The final graph name will be, * "parent graph name" + "-" + name. + * @param prm + * Graph parameter, includes model-specific parameters in this graph. * * @return * Valid graph id on success, RTE_GRAPH_ID_INVALID otherwise. */ __rte_experimental -rte_graph_t rte_graph_clone(rte_graph_t id, const char *name); +rte_graph_t rte_graph_clone(rte_graph_t id, const char *name, struct rte_graph_param *prm); /** * Get graph id from graph name. diff --git a/lib/graph/rte_graph_model_mcore_dispatch.c b/lib/graph/rte_graph_model_mcore_dispatch.c index 9df2479a10..8f4bc860ab 100644 --- a/lib/graph/rte_graph_model_mcore_dispatch.c +++ b/lib/graph/rte_graph_model_mcore_dispatch.c @@ -5,6 +5,164 @@ #include "graph_private.h" #include "rte_graph_model_mcore_dispatch.h" +int +graph_sched_wq_create(struct graph *_graph, struct graph *_parent_graph, + struct rte_graph_param *prm) +{ + struct rte_graph *parent_graph = _parent_graph->graph; + struct rte_graph *graph = _graph->graph; + unsigned int wq_size; + unsigned int flags = RING_F_SC_DEQ; + + wq_size = GRAPH_SCHED_WQ_SIZE(graph->nb_nodes); + wq_size = rte_align32pow2(wq_size + 1); + + if (prm->dispatch.wq_size_max > 0) + wq_size = wq_size <= (prm->dispatch.wq_size_max) ? wq_size : + prm->dispatch.wq_size_max; + + if (!rte_is_power_of_2(wq_size)) + flags |= RING_F_EXACT_SZ; + + graph->dispatch.wq = rte_ring_create(graph->name, wq_size, graph->socket, + flags); + if (graph->dispatch.wq == NULL) + SET_ERR_JMP(EIO, fail, "Failed to allocate graph WQ"); + + if (prm->dispatch.mp_capacity > 0) + wq_size = (wq_size <= prm->dispatch.mp_capacity) ? wq_size : + prm->dispatch.mp_capacity; + + graph->dispatch.mp = rte_mempool_create(graph->name, wq_size, + sizeof(struct graph_mcore_dispatch_wq_node), + 0, 0, NULL, NULL, NULL, NULL, + graph->socket, MEMPOOL_F_SP_PUT); + if (graph->dispatch.mp == NULL) + SET_ERR_JMP(EIO, fail_mp, + "Failed to allocate graph WQ schedule entry"); + + graph->dispatch.lcore_id = _graph->lcore_id; + + if (parent_graph->dispatch.rq == NULL) { + parent_graph->dispatch.rq = &parent_graph->dispatch.rq_head; + SLIST_INIT(parent_graph->dispatch.rq); + } + + graph->dispatch.rq = parent_graph->dispatch.rq; + SLIST_INSERT_HEAD(graph->dispatch.rq, graph, next); + + return 0; + +fail_mp: + rte_ring_free(graph->dispatch.wq); + graph->dispatch.wq = NULL; +fail: + return -rte_errno; +} + +void +graph_sched_wq_destroy(struct graph *_graph) +{ + struct rte_graph *graph = _graph->graph; + + if (graph == NULL) + return; + + rte_ring_free(graph->dispatch.wq); + graph->dispatch.wq = NULL; + + rte_mempool_free(graph->dispatch.mp); + graph->dispatch.mp = NULL; +} + +static __rte_always_inline bool +__graph_sched_node_enqueue(struct rte_node *node, struct rte_graph *graph) +{ + struct graph_mcore_dispatch_wq_node *wq_node; + uint16_t off = 0; + uint16_t size; + +submit_again: + if (rte_mempool_get(graph->dispatch.mp, (void **)&wq_node) < 0) + goto fallback; + + size = RTE_MIN(node->idx, RTE_DIM(wq_node->objs)); + wq_node->node_off = node->off; + wq_node->nb_objs = size; + rte_memcpy(wq_node->objs, &node->objs[off], size * sizeof(void *)); + + while (rte_ring_mp_enqueue_bulk_elem(graph->dispatch.wq, (void *)&wq_node, + sizeof(wq_node), 1, NULL) == 0) + rte_pause(); + + off += size; + node->idx -= size; + if (node->idx > 0) + goto submit_again; + + return true; + +fallback: + if (off != 0) + memmove(&node->objs[0], &node->objs[off], + node->idx * sizeof(void *)); + + return false; +} + +bool __rte_noinline +__rte_graph_mcore_dispatch_sched_node_enqueue(struct rte_node *node, + struct rte_graph_rq_head *rq) +{ + const unsigned int lcore_id = node->dispatch.lcore_id; + struct rte_graph *graph; + + SLIST_FOREACH(graph, rq, next) + if (graph->dispatch.lcore_id == lcore_id) + break; + + return graph != NULL ? __graph_sched_node_enqueue(node, graph) : false; +} + +void +__rte_graph_mcore_dispatch_sched_wq_process(struct rte_graph *graph) +{ +#define WQ_SZ 32 + struct graph_mcore_dispatch_wq_node *wq_node; + struct rte_mempool *mp = graph->dispatch.mp; + struct rte_ring *wq = graph->dispatch.wq; + uint16_t idx, free_space; + struct rte_node *node; + unsigned int i, n; + struct graph_mcore_dispatch_wq_node *wq_nodes[WQ_SZ]; + + n = rte_ring_sc_dequeue_burst_elem(wq, wq_nodes, sizeof(wq_nodes[0]), + RTE_DIM(wq_nodes), NULL); + if (n == 0) + return; + + for (i = 0; i < n; i++) { + wq_node = wq_nodes[i]; + node = RTE_PTR_ADD(graph, wq_node->node_off); + RTE_ASSERT(node->fence == RTE_GRAPH_FENCE); + idx = node->idx; + free_space = node->size - idx; + + if (unlikely(free_space < wq_node->nb_objs)) + __rte_node_stream_alloc_size(graph, node, node->size + wq_node->nb_objs); + + memmove(&node->objs[idx], wq_node->objs, wq_node->nb_objs * sizeof(void *)); + node->idx = idx + wq_node->nb_objs; + + __rte_node_process(graph, node); + + wq_node->nb_objs = 0; + node->idx = 0; + } + + rte_mempool_put_bulk(mp, (void **)wq_nodes, n); +} + int rte_graph_model_mcore_dispatch_node_lcore_affinity_set(const char *name, unsigned int lcore_id) { diff --git a/lib/graph/rte_graph_model_mcore_dispatch.h b/lib/graph/rte_graph_model_mcore_dispatch.h index 7da0483d13..6163f96c37 100644 --- a/lib/graph/rte_graph_model_mcore_dispatch.h +++ b/lib/graph/rte_graph_model_mcore_dispatch.h @@ -20,8 +20,53 @@ extern "C" { #endif +#include +#include +#include +#include + #include "rte_graph_worker_common.h" +#define GRAPH_SCHED_WQ_SIZE_MULTIPLIER 8 +#define GRAPH_SCHED_WQ_SIZE(nb_nodes) \ + ((typeof(nb_nodes))((nb_nodes) * GRAPH_SCHED_WQ_SIZE_MULTIPLIER)) + +/** + * @internal + * + * Schedule the node to the right graph's work queue for mcore dispatch model. + * + * @param node + * Pointer to the scheduled node object. + * @param rq + * Pointer to the scheduled run-queue for all graphs. + * + * @return + * True on success, false otherwise. + * + * @note + * This implementation is used by mcore dispatch model only and user application + * should not call it directly. + */ +__rte_experimental +bool __rte_noinline __rte_graph_mcore_dispatch_sched_node_enqueue(struct rte_node *node, + struct rte_graph_rq_head *rq); + +/** + * @internal + * + * Process all nodes (streams) in the graph's work queue for mcore dispatch model. + * + * @param graph + * Pointer to the graph object. + * + * @note + * This implementation is used by mcore dispatch model only and user application + * should not call it directly. + */ +__rte_experimental +void __rte_graph_mcore_dispatch_sched_wq_process(struct rte_graph *graph); + /** * Set lcore affinity with the node used for mcore dispatch model. * diff --git a/lib/graph/version.map b/lib/graph/version.map index 9e92b54ffa..7e985d6308 100644 --- a/lib/graph/version.map +++ b/lib/graph/version.map @@ -1,6 +1,9 @@ EXPERIMENTAL { global: + __rte_graph_mcore_dispatch_sched_node_enqueue; + __rte_graph_mcore_dispatch_sched_wq_process; + __rte_node_register; __rte_node_stream_alloc; __rte_node_stream_alloc_size; From patchwork Thu Jun 8 15:18:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yan, Zhirun" X-Patchwork-Id: 128435 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 893DC42C61; Thu, 8 Jun 2023 17:27:02 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B55AF427E9; Thu, 8 Jun 2023 17:26:45 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id DC350410FA for ; Thu, 8 Jun 2023 17:26:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686238004; x=1717774004; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=zKZLeIHr9FeSeOY0Lc9P3dlFxlXVbvZx6tLL2HmUpgQ=; b=PMFbavlT/6iqocpxHwyfrO99GNtkDtEckylePqOUFE14Bdc7FIYrs3xV V8SGW7qLk+ZvzDJ1RsaMlR3cSlBdhHvK1OLnrWPO8f28Z9mQBbkHWzf1Y sQu5Vg0YrryamcCpmAManFLVXMAdAjEBRd2d6wPhuVsZH1CYOSzKZpP9o +oa8MWPNXQeTbyUEUkv0Av/Ik1Rhnhyd3jl8cUItUYx6zo1OSa/lB43gD GQaHGsKPHgBeVDJ22CSWrIZGoQOtANxcDEOfhrMjDD+qOpxYw39NF+JeO 3miQYnEC+LO+fw/4V0rLl3raDsJSvMIrgC0if4DwY/8paO07PijIZd+Xy A==; X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="385681522" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="385681522" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Jun 2023 08:26:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="834233074" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="834233074" Received: from dpdk-zhirun-lmm.sh.intel.com ([10.67.119.94]) by orsmga004.jf.intel.com with ESMTP; 08 Jun 2023 08:26:17 -0700 From: Zhirun Yan To: dev@dpdk.org, jerinj@marvell.com, kirankumark@marvell.com, ndabilpuram@marvell.com, stephen@networkplumber.org, pbhagavatula@marvell.com, jerinjacobk@gmail.com Cc: cunming.liang@intel.com, haiyue.wang@intel.com, mattias.ronnblom@ericsson.com, Zhirun Yan Subject: [PATCH v11 11/16] graph: enable create and destroy graph scheduling workqueue Date: Thu, 8 Jun 2023 23:18:39 +0800 Message-Id: <20230608151844.1823783-12-zhirun.yan@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230608151844.1823783-1-zhirun.yan@intel.com> References: <20230608095759.1800617-1-zhirun.yan@intel.com> <20230608151844.1823783-1-zhirun.yan@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch enables to create and destroy scheduling workqueue into common graph operations. Signed-off-by: Haiyue Wang Signed-off-by: Cunming Liang Signed-off-by: Zhirun Yan Acked-by: Pavan Nikhilesh --- lib/graph/graph.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/lib/graph/graph.c b/lib/graph/graph.c index 41251e3435..0c28d925bc 100644 --- a/lib/graph/graph.c +++ b/lib/graph/graph.c @@ -451,6 +451,11 @@ rte_graph_destroy(rte_graph_t id) while (graph != NULL) { tmp = STAILQ_NEXT(graph, next); if (graph->id == id) { + /* Destroy the schedule work queue if has */ + if (rte_graph_worker_model_get(graph->graph) == + RTE_GRAPH_MODEL_MCORE_DISPATCH) + graph_sched_wq_destroy(graph); + /* Call fini() of the all the nodes in the graph */ graph_node_fini(graph); /* Destroy graph fast path memory */ @@ -524,6 +529,11 @@ graph_clone(struct graph *parent_graph, const char *name, struct rte_graph_param /* Clone the graph model */ graph->graph->model = parent_graph->graph->model; + /* Create the graph schedule work queue */ + if (rte_graph_worker_model_get(graph->graph) == RTE_GRAPH_MODEL_MCORE_DISPATCH && + graph_sched_wq_create(graph, parent_graph, prm)) + goto graph_mem_destroy; + /* Call init() of the all the nodes in the graph */ if (graph_node_init(graph)) goto graph_mem_destroy; From patchwork Thu Jun 8 15:18:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yan, Zhirun" X-Patchwork-Id: 128436 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 968FF42C61; Thu, 8 Jun 2023 17:27:08 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id ED6A942D13; Thu, 8 Jun 2023 17:26:47 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id 526D641148 for ; Thu, 8 Jun 2023 17:26:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686238005; x=1717774005; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=mr9iwvNxhZ7hGShL923CB2ZRxK/tbZrY7A7RG2Y8TbA=; b=jE4tB0Af+lHZ7r7L2/yYz1nore7m2vLcZl8e8ZvosrEtOY2KFNAI6BC8 W+mcB/XrIOlDAr0MHc/efC3sZkAr4CY6CRIzlSiBRtQpsI9Qxm7Lg2W3g L7EOM5IYJXANhxoty3BU2AC5QZ/xFqLUC6RBHQCY+/WfMU+Px60zesxUn Rlq4h6mjcSD1fFubQYvvw7MvVqKMaHUqxXHzFg+4n1Yvu3YerQuyjk/yU OmVvdqLdkTukptbrg3/83v3Iu4JL9bma8AdovRynwCjdPpymYFKbKlRoC ptnz3PsWY4UfbfXEsVvQS4DoDYb10ciorFUafgNzjyP/xL1+/z+RLjFNE A==; X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="385681549" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="385681549" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Jun 2023 08:26:23 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="834233087" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="834233087" Received: from dpdk-zhirun-lmm.sh.intel.com ([10.67.119.94]) by orsmga004.jf.intel.com with ESMTP; 08 Jun 2023 08:26:20 -0700 From: Zhirun Yan To: dev@dpdk.org, jerinj@marvell.com, kirankumark@marvell.com, ndabilpuram@marvell.com, stephen@networkplumber.org, pbhagavatula@marvell.com, jerinjacobk@gmail.com Cc: cunming.liang@intel.com, haiyue.wang@intel.com, mattias.ronnblom@ericsson.com, Zhirun Yan Subject: [PATCH v11 12/16] graph: introduce graph walk by cross-core dispatch Date: Thu, 8 Jun 2023 23:18:40 +0800 Message-Id: <20230608151844.1823783-13-zhirun.yan@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230608151844.1823783-1-zhirun.yan@intel.com> References: <20230608095759.1800617-1-zhirun.yan@intel.com> <20230608151844.1823783-1-zhirun.yan@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch introduces the task scheduler mechanism to enable dispatching tasks to another worker cores. Currently, there is only a local work queue for one graph to walk. We introduce a scheduler worker queue in each worker core for dispatching tasks. It will perform the walk on scheduler work queue first, then handle the local work queue. Signed-off-by: Haiyue Wang Signed-off-by: Cunming Liang Signed-off-by: Zhirun Yan Acked-by: Jerin Jacob Acked-by: Pavan Nikhilesh --- lib/graph/rte_graph_model_mcore_dispatch.h | 44 ++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/lib/graph/rte_graph_model_mcore_dispatch.h b/lib/graph/rte_graph_model_mcore_dispatch.h index 6163f96c37..c78a3bbdf9 100644 --- a/lib/graph/rte_graph_model_mcore_dispatch.h +++ b/lib/graph/rte_graph_model_mcore_dispatch.h @@ -83,6 +83,50 @@ __rte_experimental int rte_graph_model_mcore_dispatch_node_lcore_affinity_set(const char *name, unsigned int lcore_id); +/** + * Perform graph walk on the circular buffer and invoke the process function + * of the nodes and collect the stats. + * + * @param graph + * Graph pointer returned from rte_graph_lookup function. + * + * @see rte_graph_lookup() + */ +__rte_experimental +static inline void +rte_graph_walk_mcore_dispatch(struct rte_graph *graph) +{ + const rte_graph_off_t *cir_start = graph->cir_start; + const rte_node_t mask = graph->cir_mask; + uint32_t head = graph->head; + struct rte_node *node; + + RTE_ASSERT(graph->parent_id != RTE_GRAPH_ID_INVALID); + if (graph->dispatch.wq != NULL) + __rte_graph_mcore_dispatch_sched_wq_process(graph); + + while (likely(head != graph->tail)) { + node = (struct rte_node *)RTE_PTR_ADD(graph, cir_start[(int32_t)head++]); + + /* skip the src nodes which not bind with current worker */ + if ((int32_t)head < 0 && node->dispatch.lcore_id != graph->dispatch.lcore_id) + continue; + + /* Schedule the node until all task/objs are done */ + if (node->dispatch.lcore_id != RTE_MAX_LCORE && + graph->dispatch.lcore_id != node->dispatch.lcore_id && + graph->dispatch.rq != NULL && + __rte_graph_mcore_dispatch_sched_node_enqueue(node, graph->dispatch.rq)) + continue; + + __rte_node_process(graph, node); + + head = likely((int32_t)head > 0) ? head & mask : head; + } + + graph->tail = 0; +} + #ifdef __cplusplus } #endif From patchwork Thu Jun 8 15:18:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yan, Zhirun" X-Patchwork-Id: 128437 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BF18942C61; Thu, 8 Jun 2023 17:27:13 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0B5FE42D0B; Thu, 8 Jun 2023 17:26:49 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id 4F9FF42D12 for ; Thu, 8 Jun 2023 17:26:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686238007; x=1717774007; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=PbRAcqku2qwFKjQpRjCZ1/BAXJLK1K/Cj2WDWvQqwQU=; b=Mhbin9ZUTNzC117XUhhXSDtoAAO8KpEpVB4OjCvzbSW9cY9klUIREtA3 3ERuhH9T7uSfUW8SYjv9pXUDCkqrersP/gOwmOpdTjZMJWLbDN/sjY7cD VbnSqUc1m7nr3C7d3LOoOqe1XNco9Dk8yLLQNoEivonsVo0VyojK31Eau bKgfXrDc9X13kx2c/KZ7pQeuKZhgq/1QAHEKa1tlXyjfJknmo/c38IFZA GIS1ExuIFvpood8bnqdlF1CQH1W/PGJMWbmJQ0KXOKatqHG7OPlrIMQPF IuYvmXNrE7QXNqiu/9b1n9HzxWWgGDhaiLS+QiixFwg0ezMghJD9UFaSx g==; X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="385681571" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="385681571" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Jun 2023 08:26:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="834233123" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="834233123" Received: from dpdk-zhirun-lmm.sh.intel.com ([10.67.119.94]) by orsmga004.jf.intel.com with ESMTP; 08 Jun 2023 08:26:23 -0700 From: Zhirun Yan To: dev@dpdk.org, jerinj@marvell.com, kirankumark@marvell.com, ndabilpuram@marvell.com, stephen@networkplumber.org, pbhagavatula@marvell.com, jerinjacobk@gmail.com Cc: cunming.liang@intel.com, haiyue.wang@intel.com, mattias.ronnblom@ericsson.com, Zhirun Yan Subject: [PATCH v11 13/16] graph: enable graph multicore dispatch scheduler model Date: Thu, 8 Jun 2023 23:18:41 +0800 Message-Id: <20230608151844.1823783-14-zhirun.yan@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230608151844.1823783-1-zhirun.yan@intel.com> References: <20230608095759.1800617-1-zhirun.yan@intel.com> <20230608151844.1823783-1-zhirun.yan@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch enables to chose new scheduler model. Must define RTE_GRAPH_MODEL_SELECT before including rte_graph_worker.h to enable specific model choosing. Signed-off-by: Haiyue Wang Signed-off-by: Cunming Liang Signed-off-by: Zhirun Yan Acked-by: Jerin Jacob Acked-by: Pavan Nikhilesh --- doc/guides/prog_guide/graph_lib.rst | 71 ++++++++++++++++++++++++++--- lib/graph/rte_graph_worker.h | 13 ++++++ 2 files changed, 77 insertions(+), 7 deletions(-) diff --git a/doc/guides/prog_guide/graph_lib.rst b/doc/guides/prog_guide/graph_lib.rst index 1cfdc86433..017cc25fd3 100644 --- a/doc/guides/prog_guide/graph_lib.rst +++ b/doc/guides/prog_guide/graph_lib.rst @@ -189,13 +189,70 @@ In the above example, A graph object will be created with ethdev Rx node of port 0 and queue 0, all ipv4* nodes in the system, and ethdev tx node of all ports. -Multicore graph processing -~~~~~~~~~~~~~~~~~~~~~~~~~~ -In the current graph library implementation, specifically, -``rte_graph_walk()`` and ``rte_node_enqueue*`` fast path API functions -are designed to work on single-core to have better performance. -The fast path API works on graph object, So the multi-core graph -processing strategy would be to create graph object PER WORKER. +Graph models +~~~~~~~~~~~~ +There are two different kinds of graph walking models. User can select the model using +``rte_graph_worker_model_set()`` API. If the application decides to use only one model, +the fast path check can be avoided by defining the model with RTE_GRAPH_MODEL_SELECT. +For example: + +.. code-block:: console + +#define RTE_GRAPH_MODEL_SELECT RTE_GRAPH_MODEL_RTC +#include "rte_graph_worker.h" + +RTC (Run-To-Completion) +^^^^^^^^^^^^^^^^^^^^^^^ +This is the default graph walking model. Specifically, ``rte_graph_walk_rtc()`` and +``rte_node_enqueue*`` fast path API functions are designed to work on single-core to +have better performance. The fast path API works on graph object, So the multi-core +graph processing strategy would be to create graph object PER WORKER. + +Example: + +Graph: node-0 -> node-1 -> node-2 @Core0. + +.. code-block:: diff + + + - - - - - - - - - - - - - - - - - - - - - + + ' Core #0 ' + ' ' + ' +--------+ +---------+ +--------+ ' + ' | Node-0 | --> | Node-1 | --> | Node-2 | ' + ' +--------+ +---------+ +--------+ ' + ' ' + + - - - - - - - - - - - - - - - - - - - - - + + +Dispatch model +^^^^^^^^^^^^^^ +The dispatch model enables a cross-core dispatching mechanism which employs +a scheduling work-queue to dispatch streams to other worker cores which +being associated with the destination node. + +Use ``rte_graph_model_mcore_dispatch_lcore_affinity_set()`` to set lcore affinity +with the node. +Each worker core will have a graph repetition. Use ``rte_graph_clone()`` to clone +graph for each worker and use``rte_graph_model_mcore_dispatch_core_bind()`` to +bind graph with the worker core. + +Example: + +Graph topo: node-0 -> Core1; node-1 -> node-2; node-2 -> node-3. +Config graph: node-0 @Core0; node-1/3 @Core1; node-2 @Core2. + +.. code-block:: diff + + + - - - - - -+ +- - - - - - - - - - - - - + + - - - - - -+ + ' Core #0 ' ' Core #1 ' ' Core #2 ' + ' ' ' ' ' ' + ' +--------+ ' ' +--------+ +--------+ ' ' +--------+ ' + ' | Node-0 | - - - ->| Node-1 | | Node-3 |<- - - - | Node-2 | ' + ' +--------+ ' ' +--------+ +--------+ ' ' +--------+ ' + ' ' ' | ' ' ^ ' + + - - - - - -+ +- - -|- - - - - - - - - - + + - - -|- - -+ + | | + + - - - - - - - - - - - - - - - - + + In fast path ~~~~~~~~~~~~ diff --git a/lib/graph/rte_graph_worker.h b/lib/graph/rte_graph_worker.h index 5b58f7bda9..6685600813 100644 --- a/lib/graph/rte_graph_worker.h +++ b/lib/graph/rte_graph_worker.h @@ -11,6 +11,7 @@ extern "C" { #endif #include "rte_graph_model_rtc.h" +#include "rte_graph_model_mcore_dispatch.h" /** * Perform graph walk on the circular buffer and invoke the process function @@ -25,7 +26,19 @@ __rte_experimental static inline void rte_graph_walk(struct rte_graph *graph) { +#if defined(RTE_GRAPH_MODEL_SELECT) && (RTE_GRAPH_MODEL_SELECT == RTE_GRAPH_MODEL_RTC) rte_graph_walk_rtc(graph); +#elif defined(RTE_GRAPH_MODEL_SELECT) && (RTE_GRAPH_MODEL_SELECT == RTE_GRAPH_MODEL_MCORE_DISPATCH) + rte_graph_walk_mcore_dispatch(graph); +#else + switch (rte_graph_worker_model_no_check_get(graph)) { + case RTE_GRAPH_MODEL_MCORE_DISPATCH: + rte_graph_walk_mcore_dispatch(graph); + break; + default: + rte_graph_walk_rtc(graph); + } +#endif } #ifdef __cplusplus From patchwork Thu Jun 8 15:18:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yan, Zhirun" X-Patchwork-Id: 128438 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C301942C61; Thu, 8 Jun 2023 17:27:18 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2CCFB42D0C; Thu, 8 Jun 2023 17:26:51 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id 5CC2542D1D for ; Thu, 8 Jun 2023 17:26:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686238009; x=1717774009; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=/r34ZCpnYWsvz9CuzYxdY/G7+3LC3oCii7re9g6OA2Y=; b=R5miJWbhHoG42pA1aEMqe7AyVZ4ZfSbO7hx+3Uz3IGN1Cof3Ej440fYC UA3KAbjtHIwW2t1ryOZsNBkCT+MoahP0AJrMq/m06FFDwVEo+QslkPjEz qnLrzM2Olhu5zrpcwtR/DWW3/Ojao6MAzpxlZn3OMmaKV7OwEDcUHlvjE jxo59iDQpKU12Lmaiyig0PUF/AAb6Me+RlQwh9Uc0N4B0x+KVz99B5SjM svBhtGQNcKRKie3uvzKytvH/0GVDOwFPWXF9ZHnsOj4j16+IYYPb1wTPY ZSivIGv6FqCIMl6N9cAuuTgxUdrEvAoBzBo3uE/7faRnPomRevdP2e/Xv w==; X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="385681592" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="385681592" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Jun 2023 08:26:29 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="834233137" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="834233137" Received: from dpdk-zhirun-lmm.sh.intel.com ([10.67.119.94]) by orsmga004.jf.intel.com with ESMTP; 08 Jun 2023 08:26:26 -0700 From: Zhirun Yan To: dev@dpdk.org, jerinj@marvell.com, kirankumark@marvell.com, ndabilpuram@marvell.com, stephen@networkplumber.org, pbhagavatula@marvell.com, jerinjacobk@gmail.com Cc: cunming.liang@intel.com, haiyue.wang@intel.com, mattias.ronnblom@ericsson.com, Zhirun Yan Subject: [PATCH v11 14/16] graph: add stats for mcore dispatch model Date: Thu, 8 Jun 2023 23:18:42 +0800 Message-Id: <20230608151844.1823783-15-zhirun.yan@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230608151844.1823783-1-zhirun.yan@intel.com> References: <20230608095759.1800617-1-zhirun.yan@intel.com> <20230608151844.1823783-1-zhirun.yan@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add stats for mcore dispatch model if stats collection is enabled. Signed-off-by: Haiyue Wang Signed-off-by: Cunming Liang Signed-off-by: Zhirun Yan Acked-by: Jerin Jacob Acked-by: Pavan Nikhilesh --- lib/graph/graph_debug.c | 6 ++ lib/graph/graph_stats.c | 76 +++++++++++++++++++--- lib/graph/rte_graph.h | 10 +++ lib/graph/rte_graph_model_mcore_dispatch.c | 3 + lib/graph/rte_graph_worker_common.h | 2 + 5 files changed, 89 insertions(+), 8 deletions(-) diff --git a/lib/graph/graph_debug.c b/lib/graph/graph_debug.c index b84412f5dd..9def3067ec 100644 --- a/lib/graph/graph_debug.c +++ b/lib/graph/graph_debug.c @@ -74,6 +74,12 @@ rte_graph_obj_dump(FILE *f, struct rte_graph *g, bool all) fprintf(f, " size=%d\n", n->size); fprintf(f, " idx=%d\n", n->idx); fprintf(f, " total_objs=%" PRId64 "\n", n->total_objs); + if (rte_graph_worker_model_get(g) == RTE_GRAPH_MODEL_MCORE_DISPATCH) { + fprintf(f, " total_sched_objs=%" PRId64 "\n", + n->dispatch.total_sched_objs); + fprintf(f, " total_sched_fail=%" PRId64 "\n", + n->dispatch.total_sched_fail); + } fprintf(f, " total_calls=%" PRId64 "\n", n->total_calls); for (i = 0; i < n->nb_edges; i++) fprintf(f, " edge[%d] <%s>\n", i, diff --git a/lib/graph/graph_stats.c b/lib/graph/graph_stats.c index c0140ba922..cc32245c05 100644 --- a/lib/graph/graph_stats.c +++ b/lib/graph/graph_stats.c @@ -40,13 +40,19 @@ struct rte_graph_cluster_stats { struct cluster_node clusters[]; } __rte_cache_aligned; +#define boarder_model_dispatch() \ + fprintf(f, "+-------------------------------+---------------+--------" \ + "-------+---------------+---------------+---------------+" \ + "---------------+---------------+-" \ + "----------+\n") + #define boarder() \ fprintf(f, "+-------------------------------+---------------+--------" \ "-------+---------------+---------------+---------------+-" \ "----------+\n") static inline void -print_banner(FILE *f) +print_banner_default(FILE *f) { boarder(); fprintf(f, "%-32s%-16s%-16s%-16s%-16s%-16s%-16s\n", "|Node", "|calls", @@ -55,6 +61,28 @@ print_banner(FILE *f) boarder(); } +static inline void +print_banner_dispatch(FILE *f) +{ + boarder_model_dispatch(); + fprintf(f, "%-32s%-16s%-16s%-16s%-16s%-16s%-16s%-16s%-16s\n", + "|Node", "|calls", + "|objs", "|sched objs", "|sched fail", + "|realloc_count", "|objs/call", "|objs/sec(10E6)", + "|cycles/call|"); + boarder_model_dispatch(); +} + +static inline void +print_banner(FILE *f) +{ + if (rte_graph_worker_model_get(STAILQ_FIRST(graph_list_head_get())->graph) == + RTE_GRAPH_MODEL_MCORE_DISPATCH) + print_banner_dispatch(f); + else + print_banner_default(f); +} + static inline void print_node(FILE *f, const struct rte_graph_cluster_node_stats *stat) { @@ -76,11 +104,22 @@ print_node(FILE *f, const struct rte_graph_cluster_node_stats *stat) objs_per_sec = ts_per_hz ? (objs - prev_objs) / ts_per_hz : 0; objs_per_sec /= 1000000; - fprintf(f, - "|%-31s|%-15" PRIu64 "|%-15" PRIu64 "|%-15" PRIu64 - "|%-15.3f|%-15.6f|%-11.4f|\n", - stat->name, calls, objs, stat->realloc_count, objs_per_call, - objs_per_sec, cycles_per_call); + if (rte_graph_worker_model_get(STAILQ_FIRST(graph_list_head_get())->graph) == + RTE_GRAPH_MODEL_MCORE_DISPATCH) { + fprintf(f, + "|%-31s|%-15" PRIu64 "|%-15" PRIu64 "|%-15" PRIu64 + "|%-15" PRIu64 "|%-15" PRIu64 + "|%-15.3f|%-15.6f|%-11.4f|\n", + stat->name, calls, objs, stat->dispatch.sched_objs, + stat->dispatch.sched_fail, stat->realloc_count, objs_per_call, + objs_per_sec, cycles_per_call); + } else { + fprintf(f, + "|%-31s|%-15" PRIu64 "|%-15" PRIu64 "|%-15" PRIu64 + "|%-15.3f|%-15.6f|%-11.4f|\n", + stat->name, calls, objs, stat->realloc_count, objs_per_call, + objs_per_sec, cycles_per_call); + } } static int @@ -88,13 +127,20 @@ graph_cluster_stats_cb(bool is_first, bool is_last, void *cookie, const struct rte_graph_cluster_node_stats *stat) { FILE *f = cookie; + int model; + + model = rte_graph_worker_model_get(STAILQ_FIRST(graph_list_head_get())->graph); if (unlikely(is_first)) print_banner(f); if (stat->objs) print_node(f, stat); - if (unlikely(is_last)) - boarder(); + if (unlikely(is_last)) { + if (model == RTE_GRAPH_MODEL_MCORE_DISPATCH) + boarder_model_dispatch(); + else + boarder(); + } return 0; }; @@ -333,12 +379,20 @@ cluster_node_arregate_stats(struct cluster_node *cluster) { uint64_t calls = 0, cycles = 0, objs = 0, realloc_count = 0; struct rte_graph_cluster_node_stats *stat = &cluster->stat; + uint64_t sched_objs = 0, sched_fail = 0; struct rte_node *node; rte_node_t count; + int model; + model = rte_graph_worker_model_get(STAILQ_FIRST(graph_list_head_get())->graph); for (count = 0; count < cluster->nb_nodes; count++) { node = cluster->nodes[count]; + if (model == RTE_GRAPH_MODEL_MCORE_DISPATCH) { + sched_objs += node->dispatch.total_sched_objs; + sched_fail += node->dispatch.total_sched_fail; + } + calls += node->total_calls; objs += node->total_objs; cycles += node->total_cycles; @@ -348,6 +402,12 @@ cluster_node_arregate_stats(struct cluster_node *cluster) stat->calls = calls; stat->objs = objs; stat->cycles = cycles; + + if (model == RTE_GRAPH_MODEL_MCORE_DISPATCH) { + stat->dispatch.sched_objs = sched_objs; + stat->dispatch.sched_fail = sched_fail; + } + stat->ts = rte_get_timer_cycles(); stat->realloc_count = realloc_count; } diff --git a/lib/graph/rte_graph.h b/lib/graph/rte_graph.h index 2ffee520b1..28e50e49b8 100644 --- a/lib/graph/rte_graph.h +++ b/lib/graph/rte_graph.h @@ -220,6 +220,16 @@ struct rte_graph_cluster_node_stats { uint64_t prev_objs; /**< Previous number of processed objs. */ uint64_t prev_cycles; /**< Previous number of cycles. */ + RTE_STD_C11 + union { + struct { + uint64_t sched_objs; + /**< Previous number of scheduled objs for dispatch model. */ + uint64_t sched_fail; + /**< Previous number of failed schedule objs for dispatch model. */ + } dispatch; + }; + uint64_t realloc_count; /**< Realloc count. */ rte_node_t id; /**< Node identifier of stats. */ diff --git a/lib/graph/rte_graph_model_mcore_dispatch.c b/lib/graph/rte_graph_model_mcore_dispatch.c index 8f4bc860ab..d1291b8c57 100644 --- a/lib/graph/rte_graph_model_mcore_dispatch.c +++ b/lib/graph/rte_graph_model_mcore_dispatch.c @@ -96,6 +96,7 @@ __graph_sched_node_enqueue(struct rte_node *node, struct rte_graph *graph) rte_pause(); off += size; + node->dispatch.total_sched_objs += size; node->idx -= size; if (node->idx > 0) goto submit_again; @@ -107,6 +108,8 @@ __graph_sched_node_enqueue(struct rte_node *node, struct rte_graph *graph) memmove(&node->objs[0], &node->objs[off], node->idx * sizeof(void *)); + node->dispatch.total_sched_fail += node->idx; + return false; } diff --git a/lib/graph/rte_graph_worker_common.h b/lib/graph/rte_graph_worker_common.h index 9e519b4d9d..d433293d5c 100644 --- a/lib/graph/rte_graph_worker_common.h +++ b/lib/graph/rte_graph_worker_common.h @@ -110,6 +110,8 @@ struct rte_node { /* Fast schedule area for mcore dispatch model */ struct { unsigned int lcore_id; /**< Node running lcore. */ + uint64_t total_sched_objs; /**< Number of objects scheduled. */ + uint64_t total_sched_fail; /**< Number of scheduled failure. */ } dispatch; }; /* Fast path area */ From patchwork Thu Jun 8 15:18:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yan, Zhirun" X-Patchwork-Id: 128439 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BD24242C61; Thu, 8 Jun 2023 17:27:23 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5F9D442D30; Thu, 8 Jun 2023 17:26:54 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id B1CB042D3C for ; Thu, 8 Jun 2023 17:26:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686238012; x=1717774012; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GCHvh/8xaurPU393+p5XZALfiUafmxZqkcyAQ1hhiUI=; b=Xjn6gH72MmD24EQBelxk/wTmANkSOt+BfP0z/Y5MhORshJe1LiBqOhWs 9KzzreYW82MU2BCJTa331/BqAqUZ0ui07IZ5knzmYvbFTAeURStz00L+U ZaqryMPO5xocKZm4lLKPVpXkeZxUwLX4ibz9x1rgtr/2zBb+IJA66S1fO PJ1lrWxJ1DiSfSJv1vyFwiGDgpMkIYxdVQMnLQq30y5Jxny3r7KEVSESF 19FGfC1mss2Wf9S/8EmBoelyzqmjhkO6U6mg2S38+FI/yi/9YyQ1FLX3T mwG/K7Y/0YnkEAQcrAm46xcbh7gQ/uqvE36kQdXxP5z5X/kPlb65NNV+Y g==; X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="385681619" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="385681619" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Jun 2023 08:26:33 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="834233148" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="834233148" Received: from dpdk-zhirun-lmm.sh.intel.com ([10.67.119.94]) by orsmga004.jf.intel.com with ESMTP; 08 Jun 2023 08:26:30 -0700 From: Zhirun Yan To: dev@dpdk.org, jerinj@marvell.com, kirankumark@marvell.com, ndabilpuram@marvell.com, stephen@networkplumber.org, pbhagavatula@marvell.com, jerinjacobk@gmail.com Cc: cunming.liang@intel.com, haiyue.wang@intel.com, mattias.ronnblom@ericsson.com, Zhirun Yan Subject: [PATCH v11 15/16] test/graph: add functional tests for mcore dispatch model Date: Thu, 8 Jun 2023 23:18:43 +0800 Message-Id: <20230608151844.1823783-16-zhirun.yan@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230608151844.1823783-1-zhirun.yan@intel.com> References: <20230608095759.1800617-1-zhirun.yan@intel.com> <20230608151844.1823783-1-zhirun.yan@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add functional test for mcore dispatch model including graph clone, graph model set/get, node worker affinity, graph worker binding/unbinding. Signed-off-by: Haiyue Wang Signed-off-by: Cunming Liang Signed-off-by: Zhirun Yan Acked-by: Pavan Nikhilesh --- app/test/test_graph.c | 130 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 130 insertions(+) diff --git a/app/test/test_graph.c b/app/test/test_graph.c index 1a2d1e6fab..8609c0b3a4 100644 --- a/app/test/test_graph.c +++ b/app/test/test_graph.c @@ -660,6 +660,132 @@ test_create_graph(void) return 0; } +static int +test_graph_clone(void) +{ + rte_graph_t cloned_graph_id = RTE_GRAPH_ID_INVALID; + rte_graph_t main_graph_id = RTE_GRAPH_ID_INVALID; + struct rte_graph_param graph_conf; + int ret = 0; + + main_graph_id = rte_graph_from_name("worker0"); + if (main_graph_id == RTE_GRAPH_ID_INVALID) { + printf("Must create main graph first\n"); + ret = -1; + } + + graph_conf.dispatch.mp_capacity = 1024; + graph_conf.dispatch.wq_size_max = 32; + + cloned_graph_id = rte_graph_clone(main_graph_id, "cloned-test0", &graph_conf); + + if (cloned_graph_id == RTE_GRAPH_ID_INVALID) { + printf("Graph creation failed with error = %d\n", rte_errno); + ret = -1; + } + + if (strcmp(rte_graph_id_to_name(cloned_graph_id), "worker0-cloned-test0")) { + printf("Cloned graph should name as %s but get %s\n", "worker0-cloned-test", + rte_graph_id_to_name(cloned_graph_id)); + ret = -1; + } + + rte_graph_destroy(cloned_graph_id); + + return ret; +} + +static int +test_graph_model_mcore_dispatch_node_lcore_affinity_set(void) +{ + rte_graph_t cloned_graph_id = RTE_GRAPH_ID_INVALID; + unsigned int worker_lcore = RTE_MAX_LCORE; + rte_node_t nid = RTE_NODE_ID_INVALID; + char node_name[64] = "test_node00"; + struct rte_node *node; + int ret = 0; + + worker_lcore = rte_get_next_lcore(worker_lcore, true, 1); + ret = rte_graph_model_mcore_dispatch_node_lcore_affinity_set(node_name, worker_lcore); + if (ret == 0) + printf("Set node %s affinity to lcore %u\n", node_name, worker_lcore); + + nid = rte_node_from_name(node_name); + cloned_graph_id = rte_graph_clone(graph_id, "cloned-test1", NULL); + node = rte_graph_node_get(cloned_graph_id, nid); + + if (node->dispatch.lcore_id != worker_lcore) { + printf("set node affinity failed\n"); + ret = -1; + } + + rte_graph_destroy(cloned_graph_id); + + return ret; +} + +static int +test_graph_model_mcore_dispatch_core_bind_unbind(void) +{ + rte_graph_t cloned_graph_id = RTE_GRAPH_ID_INVALID; + unsigned int worker_lcore = RTE_MAX_LCORE; + struct rte_graph *graph; + int ret = 0; + + worker_lcore = rte_get_next_lcore(worker_lcore, true, 1); + cloned_graph_id = rte_graph_clone(graph_id, "cloned-test2", NULL); + + ret = rte_graph_model_mcore_dispatch_core_bind(cloned_graph_id, worker_lcore); + if (ret != 0) { + printf("bind graph %d to lcore %u failed\n", graph_id, worker_lcore); + ret = -1; + } + + graph = rte_graph_lookup("worker0-cloned-test2"); + + if (graph->dispatch.lcore_id != worker_lcore) { + printf("bind graph %s(id:%d) with lcore %u failed\n", + graph->name, graph->id, worker_lcore); + ret = -1; + } + + rte_graph_model_mcore_dispatch_core_unbind(cloned_graph_id); + if (graph->dispatch.lcore_id != RTE_MAX_LCORE) { + printf("unbind graph %s(id:%d) failed %d\n", + graph->name, graph->id, graph->dispatch.lcore_id); + ret = -1; + } + + rte_graph_destroy(cloned_graph_id); + + return ret; +} + +static int +test_graph_worker_model_set_get(void) +{ + rte_graph_t cloned_graph_id = RTE_GRAPH_ID_INVALID; + struct rte_graph *graph; + int ret = 0; + + cloned_graph_id = rte_graph_clone(graph_id, "cloned-test3", NULL); + ret = rte_graph_worker_model_set(RTE_GRAPH_MODEL_MCORE_DISPATCH); + if (ret != 0) { + printf("Set graph mcore dispatch model failed\n"); + ret = -1; + } + + graph = rte_graph_lookup("worker0-cloned-test3"); + if (rte_graph_worker_model_get(graph) != RTE_GRAPH_MODEL_MCORE_DISPATCH) { + printf("Get graph worker model failed\n"); + ret = -1; + } + + rte_graph_destroy(cloned_graph_id); + + return 0; +} + static int test_graph_walk(void) { @@ -837,6 +963,10 @@ static struct unit_test_suite graph_testsuite = { TEST_CASE(test_update_edges), TEST_CASE(test_lookup_functions), TEST_CASE(test_create_graph), + TEST_CASE(test_graph_clone), + TEST_CASE(test_graph_model_mcore_dispatch_node_lcore_affinity_set), + TEST_CASE(test_graph_model_mcore_dispatch_core_bind_unbind), + TEST_CASE(test_graph_worker_model_set_get), TEST_CASE(test_graph_lookup_functions), TEST_CASE(test_graph_walk), TEST_CASE(test_print_stats), From patchwork Thu Jun 8 15:18:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yan, Zhirun" X-Patchwork-Id: 128440 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D432942C61; Thu, 8 Jun 2023 17:27:30 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DAA8742D5A; Thu, 8 Jun 2023 17:26:56 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id AADC042D3B for ; Thu, 8 Jun 2023 17:26:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686238013; x=1717774013; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=IJQqBnw7n5zEHLbBUt8aYrpXAYkiwzupXGC0MKWuemQ=; b=lDVL+RTqCKi9aeF9QNqe85+IvK1fJPPvixckFlDrE68bpZymlS6AgG0p h5i+Q3ppzA9cn6Pe9mBfKfPSe+OrdiBV/nF7LAUnHxICZVy3RHEzVLVBQ QHdFV6CAiF6xVOLshSdv9jinElNzYX4TwX0QKs9IXBxWSXychn0kscXBt VKbAdf6oqXJo5+2tduump8VKn2S0wqd32k8Aw0uklv8WDp8aYCbgyIAKy kbHNeUY/EcBu1zZr+L8zfAFanKISTL1hxb8Z71MKH++6v8GICv2UX5B/P ekDLDQY2oX2tdiUw2bx6Yi18CNfNrl2ACoWHUtWXwlgOc7EfQ5ZCVB31w w==; X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="385681636" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="385681636" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Jun 2023 08:26:36 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="834233161" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="834233161" Received: from dpdk-zhirun-lmm.sh.intel.com ([10.67.119.94]) by orsmga004.jf.intel.com with ESMTP; 08 Jun 2023 08:26:33 -0700 From: Zhirun Yan To: dev@dpdk.org, jerinj@marvell.com, kirankumark@marvell.com, ndabilpuram@marvell.com, stephen@networkplumber.org, pbhagavatula@marvell.com, jerinjacobk@gmail.com Cc: cunming.liang@intel.com, haiyue.wang@intel.com, mattias.ronnblom@ericsson.com, Zhirun Yan Subject: [PATCH v11 16/16] examples/l3fwd-graph: introduce mcore dispatch worker model Date: Thu, 8 Jun 2023 23:18:44 +0800 Message-Id: <20230608151844.1823783-17-zhirun.yan@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230608151844.1823783-1-zhirun.yan@intel.com> References: <20230608095759.1800617-1-zhirun.yan@intel.com> <20230608151844.1823783-1-zhirun.yan@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add new parameter "model" to choose mcore dispatch or rtc model. And in dispatch model, the node will affinity to worker core successively. RTE_GRAPH_MODEL_SELECT is set to RTE_GRAPH_MODEL_RTC by default. Must set model the same as RTE_GRAPH_MODEL_SELECT if set it as rtc or mcore dispatch explicitly. If not define it, it could choose by param model in runtime. Only support one RX node for mcore dispatch model in current implementation. ./dpdk-l3fwd-graph -l 1,2,3,4 -n 4 -- -p 0x1 --config="(0,0,1)" -P --model="dispatch" Signed-off-by: Haiyue Wang Signed-off-by: Cunming Liang Signed-off-by: Zhirun Yan Acked-by: Jerin Jacob Acked-by: Pavan Nikhilesh --- doc/guides/sample_app_ug/l3_forward_graph.rst | 16 ++ examples/l3fwd-graph/main.c | 230 +++++++++++++++--- 2 files changed, 208 insertions(+), 38 deletions(-) diff --git a/doc/guides/sample_app_ug/l3_forward_graph.rst b/doc/guides/sample_app_ug/l3_forward_graph.rst index 585ac8c898..7189fa33ec 100644 --- a/doc/guides/sample_app_ug/l3_forward_graph.rst +++ b/doc/guides/sample_app_ug/l3_forward_graph.rst @@ -54,6 +54,7 @@ The application has a number of command line options similar to l3fwd:: [--pcap-enable] [--pcap-num-cap] [--pcap-file-name] + [--model] Where, @@ -78,6 +79,8 @@ Where, * ``--pcap-file-name:`` Optional, Pcap filename to capture packets in. +* ``--model:`` Optional, select graph walking model. + For example, consider a dual processor socket platform with 8 physical cores, where cores 0-7 and 16-23 appear on socket 0, while cores 8-15 and 24-31 appear on socket 1. @@ -122,6 +125,19 @@ In this command: * The --pcap-file-name option enables user to give filename in which packets are to be captured. +To enable mcore dispatch model, the application need change RTE_GRAPH_MODEL_SELECT to ``#define RTE_GRAPH_MODEL_SELECT RTE_GRAPH_MODEL_MCORE_DISPATCH`` +before including rte_graph_worker.h. Recompile and use following command: + +.. code-block:: console + + .//examples/dpdk-l3fwd-graph -l 1,2,3,4 -n 4 -- -p 0x1 --config="(0,0,1)" -P --model="dispatch" + +To enable graph walking model selection in run-time, remove the define of ``RTE_GRAPH_MODEL_SELECT``. Recompile and use the same command. + +In this command: + +* The --model option enables user to select ``rtc`` or ``dispatch`` model. + Refer to the *DPDK Getting Started Guide* for general information on running applications and the Environment Abstraction Layer (EAL) options. diff --git a/examples/l3fwd-graph/main.c b/examples/l3fwd-graph/main.c index 5feeab4f0f..be69fcace1 100644 --- a/examples/l3fwd-graph/main.c +++ b/examples/l3fwd-graph/main.c @@ -23,6 +23,7 @@ #include #include #include +#define RTE_GRAPH_MODEL_SELECT RTE_GRAPH_MODEL_RTC #include #include #include @@ -55,6 +56,9 @@ #define NB_SOCKETS 8 +/* Graph module */ +#define WORKER_MODEL_RTC "rtc" +#define WORKER_MODEL_MCORE_DISPATCH "dispatch" /* Static global variables used within this file. */ static uint16_t nb_rxd = RX_DESC_DEFAULT; static uint16_t nb_txd = TX_DESC_DEFAULT; @@ -88,6 +92,8 @@ struct lcore_rx_queue { char node_name[RTE_NODE_NAMESIZE]; }; +static uint8_t model_conf = RTE_GRAPH_MODEL_DEFAULT; + /* Lcore conf */ struct lcore_conf { uint16_t n_rx_queue; @@ -153,6 +159,19 @@ static struct ipv4_l3fwd_lpm_route ipv4_l3fwd_lpm_route_array[] = { {RTE_IPV4(198, 18, 6, 0), 24, 6}, {RTE_IPV4(198, 18, 7, 0), 24, 7}, }; +static int +check_worker_model_params(void) +{ + if (model_conf == RTE_GRAPH_MODEL_MCORE_DISPATCH && + nb_lcore_params > 1) { + printf("Exceeded max number of lcore params for remote model: %hu\n", + nb_lcore_params); + return -1; + } + + return 0; +} + static int check_lcore_params(void) { @@ -276,6 +295,7 @@ print_usage(const char *prgname) " --eth-dest=X,MM:MM:MM:MM:MM:MM: Ethernet destination for " "port X\n" " --max-pkt-len PKTLEN: maximum packet length in decimal (64-9600)\n" + " --model NAME: walking model name, dispatch or rtc(by default)\n" " --no-numa: Disable numa awareness\n" " --per-port-pool: Use separate buffer pool per port\n" " --pcap-enable: Enables pcap capture\n" @@ -318,6 +338,23 @@ parse_max_pkt_len(const char *pktlen) return len; } +static void +parse_worker_model(const char *model) +{ + if (strcmp(model, WORKER_MODEL_MCORE_DISPATCH) == 0) + model_conf = RTE_GRAPH_MODEL_MCORE_DISPATCH; + else if (strcmp(model, WORKER_MODEL_RTC) == 0) + model_conf = RTE_GRAPH_MODEL_RTC; + else + rte_exit(EXIT_FAILURE, "Invalid worker model: %s", model); + +#if defined(RTE_GRAPH_MODEL_SELECT) + if (model_conf != RTE_GRAPH_MODEL_SELECT) + printf("Warning: model mismatch, will use the RTE_GRAPH_MODEL_SELECT model\n"); + model_conf = RTE_GRAPH_MODEL_SELECT; +#endif +} + static int parse_portmask(const char *portmask) { @@ -434,6 +471,8 @@ static const char short_options[] = "p:" /* portmask */ #define CMD_LINE_OPT_PCAP_ENABLE "pcap-enable" #define CMD_LINE_OPT_NUM_PKT_CAP "pcap-num-cap" #define CMD_LINE_OPT_PCAP_FILENAME "pcap-file-name" +#define CMD_LINE_OPT_WORKER_MODEL "model" + enum { /* Long options mapped to a short option */ @@ -449,6 +488,7 @@ enum { CMD_LINE_OPT_PARSE_PCAP_ENABLE, CMD_LINE_OPT_PARSE_NUM_PKT_CAP, CMD_LINE_OPT_PCAP_FILENAME_CAP, + CMD_LINE_OPT_WORKER_MODEL_TYPE, }; static const struct option lgopts[] = { @@ -460,6 +500,7 @@ static const struct option lgopts[] = { {CMD_LINE_OPT_PCAP_ENABLE, 0, 0, CMD_LINE_OPT_PARSE_PCAP_ENABLE}, {CMD_LINE_OPT_NUM_PKT_CAP, 1, 0, CMD_LINE_OPT_PARSE_NUM_PKT_CAP}, {CMD_LINE_OPT_PCAP_FILENAME, 1, 0, CMD_LINE_OPT_PCAP_FILENAME_CAP}, + {CMD_LINE_OPT_WORKER_MODEL, 1, 0, CMD_LINE_OPT_WORKER_MODEL_TYPE}, {NULL, 0, 0, 0}, }; @@ -551,6 +592,11 @@ parse_args(int argc, char **argv) printf("Pcap file name: %s\n", pcap_filename); break; + case CMD_LINE_OPT_WORKER_MODEL_TYPE: + printf("Use new worker model: %s\n", optarg); + parse_worker_model(optarg); + break; + default: print_usage(prgname); return -1; @@ -788,6 +834,142 @@ config_port_max_pkt_len(struct rte_eth_conf *conf, return 0; } +static void +graph_config_mcore_dispatch(struct rte_graph_param graph_conf) +{ + uint16_t nb_patterns = graph_conf.nb_node_patterns; + int worker_count = rte_lcore_count() - 1; + int main_lcore_id = rte_get_main_lcore(); + rte_graph_t main_graph_id = 0; + struct rte_node *node_tmp; + struct lcore_conf *qconf; + struct rte_graph *graph; + rte_graph_t graph_id; + rte_graph_off_t off; + int n_rx_node = 0; + int worker_lcore; + rte_node_t count; + int i, j; + int ret; + + for (j = 0; j < nb_lcore_params; j++) { + qconf = &lcore_conf[lcore_params[j].lcore_id]; + /* Add rx node patterns of all lcore */ + for (i = 0; i < qconf->n_rx_queue; i++) { + char *node_name = qconf->rx_queue_list[i].node_name; + unsigned int lcore_id = lcore_params[j].lcore_id; + + graph_conf.node_patterns[nb_patterns + n_rx_node + i] = node_name; + n_rx_node++; + ret = rte_graph_model_mcore_dispatch_node_lcore_affinity_set(node_name, + lcore_id); + if (ret == 0) + printf("Set node %s affinity to lcore %u\n", node_name, + lcore_params[j].lcore_id); + } + } + + graph_conf.nb_node_patterns = nb_patterns + n_rx_node; + graph_conf.socket_id = rte_lcore_to_socket_id(main_lcore_id); + + qconf = &lcore_conf[main_lcore_id]; + snprintf(qconf->name, sizeof(qconf->name), "worker_%u", + main_lcore_id); + + /* create main graph */ + main_graph_id = rte_graph_create(qconf->name, &graph_conf); + if (main_graph_id == RTE_GRAPH_ID_INVALID) + rte_exit(EXIT_FAILURE, + "rte_graph_create(): main_graph_id invalid for lcore %u\n", + main_lcore_id); + + /* set the graph model for the main graph */ + rte_graph_worker_model_set(RTE_GRAPH_MODEL_MCORE_DISPATCH); + qconf->graph_id = main_graph_id; + qconf->graph = rte_graph_lookup(qconf->name); + if (!qconf->graph) + rte_exit(EXIT_FAILURE, + "rte_graph_lookup(): graph %s not found\n", + qconf->name); + + graph = qconf->graph; + worker_lcore = lcore_params[nb_lcore_params - 1].lcore_id; + rte_graph_foreach_node(count, off, graph, node_tmp) { + /* Need to set the node Lcore affinity before clone graph for each lcore */ + if (node_tmp->dispatch.lcore_id == RTE_MAX_LCORE) { + worker_lcore = rte_get_next_lcore(worker_lcore, true, 1); + ret = rte_graph_model_mcore_dispatch_node_lcore_affinity_set(node_tmp->name, + worker_lcore); + if (ret == 0) + printf("Set node %s affinity to lcore %u\n", + node_tmp->name, worker_lcore); + } + } + + worker_lcore = main_lcore_id; + for (i = 0; i < worker_count; i++) { + worker_lcore = rte_get_next_lcore(worker_lcore, true, 1); + + qconf = &lcore_conf[worker_lcore]; + snprintf(qconf->name, sizeof(qconf->name), "cloned-%u", worker_lcore); + graph_id = rte_graph_clone(main_graph_id, qconf->name, &graph_conf); + ret = rte_graph_model_mcore_dispatch_core_bind(graph_id, worker_lcore); + if (ret == 0) + printf("bind graph %d to lcore %u\n", graph_id, worker_lcore); + + /* full cloned graph name */ + snprintf(qconf->name, sizeof(qconf->name), "%s", + rte_graph_id_to_name(graph_id)); + qconf->graph_id = graph_id; + qconf->graph = rte_graph_lookup(qconf->name); + if (!qconf->graph) + rte_exit(EXIT_FAILURE, + "Failed to lookup graph %s\n", + qconf->name); + continue; + } +} + +static void +graph_config_rtc(struct rte_graph_param graph_conf) +{ + uint16_t nb_patterns = graph_conf.nb_node_patterns; + struct lcore_conf *qconf; + rte_graph_t graph_id; + uint32_t lcore_id; + rte_edge_t i; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + if (rte_lcore_is_enabled(lcore_id) == 0) + continue; + + qconf = &lcore_conf[lcore_id]; + /* Skip graph creation if no source exists */ + if (!qconf->n_rx_queue) + continue; + /* Add rx node patterns of this lcore */ + for (i = 0; i < qconf->n_rx_queue; i++) { + graph_conf.node_patterns[nb_patterns + i] = + qconf->rx_queue_list[i].node_name; + } + graph_conf.nb_node_patterns = nb_patterns + i; + graph_conf.socket_id = rte_lcore_to_socket_id(lcore_id); + snprintf(qconf->name, sizeof(qconf->name), "worker_%u", + lcore_id); + graph_id = rte_graph_create(qconf->name, &graph_conf); + if (graph_id == RTE_GRAPH_ID_INVALID) + rte_exit(EXIT_FAILURE, + "rte_graph_create(): graph_id invalid for lcore %u\n", + lcore_id); + qconf->graph_id = graph_id; + qconf->graph = rte_graph_lookup(qconf->name); + if (!qconf->graph) + rte_exit(EXIT_FAILURE, + "rte_graph_lookup(): graph %s not found\n", + qconf->name); + } +} + int main(int argc, char **argv) { @@ -840,6 +1022,9 @@ main(int argc, char **argv) if (check_lcore_params() < 0) rte_exit(EXIT_FAILURE, "check_lcore_params() failed\n"); + if (check_worker_model_params() < 0) + rte_exit(EXIT_FAILURE, "check_worker_model_params() failed\n"); + ret = init_lcore_rx_queues(); if (ret < 0) rte_exit(EXIT_FAILURE, "init_lcore_rx_queues() failed\n"); @@ -1079,51 +1264,20 @@ main(int argc, char **argv) memset(&graph_conf, 0, sizeof(graph_conf)); graph_conf.node_patterns = node_patterns; + graph_conf.nb_node_patterns = nb_patterns; /* Pcap config */ graph_conf.pcap_enable = pcap_trace_enable; graph_conf.num_pkt_to_capture = packet_to_capture; graph_conf.pcap_filename = pcap_filename; - for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { - rte_graph_t graph_id; - rte_edge_t i; - - if (rte_lcore_is_enabled(lcore_id) == 0) - continue; - - qconf = &lcore_conf[lcore_id]; - - /* Skip graph creation if no source exists */ - if (!qconf->n_rx_queue) - continue; - - /* Add rx node patterns of this lcore */ - for (i = 0; i < qconf->n_rx_queue; i++) { - graph_conf.node_patterns[nb_patterns + i] = - qconf->rx_queue_list[i].node_name; - } - - graph_conf.nb_node_patterns = nb_patterns + i; - graph_conf.socket_id = rte_lcore_to_socket_id(lcore_id); - - snprintf(qconf->name, sizeof(qconf->name), "worker_%u", - lcore_id); - - graph_id = rte_graph_create(qconf->name, &graph_conf); - if (graph_id == RTE_GRAPH_ID_INVALID) - rte_exit(EXIT_FAILURE, - "rte_graph_create(): graph_id invalid" - " for lcore %u\n", lcore_id); + if (model_conf == RTE_GRAPH_MODEL_MCORE_DISPATCH) + graph_config_mcore_dispatch(graph_conf); + else + graph_config_rtc(graph_conf); - qconf->graph_id = graph_id; - qconf->graph = rte_graph_lookup(qconf->name); - /* >8 End of graph initialization. */ - if (!qconf->graph) - rte_exit(EXIT_FAILURE, - "rte_graph_lookup(): graph %s not found\n", - qconf->name); - } + rte_graph_worker_model_set(model_conf); + /* >8 End of graph initialization. */ memset(&rewrite_data, 0, sizeof(rewrite_data)); rewrite_len = sizeof(rewrite_data);