From patchwork Thu Sep 9 13:45:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Hunt, David" X-Patchwork-Id: 98430 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BFA0DA0547; Thu, 9 Sep 2021 15:45:31 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7E5E941148; Thu, 9 Sep 2021 15:45:27 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id B03EC4003E for ; Thu, 9 Sep 2021 15:45:25 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10101"; a="243103557" X-IronPort-AV: E=Sophos;i="5.85,280,1624345200"; d="scan'208";a="243103557" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Sep 2021 06:45:25 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,280,1624345200"; d="scan'208";a="449979453" Received: from silpixa00399952.ir.intel.com ([10.55.129.13]) by orsmga002.jf.intel.com with ESMTP; 09 Sep 2021 06:45:23 -0700 From: David Hunt To: dev@dpdk.org Cc: bruce.richardson@intel.com, David Hunt Date: Thu, 9 Sep 2021 14:45:06 +0100 Message-Id: <20210909134511.18871-2-david.hunt@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210909134511.18871-1-david.hunt@intel.com> References: <20210909134511.18871-1-david.hunt@intel.com> Subject: [dpdk-dev] [PATCH v1 1/6] build: increase default of max lcores to 512 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Modern processors are coming with an ever increasing number of cores, and 128 does not seem like a sensible max limit any more, especially when you consider multi-socket systems with Hyper-Threading enabled. This patch increases max_lcores default from 128 to 512. Signed-off-by: David Hunt --- meson_options.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meson_options.txt b/meson_options.txt index 0e92734c49..0ae03677f1 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -34,7 +34,7 @@ option('machine', type: 'string', value: 'auto', description: 'Alias of cpu_instruction_set.') option('max_ethports', type: 'integer', value: 32, description: 'maximum number of Ethernet devices') -option('max_lcores', type: 'integer', value: 128, description: +option('max_lcores', type: 'integer', value: 512, description: 'maximum number of cores/threads supported by EAL') option('max_numa_nodes', type: 'integer', value: 32, description: 'maximum number of NUMA nodes supported by EAL') From patchwork Thu Sep 9 13:45:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Hunt, David" X-Patchwork-Id: 98431 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8BE6FA0547; Thu, 9 Sep 2021 15:45:37 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A9A0B41156; Thu, 9 Sep 2021 15:45:28 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id 97A8B4114C for ; Thu, 9 Sep 2021 15:45:27 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10101"; a="243103578" X-IronPort-AV: E=Sophos;i="5.85,280,1624345200"; d="scan'208";a="243103578" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Sep 2021 06:45:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,280,1624345200"; d="scan'208";a="449979470" Received: from silpixa00399952.ir.intel.com ([10.55.129.13]) by orsmga002.jf.intel.com with ESMTP; 09 Sep 2021 06:45:25 -0700 From: David Hunt To: dev@dpdk.org Cc: bruce.richardson@intel.com, David Hunt Date: Thu, 9 Sep 2021 14:45:07 +0100 Message-Id: <20210909134511.18871-3-david.hunt@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210909134511.18871-1-david.hunt@intel.com> References: <20210909134511.18871-1-david.hunt@intel.com> Subject: [dpdk-dev] [PATCH v1 2/6] lib/power: reduce memory footprint of acpi lib X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Switch from static memory allocation of core info structs to dynamic allocation. The library used to statically allocate max_lcores number of lcore_power_info structs, so change this to rte_malloc as needed. Reduces static footprint from 192K to 1K. Desirable, especially if we're changing max_lcores default from 128 to 512. Signed-off-by: David Hunt --- lib/power/power_acpi_cpufreq.c | 109 +++++++++++++++++++++++++++------ 1 file changed, 90 insertions(+), 19 deletions(-) diff --git a/lib/power/power_acpi_cpufreq.c b/lib/power/power_acpi_cpufreq.c index 1e8aeb8403..2fa239d02c 100644 --- a/lib/power/power_acpi_cpufreq.c +++ b/lib/power/power_acpi_cpufreq.c @@ -14,6 +14,7 @@ #include #include +#include #include #include "power_acpi_cpufreq.h" @@ -48,7 +49,7 @@ enum power_state { * Power info per lcore. */ struct acpi_power_info { - unsigned int lcore_id; /**< Logical core id */ + unsigned int lcore_id; /**< Logical core id */ uint32_t freqs[RTE_MAX_LCORE_FREQS]; /**< Frequency array */ uint32_t nb_freqs; /**< number of available freqs */ FILE *f; /**< FD of scaling_setspeed */ @@ -59,7 +60,7 @@ struct acpi_power_info { uint16_t turbo_enable; /**< Turbo Boost enable/disable */ } __rte_cache_aligned; -static struct acpi_power_info lcore_power_info[RTE_MAX_LCORE]; +static struct acpi_power_info *lcore_power_info[RTE_MAX_LCORE] = { NULL }; /** * It is to set specific freq for specific logical core, according to the index @@ -248,7 +249,17 @@ power_acpi_cpufreq_init(unsigned int lcore_id) return -1; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + lcore_power_info[lcore_id] = + rte_malloc(NULL, sizeof(struct acpi_power_info), 0); + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "Cannot allocate core %u\n", + lcore_id); + return -1; + } + } + + pi = lcore_power_info[lcore_id]; exp_state = POWER_IDLE; /* The power in use state works as a guard variable between * the CPU frequency control initialization and exit process. @@ -320,7 +331,13 @@ power_acpi_cpufreq_exit(unsigned int lcore_id) lcore_id, RTE_MAX_LCORE - 1U); return -1; } - pi = &lcore_power_info[lcore_id]; + + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; exp_state = POWER_USED; /* The power in use state works as a guard variable between * the CPU frequency control initialization and exit process. @@ -354,6 +371,9 @@ power_acpi_cpufreq_exit(unsigned int lcore_id) __atomic_compare_exchange_n(&(pi->state), &exp_state, POWER_IDLE, 0, __ATOMIC_RELEASE, __ATOMIC_RELAXED); + if (lcore_power_info[lcore_id] != NULL) + rte_free(lcore_power_info[lcore_id]); + return 0; fail: @@ -379,7 +399,12 @@ power_acpi_cpufreq_freqs(unsigned int lcore_id, uint32_t *freqs, uint32_t num) return 0; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; if (num < pi->nb_freqs) { RTE_LOG(ERR, POWER, "Buffer size is not enough\n"); return 0; @@ -397,7 +422,12 @@ power_acpi_cpufreq_get_freq(unsigned int lcore_id) return RTE_POWER_INVALID_FREQ_INDEX; } - return lcore_power_info[lcore_id].curr_idx; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + return lcore_power_info[lcore_id]->curr_idx; } int @@ -408,7 +438,12 @@ power_acpi_cpufreq_set_freq(unsigned int lcore_id, uint32_t index) return -1; } - return set_freq_internal(&(lcore_power_info[lcore_id]), index); + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + return set_freq_internal(lcore_power_info[lcore_id], index); } int @@ -421,7 +456,12 @@ power_acpi_cpufreq_freq_down(unsigned int lcore_id) return -1; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; if (pi->curr_idx + 1 == pi->nb_freqs) return 0; @@ -439,7 +479,12 @@ power_acpi_cpufreq_freq_up(unsigned int lcore_id) return -1; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; if (pi->curr_idx == 0 || (pi->curr_idx == 1 && pi->turbo_available && !pi->turbo_enable)) return 0; @@ -457,17 +502,17 @@ power_acpi_cpufreq_freq_max(unsigned int lcore_id) } /* Frequencies in the array are from high to low. */ - if (lcore_power_info[lcore_id].turbo_available) { - if (lcore_power_info[lcore_id].turbo_enable) + if (lcore_power_info[lcore_id]->turbo_available) { + if (lcore_power_info[lcore_id]->turbo_enable) /* Set to Turbo */ return set_freq_internal( - &lcore_power_info[lcore_id], 0); + lcore_power_info[lcore_id], 0); else /* Set to max non-turbo */ return set_freq_internal( - &lcore_power_info[lcore_id], 1); + lcore_power_info[lcore_id], 1); } else - return set_freq_internal(&lcore_power_info[lcore_id], 0); + return set_freq_internal(lcore_power_info[lcore_id], 0); } int @@ -480,7 +525,12 @@ power_acpi_cpufreq_freq_min(unsigned int lcore_id) return -1; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; /* Frequencies in the array are from high to low. */ return set_freq_internal(pi, pi->nb_freqs - 1); @@ -497,7 +547,12 @@ power_acpi_turbo_status(unsigned int lcore_id) return -1; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; return pi->turbo_enable; } @@ -513,7 +568,12 @@ power_acpi_enable_turbo(unsigned int lcore_id) return -1; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; if (pi->turbo_available) pi->turbo_enable = 1; @@ -546,7 +606,12 @@ power_acpi_disable_turbo(unsigned int lcore_id) return -1; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; pi->turbo_enable = 0; @@ -572,12 +637,18 @@ int power_acpi_get_capabilities(unsigned int lcore_id, RTE_LOG(ERR, POWER, "Invalid lcore ID\n"); return -1; } + + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + if (caps == NULL) { RTE_LOG(ERR, POWER, "Invalid argument\n"); return -1; } - pi = &lcore_power_info[lcore_id]; + pi = lcore_power_info[lcore_id]; caps->capabilities = 0; caps->turbo = !!(pi->turbo_available); From patchwork Thu Sep 9 13:45:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Hunt, David" X-Patchwork-Id: 98432 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D6B45A0547; Thu, 9 Sep 2021 15:45:42 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C80B94115F; Thu, 9 Sep 2021 15:45:29 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id 636214114C for ; Thu, 9 Sep 2021 15:45:28 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10101"; a="243103591" X-IronPort-AV: E=Sophos;i="5.85,280,1624345200"; d="scan'208";a="243103591" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Sep 2021 06:45:27 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,280,1624345200"; d="scan'208";a="449979504" Received: from silpixa00399952.ir.intel.com ([10.55.129.13]) by orsmga002.jf.intel.com with ESMTP; 09 Sep 2021 06:45:26 -0700 From: David Hunt To: dev@dpdk.org Cc: bruce.richardson@intel.com, David Hunt Date: Thu, 9 Sep 2021 14:45:08 +0100 Message-Id: <20210909134511.18871-4-david.hunt@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210909134511.18871-1-david.hunt@intel.com> References: <20210909134511.18871-1-david.hunt@intel.com> Subject: [dpdk-dev] [PATCH v1 3/6] lib/power: reduce memory footprint of pstate lib X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Switch from static memory allocation of core info structs to dynamic allocation. The library used to statically allocate max_lcores number of lcore_power_info structs, so change this to rte_malloc as needed. Reduces static footprint from 192K to 1K. Desirable, especially if we're changing max_lcores default from 128 to 512. Signed-off-by: David Hunt --- lib/power/power_pstate_cpufreq.c | 111 ++++++++++++++++++++++++++----- 1 file changed, 93 insertions(+), 18 deletions(-) diff --git a/lib/power/power_pstate_cpufreq.c b/lib/power/power_pstate_cpufreq.c index 86f8a76e46..3e6529f589 100644 --- a/lib/power/power_pstate_cpufreq.c +++ b/lib/power/power_pstate_cpufreq.c @@ -16,6 +16,7 @@ #include #include +#include #include #include "power_pstate_cpufreq.h" @@ -76,7 +77,7 @@ struct pstate_power_info { } __rte_cache_aligned; -static struct pstate_power_info lcore_power_info[RTE_MAX_LCORE]; +static struct pstate_power_info *lcore_power_info[RTE_MAX_LCORE] = { NULL }; /** * It is to read the specific MSR. @@ -518,7 +519,17 @@ power_pstate_cpufreq_init(unsigned int lcore_id) return -1; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + lcore_power_info[lcore_id] = + rte_malloc(NULL, sizeof(struct pstate_power_info), 0); + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "Cannot allocate core %u\n", + lcore_id); + return -1; + } + } + + pi = lcore_power_info[lcore_id]; exp_state = POWER_IDLE; /* The power in use state works as a guard variable between * the CPU frequency control initialization and exit process. @@ -595,7 +606,13 @@ power_pstate_cpufreq_exit(unsigned int lcore_id) lcore_id, RTE_MAX_LCORE - 1U); return -1; } - pi = &lcore_power_info[lcore_id]; + + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; exp_state = POWER_USED; /* The power in use state works as a guard variable between @@ -632,6 +649,9 @@ power_pstate_cpufreq_exit(unsigned int lcore_id) __atomic_compare_exchange_n(&(pi->state), &exp_state, POWER_IDLE, 0, __ATOMIC_RELEASE, __ATOMIC_RELAXED); + if (lcore_power_info[lcore_id] != NULL) + rte_free(lcore_power_info[lcore_id]); + return 0; fail: @@ -658,7 +678,12 @@ power_pstate_cpufreq_freqs(unsigned int lcore_id, uint32_t *freqs, uint32_t num) return 0; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; if (num < pi->nb_freqs) { RTE_LOG(ERR, POWER, "Buffer size is not enough\n"); return 0; @@ -676,7 +701,12 @@ power_pstate_cpufreq_get_freq(unsigned int lcore_id) return RTE_POWER_INVALID_FREQ_INDEX; } - return lcore_power_info[lcore_id].curr_idx; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + return lcore_power_info[lcore_id]->curr_idx; } @@ -688,7 +718,12 @@ power_pstate_cpufreq_set_freq(unsigned int lcore_id, uint32_t index) return -1; } - return set_freq_internal(&(lcore_power_info[lcore_id]), index); + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + return set_freq_internal(lcore_power_info[lcore_id], index); } int @@ -701,7 +736,12 @@ power_pstate_cpufreq_freq_up(unsigned int lcore_id) return -1; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; if (pi->curr_idx == 0 || (pi->curr_idx == 1 && pi->turbo_available && !pi->turbo_enable)) return 0; @@ -720,7 +760,12 @@ power_pstate_cpufreq_freq_down(unsigned int lcore_id) return -1; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; if (pi->curr_idx + 1 == pi->nb_freqs) return 0; @@ -736,18 +781,23 @@ power_pstate_cpufreq_freq_max(unsigned int lcore_id) return -1; } + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + /* Frequencies in the array are from high to low. */ - if (lcore_power_info[lcore_id].turbo_available) { - if (lcore_power_info[lcore_id].turbo_enable) + if (lcore_power_info[lcore_id]->turbo_available) { + if (lcore_power_info[lcore_id]->turbo_enable) /* Set to Turbo */ return set_freq_internal( - &lcore_power_info[lcore_id], 0); + lcore_power_info[lcore_id], 0); else /* Set to max non-turbo */ return set_freq_internal( - &lcore_power_info[lcore_id], 1); + lcore_power_info[lcore_id], 1); } else - return set_freq_internal(&lcore_power_info[lcore_id], 0); + return set_freq_internal(lcore_power_info[lcore_id], 0); } @@ -761,7 +811,12 @@ power_pstate_cpufreq_freq_min(unsigned int lcore_id) return -1; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; /* Frequencies in the array are from high to low. */ return set_freq_internal(pi, pi->nb_freqs - 1); @@ -778,7 +833,12 @@ power_pstate_turbo_status(unsigned int lcore_id) return -1; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; return pi->turbo_enable; } @@ -793,7 +853,12 @@ power_pstate_enable_turbo(unsigned int lcore_id) return -1; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; if (pi->turbo_available) pi->turbo_enable = 1; @@ -819,7 +884,12 @@ power_pstate_disable_turbo(unsigned int lcore_id) return -1; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; pi->turbo_enable = 0; @@ -851,7 +921,12 @@ int power_pstate_get_capabilities(unsigned int lcore_id, return -1; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; caps->capabilities = 0; caps->turbo = !!(pi->turbo_available); caps->priority = pi->priority_core; From patchwork Thu Sep 9 13:45:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Hunt, David" X-Patchwork-Id: 98433 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 05F38A0547; Thu, 9 Sep 2021 15:45:50 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4DB864116C; Thu, 9 Sep 2021 15:45:32 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id F319441162 for ; Thu, 9 Sep 2021 15:45:29 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10101"; a="243103595" X-IronPort-AV: E=Sophos;i="5.85,280,1624345200"; d="scan'208";a="243103595" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Sep 2021 06:45:29 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,280,1624345200"; d="scan'208";a="449979517" Received: from silpixa00399952.ir.intel.com ([10.55.129.13]) by orsmga002.jf.intel.com with ESMTP; 09 Sep 2021 06:45:27 -0700 From: David Hunt To: dev@dpdk.org Cc: bruce.richardson@intel.com, David Hunt Date: Thu, 9 Sep 2021 14:45:09 +0100 Message-Id: <20210909134511.18871-5-david.hunt@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210909134511.18871-1-david.hunt@intel.com> References: <20210909134511.18871-1-david.hunt@intel.com> Subject: [dpdk-dev] [PATCH v1 4/6] lib/power: reduce memory footprint of cppc lib X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Switch from static memory allocation of core info structs to dynamic allocation. The library used to statically allocate max_lcores number of lcore_power_info structs, so change this to rte_malloc as needed. Reduces static footprint from 192K to 1K. Desirable, especially if we're changing max_lcores default from 128 to 512. Signed-off-by: David Hunt --- lib/power/power_cppc_cpufreq.c | 111 +++++++++++++++++++++++++++------ 1 file changed, 93 insertions(+), 18 deletions(-) diff --git a/lib/power/power_cppc_cpufreq.c b/lib/power/power_cppc_cpufreq.c index 6afd310e4e..952daf312e 100644 --- a/lib/power/power_cppc_cpufreq.c +++ b/lib/power/power_cppc_cpufreq.c @@ -5,6 +5,7 @@ #include #include +#include #include "power_cppc_cpufreq.h" #include "power_common.h" @@ -61,7 +62,7 @@ struct cppc_power_info { uint32_t freqs[RTE_MAX_LCORE_FREQS]; /**< Frequency array */ } __rte_cache_aligned; -static struct cppc_power_info lcore_power_info[RTE_MAX_LCORE]; +static struct cppc_power_info *lcore_power_info[RTE_MAX_LCORE] = { NULL }; /** * It is to set specific freq for specific logical core, according to the index @@ -344,7 +345,17 @@ power_cppc_cpufreq_init(unsigned int lcore_id) return -1; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + lcore_power_info[lcore_id] = + rte_malloc(NULL, sizeof(struct cppc_power_info), 0); + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "Cannot allocate core %u\n", + lcore_id); + return -1; + } + } + + pi = lcore_power_info[lcore_id]; exp_state = POWER_IDLE; /* The power in use state works as a guard variable between * the CPU frequency control initialization and exit process. @@ -422,7 +433,13 @@ power_cppc_cpufreq_exit(unsigned int lcore_id) lcore_id, RTE_MAX_LCORE - 1U); return -1; } - pi = &lcore_power_info[lcore_id]; + + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; exp_state = POWER_USED; /* The power in use state works as a guard variable between * the CPU frequency control initialization and exit process. @@ -454,6 +471,9 @@ power_cppc_cpufreq_exit(unsigned int lcore_id) "original\n", lcore_id); __atomic_store_n(&(pi->state), POWER_IDLE, __ATOMIC_RELEASE); + if (lcore_power_info[lcore_id] != NULL) + rte_free(lcore_power_info[lcore_id]); + return 0; fail: @@ -477,7 +497,12 @@ power_cppc_cpufreq_freqs(unsigned int lcore_id, uint32_t *freqs, uint32_t num) return 0; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; if (num < pi->nb_freqs) { RTE_LOG(ERR, POWER, "Buffer size is not enough\n"); return 0; @@ -495,7 +520,12 @@ power_cppc_cpufreq_get_freq(unsigned int lcore_id) return RTE_POWER_INVALID_FREQ_INDEX; } - return lcore_power_info[lcore_id].curr_idx; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + return lcore_power_info[lcore_id]->curr_idx; } int @@ -506,7 +536,12 @@ power_cppc_cpufreq_set_freq(unsigned int lcore_id, uint32_t index) return -1; } - return set_freq_internal(&(lcore_power_info[lcore_id]), index); + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + return set_freq_internal(lcore_power_info[lcore_id], index); } int @@ -519,7 +554,12 @@ power_cppc_cpufreq_freq_down(unsigned int lcore_id) return -1; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; if (pi->curr_idx + 1 == pi->nb_freqs) return 0; @@ -537,7 +577,12 @@ power_cppc_cpufreq_freq_up(unsigned int lcore_id) return -1; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; if (pi->curr_idx == 0 || (pi->curr_idx == 1 && pi->turbo_available && !pi->turbo_enable)) return 0; @@ -554,18 +599,23 @@ power_cppc_cpufreq_freq_max(unsigned int lcore_id) return -1; } + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + /* Frequencies in the array are from high to low. */ - if (lcore_power_info[lcore_id].turbo_available) { - if (lcore_power_info[lcore_id].turbo_enable) + if (lcore_power_info[lcore_id]->turbo_available) { + if (lcore_power_info[lcore_id]->turbo_enable) /* Set to Turbo */ return set_freq_internal( - &lcore_power_info[lcore_id], 0); + lcore_power_info[lcore_id], 0); else /* Set to max non-turbo */ return set_freq_internal( - &lcore_power_info[lcore_id], 1); + lcore_power_info[lcore_id], 1); } else - return set_freq_internal(&lcore_power_info[lcore_id], 0); + return set_freq_internal(lcore_power_info[lcore_id], 0); } int @@ -578,7 +628,12 @@ power_cppc_cpufreq_freq_min(unsigned int lcore_id) return -1; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; /* Frequencies in the array are from high to low. */ return set_freq_internal(pi, pi->nb_freqs - 1); @@ -594,7 +649,12 @@ power_cppc_turbo_status(unsigned int lcore_id) return -1; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; return pi->turbo_enable; } @@ -609,7 +669,12 @@ power_cppc_enable_turbo(unsigned int lcore_id) return -1; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; if (pi->turbo_available) pi->turbo_enable = 1; @@ -645,7 +710,12 @@ power_cppc_disable_turbo(unsigned int lcore_id) return -1; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; pi->turbo_enable = 0; @@ -677,7 +747,12 @@ power_cppc_get_capabilities(unsigned int lcore_id, return -1; } - pi = &lcore_power_info[lcore_id]; + if (lcore_power_info[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "core %u not initialised\n", lcore_id); + return RTE_POWER_INVALID_FREQ_INDEX; + } + + pi = lcore_power_info[lcore_id]; caps->capabilities = 0; caps->turbo = !!(pi->turbo_available); From patchwork Thu Sep 9 13:45:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Hunt, David" X-Patchwork-Id: 98434 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A8B66A0547; Thu, 9 Sep 2021 15:45:55 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 71F814116F; Thu, 9 Sep 2021 15:45:33 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id C8D0941162 for ; Thu, 9 Sep 2021 15:45:30 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10101"; a="243103605" X-IronPort-AV: E=Sophos;i="5.85,280,1624345200"; d="scan'208";a="243103605" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Sep 2021 06:45:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,280,1624345200"; d="scan'208";a="449979532" Received: from silpixa00399952.ir.intel.com ([10.55.129.13]) by orsmga002.jf.intel.com with ESMTP; 09 Sep 2021 06:45:29 -0700 From: David Hunt To: dev@dpdk.org Cc: bruce.richardson@intel.com, David Hunt Date: Thu, 9 Sep 2021 14:45:10 +0100 Message-Id: <20210909134511.18871-6-david.hunt@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210909134511.18871-1-david.hunt@intel.com> References: <20210909134511.18871-1-david.hunt@intel.com> Subject: [dpdk-dev] [PATCH v1 5/6] lib/power: reduce memory footprint of channels X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Switch from static memory allocation of channel pkt structs to dynamic allocation. The library used to statically allocate max_lcores number of rte_power_channel_packet structs, so change this to rte_malloc as needed. Reduces static footprint from 236K to 1K. Desirable, especially if we're changing max_lcores default from 128 to 512. Signed-off-by: David Hunt --- lib/power/power_kvm_vm.c | 36 +++++++++++++++++++++++++++++++----- 1 file changed, 31 insertions(+), 5 deletions(-) diff --git a/lib/power/power_kvm_vm.c b/lib/power/power_kvm_vm.c index ab7d4b8cee..c60e7acf16 100644 --- a/lib/power/power_kvm_vm.c +++ b/lib/power/power_kvm_vm.c @@ -5,6 +5,7 @@ #include #include +#include #include "rte_power_guest_channel.h" #include "guest_channel.h" @@ -13,7 +14,8 @@ #define FD_PATH "/dev/virtio-ports/virtio.serial.port.poweragent" -static struct rte_power_channel_packet pkt[RTE_MAX_LCORE]; +static struct rte_power_channel_packet + *power_channel_pkt[RTE_MAX_LCORE] = { NULL }; int power_kvm_vm_check_supported(void) @@ -29,8 +31,21 @@ power_kvm_vm_init(unsigned int lcore_id) lcore_id, RTE_MAX_LCORE-1); return -1; } - pkt[lcore_id].command = RTE_POWER_CPU_POWER; - pkt[lcore_id].resource_id = lcore_id; + + if (power_channel_pkt[lcore_id] == NULL) { + power_channel_pkt[lcore_id] = + rte_malloc(NULL, + sizeof(struct rte_power_channel_packet), 0); + if (power_channel_pkt[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, + "Cannot allocate channel for core %u\n", + lcore_id); + return -1; + } + } + + power_channel_pkt[lcore_id]->command = RTE_POWER_CPU_POWER; + power_channel_pkt[lcore_id]->resource_id = lcore_id; return guest_channel_host_connect(FD_PATH, lcore_id); } @@ -38,6 +53,10 @@ int power_kvm_vm_exit(unsigned int lcore_id) { guest_channel_host_disconnect(lcore_id); + + if (power_channel_pkt[lcore_id] != NULL) + rte_free(power_channel_pkt[lcore_id]); + return 0; } @@ -78,8 +97,15 @@ send_msg(unsigned int lcore_id, uint32_t scale_direction) lcore_id, RTE_MAX_LCORE-1); return -1; } - pkt[lcore_id].unit = scale_direction; - ret = guest_channel_send_msg(&pkt[lcore_id], lcore_id); + + if (power_channel_pkt[lcore_id] == NULL) { + RTE_LOG(ERR, POWER, "channel for core %u not initialised\n", + lcore_id); + return -1; + } + + power_channel_pkt[lcore_id]->unit = scale_direction; + ret = guest_channel_send_msg(power_channel_pkt[lcore_id], lcore_id); if (ret == 0) return 1; RTE_LOG(DEBUG, POWER, "Error sending message: %s\n", From patchwork Thu Sep 9 13:45:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Hunt, David" X-Patchwork-Id: 98435 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DBA35A0547; Thu, 9 Sep 2021 15:46:00 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 90CB641175; Thu, 9 Sep 2021 15:45:34 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id 32ABC41162 for ; Thu, 9 Sep 2021 15:45:32 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10101"; a="243103609" X-IronPort-AV: E=Sophos;i="5.85,280,1624345200"; d="scan'208";a="243103609" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Sep 2021 06:45:31 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,280,1624345200"; d="scan'208";a="449979553" Received: from silpixa00399952.ir.intel.com ([10.55.129.13]) by orsmga002.jf.intel.com with ESMTP; 09 Sep 2021 06:45:30 -0700 From: David Hunt To: dev@dpdk.org Cc: bruce.richardson@intel.com, David Hunt Date: Thu, 9 Sep 2021 14:45:11 +0100 Message-Id: <20210909134511.18871-7-david.hunt@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210909134511.18871-1-david.hunt@intel.com> References: <20210909134511.18871-1-david.hunt@intel.com> Subject: [dpdk-dev] [PATCH v1 6/6] lib/power: switch empty poll to max cores config X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" empty poll algorithm uses NUM_NODES as 256, but should really use RTE_MAX_LCORE, which is configurable. Signed-off-by: David Hunt --- lib/power/rte_power_empty_poll.c | 12 ++++++------ lib/power/rte_power_empty_poll.h | 4 +--- 2 files changed, 7 insertions(+), 9 deletions(-) diff --git a/lib/power/rte_power_empty_poll.c b/lib/power/rte_power_empty_poll.c index 975aa92997..fe20b9f7b2 100644 --- a/lib/power/rte_power_empty_poll.c +++ b/lib/power/rte_power_empty_poll.c @@ -373,7 +373,7 @@ rte_empty_poll_detection(struct rte_timer *tim, void *arg) RTE_SET_USED(arg); - for (i = 0; i < NUM_NODES; i++) { + for (i = 0; i < RTE_MAX_LCORE; i++) { poll_stats = &(ep_params->wrk_data.wrk_stats[i]); @@ -436,7 +436,7 @@ rte_power_empty_poll_stat_init(struct ep_params **eptr, uint8_t *freq_tlb, *eptr = ep_params; /* initialize all wrk_stats state */ - for (i = 0; i < NUM_NODES; i++) { + for (i = 0; i < RTE_MAX_LCORE; i++) { if (rte_lcore_is_enabled(i) == 0) continue; @@ -476,7 +476,7 @@ rte_power_empty_poll_stat_update(unsigned int lcore_id) { struct priority_worker *poll_stats; - if (lcore_id >= NUM_NODES) + if (lcore_id >= RTE_MAX_LCORE) return -1; poll_stats = &(ep_params->wrk_data.wrk_stats[lcore_id]); @@ -495,7 +495,7 @@ rte_power_poll_stat_update(unsigned int lcore_id, uint8_t nb_pkt) struct priority_worker *poll_stats; - if (lcore_id >= NUM_NODES) + if (lcore_id >= RTE_MAX_LCORE) return -1; poll_stats = &(ep_params->wrk_data.wrk_stats[lcore_id]); @@ -514,7 +514,7 @@ rte_power_empty_poll_stat_fetch(unsigned int lcore_id) { struct priority_worker *poll_stats; - if (lcore_id >= NUM_NODES) + if (lcore_id >= RTE_MAX_LCORE) return -1; poll_stats = &(ep_params->wrk_data.wrk_stats[lcore_id]); @@ -530,7 +530,7 @@ rte_power_poll_stat_fetch(unsigned int lcore_id) { struct priority_worker *poll_stats; - if (lcore_id >= NUM_NODES) + if (lcore_id >= RTE_MAX_LCORE) return -1; poll_stats = &(ep_params->wrk_data.wrk_stats[lcore_id]); diff --git a/lib/power/rte_power_empty_poll.h b/lib/power/rte_power_empty_poll.h index 6ba0a37074..22678b4f4b 100644 --- a/lib/power/rte_power_empty_poll.h +++ b/lib/power/rte_power_empty_poll.h @@ -31,8 +31,6 @@ extern "C" { #define NUM_PRIORITIES 2 -#define NUM_NODES 256 /* Max core number*/ - /* Processor Power State */ enum freq_val { LOW, @@ -98,7 +96,7 @@ struct priority_worker { struct stats_data { - struct priority_worker wrk_stats[NUM_NODES]; + struct priority_worker wrk_stats[RTE_MAX_LCORE]; /* flag to stop rx threads processing packets until training over */ bool start_rx;