From patchwork Sun Apr 23 05:14:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Srikanth Yalavarthi X-Patchwork-Id: 126432 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 70C15429BA; Sun, 23 Apr 2023 07:14:34 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id ACE5042D31; Sun, 23 Apr 2023 07:14:15 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id A06A042D0E for ; Sun, 23 Apr 2023 07:14:13 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 33N3tn84011255 for ; Sat, 22 Apr 2023 22:14:12 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=cpe2ZfZd0qKW0XbLmHuh5bDrElyVe3GxXvlLREyTu6g=; b=fuZEhvTU3+zR2S03ODDVcU7s/5Okg2ncyN/ejltNIUYZP+Q+uNoVkE4n7Ph3n1EYlGk/ u/9Y/pte9/ajpEUHOWnyZeWsEX0nBKY3y+uB4nqZgPGx/ZolSgJL9AMgk+s8RnNp9Wlb 98i5fWaewnDGfWF4pTmLs88OPL3nrZSM/Ab3SIjmS/Yp4ezS3qIfTcguVBo8tUOoF1lH 74SZlVSDxFCb0h4o4Nrzy4984WSB6D0a6hvejwMffBhwszIxXnfWTXq3h7N3Va/kR4RG HGu4BDHfXxczHROkrNCNYXVrjaTN/Zy/V8rc5Z5ZLXlpbUVwjWzWOOkkDUQgegRCr+tb JA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3q4egjjvdb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 22 Apr 2023 22:14:12 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Sat, 22 Apr 2023 22:14:10 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Sat, 22 Apr 2023 22:14:10 -0700 Received: from ml-host-33.caveonetworks.com (unknown [10.110.143.233]) by maili.marvell.com (Postfix) with ESMTP id 99E6A3F706D; Sat, 22 Apr 2023 22:14:10 -0700 (PDT) From: Srikanth Yalavarthi To: Srikanth Yalavarthi CC: , , , Subject: [PATCH v1 4/5] app/mldev: enable reporting xstats Date: Sat, 22 Apr 2023 22:14:02 -0700 Message-ID: <20230423051403.31971-5-syalavarthi@marvell.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230423051403.31971-1-syalavarthi@marvell.com> References: <20230423051403.31971-1-syalavarthi@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: ShGolko8tihiEI3BswnKLDP3l3HVZtMd X-Proofpoint-GUID: ShGolko8tihiEI3BswnKLDP3l3HVZtMd X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-04-23_02,2023-04-21_01,2023-02-09_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Enabled reporting xstats in ML test application. Enabled stats option for model_ops test case. Added common files for xstats and throughput functions. Signed-off-by: Srikanth Yalavarthi --- app/test-mldev/meson.build | 1 + app/test-mldev/ml_common.h | 11 ++ app/test-mldev/ml_options.c | 5 +- app/test-mldev/test_common.h | 3 + app/test-mldev/test_inference_common.c | 60 ---------- app/test-mldev/test_inference_common.h | 1 - app/test-mldev/test_inference_interleave.c | 6 +- app/test-mldev/test_inference_ordered.c | 5 +- app/test-mldev/test_model_ops.c | 3 + app/test-mldev/test_stats.c | 129 +++++++++++++++++++++ app/test-mldev/test_stats.h | 13 +++ 11 files changed, 172 insertions(+), 65 deletions(-) create mode 100644 app/test-mldev/test_stats.c create mode 100644 app/test-mldev/test_stats.h diff --git a/app/test-mldev/meson.build b/app/test-mldev/meson.build index 15db534dc2..18e28f2713 100644 --- a/app/test-mldev/meson.build +++ b/app/test-mldev/meson.build @@ -19,6 +19,7 @@ sources = files( 'test_inference_common.c', 'test_inference_ordered.c', 'test_inference_interleave.c', + 'test_stats.c' ) deps += ['mldev', 'hash'] diff --git a/app/test-mldev/ml_common.h b/app/test-mldev/ml_common.h index 624a5aff50..8d7cc9eeb7 100644 --- a/app/test-mldev/ml_common.h +++ b/app/test-mldev/ml_common.h @@ -26,4 +26,15 @@ #define ml_dump_end printf("\b\t}\n\n") +static inline void +ml_print_line(uint16_t len) +{ + uint16_t i; + + for (i = 0; i < len; i++) + printf("-"); + + printf("\n"); +} + #endif /* ML_COMMON_H */ diff --git a/app/test-mldev/ml_options.c b/app/test-mldev/ml_options.c index 2efcc3532c..1daa229748 100644 --- a/app/test-mldev/ml_options.c +++ b/app/test-mldev/ml_options.c @@ -205,7 +205,8 @@ ml_dump_test_options(const char *testname) } if (strcmp(testname, "model_ops") == 0) { - printf("\t\t--models : comma separated list of models\n"); + printf("\t\t--models : comma separated list of models\n" + "\t\t--stats : enable reporting device statistics\n"); printf("\n"); } @@ -218,7 +219,7 @@ ml_dump_test_options(const char *testname) "\t\t--queue_size : size fo queue-pair\n" "\t\t--batches : number of batches of input\n" "\t\t--tolerance : maximum tolerance (%%) for output validation\n" - "\t\t--stats : enable reporting performance statistics\n"); + "\t\t--stats : enable reporting device and model statistics\n"); printf("\n"); } } diff --git a/app/test-mldev/test_common.h b/app/test-mldev/test_common.h index a7b2ea652a..def108d5b2 100644 --- a/app/test-mldev/test_common.h +++ b/app/test-mldev/test_common.h @@ -14,6 +14,9 @@ struct test_common { struct ml_options *opt; enum ml_test_result result; struct rte_ml_dev_info dev_info; + struct rte_ml_dev_xstats_map *xstats_map; + uint64_t *xstats_values; + int xstats_size; }; bool ml_test_cap_check(struct ml_options *opt); diff --git a/app/test-mldev/test_inference_common.c b/app/test-mldev/test_inference_common.c index 1e16608582..469ed35f6c 100644 --- a/app/test-mldev/test_inference_common.c +++ b/app/test-mldev/test_inference_common.c @@ -39,17 +39,6 @@ } \ } while (0) -static void -print_line(uint16_t len) -{ - uint16_t i; - - for (i = 0; i < len; i++) - printf("-"); - - printf("\n"); -} - /* Enqueue inference requests with burst size equal to 1 */ static int ml_enqueue_single(void *arg) @@ -1027,52 +1016,3 @@ ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t return 0; } - -int -ml_inference_throughput_get(struct ml_test *test, struct ml_options *opt) -{ - struct test_inference *t = ml_test_priv(test); - uint64_t total_cycles = 0; - uint32_t nb_filelist; - uint64_t throughput; - uint64_t avg_e2e; - uint32_t qp_id; - uint64_t freq; - - if (!opt->stats) - return 0; - - /* print end-to-end stats */ - freq = rte_get_tsc_hz(); - for (qp_id = 0; qp_id < RTE_MAX_LCORE; qp_id++) - total_cycles += t->args[qp_id].end_cycles - t->args[qp_id].start_cycles; - avg_e2e = total_cycles / opt->repetitions; - - if (freq == 0) { - avg_e2e = total_cycles / opt->repetitions; - printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (cycles)", avg_e2e); - } else { - avg_e2e = (total_cycles * NS_PER_S) / (opt->repetitions * freq); - printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (ns)", avg_e2e); - } - - /* print inference throughput */ - if (strcmp(opt->test_name, "inference_ordered") == 0) - nb_filelist = 1; - else - nb_filelist = opt->nb_filelist; - - if (freq == 0) { - throughput = (nb_filelist * t->cmn.opt->repetitions * 1000000) / total_cycles; - printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / million cycles)", - throughput); - } else { - throughput = (nb_filelist * t->cmn.opt->repetitions * freq) / total_cycles; - printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / second)", - throughput); - } - - print_line(80); - - return 0; -} diff --git a/app/test-mldev/test_inference_common.h b/app/test-mldev/test_inference_common.h index 0a9b930788..8f27af25e4 100644 --- a/app/test-mldev/test_inference_common.h +++ b/app/test-mldev/test_inference_common.h @@ -70,6 +70,5 @@ void ml_inference_mem_destroy(struct ml_test *test, struct ml_options *opt); int ml_inference_result(struct ml_test *test, struct ml_options *opt, uint16_t fid); int ml_inference_launch_cores(struct ml_test *test, struct ml_options *opt, uint16_t start_fid, uint16_t end_fid); -int ml_inference_throughput_get(struct ml_test *test, struct ml_options *opt); #endif /* TEST_INFERENCE_COMMON_H */ diff --git a/app/test-mldev/test_inference_interleave.c b/app/test-mldev/test_inference_interleave.c index 23b8efe4f0..9503dda042 100644 --- a/app/test-mldev/test_inference_interleave.c +++ b/app/test-mldev/test_inference_interleave.c @@ -7,6 +7,7 @@ #include "ml_common.h" #include "test_inference_common.h" +#include "test_stats.h" static int test_inference_interleave_driver(struct ml_test *test, struct ml_options *opt) @@ -58,7 +59,8 @@ test_inference_interleave_driver(struct ml_test *test, struct ml_options *opt) ml_inference_iomem_destroy(test, opt, fid); } - ml_inference_throughput_get(test, opt); + for (fid = 0; fid < opt->nb_filelist; fid++) + ml_stats_get(test, opt, RTE_ML_DEV_XSTATS_MODEL, fid); for (fid = 0; fid < opt->nb_filelist; fid++) { ret = ml_model_stop(test, opt, &t->model[fid], fid); @@ -70,6 +72,8 @@ test_inference_interleave_driver(struct ml_test *test, struct ml_options *opt) goto error; } + ml_stats_get(test, opt, RTE_ML_DEV_XSTATS_DEVICE, -1); + ml_throughput_get(test, opt); ml_inference_mem_destroy(test, opt); ret = ml_inference_mldev_destroy(test, opt); diff --git a/app/test-mldev/test_inference_ordered.c b/app/test-mldev/test_inference_ordered.c index f49810c152..e0e826d972 100644 --- a/app/test-mldev/test_inference_ordered.c +++ b/app/test-mldev/test_inference_ordered.c @@ -7,6 +7,7 @@ #include "ml_common.h" #include "test_inference_common.h" +#include "test_stats.h" static int test_inference_ordered_driver(struct ml_test *test, struct ml_options *opt) @@ -54,7 +55,8 @@ test_inference_ordered_driver(struct ml_test *test, struct ml_options *opt) goto error; ml_inference_iomem_destroy(test, opt, fid); - ml_inference_throughput_get(test, opt); + ml_stats_get(test, opt, RTE_ML_DEV_XSTATS_MODEL, fid); + ml_throughput_get(test, opt); /* stop model */ ret = ml_model_stop(test, opt, &t->model[fid], fid); @@ -70,6 +72,7 @@ test_inference_ordered_driver(struct ml_test *test, struct ml_options *opt) if (fid < opt->nb_filelist) goto next_model; + ml_stats_get(test, opt, RTE_ML_DEV_XSTATS_DEVICE, -1); ml_inference_mem_destroy(test, opt); ret = ml_inference_mldev_destroy(test, opt); diff --git a/app/test-mldev/test_model_ops.c b/app/test-mldev/test_model_ops.c index 5bbbcb1a6c..b15587a3c5 100644 --- a/app/test-mldev/test_model_ops.c +++ b/app/test-mldev/test_model_ops.c @@ -9,6 +9,7 @@ #include #include "test_model_ops.h" +#include "test_stats.h" static bool test_model_ops_cap_check(struct ml_options *opt) @@ -384,6 +385,8 @@ test_model_ops_driver(struct ml_test *test, struct ml_options *opt) printf("\n"); + ml_stats_get(test, opt, RTE_ML_DEV_XSTATS_DEVICE, -1); + /* device destroy */ ret = test_model_ops_mldev_destroy(test, opt); if (ret != 0) diff --git a/app/test-mldev/test_stats.c b/app/test-mldev/test_stats.c new file mode 100644 index 0000000000..ffe24a9b91 --- /dev/null +++ b/app/test-mldev/test_stats.c @@ -0,0 +1,129 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Marvell. + */ + +#include "test_stats.h" +#include "test_inference_common.h" +#include "test_model_ops.h" + +int +ml_stats_get(struct ml_test *test, struct ml_options *opt, enum rte_ml_dev_xstats_mode mode, + int32_t fid) +{ + struct test_common *t = ml_test_priv(test); + int32_t model_id; + int ret; + int i; + + if (!opt->stats) + return 0; + + if (mode == RTE_ML_DEV_XSTATS_MODEL) + model_id = ((struct test_inference *)t)->model[fid].id; + else + model_id = -1; + + /* get xstats size */ + t->xstats_size = rte_ml_dev_xstats_names_get(opt->dev_id, mode, model_id, NULL, 0); + if (t->xstats_size > 0) { + /* allocate for xstats_map and values */ + t->xstats_map = rte_malloc( + "ml_xstats_map", t->xstats_size * sizeof(struct rte_ml_dev_xstats_map), 0); + if (t->xstats_map == NULL) { + ret = -ENOMEM; + goto error; + } + + t->xstats_values = + rte_malloc("ml_xstats_values", t->xstats_size * sizeof(uint64_t), 0); + if (t->xstats_values == NULL) { + ret = -ENOMEM; + goto error; + } + + ret = rte_ml_dev_xstats_names_get(opt->dev_id, mode, model_id, t->xstats_map, + t->xstats_size); + if (ret != t->xstats_size) { + printf("Unable to get xstats names, ret = %d\n", ret); + ret = -1; + goto error; + } + + for (i = 0; i < t->xstats_size; i++) + rte_ml_dev_xstats_get(opt->dev_id, mode, model_id, &t->xstats_map[i].id, + &t->xstats_values[i], 1); + } + + /* print xstats*/ + printf("\n"); + ml_print_line(80); + if (mode == RTE_ML_DEV_XSTATS_MODEL) + printf(" Model Statistics: %s\n", + ((struct test_inference *)t)->model[fid].info.name); + else + printf(" Device Statistics\n"); + ml_print_line(80); + for (i = 0; i < t->xstats_size; i++) + printf(" %-64s = %" PRIu64 "\n", t->xstats_map[i].name, t->xstats_values[i]); + ml_print_line(80); + + rte_free(t->xstats_map); + rte_free(t->xstats_values); + + return 0; + +error: + rte_free(t->xstats_map); + rte_free(t->xstats_values); + + return ret; +} + +int +ml_throughput_get(struct ml_test *test, struct ml_options *opt) +{ + struct test_inference *t = ml_test_priv(test); + uint64_t total_cycles = 0; + uint32_t nb_filelist; + uint64_t throughput; + uint64_t avg_e2e; + uint32_t qp_id; + uint64_t freq; + + if (!opt->stats) + return 0; + + /* print inference throughput */ + if (strcmp(opt->test_name, "inference_ordered") == 0) + nb_filelist = 1; + else + nb_filelist = opt->nb_filelist; + + /* Print model end-to-end latency and throughput */ + freq = rte_get_tsc_hz(); + for (qp_id = 0; qp_id < RTE_MAX_LCORE; qp_id++) + total_cycles += t->args[qp_id].end_cycles - t->args[qp_id].start_cycles; + + avg_e2e = total_cycles / (opt->repetitions * nb_filelist); + if (freq == 0) { + printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (cycles)", avg_e2e); + } else { + avg_e2e = (avg_e2e * NS_PER_S) / freq; + printf(" %-64s = %" PRIu64 "\n", "Average End-to-End Latency (ns)", avg_e2e); + } + + /* Print model throughput */ + if (freq == 0) { + throughput = 1000000 / avg_e2e; + printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / million cycles)", + throughput); + } else { + throughput = freq / avg_e2e; + printf(" %-64s = %" PRIu64 "\n", "Average Throughput (inferences / second)", + throughput); + } + + ml_print_line(80); + + return 0; +} diff --git a/app/test-mldev/test_stats.h b/app/test-mldev/test_stats.h new file mode 100644 index 0000000000..89648490fa --- /dev/null +++ b/app/test-mldev/test_stats.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Marvell. + */ + +#include +#include + +#include "ml_test.h" +#include "test_inference_common.h" + +int ml_stats_get(struct ml_test *test, struct ml_options *opt, enum rte_ml_dev_xstats_mode, + int32_t fid); +int ml_throughput_get(struct ml_test *test, struct ml_options *opt);