From patchwork Mon Jul 25 16:35:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 114161 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 34A6AA00C4; Mon, 25 Jul 2022 18:36:03 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0E5F24280D; Mon, 25 Jul 2022 18:36:00 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by mails.dpdk.org (Postfix) with ESMTP id 4FAA741148 for ; Mon, 25 Jul 2022 18:35:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1658766958; x=1690302958; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=hJ0cmGTh9OsXMyI5rf+157EMkILCdCaBZf9lZ5yBuAM=; b=XymF1GER9XB1A3r4mSuHPrpLmQOS8HgJxKFSu00NY3WA4l4qZp3DWiBc ftuYxH+t2MDWEID+J2IgDwb3IC/65eRBUeJEVm9/8LGaxEuig0bpRLsp5 2seqSbJc9f/pNDXiAvXyga9GCIuN+rgTDKpW4w/cfekbwItJJopmTjYqJ wj5sMqzsGkOIYB2aEYeWImeLRVaqDygbvKpc7NXUpfL0Tgo0vqb2oMV35 FTcKyqeeGNSj0iFCqFa37igIeQHtwcdsEBkDdKLYisJ3XWtFci5BKZMPt JAWdliZwe0N9LG6rZ6hf4DFYkvcCf1fHfZyT/5gxxLtGbGGyMvAFCZ6ka A==; X-IronPort-AV: E=McAfee;i="6400,9594,10419"; a="274606800" X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="274606800" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jul 2022 09:35:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="575121815" Received: from silpixa00401385.ir.intel.com (HELO silpixa00401385.ger.corp.intel.com.) ([10.237.223.47]) by orsmga006.jf.intel.com with ESMTP; 25 Jul 2022 09:35:56 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power Subject: [PATCH v2 01/13] test/telemetry_json: print success or failure per subtest Date: Mon, 25 Jul 2022 17:35:30 +0100 Message-Id: <20220725163543.875775-2-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220725163543.875775-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220725163543.875775-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org rather than just printing out success or failure at the end of the test only, print out "OK" or "ERROR" for each individual test case within the overall test. As part of this, ensure each case returns 0 on success and any other value on failure. Signed-off-by: Bruce Richardson --- app/test/test_telemetry_json.c | 33 ++++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-) diff --git a/app/test/test_telemetry_json.c b/app/test/test_telemetry_json.c index 790181d316..748b7cfe5a 100644 --- a/app/test/test_telemetry_json.c +++ b/app/test/test_telemetry_json.c @@ -102,8 +102,10 @@ test_large_array_element(void) used = rte_tel_json_add_array_string(buf, sizeof(buf), used, str); printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + if (used != 0) + return -1; - return strlen(buf) != 0; + return strncmp(expected, buf, sizeof(buf)); } static int @@ -117,20 +119,33 @@ test_large_obj_element(void) used = rte_tel_json_add_obj_u64(buf, sizeof(buf), used, str, 0); printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + if (used != 0) + return -1; - return strlen(buf) != 0; + return strncmp(expected, buf, sizeof(buf)); } +typedef int (*test_fn)(void); + static int test_telemetry_json(void) { - if (test_basic_array() < 0 || - test_basic_obj() < 0 || - test_overflow_array() < 0 || - test_overflow_obj() < 0 || - test_large_array_element() < 0 || - test_large_obj_element() < 0) - return -1; + unsigned int i; + test_fn fns[] = { + test_basic_array, + test_basic_obj, + test_overflow_array, + test_overflow_obj, + test_large_array_element, + test_large_obj_element, + }; + for (i = 0; i < RTE_DIM(fns); i++) + if (fns[i]() == 0) + printf("OK\n"); + else { + printf("ERROR\n"); + return -1; + } return 0; } From patchwork Mon Jul 25 16:35:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 114162 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C32A2A00C4; Mon, 25 Jul 2022 18:36:08 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E61D142826; Mon, 25 Jul 2022 18:36:06 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id 33B1F4280B for ; Mon, 25 Jul 2022 18:36:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1658766965; x=1690302965; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yDrx1N4ZbdNlhbyDntAvBzLJtM452KOwgHL/vWp7YSw=; b=UCoyv26szLtaPbdfqvJIAvMKHjJSMQcONpgmejDVUCQIf/IYiLiaT3Ld fr/ADJfJ8XuGUqtylUM7ovYof/NZsZaUQUdZbEsfT9Cl7aRmDIlewyFbc kkfb0/fXdoXJ5N+ltvcLu73zEAVxZJi6Gt+JOcAdtIawBHvXNV+0JcjgG 5XMxdOcUtFNDOg2j8Ig9xFA7C8/U5XLHkK3DHVVVNg7/Z/DKCJJJxV/ay 1YoZi50Pv2wmwYSSq5owN83n4AROby4zuf/pNbd3s9y8WMaev+KYvGHip WarCbco9uEXMoIUv+uF5eAxeumczWs6Vy/bnRV+f1snPul1/us9yi8eeB Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10419"; a="288499039" X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="288499039" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jul 2022 09:36:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="575121846" Received: from silpixa00401385.ir.intel.com (HELO silpixa00401385.ger.corp.intel.com.) ([10.237.223.47]) by orsmga006.jf.intel.com with ESMTP; 25 Jul 2022 09:36:02 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power , Keith Wiles Subject: [PATCH v2 02/13] telemetry: fix escaping of invalid json characters Date: Mon, 25 Jul 2022 17:35:31 +0100 Message-Id: <20220725163543.875775-3-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220725163543.875775-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220725163543.875775-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org For string values returned from telemetry, escape any values that cannot normally appear in a json string. According to the json spec[1], the characters than need to be handled are control chars (char value < 0x20) and '"' and '\' characters. To handle this, we replace the snprintf call with a separate string copying and encapsulation routine which checks each character as it copies it to the final array. [1] https://www.rfc-editor.org/rfc/rfc8259.txt Fixes: 6dd571fd07c3 ("telemetry: introduce new functionality") Bugzilla ID: 1037 Signed-off-by: Bruce Richardson --- lib/telemetry/telemetry.c | 11 +++++--- lib/telemetry/telemetry_json.h | 48 +++++++++++++++++++++++++++++++++- 2 files changed, 55 insertions(+), 4 deletions(-) diff --git a/lib/telemetry/telemetry.c b/lib/telemetry/telemetry.c index c6fd03a5ab..7188b1905c 100644 --- a/lib/telemetry/telemetry.c +++ b/lib/telemetry/telemetry.c @@ -232,9 +232,14 @@ output_json(const char *cmd, const struct rte_tel_data *d, int s) MAX_CMD_LEN, cmd ? cmd : "none"); break; case RTE_TEL_STRING: - used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":\"%.*s\"}", - MAX_CMD_LEN, cmd, - RTE_TEL_MAX_SINGLE_STRING_LEN, d->data.str); + prefix_used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":", + MAX_CMD_LEN, cmd); + cb_data_buf = &out_buf[prefix_used]; + buf_len = sizeof(out_buf) - prefix_used - 1; /* space for '}' */ + + used = rte_tel_json_str(cb_data_buf, buf_len, 0, d->data.str); + used += prefix_used; + used += strlcat(out_buf + used, "}", sizeof(out_buf) - used); break; case RTE_TEL_DICT: prefix_used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":", diff --git a/lib/telemetry/telemetry_json.h b/lib/telemetry/telemetry_json.h index db70690274..13df5d07e3 100644 --- a/lib/telemetry/telemetry_json.h +++ b/lib/telemetry/telemetry_json.h @@ -44,6 +44,52 @@ __json_snprintf(char *buf, const int len, const char *format, ...) return 0; /* nothing written or modified */ } +static const char control_chars[0x20] = { + ['\n'] = 'n', + ['\r'] = 'r', + ['\t'] = 't', +}; + +/** + * @internal + * Does the same as __json_snprintf(buf, len, "\"%s\"", str) + * except that it does proper escaping as necessary. + * Drops any invalid characters we don't support + */ +static inline int +__json_format_str(char *buf, const int len, const char *str) +{ + char tmp[len]; + int tmpidx = 0; + + tmp[tmpidx++] = '"'; + while (*str != '\0') { + if (*str < (int)RTE_DIM(control_chars)) { + int idx = *str; /* compilers don't like char type as index */ + if (control_chars[idx] != 0) { + tmp[tmpidx++] = '\\'; + tmp[tmpidx++] = control_chars[idx]; + } + } else if (*str == '"' || *str == '\\') { + tmp[tmpidx++] = '\\'; + tmp[tmpidx++] = *str; + } else + tmp[tmpidx++] = *str; + /* we always need space for closing quote and null character. + * Ensuring at least two free characters also means we can always take an + * escaped character like "\n" without overflowing + */ + if (tmpidx > len - 2) + return 0; + str++; + } + tmp[tmpidx++] = '"'; + tmp[tmpidx] = '\0'; + + strcpy(buf, tmp); + return tmpidx; +} + /* Copies an empty array into the provided buffer. */ static inline int rte_tel_json_empty_array(char *buf, const int len, const int used) @@ -62,7 +108,7 @@ rte_tel_json_empty_obj(char *buf, const int len, const int used) static inline int rte_tel_json_str(char *buf, const int len, const int used, const char *str) { - return used + __json_snprintf(buf + used, len - used, "\"%s\"", str); + return used + __json_format_str(buf + used, len - used, str); } /* Appends a string into the JSON array in the provided buffer. */ From patchwork Mon Jul 25 16:35:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 114163 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 75699A00C4; Mon, 25 Jul 2022 18:36:14 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C75A04282D; Mon, 25 Jul 2022 18:36:10 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id 4768A4282D for ; Mon, 25 Jul 2022 18:36:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1658766968; x=1690302968; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=3R7YDQkLSJb0lMuB2SKKfDQinNEFHdqXX2l2iCPysLk=; b=YX8AWQQw5VRfytWxVh6YyDZ21kJ8d71ukxZ9wM/rOoBBLhG0P+l4SEEx cnb47e45p7MotASdmb7c9+EPfSMEBnP+MVUbzypeliAIF8CB7xM6P1Efp Kz9HbLMZ74sukHmNvbXycQ5JS/rotPUdOq793JqBHDacgAlHnBjT+1Apx URxjNYSgTtQUhVI6tJB9nPTd9YlCmm1LacDJcjLSoaxOjUKQDp6vu+odN t1Ev5LBfu5uRNeokeS/zPB45RjYV1HXoPVdONCrWpRkuuTdllRQb3BmA6 YXBKz04lS66+euuBP7HGggzc6VURuVLe2G6DXIW4I+J8lgdqE5/bunhJi Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10419"; a="288499057" X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="288499057" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jul 2022 09:36:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="575121874" Received: from silpixa00401385.ir.intel.com (HELO silpixa00401385.ger.corp.intel.com.) ([10.237.223.47]) by orsmga006.jf.intel.com with ESMTP; 25 Jul 2022 09:36:06 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power Subject: [PATCH v2 03/13] test/telemetry_json: add test for string character escaping Date: Mon, 25 Jul 2022 17:35:32 +0100 Message-Id: <20220725163543.875775-4-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220725163543.875775-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220725163543.875775-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add unit test to validate that when creating a string response in json, that characters such as \n or quotes are properly escaped. Signed-off-by: Bruce Richardson --- app/test/test_telemetry_json.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/app/test/test_telemetry_json.c b/app/test/test_telemetry_json.c index 748b7cfe5a..955c2e5b1b 100644 --- a/app/test/test_telemetry_json.c +++ b/app/test/test_telemetry_json.c @@ -125,6 +125,22 @@ test_large_obj_element(void) return strncmp(expected, buf, sizeof(buf)); } +static int +test_string_char_escaping(void) +{ + static const char str[] = "A string across\ntwo lines and \"with quotes\"!"; + const char *expected = "\"A string across\\ntwo lines and \\\"with quotes\\\"!\""; + char buf[sizeof(str) + 10]; + int used = 0; + + used = rte_tel_json_str(buf, sizeof(buf), used, str); + printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + if (used != (int)strlen(expected)) + return -1; + + return strncmp(expected, buf, sizeof(buf)); +} + typedef int (*test_fn)(void); static int @@ -138,6 +154,7 @@ test_telemetry_json(void) test_overflow_obj, test_large_array_element, test_large_obj_element, + test_string_char_escaping, }; for (i = 0; i < RTE_DIM(fns); i++) if (fns[i]() == 0) From patchwork Mon Jul 25 16:35:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 114164 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 17A58A00C4; Mon, 25 Jul 2022 18:36:21 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A8B7242B6D; Mon, 25 Jul 2022 18:36:14 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id 7E91142B6D for ; Mon, 25 Jul 2022 18:36:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1658766972; x=1690302972; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=p8/jQNoGnup04KD0QRg89eDZA6GFDXqWTb4/qfascmA=; b=dd/55fNjfv0lzkjiMk69KWUEQU/dm4BkT9IhCb9tEDosvQ2JyAIzCJJL LcVoSfdAaOHzs2SCuuDflKBzEBAW1z2mLRwz1oIEWz064kF3Z2MSHhfvZ Jkyug8FjJVkOeNppxrEl3/r/SuuM2wtsQ0/GK2CNKdOlB2WYnLc9hD4p6 VY+pyCDZIUJ35R+5UAdeWEMofRAAJgXGHXGD2aa8Wmfy+CMdBqCywdWh6 +rJhhq7c73Y87tN03WQD4yn+kLdxTyTVc3O4IzA7AcTUJfHJ48iNTn8Fa G588PlnTF5pOWOZrh7I2B0YqPHUryUzCgU/OUx2hWyipsrEL9ODL+Nb86 A==; X-IronPort-AV: E=McAfee;i="6400,9594,10419"; a="288499071" X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="288499071" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jul 2022 09:36:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="575121895" Received: from silpixa00401385.ir.intel.com (HELO silpixa00401385.ger.corp.intel.com.) ([10.237.223.47]) by orsmga006.jf.intel.com with ESMTP; 25 Jul 2022 09:36:10 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power Subject: [PATCH v2 04/13] telemetry: add escaping of strings in arrays Date: Mon, 25 Jul 2022 17:35:33 +0100 Message-Id: <20220725163543.875775-5-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220725163543.875775-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220725163543.875775-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org When strings are added to an array variable, we need to properly escape the invalid json characters in the strings. Signed-off-by: Bruce Richardson --- lib/telemetry/telemetry_json.h | 28 +++++++++++++++++++--------- 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/lib/telemetry/telemetry_json.h b/lib/telemetry/telemetry_json.h index 13df5d07e3..c4442a0bf0 100644 --- a/lib/telemetry/telemetry_json.h +++ b/lib/telemetry/telemetry_json.h @@ -52,17 +52,22 @@ static const char control_chars[0x20] = { /** * @internal - * Does the same as __json_snprintf(buf, len, "\"%s\"", str) - * except that it does proper escaping as necessary. + * This function acts the same as __json_snprintf(buf, len, "%s%s%s", prefix, str, suffix) + * except that it does proper escaping of "str" as necessary. Prefix and suffix should be compile- + * time constants not needing escaping. * Drops any invalid characters we don't support */ static inline int -__json_format_str(char *buf, const int len, const char *str) +__json_format_str(char *buf, const int len, const char *prefix, const char *str, const char *suffix) { char tmp[len]; int tmpidx = 0; - tmp[tmpidx++] = '"'; + while (*prefix != '\0' && tmpidx < len) + tmp[tmpidx++] = *prefix++; + if (tmpidx >= len) + return 0; + while (*str != '\0') { if (*str < (int)RTE_DIM(control_chars)) { int idx = *str; /* compilers don't like char type as index */ @@ -75,7 +80,7 @@ __json_format_str(char *buf, const int len, const char *str) tmp[tmpidx++] = *str; } else tmp[tmpidx++] = *str; - /* we always need space for closing quote and null character. + /* we always need space for (at minimum) closing quote and null character. * Ensuring at least two free characters also means we can always take an * escaped character like "\n" without overflowing */ @@ -83,7 +88,12 @@ __json_format_str(char *buf, const int len, const char *str) return 0; str++; } - tmp[tmpidx++] = '"'; + + while (*suffix != '\0' && tmpidx < len) + tmp[tmpidx++] = *suffix++; + if (tmpidx >= len) + return 0; + tmp[tmpidx] = '\0'; strcpy(buf, tmp); @@ -108,7 +118,7 @@ rte_tel_json_empty_obj(char *buf, const int len, const int used) static inline int rte_tel_json_str(char *buf, const int len, const int used, const char *str) { - return used + __json_format_str(buf + used, len - used, str); + return used + __json_format_str(buf + used, len - used, "\"", str, "\""); } /* Appends a string into the JSON array in the provided buffer. */ @@ -118,9 +128,9 @@ rte_tel_json_add_array_string(char *buf, const int len, const int used, { int ret, end = used - 1; /* strip off final delimiter */ if (used <= 2) /* assume empty, since minimum is '[]' */ - return __json_snprintf(buf, len, "[\"%s\"]", str); + return __json_format_str(buf, len, "[\"", str, "\"]"); - ret = __json_snprintf(buf + end, len - end, ",\"%s\"]", str); + ret = __json_format_str(buf + end, len - end, ",\"", str, "\"]"); return ret == 0 ? used : end + ret; } From patchwork Mon Jul 25 16:35:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 114165 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AB495A00C4; Mon, 25 Jul 2022 18:36:28 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DE3B642B6C; Mon, 25 Jul 2022 18:36:17 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id 1A52A42B6C for ; Mon, 25 Jul 2022 18:36:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1658766976; x=1690302976; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2jWvXNu7k5y+RJMnBVt7Jt0AzmkFgSDoz4YBZon6f44=; b=VGccUNDD13+FaKHErtzcyF+06MzXEWq8ckbD6A7aGnVaRY8p71XpEby1 VyoxNWCDYzDylH76PBJ7o72obcp5X8YaRCtse/voAULntJ3wqzfi3tOYB s128Xq8SnOwwwzuMruihkJcwfW2E1KDb06Ug+KUDnf2brtOcPDwlZY4Ku LaT3xtR82Jb34GqpWrv3ALCU6nS0oRGHpm2zKl+oTrb+glbudMMbXMByN Wj4u75YMhY4BUIrPwQ6GYVmATUXM+dIz0hjPVFaetbFZVu+3hg7RwzPRu ePrx0BUmtU8xrmRHaMg8vxwOe9Ud+wBRU9t/hsBUVJQHUlZWa6Oc04LXZ Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10419"; a="288499081" X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="288499081" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jul 2022 09:36:15 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="575121922" Received: from silpixa00401385.ir.intel.com (HELO silpixa00401385.ger.corp.intel.com.) ([10.237.223.47]) by orsmga006.jf.intel.com with ESMTP; 25 Jul 2022 09:36:14 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power Subject: [PATCH v2 05/13] test/telemetry-json: add test for escaping strings in arrays Date: Mon, 25 Jul 2022 17:35:34 +0100 Message-Id: <20220725163543.875775-6-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220725163543.875775-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220725163543.875775-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add test-case to validate that when adding strings to arrays, the strings are properly escaped to remove any invalid characters. Signed-off-by: Bruce Richardson --- app/test/test_telemetry_json.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/app/test/test_telemetry_json.c b/app/test/test_telemetry_json.c index 955c2e5b1b..31a13ea1d7 100644 --- a/app/test/test_telemetry_json.c +++ b/app/test/test_telemetry_json.c @@ -141,6 +141,29 @@ test_string_char_escaping(void) return strncmp(expected, buf, sizeof(buf)); } +static int +test_array_char_escaping(void) +{ + /* "meaning of life", with tab between first two words, '\n' at end, + * and "life" in quotes, followed by "all the fish" in quotes + */ + const char *expected = "[\"meaning\\tof \\\"life\\\"\\n\",\"\\\"all the fish\\\"\"]"; + char buf[1024]; + int used = 0; + + used = rte_tel_json_empty_array(buf, sizeof(buf), used); + if (used != 2 || strcmp(buf, "[]")) + return -1; + + used = rte_tel_json_add_array_string(buf, sizeof(buf), used, "meaning\tof \"life\"\n"); + used = rte_tel_json_add_array_string(buf, sizeof(buf), used, "\"all the fish\""); + + printf("buf = '%s', expected = '%s'\n", buf, expected); + if (used != (int)strlen(expected)) + return -1; + return strncmp(expected, buf, sizeof(buf)); +} + typedef int (*test_fn)(void); static int @@ -155,6 +178,7 @@ test_telemetry_json(void) test_large_array_element, test_large_obj_element, test_string_char_escaping, + test_array_char_escaping, }; for (i = 0; i < RTE_DIM(fns); i++) if (fns[i]() == 0) From patchwork Mon Jul 25 16:35:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 114166 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 76773A00C4; Mon, 25 Jul 2022 18:36:34 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 04EF442B7C; Mon, 25 Jul 2022 18:36:22 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id 741D042B76 for ; Mon, 25 Jul 2022 18:36:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1658766979; x=1690302979; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=1nxMwCwaTD1+yMyvGrZuFIwtw1bNb4w86zM8At6BbB8=; b=RNoqdhXNIrM6lTMhx2hd7k4WBr8N2JkDopQIlNZbCgs0JX5kAV2UlujD qUM5/tPR3XtOuclxGRQBcFiMnENJjVBZWhSwy+RftfHhwR0pQYucQ4VHx oKSndNt2mFekjHv8RuKf70kBLT/AGMEzi8b+HsNEdBORSFhuwcAfVeI3D 97MxoPH9DAeYuoesnlb58dVMor4ykP52G7dXjbPOdvdRXmGHSk9+x7bRx Gye81QtvgoApOUERvahm2ydFM96NKRqLpS+JN12RlMy5d1lZrvZj4WZnq /hjhsp6PrC3WxDRkFm0G33+tFy1/XG7vy4cJ8rAPnTFsJWn0rF5uzCIbe Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10419"; a="288499095" X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="288499095" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jul 2022 09:36:19 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="575121940" Received: from silpixa00401385.ir.intel.com (HELO silpixa00401385.ger.corp.intel.com.) ([10.237.223.47]) by orsmga006.jf.intel.com with ESMTP; 25 Jul 2022 09:36:17 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power Subject: [PATCH v2 06/13] telemetry: limit characters allowed in dictionary names Date: Mon, 25 Jul 2022 17:35:35 +0100 Message-Id: <20220725163543.875775-7-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220725163543.875775-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220725163543.875775-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org To save issues with encoding the names of values in dicts, we limit the allowed names to a subset of character values. This list of allowed characters can be expanded as necessary in future. Signed-off-by: Bruce Richardson --- lib/telemetry/rte_telemetry.h | 8 ++++++++ lib/telemetry/telemetry_data.c | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 40 insertions(+) diff --git a/lib/telemetry/rte_telemetry.h b/lib/telemetry/rte_telemetry.h index d586dd0fc1..a0d21d6b7f 100644 --- a/lib/telemetry/rte_telemetry.h +++ b/lib/telemetry/rte_telemetry.h @@ -64,6 +64,10 @@ rte_tel_data_start_array(struct rte_tel_data *d, enum rte_tel_value_type type); /** * Start a dictionary of values for returning from a callback * + * Dictionaries consist of key-values pairs to be returned, where the keys, + * or names, are strings and the values can be any of the types supported by telemetry. + * Name strings may only contain alphanumeric characters as well as '_' or '/' + * * @param d * The data structure passed to the callback * @return @@ -159,6 +163,7 @@ rte_tel_data_add_array_container(struct rte_tel_data *d, * The data structure passed to the callback * @param name * The name the value is to be stored under in the dict + * Must contain only alphanumeric characters or the symbols: '_' or '/' * @param val * The string to be stored in the dict * @return @@ -177,6 +182,7 @@ rte_tel_data_add_dict_string(struct rte_tel_data *d, const char *name, * The data structure passed to the callback * @param name * The name the value is to be stored under in the dict + * Must contain only alphanumeric characters or the symbols: '_' or '/' * @param val * The number to be stored in the dict * @return @@ -193,6 +199,7 @@ rte_tel_data_add_dict_int(struct rte_tel_data *d, const char *name, int val); * The data structure passed to the callback * @param name * The name the value is to be stored under in the dict + * Must contain only alphanumeric characters or the symbols: '_' or '/' * @param val * The number to be stored in the dict * @return @@ -212,6 +219,7 @@ rte_tel_data_add_dict_u64(struct rte_tel_data *d, * The data structure passed to the callback * @param name * The name the value is to be stored under in the dict. + * Must contain only alphanumeric characters or the symbols: '_' or '/' * @param val * The pointer to the container to be stored in the dict. * @param keep diff --git a/lib/telemetry/telemetry_data.c b/lib/telemetry/telemetry_data.c index e14ae3c4d4..b5cd74b25b 100644 --- a/lib/telemetry/telemetry_data.c +++ b/lib/telemetry/telemetry_data.c @@ -3,6 +3,8 @@ */ #undef RTE_USE_LIBBSD +#include + #include #include "telemetry_data.h" @@ -92,6 +94,24 @@ rte_tel_data_add_array_container(struct rte_tel_data *d, return 0; } +static bool +valid_name(const char *name) +{ + char allowed[128] = { + ['0' ... '9'] = 1, + ['A' ... 'Z'] = 1, + ['a' ... 'z'] = 1, + ['_'] = 1, + ['/'] = 1, + }; + while (*name != '\0') { + if ((size_t)*name >= RTE_DIM(allowed) || allowed[(int)*name] == 0) + return false; + name++; + } + return true; +} + int rte_tel_data_add_dict_string(struct rte_tel_data *d, const char *name, const char *val) @@ -104,6 +124,9 @@ rte_tel_data_add_dict_string(struct rte_tel_data *d, const char *name, if (d->data_len >= RTE_TEL_MAX_DICT_ENTRIES) return -ENOSPC; + if (!valid_name(name)) + return -EINVAL; + d->data_len++; e->type = RTE_TEL_STRING_VAL; vbytes = strlcpy(e->value.sval, val, RTE_TEL_MAX_STRING_LEN); @@ -123,6 +146,9 @@ rte_tel_data_add_dict_int(struct rte_tel_data *d, const char *name, int val) if (d->data_len >= RTE_TEL_MAX_DICT_ENTRIES) return -ENOSPC; + if (!valid_name(name)) + return -EINVAL; + d->data_len++; e->type = RTE_TEL_INT_VAL; e->value.ival = val; @@ -140,6 +166,9 @@ rte_tel_data_add_dict_u64(struct rte_tel_data *d, if (d->data_len >= RTE_TEL_MAX_DICT_ENTRIES) return -ENOSPC; + if (!valid_name(name)) + return -EINVAL; + d->data_len++; e->type = RTE_TEL_U64_VAL; e->value.u64val = val; @@ -161,6 +190,9 @@ rte_tel_data_add_dict_container(struct rte_tel_data *d, const char *name, if (d->data_len >= RTE_TEL_MAX_DICT_ENTRIES) return -ENOSPC; + if (!valid_name(name)) + return -EINVAL; + d->data_len++; e->type = RTE_TEL_CONTAINER; e->value.container.data = val; From patchwork Mon Jul 25 16:35:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 114167 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1A957A00C4; Mon, 25 Jul 2022 18:36:40 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C7D9B4282E; Mon, 25 Jul 2022 18:36:23 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id 6BB9542B81 for ; Mon, 25 Jul 2022 18:36:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1658766982; x=1690302982; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fdnJUlYPRuTz9Il2dzBYqFZuqYYM4NfKV7ENacocaaY=; b=JrN9TSE3IeNu5hDEy3kDybRWw0UhcdrouJE5uWkuAEXEpsomXKLttZR9 OL8SeQCkjIsYR2YfBTlA8CMVmEnfX2l+skdh76Z6vPZ2dxNhIPLam1uhM r4O/71NxppnDclhXvwTvLNzVs0UJukbueSG4imECD8w//nPTmQaMXAY+m 45OByHcuayBWQfvJNxX6rsRkvtaFNe+mqDK71Pc35W7i9flGrcigTXGkf jAk/zFEth7+dtjENw2csnIj5p2eLPDdhLLqyWLG7BNMRyNGkW02j6Y8q9 Gjm+jxn4Tk97kpLXsYihGp0ftgn/QcywCJIv/HBzxwJCGQ1Z/KPy8EzbT A==; X-IronPort-AV: E=McAfee;i="6400,9594,10419"; a="288499111" X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="288499111" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jul 2022 09:36:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="575122148" Received: from silpixa00401385.ir.intel.com (HELO silpixa00401385.ger.corp.intel.com.) ([10.237.223.47]) by orsmga006.jf.intel.com with ESMTP; 25 Jul 2022 09:36:20 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power Subject: [PATCH v2 07/13] telemetry: add escaping of strings in dicts Date: Mon, 25 Jul 2022 17:35:36 +0100 Message-Id: <20220725163543.875775-8-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220725163543.875775-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220725163543.875775-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org When strings are added to an dict variable, we need to properly escape the invalid json characters in the strings. Signed-off-by: Bruce Richardson --- lib/telemetry/telemetry_json.h | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/lib/telemetry/telemetry_json.h b/lib/telemetry/telemetry_json.h index c4442a0bf0..e3fae7c30d 100644 --- a/lib/telemetry/telemetry_json.h +++ b/lib/telemetry/telemetry_json.h @@ -54,7 +54,7 @@ static const char control_chars[0x20] = { * @internal * This function acts the same as __json_snprintf(buf, len, "%s%s%s", prefix, str, suffix) * except that it does proper escaping of "str" as necessary. Prefix and suffix should be compile- - * time constants not needing escaping. + * time constants, or values not needing escaping. * Drops any invalid characters we don't support */ static inline int @@ -219,12 +219,16 @@ static inline int rte_tel_json_add_obj_str(char *buf, const int len, const int used, const char *name, const char *val) { + char tmp_name[RTE_TEL_MAX_STRING_LEN + 5]; int ret, end = used - 1; + + /* names are limited to certain characters so need no escaping */ + snprintf(tmp_name, sizeof(tmp_name), "{\"%s\":\"", name); if (used <= 2) /* assume empty, since minimum is '{}' */ - return __json_snprintf(buf, len, "{\"%s\":\"%s\"}", name, val); + return __json_format_str(buf, len, tmp_name, val, "\"}"); - ret = __json_snprintf(buf + end, len - end, ",\"%s\":\"%s\"}", - name, val); + tmp_name[0] = ','; /* replace '{' with ',' at start */ + ret = __json_format_str(buf + end, len - end, tmp_name, val, "\"}"); return ret == 0 ? used : end + ret; } From patchwork Mon Jul 25 16:35:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 114168 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id F2B1EA00C4; Mon, 25 Jul 2022 18:36:45 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AEE3042B7B; Mon, 25 Jul 2022 18:36:27 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id 2367642905 for ; Mon, 25 Jul 2022 18:36:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1658766986; x=1690302986; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Yhghxn4H7GoQuXi4I+1jhMgapNKh74utAFoe7F2Utoc=; b=YD0oECoqZizSPMyJD4Gnr9Eh5Zily6S4IT0X3wre9QVDGNhe6lqpOYFS kRVhNFvdamxl+fW7Om4JQUJO0nbrdR6Vl91d2GQYrBEtJcuKcbvnaSWKf +TTOaf+tJVMtVQ2goJS7hlJQw6NoJ4DczUu+SnMMf7ODgOcDSA1mpd8qA 71sSrUAWE7Ta+APHxcEw5ytj6mn9Qul41EwwpbvRPh4cvJmfcylERFIKL cKTgydjvTW1XPBON546XLhoudx00Z5ILGpH7jx5IeeuAoOnDP7KvQLziB xNDeF/w7tYsQ7T31+ys1BuoGoH1GF9nz3ubDXhIjR8s6TvUgaFT3Xe3hH Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10419"; a="288499119" X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="288499119" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jul 2022 09:36:25 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="575122656" Received: from silpixa00401385.ir.intel.com (HELO silpixa00401385.ger.corp.intel.com.) ([10.237.223.47]) by orsmga006.jf.intel.com with ESMTP; 25 Jul 2022 09:36:23 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power Subject: [PATCH v2 08/13] test/telemetry_json: add test for string escaping in objects Date: Mon, 25 Jul 2022 17:35:37 +0100 Message-Id: <20220725163543.875775-9-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220725163543.875775-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220725163543.875775-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add a test-case to validate that when adding strings either as the name or the value of an entry in an object, that all values are escaped properly. Signed-off-by: Bruce Richardson --- app/test/test_telemetry_json.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/app/test/test_telemetry_json.c b/app/test/test_telemetry_json.c index 31a13ea1d7..184c3ba9f1 100644 --- a/app/test/test_telemetry_json.c +++ b/app/test/test_telemetry_json.c @@ -164,6 +164,29 @@ test_array_char_escaping(void) return strncmp(expected, buf, sizeof(buf)); } +static int +test_obj_char_escaping(void) +{ + const char *expected = "{\"good\":\"Clint Eastwood\\n\"," + "\"bad\":\"Lee\\tVan\\tCleef\"," + "\"ugly\":\"\\rEli Wallach\"}"; + char buf[1024]; + int used = 0; + + used = rte_tel_json_empty_obj(buf, sizeof(buf), used); + if (used != 2 || strcmp(buf, "{}")) + return -1; + + used = rte_tel_json_add_obj_str(buf, sizeof(buf), used, "good", "Clint Eastwood\n"); + used = rte_tel_json_add_obj_str(buf, sizeof(buf), used, "bad", "Lee\tVan\tCleef"); + used = rte_tel_json_add_obj_str(buf, sizeof(buf), used, "ugly", "\rEli Wallach"); + + printf("buf = '%s', expected = '%s'\n", buf, expected); + if (used != (int)strlen(expected)) + return -1; + return strncmp(expected, buf, sizeof(buf)); +} + typedef int (*test_fn)(void); static int @@ -179,6 +202,7 @@ test_telemetry_json(void) test_large_obj_element, test_string_char_escaping, test_array_char_escaping, + test_obj_char_escaping }; for (i = 0; i < RTE_DIM(fns); i++) if (fns[i]() == 0) From patchwork Mon Jul 25 16:35:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 114169 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 12402A00C4; Mon, 25 Jul 2022 18:36:51 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2FF3442B72; Mon, 25 Jul 2022 18:36:30 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id A6A7842B85 for ; Mon, 25 Jul 2022 18:36:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1658766988; x=1690302988; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=RmkWV8JLBCLs6vpJ2qD29oK9VqHjzT8RCgZmeh/ciFM=; b=Cm4RLTMh1qJaDW5WcYx6r8g/A7xY8/naZyKnOE5Ei6C9itGwvMSd0qQ9 57a4qG1Ufu+k3RhW89rubRiZ5V5Ok9qDnTPXHiOkf+P3eBfG9cyo1Ok4r MvJHskfTWn688L2zi9/HiFXttIbODyl/0rYJKhSjIuoqwneS+/VB2AM+p z5X4JaU4Md40yTEQkxb66HXr+ei7CXtnOt19cOOOyDEzZW+wp62rdo8uW e+8VZjYm46iPF3mWv9mGVx1Ca2PxVXt7fcq5mKS6WdRbMAjZnt0ojOLeV PlpMu0QW6c8aO8ldcUaMy4IDuqlB9znTZXs8SvZvil+g4nchh2RCAF+Yt w==; X-IronPort-AV: E=McAfee;i="6400,9594,10419"; a="288499123" X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="288499123" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jul 2022 09:36:28 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="575122732" Received: from silpixa00401385.ir.intel.com (HELO silpixa00401385.ger.corp.intel.com.) ([10.237.223.47]) by orsmga006.jf.intel.com with ESMTP; 25 Jul 2022 09:36:26 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ray Kinsella , Ciara Power Subject: [PATCH v2 09/13] telemetry: limit command characters Date: Mon, 25 Jul 2022 17:35:38 +0100 Message-Id: <20220725163543.875775-10-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220725163543.875775-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220725163543.875775-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Limit the telemetry command characters to the minimum set needed for current implementations. This prevents issues with invalid json characters needing to be escaped on replies. Signed-off-by: Bruce Richardson --- doc/guides/rel_notes/deprecation.rst | 8 -------- lib/telemetry/telemetry.c | 7 +++++++ 2 files changed, 7 insertions(+), 8 deletions(-) diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst index e7583cae4c..d1c93ca7e3 100644 --- a/doc/guides/rel_notes/deprecation.rst +++ b/doc/guides/rel_notes/deprecation.rst @@ -212,14 +212,6 @@ Deprecation Notices * metrics: The function ``rte_metrics_init`` will have a non-void return in order to notify errors instead of calling ``rte_exit``. -* telemetry: The allowed characters in names for dictionary values - will be limited to alphanumeric characters - and a small subset of additional printable characters. - This will ensure that all dictionary parameter names can be output - without escaping in JSON - or in any future output format used. - Names for the telemetry commands will be similarly limited. - The parameters for telemetry commands are unaffected by this change. - * net/octeontx_ep: The driver ``octeontx_ep`` was to support OCTEON TX line of products. It will be renamed to ``octeon_ep`` in DPDK 22.11 to apply for diff --git a/lib/telemetry/telemetry.c b/lib/telemetry/telemetry.c index 7188b1905c..03651e947d 100644 --- a/lib/telemetry/telemetry.c +++ b/lib/telemetry/telemetry.c @@ -70,12 +70,19 @@ int rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn, const char *help) { struct cmd_callback *new_callbacks; + const char *cmdp = cmd; int i = 0; if (strlen(cmd) >= MAX_CMD_LEN || fn == NULL || cmd[0] != '/' || strlen(help) >= RTE_TEL_MAX_STRING_LEN) return -EINVAL; + while (*cmdp != '\0') { + if (!isalnum(*cmdp) && *cmdp != '_' && *cmdp != '/') + return -EINVAL; + cmdp++; + } + rte_spinlock_lock(&callback_sl); new_callbacks = realloc(callbacks, sizeof(callbacks[0]) * (num_callbacks + 1)); if (new_callbacks == NULL) { From patchwork Mon Jul 25 16:35:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 114170 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7E000A00C4; Mon, 25 Jul 2022 18:36:56 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8750C42B87; Mon, 25 Jul 2022 18:36:32 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id C91E64280B for ; Mon, 25 Jul 2022 18:36:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1658766991; x=1690302991; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=4Y9Uty0wwE9TpkiG3EJhvCcqW3Yf+8pCH4DyHyMrg4c=; b=JqvICvutQJhPlm4fbaGZEpUf2XF1N2E1SGiCE/MHKEm4VrIa97Jyw+uV a5K1toP7iG33wxB1OWD/v3Pr7jOhVmZZg73To+JQlM2Txb+XRTSvxEzaA ut9WMUNIiU2CB+8jkuGn+ti9U8M0WLPtWnZtTTCZ+m5X1EVFLffcq9Bxp wMIFKtVD/usOhaJ9520a/vF37qbKfqLr/hQ7k6oMdLMkrpudB7Qz5F0Ao igOeBPme3b5vfE7+bVINoZZyyxeIXo//I1HbRmqlnyS8hY34VXkHaLAH5 EPngBawpn8v8bPcXxND4M4YtQUSt3Gs81OtdBTxY0fLJ9Gx7rENvSNA24 g==; X-IronPort-AV: E=McAfee;i="6400,9594,10419"; a="288499128" X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="288499128" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jul 2022 09:36:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="575122747" Received: from silpixa00401385.ir.intel.com (HELO silpixa00401385.ger.corp.intel.com.) ([10.237.223.47]) by orsmga006.jf.intel.com with ESMTP; 25 Jul 2022 09:36:29 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power Subject: [PATCH v2 10/13] test/telemetry_data: refactor for maintainability Date: Mon, 25 Jul 2022 17:35:39 +0100 Message-Id: <20220725163543.875775-11-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220725163543.875775-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220725163543.875775-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org To help with the writing and maintaining of test cases in this file we can make the following changes to it: - rename non-test-case functions i.e. the infrastructure functions, to not start with "test_", so that each sub-test case can be identified by starting with that prefix. - add a comment at the start of the file explaining how tests are to be written and managed, so as to keep consistency. - add a trivial test-case for returning a simple string value to use as a reference example for those wanting to add test cases. - improve the key macro used for validating the output from each function, so that the standard json preamble can be skipped for each function. This hides more of the infrastructure implementation from the user i.e. they don't need to worry what the actual command used is called, and also shortens the output strings so we can avoid line splitting in most cases. - add clearing the "response_data" structure to the loop calling each test to avoid each test function having to do so individually. Signed-off-by: Bruce Richardson --- app/test/test_telemetry_data.c | 101 ++++++++++++++++++++------------- 1 file changed, 60 insertions(+), 41 deletions(-) diff --git a/app/test/test_telemetry_data.c b/app/test/test_telemetry_data.c index 73eee293a1..5a85e790d3 100644 --- a/app/test/test_telemetry_data.c +++ b/app/test/test_telemetry_data.c @@ -21,18 +21,45 @@ #define TELEMETRY_VERSION "v2" #define REQUEST_CMD "/test" #define BUF_SIZE 1024 -#define TEST_OUTPUT(exp) test_output(__func__, exp) +#define CHECK_OUTPUT(exp) check_output(__func__, "{\"" REQUEST_CMD "\":" exp "}") + +/* + * Runs a series of test cases, checking the output of telemetry for various different types of + * responses. On init, a single connection to DPDK telemetry is made, and a single telemetry + * callback "/test" is registered. That callback always returns the value of the static global + * variable "response_data", so each test case builds up that structure, and then calls the + * "check_output" function to ensure the response received over the socket for "/test" matches + * that expected for the response_data value populated. + * + * NOTE: + * - each test case function in this file should be added to the "test_cases" array in + * test_telemetry_data function at the bottom of the file. + * - each test case function should populate the "response_data" global variable (below) + * with the appropriate values which would be returned from a simulated telemetry function. + * Then the test case function should have "return TEST_OUTPUT();" as it's + * last line. The test infrastructure will then validate that the output when returning + * "response_data" structure matches that in "". + * - the response_data structure will be zeroed on entry to each test function, so each function + * can begin with a call to "rte_tel_data_string/start_array/start_dict" as so desired. + * - the expected_output for each function can be just the actual json data from the + * "response_data" value. The CHECK_OUTPUT macro will include the appropriate "{\"/test\": ... }" + * structure around the json output. + * + * See test_simple_string(), or test_case_array_int() for a basic examples of test cases. + */ + static struct rte_tel_data response_data; static int sock; + /* * This function is the callback registered with Telemetry to be used when * the /test command is requested. This callback returns the global data built * up by the individual test cases. */ static int -test_cb(const char *cmd __rte_unused, const char *params __rte_unused, +telemetry_test_cb(const char *cmd __rte_unused, const char *params __rte_unused, struct rte_tel_data *d) { *d = response_data; @@ -46,7 +73,7 @@ test_cb(const char *cmd __rte_unused, const char *params __rte_unused, * and is compared to the actual response received from Telemetry. */ static int -test_output(const char *func_name, const char *expected) +check_output(const char *func_name, const char *expected) { int bytes; char buf[BUF_SIZE * 16]; @@ -66,6 +93,14 @@ test_output(const char *func_name, const char *expected) return strncmp(expected, buf, sizeof(buf)); } +static int +test_simple_string(void) +{ + rte_tel_data_string(&response_data, "Simple string"); + + return CHECK_OUTPUT("\"Simple string\""); +} + static int test_dict_with_array_int_values(void) { @@ -77,7 +112,6 @@ test_dict_with_array_int_values(void) struct rte_tel_data *child_data2 = rte_tel_data_alloc(); rte_tel_data_start_array(child_data2, RTE_TEL_INT_VAL); - memset(&response_data, 0, sizeof(response_data)); rte_tel_data_start_dict(&response_data); for (i = 0; i < 5; i++) { @@ -90,8 +124,7 @@ test_dict_with_array_int_values(void) rte_tel_data_add_dict_container(&response_data, "dict_1", child_data2, 0); - return TEST_OUTPUT("{\"/test\":{\"dict_0\":[0,1,2,3,4]," - "\"dict_1\":[0,1,2,3,4]}}"); + return CHECK_OUTPUT("{\"dict_0\":[0,1,2,3,4],\"dict_1\":[0,1,2,3,4]}"); } static int @@ -105,7 +138,6 @@ test_array_with_array_int_values(void) struct rte_tel_data *child_data2 = rte_tel_data_alloc(); rte_tel_data_start_array(child_data2, RTE_TEL_INT_VAL); - memset(&response_data, 0, sizeof(response_data)); rte_tel_data_start_array(&response_data, RTE_TEL_CONTAINER); for (i = 0; i < 5; i++) { @@ -115,18 +147,18 @@ test_array_with_array_int_values(void) rte_tel_data_add_array_container(&response_data, child_data, 0); rte_tel_data_add_array_container(&response_data, child_data2, 0); - return TEST_OUTPUT("{\"/test\":[[0,1,2,3,4],[0,1,2,3,4]]}"); + return CHECK_OUTPUT("[[0,1,2,3,4],[0,1,2,3,4]]"); } static int test_case_array_int(void) { int i; - memset(&response_data, 0, sizeof(response_data)); + rte_tel_data_start_array(&response_data, RTE_TEL_INT_VAL); for (i = 0; i < 5; i++) rte_tel_data_add_array_int(&response_data, i); - return TEST_OUTPUT("{\"/test\":[0,1,2,3,4]}"); + return CHECK_OUTPUT("[0,1,2,3,4]"); } static int @@ -135,7 +167,6 @@ test_case_add_dict_int(void) int i = 0; char name_of_value[8]; - memset(&response_data, 0, sizeof(response_data)); rte_tel_data_start_dict(&response_data); for (i = 0; i < 5; i++) { @@ -143,14 +174,12 @@ test_case_add_dict_int(void) rte_tel_data_add_dict_int(&response_data, name_of_value, i); } - return TEST_OUTPUT("{\"/test\":{\"dict_0\":0,\"dict_1\":1,\"dict_2\":2," - "\"dict_3\":3,\"dict_4\":4}}"); + return CHECK_OUTPUT("{\"dict_0\":0,\"dict_1\":1,\"dict_2\":2,\"dict_3\":3,\"dict_4\":4}"); } static int test_case_array_string(void) { - memset(&response_data, 0, sizeof(response_data)); rte_tel_data_start_array(&response_data, RTE_TEL_STRING_VAL); rte_tel_data_add_array_string(&response_data, "aaaa"); rte_tel_data_add_array_string(&response_data, "bbbb"); @@ -158,14 +187,12 @@ test_case_array_string(void) rte_tel_data_add_array_string(&response_data, "dddd"); rte_tel_data_add_array_string(&response_data, "eeee"); - return TEST_OUTPUT("{\"/test\":[\"aaaa\",\"bbbb\",\"cccc\",\"dddd\"," - "\"eeee\"]}"); + return CHECK_OUTPUT("[\"aaaa\",\"bbbb\",\"cccc\",\"dddd\",\"eeee\"]"); } static int test_case_add_dict_string(void) { - memset(&response_data, 0, sizeof(response_data)); rte_tel_data_start_dict(&response_data); rte_tel_data_add_dict_string(&response_data, "dict_0", "aaaa"); @@ -173,8 +200,7 @@ test_case_add_dict_string(void) rte_tel_data_add_dict_string(&response_data, "dict_2", "cccc"); rte_tel_data_add_dict_string(&response_data, "dict_3", "dddd"); - return TEST_OUTPUT("{\"/test\":{\"dict_0\":\"aaaa\",\"dict_1\":" - "\"bbbb\",\"dict_2\":\"cccc\",\"dict_3\":\"dddd\"}}"); + return CHECK_OUTPUT("{\"dict_0\":\"aaaa\",\"dict_1\":\"bbbb\",\"dict_2\":\"cccc\",\"dict_3\":\"dddd\"}"); } @@ -187,7 +213,6 @@ test_dict_with_array_string_values(void) struct rte_tel_data *child_data2 = rte_tel_data_alloc(); rte_tel_data_start_array(child_data2, RTE_TEL_STRING_VAL); - memset(&response_data, 0, sizeof(response_data)); rte_tel_data_start_dict(&response_data); rte_tel_data_add_array_string(child_data, "aaaa"); @@ -198,8 +223,7 @@ test_dict_with_array_string_values(void) rte_tel_data_add_dict_container(&response_data, "dict_1", child_data2, 0); - return TEST_OUTPUT("{\"/test\":{\"dict_0\":[\"aaaa\"],\"dict_1\":" - "[\"bbbb\"]}}"); + return CHECK_OUTPUT("{\"dict_0\":[\"aaaa\"],\"dict_1\":[\"bbbb\"]}"); } static int @@ -214,7 +238,6 @@ test_dict_with_dict_values(void) struct rte_tel_data *child_data2 = rte_tel_data_alloc(); rte_tel_data_start_array(child_data2, RTE_TEL_STRING_VAL); - memset(&response_data, 0, sizeof(response_data)); rte_tel_data_start_dict(&response_data); rte_tel_data_add_array_string(child_data, "aaaa"); @@ -226,8 +249,7 @@ test_dict_with_dict_values(void) rte_tel_data_add_dict_container(&response_data, "dict_of_dicts", dict_of_dicts, 0); - return TEST_OUTPUT("{\"/test\":{\"dict_of_dicts\":{\"dict_0\":" - "[\"aaaa\"],\"dict_1\":[\"bbbb\"]}}}"); + return CHECK_OUTPUT("{\"dict_of_dicts\":{\"dict_0\":[\"aaaa\"],\"dict_1\":[\"bbbb\"]}}"); } static int @@ -239,7 +261,6 @@ test_array_with_array_string_values(void) struct rte_tel_data *child_data2 = rte_tel_data_alloc(); rte_tel_data_start_array(child_data2, RTE_TEL_STRING_VAL); - memset(&response_data, 0, sizeof(response_data)); rte_tel_data_start_array(&response_data, RTE_TEL_CONTAINER); rte_tel_data_add_array_string(child_data, "aaaa"); @@ -248,18 +269,18 @@ test_array_with_array_string_values(void) rte_tel_data_add_array_container(&response_data, child_data, 0); rte_tel_data_add_array_container(&response_data, child_data2, 0); - return TEST_OUTPUT("{\"/test\":[[\"aaaa\"],[\"bbbb\"]]}"); + return CHECK_OUTPUT("[[\"aaaa\"],[\"bbbb\"]]"); } static int test_case_array_u64(void) { int i; - memset(&response_data, 0, sizeof(response_data)); + rte_tel_data_start_array(&response_data, RTE_TEL_U64_VAL); for (i = 0; i < 5; i++) rte_tel_data_add_array_u64(&response_data, i); - return TEST_OUTPUT("{\"/test\":[0,1,2,3,4]}"); + return CHECK_OUTPUT("[0,1,2,3,4]"); } static int @@ -268,15 +289,13 @@ test_case_add_dict_u64(void) int i = 0; char name_of_value[8]; - memset(&response_data, 0, sizeof(response_data)); rte_tel_data_start_dict(&response_data); for (i = 0; i < 5; i++) { sprintf(name_of_value, "dict_%d", i); rte_tel_data_add_dict_u64(&response_data, name_of_value, i); } - return TEST_OUTPUT("{\"/test\":{\"dict_0\":0,\"dict_1\":1,\"dict_2\":2," - "\"dict_3\":3,\"dict_4\":4}}"); + return CHECK_OUTPUT("{\"dict_0\":0,\"dict_1\":1,\"dict_2\":2,\"dict_3\":3,\"dict_4\":4}"); } static int @@ -290,7 +309,6 @@ test_dict_with_array_u64_values(void) struct rte_tel_data *child_data2 = rte_tel_data_alloc(); rte_tel_data_start_array(child_data2, RTE_TEL_U64_VAL); - memset(&response_data, 0, sizeof(response_data)); rte_tel_data_start_dict(&response_data); for (i = 0; i < 10; i++) { @@ -303,8 +321,7 @@ test_dict_with_array_u64_values(void) rte_tel_data_add_dict_container(&response_data, "dict_1", child_data2, 0); - return TEST_OUTPUT("{\"/test\":{\"dict_0\":[0,1,2,3,4,5,6,7,8,9]," - "\"dict_1\":[0,1,2,3,4,5,6,7,8,9]}}"); + return CHECK_OUTPUT("{\"dict_0\":[0,1,2,3,4,5,6,7,8,9],\"dict_1\":[0,1,2,3,4,5,6,7,8,9]}"); } static int @@ -318,7 +335,6 @@ test_array_with_array_u64_values(void) struct rte_tel_data *child_data2 = rte_tel_data_alloc(); rte_tel_data_start_array(child_data2, RTE_TEL_U64_VAL); - memset(&response_data, 0, sizeof(response_data)); rte_tel_data_start_array(&response_data, RTE_TEL_CONTAINER); for (i = 0; i < 5; i++) { @@ -328,7 +344,7 @@ test_array_with_array_u64_values(void) rte_tel_data_add_array_container(&response_data, child_data, 0); rte_tel_data_add_array_container(&response_data, child_data2, 0); - return TEST_OUTPUT("{\"/test\":[[0,1,2,3,4],[0,1,2,3,4]]}"); + return CHECK_OUTPUT("[[0,1,2,3,4],[0,1,2,3,4]]"); } static int @@ -369,7 +385,7 @@ connect_to_socket(void) } static int -test_telemetry_data(void) +telemetry_data_autotest(void) { typedef int (*test_case)(void); unsigned int i = 0; @@ -378,7 +394,9 @@ test_telemetry_data(void) if (sock <= 0) return -1; - test_case test_cases[] = {test_case_array_string, + test_case test_cases[] = { + test_simple_string, + test_case_array_string, test_case_array_int, test_case_array_u64, test_case_add_dict_int, test_case_add_dict_u64, test_case_add_dict_string, @@ -390,8 +408,9 @@ test_telemetry_data(void) test_array_with_array_u64_values, test_array_with_array_string_values }; - rte_telemetry_register_cmd(REQUEST_CMD, test_cb, "Test"); + rte_telemetry_register_cmd(REQUEST_CMD, telemetry_test_cb, "Test"); for (i = 0; i < RTE_DIM(test_cases); i++) { + memset(&response_data, 0, sizeof(response_data)); if (test_cases[i]() != 0) { close(sock); return -1; @@ -401,4 +420,4 @@ test_telemetry_data(void) return 0; } -REGISTER_TEST_COMMAND(telemetry_data_autotest, test_telemetry_data); +REGISTER_TEST_COMMAND(telemetry_data_autotest, telemetry_data_autotest); From patchwork Mon Jul 25 16:35:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 114171 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C4BFCA00C4; Mon, 25 Jul 2022 18:37:03 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C1F7C42B8D; Mon, 25 Jul 2022 18:36:34 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id 11F5342B89 for ; Mon, 25 Jul 2022 18:36:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1658766993; x=1690302993; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ulk7dfNr+IpskgW6mtTN0ffCvjyImwIkl400BPLel7g=; b=P/Y4lDzFLP0oO2uvqTgqsBy8YF3vbkJ9LiCZXkEnxAzO9zb6iliT019B uYVappPqpAr+Y9Ug8i+QMki7OLZvbdYsU9/XivJ8u/HE4NvfLJBWKUxAT 3/MJstcsNxx2RACm6XmBQmsI6NY81K8cFqAJY6Rk438hfZE/hEs43JMH+ Y32WqIOlkKpvYa7mOxIe7deSY9R/ktuBB3KHLnKeDQkPAwyKLWUPjmUjp DQHG+wc5oZOURm+ZDP4nyRo7tVzBHP5YXBMMjdjE/zWeg6p1A6/VSuuRg F4fbSHh6G8PZjyl7O/rpWaeS7n/+Gy1vKIHZE/hfPuy9xUws86R/k1fhC A==; X-IronPort-AV: E=McAfee;i="6400,9594,10419"; a="288499134" X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="288499134" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jul 2022 09:36:32 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="575122755" Received: from silpixa00401385.ir.intel.com (HELO silpixa00401385.ger.corp.intel.com.) ([10.237.223.47]) by orsmga006.jf.intel.com with ESMTP; 25 Jul 2022 09:36:31 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power Subject: [PATCH v2 11/13] test/telemetry_data: add test cases for character escaping Date: Mon, 25 Jul 2022 17:35:40 +0100 Message-Id: <20220725163543.875775-12-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220725163543.875775-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220725163543.875775-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add in some basic unit tests to validate the character escaping being done on string data values, which tests end-to-end processing of those values beyond just the json-encoding steps tested by the "telemetry_json_autotest". Signed-off-by: Bruce Richardson --- app/test/test_telemetry_data.c | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/app/test/test_telemetry_data.c b/app/test/test_telemetry_data.c index 5a85e790d3..cfa4d636f0 100644 --- a/app/test/test_telemetry_data.c +++ b/app/test/test_telemetry_data.c @@ -347,6 +347,30 @@ test_array_with_array_u64_values(void) return CHECK_OUTPUT("[[0,1,2,3,4],[0,1,2,3,4]]"); } +static int +test_string_char_escaping(void) +{ + rte_tel_data_string(&response_data, "hello,\nworld\n"); + return CHECK_OUTPUT("\"hello,\\nworld\\n\""); +} + +static int +test_array_char_escaping(void) +{ + rte_tel_data_start_array(&response_data, RTE_TEL_STRING_VAL); + rte_tel_data_add_array_string(&response_data, "\\escape\r"); + rte_tel_data_add_array_string(&response_data, "characters\n"); + return CHECK_OUTPUT("[\"\\\\escape\\r\",\"characters\\n\"]"); +} + +static int +test_dict_char_escaping(void) +{ + rte_tel_data_start_dict(&response_data); + rte_tel_data_add_dict_string(&response_data, "name", "escaped\n\tvalue"); + return CHECK_OUTPUT("{\"name\":\"escaped\\n\\tvalue\"}"); +} + static int connect_to_socket(void) { @@ -406,7 +430,11 @@ telemetry_data_autotest(void) test_dict_with_dict_values, test_array_with_array_int_values, test_array_with_array_u64_values, - test_array_with_array_string_values }; + test_array_with_array_string_values, + test_string_char_escaping, + test_array_char_escaping, + test_dict_char_escaping, + }; rte_telemetry_register_cmd(REQUEST_CMD, telemetry_test_cb, "Test"); for (i = 0; i < RTE_DIM(test_cases); i++) { From patchwork Mon Jul 25 16:35:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 114172 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 54645A00C4; Mon, 25 Jul 2022 18:37:09 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9845742B90; Mon, 25 Jul 2022 18:36:36 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id 5BF8142B90 for ; Mon, 25 Jul 2022 18:36:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1658766995; x=1690302995; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8l1EE9X+6YNZwhHtAXzkAYONZuImJwtH8hLQmxTo+5w=; b=QpBGcHeexOsijciFyA0tt29F+bZSwqt+SqwXEqkYChbfIb9QkUUbAmBK TYhHnWRL6QkVR8TDzjid9eOXsTgXBoPcTm0A4nf/eedaziY6WL+7lpTs4 V59WhnsovCND3Jsg3A+ze7C3kb6c2DrP958HLeLkAGOoSnIUiIlKYN8IU wH0ZBKwW0tIjZNqMnpI57qSQbPqX6X6coUuAHTfRi/MRIBrQzXkCoGdzr ModkD81t+o/MX0w0dwReH1TFoBmXVBWJS49rYkYjjT61yoCkX0vadmlwE QdNIOYKgICl8atlsw/cDwmgUqcaCJWTVa2d6HyC4Vx+NALTBuT/rZsp1X A==; X-IronPort-AV: E=McAfee;i="6400,9594,10419"; a="288499142" X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="288499142" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jul 2022 09:36:35 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="575122764" Received: from silpixa00401385.ir.intel.com (HELO silpixa00401385.ger.corp.intel.com.) ([10.237.223.47]) by orsmga006.jf.intel.com with ESMTP; 25 Jul 2022 09:36:33 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power Subject: [PATCH v2 12/13] telemetry: eliminate duplicate code for json output Date: Mon, 25 Jul 2022 17:35:41 +0100 Message-Id: <20220725163543.875775-13-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220725163543.875775-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220725163543.875775-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org When preparing the json response to a telemetry socket query, the code for prefixing the command name, and appending the file "}" on the end of the response was duplicated for multiple reply types. Taking this code out of the switch statement reduces the duplication and makes the code more maintainable. For completeness of testing, add in a test case to validate the "null" response type - the only leg of the switch statment not already covered by an existing test case in the telemetry_data tests. Signed-off-by: Bruce Richardson --- app/test/test_telemetry_data.c | 7 +++++++ lib/telemetry/telemetry.c | 35 ++++++++++++---------------------- 2 files changed, 19 insertions(+), 23 deletions(-) diff --git a/app/test/test_telemetry_data.c b/app/test/test_telemetry_data.c index cfa4d636f0..d0fc78474e 100644 --- a/app/test/test_telemetry_data.c +++ b/app/test/test_telemetry_data.c @@ -93,6 +93,12 @@ check_output(const char *func_name, const char *expected) return strncmp(expected, buf, sizeof(buf)); } +static int +test_null_return(void) +{ + return CHECK_OUTPUT("null"); +} + static int test_simple_string(void) { @@ -419,6 +425,7 @@ telemetry_data_autotest(void) return -1; test_case test_cases[] = { + test_null_return, test_simple_string, test_case_array_string, test_case_array_int, test_case_array_u64, diff --git a/lib/telemetry/telemetry.c b/lib/telemetry/telemetry.c index 03651e947d..cf60d27bd4 100644 --- a/lib/telemetry/telemetry.c +++ b/lib/telemetry/telemetry.c @@ -233,27 +233,22 @@ output_json(const char *cmd, const struct rte_tel_data *d, int s) RTE_BUILD_BUG_ON(sizeof(out_buf) < MAX_CMD_LEN + RTE_TEL_MAX_SINGLE_STRING_LEN + 10); + + prefix_used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":", + MAX_CMD_LEN, cmd); + cb_data_buf = &out_buf[prefix_used]; + buf_len = sizeof(out_buf) - prefix_used - 1; /* space for '}' */ + switch (d->type) { case RTE_TEL_NULL: - used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":null}", - MAX_CMD_LEN, cmd ? cmd : "none"); + used = strlcpy(cb_data_buf, "null", buf_len); break; - case RTE_TEL_STRING: - prefix_used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":", - MAX_CMD_LEN, cmd); - cb_data_buf = &out_buf[prefix_used]; - buf_len = sizeof(out_buf) - prefix_used - 1; /* space for '}' */ + case RTE_TEL_STRING: used = rte_tel_json_str(cb_data_buf, buf_len, 0, d->data.str); - used += prefix_used; - used += strlcat(out_buf + used, "}", sizeof(out_buf) - used); break; - case RTE_TEL_DICT: - prefix_used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":", - MAX_CMD_LEN, cmd); - cb_data_buf = &out_buf[prefix_used]; - buf_len = sizeof(out_buf) - prefix_used - 1; /* space for '}' */ + case RTE_TEL_DICT: used = rte_tel_json_empty_obj(cb_data_buf, buf_len, 0); for (i = 0; i < d->data_len; i++) { const struct tel_dict_entry *v = &d->data.dict[i]; @@ -289,18 +284,12 @@ output_json(const char *cmd, const struct rte_tel_data *d, int s) } } } - used += prefix_used; - used += strlcat(out_buf + used, "}", sizeof(out_buf) - used); break; + case RTE_TEL_ARRAY_STRING: case RTE_TEL_ARRAY_INT: case RTE_TEL_ARRAY_U64: case RTE_TEL_ARRAY_CONTAINER: - prefix_used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":", - MAX_CMD_LEN, cmd); - cb_data_buf = &out_buf[prefix_used]; - buf_len = sizeof(out_buf) - prefix_used - 1; /* space for '}' */ - used = rte_tel_json_empty_array(cb_data_buf, buf_len, 0); for (i = 0; i < d->data_len; i++) if (d->type == RTE_TEL_ARRAY_STRING) @@ -328,10 +317,10 @@ output_json(const char *cmd, const struct rte_tel_data *d, int s) if (!rec_data->keep) rte_tel_data_free(rec_data->data); } - used += prefix_used; - used += strlcat(out_buf + used, "}", sizeof(out_buf) - used); break; } + used += prefix_used; + used += strlcat(out_buf + used, "}", sizeof(out_buf) - used); if (write(s, out_buf, used) < 0) perror("Error writing to socket"); } From patchwork Mon Jul 25 16:35:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Richardson X-Patchwork-Id: 114173 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 216DEA00C4; Mon, 25 Jul 2022 18:37:15 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E8DB84280B; Mon, 25 Jul 2022 18:36:38 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id AE2C64280B for ; Mon, 25 Jul 2022 18:36:37 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1658766997; x=1690302997; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=o4JqiESRzUpvj53szCHZ4ZhHLo/poJuQ0zmzj/oIa0c=; b=lCZ1B/TDl2AuIDvrtsqcGsLGJ1Q8Z8QnTr+EE01toMAE+dkpFN+sNoz8 hHoi+CGJZThe/aX8v+h9ZJ/6DKfe5ZICd3E4UbsgZOIH/P/mInfmiHFTM XxALT1/mKzmiEBpeyQOrr1XF9goMHO3cUqCqw2APVJROSreG2xi+9LDMg z2KaEnaRdoIJHl3OT+t6TxthoQOJE3dpSaCe8VBdHsOxt1kmNFb43Qt6a Lu9y4V9pNqip/xrr+TUPg/xQTF6EfW4VMDskbNqLs9Rzl8uT4EdrBP+va 4CtpIEQYK5eSFVfEDR0QlTq4bHYLvNPs3XcDYJgyhkfLGUnrpRW/XdBBT Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10419"; a="288499152" X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="288499152" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jul 2022 09:36:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,193,1654585200"; d="scan'208";a="575122771" Received: from silpixa00401385.ir.intel.com (HELO silpixa00401385.ger.corp.intel.com.) ([10.237.223.47]) by orsmga006.jf.intel.com with ESMTP; 25 Jul 2022 09:36:36 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: Bruce Richardson , Ciara Power Subject: [PATCH v2 13/13] telemetry: make help command more helpful Date: Mon, 25 Jul 2022 17:35:42 +0100 Message-Id: <20220725163543.875775-14-bruce.richardson@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220725163543.875775-1-bruce.richardson@intel.com> References: <20220623164245.561371-1-bruce.richardson@intel.com> <20220725163543.875775-1-bruce.richardson@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The /help telemetry command prints out the help text for the given command passed in as parameter. However, entering /help without any parameters does not give any useful information as to the fact that you need to pass in a command to get help on. Update the command so it prints its own help text when called without any parameters. Signed-off-by: Bruce Richardson --- lib/telemetry/telemetry.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/lib/telemetry/telemetry.c b/lib/telemetry/telemetry.c index cf60d27bd4..09febff0ae 100644 --- a/lib/telemetry/telemetry.c +++ b/lib/telemetry/telemetry.c @@ -139,15 +139,17 @@ command_help(const char *cmd __rte_unused, const char *params, struct rte_tel_data *d) { int i; + /* if no parameters return our own help text */ + const char *to_lookup = (params == NULL ? cmd : params); - if (!params) - return -1; rte_tel_data_start_dict(d); rte_spinlock_lock(&callback_sl); for (i = 0; i < num_callbacks; i++) - if (strcmp(params, callbacks[i].cmd) == 0) { - rte_tel_data_add_dict_string(d, params, - callbacks[i].help); + if (strcmp(to_lookup, callbacks[i].cmd) == 0) { + if (params == NULL) + rte_tel_data_string(d, callbacks[i].help); + else + rte_tel_data_add_dict_string(d, params, callbacks[i].help); break; } rte_spinlock_unlock(&callback_sl);