Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/132555/?format=api
{ "id": 132555, "url": "http://patchwork.dpdk.org/api/patches/132555/?format=api", "web_url": "http://patchwork.dpdk.org/project/dpdk/patch/20231011133357.111058-3-bruce.richardson@intel.com/", "project": { "id": 1, "url": "http://patchwork.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<20231011133357.111058-3-bruce.richardson@intel.com>", "list_archive_url": "https://inbox.dpdk.org/dev/20231011133357.111058-3-bruce.richardson@intel.com", "date": "2023-10-11T13:33:54", "name": "[v3,2/5] buildtools: script to generate cmdline boilerplate", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": true, "hash": "c7f33278019bdcbd1ecf127246af380ddd669c76", "submitter": { "id": 20, "url": "http://patchwork.dpdk.org/api/people/20/?format=api", "name": "Bruce Richardson", "email": "bruce.richardson@intel.com" }, "delegate": { "id": 1, "url": "http://patchwork.dpdk.org/api/users/1/?format=api", "username": "tmonjalo", "first_name": "Thomas", "last_name": "Monjalon", "email": "thomas@monjalon.net" }, "mbox": "http://patchwork.dpdk.org/project/dpdk/patch/20231011133357.111058-3-bruce.richardson@intel.com/mbox/", "series": [ { "id": 29820, "url": "http://patchwork.dpdk.org/api/series/29820/?format=api", "web_url": "http://patchwork.dpdk.org/project/dpdk/list/?series=29820", "date": "2023-10-11T13:33:52", "name": "document and simplify use of cmdline", "version": 3, "mbox": "http://patchwork.dpdk.org/series/29820/mbox/" } ], "comments": "http://patchwork.dpdk.org/api/patches/132555/comments/", "check": "success", "checks": "http://patchwork.dpdk.org/api/patches/132555/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from mails.dpdk.org (mails.dpdk.org [217.70.189.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id C6BEE42361;\n\tWed, 11 Oct 2023 15:34:22 +0200 (CEST)", "from mails.dpdk.org (localhost [127.0.0.1])\n\tby mails.dpdk.org (Postfix) with ESMTP id 9662C4067E;\n\tWed, 11 Oct 2023 15:34:13 +0200 (CEST)", "from mgamail.intel.com (mgamail.intel.com [192.55.52.136])\n by mails.dpdk.org (Postfix) with ESMTP id 1B68C4067C\n for <dev@dpdk.org>; Wed, 11 Oct 2023 15:34:10 +0200 (CEST)", "from fmsmga006.fm.intel.com ([10.253.24.20])\n by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;\n 11 Oct 2023 06:34:10 -0700", "from unknown (HELO silpixa00401385.ir.intel.com) ([10.237.214.46])\n by fmsmga006.fm.intel.com with ESMTP; 11 Oct 2023 06:34:09 -0700" ], "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/simple;\n d=intel.com; i=@intel.com; q=dns/txt; s=Intel;\n t=1697031251; x=1728567251;\n h=from:to:cc:subject:date:message-id:in-reply-to:\n references:mime-version:content-transfer-encoding;\n bh=02yVwb6DGN0WGfJOofmh8a9zr/hfKwH3DY55OpqI3do=;\n b=DZZ1NJNRclJkiL+3clx0PwWb8KAAC5tMjwKvt9gAx7fMrH+KYu7ipXwG\n 2jbefqw7H5GQCj7OZXAX96r1aalkTovIRKTx7OsaH7VKeW88kUUIPZON8\n vRIzd9dzsoCzbWIbHi3A5GvO15LtRLEKMKn1uhDDP8aznM+E7BOD07cwh\n szYTbUnMsAomlONvg3ibqAUIzKSn8HpmlN9Y64GG+nbKfxUuDfvZDfRqG\n MITp6khJ9U/L5OeWIbsz0pFJhnqU3QHcq1KSBI9N+mOQH9Fc5x5gdLkO/\n j/ynQnIbNL8evx+Y6HWqVePpU3UWsGDOG2RzpIJgzLy9+fyVxCOC9XtDT A==;", "X-IronPort-AV": [ "E=McAfee;i=\"6600,9927,10860\"; a=\"364025725\"", "E=Sophos;i=\"6.03,216,1694761200\"; d=\"scan'208\";a=\"364025725\"", "E=McAfee;i=\"6600,9927,10860\"; a=\"1001111871\"", "E=Sophos;i=\"6.03,216,1694761200\"; d=\"scan'208\";a=\"1001111871\"" ], "X-ExtLoop1": "1", "From": "Bruce Richardson <bruce.richardson@intel.com>", "To": "dev@dpdk.org", "Cc": "Bruce Richardson <bruce.richardson@intel.com>", "Subject": "[PATCH v3 2/5] buildtools: script to generate cmdline boilerplate", "Date": "Wed, 11 Oct 2023 14:33:54 +0100", "Message-Id": "<20231011133357.111058-3-bruce.richardson@intel.com>", "X-Mailer": "git-send-email 2.39.2", "In-Reply-To": "<20231011133357.111058-1-bruce.richardson@intel.com>", "References": "<20230802170052.955323-1-bruce.richardson@intel.com>\n <20231011133357.111058-1-bruce.richardson@intel.com>", "MIME-Version": "1.0", "Content-Transfer-Encoding": "8bit", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org" }, "content": "Provide a \"dpdk-cmdline-gen.py\" script for application developers to\nquickly generate the boilerplate code necessary for using the cmdline\nlibrary.\n\nExample of use:\nThe script takes an input file with a list of commands the user wants in\nthe app, where the parameter variables are tagged with the type.\nFor example:\n\n\t$ cat commands.list\n\tlist\n\tadd <UINT16>x <UINT16>y\n\techo <STRING>message\n\tadd socket <STRING>path\n\tquit\n\nWhen run through the script as \"./dpdk-cmdline-gen.py commands.list\",\nthe output will be the contents of a header file with all the\nboilerplate necessary for a commandline instance with those commands.\n\nIf the flag --stubs is passed, an output header filename must also be\npassed, in which case both a header file with the definitions and a C\nfile with function stubs in it is written to disk. The separation is so\nthat the header file can be rewritten at any future point to add more\ncommands, while the C file can be kept as-is and extended by the user\nwith any additional functions needed.\n\nSigned-off-by: Bruce Richardson <bruce.richardson@intel.com>\n---\n buildtools/dpdk-cmdline-gen.py | 167 ++++++++++++++++++++++++++++++\n buildtools/meson.build | 7 ++\n doc/guides/prog_guide/cmdline.rst | 131 ++++++++++++++++++++++-\n 3 files changed, 304 insertions(+), 1 deletion(-)\n create mode 100755 buildtools/dpdk-cmdline-gen.py", "diff": "diff --git a/buildtools/dpdk-cmdline-gen.py b/buildtools/dpdk-cmdline-gen.py\nnew file mode 100755\nindex 0000000000..3b41fb0493\n--- /dev/null\n+++ b/buildtools/dpdk-cmdline-gen.py\n@@ -0,0 +1,167 @@\n+#!/usr/bin/env python3\n+# SPDX-License-Identifier: BSD-3-Clause\n+# Copyright(c) 2023 Intel Corporation\n+#\n+\"\"\"Script to automatically generate boilerplate for using DPDK cmdline library.\"\"\"\n+\n+import argparse\n+import sys\n+\n+PARSE_FN_PARAMS = 'void *parsed_result, struct cmdline *cl, void *data'\n+PARSE_FN_BODY = \"\"\"\n+ /* TODO: command action */\n+ RTE_SET_USED(parsed_result);\n+ RTE_SET_USED(cl);\n+ RTE_SET_USED(data);\n+\"\"\"\n+\n+\n+def process_command(tokens, cfile, comment):\n+ \"\"\"Generate the structures and definitions for a single command.\"\"\"\n+ name = []\n+\n+ if tokens[0].startswith('<'):\n+ print('Error: each command must start with at least one literal string', file=sys.stderr)\n+ sys.exit(1)\n+ for t in tokens:\n+ if t.startswith('<'):\n+ break\n+ name.append(t)\n+ name = '_'.join(name)\n+\n+ result_struct = []\n+ initializers = []\n+ token_list = []\n+ for t in tokens:\n+ if t.startswith('<'):\n+ t_type, t_name = t[1:].split('>')\n+ t_val = 'NULL'\n+ else:\n+ t_type = 'STRING'\n+ t_name = t\n+ t_val = f'\"{t}\"'\n+\n+ if t_type == 'STRING':\n+ result_struct.append(f'\\tcmdline_fixed_string_t {t_name};')\n+ initializers.append(\n+ f'static cmdline_parse_token_string_t cmd_{name}_{t_name}_tok =\\n' +\n+ f'\\tTOKEN_STRING_INITIALIZER(struct cmd_{name}_result, {t_name}, {t_val});')\n+ elif t_type in ['UINT8', 'UINT16', 'UINT32', 'UINT64', 'INT8', 'INT16', 'INT32', 'INT64']:\n+ result_struct.append(f'\\t{t_type.lower()}_t {t_name};')\n+ initializers.append(\n+ f'static cmdline_parse_token_num_t cmd_{name}_{t_name}_tok =\\n' +\n+ f'\\tTOKEN_NUM_INITIALIZER(struct cmd_{name}_result, {t_name}, RTE_{t_type});')\n+ elif t_type in ['IP', 'IP_ADDR', 'IPADDR']:\n+ result_struct.append(f'\\tcmdline_ipaddr_t {t_name};')\n+ initializers.append(\n+ f'cmdline_parse_token_ipaddr_t cmd_{name}_{t_name}_tok =\\n' +\n+ f'\\tTOKEN_IPV4_INITIALIZER(struct cmd_{name}_result, {t_name});')\n+ else:\n+ print(f'Error: unknown token-type {t}', file=sys.stderr)\n+ sys.exit(1)\n+ token_list.append(f'cmd_{name}_{t_name}_tok')\n+\n+ print(f'/* Auto-generated handling for command \"{\" \".join(tokens)}\" */')\n+ # output function prototype\n+ func_sig = f'void\\ncmd_{name}_parsed({PARSE_FN_PARAMS})'\n+ print(f'extern {func_sig};\\n')\n+ # output function template if C file being written\n+ if (cfile):\n+ print(f'{func_sig}\\n{{{PARSE_FN_BODY}}}\\n', file=cfile)\n+ # output result data structure\n+ print(\n+ f'struct cmd_{name}_result {{\\n' +\n+ '\\n'.join(result_struct) +\n+ '\\n};\\n')\n+ # output the initializer tokens\n+ print('\\n'.join(initializers) + '\\n')\n+ # output the instance structure\n+ print(\n+ f'static cmdline_parse_inst_t cmd_{name} = {{\\n' +\n+ f'\\t.f = cmd_{name}_parsed,\\n' +\n+ '\\t.data = NULL,\\n' +\n+ f'\\t.help_str = \"{comment}\",\\n' +\n+ '\\t.tokens = {')\n+ for t in token_list:\n+ print(f'\\t\\t(void *)&{t},')\n+ print('\\t\\tNULL\\n' + '\\t}\\n' + '};\\n')\n+\n+ # return the instance structure name\n+ return f'cmd_{name}'\n+\n+\n+def process_commands(infile, hfile, cfile, ctxname):\n+ \"\"\"Generate boilerplate output for a list of commands from infile.\"\"\"\n+ instances = []\n+\n+ # redirect stdout to output the header, to save passing file= each print\n+ old_sys_stdout = sys.stdout\n+ sys.stdout = hfile\n+\n+ print(f'/* File autogenerated by {sys.argv[0]} */')\n+ print('#ifndef GENERATED_COMMANDS_H')\n+ print('#define GENERATED_COMMANDS_H')\n+ print('#include <rte_common.h>')\n+ print('#include <cmdline.h>')\n+ print('#include <cmdline_parse_string.h>')\n+ print('#include <cmdline_parse_num.h>')\n+ print('#include <cmdline_parse_ipaddr.h>')\n+ print('')\n+\n+ for line in infile.readlines():\n+ if line.lstrip().startswith('#'):\n+ continue\n+ if '#' not in line:\n+ line = line + '#' # ensure split always works, even if no help text\n+ tokens, comment = line.split('#', 1)\n+ instances.append(process_command(tokens.strip().split(), cfile, comment.strip()))\n+\n+ print(f'static __rte_used cmdline_parse_ctx_t {ctxname}[] = {{')\n+ for inst in instances:\n+ print(f'\\t&{inst},')\n+ print('\\tNULL')\n+ print('};\\n')\n+ print('#endif /* GENERATED_COMMANDS_H */')\n+\n+ sys.stdout = old_sys_stdout\n+\n+\n+def main():\n+ \"\"\"Application main entry point.\"\"\"\n+ ap = argparse.ArgumentParser()\n+ ap.add_argument(\n+ '--stubs', action='store_true',\n+ help='Produce C file with empty function stubs for each command')\n+ ap.add_argument(\n+ '--output-file', '-o', default='-',\n+ help='Output header filename [default to stdout]')\n+ ap.add_argument(\n+ '--context-name', default='ctx',\n+ help='Name given to the cmdline context variable in the output header [default=ctx]')\n+ ap.add_argument(\n+ 'infile', type=argparse.FileType('r'),\n+ help='File with list of commands')\n+ args = ap.parse_args()\n+\n+ if not args.stubs:\n+ if args.output_file == '-':\n+ process_commands(args.infile, sys.stdout, None, args.context_name)\n+ else:\n+ with open(args.output_file, 'w') as hfile:\n+ process_commands(args.infile, hfile, None, args.context_name)\n+ else:\n+ if not args.output_file.endswith('.h'):\n+ print(\n+ 'Error: output filename must end with \".h\" extension when creating stubs',\n+ file=sys.stderr)\n+ sys.exit(1)\n+\n+ cfilename = args.output_file[:-2] + '.c'\n+ with open(args.output_file, 'w') as hfile:\n+ with open(cfilename, 'w') as cfile:\n+ print(f'#include \"{args.output_file}\"\\n', file=cfile)\n+ process_commands(args.infile, hfile, cfile, args.context_name)\n+\n+\n+if __name__ == '__main__':\n+ main()\ndiff --git a/buildtools/meson.build b/buildtools/meson.build\nindex 948ac17dd2..72447b60a0 100644\n--- a/buildtools/meson.build\n+++ b/buildtools/meson.build\n@@ -19,6 +19,13 @@ get_cpu_count_cmd = py3 + files('get-cpu-count.py')\n get_numa_count_cmd = py3 + files('get-numa-count.py')\n get_test_suites_cmd = py3 + files('get-test-suites.py')\n has_hugepages_cmd = py3 + files('has-hugepages.py')\n+cmdline_gen_cmd = py3 + files('dpdk-cmdline-gen.py')\n+\n+# install any build tools that end-users might want also\n+install_data([\n+ 'dpdk-cmdline-gen.py',\n+ ],\n+ install_dir: 'bin')\n \n # select library and object file format\n pmdinfo = py3 + files('gen-pmdinfo-cfile.py') + [meson.current_build_dir()]\ndiff --git a/doc/guides/prog_guide/cmdline.rst b/doc/guides/prog_guide/cmdline.rst\nindex 40f49a30cc..0b96b770e2 100644\n--- a/doc/guides/prog_guide/cmdline.rst\n+++ b/doc/guides/prog_guide/cmdline.rst\n@@ -44,7 +44,136 @@ Adding a command-line instance to an application involves a number of coding ste\n \n 6. Within your main application code, create a new command-line instance passing in the context.\n \n-The next few subsections will cover each of these steps in more detail,\n+Many of these steps can be automated using the script ``dpdk-cmdline-gen.py`` installed by DPDK,\n+and found in the ``buildtools`` folder in the source tree.\n+This section covers adding a command-line using this script to generate the boiler plate,\n+while the following section,\n+`Worked Example of Adding Command-line to an Application`_ covers the steps to do so manually.\n+\n+Creating a Command List File\n+~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n+\n+The ``dpdk-cmdline-gen.py`` script takes as input a list of commands to be used by the application.\n+While these can be piped to it via standard input, using a list file is probably best.\n+\n+The format of the list file must be:\n+\n+* Comment lines start with '#' as first non-whitespace character\n+\n+* One command per line\n+\n+* Variable fields are prefixed by the type-name in angle-brackets, for example:\n+\n+ * ``<STRING>message``\n+\n+ * ``<UINT16>port_id``\n+\n+ * ``<IP>src_ip``\n+\n+* The help text for a command is given in the form of a comment on the same line as the command\n+\n+An example list file, with a variety of (unrelated) commands, is shown below::\n+\n+ # example list file\n+ list # show all entries\n+ add <UINT16>x <UINT16>y # add x and y\n+ echo <STRING>message # print message to screen\n+ add socket <STRING>path # add unix socket with the given path\n+ quit # close the application\n+\n+Running the Generator Script\n+~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n+\n+To generate the necessary definitions for a command-line, run ``dpdk-cmdline-gen.py`` passing the list file as parameter.\n+The script will output the generated C code to standard output,\n+the contents of which are in the form of a C header file.\n+Optionally, an output filename may be specified via the ``-o/--output-file`` argument.\n+\n+The generated content includes:\n+\n+* The result structure definitions for each command\n+\n+* The token initializers for each structure field\n+\n+* An \"extern\" function prototype for the callback for each command\n+\n+* A parse context for each command, including the per-command comments as help string\n+\n+* A command-line context array definition, suitable for passing to ``cmdline_new``\n+\n+If so desired, the script can also output function stubs for the callback functions for each command.\n+This behaviour is triggered by passing the ``--stubs`` flag to the script.\n+In this case, an output file must be provided with a filename ending in \".h\",\n+and the callback stubs will be written to an equivalent \".c\" file.\n+\n+.. note::\n+\n+ The stubs are written to a separate file,\n+ to allow continuous use of the script to regenerate the command-line header,\n+ without overwriting any code the user has added to the callback functions.\n+ This makes it easy to incrementally add new commands to an existing application.\n+\n+Providing the Function Callbacks\n+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n+\n+As discussed above, the script output is a header file, containing structure definitions,\n+but the callback functions themselves obviously have to be provided by the user.\n+These callback functions must be provided as non-static functions in a C file,\n+and named ``cmd_<cmdname>_parsed``.\n+The function prototypes can be seen in the generated output header.\n+\n+The \"cmdname\" part of the function name is built up by combining the non-variable initial tokens in the command.\n+So, given the commands in our worked example below: ``quit`` and ``show port stats <n>``,\n+the callback functions would be:\n+\n+.. code:: c\n+\n+ void\n+ cmd_quit_parsed(void *parsed_result, struct cmdline *cl, void *data)\n+ {\n+ ...\n+ }\n+\n+ void\n+ cmd_show_port_stats_parsed(void *parsed_result, struct cmdline *cl, void *data)\n+ {\n+ ...\n+ }\n+\n+These functions must be provided by the developer, but, as stated above,\n+stub functions may be generated by the script automatically using the ``--stubs`` parameter.\n+\n+The same \"cmdname\" stem is used in the naming of the generated structures too.\n+To get at the results structure for each command above,\n+the ``parsed_result`` parameter should be cast to ``struct cmd_quit_result``\n+or ``struct cmd_show_port_stats_result`` respectively.\n+\n+Integrating with the Application\n+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n+\n+To integrate the script output with the application,\n+we must ``#include`` the generated header into our applications C file,\n+and then have the command-line created via either ``cmdline_new`` or ``cmdline_stdin_new``.\n+The first parameter to the function call should be the context array in the generated header file,\n+``ctx`` by default. (Modifiable via script parameter).\n+\n+The callback functions may be in this same file, or in a separate one -\n+they just need to be available to the linker at build-time.\n+\n+Limitations of the Script Approach\n+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n+\n+The script approach works for most commands that a user may wish to add to an application.\n+However, it does not support the full range of functions possible with the DPDK command-line library.\n+For example,\n+it is not possible using the script to multiplex multiple commands into a single callback function.\n+To use this functionality, the user should follow the instructions in the next section\n+`Worked Example of Adding Command-line to an Application`_ to manually configure a command-line instance.\n+\n+Worked Example of Adding Command-line to an Application\n+-------------------------------------------------------\n+\n+The next few subsections will cover each of the steps listed in `Adding Command-line to an Application`_ in more detail,\n working through an example to add two commands to a command-line instance.\n Those two commands will be:\n \n", "prefixes": [ "v3", "2/5" ] }