[RFC,v2,1/1] devtools: add vscode configuration generator
Checks
Commit Message
A lot of developers use Visual Studio Code as their primary IDE. This
script generates a configuration file for VSCode that sets up basic build
tasks, launch tasks, as well as C/C++ code analysis settings that will
take into account compile_commands.json that is automatically generated
by meson.
Files generated by script:
- .vscode/settings.json: stores variables needed by other files
- .vscode/tasks.json: defines build tasks
- .vscode/launch.json: defines launch tasks
- .vscode/c_cpp_properties.json: defines code analysis settings
The script uses a combination of globbing and meson file parsing to
discover available apps, examples, and drivers, and generates a
project-wide settings file, so that the user can later switch between
debug/release/etc. configurations while keeping their desired apps,
examples, and drivers, built by meson, and ensuring launch configurations
still work correctly whatever the configuration selected.
This script uses whiptail as TUI, which is expected to be universally
available as it is shipped by default on most major distributions.
However, the script is also designed to be scriptable and can be run
without user interaction, and have its configuration supplied from
command-line arguments.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
Notes:
RFCv1 -> RFCv2:
- No longer disable apps and drivers if nothing was specified via command line
or TUI, and warn user about things being built by default
- Generate app launch configuration by default for when no apps are selected
- Added paramters:
- --force to avoid overwriting existing config
- --common-conf to specify global meson flags applicable to all configs
- --gdbsudo/--no-gdbsudo to specify gdbsudo behavior
- Autodetect gdbsudo/gdb from UID
- Updated comments, error messages, fixed issues with user interaction
- Improved handling of wildcards and driver dependencies
- Fixed a few bugs in dependency detection due to incorrect parsing
- [Stephen] flake8 is happy
devtools/gen-vscode-config.py | 871 ++++++++++++++++++++++++++++++++++
1 file changed, 871 insertions(+)
create mode 100755 devtools/gen-vscode-config.py
Comments
On Mon, Jul 29, 2024 at 02:05:52PM +0100, Anatoly Burakov wrote:
> A lot of developers use Visual Studio Code as their primary IDE. This
> script generates a configuration file for VSCode that sets up basic build
> tasks, launch tasks, as well as C/C++ code analysis settings that will
> take into account compile_commands.json that is automatically generated
> by meson.
>
> Files generated by script:
> - .vscode/settings.json: stores variables needed by other files
> - .vscode/tasks.json: defines build tasks
> - .vscode/launch.json: defines launch tasks
> - .vscode/c_cpp_properties.json: defines code analysis settings
>
> The script uses a combination of globbing and meson file parsing to
> discover available apps, examples, and drivers, and generates a
> project-wide settings file, so that the user can later switch between
> debug/release/etc. configurations while keeping their desired apps,
> examples, and drivers, built by meson, and ensuring launch configurations
> still work correctly whatever the configuration selected.
>
> This script uses whiptail as TUI, which is expected to be universally
> available as it is shipped by default on most major distributions.
> However, the script is also designed to be scriptable and can be run
> without user interaction, and have its configuration supplied from
> command-line arguments.
>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---
>
Not sure where it would go - contributors guide probably - but I think this
script could do with some docs, especially a quick-setup example on how to
use. Having it in the docs will also make it more likely someone will use
this.
/Bruce
On 7/29/2024 3:14 PM, Bruce Richardson wrote:
> On Mon, Jul 29, 2024 at 02:05:52PM +0100, Anatoly Burakov wrote:
>> A lot of developers use Visual Studio Code as their primary IDE. This
>> script generates a configuration file for VSCode that sets up basic build
>> tasks, launch tasks, as well as C/C++ code analysis settings that will
>> take into account compile_commands.json that is automatically generated
>> by meson.
>>
>> Files generated by script:
>> - .vscode/settings.json: stores variables needed by other files
>> - .vscode/tasks.json: defines build tasks
>> - .vscode/launch.json: defines launch tasks
>> - .vscode/c_cpp_properties.json: defines code analysis settings
>>
>> The script uses a combination of globbing and meson file parsing to
>> discover available apps, examples, and drivers, and generates a
>> project-wide settings file, so that the user can later switch between
>> debug/release/etc. configurations while keeping their desired apps,
>> examples, and drivers, built by meson, and ensuring launch configurations
>> still work correctly whatever the configuration selected.
>>
>> This script uses whiptail as TUI, which is expected to be universally
>> available as it is shipped by default on most major distributions.
>> However, the script is also designed to be scriptable and can be run
>> without user interaction, and have its configuration supplied from
>> command-line arguments.
>>
>> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
>> ---
>>
> Not sure where it would go - contributors guide probably - but I think this
> script could do with some docs, especially a quick-setup example on how to
> use. Having it in the docs will also make it more likely someone will use
> this.
>
> /Bruce
Yep, this is the next step :) I've left it until v1 to add docs.
On Mon, Jul 29, 2024 at 02:05:52PM +0100, Anatoly Burakov wrote:
> A lot of developers use Visual Studio Code as their primary IDE. This
> script generates a configuration file for VSCode that sets up basic build
> tasks, launch tasks, as well as C/C++ code analysis settings that will
> take into account compile_commands.json that is automatically generated
> by meson.
>
> Files generated by script:
> - .vscode/settings.json: stores variables needed by other files
> - .vscode/tasks.json: defines build tasks
> - .vscode/launch.json: defines launch tasks
> - .vscode/c_cpp_properties.json: defines code analysis settings
>
> The script uses a combination of globbing and meson file parsing to
> discover available apps, examples, and drivers, and generates a
> project-wide settings file, so that the user can later switch between
> debug/release/etc. configurations while keeping their desired apps,
> examples, and drivers, built by meson, and ensuring launch configurations
> still work correctly whatever the configuration selected.
>
> This script uses whiptail as TUI, which is expected to be universally
> available as it is shipped by default on most major distributions.
> However, the script is also designed to be scriptable and can be run
> without user interaction, and have its configuration supplied from
> command-line arguments.
>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---
>
Just was trying this out, nice script, thanks.
Initial thoughts concerning the build directory:
- the script doesn't actually create the build directory, so there is no
guarantee that the build directory created will have the same parameters
as that specified in the script run. I'd suggest in the case where the
user runs the script and specifies build settings, that the build
directory is then configured using those settings.
- On the other hand, when the build directory already exists - I think the
script should pull all settings from there, rather than prompting the
user.
- I'm not sure I like the idea for reconfiguring of just removing the build
directory and doing a whole meson setup command all over again. This
seems excessive and also removes the possibility of the user having made
changes in config to the build dir without re-running the whole config
script. For example, having tweaked the LTO setting, or the
instruction_set_isa_setting. Rather than deleting it and running meson
setup, it would be better to use "meson configure" to adjust the one
required setting and let ninja figure out how to propagate that change.
That saves the script from having to track all meson parameters itself.
- Finally, and semi-related, this script assumes that the user does
everything in a single build directory. Just something to consider, but
my own workflow till now has tended to keep multiple build directories
around, generally a "build" directory, which is either release or
debugoptimized type, and a separate "build-debug" directory + occasionally
others for build testing. When doing incremental builds, the time taken to
do two builds following a change is a lot less noticable than the time taken
for periodic switches of a single build directory between debug and release
mode.
Final thoughts on usability:
- Please don't write gdbsudo to /usr/local/bin without asking the user
first. Instead I think it should default to $HOME/.local/bin, but with a
prompt for the user to specify a path.
- While I realise your primary concern here is an interactive script, I'd
tend towards requiring a cmdline arg to run in interactive mode and
instead printing the help usage when run without parameters. Just a
personal preference on my part though.
/Bruce
On 7/29/2024 4:30 PM, Bruce Richardson wrote:
> On Mon, Jul 29, 2024 at 02:05:52PM +0100, Anatoly Burakov wrote:
>> A lot of developers use Visual Studio Code as their primary IDE. This
>> script generates a configuration file for VSCode that sets up basic build
>> tasks, launch tasks, as well as C/C++ code analysis settings that will
>> take into account compile_commands.json that is automatically generated
>> by meson.
>>
>> Files generated by script:
>> - .vscode/settings.json: stores variables needed by other files
>> - .vscode/tasks.json: defines build tasks
>> - .vscode/launch.json: defines launch tasks
>> - .vscode/c_cpp_properties.json: defines code analysis settings
>>
>> The script uses a combination of globbing and meson file parsing to
>> discover available apps, examples, and drivers, and generates a
>> project-wide settings file, so that the user can later switch between
>> debug/release/etc. configurations while keeping their desired apps,
>> examples, and drivers, built by meson, and ensuring launch configurations
>> still work correctly whatever the configuration selected.
>>
>> This script uses whiptail as TUI, which is expected to be universally
>> available as it is shipped by default on most major distributions.
>> However, the script is also designed to be scriptable and can be run
>> without user interaction, and have its configuration supplied from
>> command-line arguments.
>>
>> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
>> ---
>>
> Just was trying this out, nice script, thanks.
Thanks for the feedback! Comments below.
>
> Initial thoughts concerning the build directory:
> - the script doesn't actually create the build directory, so there is no
> guarantee that the build directory created will have the same parameters
> as that specified in the script run. I'd suggest in the case where the
> user runs the script and specifies build settings, that the build
> directory is then configured using those settings.
I'm not sure I follow.
The script creates a command for VSCode to create a build directory
using configuration the user has supplied at script's run time. The
directory is not created by the script, that is the job of meson build
system. This script is merely codifying commands for meson to do that,
with the expectation that the user is familiar with VSCode workflow and
will go straight to build commands anyway, and will pick one of them.
Are you suggesting running `meson setup` right after?
Assuming we do that, it would actually then be possible to adjust launch
tasks to only include *actual* built apps/examples (as well as infer
things like platform, compiler etc.), as that's one weakness of my
current "flying blind" approach, so I wouldn't be opposed to adding an
extra step here, just want to make sure I understand what you're saying
correctly.
>
> - On the other hand, when the build directory already exists - I think the
> script should pull all settings from there, rather than prompting the
> user.
>
That can be done, however, my own workflow has been that I do not ever
keep build directories inside my source directory, so it would not be
possible to pick up directories anywhere but the source directory.
I also think from the point of view of the script it would be easier to
start from known quantities rather than guess what user was trying to do
from current configuration, but I guess a few common-sense heuristics
should suffice for most use cases, such as e.g. inferring debug builds.
> - I'm not sure I like the idea for reconfiguring of just removing the build
> directory and doing a whole meson setup command all over again. This
> seems excessive and also removes the possibility of the user having made
> changes in config to the build dir without re-running the whole config
> script. For example, having tweaked the LTO setting, or the
> instruction_set_isa_setting. Rather than deleting it and running meson
> setup, it would be better to use "meson configure" to adjust the one
> required setting and let ninja figure out how to propagate that change.
> That saves the script from having to track all meson parameters itself.
Last I checked, meson doesn't have a command that would "setup or
configure existing" a directory, it's either "set up new one" or
"configure existing one". I guess we could set up a fallback of
"configure || setup".
>
> - Finally, and semi-related, this script assumes that the user does
> everything in a single build directory. Just something to consider, but
> my own workflow till now has tended to keep multiple build directories
> around, generally a "build" directory, which is either release or
> debugoptimized type, and a separate "build-debug" directory + occasionally
> others for build testing. When doing incremental builds, the time taken to
> do two builds following a change is a lot less noticable than the time taken
> for periodic switches of a single build directory between debug and release
> mode.
The problem with that approach is the launch tasks, because a launch
task can only ever point to one executable, so if you have multiple
build directories, you'll have to have multiple launch tasks per
app/example. I guess we can either tag them (e.g. "Launch dpdk-testpmd
[debug]", "Launch dpdk-testpmd [asan]" etc.), or use some kind of
indirection to "select active build configuration" (e.g. have one launch
task but overwrite ${config:BUILDDIR} after request for configuration,
so that launch tasks would pick up actual executable path at run time
from settings). I would prefer the latter to be honest, as it's much
easier to drop a script into ./vscode and run it together with
"configure" command to switch between different build/launch
configurations. What do you think?
>
> Final thoughts on usability:
>
> - Please don't write gdbsudo to /usr/local/bin without asking the user
> first. Instead I think it should default to $HOME/.local/bin, but with a
> prompt for the user to specify a path.
It's not creating anything, it's just printing out a snippet, which, if
run by user, would do that - the implication is obviously that the user
may correct it if necessary. The script actually picks up path to
`gdbsudo` from `which` command, so if the user puts their command to
$HOME/.local/bin or something, it would get picked up if it's in PATH. I
see your point about maybe suggesting using a home directory path
instead of a system wide path, I can change that.
>
> - While I realise your primary concern here is an interactive script, I'd
> tend towards requiring a cmdline arg to run in interactive mode and
> instead printing the help usage when run without parameters. Just a
> personal preference on my part though.
I found it to be much faster to pick my targets, apps etc. using a
couple of interactive windows than to type out parameters I probably
don't even remember ahead of time (especially build configurations!),
and I believe it's more newbie-friendly that way, as I imagine very few
people will want to learn arguments for yet-another-script just to start
using VSCode. It would be my personal preference to leave it as
default-to-TUI, but maybe recognizing a widely used `-i` parameter would
be a good compromise for instant familiarity.
>
> /Bruce
On Mon, Jul 29, 2024 at 06:16:48PM +0200, Burakov, Anatoly wrote:
> On 7/29/2024 4:30 PM, Bruce Richardson wrote:
> > On Mon, Jul 29, 2024 at 02:05:52PM +0100, Anatoly Burakov wrote:
> > > A lot of developers use Visual Studio Code as their primary IDE. This
> > > script generates a configuration file for VSCode that sets up basic build
> > > tasks, launch tasks, as well as C/C++ code analysis settings that will
> > > take into account compile_commands.json that is automatically generated
> > > by meson.
> > >
> > > Files generated by script:
> > > - .vscode/settings.json: stores variables needed by other files
> > > - .vscode/tasks.json: defines build tasks
> > > - .vscode/launch.json: defines launch tasks
> > > - .vscode/c_cpp_properties.json: defines code analysis settings
> > >
> > > The script uses a combination of globbing and meson file parsing to
> > > discover available apps, examples, and drivers, and generates a
> > > project-wide settings file, so that the user can later switch between
> > > debug/release/etc. configurations while keeping their desired apps,
> > > examples, and drivers, built by meson, and ensuring launch configurations
> > > still work correctly whatever the configuration selected.
> > >
> > > This script uses whiptail as TUI, which is expected to be universally
> > > available as it is shipped by default on most major distributions.
> > > However, the script is also designed to be scriptable and can be run
> > > without user interaction, and have its configuration supplied from
> > > command-line arguments.
> > >
> > > Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> > > ---
> > >
> > Just was trying this out, nice script, thanks.
>
> Thanks for the feedback! Comments below.
>
> >
> > Initial thoughts concerning the build directory:
> > - the script doesn't actually create the build directory, so there is no
> > guarantee that the build directory created will have the same parameters
> > as that specified in the script run. I'd suggest in the case where the
> > user runs the script and specifies build settings, that the build
> > directory is then configured using those settings.
>
> I'm not sure I follow.
>
> The script creates a command for VSCode to create a build directory using
> configuration the user has supplied at script's run time. The directory is
> not created by the script, that is the job of meson build system. This
> script is merely codifying commands for meson to do that, with the
> expectation that the user is familiar with VSCode workflow and will go
> straight to build commands anyway, and will pick one of them. Are you
> suggesting running `meson setup` right after?
>
Yes, that is what I was thinking, based on the assumption that running
"meson setup" should be a once-only task. I suppose overall it's a
different workflow to what you have - where you run meson setup repeatedly
each time you change a build type. My thinking for the approach to take
here is that your script firstly asks for a build directory, and then:
* if build-dir exists, pull settings you need from there, such as build
type and the apps being built.
* if not existing, ask for config settings as you do now, and then run
meson setup to create the build dir.
Thereafter, the source for all build settings is not in vscode, but in
meson, and you just use "meson configure" from vscode to tweak whatever
needs tweaking, without affecting any other settings. Since you don't
affect any other settings there is no need to record what they should be.
> Assuming we do that, it would actually then be possible to adjust launch
> tasks to only include *actual* built apps/examples (as well as infer things
> like platform, compiler etc.), as that's one weakness of my current "flying
> blind" approach, so I wouldn't be opposed to adding an extra step here, just
> want to make sure I understand what you're saying correctly.
>
> >
> > - On the other hand, when the build directory already exists - I think the
> > script should pull all settings from there, rather than prompting the
> > user.
> >
>
> That can be done, however, my own workflow has been that I do not ever keep
> build directories inside my source directory, so it would not be possible to
> pick up directories anywhere but the source directory.
>
Why is that, and how does it work now, for e.g. getting the
compile_commands.json file from your build folder?
> I also think from the point of view of the script it would be easier to
> start from known quantities rather than guess what user was trying to do
> from current configuration, but I guess a few common-sense heuristics should
> suffice for most use cases, such as e.g. inferring debug builds.
>
What you need depends on whether you want to keep running "meson setup" -
which means you need to track all settings - or want to use "meson
configure" where you don't really need to track much at all.
> > - I'm not sure I like the idea for reconfiguring of just removing the build
> > directory and doing a whole meson setup command all over again. This
> > seems excessive and also removes the possibility of the user having made
> > changes in config to the build dir without re-running the whole config
> > script. For example, having tweaked the LTO setting, or the
> > instruction_set_isa_setting. Rather than deleting it and running meson
> > setup, it would be better to use "meson configure" to adjust the one
> > required setting and let ninja figure out how to propagate that change.
> > That saves the script from having to track all meson parameters itself.
>
> Last I checked, meson doesn't have a command that would "setup or configure
> existing" a directory, it's either "set up new one" or "configure existing
> one". I guess we could set up a fallback of "configure || setup".
>
This goes back to the whole "create build directory after running the
script" option. If the script creates the build dir, the vscode commands
never need to use meson setup, only ever meson configure.
> >
> > - Finally, and semi-related, this script assumes that the user does
> > everything in a single build directory. Just something to consider, but
> > my own workflow till now has tended to keep multiple build directories
> > around, generally a "build" directory, which is either release or
> > debugoptimized type, and a separate "build-debug" directory + occasionally
> > others for build testing. When doing incremental builds, the time taken to
> > do two builds following a change is a lot less noticable than the time taken
> > for periodic switches of a single build directory between debug and release
> > mode.
>
> The problem with that approach is the launch tasks, because a launch task
> can only ever point to one executable, so if you have multiple build
> directories, you'll have to have multiple launch tasks per app/example. I
> guess we can either tag them (e.g. "Launch dpdk-testpmd [debug]", "Launch
> dpdk-testpmd [asan]" etc.), or use some kind of indirection to "select
> active build configuration" (e.g. have one launch task but overwrite
> ${config:BUILDDIR} after request for configuration, so that launch tasks
> would pick up actual executable path at run time from settings). I would
> prefer the latter to be honest, as it's much easier to drop a script into
> ./vscode and run it together with "configure" command to switch between
> different build/launch configurations. What do you think?
>
I think I'd prefer the former actually - to have explicit tasks always
listed for debug and release builds.
Not a big deal for me either way, I'll just hack in the extra tasks as I
need them, so it's a low-priority support item for me.
> >
> > Final thoughts on usability:
> >
> > - Please don't write gdbsudo to /usr/local/bin without asking the user
> > first. Instead I think it should default to $HOME/.local/bin, but with a
> > prompt for the user to specify a path.
>
> It's not creating anything, it's just printing out a snippet, which, if run
> by user, would do that - the implication is obviously that the user may
> correct it if necessary. The script actually picks up path to `gdbsudo` from
> `which` command, so if the user puts their command to $HOME/.local/bin or
> something, it would get picked up if it's in PATH. I see your point about
> maybe suggesting using a home directory path instead of a system wide path,
> I can change that.
Yep, thanks, and thanks for the explanation.
BTW: even if the user is running as non-root user, they don't always need
to use sudo (I set up my system to not need it for running DPDK). I see
there is a cmdline option for "no-gdbsudo" but I think you should make that
accessible via TUI also if non-root.
And for the cmdline parameters, how about shortening them to "--sudo" and
"--nosudo". For debug vs release builds you may want to have the latter run
without gdb at all, just with or without sudo.]
>
> >
> > - While I realise your primary concern here is an interactive script, I'd
> > tend towards requiring a cmdline arg to run in interactive mode and
> > instead printing the help usage when run without parameters. Just a
> > personal preference on my part though.
>
> I found it to be much faster to pick my targets, apps etc. using a couple of
> interactive windows than to type out parameters I probably don't even
> remember ahead of time (especially build configurations!), and I believe
> it's more newbie-friendly that way, as I imagine very few people will want
> to learn arguments for yet-another-script just to start using VSCode. It
> would be my personal preference to leave it as default-to-TUI, but maybe
> recognizing a widely used `-i` parameter would be a good compromise for
> instant familiarity.
>
Ok. There's always a -h option for me to get the cmdline parameters.
I also think if the script is ok with working off an existing build
directory (or directories!), and only prompting for that, it would remove
for me the real necessity of asking for a more cmdline-fieldly version.
On 7/29/2024 6:41 PM, Bruce Richardson wrote:
> On Mon, Jul 29, 2024 at 06:16:48PM +0200, Burakov, Anatoly wrote:
>> On 7/29/2024 4:30 PM, Bruce Richardson wrote:
>>> On Mon, Jul 29, 2024 at 02:05:52PM +0100, Anatoly Burakov wrote:
>>>> A lot of developers use Visual Studio Code as their primary IDE. This
>>>> script generates a configuration file for VSCode that sets up basic build
>>>> tasks, launch tasks, as well as C/C++ code analysis settings that will
>>>> take into account compile_commands.json that is automatically generated
>>>> by meson.
>>>>
>>>> Files generated by script:
>>>> - .vscode/settings.json: stores variables needed by other files
>>>> - .vscode/tasks.json: defines build tasks
>>>> - .vscode/launch.json: defines launch tasks
>>>> - .vscode/c_cpp_properties.json: defines code analysis settings
>>>>
>>>> The script uses a combination of globbing and meson file parsing to
>>>> discover available apps, examples, and drivers, and generates a
>>>> project-wide settings file, so that the user can later switch between
>>>> debug/release/etc. configurations while keeping their desired apps,
>>>> examples, and drivers, built by meson, and ensuring launch configurations
>>>> still work correctly whatever the configuration selected.
>>>>
>>>> This script uses whiptail as TUI, which is expected to be universally
>>>> available as it is shipped by default on most major distributions.
>>>> However, the script is also designed to be scriptable and can be run
>>>> without user interaction, and have its configuration supplied from
>>>> command-line arguments.
>>>>
>>>> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
>>>> ---
>>>>
>>> Just was trying this out, nice script, thanks.
>>
>> Thanks for the feedback! Comments below.
>>
>>>
>>> Initial thoughts concerning the build directory:
>>> - the script doesn't actually create the build directory, so there is no
>>> guarantee that the build directory created will have the same parameters
>>> as that specified in the script run. I'd suggest in the case where the
>>> user runs the script and specifies build settings, that the build
>>> directory is then configured using those settings.
>>
>> I'm not sure I follow.
>>
>> The script creates a command for VSCode to create a build directory using
>> configuration the user has supplied at script's run time. The directory is
>> not created by the script, that is the job of meson build system. This
>> script is merely codifying commands for meson to do that, with the
>> expectation that the user is familiar with VSCode workflow and will go
>> straight to build commands anyway, and will pick one of them. Are you
>> suggesting running `meson setup` right after?
>>
>
> Yes, that is what I was thinking, based on the assumption that running
> "meson setup" should be a once-only task. I suppose overall it's a
> different workflow to what you have - where you run meson setup repeatedly
> each time you change a build type. My thinking for the approach to take
> here is that your script firstly asks for a build directory, and then:
> * if build-dir exists, pull settings you need from there, such as build
> type and the apps being built.
> * if not existing, ask for config settings as you do now, and then run
> meson setup to create the build dir.
I guess the disconnect comes from the fact that I treat meson build
directories as transient and disposable and never hesitate to wipe them
and start over, whereas you seem to favor persistence. Since 99% of the
time I'm using heavily reduced builds anyway (e.g. one app, one driver),
repeatedly doing meson setup doesn't really hinder me in any way, but I
suppose it would if I had more meaty builds.
I think we also have a slightly different view of what this script
should be - I envisioned a "one-stop-shop" for "open freshly cloned DPDK
directory, run one script and off you go" (which stems from the fact
that I heavily rely on Git Worktrees [1] in my workflow, so having an
untouched source directory without any configuration in it is something
I am constantly faced with), while you seem to favor picking up existing
meson build and building a VSCode configuration around it.
I can see point in this, and I guess I actually unwittingly rolled two
scripts into one - a TUI meson frontend, and a VSCode configuration
generator. Perhaps it should very well be two scripts, not one? Because
honestly, having something like what I built for TUI (the meson
configuration frontend) is something I missed a lot and something I
always wished our long-gone `setup.sh` script had, but didn't, and now
with meson it's much simpler to do but we still don't have anything like
that. Maybe this is our opportunity to provide a quicker "quick start"
script, one that I could run and configure meson build without having to
type anything. WDYT?
>
> Thereafter, the source for all build settings is not in vscode, but in
> meson, and you just use "meson configure" from vscode to tweak whatever
> needs tweaking, without affecting any other settings. Since you don't
> affect any other settings there is no need to record what they should be.
Why would I be using meson configure from vscode then? I mean, the whole
notion of having tasks in VSCode is so that I define a few common
configurations and never touch them until I'm working on something else.
If I have to type in configure commands anyway (because I would have to,
to adjust configuration in meson), I might as well do so using terminal,
and avoid dealing with meson from VSCode entirely?
(technically, there's a way to read arguments from a popup window - I
suppose we could have this step recorded as a VSCode task)
>
>
>> Assuming we do that, it would actually then be possible to adjust launch
>> tasks to only include *actual* built apps/examples (as well as infer things
>> like platform, compiler etc.), as that's one weakness of my current "flying
>> blind" approach, so I wouldn't be opposed to adding an extra step here, just
>> want to make sure I understand what you're saying correctly.
>>
>>>
>>> - On the other hand, when the build directory already exists - I think the
>>> script should pull all settings from there, rather than prompting the
>>> user.
>>>
>>
>> That can be done, however, my own workflow has been that I do not ever keep
>> build directories inside my source directory, so it would not be possible to
>> pick up directories anywhere but the source directory.
>>
>
> Why is that, and how does it work now, for e.g. getting the
> compile_commands.json file from your build folder?
Right now, at configuration time I store build directory in
settings.json, and all of the other tasks (build, launch, C++ analysis)
pick it up via ${config:...} variable. This is why I suggested having a
notion of "active configuration" - if we just rewrite that variable,
both ninja and launch tasks can be switched to a different build
directory without actually having to create new tasks. More on that
below, as that raises a few questions.
As for why, it's a personal preference - it's annoying to have the
build/ directory in my file view (it's possible to hide it, I guess I
could automatically add a setting to do that in settings.json), and it
interferes with e.g. source-tree-wide searches and stuff like that. This
isn't a hard requirement though, I can switch to in-tree builds and
automatic directory hiding if it means more automation :)
>
>> I also think from the point of view of the script it would be easier to
>> start from known quantities rather than guess what user was trying to do
>> from current configuration, but I guess a few common-sense heuristics should
>> suffice for most use cases, such as e.g. inferring debug builds.
>>
>
> What you need depends on whether you want to keep running "meson setup" -
> which means you need to track all settings - or want to use "meson
> configure" where you don't really need to track much at all.
I guess I was aiming for the former, but doing only the latter makes
sense if we assume we want to separate setup from VSCode config
generation (which it seems like we're heading that way).
>
>>> - I'm not sure I like the idea for reconfiguring of just removing the build
>>> directory and doing a whole meson setup command all over again. This
>>> seems excessive and also removes the possibility of the user having made
>>> changes in config to the build dir without re-running the whole config
>>> script. For example, having tweaked the LTO setting, or the
>>> instruction_set_isa_setting. Rather than deleting it and running meson
>>> setup, it would be better to use "meson configure" to adjust the one
>>> required setting and let ninja figure out how to propagate that change.
>>> That saves the script from having to track all meson parameters itself.
>>
>> Last I checked, meson doesn't have a command that would "setup or configure
>> existing" a directory, it's either "set up new one" or "configure existing
>> one". I guess we could set up a fallback of "configure || setup".
>>
>
> This goes back to the whole "create build directory after running the
> script" option. If the script creates the build dir, the vscode commands
> never need to use meson setup, only ever meson configure.
Sure, and like I suggested above, it doesn't even have to be *this*
script, it can be another, a Python-based TUI reimplementation of
setup.sh :)
>
>>>
>>> - Finally, and semi-related, this script assumes that the user does
>>> everything in a single build directory. Just something to consider, but
>>> my own workflow till now has tended to keep multiple build directories
>>> around, generally a "build" directory, which is either release or
>>> debugoptimized type, and a separate "build-debug" directory + occasionally
>>> others for build testing. When doing incremental builds, the time taken to
>>> do two builds following a change is a lot less noticable than the time taken
>>> for periodic switches of a single build directory between debug and release
>>> mode.
>>
>> The problem with that approach is the launch tasks, because a launch task
>> can only ever point to one executable, so if you have multiple build
>> directories, you'll have to have multiple launch tasks per app/example. I
>> guess we can either tag them (e.g. "Launch dpdk-testpmd [debug]", "Launch
>> dpdk-testpmd [asan]" etc.), or use some kind of indirection to "select
>> active build configuration" (e.g. have one launch task but overwrite
>> ${config:BUILDDIR} after request for configuration, so that launch tasks
>> would pick up actual executable path at run time from settings). I would
>> prefer the latter to be honest, as it's much easier to drop a script into
>> ./vscode and run it together with "configure" command to switch between
>> different build/launch configurations. What do you think?
>>
>
> I think I'd prefer the former actually - to have explicit tasks always
> listed for debug and release builds.
> Not a big deal for me either way, I'll just hack in the extra tasks as I
> need them, so it's a low-priority support item for me.
There's another issue though: code analysis. If you have multiple build
directories, your C++ analysis settings (the
.vscode/c_cpp_properties.json file) can only ever use one specific
compile_commands.json from a specific build directory. I think if we
were to support having multiple build dirs, we would *have to* implement
something like "switch active configuration" thing anyway.
Whether we add tagged launch tasks is honestly a problem to be solved,
because on the one hand we want them to be generated automatically
(ideally after configure step), and on the other we also want to not
interfere with any custom configuration the user has already added (such
as e.g. command line arguments to existing launch tasks). We'll probably
have to do config file parsing and updating the configuration, rather
than regenerating it each time. Python's JSON also doesn't support
comments, so for any comments that were added to configurations, we'd
either lose them, or find a way to reinsert them post-generation.
>
>>>
>>> Final thoughts on usability:
>>>
>>> - Please don't write gdbsudo to /usr/local/bin without asking the user
>>> first. Instead I think it should default to $HOME/.local/bin, but with a
>>> prompt for the user to specify a path.
>>
>> It's not creating anything, it's just printing out a snippet, which, if run
>> by user, would do that - the implication is obviously that the user may
>> correct it if necessary. The script actually picks up path to `gdbsudo` from
>> `which` command, so if the user puts their command to $HOME/.local/bin or
>> something, it would get picked up if it's in PATH. I see your point about
>> maybe suggesting using a home directory path instead of a system wide path,
>> I can change that.
>
> Yep, thanks, and thanks for the explanation.
> BTW: even if the user is running as non-root user, they don't always need
> to use sudo (I set up my system to not need it for running DPDK). I see
> there is a cmdline option for "no-gdbsudo" but I think you should make that
> accessible via TUI also if non-root.
Sure, all of that can be done.
>
> And for the cmdline parameters, how about shortening them to "--sudo" and
> "--nosudo". For debug vs release builds you may want to have the latter run
> without gdb at all, just with or without sudo.]
I honestly never run anything outside gdb, but that's a consequence of
me not really working on things that routinely require it (e.g.
performace code). We can add that no problem though.
>
>>
>>>
>>> - While I realise your primary concern here is an interactive script, I'd
>>> tend towards requiring a cmdline arg to run in interactive mode and
>>> instead printing the help usage when run without parameters. Just a
>>> personal preference on my part though.
>>
>> I found it to be much faster to pick my targets, apps etc. using a couple of
>> interactive windows than to type out parameters I probably don't even
>> remember ahead of time (especially build configurations!), and I believe
>> it's more newbie-friendly that way, as I imagine very few people will want
>> to learn arguments for yet-another-script just to start using VSCode. It
>> would be my personal preference to leave it as default-to-TUI, but maybe
>> recognizing a widely used `-i` parameter would be a good compromise for
>> instant familiarity.
>>
>
> Ok. There's always a -h option for me to get the cmdline parameters.
>
> I also think if the script is ok with working off an existing build
> directory (or directories!), and only prompting for that, it would remove
> for me the real necessity of asking for a more cmdline-fieldly version.
It sounds like this would really be something that a setup script would
do better than a VSCode config generator.
So, assuming we want to move setup steps to another script and
concentrate on VSCode configuration exclusively, my thinking of how it
would work is as follows:
1) Assume we want multiple build directories, suggest automatically
picking them up from source directory but support specifying one or more
from command line arguments (or TUI, although I suspect if setup moves
to a separate script, there's very little need for TUI in VSCode config
generator - it should be a mostly mechanical process at that point)
2) Now that we track multiple build directories, we can store them in a
YAML file or something (e.g. .vscode/.dpdk_builds.yaml), and create
tasks to switch between them as "active" to support e.g. different code
analysis settings and stuff like that
3) All build tasks can work off "active configuration" which means we
don't need multiple compile tasks, but for launch tasks we may need
different ones because different build dirs may have different launch tasks
Let's assume user just ran `meson configure` and changed something about
one of their configurations. What do you think should happen next? I
mean, if they added/removed an app/example to the build, we can detect
that and auto-generate (or remove) a launch task, for example? Or do you
think it should be a manual step, e.g. user should explicitly request
regenerating/updating launch tasks? Maybe there should be an --update
flag, to indicate that we're not creating new configurations but merely
refreshing existing ones?
[1] https://git-scm.com/docs/git-worktree
On Tue, Jul 30, 2024 at 11:21:25AM +0200, Burakov, Anatoly wrote:
> On 7/29/2024 6:41 PM, Bruce Richardson wrote:
> > On Mon, Jul 29, 2024 at 06:16:48PM +0200, Burakov, Anatoly wrote:
> > > On 7/29/2024 4:30 PM, Bruce Richardson wrote:
> > > > On Mon, Jul 29, 2024 at 02:05:52PM +0100, Anatoly Burakov wrote:
> > > > > A lot of developers use Visual Studio Code as their primary IDE. This
> > > > > script generates a configuration file for VSCode that sets up basic build
> > > > > tasks, launch tasks, as well as C/C++ code analysis settings that will
> > > > > take into account compile_commands.json that is automatically generated
> > > > > by meson.
> > > > >
> > > > > Files generated by script:
> > > > > - .vscode/settings.json: stores variables needed by other files
> > > > > - .vscode/tasks.json: defines build tasks
> > > > > - .vscode/launch.json: defines launch tasks
> > > > > - .vscode/c_cpp_properties.json: defines code analysis settings
> > > > >
> > > > > The script uses a combination of globbing and meson file parsing to
> > > > > discover available apps, examples, and drivers, and generates a
> > > > > project-wide settings file, so that the user can later switch between
> > > > > debug/release/etc. configurations while keeping their desired apps,
> > > > > examples, and drivers, built by meson, and ensuring launch configurations
> > > > > still work correctly whatever the configuration selected.
> > > > >
> > > > > This script uses whiptail as TUI, which is expected to be universally
> > > > > available as it is shipped by default on most major distributions.
> > > > > However, the script is also designed to be scriptable and can be run
> > > > > without user interaction, and have its configuration supplied from
> > > > > command-line arguments.
> > > > >
> > > > > Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> > > > > ---
> > > > >
> > > > Just was trying this out, nice script, thanks.
> > >
> > > Thanks for the feedback! Comments below.
> > >
More comments inline below, but summarising after the fact here.
Still not entirely sure what way is best for all this so please take all
current and previous suggestions with a pinch of salt. Based off what you
suggest and the ongoing discuss my current thinking is:
* +1 to split the vscode config generation from the TUI. Both are also
targetting different end-users - the TUI is for everyone looking to build
DPDK, both devs and users, while the vscode config is for developers only.
* Let's ignore the multi-build-directory setup for now, if it makes it more
complex for the simple cases of one build-dir.
* I think we should investigate having the vscode config generated from
meson rather than the other way around.
See also below.
/Bruce
> > > >
> > > > Initial thoughts concerning the build directory:
> > > > - the script doesn't actually create the build directory, so there is no
> > > > guarantee that the build directory created will have the same parameters
> > > > as that specified in the script run. I'd suggest in the case where the
> > > > user runs the script and specifies build settings, that the build
> > > > directory is then configured using those settings.
> > >
> > > I'm not sure I follow.
> > >
> > > The script creates a command for VSCode to create a build directory using
> > > configuration the user has supplied at script's run time. The directory is
> > > not created by the script, that is the job of meson build system. This
> > > script is merely codifying commands for meson to do that, with the
> > > expectation that the user is familiar with VSCode workflow and will go
> > > straight to build commands anyway, and will pick one of them. Are you
> > > suggesting running `meson setup` right after?
> > >
> >
> > Yes, that is what I was thinking, based on the assumption that running
> > "meson setup" should be a once-only task. I suppose overall it's a
> > different workflow to what you have - where you run meson setup repeatedly
> > each time you change a build type. My thinking for the approach to take
> > here is that your script firstly asks for a build directory, and then:
> > * if build-dir exists, pull settings you need from there, such as build
> > type and the apps being built.
> > * if not existing, ask for config settings as you do now, and then run
> > meson setup to create the build dir.
>
> I guess the disconnect comes from the fact that I treat meson build
> directories as transient and disposable and never hesitate to wipe them and
> start over, whereas you seem to favor persistence. Since 99% of the time I'm
> using heavily reduced builds anyway (e.g. one app, one driver), repeatedly
> doing meson setup doesn't really hinder me in any way, but I suppose it
> would if I had more meaty builds.
>
It mainly just seems inefficient to me. Ninja does a lot of processing to
minimise the work done whenever you make a configuration change, and you
bypass all that by nuking the directory from orbit (only way to be sure!)
and then creating a new one!
The other main downside of it (to my mind), is that the tracking of
settings for the build needs to be in vscode. I'd prefer meson itself to
be the "one source of truth" and vscode to be tweaking that, rather than
tracking everything itself.
> I think we also have a slightly different view of what this script should be
> - I envisioned a "one-stop-shop" for "open freshly cloned DPDK directory,
> run one script and off you go" (which stems from the fact that I heavily
> rely on Git Worktrees [1] in my workflow, so having an untouched source
> directory without any configuration in it is something I am constantly faced
> with), while you seem to favor picking up existing meson build and building
> a VSCode configuration around it.
>
> I can see point in this, and I guess I actually unwittingly rolled two
> scripts into one - a TUI meson frontend, and a VSCode configuration
> generator. Perhaps it should very well be two scripts, not one? Because
> honestly, having something like what I built for TUI (the meson
> configuration frontend) is something I missed a lot and something I always
> wished our long-gone `setup.sh` script had, but didn't, and now with meson
> it's much simpler to do but we still don't have anything like that. Maybe
> this is our opportunity to provide a quicker "quick start" script, one that
> I could run and configure meson build without having to type anything. WDYT?
>
Splitting it into two makes sense to me, yes, since as you point out there
are really two separate jobs involved here that one may want to roll with
separately.
> >
> > Thereafter, the source for all build settings is not in vscode, but in
> > meson, and you just use "meson configure" from vscode to tweak whatever
> > needs tweaking, without affecting any other settings. Since you don't
> > affect any other settings there is no need to record what they should be.
>
> Why would I be using meson configure from vscode then? I mean, the whole
> notion of having tasks in VSCode is so that I define a few common
> configurations and never touch them until I'm working on something else. If
> I have to type in configure commands anyway (because I would have to, to
> adjust configuration in meson), I might as well do so using terminal, and
> avoid dealing with meson from VSCode entirely?
>
> (technically, there's a way to read arguments from a popup window - I
> suppose we could have this step recorded as a VSCode task)
>
The reason I was thinking about this is that you don't want to expose
dozens of tasks in vscode for tweaking every possible meson setting. Sure,
have build and run tasks for the common options, but for "advanced use"
where the user wants to tweak a build setting, let them just do it from
commandline without having to re-run a config script to adjust the
settings.
<Snip>
> > > >
> > > > - Finally, and semi-related, this script assumes that the user does
> > > > everything in a single build directory. Just something to consider, but
> > > > my own workflow till now has tended to keep multiple build directories
> > > > around, generally a "build" directory, which is either release or
> > > > debugoptimized type, and a separate "build-debug" directory + occasionally
> > > > others for build testing. When doing incremental builds, the time taken to
> > > > do two builds following a change is a lot less noticable than the time taken
> > > > for periodic switches of a single build directory between debug and release
> > > > mode.
> > >
> > > The problem with that approach is the launch tasks, because a launch task
> > > can only ever point to one executable, so if you have multiple build
> > > directories, you'll have to have multiple launch tasks per app/example. I
> > > guess we can either tag them (e.g. "Launch dpdk-testpmd [debug]", "Launch
> > > dpdk-testpmd [asan]" etc.), or use some kind of indirection to "select
> > > active build configuration" (e.g. have one launch task but overwrite
> > > ${config:BUILDDIR} after request for configuration, so that launch tasks
> > > would pick up actual executable path at run time from settings). I would
> > > prefer the latter to be honest, as it's much easier to drop a script into
> > > ./vscode and run it together with "configure" command to switch between
> > > different build/launch configurations. What do you think?
> > >
> >
> > I think I'd prefer the former actually - to have explicit tasks always
> > listed for debug and release builds.
> > Not a big deal for me either way, I'll just hack in the extra tasks as I
> > need them, so it's a low-priority support item for me.
>
> There's another issue though: code analysis. If you have multiple build
> directories, your C++ analysis settings (the .vscode/c_cpp_properties.json
> file) can only ever use one specific compile_commands.json from a specific
> build directory. I think if we were to support having multiple build dirs,
> we would *have to* implement something like "switch active configuration"
> thing anyway.
>
Strictly speaking, yes. However, in my experience using eclipse as an IDE
in the past it doesn't matter that much which or how many build directories
are analysed. However, vscode may well be different in this regard.
Since I don't ever envisage myself doing everything always through vscode,
I'm happy enough with vscode managing a single build directory, and I can
manually worry about a second build directory myself. Maybe let's park the
multi-build-dir stuff for now, unless others feel that it's something they
need.
> Whether we add tagged launch tasks is honestly a problem to be solved,
> because on the one hand we want them to be generated automatically (ideally
> after configure step), and on the other we also want to not interfere with
> any custom configuration the user has already added (such as e.g. command
> line arguments to existing launch tasks). We'll probably have to do config
> file parsing and updating the configuration, rather than regenerating it
> each time. Python's JSON also doesn't support comments, so for any comments
> that were added to configurations, we'd either lose them, or find a way to
> reinsert them post-generation.
>
Have you considered generating the launch tasks from a script launched from
meson itself? Any time the configuration is changed, meson will re-run at
the next build and that can trigger re-generation of whatever vscode config
you need, including launch tasks for all the binaries. This would be
another advantage of splitting the script into two - one should look to make
the vscode-settings generation script usable from meson.
<snip>
> It sounds like this would really be something that a setup script would do
> better than a VSCode config generator.
>
> So, assuming we want to move setup steps to another script and concentrate
> on VSCode configuration exclusively, my thinking of how it would work is as
> follows:
>
> 1) Assume we want multiple build directories, suggest automatically picking
> them up from source directory but support specifying one or more from
> command line arguments (or TUI, although I suspect if setup moves to a
> separate script, there's very little need for TUI in VSCode config generator
> - it should be a mostly mechanical process at that point)
>
I'm probably an outlier, so lets not over-design things for the
multi-build-directory case.
If we think about generating the vscode config from a meson run (via a
"generate-vscode-config" setting or something), that may switch the way in
which things are actually being done. In that case, a build directory would
register itself with the vscode config - creating a new one if not already
present.
> 2) Now that we track multiple build directories, we can store them in a YAML
> file or something (e.g. .vscode/.dpdk_builds.yaml), and create tasks to
> switch between them as "active" to support e.g. different code analysis
> settings and stuff like that
>
Initially, maybe don't add this. For first draft supporting
multi-directories, I'd start by adding prefixed duplicate tasks for each
directory registered.
> 3) All build tasks can work off "active configuration" which means we don't
> need multiple compile tasks, but for launch tasks we may need different ones
> because different build dirs may have different launch tasks
>
Again, I'd just add tasks rather than bothering with active configs.
> Let's assume user just ran `meson configure` and changed something about one
> of their configurations. What do you think should happen next? I mean, if
> they added/removed an app/example to the build, we can detect that and
> auto-generate (or remove) a launch task, for example? Or do you think it
> should be a manual step, e.g. user should explicitly request
> regenerating/updating launch tasks? Maybe there should be an --update flag,
> to indicate that we're not creating new configurations but merely refreshing
> existing ones?
See above, this is a case where having the vscode config script callable
from meson would be perfect.
>
> [1] https://git-scm.com/docs/git-worktree
>
Thanks for the link - seems to be automating a setup which I've been
approximately doing manually for some years now! :-)
On 7/30/2024 12:31 PM, Bruce Richardson wrote:
> On Tue, Jul 30, 2024 at 11:21:25AM +0200, Burakov, Anatoly wrote:
>> On 7/29/2024 6:41 PM, Bruce Richardson wrote:
>>> On Mon, Jul 29, 2024 at 06:16:48PM +0200, Burakov, Anatoly wrote:
>>>> On 7/29/2024 4:30 PM, Bruce Richardson wrote:
>>>>> On Mon, Jul 29, 2024 at 02:05:52PM +0100, Anatoly Burakov wrote:
>>>>>> A lot of developers use Visual Studio Code as their primary IDE. This
>>>>>> script generates a configuration file for VSCode that sets up basic build
>>>>>> tasks, launch tasks, as well as C/C++ code analysis settings that will
>>>>>> take into account compile_commands.json that is automatically generated
>>>>>> by meson.
>>>>>>
>>>>>> Files generated by script:
>>>>>> - .vscode/settings.json: stores variables needed by other files
>>>>>> - .vscode/tasks.json: defines build tasks
>>>>>> - .vscode/launch.json: defines launch tasks
>>>>>> - .vscode/c_cpp_properties.json: defines code analysis settings
>>>>>>
>>>>>> The script uses a combination of globbing and meson file parsing to
>>>>>> discover available apps, examples, and drivers, and generates a
>>>>>> project-wide settings file, so that the user can later switch between
>>>>>> debug/release/etc. configurations while keeping their desired apps,
>>>>>> examples, and drivers, built by meson, and ensuring launch configurations
>>>>>> still work correctly whatever the configuration selected.
>>>>>>
>>>>>> This script uses whiptail as TUI, which is expected to be universally
>>>>>> available as it is shipped by default on most major distributions.
>>>>>> However, the script is also designed to be scriptable and can be run
>>>>>> without user interaction, and have its configuration supplied from
>>>>>> command-line arguments.
>>>>>>
>>>>>> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
>>>>>> ---
>>>>>>
>>>>> Just was trying this out, nice script, thanks.
>>>>
>>>> Thanks for the feedback! Comments below.
>>>>
>
> More comments inline below, but summarising after the fact here.
>
> Still not entirely sure what way is best for all this so please take all
> current and previous suggestions with a pinch of salt. Based off what you
> suggest and the ongoing discuss my current thinking is:
>
> * +1 to split the vscode config generation from the TUI. Both are also
> targetting different end-users - the TUI is for everyone looking to build
> DPDK, both devs and users, while the vscode config is for developers only.
> * Let's ignore the multi-build-directory setup for now, if it makes it more
> complex for the simple cases of one build-dir.
Not really *that* much more complex, IMO. The only real issue is
possible differences in code analysis behavior stemming from having
"wrong" build directory set up as a source of compile_commands.json. If
you're OK with adding multiple tasks per multiple build directories,
then all of the rest of it becomes trivial because if launch tasks are
per-build, they can reference per-build build commands and work
seamlessly using "duplicate" commands.
And even then, for first version we can probably drop the code analysis
angle (just use the first detected config as the source), in which case
we can pretty much support multiple build dirs for free as we'd have to
build all the infrastructure (e.g. config updates etc.) anyway if we
want this process to be seamless.
> * I think we should investigate having the vscode config generated from
> meson rather than the other way around.
It didn't occur to me that it was possible, it sounds like that's really
the way to go!
<snip>
> Strictly speaking, yes. However, in my experience using eclipse as an IDE
> in the past it doesn't matter that much which or how many build directories
> are analysed. However, vscode may well be different in this regard.
Unless the user does *wildly* different things in their build
directories (i.e. two dirs, one of which used for cross-build or
something), I expect things to work without any additional effort, so
you're right in that for most practical purposes, the result wouldn't
really be different to having "proper" C++ analysis configurations.
> Since I don't ever envisage myself doing everything always through vscode,
> I'm happy enough with vscode managing a single build directory, and I can
> manually worry about a second build directory myself. Maybe let's park the
> multi-build-dir stuff for now, unless others feel that it's something they
> need.
Well, I do strive to do most things with VSCode (that's why I had
multiple configurations to begin with!), so it would benefit *my*
workflow to support that :)
new file mode 100755
@@ -0,0 +1,871 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 Intel Corporation
+#
+
+"""Visual Studio Code configuration generator script."""
+
+import os
+import json
+import argparse
+import fnmatch
+import shutil
+from typing import List, Dict, Tuple, Any
+from sys import exit as _exit, stderr
+from subprocess import run, CalledProcessError, PIPE
+from mesonbuild import mparser
+from mesonbuild.mesonlib import MesonException
+
+
+class DPDKBuildTask:
+ """A build task for DPDK"""
+
+ def __init__(self, label: str, description: str, param: str):
+ # label as it appears in build configuration
+ self.label = label
+ # description to be given in menu
+ self.description = description
+ # task-specific configuration parameters
+ self.param = param
+
+ def to_json_dict(self) -> Dict[str, Any]:
+ """Generate JSON dictionary for this task"""
+ return {
+ "label": f"Configure {self.label}",
+ "detail": self.description,
+ "type": "shell",
+ "dependsOn": "Remove builddir",
+ # take configuration from settings.json using config: namespace
+ "command": f"meson setup ${{config:BUILDCONFIG}} " \
+ f"{self.param} ${{config:BUILDDIR}}",
+ "problemMatcher": [],
+ "group": "build"
+ }
+
+
+class DPDKLaunchTask:
+ """A launch task for DPDK"""
+
+ def __init__(self, label: str, exe: str, gdb_path: str):
+ # label as it appears in launch configuration
+ self.label = label
+ # path to executable
+ self.exe = exe
+ self.gdb_path = gdb_path
+
+ def to_json_dict(self) -> Dict[str, Any]:
+ """Generate JSON dictionary for this task"""
+ return {
+ "name": f"Run {self.label}",
+ "type": "cppdbg",
+ "request": "launch",
+ # take configuration from settings.json using config: namespace
+ "program": f"${{config:BUILDDIR}}/{self.exe}",
+ "args": [],
+ "stopAtEntry": False,
+ "cwd": "${workspaceFolder}",
+ "externalConsole": False,
+ "preLaunchTask": "Build",
+ "MIMode": "gdb",
+ "miDebuggerPath": self.gdb_path,
+ "setupCommands": [
+ {
+ "description": "Enable pretty-printing for gdb",
+ "text": "-gdb-set print pretty on",
+ "ignoreFailures": True
+ }
+ ]
+ }
+
+
+class VSCodeConfig:
+ """Configuration for VSCode"""
+
+ def __init__(self, builddir: str, commoncfg: str):
+ # where will our build dir be located
+ self.builddir = builddir
+ # meson configuration common to all configs
+ self.commonconfig = commoncfg
+ # meson build configurations
+ self.build_tasks: List[DPDKBuildTask] = []
+ # meson launch configurations
+ self.launch_tasks: List[DPDKLaunchTask] = []
+
+ def settings_to_json_dict(self) -> Dict[str, Any]:
+ """Generate settings.json"""
+ return {
+ "BUILDDIR": self.builddir,
+ "BUILDCONFIG": self.commonconfig,
+ }
+
+ def tasks_to_json_dict(self) -> Dict[str, Any]:
+ """Generate tasks.json"""
+ # generate outer layer
+ build_tasks: Dict[str, Any] = {
+ "version": "2.0.0",
+ "tasks": []
+ }
+ # generate inner layer
+ tasks = build_tasks["tasks"]
+ # add common tasks
+ tasks.append({
+ "label": "Remove builddir",
+ "type": "shell",
+ "command": "rm -rf ${config:BUILDDIR}",
+ })
+ tasks.append({
+ "label": "Build",
+ "detail": "Run build command",
+ "type": "shell",
+ "command": "ninja",
+ "options": {
+ "cwd": "${config:BUILDDIR}"
+ },
+ "problemMatcher": {
+ "base": "$gcc",
+ "fileLocation": ["relative", "${config:BUILDDIR}"]
+ },
+ "group": "build"
+ })
+ # now, add generated tasks
+ tasks.extend([task.to_json_dict() for task in self.build_tasks])
+
+ # we're done
+ return build_tasks
+
+ def launch_to_json_dict(self) -> Dict[str, Any]:
+ """Generate launch.json"""
+ return {
+ "version": "0.2.0",
+ "configurations": [task.to_json_dict()
+ for task in self.launch_tasks]
+ }
+
+ def c_cpp_properties_to_json_dict(self) -> Dict[str, Any]:
+ """Generate c_cpp_properties.json"""
+ return {
+ "configurations": [
+ {
+ "name": "Linux",
+ "includePath": [
+ "${config:BUILDDIR}/",
+ "${workspaceFolder}/lib/eal/x86",
+ "${workspaceFolder}/lib/eal/linux",
+ "${workspaceFolder}/**"
+ ],
+ "compilerPath": "/usr/bin/gcc",
+ "cStandard": "c99",
+ "cppStandard": "c++17",
+ "intelliSenseMode": "${default}",
+ "compileCommands":
+ "${config:BUILDDIR}/compile_commands.json"
+ }
+ ],
+ "version": 4
+ }
+
+
+class CmdlineCtx:
+ """POD class to set up command line parameters"""
+
+ def __init__(self):
+ self.use_ui = False
+ self.use_gdbsudo = False
+ self.force_overwrite = False
+ self.build_dir = ""
+ self.dpdk_dir = ""
+ self.gdb_path = ""
+
+ self.avail_configs: List[Tuple[str, str, str]] = []
+ self.avail_apps: List[str] = []
+ self.avail_examples: List[str] = []
+ self.avail_drivers: List[str] = []
+
+ self.enabled_configs_str = ""
+ self.enabled_apps_str = ""
+ self.enabled_examples_str = ""
+ self.enabled_drivers_str = ""
+ self.enabled_configs: List[Tuple[str, str, str]] = []
+ self.enabled_apps: List[str] = []
+ self.enabled_examples: List[str] = []
+ self.enabled_drivers: List[str] = []
+
+ self.driver_dep_map: Dict[str, List[str]] = {}
+ self.common_conf = ""
+
+ # this is only used by TUI to decide which windows to show
+ self.show_apps = False
+ self.show_examples = False
+ self.show_drivers = False
+ self.show_configs = False
+ self.show_common_config = False
+
+
+def _whiptail_msgbox(message: str) -> None:
+ """Display a message box."""
+ args = ["whiptail", "--msgbox", message, "10", "70"]
+ run(args, check=True)
+
+
+def _whiptail_checklist(title: str, prompt: str,
+ options: List[Tuple[str, str]],
+ checked: List[str]) -> List[str]:
+ """Display a checklist and get user input."""
+ # build whiptail checklist
+ checklist = [
+ (label, desc, "on" if label in checked else "off")
+ for label, desc in options
+ ]
+ # flatten the list
+ flat = [item for sublist in checklist for item in sublist]
+ # build whiptail arguments
+ args = [
+ "whiptail", "--separate-output", "--checklist",
+ "--title", title, prompt, "15", "80", "8"
+ ] + flat
+
+ result = run(args, stderr=PIPE, check=True)
+ # capture selected options
+ return result.stderr.decode().strip().split()
+
+
+def _whiptail_inputbox(title: str, prompt: str, default: str = "") -> str:
+ """Display an input box and get user input."""
+ args = [
+ "whiptail", "--inputbox",
+ "--title", title,
+ prompt, "10", "70", default
+ ]
+ result = run(args, stderr=PIPE, check=True)
+ return result.stderr.decode().strip()
+
+
+def _get_enabled_configurations(configs: List[Tuple[str, str, str]],
+ checked: List[str]) \
+ -> List[Tuple[str, str, str]]:
+ """Ask user which build configurations they want."""
+ stop = False
+ while not stop:
+ opts = [
+ (c[0], c[1]) for c in configs
+ ]
+ # when interacting using UI, allow adding items
+ opts += [("add", "Add new option")]
+
+ # ask user to select options
+ checked = _whiptail_checklist(
+ "Build configurations", "Select build configurations to enable:",
+ opts, checked)
+
+ # if user selected "add", ask for custom meson configuration
+ if "add" in checked:
+ # remove "" from checked because it's a special option
+ checked.remove("add")
+ while True:
+ custom_label = _whiptail_inputbox(
+ "Configuration name",
+ "Enter custom meson configuration label:")
+ custom_description = _whiptail_inputbox(
+ "Configuration description",
+ "Enter custom meson configuration description:")
+ custom_mesonstr = _whiptail_inputbox(
+ "Configuration parameters",
+ "Enter custom meson configuration string:")
+ # do we have meaningful label?
+ if not custom_label:
+ _whiptail_msgbox("Configuration label cannot be empty!")
+ continue
+ # don't allow "add", don't allow duplicates
+ existing = [task[0] for task in configs] + ["add"]
+ if custom_label in existing:
+ _whiptail_msgbox(
+ f"Label '{custom_label}' is not allowed!")
+ continue
+ # we passed all checks, stop
+ break
+ new_task = (custom_label, custom_description, custom_mesonstr)
+ configs += [new_task]
+ # enable new configuration
+ checked += [custom_label]
+ else:
+ stop = True
+ # return our list of enabled configurations
+ return [
+ c for c in configs if c[0] in checked
+ ]
+
+
+def _select_from_list(title: str, prompt: str, items: List[str],
+ enabled: List[str]) -> List[str]:
+ """Display a list of items, optionally some enabled by default."""
+ opts = [
+ (item, "") for item in items
+ ]
+ # ask user to select options
+ return _whiptail_checklist(title, prompt, opts, enabled)
+
+
+def _extract_var(path: str, var: str) -> Any:
+ """Extract a variable from a meson.build file."""
+ try:
+ # we don't want to deal with multiline variable assignments
+ # so just read entire file in one go
+ with open(path, 'r', encoding='utf-8') as file:
+ content = file.read()
+ parser = mparser.Parser(content, path)
+ ast = parser.parse()
+
+ for node in ast.lines:
+ # we're only interested in variable assignments
+ if not isinstance(node, mparser.AssignmentNode):
+ continue
+ # we're only interested in the variable we're looking for
+ if node.var_name.value != var:
+ continue
+ # we're expecting string or array
+ if isinstance(node.value, mparser.StringNode):
+ return node.value.value
+ if isinstance(node.value, mparser.ArrayNode):
+ return [item.value for item in node.value.args.arguments]
+ except (MesonException, FileNotFoundError):
+ return None
+ return None
+
+
+def _pick_ui_options(ctx: CmdlineCtx) -> None:
+ """Use whiptail dialogs to decide which setup options to show."""
+ opts = [
+ ("config", "Select build configurations to enable"),
+ ("common", "Customize meson flags"),
+ ("apps", "Select apps to enable"),
+ ("examples", "Select examples to enable"),
+ ("drivers", "Select drivers to enable"),
+ ]
+ # whether any options are enabled depends on whether user has specified
+ # anything on the command-line, but also enable examples by default
+ checked_opts = ["examples"]
+ if ctx.enabled_configs_str:
+ checked_opts.append("config")
+ if ctx.enabled_apps_str:
+ checked_opts.append("apps")
+ if ctx.enabled_drivers_str:
+ checked_opts.append("drivers")
+ if ctx.common_conf:
+ checked_opts.append("common")
+
+ enabled = _whiptail_checklist(
+ "Options",
+ "Select options to configure (deselecting will pick defaults):",
+ opts, checked_opts)
+ for opt in enabled:
+ if opt == "config":
+ ctx.show_configs = True
+ elif opt == "common":
+ ctx.show_common_config = True
+ elif opt == "apps":
+ ctx.show_apps = True
+ elif opt == "examples":
+ ctx.show_examples = True
+ elif opt == "drivers":
+ ctx.show_drivers = True
+
+
+def _build_configs(ctx: CmdlineCtx) -> int:
+ """Build VSCode configuration files."""
+ # if builddir is a relative path, make it absolute
+ if not os.path.isabs(ctx.build_dir):
+ ctx.build_dir = os.path.realpath(ctx.build_dir)
+
+ # first, build our common meson param string
+ force_apps = False
+ force_drivers = False
+ common_param = ctx.common_conf
+
+ # if no apps are specified, all apps are built, so enable all of them. this
+ # isn't ideal because some of them might not be able to run because in
+ # actuality they don't get built due to missing dependencies. however, the
+ # alternative is to not generate any apps in launch configuration at all,
+ # which is worse than having some apps defined in config but not available.
+ if ctx.enabled_apps_str:
+ common_param += f" -Denable_apps={ctx.enabled_apps_str}"
+ else:
+ # special case: user might have specified -Dtests or apps flags in
+ # common param, so if the user did that, assume user knows what they're
+ # doing and don't display any warnings about enabling apps, and don't
+ # enable them in launch config and leave it up to the user to handle.
+ avoid_opts = ['-Dtests=', '-Denable_apps=', '-Ddisable_apps=']
+ if not any(opt in common_param for opt in avoid_opts):
+ force_apps = True
+ ctx.enabled_apps = ctx.avail_apps
+
+ # examples don't get build unless user asks
+ if ctx.enabled_examples_str:
+ common_param += f" -Dexamples={ctx.enabled_examples_str}"
+
+ # if no drivers enabled, let user know they will be built anyway
+ if ctx.enabled_drivers_str:
+ common_param += f" -Denable_drivers={ctx.enabled_drivers_str}"
+ else:
+ avoid_opts = ['-Denable_drivers=', '-Ddisable_drivers=']
+ if not any(opt in common_param for opt in avoid_opts):
+ # special case: user might have specified driver flags in common
+ # param, so if the user did that, assume user knows what they're
+ # doing and don't display any warnings about enabling drivers.
+ force_drivers = True
+
+ if force_drivers or force_apps:
+ ena: List[str] = []
+ dis: List[str] = []
+ if force_apps:
+ ena += ["apps"]
+ dis += ["-Ddisable_apps=*"]
+ if force_drivers:
+ ena += ["drivers"]
+ dis += ["-Ddisable_drivers=*/*"]
+ ena_str = " or ".join(ena)
+ dis_str = " or ".join(dis)
+ msg = f"""\
+No {ena_str} are specified in configuration, so all of them will be built. \
+To disable {ena_str}, add {dis_str} to common meson flags."""
+
+ _whiptail_msgbox(msg)
+
+ # create build tasks
+ build_tasks = [DPDKBuildTask(n, d, p) for n, d, p in ctx.enabled_configs]
+
+ # create launch tasks
+ launch_tasks: List[DPDKLaunchTask] = []
+ for app in ctx.enabled_apps:
+ label = app
+ exe = os.path.join("app", f"dpdk-{app}")
+ launch_tasks.append(DPDKLaunchTask(label, exe, ctx.gdb_path))
+ for app in ctx.enabled_examples:
+ # examples may have complex paths but they always flatten
+ label = os.path.basename(app)
+ exe = os.path.join("examples", f"dpdk-{label}")
+ launch_tasks.append(DPDKLaunchTask(label, exe, ctx.gdb_path))
+
+ # build our config
+ vscode_cfg = VSCodeConfig(ctx.build_dir, common_param)
+ vscode_cfg.build_tasks = build_tasks
+ vscode_cfg.launch_tasks = launch_tasks
+
+ # we're done! now, create .vscode directory
+ config_root = os.path.join(ctx.dpdk_dir, ".vscode")
+ os.makedirs(config_root, exist_ok=True)
+
+ # ...and create VSCode configuration
+ print("Creating VSCode configuration files...")
+ func_map = {
+ "settings.json": vscode_cfg.settings_to_json_dict,
+ "tasks.json": vscode_cfg.tasks_to_json_dict,
+ "launch.json": vscode_cfg.launch_to_json_dict,
+ "c_cpp_properties.json": vscode_cfg.c_cpp_properties_to_json_dict
+ }
+ # check if any of the files exist, and refuse to overwrite them unless
+ # --force was specified on the command line
+ for filename in func_map.keys():
+ fpath = os.path.join(config_root, filename)
+ if os.path.exists(fpath) and not ctx.force_overwrite:
+ print(f"Error: {filename} already exists! \
+ Use --force to overwrite.", file=stderr)
+ return 1
+ for filename, func in func_map.items():
+ with open(os.path.join(config_root, filename),
+ "w", encoding="utf-8") as f:
+ print(f"Writing {filename}...")
+ f.write(json.dumps(func(), indent=4))
+ print("Done!")
+ return 0
+
+
+def _resolve_deps(ctx: CmdlineCtx) -> None:
+ """Resolve driver dependencies."""
+ # resolving dependencies is not straightforward, because DPDK build system
+ # treats wildcards differently from explicitly requested drivers: namely,
+ # it will treat wildcard-matched drivers on a best-effort basis, and will
+ # skip them if driver's dependencies aren't met without error. contrary to
+ # that, when a driver is explicitly requested, it will cause an error if
+ # any of its dependencies are unmet.
+ #
+ # to resolve this, we need to be smarter about how we add dependencies.
+ # specifically, when we're dealing with wildcards, we will need to add
+ # wildcard dependencies, whereas when we're dealing with explicitly
+ # requested drivers, we will add explicit dependencies. for example,
+ # requesting net/ice will add common/iavf, but requesting net/*ce will
+ # add common/* as a dependency. We will build more that we would've
+ # otherwise, but that's an acceptable compromise to enable as many drivers
+ # as we can while avoiding build errors due to erroneous wildcard matches.
+ new_deps: List[str] = []
+ for driver in ctx.enabled_drivers_str.split(","):
+ # is this a wildcard?
+ if "*" in driver:
+ # find all drivers matching this wildcard, figure out which
+ # category (bus, common, etc.) of driver they request as
+ # dependency, and add a wildcarded match on that category
+ wc_matches = fnmatch.filter(ctx.avail_drivers, driver)
+ # find all of their dependencies
+ deps = [d
+ for dl in wc_matches
+ for d in ctx.driver_dep_map.get(dl, [])]
+ categories: List[str] = []
+ for d in deps:
+ category, _ = d.split("/")
+ categories += [category]
+ # find all categories we've added
+ categories = sorted(set(categories))
+ # add them as dependencies
+ new_deps += [f"{cat}/*" for cat in categories]
+ continue
+ # this is a driver, so add its dependencies explicitly
+ new_deps += ctx.driver_dep_map.get(driver, [])
+
+ # add them to enabled_drivers_str, this will be resolved later
+ if new_deps:
+ # this might add some dupes but we don't really care
+ ctx.enabled_drivers_str += f',{",".join(new_deps)}'
+
+
+def _update_ctx_from_ui(ctx: CmdlineCtx) -> int:
+ """Use whiptail dialogs to update context contents."""
+ try:
+ # update build dir
+ ctx.build_dir = _whiptail_inputbox(
+ "Build directory", "Enter build directory:", ctx.build_dir)
+
+ # first, decide what we are going to set up
+ _pick_ui_options(ctx)
+
+ # update configs
+ if ctx.show_configs:
+ ctx.enabled_configs = _get_enabled_configurations(
+ ctx.avail_configs, [c[0] for c in ctx.enabled_configs])
+
+ # update common config
+ if ctx.show_common_config:
+ ctx.common_conf = _whiptail_inputbox(
+ "Meson configuration",
+ "Enter common meson configuration flags (if any):",
+ ctx.common_conf)
+
+ # when user interaction is requestted, we cannot really keep any values
+ # we got from arguments, because if user has changed something in those
+ # checklists, any wildcards will become invalid. however, we can do a
+ # heuristic: if user didn't *change* anything, we can infer that
+ # they're happy with the configuration they have picked, so we will
+ # only update meson param strings if the user has changed the
+ # configuration from TUI, or if we didn't have any to begin with
+
+ old_enabled_apps = ctx.enabled_apps.copy()
+ old_enabled_examples = ctx.enabled_examples.copy()
+ old_enabled_drivers = ctx.enabled_drivers.copy()
+ if ctx.show_apps:
+ ctx.enabled_apps = _select_from_list(
+ "Apps", "Select apps to enable:",
+ ctx.avail_apps, ctx.enabled_apps)
+ if ctx.show_examples:
+ ctx.enabled_examples = _select_from_list(
+ "Examples", "Select examples to enable:",
+ ctx.avail_examples, ctx.enabled_examples)
+ if ctx.show_drivers:
+ ctx.enabled_drivers = _select_from_list(
+ "Drivers", "Select drivers to enable:",
+ ctx.avail_drivers, ctx.enabled_drivers)
+
+ # did we change anything, assuming we even had anything at all?
+ if not ctx.enabled_apps_str or \
+ set(old_enabled_apps) != set(ctx.enabled_apps):
+ ctx.enabled_apps_str = ",".join(ctx.enabled_apps)
+ if not ctx.enabled_examples_str or \
+ set(old_enabled_examples) != set(ctx.enabled_examples):
+ ctx.enabled_examples_str = ",".join(ctx.enabled_examples)
+ if not ctx.enabled_drivers_str or \
+ set(old_enabled_drivers) != set(ctx.enabled_drivers):
+ ctx.enabled_drivers_str = ",".join(ctx.enabled_drivers)
+
+ return 0
+ except CalledProcessError:
+ # use probably pressed cancel, so bail out
+ return 1
+
+
+def _resolve_ctx(ctx: CmdlineCtx) -> int:
+ """Map command-line enabled options to available options."""
+ # for each enabled app, see if it's a wildcard and if so, do a wildcard
+ # match
+ for app in ctx.enabled_apps_str.split(","):
+ if "*" in app:
+ ctx.enabled_apps.extend(fnmatch.filter(ctx.avail_apps, app))
+ elif app in ctx.avail_apps:
+ ctx.enabled_apps.append(app)
+ elif app:
+ print(f"Error: Unknown app: {app}", file=stderr)
+ return 1
+
+ # do the same with examples
+ for example in ctx.enabled_examples_str.split(","):
+ if "*" in example:
+ ctx.enabled_examples.extend(
+ fnmatch.filter(ctx.avail_examples, example))
+ elif example in ctx.avail_examples:
+ ctx.enabled_examples.append(example)
+ elif example:
+ print(f"Error: Unknown example: {example}", file=stderr)
+ return 1
+
+ # do the same with drivers
+ for driver in ctx.enabled_drivers_str.split(","):
+ if "*" in driver:
+ ctx.enabled_drivers.extend(
+ fnmatch.filter(ctx.avail_drivers, driver))
+ elif driver in ctx.avail_drivers:
+ ctx.enabled_drivers.append(driver)
+ elif driver:
+ print(f"Error: Unknown driver: {driver}", file=stderr)
+ return 1
+
+ # due to wildcard, there may be dupes, so sort(set()) everything
+ ctx.enabled_apps = sorted(set(ctx.enabled_apps))
+ ctx.enabled_examples = sorted(set(ctx.enabled_examples))
+ ctx.enabled_drivers = sorted(set(ctx.enabled_drivers))
+
+ return 0
+
+
+def _discover_ctx(ctx: CmdlineCtx) -> int:
+ """Discover available apps/drivers etc. from DPDK."""
+ # find out where DPDK root is located
+ _self = os.path.realpath(__file__)
+ dpdk_root = os.path.realpath(os.path.join(os.path.dirname(_self), ".."))
+ ctx.dpdk_dir = dpdk_root
+
+ # find gdb path
+ if ctx.use_gdbsudo:
+ gdb = "gdbsudo"
+ else:
+ gdb = "gdb"
+ ctx.gdb_path = shutil.which(gdb)
+ if not ctx.gdb_path:
+ print(f"Error: Cannot find {gdb} in PATH!", file=stderr)
+ return 1
+
+ # we want to extract information from DPDK build files, but we don't have a
+ # good way of doing it without already having a meson build directory. for
+ # some things we can use meson AST parsing to extract this information, but
+ # for drivers extracting this information is not straightforward because
+ # they have complex build-time logic to determine which drivers need to be
+ # built (e.g. qat). so, we'll use meson AST for apps and examples, but for
+ # drivers we'll do it the old-fashioned way: by globbing directories.
+
+ apps: List[str] = []
+ examples: List[str] = []
+ drivers: List[str] = []
+
+ app_root = os.path.join(dpdk_root, "app")
+ examples_root = os.path.join(dpdk_root, "examples")
+ drivers_root = os.path.join(dpdk_root, "drivers")
+
+ apps = _extract_var(os.path.join(app_root, "meson.build"), "apps")
+ # special case for apps: test isn't added by default
+ apps.append("test")
+ # some apps will have overridden names using 'name' variable, extract it
+ for i, app in enumerate(apps[:]):
+ new_name = _extract_var(os.path.join(
+ app_root, app, "meson.build"), "name")
+ if new_name:
+ apps[i] = new_name
+
+ # examples don't have any special cases
+ examples = _extract_var(os.path.join(
+ examples_root, "meson.build"), "all_examples")
+
+ for root, _, _ in os.walk(drivers_root):
+ # some directories are drivers, while some are there simply to
+ # organize source in a certain way (e.g. base drivers), so we're
+ # going to cheat a little and only consider directories that have
+ # exactly two levels (e.g. net/ixgbe) and no others.
+ if root == drivers_root:
+ continue
+ rel_root = os.path.relpath(root, drivers_root)
+ if len(rel_root.split(os.sep)) != 2:
+ continue
+ category = os.path.dirname(rel_root)
+ # see if there's a name override
+ name = os.path.basename(rel_root)
+ new_name = _extract_var(os.path.join(root, "meson.build"), "name")
+ if new_name:
+ name = new_name
+ driver_name = os.path.join(category, name)
+ drivers.append(driver_name)
+
+ # some drivers depend on other drivers, so parse these dependencies
+ # using the "deps" variable
+ deps: Any = _extract_var(
+ os.path.join(root, "meson.build"), "deps")
+ if not deps:
+ continue
+ # occasionally, deps will be a string, so convert it to a list
+ if isinstance(deps, str):
+ deps = [deps]
+ for dep in deps:
+ # by convention, drivers are named as <category>_<name>, so we can
+ # infer that dependency is a driver if it has an underscore
+ if "_" not in dep:
+ continue
+ dep_driver = dep.replace("_", "/", 1)
+ ctx.driver_dep_map.setdefault(driver_name, []).append(dep_driver)
+
+ # sort all lists alphabetically
+ apps.sort()
+ examples.sort()
+ drivers.sort()
+
+ # save all of this information into our context
+ ctx.avail_apps = apps
+ ctx.avail_examples = examples
+ ctx.avail_drivers = drivers
+
+ return 0
+
+
+def _main() -> int:
+ """Parse command line arguments and direct program flow."""
+ # this is primarily a TUI script, but we also want to be able to automate
+ # everything, or set defaults to enhance user interaction and
+ # customization.
+
+ # valid parameters:
+ # --no-ui: run without any user interaction
+ # --no-gdbsudo: set up launch targets to use gdb directly
+ # --gdbsudo: set up launch targets to use gdbsudo
+ # --no-defaults: do not enable built-in build configurations
+ # --help: show help message
+ # -B/--build-dir: set build directory
+ # -b/--build-config: set default build configurations
+ # format: <label>,<description>,<meson-param>
+ # can be specified multiple times
+ # -c/--common-conf: additional configuration common to all build tasks
+ # -a/--apps: comma-separated list of enabled apps
+ # -e/--examples: comma-separated list of enabled examples
+ # -d/--drivers: comma-separated list of enabled drivers
+ # -f/--force: overwrite existing configuration
+ ap = argparse.ArgumentParser(
+ description="Generate VSCode configuration for DPDK")
+ ap.add_argument("--no-ui", action="store_true",
+ help="Run without any user interaction")
+ gdbgrp = ap.add_mutually_exclusive_group()
+ gdbgrp.add_argument("--no-gdbsudo", action="store_true",
+ help="Set up launch targets to use gdb directly")
+ gdbgrp.add_argument("--gdbsudo", action="store_true",
+ help="Set up launch targets to use gdbsudo")
+ ap.add_argument("--no-defaults", action="store_true",
+ help="Do not enable built-in build configurations")
+ ap.add_argument("-B", "--build-dir", default="build",
+ help="Set build directory")
+ ap.add_argument("-b", "--build-config", action="append", default=[],
+ help="Comma-separated build task configuration of format \
+ [label,description,meson setup arguments]")
+ ap.add_argument("-c", "--common-conf",
+ help="Additional configuration common to all build tasks",
+ default="")
+ ap.add_argument("-a", "--apps", default="",
+ help="Comma-separated list of enabled apps \
+ (wildcards accepted)")
+ ap.add_argument("-e", "--examples", default="",
+ help="Comma-separated list of enabled examples \
+ (wildcards accepted)")
+ ap.add_argument("-d", "--drivers", default="",
+ help="Comma-separated list of enabled drivers \
+ (wildcards accepted)")
+ ap.add_argument("-f", "--force", action="store_true",
+ help="Overwrite existing configuration")
+ ap.epilog = """\
+When script is run in interactive mode, parameters will be \
+used to set up dialog defaults. Otherwise, they will be used \
+to create configuration directly."""
+ args = ap.parse_args()
+
+ def_configs = [
+ ("debug", "Debug build", "--buildtype=debug"),
+ ("debugopt", "Debug build with optimizations",
+ "--buildtype=debugoptimized"),
+ ("release", "Release build with documentation",
+ "--buildtype=release -Denable_docs=true"),
+ ("asan", "Address Sanitizer build",
+ "--buildtype=debugoptimized -Db_sanitize=address -Db_lundef=false"),
+ ]
+ # parse build configs
+ arg_configs: List[Tuple[str, str, str]] = []
+ for c in args.build_config:
+ parts: List[str] = c.split(",")
+ if len(parts) != 3:
+ print(
+ f"Error: Invalid build configuration format: {c}", file=stderr)
+ return 1
+ arg_configs.append(tuple(parts))
+
+ # set up command line context. all wildcards will be passed directly to
+ # _main, and will be resolved later, when we have a list of things to
+ # enable/disable.
+ ctx = CmdlineCtx()
+ ctx.use_ui = not args.no_ui
+ ctx.force_overwrite = args.force
+ ctx.build_dir = args.build_dir
+ ctx.common_conf = args.common_conf
+ ctx.enabled_configs_str = args.build_config
+ ctx.enabled_apps_str = args.apps
+ ctx.enabled_examples_str = args.examples
+ ctx.enabled_drivers_str = args.drivers
+ ctx.enabled_configs = arg_configs
+ ctx.avail_configs = def_configs + ctx.enabled_configs
+
+ # if user has specified gdbsudo argument, use that
+ if args.gdbsudo or args.no_gdbsudo:
+ ctx.use_gdbsudo = args.gdbsudo or not args.no_gdbsudo
+ else:
+ # use gdb if we're root
+ ctx.use_gdbsudo = os.geteuid() != 0
+ print(f"Autodetected gdbsudo usage: {ctx.use_gdbsudo}")
+
+ if not args.no_defaults:
+ # enable default configs
+ ctx.enabled_configs.extend(def_configs)
+
+ # if UI interaction is requested, check if whiptail is installed
+ if ctx.use_ui and os.system("which whiptail &> /dev/null") != 0:
+ print("whiptail is not installed! Please install it and try again.",
+ file=stderr)
+ return 1
+
+ # check if gdbsudo is available
+ if ctx.use_gdbsudo and os.system("which gdbsudo &> /dev/null") != 0:
+ print("Generated configuration will use \
+ gdbsudo script to run applications.", file=stderr)
+ print("If you want to use gdb directly, \
+ please run with --no-gdbsudo argument.", file=stderr)
+ print("Otherwise, run the following snippet \
+ in your terminal and try again:", file=stderr)
+ print("""\
+sudo tee <<EOF /usr/local/bin/gdbsudo &> /dev/null
+#!/usr/bin/bash
+sudo gdb $@
+EOF
+sudo chmod a+x /usr/local/bin/gdbsudo
+""", file=stderr)
+ return 1
+
+ if _discover_ctx(ctx):
+ return 1
+ if _resolve_ctx(ctx):
+ return 1
+ if ctx.use_ui and _update_ctx_from_ui(ctx):
+ return 1
+ _resolve_deps(ctx)
+ # resolve again because we might have added some dependencies
+ if _resolve_ctx(ctx):
+ return 1
+ return _build_configs(ctx)
+
+
+if __name__ == "__main__":
+ _exit(_main())