[v4,5/6] doc/guides: rebuild with meson whenever a file changes
Checks
Commit Message
Add proper support for calling sphinx whenever a file in the doc
directory changes. This is accomplished by using a wrapper script
for sphinx, which runs sphinx but also emits a gcc-format dependency
file listing all the doc files. This is used by ninja so that any
change to the doc files triggers a rebuild of the docs.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Aaron Conole <aconole@redhat.com>
Acked-by: Luca Boccassi <bluca@debian.org>
---
MAINTAINERS | 1 +
buildtools/call-sphinx-build.py | 31 +++++++++++++++++++++++++++++++
buildtools/meson.build | 6 ++++--
doc/guides/meson.build | 22 ++++++++--------------
4 files changed, 44 insertions(+), 16 deletions(-)
create mode 100755 buildtools/call-sphinx-build.py
Comments
On Fri, Jan 10, 2020 at 10:52 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> Add proper support for calling sphinx whenever a file in the doc
> directory changes. This is accomplished by using a wrapper script
> for sphinx, which runs sphinx but also emits a gcc-format dependency
> file listing all the doc files. This is used by ninja so that any
> change to the doc files triggers a rebuild of the docs.
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> Acked-by: Aaron Conole <aconole@redhat.com>
> Acked-by: Luca Boccassi <bluca@debian.org>
> ---
> MAINTAINERS | 1 +
> buildtools/call-sphinx-build.py | 31 +++++++++++++++++++++++++++++++
> buildtools/meson.build | 6 ++++--
> doc/guides/meson.build | 22 ++++++++--------------
> 4 files changed, 44 insertions(+), 16 deletions(-)
> create mode 100755 buildtools/call-sphinx-build.py
Caught an issue on fc30:
[2094/2338] Generating html_guides with a custom command.
FAILED: doc/guides/html
/usr/bin/python3 ../../dpdk/buildtools/call-sphinx-build.py
/usr/libexec/python2-sphinx/sphinx-build
/home/dmarchan/dpdk/doc/guides
/home/dmarchan/builds/build-x86-default/doc/guides
sphinx-build 1.8.4
Traceback (most recent call last):
File "../../dpdk/buildtools/call-sphinx-build.py", line 15, in <module>
ver = run([sphinx, '--version'], stdout=PIPE).stdout.decode().split()[-1]
IndexError: list index out of range
Double checked, the version went to stderr:
$ sphinx-build --version >/dev/null
sphinx-build 1.8.4
$ sphinx-build --version 2>/dev/null
On Tue, Jan 14, 2020 at 04:59:24PM +0100, David Marchand wrote:
> On Fri, Jan 10, 2020 at 10:52 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > Add proper support for calling sphinx whenever a file in the doc
> > directory changes. This is accomplished by using a wrapper script
> > for sphinx, which runs sphinx but also emits a gcc-format dependency
> > file listing all the doc files. This is used by ninja so that any
> > change to the doc files triggers a rebuild of the docs.
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > Acked-by: Aaron Conole <aconole@redhat.com>
> > Acked-by: Luca Boccassi <bluca@debian.org>
> > ---
> > MAINTAINERS | 1 +
> > buildtools/call-sphinx-build.py | 31 +++++++++++++++++++++++++++++++
> > buildtools/meson.build | 6 ++++--
> > doc/guides/meson.build | 22 ++++++++--------------
> > 4 files changed, 44 insertions(+), 16 deletions(-)
> > create mode 100755 buildtools/call-sphinx-build.py
>
> Caught an issue on fc30:
>
> [2094/2338] Generating html_guides with a custom command.
> FAILED: doc/guides/html
> /usr/bin/python3 ../../dpdk/buildtools/call-sphinx-build.py
> /usr/libexec/python2-sphinx/sphinx-build
> /home/dmarchan/dpdk/doc/guides
> /home/dmarchan/builds/build-x86-default/doc/guides
> sphinx-build 1.8.4
> Traceback (most recent call last):
> File "../../dpdk/buildtools/call-sphinx-build.py", line 15, in <module>
> ver = run([sphinx, '--version'], stdout=PIPE).stdout.decode().split()[-1]
> IndexError: list index out of range
>
> Double checked, the version went to stderr:
>
> $ sphinx-build --version >/dev/null
> sphinx-build 1.8.4
> $ sphinx-build --version 2>/dev/null
>
That is strange. Didn't hit any issues on fedora 31 which I am using here.
I'll see if I can make it work with both stderr and stdout reporting.
/Bruce
> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Tuesday, January 14, 2020 3:59 PM
> To: Richardson, Bruce <bruce.richardson@intel.com>
> Cc: Mcnamara, John <john.mcnamara@intel.com>; Luca Boccassi
> <bluca@debian.org>; dev <dev@dpdk.org>; Aaron Conole <aconole@redhat.com>;
> Thomas Monjalon <thomas@monjalon.net>
> Subject: Re: [PATCH v4 5/6] doc/guides: rebuild with meson whenever a file
> changes
>
> On Fri, Jan 10, 2020 at 10:52 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > Add proper support for calling sphinx whenever a file in the doc
> > directory changes. This is accomplished by using a wrapper script for
> > sphinx, which runs sphinx but also emits a gcc-format dependency file
> > listing all the doc files. This is used by ninja so that any change to
> > the doc files triggers a rebuild of the docs.
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > Acked-by: Aaron Conole <aconole@redhat.com>
> > Acked-by: Luca Boccassi <bluca@debian.org>
> > ---
> > MAINTAINERS | 1 +
> > buildtools/call-sphinx-build.py | 31 +++++++++++++++++++++++++++++++
> > buildtools/meson.build | 6 ++++--
> > doc/guides/meson.build | 22 ++++++++--------------
> > 4 files changed, 44 insertions(+), 16 deletions(-) create mode
> > 100755 buildtools/call-sphinx-build.py
>
> Caught an issue on fc30:
>
> [2094/2338] Generating html_guides with a custom command.
> FAILED: doc/guides/html
> /usr/bin/python3 ../../dpdk/buildtools/call-sphinx-build.py
> /usr/libexec/python2-sphinx/sphinx-build
> /home/dmarchan/dpdk/doc/guides
> /home/dmarchan/builds/build-x86-default/doc/guides
> sphinx-build 1.8.4
> Traceback (most recent call last):
> File "../../dpdk/buildtools/call-sphinx-build.py", line 15, in <module>
> ver = run([sphinx, '--version'],
> stdout=PIPE).stdout.decode().split()[-1]
> IndexError: list index out of range
>
> Double checked, the version went to stderr:
>
> $ sphinx-build --version >/dev/null
> sphinx-build 1.8.4
> $ sphinx-build --version 2>/dev/null
>
The series http://patches.dpdk.org/project/dpdk/list/?series=8190 should fix
this issue and also adds support for the "werror" option for building the guides.
/Bruce
@@ -128,6 +128,7 @@ F: meson.build
F: lib/librte_eal/freebsd/BSDmakefile.meson
F: meson_options.txt
F: config/rte_config.h
+F: buildtools/call-sphinx-build.py
F: buildtools/gen-pmdinfo-cfile.sh
F: buildtools/map_to_def.py
F: buildtools/symlink-drivers-solibs.sh
new file mode 100755
@@ -0,0 +1,31 @@
+#! /usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2019 Intel Corporation
+#
+
+import sys
+import os
+from os.path import join
+from subprocess import run, PIPE
+from distutils.version import StrictVersion
+
+(sphinx, src, dst) = sys.argv[1:] # assign parameters to variables
+
+# for sphinx version >= 1.7 add parallelism using "-j auto"
+ver = run([sphinx, '--version'], stdout=PIPE).stdout.decode().split()[-1]
+sphinx_cmd = [sphinx]
+if StrictVersion(ver) >= StrictVersion('1.7'):
+ sphinx_cmd += ['-j', 'auto']
+
+# find all the files sphinx will process so we can write them as dependencies
+srcfiles = []
+for root, dirs, files in os.walk(src):
+ srcfiles.extend([join(root, f) for f in files])
+
+# run sphinx, putting the html output in a "html" directory
+process = run(sphinx_cmd + ['-b', 'html', src, join(dst, 'html')], check=True)
+print(str(process.args) + ' Done OK')
+
+# create a gcc format .d file giving all the dependencies of this doc build
+with open(join(dst, '.html.d'), 'w') as d:
+ d.write('html: ' + ' '.join(srcfiles) + '\n')
@@ -10,10 +10,12 @@ check_experimental_syms = find_program('check-experimental-syms.sh')
# set up map-to-def script using python, either built-in or external
python3 = import('python').find_installation(required: false)
if python3.found()
- map_to_def_cmd = [python3, files('map_to_def.py')]
+ py3 = [python3]
else
- map_to_def_cmd = ['meson', 'runpython', files('map_to_def.py')]
+ py3 = ['meson', 'runpython']
endif
+map_to_def_cmd = py3 + files('map_to_def.py')
+sphinx_wrapper = py3 + files('call-sphinx-build.py')
# stable ABI always starts with "DPDK_"
is_experimental_cmd = [find_program('grep', 'findstr'), '^DPDK_']
@@ -7,24 +7,18 @@ if not sphinx.found()
subdir_done()
endif
-htmldir = join_paths('share', 'doc', 'dpdk')
+htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk')
html_guides = custom_target('html_guides',
- input: meson.current_source_dir(),
- output: 'guides',
- command: [sphinx, '-b', 'html',
- '-d', meson.current_build_dir() + '/.doctrees',
- '@INPUT@', meson.current_build_dir() + '/guides'],
+ input: files('index.rst'),
+ output: 'html',
+ command: [sphinx_wrapper, sphinx, meson.current_source_dir(), meson.current_build_dir()],
+ depfile: '.html.d',
build_by_default: get_option('enable_docs'),
install: get_option('enable_docs'),
install_dir: htmldir)
+install_data(files('custom.css'),
+ install_dir: join_paths(htmldir,'_static', 'css'))
+
doc_targets += html_guides
doc_target_names += 'HTML_Guides'
-
-# sphinx leaves a .buildinfo in the target directory, which we don't
-# want to install. Note that sh -c has to be used, otherwise the
-# env var does not get expanded if calling rm/install directly.
-meson.add_install_script('sh', '-c',
- 'rm -f $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/.buildinfo')
-meson.add_install_script('sh', '-c',
- 'install -D -m0644 $MESON_SOURCE_ROOT/doc/guides/custom.css $MESON_INSTALL_DESTDIR_PREFIX/share/doc/dpdk/guides/_static/css/custom.css')