Re: [PATCH 0/3] Translate sphinx-pre-install to Python

[Mauro Carvalho Chehab <mchehab+huawei@kernel.org>]

Em Thu, 26 Jun 2025 10:06:28 +0200
Mauro Carvalho Chehab <mchehab+huawei@kernel.org> escreveu:

> As part of my efforts to make the doc build system more modern, it
> follows a patch series converting sphinx-pre-install to python.
> 
> The first patch contains a small fix to the Perl version. It is
> interesting to have, as otherwise regression tests fail.
> 
> The conversion itself was not hard. Most of the time I spent writing
> scripts to test it.

Hi Jon,

Please see this as a RFC. There is a trivial bug here when sphinx-build
is found (m1 and m2 variables were renamed to match). Easy to fix,
but I'm doing other cleanups, and I'm planning to improve the regression
test coverage.

I'll be sending you a new version next week.

> 
> The new version is meant to be bug-compatible with the previous
> version, with some exceptions:
> 
> - I dropped support for too old Fedora distros. They're OOL;
> - as it runs on Python, the tests for python could be bogus. I 
>   added some FIXME entries to remember about that.;
> - The package lists are now alphabetically sorted, whith alters
>    the warning/eror order.
> - It supports additional variants that I could install via lxc download
>    template.
> - I didn't test a couple of distro variants (OpenMandriva, RHEL,
>   and Scientific Linux.
> 
> I did regression tests. Those are the test results summary:
> 
> +-----------------+---------+-------+-------------+----------------------------------------+
> | distro          | status  | stage | return code | log file                               |
> +-----------------+---------+-------+-------------+----------------------------------------+
> | almalinux       | N/A     | Tests | 0           | ./test_logs/almalinux_report.log       |
> | amazonlinux     | N/A     | Tests | 0           | ./test_logs/amazonlinux_report.log     |
> | archlinux       | SUCCESS | Tests | 0           | ./test_logs/archlinux_report.log       |
> | centos          | SUCCESS | Tests | 0           | ./test_logs/centos_report.log          |
> | debian          | SUCCESS | Tests | 0           | ./test_logs/debian_report.log          |
> | devuan          | N/A     | Tests | 0           | ./test_logs/devuan_report.log          |
> | fedora          | SUCCESS | Tests | 0           | ./test_logs/fedora_report.log          |
> | gentoo          | FAILED  | Tests | 3           | ./test_logs/gentoo_report.log          |
> | kali            | N/A     | Tests | 0           | ./test_logs/kali_report.log            |
> | mageia          | SUCCESS | Tests | 0           | ./test_logs/mageia_report.log          |
> | mint            | N/A     | Tests | 0           | ./test_logs/mint_report.log            |
> | openeuler       | N/A     | Tests | 0           | ./test_logs/openeuler_report.log       |
> | opensuse        | SUCCESS | Tests | 0           | ./test_logs/opensuse_report.log        |
> | oracle          | SUCCESS | Tests | 0           | ./test_logs/oracle_report.log          |
> | rockylinux      | N/A     | Tests | 0           | ./test_logs/rockylinux_report.log      |
> | springdalelinux | N/A     | Tests | 0           | ./test_logs/springdalelinux_report.log |
> | ubuntu          | SUCCESS | Tests | 0           | ./test_logs/ubuntu_report.log          |
> | ubuntu-lts      | SUCCESS | Tests | 0           | ./test_logs/ubuntu-lts_report.log      |
> +-----------------+---------+-------+-------------+----------------------------------------+
> 
> N/A means that the version were not supported originally, so it can't be
> properly compared.
> 
> The only one marked as failed is gentoo. This is actually a false positive.
> That's the test output:
> 
> TEST REPORT: GENTOO
> 
> Generated: 2025-06-26 10:02:57
> Stage: Tests
> 
> ERROR DETAILS:
> 
> CustomError: 6 of 9 validate commands succeeded
>   File "/home/mchehab/bin/container_tests.py", line 1779, in <module>
>     asyncio.run(main())
>   File "/usr/lib64/python3.13/asyncio/runners.py", line 195, in run
>     return runner.run(main)
>   File "/usr/lib64/python3.13/asyncio/runners.py", line 118, in run
>     return self._loop.run_until_complete(task)
>   File "/usr/lib64/python3.13/asyncio/base_events.py", line 706, in run_until_complete
>     self.run_forever()
>   File "/usr/lib64/python3.13/asyncio/base_events.py", line 677, in run_forever
>     self._run_once()
>   File "/usr/lib64/python3.13/asyncio/base_events.py", line 2034, in _run_once
>     handle._run()
>   File "/usr/lib64/python3.13/asyncio/events.py", line 89, in _run
>     self._context.run(self._callback, *self._args)
>   File "/home/mchehab/bin/container_tests.py", line 1773, in main
>     await manager.run()
>   File "/home/mchehab/bin/container_tests.py", line 1625, in run
>     await self.run_distro(distro)
>   File "/home/mchehab/bin/container_tests.py", line 1605, in run_distro
>     await self.run_test(distro)
>   File "/home/mchehab/bin/container_tests.py", line 1434, in run_test
>     self.record_status(
> 
> 
> TEST RESULTS:
> 
> Status: FAILED
> Return Code: 3
> 
> STANDARD OUTPUT:
> 
> $ cd /root && LC_ALL=C ./scripts/sphinx-pre-install.py
> -------------------------------------------------------
> 
> Detected OS: Gentoo Base System release 2.17.
> Warning: better to also install "convert".
> Warning: better to also install "dot".
> Warning: better to also install "latexmk".
> Warning: better to also install "media-fonts/dejavu".
> Warning: better to also install "media-fonts/noto-cjk".
> Warning: better to also install "rsvg-convert".
> Warning: better to also install "xelatex".
> ERROR: please install "yaml", otherwise, build won't work.
> You should run:
> 
> 
> 	sudo su -c 'echo "media-gfx/imagemagick svg png" > /etc/portage/package.use/imagemagick'
> 	sudo su -c 'echo "media-gfx/graphviz cairo pdf" > /etc/portage/package.use/graphviz'
> 	sudo emerge --ask media-gfx/imagemagick media-gfx/graphviz latexmk media-fonts/dejavu media-fonts/noto-cjk gnome-base/librsvg dev-texlive/texlive-xetex media-fonts/dejavu dev-python/pyyaml
> 
> Sphinx needs to be installed either:
> 1) via pip/pypi with:
> 
> 	/usr/sbin/python3 -m venv sphinx_latest
> 	. sphinx_latest/bin/activate
> 	pip install -r ./Documentation/sphinx/requirements.txt
> 
>     If you want to exit the virtualenv, you can use:
> 	deactivate
> 
> 2) As a package with:
> 
> 
> 	sudo su -c 'echo "media-gfx/imagemagick svg png" > /etc/portage/package.use/imagemagick'
> 	sudo su -c 'echo "media-gfx/graphviz cairo pdf" > /etc/portage/package.use/graphviz'
> 	sudo emerge --ask python-sphinx
> 
>     Please note that Sphinx >= 3.0 will currently produce false-positive
>    warning when the same name is used for more than one type (functions,
>    structs, enums,...). This is known Sphinx bug. For more details, see:
> 	https://github.com/sphinx-doc/sphinx/pull/8313
> Can't build as 2 mandatory dependencies are missing
> 
> $ cd /root && LC_ALL=C ./scripts/sphinx-pre-install
> ----------------------------------------------------
> 
> Detected OS: Gentoo Base System release 2.17.
> Warning: better to also install "convert".
> Warning: better to also install "dot".
> Warning: better to also install "latexmk".
> Warning: better to also install "media-fonts/dejavu".
> Warning: better to also install "media-fonts/noto-cjk".
> Warning: better to also install "rsvg-convert".
> Warning: better to also install "xelatex".
> ERROR: please install "yaml", otherwise, build won't work.
> You should run:
> 
> 	sudo su -c 'echo "media-gfx/imagemagick svg png" > /etc/portage/package.use/imagemagick'
> 	sudo su -c 'echo "media-gfx/graphviz cairo pdf" > /etc/portage/package.use/graphviz'
> 	sudo emerge --ask media-gfx/imagemagick media-gfx/graphviz latexmk media-fonts/dejavu media-fonts/noto-cjk gnome-base/librsvg dev-texlive/texlive-xetex media-fonts/dejavu dev-python/pyyaml
> 
> Sphinx needs to be installed either:
> 1) via pip/pypi with:
> 
> 	/usr/sbin/python3 -m venv sphinx_latest
> 	. sphinx_latest/bin/activate
> 	pip install -r ./Documentation/sphinx/requirements.txt
> 
>     If you want to exit the virtualenv, you can use:
> 	deactivate
> 
> 2) As a package with:
> 
> 	sudo su -c 'echo "media-gfx/imagemagick svg png" > /etc/portage/package.use/imagemagick'
> 	sudo su -c 'echo "media-gfx/graphviz cairo pdf" > /etc/portage/package.use/graphviz'
> 	sudo emerge --ask python-sphinx
> 
>     Please note that Sphinx >= 3.0 will currently produce false-positive
>    warning when the same name is used for more than one type (functions,
>    structs, enums,...). This is known Sphinx bug. For more details, see:
> 	https://github.com/sphinx-doc/sphinx/pull/8313
> Can't build as 2 mandatory dependencies are missing at ./scripts/sphinx-pre-install line 990.
> 
> $ cd /root && LC_ALL=C ./scripts/sphinx-pre-install >/tmp/perl_venv
> --------------------------------------------------------------------
> 
> Can't build as 2 mandatory dependencies are missing at ./scripts/sphinx-pre-install line 990.
> 
> $ cd /root && LC_ALL=C ./scripts/sphinx-pre-install.py >/tmp/python_venv
> -------------------------------------------------------------------------
> 
> Can't build as 2 mandatory dependencies are missing
> 
> $ cd /root && LC_ALL=C ./scripts/sphinx-pre-install --no-virtualenv >/tmp/perl_no_venv
> ---------------------------------------------------------------------------------------
> 
> Can't build as 2 mandatory dependencies are missing at ./scripts/sphinx-pre-install line 990.
> 
> $ cd /root && LC_ALL=C ./scripts/sphinx-pre-install.py --no-virtualenv >/tmp/python_no_venv
> --------------------------------------------------------------------------------------------
> 
> Can't build as 2 mandatory dependencies are missing
> 
> $ cd /root && LC_ALL=C ./scripts/sphinx-pre-install --no-pdf >/tmp/perl_no_pdf
> -------------------------------------------------------------------------------
> 
> Can't build as 2 mandatory dependencies are missing at ./scripts/sphinx-pre-install line 990.
> 
> $ cd /root && LC_ALL=C ./scripts/sphinx-pre-install.py --no-pdf >/tmp/python_no_pdf
> ------------------------------------------------------------------------------------
> 
> Can't build as 2 mandatory dependencies are missing
> 
> $ sort /tmp/perl_venv -o /tmp/perl_venv.sorted
> -----------------------------------------------
> 
> 
> $ sort /tmp/python_venv -o /tmp/python_venv.sorted
> ---------------------------------------------------
> 
> 
> $ sort /tmp/perl_no_venv -o /tmp/perl_no_venv.sorted
> -----------------------------------------------------
> 
> 
> $ sort /tmp/python_no_venv -o /tmp/python_no_venv.sorted
> ---------------------------------------------------------
> 
> 
> $ sort /tmp/perl_no_pdf -o /tmp/perl_no_pdf.sorted
> ---------------------------------------------------
> 
> 
> $ sort /tmp/python_no_pdf -o /tmp/python_no_pdf.sorted
> -------------------------------------------------------
> 
> 
> $ diff -q /tmp/perl_venv.sorted /tmp/python_venv.sorted
> --------------------------------------------------------
> 
> Files /tmp/perl_venv.sorted and /tmp/python_venv.sorted differ
> 
> $ diff -q /tmp/perl_no_venv.sorted /tmp/python_no_venv.sorted
> --------------------------------------------------------------
> 
> Files /tmp/perl_no_venv.sorted and /tmp/python_no_venv.sorted differ
> 
> $ diff -q /tmp/perl_no_pdf.sorted /tmp/python_no_pdf.sorted
> ------------------------------------------------------------
> 
> Files /tmp/perl_no_pdf.sorted and /tmp/python_no_pdf.sorted differ
> 
> 
> 
> ERROR OUTPUT:
> 6 of 9 validate commands succeeded
> 
> 
> 
> 
> 
> Mauro Carvalho Chehab (3):
>   scripts: : fix version check for Fedora
>   scripts: : Convert script to Python
>   scripts: sphinx-pre-install: replace it by a Python version
> 
>  scripts/sphinx-pre-install | 2144 ++++++++++++++++++------------------
>  1 file changed, 1095 insertions(+), 1049 deletions(-)
>