[PATCH 0/3] Translate sphinx-pre-install to Python

[Mauro Carvalho Chehab <mchehab+huawei@kernel.org>]

As part of my efforts to make the doc build system more modern, it
follows a patch series converting sphinx-pre-install to python.

The first patch contains a small fix to the Perl version. It is
interesting to have, as otherwise regression tests fail.

The conversion itself was not hard. Most of the time I spent writing
scripts to test it.

The new version is meant to be bug-compatible with the previous
version, with some exceptions:

- I dropped support for too old Fedora distros. They're OOL;
- as it runs on Python, the tests for python could be bogus. I 
  added some FIXME entries to remember about that.;
- The package lists are now alphabetically sorted, whith alters
   the warning/eror order.
- It supports additional variants that I could install via lxc download
   template.
- I didn't test a couple of distro variants (OpenMandriva, RHEL,
  and Scientific Linux.

I did regression tests. Those are the test results summary:

+-----------------+---------+-------+-------------+----------------------------------------+
| distro          | status  | stage | return code | log file                               |
+-----------------+---------+-------+-------------+----------------------------------------+
| almalinux       | N/A     | Tests | 0           | ./test_logs/almalinux_report.log       |
| amazonlinux     | N/A     | Tests | 0           | ./test_logs/amazonlinux_report.log     |
| archlinux       | SUCCESS | Tests | 0           | ./test_logs/archlinux_report.log       |
| centos          | SUCCESS | Tests | 0           | ./test_logs/centos_report.log          |
| debian          | SUCCESS | Tests | 0           | ./test_logs/debian_report.log          |
| devuan          | N/A     | Tests | 0           | ./test_logs/devuan_report.log          |
| fedora          | SUCCESS | Tests | 0           | ./test_logs/fedora_report.log          |
| gentoo          | FAILED  | Tests | 3           | ./test_logs/gentoo_report.log          |
| kali            | N/A     | Tests | 0           | ./test_logs/kali_report.log            |
| mageia          | SUCCESS | Tests | 0           | ./test_logs/mageia_report.log          |
| mint            | N/A     | Tests | 0           | ./test_logs/mint_report.log            |
| openeuler       | N/A     | Tests | 0           | ./test_logs/openeuler_report.log       |
| opensuse        | SUCCESS | Tests | 0           | ./test_logs/opensuse_report.log        |
| oracle          | SUCCESS | Tests | 0           | ./test_logs/oracle_report.log          |
| rockylinux      | N/A     | Tests | 0           | ./test_logs/rockylinux_report.log      |
| springdalelinux | N/A     | Tests | 0           | ./test_logs/springdalelinux_report.log |
| ubuntu          | SUCCESS | Tests | 0           | ./test_logs/ubuntu_report.log          |
| ubuntu-lts      | SUCCESS | Tests | 0           | ./test_logs/ubuntu-lts_report.log      |
+-----------------+---------+-------+-------------+----------------------------------------+

N/A means that the version were not supported originally, so it can't be
properly compared.

The only one marked as failed is gentoo. This is actually a false positive.
That's the test output:

TEST REPORT: GENTOO

Generated: 2025-06-26 10:02:57
Stage: Tests

ERROR DETAILS:

CustomError: 6 of 9 validate commands succeeded
  File "/home/mchehab/bin/container_tests.py", line 1779, in <module>
    asyncio.run(main())
  File "/usr/lib64/python3.13/asyncio/runners.py", line 195, in run
    return runner.run(main)
  File "/usr/lib64/python3.13/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 706, in run_until_complete
    self.run_forever()
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 677, in run_forever
    self._run_once()
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 2034, in _run_once
    handle._run()
  File "/usr/lib64/python3.13/asyncio/events.py", line 89, in _run
    self._context.run(self._callback, *self._args)
  File "/home/mchehab/bin/container_tests.py", line 1773, in main
    await manager.run()
  File "/home/mchehab/bin/container_tests.py", line 1625, in run
    await self.run_distro(distro)
  File "/home/mchehab/bin/container_tests.py", line 1605, in run_distro
    await self.run_test(distro)
  File "/home/mchehab/bin/container_tests.py", line 1434, in run_test
    self.record_status(


TEST RESULTS:

Status: FAILED
Return Code: 3

STANDARD OUTPUT:

$ cd /root && LC_ALL=C ./scripts/sphinx-pre-install.py
-------------------------------------------------------

Detected OS: Gentoo Base System release 2.17.
Warning: better to also install "convert".
Warning: better to also install "dot".
Warning: better to also install "latexmk".
Warning: better to also install "media-fonts/dejavu".
Warning: better to also install "media-fonts/noto-cjk".
Warning: better to also install "rsvg-convert".
Warning: better to also install "xelatex".
ERROR: please install "yaml", otherwise, build won't work.
You should run:


	sudo su -c 'echo "media-gfx/imagemagick svg png" > /etc/portage/package.use/imagemagick'
	sudo su -c 'echo "media-gfx/graphviz cairo pdf" > /etc/portage/package.use/graphviz'
	sudo emerge --ask media-gfx/imagemagick media-gfx/graphviz latexmk media-fonts/dejavu media-fonts/noto-cjk gnome-base/librsvg dev-texlive/texlive-xetex media-fonts/dejavu dev-python/pyyaml

Sphinx needs to be installed either:
1) via pip/pypi with:

	/usr/sbin/python3 -m venv sphinx_latest
	. sphinx_latest/bin/activate
	pip install -r ./Documentation/sphinx/requirements.txt

    If you want to exit the virtualenv, you can use:
	deactivate

2) As a package with:


	sudo su -c 'echo "media-gfx/imagemagick svg png" > /etc/portage/package.use/imagemagick'
	sudo su -c 'echo "media-gfx/graphviz cairo pdf" > /etc/portage/package.use/graphviz'
	sudo emerge --ask python-sphinx

    Please note that Sphinx >= 3.0 will currently produce false-positive
   warning when the same name is used for more than one type (functions,
   structs, enums,...). This is known Sphinx bug. For more details, see:
	https://github.com/sphinx-doc/sphinx/pull/8313
Can't build as 2 mandatory dependencies are missing

$ cd /root && LC_ALL=C ./scripts/sphinx-pre-install
----------------------------------------------------

Detected OS: Gentoo Base System release 2.17.
Warning: better to also install "convert".
Warning: better to also install "dot".
Warning: better to also install "latexmk".
Warning: better to also install "media-fonts/dejavu".
Warning: better to also install "media-fonts/noto-cjk".
Warning: better to also install "rsvg-convert".
Warning: better to also install "xelatex".
ERROR: please install "yaml", otherwise, build won't work.
You should run:

	sudo su -c 'echo "media-gfx/imagemagick svg png" > /etc/portage/package.use/imagemagick'
	sudo su -c 'echo "media-gfx/graphviz cairo pdf" > /etc/portage/package.use/graphviz'
	sudo emerge --ask media-gfx/imagemagick media-gfx/graphviz latexmk media-fonts/dejavu media-fonts/noto-cjk gnome-base/librsvg dev-texlive/texlive-xetex media-fonts/dejavu dev-python/pyyaml

Sphinx needs to be installed either:
1) via pip/pypi with:

	/usr/sbin/python3 -m venv sphinx_latest
	. sphinx_latest/bin/activate
	pip install -r ./Documentation/sphinx/requirements.txt

    If you want to exit the virtualenv, you can use:
	deactivate

2) As a package with:

	sudo su -c 'echo "media-gfx/imagemagick svg png" > /etc/portage/package.use/imagemagick'
	sudo su -c 'echo "media-gfx/graphviz cairo pdf" > /etc/portage/package.use/graphviz'
	sudo emerge --ask python-sphinx

    Please note that Sphinx >= 3.0 will currently produce false-positive
   warning when the same name is used for more than one type (functions,
   structs, enums,...). This is known Sphinx bug. For more details, see:
	https://github.com/sphinx-doc/sphinx/pull/8313
Can't build as 2 mandatory dependencies are missing at ./scripts/sphinx-pre-install line 990.

$ cd /root && LC_ALL=C ./scripts/sphinx-pre-install >/tmp/perl_venv
--------------------------------------------------------------------

Can't build as 2 mandatory dependencies are missing at ./scripts/sphinx-pre-install line 990.

$ cd /root && LC_ALL=C ./scripts/sphinx-pre-install.py >/tmp/python_venv
-------------------------------------------------------------------------

Can't build as 2 mandatory dependencies are missing

$ cd /root && LC_ALL=C ./scripts/sphinx-pre-install --no-virtualenv >/tmp/perl_no_venv
---------------------------------------------------------------------------------------

Can't build as 2 mandatory dependencies are missing at ./scripts/sphinx-pre-install line 990.

$ cd /root && LC_ALL=C ./scripts/sphinx-pre-install.py --no-virtualenv >/tmp/python_no_venv
--------------------------------------------------------------------------------------------

Can't build as 2 mandatory dependencies are missing

$ cd /root && LC_ALL=C ./scripts/sphinx-pre-install --no-pdf >/tmp/perl_no_pdf
-------------------------------------------------------------------------------

Can't build as 2 mandatory dependencies are missing at ./scripts/sphinx-pre-install line 990.

$ cd /root && LC_ALL=C ./scripts/sphinx-pre-install.py --no-pdf >/tmp/python_no_pdf
------------------------------------------------------------------------------------

Can't build as 2 mandatory dependencies are missing

$ sort /tmp/perl_venv -o /tmp/perl_venv.sorted
-----------------------------------------------


$ sort /tmp/python_venv -o /tmp/python_venv.sorted
---------------------------------------------------


$ sort /tmp/perl_no_venv -o /tmp/perl_no_venv.sorted
-----------------------------------------------------


$ sort /tmp/python_no_venv -o /tmp/python_no_venv.sorted
---------------------------------------------------------


$ sort /tmp/perl_no_pdf -o /tmp/perl_no_pdf.sorted
---------------------------------------------------


$ sort /tmp/python_no_pdf -o /tmp/python_no_pdf.sorted
-------------------------------------------------------


$ diff -q /tmp/perl_venv.sorted /tmp/python_venv.sorted
--------------------------------------------------------

Files /tmp/perl_venv.sorted and /tmp/python_venv.sorted differ

$ diff -q /tmp/perl_no_venv.sorted /tmp/python_no_venv.sorted
--------------------------------------------------------------

Files /tmp/perl_no_venv.sorted and /tmp/python_no_venv.sorted differ

$ diff -q /tmp/perl_no_pdf.sorted /tmp/python_no_pdf.sorted
------------------------------------------------------------

Files /tmp/perl_no_pdf.sorted and /tmp/python_no_pdf.sorted differ



ERROR OUTPUT:
6 of 9 validate commands succeeded





Mauro Carvalho Chehab (3):
  scripts: : fix version check for Fedora
  scripts: : Convert script to Python
  scripts: sphinx-pre-install: replace it by a Python version

 scripts/sphinx-pre-install | 2144 ++++++++++++++++++------------------
 1 file changed, 1095 insertions(+), 1049 deletions(-)

-- 
2.49.0