* [PATCH v2 01/39] scripts: sphinx-pre-install: fix version check for Fedora
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 02/39] scripts: sphinx-pre-install: rename it to scripts/sphinx-pre-install.pl Mauro Carvalho Chehab
` (38 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
The script is now picking the wrong version. Fix it.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index 3f8d6925e896..07234d482fa8 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -413,7 +413,7 @@ sub give_redhat_hints()
my $old = 0;
my $rel;
my $noto_sans_redhat = "google-noto-sans-cjk-ttc-fonts";
- $rel = $1 if ($system_release =~ /(release|Linux)\s+(\d+)/);
+ $rel = $2 if ($system_release =~ /(release|Linux)\s+(\d+)/);
if (!($system_release =~ /Fedora/)) {
$map{"virtualenv"} = "python-virtualenv";
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 02/39] scripts: sphinx-pre-install: rename it to scripts/sphinx-pre-install.pl
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 01/39] scripts: sphinx-pre-install: fix version check for Fedora Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 03/39] scripts: sphinx-pre-install: Convert script to Python Mauro Carvalho Chehab
` (37 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
That helps us to later replace the scripts.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/Makefile | 14 +++++++-------
.../{sphinx-pre-install => sphinx-pre-install.pl} | 0
2 files changed, 7 insertions(+), 7 deletions(-)
rename scripts/{sphinx-pre-install => sphinx-pre-install.pl} (100%)
diff --git a/Documentation/Makefile b/Documentation/Makefile
index b98477df5ddf..c486fe3cc5e1 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -46,7 +46,7 @@ ifeq ($(HAVE_SPHINX),0)
.DEFAULT:
$(warning The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed and in PATH, or set the SPHINXBUILD make variable to point to the full path of the '$(SPHINXBUILD)' executable.)
@echo
- @$(srctree)/scripts/sphinx-pre-install
+ @$(srctree)/scripts/sphinx-pre-install.pl
@echo " SKIP Sphinx $@ target."
else # HAVE_SPHINX
@@ -121,7 +121,7 @@ $(YNL_RST_DIR)/%.rst: $(YNL_YAML_DIR)/%.yaml $(YNL_TOOL)
htmldocs texinfodocs latexdocs epubdocs xmldocs: $(YNL_INDEX)
htmldocs:
- @$(srctree)/scripts/sphinx-pre-install --version-check
+ @$(srctree)/scripts/sphinx-pre-install.pl --version-check
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,html,$(var),,$(var)))
# If Rust support is available and .config exists, add rustdoc generated contents.
@@ -135,7 +135,7 @@ endif
endif
texinfodocs:
- @$(srctree)/scripts/sphinx-pre-install --version-check
+ @$(srctree)/scripts/sphinx-pre-install.pl --version-check
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,texinfo,$(var),texinfo,$(var)))
# Note: the 'info' Make target is generated by sphinx itself when
@@ -147,7 +147,7 @@ linkcheckdocs:
@$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,linkcheck,$(var),,$(var)))
latexdocs:
- @$(srctree)/scripts/sphinx-pre-install --version-check
+ @$(srctree)/scripts/sphinx-pre-install.pl --version-check
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,latex,$(var),latex,$(var)))
ifeq ($(HAVE_PDFLATEX),0)
@@ -160,7 +160,7 @@ else # HAVE_PDFLATEX
pdfdocs: DENY_VF = XDG_CONFIG_HOME=$(FONTS_CONF_DENY_VF)
pdfdocs: latexdocs
- @$(srctree)/scripts/sphinx-pre-install --version-check
+ @$(srctree)/scripts/sphinx-pre-install.pl --version-check
$(foreach var,$(SPHINXDIRS), \
$(MAKE) PDFLATEX="$(PDFLATEX)" LATEXOPTS="$(LATEXOPTS)" $(DENY_VF) -C $(BUILDDIR)/$(var)/latex || sh $(srctree)/scripts/check-variable-fonts.sh || exit; \
mkdir -p $(BUILDDIR)/$(var)/pdf; \
@@ -170,11 +170,11 @@ pdfdocs: latexdocs
endif # HAVE_PDFLATEX
epubdocs:
- @$(srctree)/scripts/sphinx-pre-install --version-check
+ @$(srctree)/scripts/sphinx-pre-install.pl --version-check
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,epub,$(var),epub,$(var)))
xmldocs:
- @$(srctree)/scripts/sphinx-pre-install --version-check
+ @$(srctree)/scripts/sphinx-pre-install.pl --version-check
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,xml,$(var),xml,$(var)))
endif # HAVE_SPHINX
diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install.pl
similarity index 100%
rename from scripts/sphinx-pre-install
rename to scripts/sphinx-pre-install.pl
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 03/39] scripts: sphinx-pre-install: Convert script to Python
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 01/39] scripts: sphinx-pre-install: fix version check for Fedora Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 02/39] scripts: sphinx-pre-install: rename it to scripts/sphinx-pre-install.pl Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 04/39] scripts: sphinx-pre-install: Make it compatible with Python 3.6 Mauro Carvalho Chehab
` (36 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Port scripts/sphinx-pre-install to Python. That allows
a better maintainance of the file.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 1102 +++++++++++++++++++++++++++++++++
1 file changed, 1102 insertions(+)
create mode 100755 scripts/sphinx-pre-install.py
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
new file mode 100755
index 000000000000..dcee2181b72f
--- /dev/null
+++ b/scripts/sphinx-pre-install.py
@@ -0,0 +1,1102 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2017-2025 Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
+#
+# pylint: disable=C0103,C0114,C0115,C0116,C0301
+# pylint: disable=R0902,R0904,R0912,R0915,R1705,R1710,E1121
+
+
+import argparse
+import os
+import re
+import subprocess
+import sys
+from glob import glob
+
+
+def parse_version(version):
+ """Convert a major.minor.patch version into a tuple"""
+#
+ return tuple(int(x) for x in version.split("."))
+
+
+def ver_str(version):
+ """Returns a version tuple as major.minor.patch"""
+
+ return ".".join([str(x) for x in version])
+
+
+RECOMMENDED_VERSION = parse_version("3.4.3")
+
+
+class SphinxDependencyChecker:
+ # List of required texlive packages on Fedora and OpenSuse
+ texlive = {
+ "amsfonts.sty": "texlive-amsfonts",
+ "amsmath.sty": "texlive-amsmath",
+ "amssymb.sty": "texlive-amsfonts",
+ "amsthm.sty": "texlive-amscls",
+ "anyfontsize.sty": "texlive-anyfontsize",
+ "atbegshi.sty": "texlive-oberdiek",
+ "bm.sty": "texlive-tools",
+ "capt-of.sty": "texlive-capt-of",
+ "cmap.sty": "texlive-cmap",
+ "ctexhook.sty": "texlive-ctex",
+ "ecrm1000.tfm": "texlive-ec",
+ "eqparbox.sty": "texlive-eqparbox",
+ "eu1enc.def": "texlive-euenc",
+ "fancybox.sty": "texlive-fancybox",
+ "fancyvrb.sty": "texlive-fancyvrb",
+ "float.sty": "texlive-float",
+ "fncychap.sty": "texlive-fncychap",
+ "footnote.sty": "texlive-mdwtools",
+ "framed.sty": "texlive-framed",
+ "luatex85.sty": "texlive-luatex85",
+ "multirow.sty": "texlive-multirow",
+ "needspace.sty": "texlive-needspace",
+ "palatino.sty": "texlive-psnfss",
+ "parskip.sty": "texlive-parskip",
+ "polyglossia.sty": "texlive-polyglossia",
+ "tabulary.sty": "texlive-tabulary",
+ "threeparttable.sty": "texlive-threeparttable",
+ "titlesec.sty": "texlive-titlesec",
+ "ucs.sty": "texlive-ucs",
+ "upquote.sty": "texlive-upquote",
+ "wrapfig.sty": "texlive-wrapfig",
+ }
+
+ def __init__(self, args):
+ self.pdf = args.pdf
+ self.virtualenv = args.virtualenv
+ self.version_check = args.version_check
+
+ self.missing = {}
+
+ self.need = 0
+ self.optional = 0
+ self.need_symlink = 0
+ self.need_sphinx = 0
+ self.need_pip = 0
+ self.need_virtualenv = 0
+ self.rec_sphinx_upgrade = 0
+ self.verbose_warn_install = 1
+
+ self.system_release = ""
+ self.install = ""
+ self.virtenv_dir = ""
+ self.python_cmd = ""
+ self.activate_cmd = ""
+
+ self.min_version = (0, 0, 0)
+ self.cur_version = (0, 0, 0)
+ self.latest_avail_ver = (0, 0, 0)
+ self.venv_ver = (0, 0, 0)
+
+ prefix = os.environ.get("srctree", ".") + "/"
+
+ self.conf = prefix + "Documentation/conf.py"
+ self.requirement_file = prefix + "Documentation/sphinx/requirements.txt"
+ self.virtenv_prefix = ["sphinx_", "Sphinx_" ]
+
+ #
+ # Ancillary methods that don't depend on self
+ #
+
+ @staticmethod
+ def which(prog):
+ for path in os.environ.get("PATH", "").split(":"):
+ full_path = os.path.join(path, prog)
+ if os.access(full_path, os.X_OK):
+ return full_path
+
+ return None
+
+ @staticmethod
+ def find_python_no_venv():
+ # FIXME: does it makes sense now that this script is in Python?
+
+ result = subprocess.run(["pwd"], capture_output=True, text=True)
+ cur_dir = result.stdout.strip()
+
+ python_names = ["python3", "python"]
+
+ for d in os.environ.get("PATH", "").split(":"):
+ if f"{cur_dir}/sphinx" in d:
+ continue
+
+ for p in python_names:
+ if os.access(os.path.join(d, p), os.X_OK):
+ return os.path.join(d, p)
+
+ # Python not found at the PATH
+ return python_names[-1]
+
+ @staticmethod
+ def run(*args, **kwargs):
+ """Excecute a command, hiding its output by default"""
+
+ if not kwargs.get('capture_output', False):
+ if 'stdout' not in kwargs:
+ kwargs['stdout'] = subprocess.DEVNULL
+ if 'stderr' not in kwargs:
+ kwargs['stderr'] = subprocess.DEVNULL
+
+ return subprocess.run(*args, **kwargs)
+
+ #
+ # Methods to check if a feature exists
+ #
+
+ # Note: is_optional has 3 states:
+ # - 0: mandatory
+ # - 1: optional, but nice to have
+ # - 2: LaTeX optional - pdf builds without it, but may have visual impact
+
+ def check_missing(self, progs):
+ for prog, is_optional in sorted(self.missing.items()):
+ # At least on some LTS distros like CentOS 7, texlive doesn't
+ # provide all packages we need. When such distros are
+ # detected, we have to disable PDF output.
+ #
+ # So, we need to ignore the packages that distros would
+ # need for LaTeX to work
+ if is_optional == 2 and not self.pdf:
+ self.optional -= 1
+ continue
+
+ if self.verbose_warn_install:
+ if is_optional:
+ print(f'Warning: better to also install "{prog}".')
+ else:
+ print(f'ERROR: please install "{prog}", otherwise, build won\'t work.')
+
+ self.install += " " + progs.get(prog, prog)
+
+ self.install = self.install.lstrip()
+
+ def add_package(self, package, is_optional):
+ self.missing[package] = is_optional
+ if is_optional:
+ self.optional += 1
+ else:
+ self.need += 1
+
+ def check_missing_file(self, files, package, is_optional):
+ for f in files:
+ if os.path.exists(f):
+ return
+ self.add_package(package, is_optional)
+
+ def check_program(self, prog, is_optional):
+ found = self.which(prog)
+ if found:
+ return found
+
+ self.add_package(prog, is_optional)
+
+ return None
+
+ def check_perl_module(self, prog, is_optional):
+ # While testing with lxc download template, one of the
+ # distros (Oracle) didn't have perl - nor even an option to install
+ # before installing oraclelinux-release-el9 package.
+ #
+ # Check it before running an error. If perl is not there,
+ # add it as a mandatory package, as some parts of the doc builder
+ # needs it.
+ if not self.which("perl"):
+ self.add_package("perl", 0)
+ self.add_package(prog, is_optional)
+ return
+
+ try:
+ self.run(["perl", f"-M{prog}", "-e", "1"], check=True)
+ except subprocess.CalledProcessError:
+ self.add_package(prog, is_optional)
+
+ def check_python_module(self, module, is_optional):
+ # FIXME: is it needed at the Python version? Maybe due to venv?
+ if not self.python_cmd:
+ return
+
+ try:
+ self.run([self.python_cmd, "-c", f"import {module}"], check=True)
+ except subprocess.CalledProcessError:
+ self.add_package(module, is_optional)
+
+ def check_rpm_missing(self, pkgs, is_optional):
+ for prog in pkgs:
+ try:
+ self.run(["rpm", "-q", prog], check=True)
+ except subprocess.CalledProcessError:
+ self.add_package(prog, is_optional)
+
+ def check_pacman_missing(self, pkgs, is_optional):
+ for prog in pkgs:
+ try:
+ self.run(["pacman", "-Q", prog], check=True)
+ except subprocess.CalledProcessError:
+ self.add_package(prog, is_optional)
+
+ def check_missing_tex(self, is_optional):
+ kpsewhich = self.which("kpsewhich")
+ for prog, package in self.texlive.items():
+
+ # If kpsewhich is not there, just add it to deps
+ if not kpsewhich:
+ self.add_package(package, is_optional)
+ continue
+
+ # Check if the package is needed
+ try:
+ result = self.run(
+ [kpsewhich, prog], stdout=subprocess.PIPE, text=True, check=True
+ )
+
+ # Didn't find. Add it
+ if not result.stdout.strip():
+ self.add_package(package, is_optional)
+
+ except subprocess.CalledProcessError:
+ # kpsewhich returned an error. Add it, just in case
+ self.add_package(package, is_optional)
+
+ def get_sphinx_fname(self):
+ if "SPHINXBUILD" in os.environ:
+ return os.environ["SPHINXBUILD"]
+
+ fname = "sphinx-build"
+ if self.which(fname):
+ return fname
+
+ fname = "sphinx-build-3"
+ if self.which(fname):
+ self.need_symlink = 1
+ return fname
+
+ return ""
+
+ def get_sphinx_version(self, cmd):
+ try:
+ result = self.run([cmd, "--version"],
+ stdout=subprocess.PIPE,
+ stderr=subprocess.STDOUT,
+ text=True, check=True)
+ except (subprocess.CalledProcessError, FileNotFoundError):
+ return None
+
+ for line in result.stdout.split("\n"):
+ match = re.match(r"^sphinx-build\s+([\d\.]+)(?:\+(?:/[\da-f]+)|b\d+)?\s*$", line)
+ if match:
+ return parse_version(match.group(1))
+
+ match = re.match(r"^Sphinx.*\s+([\d\.]+)\s*$", line)
+ if match:
+ return parse_version(match.group(1))
+
+ def check_sphinx(self):
+ try:
+ with open(self.conf, "r", encoding="utf-8") as f:
+ for line in f:
+ match = re.match(r"^\s*needs_sphinx\s*=\s*[\'\"]([\d\.]+)[\'\"]", line)
+ if match:
+ self.min_version = parse_version(match.group(1))
+ break
+ except IOError:
+ sys.exit(f"Can't open {self.conf}")
+
+ if not self.min_version:
+ sys.exit(f"Can't get needs_sphinx version from {self.conf}")
+
+ self.virtenv_dir = self.virtenv_prefix[0] + "latest"
+
+ sphinx = self.get_sphinx_fname()
+ if not sphinx:
+ self.need_sphinx = 1
+ return
+
+ self.cur_version = self.get_sphinx_version(sphinx)
+ if not self.cur_version:
+ sys.exit(f"{sphinx} didn't return its version")
+
+ if self.cur_version < self.min_version:
+ curver = ver_str(self.cur_version)
+ minver = ver_str(self.min_version)
+
+ print(f"ERROR: Sphinx version is {curver}. It should be >= {minver}")
+ self.need_sphinx = 1
+ return
+
+ # On version check mode, just assume Sphinx has all mandatory deps
+ if self.version_check and self.cur_version >= RECOMMENDED_VERSION:
+ sys.exit(0)
+
+ def catcheck(self, filename):
+ if os.path.exists(filename):
+ with open(filename, "r", encoding="utf-8") as f:
+ return f.read().strip()
+ return ""
+
+ #
+ # Distro-specific hints methods
+ #
+
+ def give_debian_hints(self):
+ progs = {
+ "Pod::Usage": "perl-modules",
+ "convert": "imagemagick",
+ "dot": "graphviz",
+ "ensurepip": "python3-venv",
+ "python-sphinx": "python3-sphinx",
+ "rsvg-convert": "librsvg2-bin",
+ "virtualenv": "virtualenv",
+ "xelatex": "texlive-xetex",
+ "yaml": "python3-yaml",
+ }
+
+ if self.pdf:
+ pdf_pkgs = {
+ "texlive-lang-chinese": [
+ "/usr/share/texlive/texmf-dist/tex/latex/ctex/ctexhook.sty",
+ ],
+ "fonts-dejavu": [
+ "/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf",
+ ],
+ "fonts-noto-cjk": [
+ "/usr/share/fonts/noto-cjk/NotoSansCJK-Regular.ttc",
+ "/usr/share/fonts/opentype/noto/NotoSansCJK-Regular.ttc",
+ "/usr/share/fonts/opentype/noto/NotoSerifCJK-Regular.ttc",
+ ],
+ }
+
+ for package, files in pdf_pkgs.items():
+ self.check_missing_file(files, package, 2)
+
+ self.check_program("dvipng", 2)
+
+ self.check_missing(progs)
+
+ if not self.need and not self.optional:
+ return
+
+ if self.verbose_warn_install:
+ print("You should run:")
+ print(f"\n\tsudo apt-get install {self.install}")
+
+ def give_redhat_hints(self):
+ progs = {
+ "Pod::Usage": "perl-Pod-Usage",
+ "convert": "ImageMagick",
+ "dot": "graphviz",
+ "python-sphinx": "python3-sphinx",
+ "rsvg-convert": "librsvg2-tools",
+ "virtualenv": "python3-virtualenv",
+ "xelatex": "texlive-xetex-bin",
+ "yaml": "python3-pyyaml",
+ }
+
+ fedora26_opt_pkgs = [
+ "graphviz-gd", # Fedora 26: needed for PDF support
+ ]
+
+ fedora_tex_pkgs = [
+ "dejavu-sans-fonts",
+ "dejavu-sans-mono-fonts",
+ "dejavu-serif-fonts",
+ "texlive-collection-fontsrecommended",
+ "texlive-collection-latex",
+ "texlive-xecjk",
+ ]
+
+ old = 0
+ rel = None
+ pkg_manager = "dnf"
+
+ match = re.search(r"(release|Linux)\s+(\d+)", self.system_release)
+ if match:
+ rel = int(match.group(2))
+
+ if not rel:
+ print("Couldn't identify release number")
+ old = 1
+ self.pdf = False
+ elif re.search("Fedora", self.system_release):
+ # Fedora 38 and upper use this CJK font
+
+ noto_sans_redhat = "google-noto-sans-cjk-fonts"
+ else:
+ # Almalinux, CentOS, RHEL, ...
+
+ # at least up to version 9 (and Fedora < 38), that's the CJK font
+ noto_sans_redhat = "google-noto-sans-cjk-ttc-fonts"
+
+ progs["virtualenv"] = "python-virtualenv"
+
+ if rel and rel < 8:
+ old = 1
+ self.pdf = False
+
+ # RHEL 7 is in ELS, currently up to Jun, 2026
+
+ print("Note: texlive packages on RHEL/CENTOS <= 7 are incomplete. Can't support PDF output")
+ print("If you want to build PDF, please read:")
+ print("\thttps://www.systutorials.com/241660/how-to-install-tex-live-on-centos-7-linux/")
+
+ if self.pdf:
+ pdf_pkgs = [
+ "/usr/share/fonts/google-noto-cjk/NotoSansCJK-Regular.ttc",
+ "/usr/share/fonts/google-noto-sans-cjk-fonts/NotoSansCJK-Regular.ttc",
+ ]
+
+ self.check_missing_file(pdf_pkgs, noto_sans_redhat, 2)
+
+ if not old:
+ self.check_rpm_missing(fedora26_opt_pkgs, 2)
+ self.check_rpm_missing(fedora_tex_pkgs, 2)
+
+ self.check_missing_tex(2)
+
+ self.check_missing(progs)
+
+ if not self.need and not self.optional:
+ return
+
+ if self.verbose_warn_install:
+ print("You should run:")
+
+ if old:
+ # dnf is there since Fedora 18+ and RHEL 8
+ pkg_manager = "yum"
+
+ print(f"\n\tsudo {pkg_manager} install -y {self.install}")
+
+ def give_opensuse_hints(self):
+ progs = {
+ "Pod::Usage": "perl-Pod-Usage",
+ "convert": "ImageMagick",
+ "dot": "graphviz",
+ "python-sphinx": "python3-sphinx",
+ "virtualenv": "python3-virtualenv",
+ "xelatex": "texlive-xetex-bin",
+ "yaml": "python3-pyyaml",
+ }
+
+ # On Tumbleweed, this package is also named rsvg-convert
+ if not re.search(r"Tumbleweed", self.system_release):
+ progs["rsvg-convert"] = "rsvg-view"
+
+ suse_tex_pkgs = [
+ "texlive-babel-english",
+ "texlive-caption",
+ "texlive-colortbl",
+ "texlive-courier",
+ "texlive-dvips",
+ "texlive-helvetic",
+ "texlive-makeindex",
+ "texlive-metafont",
+ "texlive-metapost",
+ "texlive-palatino",
+ "texlive-preview",
+ "texlive-times",
+ "texlive-zapfchan",
+ "texlive-zapfding",
+ ]
+
+ progs["latexmk"] = "texlive-latexmk-bin"
+
+ # FIXME: add support for installing CJK fonts
+ #
+ # I tried hard, but was unable to find a way to install
+ # "Noto Sans CJK SC" on openSUSE
+
+ if self.pdf:
+ self.check_rpm_missing(suse_tex_pkgs, 2)
+ if self.pdf:
+ self.check_missing_tex(2)
+ self.check_missing(progs)
+
+ if not self.need and not self.optional:
+ return
+
+ if self.verbose_warn_install:
+ print("You should run:")
+ print(f"\n\tsudo zypper install --no-recommends {self.install}")
+
+ def give_mageia_hints(self):
+ progs = {
+ "Pod::Usage": "perl-Pod-Usage",
+ "convert": "ImageMagick",
+ "dot": "graphviz",
+ "python-sphinx": "python3-sphinx",
+ "rsvg-convert": "librsvg2",
+ "virtualenv": "python3-virtualenv",
+ "xelatex": "texlive",
+ "yaml": "python3-yaml",
+ }
+
+ tex_pkgs = [
+ "texlive-fontsextra",
+ ]
+
+ if re.search(r"OpenMandriva", self.system_release):
+ packager_cmd = "dnf install"
+ noto_sans = "noto-sans-cjk-fonts"
+ tex_pkgs = ["texlive-collection-fontsextra"]
+ else:
+ packager_cmd = "urpmi"
+ noto_sans = "google-noto-sans-cjk-ttc-fonts"
+
+ progs["latexmk"] = "texlive-collection-basic"
+
+ if self.pdf:
+ pdf_pkgs = [
+ "/usr/share/fonts/google-noto-cjk/NotoSansCJK-Regular.ttc",
+ "/usr/share/fonts/TTF/NotoSans-Regular.ttf",
+ ]
+
+ self.check_missing_file(pdf_pkgs, noto_sans, 2)
+ self.check_rpm_missing(tex_pkgs, 2)
+
+ self.check_missing(progs)
+
+ if not self.need and not self.optional:
+ return
+ if self.verbose_warn_install:
+ print("You should run:")
+ print(f"\n\tsudo {packager_cmd} {self.install}")
+
+ def give_arch_linux_hints(self):
+ progs = {
+ "convert": "imagemagick",
+ "dot": "graphviz",
+ "latexmk": "texlive-core",
+ "rsvg-convert": "extra/librsvg",
+ "virtualenv": "python-virtualenv",
+ "xelatex": "texlive-xetex",
+ "yaml": "python-yaml",
+ }
+
+ archlinux_tex_pkgs = [
+ "texlive-core",
+ "texlive-latexextra",
+ "ttf-dejavu",
+ ]
+
+ if self.pdf:
+ self.check_pacman_missing(archlinux_tex_pkgs, 2)
+
+ self.check_missing_file(
+ ["/usr/share/fonts/noto-cjk/NotoSansCJK-Regular.ttc"],
+ "noto-fonts-cjk",
+ 2,
+ )
+
+ self.check_missing(progs)
+
+ if not self.need and not self.optional:
+ return
+ if self.verbose_warn_install:
+ print("You should run:")
+ print(f"\n\tsudo pacman -S {self.install}")
+
+ def give_gentoo_hints(self):
+ progs = {
+ "convert": "media-gfx/imagemagick",
+ "dot": "media-gfx/graphviz",
+ "rsvg-convert": "gnome-base/librsvg",
+ "virtualenv": "dev-python/virtualenv",
+ "xelatex": "dev-texlive/texlive-xetex media-fonts/dejavu",
+ "yaml": "dev-python/pyyaml",
+ }
+
+ if self.pdf:
+ pdf_pkgs = {
+ "media-fonts/dejavu": [
+ "/usr/share/fonts/dejavu/DejaVuSans.ttf",
+ ],
+ "media-fonts/noto-cjk": [
+ "/usr/share/fonts/noto-cjk/NotoSansCJKsc-Regular.otf",
+ "/usr/share/fonts/noto-cjk/NotoSerifCJK-Regular.ttc",
+ ],
+ }
+ for package, files in pdf_pkgs.items():
+ self.check_missing_file(files, package, 2)
+
+ self.check_missing(progs)
+
+ if not self.need and not self.optional:
+ return
+
+ if self.verbose_warn_install:
+ print("You should run:")
+ print("\n")
+
+ imagemagick = "media-gfx/imagemagick svg png"
+ cairo = "media-gfx/graphviz cairo pdf"
+ portage_imagemagick = "/etc/portage/package.use/imagemagick"
+ portage_cairo = "/etc/portage/package.use/graphviz"
+
+ result = self.run(["grep", "imagemagick", portage_imagemagick],
+ stdout=subprocess.PIPE, text=True)
+ if not result.stdout.strip():
+ print(f"\tsudo su -c 'echo \"{imagemagick}\" > {portage_imagemagick}'")
+
+ result = self.run(["grep", "graphviz", portage_cairo],
+ stdout=subprocess.PIPE, text=True)
+
+ if not result.stdout.strip():
+ print(f"\tsudo su -c 'echo \"{cairo}\" > {portage_cairo}'")
+
+ print(f"\tsudo emerge --ask {self.install}")
+
+ #
+ # Dispatch the check to an os_specific hinter
+ #
+
+ def check_distros(self):
+ # OS-specific hints logic
+ os_hints = {
+ re.compile("Red Hat Enterprise Linux"): self.give_redhat_hints,
+ re.compile("Fedora"): self.give_redhat_hints,
+ re.compile("AlmaLinux"): self.give_redhat_hints,
+ re.compile("Amazon Linux"): self.give_redhat_hints,
+ re.compile("CentOS"): self.give_redhat_hints,
+ re.compile("openEuler"): self.give_redhat_hints,
+ re.compile("Oracle Linux Server"): self.give_redhat_hints,
+ re.compile("Rocky Linux"): self.give_redhat_hints,
+ re.compile("Scientific Linux"): self.give_redhat_hints,
+ re.compile("Springdale Open Enterprise"): self.give_redhat_hints,
+
+ re.compile("Ubuntu"): self.give_debian_hints,
+ re.compile("Debian"): self.give_debian_hints,
+ re.compile("Devuan"): self.give_debian_hints,
+ re.compile("Kali"): self.give_debian_hints,
+ re.compile("Mint"): self.give_debian_hints,
+
+ re.compile("openSUSE"): self.give_opensuse_hints,
+
+ re.compile("Mageia"): self.give_mageia_hints,
+ re.compile("OpenMandriva"): self.give_mageia_hints,
+
+ re.compile("Arch Linux"): self.give_arch_linux_hints,
+ re.compile("Gentoo"): self.give_gentoo_hints,
+ }
+
+ # If the OS is detected, use per-OS hint logic
+ for regex, os_hint in os_hints.items():
+ if regex.search(self.system_release):
+ os_hint()
+
+ return
+
+ #
+ # Fall-back to generic hint code for other distros
+ # That's far from ideal, specially for LaTeX dependencies.
+ #
+ progs = {"sphinx-build": "sphinx"}
+ if self.pdf:
+ self.check_missing_tex(2)
+
+ self.check_missing(progs)
+
+ print(f"I don't know distro {self.system_release}.")
+ print("So, I can't provide you a hint with the install procedure.")
+ print("There are likely missing dependencies.")
+
+ #
+ # Common dependencies
+ #
+ def deactivate_help(self):
+ print("\n If you want to exit the virtualenv, you can use:")
+ print("\tdeactivate")
+
+ def get_virtenv(self):
+ cwd = os.getcwd()
+
+ activates = []
+
+ # Add all sphinx prefixes with possible version numbers
+ for p in self.virtenv_prefix:
+ activates += glob(f"{cwd}/{p}[0-9]*/bin/activate")
+
+ activates.sort(reverse=True, key=str.lower)
+
+ # Place sphinx_latest first, if it exists
+ for p in self.virtenv_prefix:
+ activates = glob(f"{cwd}/{p}*latest/bin/activate") + activates
+
+ ver = (0, 0, 0)
+ for f in activates:
+ # Discard too old Sphinx virtual environments
+ match = re.search(r"(\d+)\.(\d+)\.(\d+)", f)
+ if match:
+ ver = (int(match.group(1)), int(match.group(2)), int(match.group(3)))
+
+ if ver < self.min_version:
+ continue
+
+ sphinx_cmd = f.replace("activate", "sphinx-build")
+ if not os.path.isfile(sphinx_cmd):
+ continue
+
+ ver = self.get_sphinx_version(sphinx_cmd)
+
+ if not ver:
+ venv_dir = f.replace("/bin/activate", "")
+ print(f"Warning: virtual environment {venv_dir} is not working.\n" \
+ "Python version upgrade? Remove it with:\n\n" \
+ "\trm -rf {venv_dir}\n\n")
+ else:
+ if self.need_sphinx and ver >= self.min_version:
+ return (f, ver)
+ elif parse_version(ver) > self.cur_version:
+ return (f, ver)
+
+ return ("", ver)
+
+ def recommend_sphinx_upgrade(self):
+ # Avoid running sphinx-builds from venv if cur_version is good
+ if self.cur_version and self.cur_version >= RECOMMENDED_VERSION:
+ self.latest_avail_ver = self.cur_version
+ return None
+
+ # Get the highest version from sphinx_*/bin/sphinx-build and the
+ # corresponding command to activate the venv/virtenv
+ self.activate_cmd, self.venv_ver = self.get_virtenv()
+
+ # Store the highest version from Sphinx existing virtualenvs
+ if self.activate_cmd and self.venv_ver > self.cur_version:
+ self.latest_avail_ver = self.venv_ver
+ else:
+ if self.cur_version:
+ self.latest_avail_ver = self.cur_version
+ else:
+ self.latest_avail_ver = (0, 0, 0)
+
+ # As we don't know package version of Sphinx, and there's no
+ # virtual environments, don't check if upgrades are needed
+ if not self.virtualenv:
+ if not self.latest_avail_ver:
+ return None
+
+ return self.latest_avail_ver
+
+ # Either there are already a virtual env or a new one should be created
+ self.need_pip = 1
+
+ if not self.latest_avail_ver:
+ return None
+
+ # Return if the reason is due to an upgrade or not
+ if self.latest_avail_ver != (0, 0, 0):
+ if self.latest_avail_ver < RECOMMENDED_VERSION:
+ self.rec_sphinx_upgrade = 1
+
+ return self.latest_avail_ver
+
+ def recommend_sphinx_version(self, virtualenv_cmd):
+ # The logic here is complex, as it have to deal with different versions:
+ # - minimal supported version;
+ # - minimal PDF version;
+ # - recommended version.
+ # It also needs to work fine with both distro's package and venv/virtualenv
+
+ # Version is OK. Nothing to do.
+ if self.cur_version != (0, 0, 0) and self.cur_version >= RECOMMENDED_VERSION:
+ return
+
+ if not self.need_sphinx:
+ # sphinx-build is present and its version is >= $min_version
+
+ # only recommend enabling a newer virtenv version if makes sense.
+ if self.latest_avail_ver and self.latest_avail_ver > self.cur_version:
+ print("\nYou may also use the newer Sphinx version {self.latest_avail_ver} with:")
+ if f"{self.virtenv_prefix}" in os.getcwd():
+ print("\tdeactivate")
+ print(f"\t. {self.activate_cmd}")
+ self.deactivate_help()
+ return
+
+ if self.latest_avail_ver and self.latest_avail_ver >= RECOMMENDED_VERSION:
+ return
+
+ if not self.virtualenv:
+ # No sphinx either via package or via virtenv. As we can't
+ # Compare the versions here, just return, recommending the
+ # user to install it from the package distro.
+ if not self.latest_avail_ver or self.latest_avail_ver == (0, 0, 0):
+ return
+
+ # User doesn't want a virtenv recommendation, but he already
+ # installed one via virtenv with a newer version.
+ # So, print commands to enable it
+ if self.latest_avail_ver > self.cur_version:
+ print("\nYou may also use the Sphinx virtualenv version {self.latest_avail_ver} with:")
+ if f"{self.virtenv_prefix}" in os.getcwd():
+ print("\tdeactivate")
+ print(f"\t. {self.activate_cmd}")
+ self.deactivate_help()
+ return
+ print("\n")
+ else:
+ if self.need_sphinx:
+ self.need += 1
+
+ # Suggest newer versions if current ones are too old
+ if self.latest_avail_ver and self.latest_avail_ver >= self.min_version:
+ if self.latest_avail_ver >= RECOMMENDED_VERSION:
+ print("\nNeed to activate Sphinx (version {self.latest_avail_ver}) on virtualenv with:")
+ print(f"\t. {self.activate_cmd}")
+ self.deactivate_help()
+ return
+
+ # Version is above the minimal required one, but may be
+ # below the recommended one. So, print warnings/notes
+ if self.latest_avail_ver < RECOMMENDED_VERSION:
+ print(f"Warning: It is recommended at least Sphinx version {RECOMMENDED_VERSION}.")
+
+ # At this point, either it needs Sphinx or upgrade is recommended,
+ # both via pip
+
+ if self.rec_sphinx_upgrade:
+ if not self.virtualenv:
+ print("Instead of install/upgrade Python Sphinx pkg, you could use pip/pypi with:\n\n")
+ else:
+ print("To upgrade Sphinx, use:\n\n")
+ else:
+ print("\nSphinx needs to be installed either:\n1) via pip/pypi with:\n")
+
+ self.python_cmd = self.find_python_no_venv()
+
+ print(f"\t{virtualenv_cmd} {self.virtenv_dir}")
+ print(f"\t. {self.virtenv_dir}/bin/activate")
+ print(f"\tpip install -r {self.requirement_file}")
+ self.deactivate_help()
+
+ print("\n2) As a package with:")
+
+ old_need = self.need
+ old_optional = self.optional
+ self.missing = {}
+ self.pdf = False
+ self.optional = 0
+ self.install = ""
+ old_verbose = self.verbose_warn_install
+ self.verbose_warn_install = 0
+
+ self.add_package("python-sphinx", 0)
+
+ self.check_distros()
+
+ self.need = old_need
+ self.optional = old_optional
+ self.verbose_warn_install = old_verbose
+
+ print("\n" \
+ " Please note that Sphinx >= 3.0 will currently produce false-positive\n" \
+ " warning when the same name is used for more than one type (functions,\n" \
+ " structs, enums,...). This is known Sphinx bug. For more details, see:\n" \
+ "\thttps://github.com/sphinx-doc/sphinx/pull/8313")
+
+ def check_needs(self):
+ self.get_system_release()
+
+ # Check if Sphinx is already accessible from current environment
+ self.check_sphinx()
+
+ if self.system_release:
+ print(f"Detected OS: {self.system_release}.")
+ else:
+ print("Unknown OS")
+ if self.cur_version != (0, 0, 0):
+ ver = ver_str(self.cur_version)
+ print(f"Sphinx version: {ver}\n")
+
+ # FIXME: Check python command line, trying first python3
+ self.python_cmd = self.which("python3")
+ if not self.python_cmd:
+ self.python_cmd = self.check_program("python", 0)
+
+ # Check the type of virtual env, depending on Python version
+ if self.python_cmd:
+ if self.virtualenv:
+ try:
+ result = self.run(
+ [self.python_cmd, "--version"],
+ capture_output=True,
+ text=True,
+ check=True,
+ )
+
+ output = result.stdout + result.stderr
+
+ match = re.search(r"(\d+)\.(\d+)\.", output)
+ if match:
+ major = int(match.group(1))
+ minor = int(match.group(2))
+
+ if major < 3:
+ sys.exit("Python 3 is required to build the kernel docs")
+ if major == 3 and minor < 3:
+ self.need_virtualenv = True
+ else:
+ sys.exit(f"Warning: couldn't identify {self.python_cmd} version!")
+
+ except subprocess.CalledProcessError as e:
+ sys.exit(f"Error checking Python version: {e}")
+ else:
+ self.add_package("python-sphinx", 0)
+
+ self.venv_ver = self.recommend_sphinx_upgrade()
+
+ virtualenv_cmd = ""
+
+ if self.need_pip:
+ # Set virtualenv command line, if python < 3.3
+ # FIXME: can be removed as we're now with an upper min requirement
+ # but then we need to check python version
+ if self.need_virtualenv:
+ virtualenv_cmd = self.which("virtualenv-3")
+ if not virtualenv_cmd:
+ virtualenv_cmd = self.which("virtualenv-3.5")
+ if not virtualenv_cmd:
+ self.check_program("virtualenv", 0)
+ virtualenv_cmd = "virtualenv"
+ else:
+ virtualenv_cmd = f"{self.python_cmd} -m venv"
+ self.check_python_module("ensurepip", 0)
+
+ # Check for needed programs/tools
+ self.check_perl_module("Pod::Usage", 0)
+ self.check_python_module("yaml", 0)
+ self.check_program("make", 0)
+ self.check_program("gcc", 0)
+ self.check_program("dot", 1)
+ self.check_program("convert", 1)
+
+ if self.pdf:
+ # Extra PDF files - should use 2 for LaTeX is_optional
+ self.check_program("xelatex", 2)
+ self.check_program("rsvg-convert", 2)
+ self.check_program("latexmk", 2)
+
+ # Do distro-specific checks and output distro-install commands
+ self.check_distros()
+
+ if not self.python_cmd:
+ if self.need == 1:
+ sys.exit("Can't build as 1 mandatory dependency is missing")
+ elif self.need:
+ sys.exit(f"Can't build as {self.need} mandatory dependencies are missing")
+
+ # Check if sphinx-build is called sphinx-build-3
+ if self.need_symlink:
+ sphinx_path = self.which("sphinx-build-3")
+ if sphinx_path:
+ print(f"\tsudo ln -sf {sphinx_path} /usr/bin/sphinx-build\n")
+
+ self.recommend_sphinx_version(virtualenv_cmd)
+ print("")
+
+ if not self.optional:
+ print("All optional dependencies are met.")
+
+ if self.need == 1:
+ sys.exit("Can't build as 1 mandatory dependency is missing")
+ elif self.need:
+ sys.exit(f"Can't build as {self.need} mandatory dependencies are missing")
+
+ print("Needed package dependencies are met.")
+
+ def get_system_release(self):
+ """
+ Determine the system type. There's no unique way that would work
+ with all distros with a minimal package install. So, several
+ methods are used here.
+
+ By default, it will use lsb_release function. If not available, it will
+ fail back to reading the known different places where the distro name
+ is stored.
+
+ Several modern distros now have /etc/os-release, which usually have
+ a decent coverage.
+ """
+
+ if self.which("lsb_release"):
+ result = self.run(["lsb_release", "-d"], capture_output=True, text=True)
+ self.system_release = result.stdout.replace("Description:", "").strip()
+
+ release_files = [
+ "/etc/system-release",
+ "/etc/redhat-release",
+ "/etc/lsb-release",
+ "/etc/gentoo-release",
+ ]
+
+ if not self.system_release:
+ for f in release_files:
+ self.system_release = self.catcheck(f)
+ if self.system_release:
+ break
+
+ # This seems more common than LSB these days
+ if not self.system_release:
+ os_var = {}
+ try:
+ with open("/etc/os-release", "r", encoding="utf-8") as f:
+ for line in f:
+ match = re.match(r"^([\w\d\_]+)=\"?([^\"]*)\"?\n", line)
+ if match:
+ os_var[match.group(1)] = match.group(2)
+
+ self.system_release = os_var.get("NAME", "")
+ if "VERSION_ID" in os_var:
+ self.system_release += " " + os_var["VERSION_ID"]
+ elif "VERSION" in os_var:
+ self.system_release += " " + os_var["VERSION"]
+ except IOError:
+ pass
+
+ if not self.system_release:
+ self.system_release = self.catcheck("/etc/issue")
+
+ self.system_release = self.system_release.strip()
+
+DESCRIPTION = """
+Process some flags related to Sphinx installation and documentation build.
+"""
+
+
+def main():
+ parser = argparse.ArgumentParser(description=DESCRIPTION)
+
+ parser.add_argument(
+ "--no-virtualenv",
+ action="store_false",
+ dest="virtualenv",
+ help="Recommend installing Sphinx instead of using a virtualenv",
+ )
+
+ parser.add_argument(
+ "--no-pdf",
+ action="store_false",
+ dest="pdf",
+ help="Don't check for dependencies required to build PDF docs",
+ )
+
+ parser.add_argument(
+ "--version-check",
+ action="store_true",
+ dest="version_check",
+ help="If version is compatible, don't check for missing dependencies",
+ )
+
+ args = parser.parse_args()
+
+ checker = SphinxDependencyChecker(args)
+
+ checker.check_needs()
+
+
+if __name__ == "__main__":
+ main()
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 04/39] scripts: sphinx-pre-install: Make it compatible with Python 3.6
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (2 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 03/39] scripts: sphinx-pre-install: Convert script to Python Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 05/39] scripts: sphinx-pre-install: run on a supported version Mauro Carvalho Chehab
` (35 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
The minimal version requirements we have is 3.9. Yet, the
script which detects it is this one. So, let's try supporting
an old version here, as we may want to suggest to upgrade
Python version to build the docs.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 16 ++++++++++++++--
1 file changed, 14 insertions(+), 2 deletions(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index dcee2181b72f..71d86b230b22 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -115,7 +115,8 @@ class SphinxDependencyChecker:
def find_python_no_venv():
# FIXME: does it makes sense now that this script is in Python?
- result = subprocess.run(["pwd"], capture_output=True, text=True)
+ result = SphinxDependencyChecker.run(["pwd"], capture_output=True,
+ text=True)
cur_dir = result.stdout.strip()
python_names = ["python3", "python"]
@@ -135,12 +136,23 @@ class SphinxDependencyChecker:
def run(*args, **kwargs):
"""Excecute a command, hiding its output by default"""
- if not kwargs.get('capture_output', False):
+ capture_output = kwargs.pop('capture_output', False)
+
+ if capture_output:
+ if 'stdout' not in kwargs:
+ kwargs['stdout'] = subprocess.PIPE
+ if 'stderr' not in kwargs:
+ kwargs['stderr'] = subprocess.PIPE
+ else:
if 'stdout' not in kwargs:
kwargs['stdout'] = subprocess.DEVNULL
if 'stderr' not in kwargs:
kwargs['stderr'] = subprocess.DEVNULL
+ # Don't break with older Python versions
+ if 'text' in kwargs and sys.version_info < (3, 7):
+ kwargs['universal_newlines'] = kwargs.pop('text')
+
return subprocess.run(*args, **kwargs)
#
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 05/39] scripts: sphinx-pre-install: run on a supported version
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (3 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 04/39] scripts: sphinx-pre-install: Make it compatible with Python 3.6 Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 06/39] scripts: sphinx-pre-install: drop obsolete routines Mauro Carvalho Chehab
` (34 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
The scripts/sphinx-pre-install is used to detect problems at
the system environment and adjust it to build the Kernel
documentation. If the version is too old, it won't run, though.
Check if the version which started the script is valid. If not,
seek for a new one that is compatible with documentation
build.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 65 ++++++++++++++++++++++++++++++++++-
1 file changed, 64 insertions(+), 1 deletion(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index 71d86b230b22..3912359d2bae 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -5,6 +5,9 @@
# pylint: disable=C0103,C0114,C0115,C0116,C0301
# pylint: disable=R0902,R0904,R0912,R0915,R1705,R1710,E1121
+# Note: this script requires at least Python 3.6 to run.
+# Don't add changes not compatible with it, it is meant to report
+# incompatible python versions.
import argparse
import os
@@ -16,7 +19,6 @@ from glob import glob
def parse_version(version):
"""Convert a major.minor.patch version into a tuple"""
-#
return tuple(int(x) for x in version.split("."))
@@ -27,6 +29,7 @@ def ver_str(version):
RECOMMENDED_VERSION = parse_version("3.4.3")
+MIN_PYTHON_VERSION = parse_version("3.7")
class SphinxDependencyChecker:
@@ -132,6 +135,65 @@ class SphinxDependencyChecker:
# Python not found at the PATH
return python_names[-1]
+ @staticmethod
+ def get_python_version(cmd):
+
+ result = SphinxDependencyChecker.run([cmd, "--version"],
+ capture_output=True, text=True)
+ version = result.stdout.strip()
+
+ match = re.search(r"(\d+\.\d+\.\d+)", version)
+ if match:
+ return parse_version(match.group(1))
+
+ print(f"Can't parse version {version}")
+ return (0, 0, 0)
+
+ @staticmethod
+ def find_python():
+
+ patterns = [
+ "python3.[0-9]",
+ "python3.[0-9][0-9]",
+ ]
+
+ new_python_cmd = None
+
+ # Seek for a python binary newer than MIN_PYTHON_VERSION
+ for path in os.getenv("PATH", "").split(":"):
+ for pattern in patterns:
+ for cmd in glob(os.path.join(path, pattern)):
+ if os.path.isfile(cmd) and os.access(cmd, os.X_OK):
+ version = SphinxDependencyChecker.get_python_version(cmd)
+ if version >= MIN_PYTHON_VERSION:
+ return(cmd)
+
+ @staticmethod
+ def check_python():
+
+ cur_ver = sys.version_info[:3]
+ if cur_ver >= MIN_PYTHON_VERSION:
+ return
+
+ python_ver = ver_str(cur_ver)
+
+ new_python_cmd = SphinxDependencyChecker.find_python()
+ if not new_python_cmd:
+ print(f"ERROR: Python version {python_ver} is not spported anymore")
+ print(f" Can't find a new version. This script may fail")
+ return
+
+ # Restart script using the newer version
+ script_path = os.path.abspath(sys.argv[0])
+ args = [new_python_cmd, script_path] + sys.argv[1:]
+
+ print(f"Python {python_ver} not supported. Changing to {new_python_cmd}")
+
+ try:
+ os.execv(new_python_cmd, args)
+ except OSError as e:
+ sys.exit(f"Failed to restart with {new_python_cmd}: {e}")
+
@staticmethod
def run(*args, **kwargs):
"""Excecute a command, hiding its output by default"""
@@ -1107,6 +1169,7 @@ def main():
checker = SphinxDependencyChecker(args)
+ checker.check_python()
checker.check_needs()
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 06/39] scripts: sphinx-pre-install: drop obsolete routines
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (4 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 05/39] scripts: sphinx-pre-install: run on a supported version Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 07/39] scripts: sphinx-pre-install: drop support for old virtualenv Mauro Carvalho Chehab
` (33 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 23 +----------------------
1 file changed, 1 insertion(+), 22 deletions(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index 3912359d2bae..b639acd455cc 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -114,27 +114,6 @@ class SphinxDependencyChecker:
return None
- @staticmethod
- def find_python_no_venv():
- # FIXME: does it makes sense now that this script is in Python?
-
- result = SphinxDependencyChecker.run(["pwd"], capture_output=True,
- text=True)
- cur_dir = result.stdout.strip()
-
- python_names = ["python3", "python"]
-
- for d in os.environ.get("PATH", "").split(":"):
- if f"{cur_dir}/sphinx" in d:
- continue
-
- for p in python_names:
- if os.access(os.path.join(d, p), os.X_OK):
- return os.path.join(d, p)
-
- # Python not found at the PATH
- return python_names[-1]
-
@staticmethod
def get_python_version(cmd):
@@ -940,7 +919,7 @@ class SphinxDependencyChecker:
else:
print("\nSphinx needs to be installed either:\n1) via pip/pypi with:\n")
- self.python_cmd = self.find_python_no_venv()
+ self.python_cmd = os.path.abspath(sys.argv[0])
print(f"\t{virtualenv_cmd} {self.virtenv_dir}")
print(f"\t. {self.virtenv_dir}/bin/activate")
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 07/39] scripts: sphinx-pre-install: drop support for old virtualenv
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (5 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 06/39] scripts: sphinx-pre-install: drop obsolete routines Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 08/39] scripts: sphinx-pre-install: Address issues with OpenSUSE Leap 15.x Mauro Carvalho Chehab
` (32 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Up to Python 3.2, the virtual environment were created
via virtualenv binary.
As we dropped support for such old version, clean up the code.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 69 +++++++++--------------------------
1 file changed, 18 insertions(+), 51 deletions(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index b639acd455cc..0a73b1b33842 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -80,7 +80,6 @@ class SphinxDependencyChecker:
self.need_symlink = 0
self.need_sphinx = 0
self.need_pip = 0
- self.need_virtualenv = 0
self.rec_sphinx_upgrade = 0
self.verbose_warn_install = 1
@@ -919,12 +918,14 @@ class SphinxDependencyChecker:
else:
print("\nSphinx needs to be installed either:\n1) via pip/pypi with:\n")
- self.python_cmd = os.path.abspath(sys.argv[0])
-
- print(f"\t{virtualenv_cmd} {self.virtenv_dir}")
- print(f"\t. {self.virtenv_dir}/bin/activate")
- print(f"\tpip install -r {self.requirement_file}")
- self.deactivate_help()
+ if not virtualenv_cmd:
+ print(" Currently not possible.\n")
+ print(" Please upgrade Python to a newer version and run this script again")
+ else:
+ print(f"\t{virtualenv_cmd} {self.virtenv_dir}")
+ print(f"\t. {self.virtenv_dir}/bin/activate")
+ print(f"\tpip install -r {self.requirement_file}")
+ self.deactivate_help()
print("\n2) As a package with:")
@@ -953,6 +954,7 @@ class SphinxDependencyChecker:
def check_needs(self):
self.get_system_release()
+ self.python_cmd = sys.executable
# Check if Sphinx is already accessible from current environment
self.check_sphinx()
@@ -965,56 +967,21 @@ class SphinxDependencyChecker:
ver = ver_str(self.cur_version)
print(f"Sphinx version: {ver}\n")
- # FIXME: Check python command line, trying first python3
- self.python_cmd = self.which("python3")
- if not self.python_cmd:
- self.python_cmd = self.check_program("python", 0)
-
# Check the type of virtual env, depending on Python version
- if self.python_cmd:
- if self.virtualenv:
- try:
- result = self.run(
- [self.python_cmd, "--version"],
- capture_output=True,
- text=True,
- check=True,
- )
+ virtualenv_cmd = None
- output = result.stdout + result.stderr
-
- match = re.search(r"(\d+)\.(\d+)\.", output)
- if match:
- major = int(match.group(1))
- minor = int(match.group(2))
-
- if major < 3:
- sys.exit("Python 3 is required to build the kernel docs")
- if major == 3 and minor < 3:
- self.need_virtualenv = True
- else:
- sys.exit(f"Warning: couldn't identify {self.python_cmd} version!")
-
- except subprocess.CalledProcessError as e:
- sys.exit(f"Error checking Python version: {e}")
- else:
- self.add_package("python-sphinx", 0)
+ if sys.version_info < MIN_PYTHON_VERSION:
+ min_ver = ver_str(MIN_PYTHON_VERSION)
+ print(f"ERROR: at least python {min_ver} is required to build the kernel docs")
+ self.need_sphinx = 1
self.venv_ver = self.recommend_sphinx_upgrade()
- virtualenv_cmd = ""
-
if self.need_pip:
- # Set virtualenv command line, if python < 3.3
- # FIXME: can be removed as we're now with an upper min requirement
- # but then we need to check python version
- if self.need_virtualenv:
- virtualenv_cmd = self.which("virtualenv-3")
- if not virtualenv_cmd:
- virtualenv_cmd = self.which("virtualenv-3.5")
- if not virtualenv_cmd:
- self.check_program("virtualenv", 0)
- virtualenv_cmd = "virtualenv"
+ if sys.version_info < MIN_PYTHON_VERSION:
+ self.need_pip = False
+ print("Warning: python version is not supported.")
+
else:
virtualenv_cmd = f"{self.python_cmd} -m venv"
self.check_python_module("ensurepip", 0)
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 08/39] scripts: sphinx-pre-install: Address issues with OpenSUSE Leap 15.x
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (6 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 07/39] scripts: sphinx-pre-install: drop support for old virtualenv Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 09/39] scripts: sphinx-pre-install: fix opensuse Leap hint for PyYAML Mauro Carvalho Chehab
` (31 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
On openSUSE Leap 15.6, which is the current LTS version, has two
Sphinx packages. The normal one requires Python 3.6, which we
don't support anymore. However, it also has Python 3.11 with a
newer Sphinx version (7.2.6).
Suggest the newer version:
Detected OS: openSUSE Leap 15.6.
ERROR: at least python 3.7 is required to build the kernel docs
Warning: python version is not supported.
Warning: better to also install "convert".
Warning: better to also install "dot".
ERROR: please install "yaml", otherwise, build won't work.
You should run:
sudo zypper install --no-recommends ImageMagick graphviz python311-pyyaml
Sphinx needs to be installed either:
1) via pip/pypi with:
Currently not possible.
Please upgrade Python to a newer version and run this script again
2) As a package with:
sudo zypper install --no-recommends python311-Sphinx
Please note that Sphinx >= 3.0 will currently produce false-positive
warning when the same name is used for more than one type (functions,
structs, enums,...). This is known Sphinx bug. For more details, see:
https://github.com/sphinx-doc/sphinx/pull/8313
Can't build as 2 mandatory dependencies are missing
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index 0a73b1b33842..eca42d90ed01 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -556,6 +556,22 @@ class SphinxDependencyChecker:
progs["latexmk"] = "texlive-latexmk-bin"
+ match = re.search(r"(Leap)\s+(\d+).(\d)", self.system_release)
+ if match:
+ rel = int(match.group(2))
+
+ # Leap 15.x uses Python 3.6, which is not compatible with
+ # the build system anymore. Suggest Python 3.11
+ if rel == 15:
+ if not self.which(self.python_cmd):
+ self.add_package(self.python_cmd, 0)
+
+ progs.update({
+ "python-sphinx": "python311-Sphinx",
+ "virtualenv": "python311-virtualenv",
+ "yaml": "python311-pyyaml",
+ })
+
# FIXME: add support for installing CJK fonts
#
# I tried hard, but was unable to find a way to install
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 09/39] scripts: sphinx-pre-install: fix opensuse Leap hint for PyYAML
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (7 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 08/39] scripts: sphinx-pre-install: Address issues with OpenSUSE Leap 15.x Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 10/39] scripts: sphinx-pre-install: fix support for gentoo Mauro Carvalho Chehab
` (30 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
On Leap, the name of the package is python311-PyYAML.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index eca42d90ed01..65438c198674 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -569,7 +569,7 @@ class SphinxDependencyChecker:
progs.update({
"python-sphinx": "python311-Sphinx",
"virtualenv": "python311-virtualenv",
- "yaml": "python311-pyyaml",
+ "yaml": "python311-PyYAML",
})
# FIXME: add support for installing CJK fonts
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 10/39] scripts: sphinx-pre-install: fix support for gentoo
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (8 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 09/39] scripts: sphinx-pre-install: fix opensuse Leap hint for PyYAML Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 11/39] scripts: sphinx-pre-install: Address issues with OpenSUSE Tumbleweed Mauro Carvalho Chehab
` (29 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Some gentoo packages have changes. Fix it.
While here, improve emerge portage use logic.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 35 ++++++++++++++++-------------------
1 file changed, 16 insertions(+), 19 deletions(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index 65438c198674..b793796329c8 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -669,12 +669,13 @@ class SphinxDependencyChecker:
def give_gentoo_hints(self):
progs = {
- "convert": "media-gfx/imagemagick",
- "dot": "media-gfx/graphviz",
- "rsvg-convert": "gnome-base/librsvg",
- "virtualenv": "dev-python/virtualenv",
- "xelatex": "dev-texlive/texlive-xetex media-fonts/dejavu",
- "yaml": "dev-python/pyyaml",
+ "convert": "media-gfx/imagemagick",
+ "dot": "media-gfx/graphviz",
+ "rsvg-convert": "gnome-base/librsvg",
+ "virtualenv": "dev-python/virtualenv",
+ "xelatex": "dev-texlive/texlive-xetex media-fonts/dejavu",
+ "yaml": "dev-python/pyyaml",
+ "python-sphinx": "dev-python/sphinx",
}
if self.pdf:
@@ -699,21 +700,17 @@ class SphinxDependencyChecker:
print("You should run:")
print("\n")
- imagemagick = "media-gfx/imagemagick svg png"
- cairo = "media-gfx/graphviz cairo pdf"
- portage_imagemagick = "/etc/portage/package.use/imagemagick"
- portage_cairo = "/etc/portage/package.use/graphviz"
- result = self.run(["grep", "imagemagick", portage_imagemagick],
- stdout=subprocess.PIPE, text=True)
- if not result.stdout.strip():
- print(f"\tsudo su -c 'echo \"{imagemagick}\" > {portage_imagemagick}'")
+ portages = [
+ "media-gfx/imagemagick",
+ "media-gfx/graphviz",
+ ]
- result = self.run(["grep", "graphviz", portage_cairo],
- stdout=subprocess.PIPE, text=True)
-
- if not result.stdout.strip():
- print(f"\tsudo su -c 'echo \"{cairo}\" > {portage_cairo}'")
+ for p in portages:
+ result = self.run(["grep", p, "/etc/portage/package.use/*"],
+ stdout=subprocess.PIPE, text=True)
+ if not result.stdout.strip():
+ print(f"\tsudo emerge -av1 {p}")
print(f"\tsudo emerge --ask {self.install}")
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 11/39] scripts: sphinx-pre-install: Address issues with OpenSUSE Tumbleweed
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (9 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 10/39] scripts: sphinx-pre-install: fix support for gentoo Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 12/39] scripts: sphinx-pre-install: only show portage hints once Mauro Carvalho Chehab
` (28 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
On Tumbleweed, package names are named after python-313*, as it also
has older python versions on it.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index b793796329c8..0e165ad05fdb 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -571,6 +571,14 @@ class SphinxDependencyChecker:
"virtualenv": "python311-virtualenv",
"yaml": "python311-PyYAML",
})
+ else:
+ # Tumbleweed defaults to Python 3.11
+
+ progs.update({
+ "python-sphinx": "python313-Sphinx",
+ "virtualenv": "python313-virtualenv",
+ "yaml": "python313-PyYAML",
+ })
# FIXME: add support for installing CJK fonts
#
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 12/39] scripts: sphinx-pre-install: only show portage hints once
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (10 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 11/39] scripts: sphinx-pre-install: Address issues with OpenSUSE Tumbleweed Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 13/39] scripts: sphinx-pre-install: cleanup rhel support Mauro Carvalho Chehab
` (27 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
On gentoo, doesn't repeat instructions about how to enable
portage.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 16 +++++++++++-----
1 file changed, 11 insertions(+), 5 deletions(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index 0e165ad05fdb..808d31bfa790 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -89,6 +89,9 @@ class SphinxDependencyChecker:
self.python_cmd = ""
self.activate_cmd = ""
+ # Certain hints are meant to be shown only once
+ self.first_hint = True
+
self.min_version = (0, 0, 0)
self.cur_version = (0, 0, 0)
self.latest_avail_ver = (0, 0, 0)
@@ -714,11 +717,14 @@ class SphinxDependencyChecker:
"media-gfx/graphviz",
]
- for p in portages:
- result = self.run(["grep", p, "/etc/portage/package.use/*"],
- stdout=subprocess.PIPE, text=True)
- if not result.stdout.strip():
- print(f"\tsudo emerge -av1 {p}")
+ if self.first_hint:
+ for p in portages:
+ result = self.run(["grep", p, "/etc/portage/package.use/*"],
+ stdout=subprocess.PIPE, text=True)
+ if not result.stdout.strip():
+ print(f"\tsudo emerge -av1 {p}")
+
+ self.first_hint = False
print(f"\tsudo emerge --ask {self.install}")
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 13/39] scripts: sphinx-pre-install: cleanup rhel support
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (11 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 12/39] scripts: sphinx-pre-install: only show portage hints once Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 14/39] scripts: sphinx-pre-install: output Python and docutils version Mauro Carvalho Chehab
` (26 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Rhel < 8.0 is not supported anymore. Drop support for it.
Rhel 8 is problematic: at least on the tests I did with a
docker repo, it didn't work, but it could be due to the issue
that it is actually different than a real One.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 26 +++++++++++++++++++++-----
1 file changed, 21 insertions(+), 5 deletions(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index 808d31bfa790..42f55e67256d 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -487,15 +487,31 @@ class SphinxDependencyChecker:
progs["virtualenv"] = "python-virtualenv"
- if rel and rel < 8:
+ if not rel or rel < 8:
old = 1
self.pdf = False
- # RHEL 7 is in ELS, currently up to Jun, 2026
+ print("ERROR: Distro not supported. Too old?")
+ return
+
+ # TODO: check if RHEL8 still works.
+ # On my tests with docker "redhat/ubi8" image, there's no
+ # python3-sphinx (or similar) package. It comes with Python 3.6,
+ # but there are other python packages over there, so it may be
+ # possible to work with venv.
+
+ if self.first_hint:
+ print("Note: RHEL-based distros typically require extra repositories.\n" \
+ "For most, enabling epel and crb are enough:\n" \
+ "\tsudo dnf install -y epel-release", \
+ "\tsudo dnf config-manager --set-enabled crb\n" \
+ "Yet, some may have other required repositories. Those commands could be useful:" \
+ "\tsudo dnf repolist all" \
+ "\tsudo repoquery --available --info <pkgs>",
+ "\tsudo dnf config-manager --set-enabled '*' # enable all - probably not what you want")
+
+ self.first_hint = False
- print("Note: texlive packages on RHEL/CENTOS <= 7 are incomplete. Can't support PDF output")
- print("If you want to build PDF, please read:")
- print("\thttps://www.systutorials.com/241660/how-to-install-tex-live-on-centos-7-linux/")
if self.pdf:
pdf_pkgs = [
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 14/39] scripts: sphinx-pre-install: output Python and docutils version
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (12 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 13/39] scripts: sphinx-pre-install: cleanup rhel support Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 15/39] scripts: sphinx-pre-install: add a missing f-string marker Mauro Carvalho Chehab
` (25 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Specially when debugging issues, knowing the versions is
important. Add it to the script output.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index 42f55e67256d..de5bcfd052b5 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -154,6 +154,20 @@ class SphinxDependencyChecker:
cur_ver = sys.version_info[:3]
if cur_ver >= MIN_PYTHON_VERSION:
+ ver = ver_str(cur_ver)
+ print(f"Python version: {ver}")
+
+ # This could be useful for debugging purposes
+ if SphinxDependencyChecker.which("docutils"):
+ result = SphinxDependencyChecker.run(["docutils", "--version"],
+ capture_output=True, text=True)
+ ver = result.stdout.strip()
+ match = re.search(r"(\d+\.\d+\.\d+)", ver)
+ if match:
+ ver = match.group(1)
+
+ print(f"Docutils version: {ver}")
+
return
python_ver = ver_str(cur_ver)
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 15/39] scripts: sphinx-pre-install: add a missing f-string marker
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (13 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 14/39] scripts: sphinx-pre-install: output Python and docutils version Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 16/39] scripts: sphinx-pre-install: fix Leap support for rsvg-convert Mauro Carvalho Chehab
` (24 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
I forgot one f-string marker, with turned to be affecting 3
lines, because of cut-and-paste ;-)
Use the proper f-string marker to print Sphinx version at
the hint lines. Yet, we don't want to print as a tuple, so
call ver_str() for it.
Ideally, we would be placing it directly at the f-string, but
Python 3.6 f-string support was pretty much limited. Only
3.12 (PEP 701) makes it similar to Perl, allowing expressions
inside it. It sounds that function call itself was introduced
on 3.7.
As we explicitly want this one to run on 3.6, as latest Leap
comes with it, we can't use function calls on f-string.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index de5bcfd052b5..6a244105f7ef 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -914,12 +914,15 @@ class SphinxDependencyChecker:
if self.cur_version != (0, 0, 0) and self.cur_version >= RECOMMENDED_VERSION:
return
+ if self.latest_avail_ver:
+ latest_avail_ver = ver_str(self.latest_avail_ver)
+
if not self.need_sphinx:
# sphinx-build is present and its version is >= $min_version
# only recommend enabling a newer virtenv version if makes sense.
if self.latest_avail_ver and self.latest_avail_ver > self.cur_version:
- print("\nYou may also use the newer Sphinx version {self.latest_avail_ver} with:")
+ print(f"\nYou may also use the newer Sphinx version {latest_avail_ver} with:")
if f"{self.virtenv_prefix}" in os.getcwd():
print("\tdeactivate")
print(f"\t. {self.activate_cmd}")
@@ -940,7 +943,7 @@ class SphinxDependencyChecker:
# installed one via virtenv with a newer version.
# So, print commands to enable it
if self.latest_avail_ver > self.cur_version:
- print("\nYou may also use the Sphinx virtualenv version {self.latest_avail_ver} with:")
+ print(f"\nYou may also use the Sphinx virtualenv version {latest_avail_ver} with:")
if f"{self.virtenv_prefix}" in os.getcwd():
print("\tdeactivate")
print(f"\t. {self.activate_cmd}")
@@ -954,7 +957,7 @@ class SphinxDependencyChecker:
# Suggest newer versions if current ones are too old
if self.latest_avail_ver and self.latest_avail_ver >= self.min_version:
if self.latest_avail_ver >= RECOMMENDED_VERSION:
- print("\nNeed to activate Sphinx (version {self.latest_avail_ver}) on virtualenv with:")
+ print(f"\nNeed to activate Sphinx (version {latest_avail_ver}) on virtualenv with:")
print(f"\t. {self.activate_cmd}")
self.deactivate_help()
return
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 16/39] scripts: sphinx-pre-install: fix Leap support for rsvg-convert
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (14 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 15/39] scripts: sphinx-pre-install: add a missing f-string marker Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 17/39] scripts: sphinx-pre-install: fix rhel recomendations Mauro Carvalho Chehab
` (23 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
There is a test logic meant to be for Leap, renaming rsvg-convert
package. Well, at least on latest Leap releases, this is wrong,
causing install to fail. Drop it.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 4 ----
1 file changed, 4 deletions(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index 6a244105f7ef..6b5f9eda91bb 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -566,10 +566,6 @@ class SphinxDependencyChecker:
"yaml": "python3-pyyaml",
}
- # On Tumbleweed, this package is also named rsvg-convert
- if not re.search(r"Tumbleweed", self.system_release):
- progs["rsvg-convert"] = "rsvg-view"
-
suse_tex_pkgs = [
"texlive-babel-english",
"texlive-caption",
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 17/39] scripts: sphinx-pre-install: fix rhel recomendations
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (15 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 16/39] scripts: sphinx-pre-install: fix Leap support for rsvg-convert Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 18/39] scripts: sphinx-pre-install: remove Scientific Linux Mauro Carvalho Chehab
` (22 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index 6b5f9eda91bb..1f64909953b9 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -519,9 +519,9 @@ class SphinxDependencyChecker:
"For most, enabling epel and crb are enough:\n" \
"\tsudo dnf install -y epel-release", \
"\tsudo dnf config-manager --set-enabled crb\n" \
- "Yet, some may have other required repositories. Those commands could be useful:" \
- "\tsudo dnf repolist all" \
- "\tsudo repoquery --available --info <pkgs>",
+ "Yet, some may have other required repositories. Those commands could be useful:\n" \
+ "\tsudo dnf repolist all\n" \
+ "\tsudo dnf repoquery --available --info <pkgs>\n",
"\tsudo dnf config-manager --set-enabled '*' # enable all - probably not what you want")
self.first_hint = False
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 18/39] scripts: sphinx-pre-install: remove Scientific Linux
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (16 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 17/39] scripts: sphinx-pre-install: fix rhel recomendations Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 19/39] scripts: sphinx-pre-install: improve Gentoo package deps logic Mauro Carvalho Chehab
` (21 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
According with its website: https://scientificlinux.org/
Scientific Linux reached end of life in June 30, 2024.
Also, it was based on RHEL 7, which is not compatible with
our build system anymore.
So, drop support for it.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 1 -
1 file changed, 1 deletion(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index 1f64909953b9..cd9c83b17971 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -769,7 +769,6 @@ class SphinxDependencyChecker:
re.compile("openEuler"): self.give_redhat_hints,
re.compile("Oracle Linux Server"): self.give_redhat_hints,
re.compile("Rocky Linux"): self.give_redhat_hints,
- re.compile("Scientific Linux"): self.give_redhat_hints,
re.compile("Springdale Open Enterprise"): self.give_redhat_hints,
re.compile("Ubuntu"): self.give_debian_hints,
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 19/39] scripts: sphinx-pre-install: improve Gentoo package deps logic
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (17 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 18/39] scripts: sphinx-pre-install: remove Scientific Linux Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 20/39] scripts: sphinx-pre-install: fix OpenMandriva support Mauro Carvalho Chehab
` (20 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
It took me a lot of time, but I guess understand now what it
takes to install a package on Gentoo.
Handling dependencies is a nightmare, as Gentoo refuses to emerge
some packages if there's no package.use file describing them.
To make it worse, compilation flags shall also be present there
for some packages. If USE is not perfect, error/warning messages
like those are shown:
gnome-base/librsvg dev-texlive/texlive-xetex media-fonts/dejavu dev-python/pyyaml
...
!!! The following binary packages have been ignored due to non matching USE:
=media-gfx/graphviz-12.2.1-r1 X pdf -python_single_target_python3_13 qt6 svg
=media-gfx/graphviz-12.2.1-r1 X pdf python_single_target_python3_12 -python_single_target_python3_13 qt6 svg
=media-gfx/graphviz-12.2.1-r1 X pdf qt6 svg
=media-gfx/graphviz-12.2.1-r1 X pdf -python_single_target_python3_10 qt6 svg
=media-gfx/graphviz-12.2.1-r1 X pdf -python_single_target_python3_10 python_single_target_python3_12 -python_single_target_python3_13 qt6 svg
=media-fonts/noto-cjk-20190416 X
=app-text/texlive-core-2024-r1 X cjk -xetex
=app-text/texlive-core-2024-r1 X -xetex
=app-text/texlive-core-2024-r1 -xetex
=dev-libs/zziplib-0.13.79-r1 sdl
If emerge is allowed, it will simply ignore the above packages,
creating an incomplete installation, which will later fail when
one tries to build docs with images or build PDFs.
After the fix, command line commands to produce the needed USE
chain will be emitted, if they don't exist yet.
sudo su -c 'echo "media-gfx/graphviz" > /etc/portage/package.use/graphviz'
sudo su -c 'echo "media-gfx/imagemagick" > /etc/portage/package.use/imagemagick'
sudo su -c 'echo "media-libs/harfbuzz icu" > /etc/portage/package.use/media-libs'
sudo su -c 'echo "media-fonts/noto-cjk" > /etc/portage/package.use/media-fonts'
sudo su -c 'echo "app-text/texlive-core xetex" > /etc/portage/package.use/texlive'
sudo su -c 'echo "dev-libs/zziplib sdl" > /etc/portage/package.use/zziblib'
The new logic tries to be smart enough to detect for missing files
and missing arguments. Yet, as Gentoo seems to require users to
manage those package.use files by hand, the logic isn't perfect:
users may still need to verify for conflicts on different use
files.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 86 +++++++++++++++++++++++++++++++----
1 file changed, 76 insertions(+), 10 deletions(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index cd9c83b17971..94f3d2e32fd6 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -737,22 +737,88 @@ class SphinxDependencyChecker:
print("You should run:")
print("\n")
+ # Handling dependencies is a nightmare, as Gentoo refuses to emerge
+ # some packages if there's no package.use file describing them.
+ # To make it worse, compilation flags shall also be present there
+ # for some packages. If USE is not perfect, error/warning messages
+ # like those are shown:
+ #
+ # !!! The following binary packages have been ignored due to non matching USE:
+ #
+ # =media-gfx/graphviz-12.2.1-r1 X pdf -python_single_target_python3_13 qt6 svg
+ # =media-gfx/graphviz-12.2.1-r1 X pdf python_single_target_python3_12 -python_single_target_python3_13 qt6 svg
+ # =media-gfx/graphviz-12.2.1-r1 X pdf qt6 svg
+ # =media-gfx/graphviz-12.2.1-r1 X pdf -python_single_target_python3_10 qt6 svg
+ # =media-gfx/graphviz-12.2.1-r1 X pdf -python_single_target_python3_10 python_single_target_python3_12 -python_single_target_python3_13 qt6 svg
+ # =media-fonts/noto-cjk-20190416 X
+ # =app-text/texlive-core-2024-r1 X cjk -xetex
+ # =app-text/texlive-core-2024-r1 X -xetex
+ # =app-text/texlive-core-2024-r1 -xetex
+ # =dev-libs/zziplib-0.13.79-r1 sdl
+ #
+ # And will ignore such packages, installing the remaining ones. That
+ # affects mostly the image extension and PDF generation.
- portages = [
- "media-gfx/imagemagick",
- "media-gfx/graphviz",
- ]
+ # Package dependencies and the minimal needed args:
+ portages = {
+ "graphviz": "media-gfx/graphviz",
+ "imagemagick": "media-gfx/imagemagick",
+ "media-libs": "media-libs/harfbuzz icu",
+ "media-fonts": "media-fonts/noto-cjk",
+ "texlive": "app-text/texlive-core xetex",
+ "zziblib": "dev-libs/zziplib sdl",
+ }
if self.first_hint:
- for p in portages:
- result = self.run(["grep", p, "/etc/portage/package.use/*"],
- stdout=subprocess.PIPE, text=True)
- if not result.stdout.strip():
- print(f"\tsudo emerge -av1 {p}")
+ use_base = "/etc/portage/package.use"
+ files = glob(f"{use_base}/*")
+
+ for fname, portage in portages.items():
+ install = False
+
+ while install == False:
+ if not files:
+ # No files under package.usage. Install all
+ install = True
+ break
+
+ args = portage.split(" ")
+
+ name = args.pop(0)
+
+ cmd = ["grep", "-l", "-E", rf"^{name}\b" ] + files
+ result = self.run(cmd, stdout=subprocess.PIPE, text=True)
+ if result.returncode or not result.stdout.strip():
+ # File containing portage name not found
+ install = True
+ break
+
+ # Ensure that needed USE flags are present
+ if args:
+ match_fname = result.stdout.strip()
+ with open(match_fname, 'r', encoding='utf8',
+ errors='backslashreplace') as fp:
+ for line in fp:
+ for arg in args:
+ if arg.startswith("-"):
+ continue
+
+ if not re.search(rf"\s*{arg}\b", line):
+ # Needed file argument not found
+ install = True
+ break
+
+ # Everything looks ok, don't install
+ break
+
+ # emit a code to setup missing USE
+ if install:
+ print(f"\tsudo su -c 'echo \"{portage}\" > {use_base}/{fname}'")
self.first_hint = False
- print(f"\tsudo emerge --ask {self.install}")
+ # Now, we can use emerge and let it respect USE
+ print(f"\tsudo emerge --ask --changed-use --binpkg-respect-use=y {self.install}")
#
# Dispatch the check to an os_specific hinter
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 20/39] scripts: sphinx-pre-install: fix OpenMandriva support
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (18 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 19/39] scripts: sphinx-pre-install: improve Gentoo package deps logic Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 21/39] scripts: sphinx-pre-install: move package instructions to a new func Mauro Carvalho Chehab
` (19 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
OpenMandriva Lx 4.3 has different package names for ImageMagick
and yaml. Fix them to ensure that system setup will pass.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index 94f3d2e32fd6..2f6036eadc94 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -647,6 +647,11 @@ class SphinxDependencyChecker:
packager_cmd = "dnf install"
noto_sans = "noto-sans-cjk-fonts"
tex_pkgs = ["texlive-collection-fontsextra"]
+
+ # Tested on OpenMandriva Lx 4.3
+ progs["convert"] = "imagemagick"
+ progs["yaml"] = "python-pyyaml"
+
else:
packager_cmd = "urpmi"
noto_sans = "google-noto-sans-cjk-ttc-fonts"
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 21/39] scripts: sphinx-pre-install: move package instructions to a new func
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (19 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 20/39] scripts: sphinx-pre-install: fix OpenMandriva support Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 22/39] scripts: sphinx-pre-install: adjust a warning message Mauro Carvalho Chehab
` (18 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Currently, if Python < 3.7, package install will fail. That happens
with OpenSuse Leap and RHEL-based ver 8 distros.
OpenSuse allows installing Sphinx with Python 3.11, but RHEL-based
distros don't.
Prepare to recomend only venv on such cases. For now, just split
the recomendation on a new function that will check for a
paramtere to be called.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 44 +++++++++++++++++++++--------------
1 file changed, 26 insertions(+), 18 deletions(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index 2f6036eadc94..2360ca2ed21c 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -89,6 +89,9 @@ class SphinxDependencyChecker:
self.python_cmd = ""
self.activate_cmd = ""
+ # Some distros may not have a Sphinx shipped package compatible with
+ # our minimal requirements
+ self.package_supported = True
# Certain hints are meant to be shown only once
self.first_hint = True
@@ -969,6 +972,27 @@ class SphinxDependencyChecker:
return self.latest_avail_ver
+ def recommend_package(self):
+
+ print("\n2) As a package with:")
+
+ old_need = self.need
+ old_optional = self.optional
+ self.missing = {}
+ self.pdf = False
+ self.optional = 0
+ self.install = ""
+ old_verbose = self.verbose_warn_install
+ self.verbose_warn_install = 0
+
+ self.add_package("python-sphinx", 0)
+
+ self.check_distros()
+
+ self.need = old_need
+ self.optional = old_optional
+ self.verbose_warn_install = old_verbose
+
def recommend_sphinx_version(self, virtualenv_cmd):
# The logic here is complex, as it have to deal with different versions:
# - minimal supported version;
@@ -1053,24 +1077,8 @@ class SphinxDependencyChecker:
print(f"\tpip install -r {self.requirement_file}")
self.deactivate_help()
- print("\n2) As a package with:")
-
- old_need = self.need
- old_optional = self.optional
- self.missing = {}
- self.pdf = False
- self.optional = 0
- self.install = ""
- old_verbose = self.verbose_warn_install
- self.verbose_warn_install = 0
-
- self.add_package("python-sphinx", 0)
-
- self.check_distros()
-
- self.need = old_need
- self.optional = old_optional
- self.verbose_warn_install = old_verbose
+ if self.package_supported:
+ self.recommend_package()
print("\n" \
" Please note that Sphinx >= 3.0 will currently produce false-positive\n" \
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 22/39] scripts: sphinx-pre-install: adjust a warning message
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (20 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 21/39] scripts: sphinx-pre-install: move package instructions to a new func Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 23/39] scripts: sphinx-pre-install: better handle Python min version Mauro Carvalho Chehab
` (17 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
There is one extra space at the first line. Also, as now we only
support Python 3.4+, update the text.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index 2360ca2ed21c..365590f81551 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -1081,8 +1081,8 @@ class SphinxDependencyChecker:
self.recommend_package()
print("\n" \
- " Please note that Sphinx >= 3.0 will currently produce false-positive\n" \
- " warning when the same name is used for more than one type (functions,\n" \
+ " Please note that Sphinx currentlys produce false-positive\n" \
+ " warnings when the same name is used for more than one type (functions,\n" \
" structs, enums,...). This is known Sphinx bug. For more details, see:\n" \
"\thttps://github.com/sphinx-doc/sphinx/pull/8313")
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 23/39] scripts: sphinx-pre-install: better handle Python min version
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (21 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 22/39] scripts: sphinx-pre-install: adjust a warning message Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 24/39] scripts: sphinx-pre-install: convert is_optional to a class Mauro Carvalho Chehab
` (16 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Don't do any recommendations about Sphinx install with too
old python versions.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 20 +++++++++++++++-----
1 file changed, 15 insertions(+), 5 deletions(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index 365590f81551..a5c777e529ec 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -92,6 +92,10 @@ class SphinxDependencyChecker:
# Some distros may not have a Sphinx shipped package compatible with
# our minimal requirements
self.package_supported = True
+
+ # Recommend a new python version
+ self.recommend_python = None
+
# Certain hints are meant to be shown only once
self.first_hint = True
@@ -511,11 +515,11 @@ class SphinxDependencyChecker:
print("ERROR: Distro not supported. Too old?")
return
- # TODO: check if RHEL8 still works.
- # On my tests with docker "redhat/ubi8" image, there's no
- # python3-sphinx (or similar) package. It comes with Python 3.6,
- # but there are other python packages over there, so it may be
- # possible to work with venv.
+ # RHEL 8 uses Python 3.6, which is not compatible with
+ # the build system anymore. Suggest Python 3.11
+ if rel == 8:
+ self.add_package("python39", 0)
+ self.recommend_python = True
if self.first_hint:
print("Note: RHEL-based distros typically require extra repositories.\n" \
@@ -596,6 +600,7 @@ class SphinxDependencyChecker:
# the build system anymore. Suggest Python 3.11
if rel == 15:
if not self.which(self.python_cmd):
+ self.recommend_python = True
self.add_package(self.python_cmd, 0)
progs.update({
@@ -1000,6 +1005,11 @@ class SphinxDependencyChecker:
# - recommended version.
# It also needs to work fine with both distro's package and venv/virtualenv
+ if self.recommend_python:
+ print("\nPython version is incompatible with doc build.\n" \
+ "Please upgrade it and re-run.\n")
+ return
+
# Version is OK. Nothing to do.
if self.cur_version != (0, 0, 0) and self.cur_version >= RECOMMENDED_VERSION:
return
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 24/39] scripts: sphinx-pre-install: convert is_optional to a class
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (22 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 23/39] scripts: sphinx-pre-install: better handle Python min version Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 25/39] scripts: sphinx-pre-install: better handle RHEL-based distros Mauro Carvalho Chehab
` (15 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
When is_optional was added in Perl, it was a boolean. With
time, it ended becoming a sort of enum, which makes the
module harder to maintain.
Convert it to a enum-like class and add more options to it.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 192 +++++++++++++++++++++++-----------
1 file changed, 130 insertions(+), 62 deletions(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index a5c777e529ec..0963da21c27b 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -32,6 +32,53 @@ RECOMMENDED_VERSION = parse_version("3.4.3")
MIN_PYTHON_VERSION = parse_version("3.7")
+class DepType:
+
+ # Internal types of dependencies.
+ _SYS_TYPE = 0
+ _PHY_TYPE = 1
+ _PDF_TYPE = 2
+
+ # Let's define keys as a tuple with the type and mandatory/optional.
+ # This way, checking for optional or type is easy.
+
+ SYSTEM_MANDATORY = (_SYS_TYPE, True)
+ PYTHON_MANDATORY = (_PHY_TYPE, True)
+ PDF_MANDATORY = (_PDF_TYPE, True)
+
+ # Currently we're not using all optional types, but let's keep all
+ # combinations here, as we may end needing them in the future. Also,
+ # it allows a name() function that handles all possibilities.
+ SYSTEM_OPTIONAL = (_SYS_TYPE, False)
+ PYTHON_OPTIONAL = (_PHY_TYPE, False)
+ PDF_OPTIONAL = (_PDF_TYPE, True)
+
+ @staticmethod
+ def name(dtype):
+ if dtype[0] == DepType._SYS_TYPE:
+ msg = "build"
+ elif dtype[0] == DepType._PHY_TYPE:
+ msg = "Python"
+ else:
+ msg = "PDF"
+
+ if dtype[1]:
+ return f"ERROR: {msg} mandatory deps missing"
+ else:
+ out = f"Warning: {msg} optional deps missing"
+
+ @staticmethod
+ def is_optional(dtype):
+ return not dtype[1]
+
+ @staticmethod
+ def is_pdf(dtype):
+ if (dtype[0] == DepType._PDF_TYPE):
+ return True
+
+ return False
+
+
class SphinxDependencyChecker:
# List of required texlive packages on Fedora and OpenSuse
texlive = {
@@ -223,56 +270,68 @@ class SphinxDependencyChecker:
# Methods to check if a feature exists
#
- # Note: is_optional has 3 states:
- # - 0: mandatory
- # - 1: optional, but nice to have
- # - 2: LaTeX optional - pdf builds without it, but may have visual impact
-
def check_missing(self, progs):
- for prog, is_optional in sorted(self.missing.items()):
+ run = {}
+
+ for prog, dtype in sorted(self.missing.items()):
# At least on some LTS distros like CentOS 7, texlive doesn't
# provide all packages we need. When such distros are
# detected, we have to disable PDF output.
#
# So, we need to ignore the packages that distros would
# need for LaTeX to work
- if is_optional == 2 and not self.pdf:
+ if DepType.is_pdf(dtype) and not self.pdf:
self.optional -= 1
continue
+ if not dtype in run:
+ run[dtype] = []
+
+ run[dtype].append(prog)
+
+ output_msg = ""
+
+ for dtype in sorted(run.keys()):
+ progs = " ".join(run[dtype])
+
if self.verbose_warn_install:
- if is_optional:
- print(f'Warning: better to also install "{prog}".')
- else:
- print(f'ERROR: please install "{prog}", otherwise, build won\'t work.')
+ try:
+ name = DepType.name(dtype)
+ output_msg += f'{name}:\t{progs}\n'
+ except KeyError:
+ raise KeyError(f"ERROR!!!: invalid dtype for {progs}: {dtype}")
- self.install += " " + progs.get(prog, prog)
+ self.install += " " + progs
+
+ if output_msg:
+ print(f"\n{output_msg}\n")
self.install = self.install.lstrip()
- def add_package(self, package, is_optional):
- self.missing[package] = is_optional
+ def add_package(self, package, dtype):
+ is_optional = DepType.is_optional(dtype)
+ self.missing[package] = dtype
if is_optional:
self.optional += 1
else:
self.need += 1
- def check_missing_file(self, files, package, is_optional):
+ def check_missing_file(self, files, package, dtype):
for f in files:
if os.path.exists(f):
return
- self.add_package(package, is_optional)
+ self.add_package(package, dtype)
- def check_program(self, prog, is_optional):
+ def check_program(self, prog, dtype):
found = self.which(prog)
if found:
return found
- self.add_package(prog, is_optional)
+ self.add_package(prog, dtype)
return None
- def check_perl_module(self, prog, is_optional):
+ def check_perl_module(self, prog, dtype):
# While testing with lxc download template, one of the
# distros (Oracle) didn't have perl - nor even an option to install
# before installing oraclelinux-release-el9 package.
@@ -281,46 +340,52 @@ class SphinxDependencyChecker:
# add it as a mandatory package, as some parts of the doc builder
# needs it.
if not self.which("perl"):
- self.add_package("perl", 0)
- self.add_package(prog, is_optional)
+ self.add_package("perl", DepType.SYSTEM_MANDATORY)
+ self.add_package(prog, dtype)
return
try:
self.run(["perl", f"-M{prog}", "-e", "1"], check=True)
except subprocess.CalledProcessError:
- self.add_package(prog, is_optional)
+ self.add_package(prog, dtype)
- def check_python_module(self, module, is_optional):
- # FIXME: is it needed at the Python version? Maybe due to venv?
- if not self.python_cmd:
- return
+ def check_python_module(self, module, is_optional=False):
+ if is_optional:
+ dtype = DepType.PYTHON_OPTIONAL
+ else:
+ dtype = DepType.PYTHON_MANDATORY
try:
self.run([self.python_cmd, "-c", f"import {module}"], check=True)
except subprocess.CalledProcessError:
- self.add_package(module, is_optional)
+ self.add_package(module, dtype)
- def check_rpm_missing(self, pkgs, is_optional):
+ def check_rpm_missing(self, pkgs, dtype):
for prog in pkgs:
try:
self.run(["rpm", "-q", prog], check=True)
except subprocess.CalledProcessError:
- self.add_package(prog, is_optional)
+ self.add_package(prog, dtype)
- def check_pacman_missing(self, pkgs, is_optional):
+ def check_pacman_missing(self, pkgs, dtype):
for prog in pkgs:
try:
self.run(["pacman", "-Q", prog], check=True)
except subprocess.CalledProcessError:
- self.add_package(prog, is_optional)
+ self.add_package(prog, dtype)
+
+ def check_missing_tex(self, is_optional=False):
+ if is_optional:
+ dtype = DepType.PDF_OPTIONAL
+ else:
+ dtype = DepType.PDF_MANDATORY
- def check_missing_tex(self, is_optional):
kpsewhich = self.which("kpsewhich")
for prog, package in self.texlive.items():
# If kpsewhich is not there, just add it to deps
if not kpsewhich:
- self.add_package(package, is_optional)
+ self.add_package(package, dtype)
continue
# Check if the package is needed
@@ -331,11 +396,11 @@ class SphinxDependencyChecker:
# Didn't find. Add it
if not result.stdout.strip():
- self.add_package(package, is_optional)
+ self.add_package(package, dtype)
except subprocess.CalledProcessError:
# kpsewhich returned an error. Add it, just in case
- self.add_package(package, is_optional)
+ self.add_package(package, dtype)
def get_sphinx_fname(self):
if "SPHINXBUILD" in os.environ:
@@ -446,9 +511,9 @@ class SphinxDependencyChecker:
}
for package, files in pdf_pkgs.items():
- self.check_missing_file(files, package, 2)
+ self.check_missing_file(files, package, DepType.PDF_MANDATORY)
- self.check_program("dvipng", 2)
+ self.check_program("dvipng", DepType.PDF_MANDATORY)
self.check_missing(progs)
@@ -518,7 +583,7 @@ class SphinxDependencyChecker:
# RHEL 8 uses Python 3.6, which is not compatible with
# the build system anymore. Suggest Python 3.11
if rel == 8:
- self.add_package("python39", 0)
+ self.add_package("python39", DepType.SYSTEM_MANDATORY)
self.recommend_python = True
if self.first_hint:
@@ -540,13 +605,13 @@ class SphinxDependencyChecker:
"/usr/share/fonts/google-noto-sans-cjk-fonts/NotoSansCJK-Regular.ttc",
]
- self.check_missing_file(pdf_pkgs, noto_sans_redhat, 2)
+ self.check_missing_file(pdf_pkgs, noto_sans_redhat, DepType.PDF_MANDATORY)
if not old:
- self.check_rpm_missing(fedora26_opt_pkgs, 2)
- self.check_rpm_missing(fedora_tex_pkgs, 2)
+ self.check_rpm_missing(fedora26_opt_pkgs, DepType.PDF_MANDATORY)
+ self.check_rpm_missing(fedora_tex_pkgs, DepType.PDF_MANDATORY)
- self.check_missing_tex(2)
+ self.check_missing_tex()
self.check_missing(progs)
@@ -601,7 +666,7 @@ class SphinxDependencyChecker:
if rel == 15:
if not self.which(self.python_cmd):
self.recommend_python = True
- self.add_package(self.python_cmd, 0)
+ self.add_package(self.python_cmd, DepType.SYSTEM_MANDATORY)
progs.update({
"python-sphinx": "python311-Sphinx",
@@ -623,9 +688,9 @@ class SphinxDependencyChecker:
# "Noto Sans CJK SC" on openSUSE
if self.pdf:
- self.check_rpm_missing(suse_tex_pkgs, 2)
+ self.check_rpm_missing(suse_tex_pkgs, DepType.PDF_MANDATORY)
if self.pdf:
- self.check_missing_tex(2)
+ self.check_missing_tex()
self.check_missing(progs)
if not self.need and not self.optional:
@@ -672,8 +737,8 @@ class SphinxDependencyChecker:
"/usr/share/fonts/TTF/NotoSans-Regular.ttf",
]
- self.check_missing_file(pdf_pkgs, noto_sans, 2)
- self.check_rpm_missing(tex_pkgs, 2)
+ self.check_missing_file(pdf_pkgs, noto_sans, DepType.PDF_MANDATORY)
+ self.check_rpm_missing(tex_pkgs, DepType.PDF_MANDATORY)
self.check_missing(progs)
@@ -701,7 +766,7 @@ class SphinxDependencyChecker:
]
if self.pdf:
- self.check_pacman_missing(archlinux_tex_pkgs, 2)
+ self.check_pacman_missing(archlinux_tex_pkgs, DepType.PDF_MANDATORY)
self.check_missing_file(
["/usr/share/fonts/noto-cjk/NotoSansCJK-Regular.ttc"],
@@ -739,7 +804,7 @@ class SphinxDependencyChecker:
],
}
for package, files in pdf_pkgs.items():
- self.check_missing_file(files, package, 2)
+ self.check_missing_file(files, package, DepType.PDF_MANDATORY)
self.check_missing(progs)
@@ -878,7 +943,7 @@ class SphinxDependencyChecker:
#
progs = {"sphinx-build": "sphinx"}
if self.pdf:
- self.check_missing_tex(2)
+ self.check_missing_tex()
self.check_missing(progs)
@@ -990,7 +1055,7 @@ class SphinxDependencyChecker:
old_verbose = self.verbose_warn_install
self.verbose_warn_install = 0
- self.add_package("python-sphinx", 0)
+ self.add_package("python-sphinx", DepType.PYTHON_MANDATORY)
self.check_distros()
@@ -1010,6 +1075,7 @@ class SphinxDependencyChecker:
"Please upgrade it and re-run.\n")
return
+
# Version is OK. Nothing to do.
if self.cur_version != (0, 0, 0) and self.cur_version >= RECOMMENDED_VERSION:
return
@@ -1128,21 +1194,23 @@ class SphinxDependencyChecker:
else:
virtualenv_cmd = f"{self.python_cmd} -m venv"
- self.check_python_module("ensurepip", 0)
+ self.check_python_module("ensurepip")
# Check for needed programs/tools
- self.check_perl_module("Pod::Usage", 0)
- self.check_python_module("yaml", 0)
- self.check_program("make", 0)
- self.check_program("gcc", 0)
- self.check_program("dot", 1)
- self.check_program("convert", 1)
+ self.check_perl_module("Pod::Usage", DepType.SYSTEM_MANDATORY)
+
+ self.check_program("make", DepType.SYSTEM_MANDATORY)
+ self.check_program("gcc", DepType.SYSTEM_MANDATORY)
+
+ self.check_program("dot", DepType.SYSTEM_OPTIONAL)
+ self.check_program("convert", DepType.SYSTEM_OPTIONAL)
+
+ self.check_python_module("yaml")
if self.pdf:
- # Extra PDF files - should use 2 for LaTeX is_optional
- self.check_program("xelatex", 2)
- self.check_program("rsvg-convert", 2)
- self.check_program("latexmk", 2)
+ self.check_program("xelatex", DepType.PDF_MANDATORY)
+ self.check_program("rsvg-convert", DepType.PDF_MANDATORY)
+ self.check_program("latexmk", DepType.PDF_MANDATORY)
# Do distro-specific checks and output distro-install commands
self.check_distros()
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 25/39] scripts: sphinx-pre-install: better handle RHEL-based distros
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (23 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 24/39] scripts: sphinx-pre-install: convert is_optional to a class Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 26/39] scripts: sphinx-pre-install: move missing logic to a separate class Mauro Carvalho Chehab
` (14 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Better implement support for RHEL-based distros. While here,
get rid of a Fedora 28 support which cause troubles with
server distros. Also, get rid of yum, as RHEL8 already
suppords dnf, and this is not the minimal version we may
still support.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 31 ++++++++++++-------------------
1 file changed, 12 insertions(+), 19 deletions(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index 0963da21c27b..592223fa686f 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -536,10 +536,6 @@ class SphinxDependencyChecker:
"yaml": "python3-pyyaml",
}
- fedora26_opt_pkgs = [
- "graphviz-gd", # Fedora 26: needed for PDF support
- ]
-
fedora_tex_pkgs = [
"dejavu-sans-fonts",
"dejavu-sans-mono-fonts",
@@ -549,9 +545,8 @@ class SphinxDependencyChecker:
"texlive-xecjk",
]
- old = 0
+ fedora = False
rel = None
- pkg_manager = "dnf"
match = re.search(r"(release|Linux)\s+(\d+)", self.system_release)
if match:
@@ -559,12 +554,12 @@ class SphinxDependencyChecker:
if not rel:
print("Couldn't identify release number")
- old = 1
self.pdf = False
elif re.search("Fedora", self.system_release):
# Fedora 38 and upper use this CJK font
noto_sans_redhat = "google-noto-sans-cjk-fonts"
+ fedora = True
else:
# Almalinux, CentOS, RHEL, ...
@@ -574,9 +569,6 @@ class SphinxDependencyChecker:
progs["virtualenv"] = "python-virtualenv"
if not rel or rel < 8:
- old = 1
- self.pdf = False
-
print("ERROR: Distro not supported. Too old?")
return
@@ -607,11 +599,16 @@ class SphinxDependencyChecker:
self.check_missing_file(pdf_pkgs, noto_sans_redhat, DepType.PDF_MANDATORY)
- if not old:
- self.check_rpm_missing(fedora26_opt_pkgs, DepType.PDF_MANDATORY)
- self.check_rpm_missing(fedora_tex_pkgs, DepType.PDF_MANDATORY)
+ self.check_rpm_missing(fedora_tex_pkgs, DepType.PDF_MANDATORY)
+
+ self.check_missing_tex(DepType.PDF_MANDATORY)
+
+ # There's no texlive-ctex on RHEL 8 repositories. This will
+ # likely affect CJK pdf build only.
+ if not fedora and rel == 8:
+ if "texlive-ctex" in self.missing:
+ del self.missing["texlive-ctex"]
- self.check_missing_tex()
self.check_missing(progs)
@@ -621,11 +618,7 @@ class SphinxDependencyChecker:
if self.verbose_warn_install:
print("You should run:")
- if old:
- # dnf is there since Fedora 18+ and RHEL 8
- pkg_manager = "yum"
-
- print(f"\n\tsudo {pkg_manager} install -y {self.install}")
+ print(f"\n\tsudo dnf install -y {self.install}")
def give_opensuse_hints(self):
progs = {
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 26/39] scripts: sphinx-pre-install: move missing logic to a separate class
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (24 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 25/39] scripts: sphinx-pre-install: better handle RHEL-based distros Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 27/39] scripts: sphinx-pre-install: move ancillary checkers " Mauro Carvalho Chehab
` (13 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Better manage dependencies by placing them on a distro-independent
class.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 251 +++++++++++++++++++---------------
1 file changed, 138 insertions(+), 113 deletions(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index 592223fa686f..47dce1fcddfb 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -32,7 +32,7 @@ RECOMMENDED_VERSION = parse_version("3.4.3")
MIN_PYTHON_VERSION = parse_version("3.7")
-class DepType:
+class DepManager:
# Internal types of dependencies.
_SYS_TYPE = 0
@@ -53,11 +53,18 @@ class DepType:
PYTHON_OPTIONAL = (_PHY_TYPE, False)
PDF_OPTIONAL = (_PDF_TYPE, True)
+ def __init__(self, pdf):
+ self.missing = {}
+ self.run = {}
+ self.need = 0
+ self.optional = 0
+ self.pdf = pdf
+
@staticmethod
def name(dtype):
- if dtype[0] == DepType._SYS_TYPE:
+ if dtype[0] == DepManager._SYS_TYPE:
msg = "build"
- elif dtype[0] == DepType._PHY_TYPE:
+ elif dtype[0] == DepManager._PHY_TYPE:
msg = "Python"
else:
msg = "PDF"
@@ -73,11 +80,75 @@ class DepType:
@staticmethod
def is_pdf(dtype):
- if (dtype[0] == DepType._PDF_TYPE):
+ if (dtype[0] == DepManager._PDF_TYPE):
return True
return False
+ def add_package(self, package, dtype):
+ is_optional = DepManager.is_optional(dtype)
+ self.missing[package] = dtype
+ if is_optional:
+ self.optional += 1
+ else:
+ self.need += 1
+
+ def del_package(self, package):
+ if package in self.missing:
+ del self.missing[package]
+
+ def clear_deps(self):
+ """
+ Clear dependencies without changing needed/optional.
+
+ This is an ackward way to have a separate section to recommend
+ a package after system main dependencies.
+
+ TODO: rework the logic to prevent needing it
+ """
+
+ self.missing = {}
+
+ def check_missing(self, progs):
+ self.run = {}
+
+ for prog, dtype in sorted(self.missing.items()):
+ # At least on some LTS distros like CentOS 7, texlive doesn't
+ # provide all packages we need. When such distros are
+ # detected, we have to disable PDF output.
+ #
+ # So, we need to ignore the packages that distros would
+ # need for LaTeX to work
+ if DepManager.is_pdf(dtype) and not self.pdf:
+ self.optional -= 1
+ continue
+
+ if not dtype in self.run:
+ self.run[dtype] = []
+
+ self.run[dtype].append(progs.get(prog, prog))
+
+ install = []
+ for dtype in self.run.keys():
+ install += self.run[dtype]
+
+ return " ".join(sorted(set(install)))
+
+ def warn_install(self):
+
+ output_msg = ""
+
+ for dtype in sorted(self.run.keys()):
+ progs = " ".join(sorted(set(self.run[dtype])))
+
+ try:
+ name = DepManager.name(dtype)
+ output_msg += f'{name}:\t{progs}\n'
+ except KeyError:
+ raise KeyError(f"ERROR!!!: invalid dtype for {progs}: {dtype}")
+
+ if output_msg:
+ print(f"\n{output_msg}\n")
class SphinxDependencyChecker:
# List of required texlive packages on Fedora and OpenSuse
@@ -120,10 +191,8 @@ class SphinxDependencyChecker:
self.virtualenv = args.virtualenv
self.version_check = args.version_check
- self.missing = {}
+ self.deps = DepManager(self.pdf)
- self.need = 0
- self.optional = 0
self.need_symlink = 0
self.need_sphinx = 0
self.need_pip = 0
@@ -270,64 +339,18 @@ class SphinxDependencyChecker:
# Methods to check if a feature exists
#
- def check_missing(self, progs):
- run = {}
-
- for prog, dtype in sorted(self.missing.items()):
- # At least on some LTS distros like CentOS 7, texlive doesn't
- # provide all packages we need. When such distros are
- # detected, we have to disable PDF output.
- #
- # So, we need to ignore the packages that distros would
- # need for LaTeX to work
- if DepType.is_pdf(dtype) and not self.pdf:
- self.optional -= 1
- continue
-
- if not dtype in run:
- run[dtype] = []
-
- run[dtype].append(prog)
-
- output_msg = ""
-
- for dtype in sorted(run.keys()):
- progs = " ".join(run[dtype])
-
- if self.verbose_warn_install:
- try:
- name = DepType.name(dtype)
- output_msg += f'{name}:\t{progs}\n'
- except KeyError:
- raise KeyError(f"ERROR!!!: invalid dtype for {progs}: {dtype}")
-
- self.install += " " + progs
-
- if output_msg:
- print(f"\n{output_msg}\n")
-
- self.install = self.install.lstrip()
-
- def add_package(self, package, dtype):
- is_optional = DepType.is_optional(dtype)
- self.missing[package] = dtype
- if is_optional:
- self.optional += 1
- else:
- self.need += 1
-
def check_missing_file(self, files, package, dtype):
for f in files:
if os.path.exists(f):
return
- self.add_package(package, dtype)
+ self.deps.add_package(package, dtype)
def check_program(self, prog, dtype):
found = self.which(prog)
if found:
return found
- self.add_package(prog, dtype)
+ self.deps.add_package(prog, dtype)
return None
@@ -340,52 +363,52 @@ class SphinxDependencyChecker:
# add it as a mandatory package, as some parts of the doc builder
# needs it.
if not self.which("perl"):
- self.add_package("perl", DepType.SYSTEM_MANDATORY)
- self.add_package(prog, dtype)
+ self.deps.add_package("perl", DepManager.SYSTEM_MANDATORY)
+ self.deps.add_package(prog, dtype)
return
try:
self.run(["perl", f"-M{prog}", "-e", "1"], check=True)
except subprocess.CalledProcessError:
- self.add_package(prog, dtype)
+ self.deps.add_package(prog, dtype)
def check_python_module(self, module, is_optional=False):
if is_optional:
- dtype = DepType.PYTHON_OPTIONAL
+ dtype = DepManager.PYTHON_OPTIONAL
else:
- dtype = DepType.PYTHON_MANDATORY
+ dtype = DepManager.PYTHON_MANDATORY
try:
self.run([self.python_cmd, "-c", f"import {module}"], check=True)
except subprocess.CalledProcessError:
- self.add_package(module, dtype)
+ self.deps.add_package(module, dtype)
def check_rpm_missing(self, pkgs, dtype):
for prog in pkgs:
try:
self.run(["rpm", "-q", prog], check=True)
except subprocess.CalledProcessError:
- self.add_package(prog, dtype)
+ self.deps.add_package(prog, dtype)
def check_pacman_missing(self, pkgs, dtype):
for prog in pkgs:
try:
self.run(["pacman", "-Q", prog], check=True)
except subprocess.CalledProcessError:
- self.add_package(prog, dtype)
+ self.deps.add_package(prog, dtype)
def check_missing_tex(self, is_optional=False):
if is_optional:
- dtype = DepType.PDF_OPTIONAL
+ dtype = DepManager.PDF_OPTIONAL
else:
- dtype = DepType.PDF_MANDATORY
+ dtype = DepManager.PDF_MANDATORY
kpsewhich = self.which("kpsewhich")
for prog, package in self.texlive.items():
# If kpsewhich is not there, just add it to deps
if not kpsewhich:
- self.add_package(package, dtype)
+ self.deps.add_package(package, dtype)
continue
# Check if the package is needed
@@ -396,11 +419,11 @@ class SphinxDependencyChecker:
# Didn't find. Add it
if not result.stdout.strip():
- self.add_package(package, dtype)
+ self.deps.add_package(package, dtype)
except subprocess.CalledProcessError:
# kpsewhich returned an error. Add it, just in case
- self.add_package(package, dtype)
+ self.deps.add_package(package, dtype)
def get_sphinx_fname(self):
if "SPHINXBUILD" in os.environ:
@@ -478,6 +501,17 @@ class SphinxDependencyChecker:
return f.read().strip()
return ""
+ def check_missing(self, progs):
+ self.install += self.deps.check_missing(progs)
+ if self.verbose_warn_install:
+ self.deps.warn_install()
+
+ if not self.deps.need and not self.deps.optional:
+ return False
+
+ return True
+
+
#
# Distro-specific hints methods
#
@@ -511,13 +545,11 @@ class SphinxDependencyChecker:
}
for package, files in pdf_pkgs.items():
- self.check_missing_file(files, package, DepType.PDF_MANDATORY)
+ self.check_missing_file(files, package, DepManager.PDF_MANDATORY)
- self.check_program("dvipng", DepType.PDF_MANDATORY)
+ self.check_program("dvipng", DepManager.PDF_MANDATORY)
- self.check_missing(progs)
-
- if not self.need and not self.optional:
+ if self.check_missing(progs):
return
if self.verbose_warn_install:
@@ -575,7 +607,7 @@ class SphinxDependencyChecker:
# RHEL 8 uses Python 3.6, which is not compatible with
# the build system anymore. Suggest Python 3.11
if rel == 8:
- self.add_package("python39", DepType.SYSTEM_MANDATORY)
+ self.deps.add_package("python39", DepManager.SYSTEM_MANDATORY)
self.recommend_python = True
if self.first_hint:
@@ -597,22 +629,18 @@ class SphinxDependencyChecker:
"/usr/share/fonts/google-noto-sans-cjk-fonts/NotoSansCJK-Regular.ttc",
]
- self.check_missing_file(pdf_pkgs, noto_sans_redhat, DepType.PDF_MANDATORY)
+ self.check_missing_file(pdf_pkgs, noto_sans_redhat, DepManager.PDF_MANDATORY)
- self.check_rpm_missing(fedora_tex_pkgs, DepType.PDF_MANDATORY)
+ self.check_rpm_missing(fedora_tex_pkgs, DepManager.PDF_MANDATORY)
- self.check_missing_tex(DepType.PDF_MANDATORY)
+ self.check_missing_tex(DepManager.PDF_MANDATORY)
# There's no texlive-ctex on RHEL 8 repositories. This will
# likely affect CJK pdf build only.
if not fedora and rel == 8:
- if "texlive-ctex" in self.missing:
- del self.missing["texlive-ctex"]
+ self.deps.del_package("texlive-ctex")
-
- self.check_missing(progs)
-
- if not self.need and not self.optional:
+ if self.check_missing(progs):
return
if self.verbose_warn_install:
@@ -659,7 +687,7 @@ class SphinxDependencyChecker:
if rel == 15:
if not self.which(self.python_cmd):
self.recommend_python = True
- self.add_package(self.python_cmd, DepType.SYSTEM_MANDATORY)
+ self.deps.add_package(self.python_cmd, DepManager.SYSTEM_MANDATORY)
progs.update({
"python-sphinx": "python311-Sphinx",
@@ -681,12 +709,11 @@ class SphinxDependencyChecker:
# "Noto Sans CJK SC" on openSUSE
if self.pdf:
- self.check_rpm_missing(suse_tex_pkgs, DepType.PDF_MANDATORY)
+ self.check_rpm_missing(suse_tex_pkgs, DepManager.PDF_MANDATORY)
if self.pdf:
self.check_missing_tex()
- self.check_missing(progs)
- if not self.need and not self.optional:
+ if self.check_missing(progs):
return
if self.verbose_warn_install:
@@ -730,13 +757,12 @@ class SphinxDependencyChecker:
"/usr/share/fonts/TTF/NotoSans-Regular.ttf",
]
- self.check_missing_file(pdf_pkgs, noto_sans, DepType.PDF_MANDATORY)
- self.check_rpm_missing(tex_pkgs, DepType.PDF_MANDATORY)
+ self.check_missing_file(pdf_pkgs, noto_sans, DepManager.PDF_MANDATORY)
+ self.check_rpm_missing(tex_pkgs, DepManager.PDF_MANDATORY)
- self.check_missing(progs)
-
- if not self.need and not self.optional:
+ if self.check_missing(progs):
return
+
if self.verbose_warn_install:
print("You should run:")
print(f"\n\tsudo {packager_cmd} {self.install}")
@@ -759,7 +785,7 @@ class SphinxDependencyChecker:
]
if self.pdf:
- self.check_pacman_missing(archlinux_tex_pkgs, DepType.PDF_MANDATORY)
+ self.check_pacman_missing(archlinux_tex_pkgs, DepManager.PDF_MANDATORY)
self.check_missing_file(
["/usr/share/fonts/noto-cjk/NotoSansCJK-Regular.ttc"],
@@ -767,10 +793,9 @@ class SphinxDependencyChecker:
2,
)
- self.check_missing(progs)
-
- if not self.need and not self.optional:
+ if self.check_missing(progs):
return
+
if self.verbose_warn_install:
print("You should run:")
print(f"\n\tsudo pacman -S {self.install}")
@@ -797,11 +822,9 @@ class SphinxDependencyChecker:
],
}
for package, files in pdf_pkgs.items():
- self.check_missing_file(files, package, DepType.PDF_MANDATORY)
+ self.check_missing_file(files, package, DepManager.PDF_MANDATORY)
- self.check_missing(progs)
-
- if not self.need and not self.optional:
+ if self.check_missing(progs):
return
if self.verbose_warn_install:
@@ -1039,16 +1062,18 @@ class SphinxDependencyChecker:
print("\n2) As a package with:")
- old_need = self.need
- old_optional = self.optional
- self.missing = {}
+ old_need = self.deps.need
+ old_optional = self.deps.optional
+
self.pdf = False
self.optional = 0
self.install = ""
old_verbose = self.verbose_warn_install
self.verbose_warn_install = 0
- self.add_package("python-sphinx", DepType.PYTHON_MANDATORY)
+ self.deps.clear_deps()
+
+ self.deps.add_package("python-sphinx", DepManager.PYTHON_MANDATORY)
self.check_distros()
@@ -1111,7 +1136,7 @@ class SphinxDependencyChecker:
print("\n")
else:
if self.need_sphinx:
- self.need += 1
+ self.deps.need += 1
# Suggest newer versions if current ones are too old
if self.latest_avail_ver and self.latest_avail_ver >= self.min_version:
@@ -1190,20 +1215,20 @@ class SphinxDependencyChecker:
self.check_python_module("ensurepip")
# Check for needed programs/tools
- self.check_perl_module("Pod::Usage", DepType.SYSTEM_MANDATORY)
+ self.check_perl_module("Pod::Usage", DepManager.SYSTEM_MANDATORY)
- self.check_program("make", DepType.SYSTEM_MANDATORY)
- self.check_program("gcc", DepType.SYSTEM_MANDATORY)
+ self.check_program("make", DepManager.SYSTEM_MANDATORY)
+ self.check_program("gcc", DepManager.SYSTEM_MANDATORY)
- self.check_program("dot", DepType.SYSTEM_OPTIONAL)
- self.check_program("convert", DepType.SYSTEM_OPTIONAL)
+ self.check_program("dot", DepManager.SYSTEM_OPTIONAL)
+ self.check_program("convert", DepManager.SYSTEM_OPTIONAL)
self.check_python_module("yaml")
if self.pdf:
- self.check_program("xelatex", DepType.PDF_MANDATORY)
- self.check_program("rsvg-convert", DepType.PDF_MANDATORY)
- self.check_program("latexmk", DepType.PDF_MANDATORY)
+ self.check_program("xelatex", DepManager.PDF_MANDATORY)
+ self.check_program("rsvg-convert", DepManager.PDF_MANDATORY)
+ self.check_program("latexmk", DepManager.PDF_MANDATORY)
# Do distro-specific checks and output distro-install commands
self.check_distros()
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 27/39] scripts: sphinx-pre-install: move ancillary checkers to a separate class
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (25 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 26/39] scripts: sphinx-pre-install: move missing logic to a separate class Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 28/39] scripts: sphinx-pre-install: add more generic checkers on a class Mauro Carvalho Chehab
` (12 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
The code there are just a bunch of static functions that are used by
the main class. group them altogether to better organize the code.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 160 +++++++++++++++++-----------------
1 file changed, 81 insertions(+), 79 deletions(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index 47dce1fcddfb..b00e50028f4d 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -150,85 +150,11 @@ class DepManager:
if output_msg:
print(f"\n{output_msg}\n")
-class SphinxDependencyChecker:
- # List of required texlive packages on Fedora and OpenSuse
- texlive = {
- "amsfonts.sty": "texlive-amsfonts",
- "amsmath.sty": "texlive-amsmath",
- "amssymb.sty": "texlive-amsfonts",
- "amsthm.sty": "texlive-amscls",
- "anyfontsize.sty": "texlive-anyfontsize",
- "atbegshi.sty": "texlive-oberdiek",
- "bm.sty": "texlive-tools",
- "capt-of.sty": "texlive-capt-of",
- "cmap.sty": "texlive-cmap",
- "ctexhook.sty": "texlive-ctex",
- "ecrm1000.tfm": "texlive-ec",
- "eqparbox.sty": "texlive-eqparbox",
- "eu1enc.def": "texlive-euenc",
- "fancybox.sty": "texlive-fancybox",
- "fancyvrb.sty": "texlive-fancyvrb",
- "float.sty": "texlive-float",
- "fncychap.sty": "texlive-fncychap",
- "footnote.sty": "texlive-mdwtools",
- "framed.sty": "texlive-framed",
- "luatex85.sty": "texlive-luatex85",
- "multirow.sty": "texlive-multirow",
- "needspace.sty": "texlive-needspace",
- "palatino.sty": "texlive-psnfss",
- "parskip.sty": "texlive-parskip",
- "polyglossia.sty": "texlive-polyglossia",
- "tabulary.sty": "texlive-tabulary",
- "threeparttable.sty": "texlive-threeparttable",
- "titlesec.sty": "texlive-titlesec",
- "ucs.sty": "texlive-ucs",
- "upquote.sty": "texlive-upquote",
- "wrapfig.sty": "texlive-wrapfig",
- }
-
- def __init__(self, args):
- self.pdf = args.pdf
- self.virtualenv = args.virtualenv
- self.version_check = args.version_check
-
- self.deps = DepManager(self.pdf)
-
- self.need_symlink = 0
- self.need_sphinx = 0
- self.need_pip = 0
- self.rec_sphinx_upgrade = 0
- self.verbose_warn_install = 1
-
- self.system_release = ""
- self.install = ""
- self.virtenv_dir = ""
- self.python_cmd = ""
- self.activate_cmd = ""
-
- # Some distros may not have a Sphinx shipped package compatible with
- # our minimal requirements
- self.package_supported = True
-
- # Recommend a new python version
- self.recommend_python = None
-
- # Certain hints are meant to be shown only once
- self.first_hint = True
-
- self.min_version = (0, 0, 0)
- self.cur_version = (0, 0, 0)
- self.latest_avail_ver = (0, 0, 0)
- self.venv_ver = (0, 0, 0)
-
- prefix = os.environ.get("srctree", ".") + "/"
-
- self.conf = prefix + "Documentation/conf.py"
- self.requirement_file = prefix + "Documentation/sphinx/requirements.txt"
- self.virtenv_prefix = ["sphinx_", "Sphinx_" ]
-
- #
- # Ancillary methods that don't depend on self
- #
+class AncillaryCheckers:
+ """
+ Ancillary methods that checks for missing dependencies for different
+ types of types, like binaries, python modules, rpm deps, etc.
+ """
@staticmethod
def which(prog):
@@ -335,6 +261,82 @@ class SphinxDependencyChecker:
return subprocess.run(*args, **kwargs)
+class SphinxDependencyChecker(AncillaryCheckers):
+ # List of required texlive packages on Fedora and OpenSuse
+ texlive = {
+ "amsfonts.sty": "texlive-amsfonts",
+ "amsmath.sty": "texlive-amsmath",
+ "amssymb.sty": "texlive-amsfonts",
+ "amsthm.sty": "texlive-amscls",
+ "anyfontsize.sty": "texlive-anyfontsize",
+ "atbegshi.sty": "texlive-oberdiek",
+ "bm.sty": "texlive-tools",
+ "capt-of.sty": "texlive-capt-of",
+ "cmap.sty": "texlive-cmap",
+ "ctexhook.sty": "texlive-ctex",
+ "ecrm1000.tfm": "texlive-ec",
+ "eqparbox.sty": "texlive-eqparbox",
+ "eu1enc.def": "texlive-euenc",
+ "fancybox.sty": "texlive-fancybox",
+ "fancyvrb.sty": "texlive-fancyvrb",
+ "float.sty": "texlive-float",
+ "fncychap.sty": "texlive-fncychap",
+ "footnote.sty": "texlive-mdwtools",
+ "framed.sty": "texlive-framed",
+ "luatex85.sty": "texlive-luatex85",
+ "multirow.sty": "texlive-multirow",
+ "needspace.sty": "texlive-needspace",
+ "palatino.sty": "texlive-psnfss",
+ "parskip.sty": "texlive-parskip",
+ "polyglossia.sty": "texlive-polyglossia",
+ "tabulary.sty": "texlive-tabulary",
+ "threeparttable.sty": "texlive-threeparttable",
+ "titlesec.sty": "texlive-titlesec",
+ "ucs.sty": "texlive-ucs",
+ "upquote.sty": "texlive-upquote",
+ "wrapfig.sty": "texlive-wrapfig",
+ }
+
+ def __init__(self, args):
+ self.pdf = args.pdf
+ self.virtualenv = args.virtualenv
+ self.version_check = args.version_check
+
+ self.deps = DepManager(self.pdf)
+
+ self.need_symlink = 0
+ self.need_sphinx = 0
+ self.need_pip = 0
+ self.rec_sphinx_upgrade = 0
+ self.verbose_warn_install = 1
+
+ self.system_release = ""
+ self.install = ""
+ self.virtenv_dir = ""
+ self.python_cmd = ""
+ self.activate_cmd = ""
+
+ # Some distros may not have a Sphinx shipped package compatible with
+ # our minimal requirements
+ self.package_supported = True
+
+ # Recommend a new python version
+ self.recommend_python = None
+
+ # Certain hints are meant to be shown only once
+ self.first_hint = True
+
+ self.min_version = (0, 0, 0)
+ self.cur_version = (0, 0, 0)
+ self.latest_avail_ver = (0, 0, 0)
+ self.venv_ver = (0, 0, 0)
+
+ prefix = os.environ.get("srctree", ".") + "/"
+
+ self.conf = prefix + "Documentation/conf.py"
+ self.requirement_file = prefix + "Documentation/sphinx/requirements.txt"
+ self.virtenv_prefix = ["sphinx_", "Sphinx_" ]
+
#
# Methods to check if a feature exists
#
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 28/39] scripts: sphinx-pre-install: add more generic checkers on a class
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (26 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 27/39] scripts: sphinx-pre-install: move ancillary checkers " Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 29/39] scripts: sphinx-pre-install: move get_system_release() Mauro Carvalho Chehab
` (11 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Better organize the code by moving the more generic methods
to MissingCheckers. Such class contain only binary and package
dependent missing checkers, but no distro-specific data or code.
All distro-specific data/code remains at SphinxDependencyChecker
class.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 142 ++++++++++++++++++----------------
1 file changed, 76 insertions(+), 66 deletions(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index b00e50028f4d..9127487bd4d7 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -150,7 +150,7 @@ class DepManager:
if output_msg:
print(f"\n{output_msg}\n")
-class AncillaryCheckers:
+class AncillaryMethods:
"""
Ancillary methods that checks for missing dependencies for different
types of types, like binaries, python modules, rpm deps, etc.
@@ -261,81 +261,24 @@ class AncillaryCheckers:
return subprocess.run(*args, **kwargs)
-class SphinxDependencyChecker(AncillaryCheckers):
- # List of required texlive packages on Fedora and OpenSuse
- texlive = {
- "amsfonts.sty": "texlive-amsfonts",
- "amsmath.sty": "texlive-amsmath",
- "amssymb.sty": "texlive-amsfonts",
- "amsthm.sty": "texlive-amscls",
- "anyfontsize.sty": "texlive-anyfontsize",
- "atbegshi.sty": "texlive-oberdiek",
- "bm.sty": "texlive-tools",
- "capt-of.sty": "texlive-capt-of",
- "cmap.sty": "texlive-cmap",
- "ctexhook.sty": "texlive-ctex",
- "ecrm1000.tfm": "texlive-ec",
- "eqparbox.sty": "texlive-eqparbox",
- "eu1enc.def": "texlive-euenc",
- "fancybox.sty": "texlive-fancybox",
- "fancyvrb.sty": "texlive-fancyvrb",
- "float.sty": "texlive-float",
- "fncychap.sty": "texlive-fncychap",
- "footnote.sty": "texlive-mdwtools",
- "framed.sty": "texlive-framed",
- "luatex85.sty": "texlive-luatex85",
- "multirow.sty": "texlive-multirow",
- "needspace.sty": "texlive-needspace",
- "palatino.sty": "texlive-psnfss",
- "parskip.sty": "texlive-parskip",
- "polyglossia.sty": "texlive-polyglossia",
- "tabulary.sty": "texlive-tabulary",
- "threeparttable.sty": "texlive-threeparttable",
- "titlesec.sty": "texlive-titlesec",
- "ucs.sty": "texlive-ucs",
- "upquote.sty": "texlive-upquote",
- "wrapfig.sty": "texlive-wrapfig",
- }
+class MissingCheckers(AncillaryMethods):
- def __init__(self, args):
+ def __init__(self, args, texlive):
self.pdf = args.pdf
self.virtualenv = args.virtualenv
self.version_check = args.version_check
+ self.texlive = texlive
self.deps = DepManager(self.pdf)
self.need_symlink = 0
self.need_sphinx = 0
- self.need_pip = 0
- self.rec_sphinx_upgrade = 0
+
self.verbose_warn_install = 1
- self.system_release = ""
- self.install = ""
self.virtenv_dir = ""
- self.python_cmd = ""
- self.activate_cmd = ""
- # Some distros may not have a Sphinx shipped package compatible with
- # our minimal requirements
- self.package_supported = True
-
- # Recommend a new python version
- self.recommend_python = None
-
- # Certain hints are meant to be shown only once
- self.first_hint = True
-
- self.min_version = (0, 0, 0)
- self.cur_version = (0, 0, 0)
- self.latest_avail_ver = (0, 0, 0)
- self.venv_ver = (0, 0, 0)
-
- prefix = os.environ.get("srctree", ".") + "/"
-
- self.conf = prefix + "Documentation/conf.py"
- self.requirement_file = prefix + "Documentation/sphinx/requirements.txt"
- self.virtenv_prefix = ["sphinx_", "Sphinx_" ]
+ self.install = ""
#
# Methods to check if a feature exists
@@ -460,9 +403,9 @@ class SphinxDependencyChecker(AncillaryCheckers):
if match:
return parse_version(match.group(1))
- def check_sphinx(self):
+ def check_sphinx(self, conf):
try:
- with open(self.conf, "r", encoding="utf-8") as f:
+ with open(conf, "r", encoding="utf-8") as f:
for line in f:
match = re.match(r"^\s*needs_sphinx\s*=\s*[\'\"]([\d\.]+)[\'\"]", line)
if match:
@@ -513,6 +456,73 @@ class SphinxDependencyChecker(AncillaryCheckers):
return True
+class SphinxDependencyChecker(MissingCheckers):
+
+ def __init__(self, args):
+ # List of required texlive packages on Fedora and OpenSuse
+ texlive = {
+ "amsfonts.sty": "texlive-amsfonts",
+ "amsmath.sty": "texlive-amsmath",
+ "amssymb.sty": "texlive-amsfonts",
+ "amsthm.sty": "texlive-amscls",
+ "anyfontsize.sty": "texlive-anyfontsize",
+ "atbegshi.sty": "texlive-oberdiek",
+ "bm.sty": "texlive-tools",
+ "capt-of.sty": "texlive-capt-of",
+ "cmap.sty": "texlive-cmap",
+ "ctexhook.sty": "texlive-ctex",
+ "ecrm1000.tfm": "texlive-ec",
+ "eqparbox.sty": "texlive-eqparbox",
+ "eu1enc.def": "texlive-euenc",
+ "fancybox.sty": "texlive-fancybox",
+ "fancyvrb.sty": "texlive-fancyvrb",
+ "float.sty": "texlive-float",
+ "fncychap.sty": "texlive-fncychap",
+ "footnote.sty": "texlive-mdwtools",
+ "framed.sty": "texlive-framed",
+ "luatex85.sty": "texlive-luatex85",
+ "multirow.sty": "texlive-multirow",
+ "needspace.sty": "texlive-needspace",
+ "palatino.sty": "texlive-psnfss",
+ "parskip.sty": "texlive-parskip",
+ "polyglossia.sty": "texlive-polyglossia",
+ "tabulary.sty": "texlive-tabulary",
+ "threeparttable.sty": "texlive-threeparttable",
+ "titlesec.sty": "texlive-titlesec",
+ "ucs.sty": "texlive-ucs",
+ "upquote.sty": "texlive-upquote",
+ "wrapfig.sty": "texlive-wrapfig",
+ }
+
+ super().__init__(args, texlive)
+
+ self.need_pip = 0
+ self.rec_sphinx_upgrade = 0
+
+ self.system_release = ""
+ self.python_cmd = ""
+ self.activate_cmd = ""
+
+ # Some distros may not have a Sphinx shipped package compatible with
+ # our minimal requirements
+ self.package_supported = True
+
+ # Recommend a new python version
+ self.recommend_python = None
+
+ # Certain hints are meant to be shown only once
+ self.first_hint = True
+
+ self.min_version = (0, 0, 0)
+ self.cur_version = (0, 0, 0)
+ self.latest_avail_ver = (0, 0, 0)
+ self.venv_ver = (0, 0, 0)
+
+ prefix = os.environ.get("srctree", ".") + "/"
+
+ self.conf = prefix + "Documentation/conf.py"
+ self.requirement_file = prefix + "Documentation/sphinx/requirements.txt"
+ self.virtenv_prefix = ["sphinx_", "Sphinx_" ]
#
# Distro-specific hints methods
@@ -1187,7 +1197,7 @@ class SphinxDependencyChecker(AncillaryCheckers):
self.python_cmd = sys.executable
# Check if Sphinx is already accessible from current environment
- self.check_sphinx()
+ self.check_sphinx(self.conf)
if self.system_release:
print(f"Detected OS: {self.system_release}.")
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 29/39] scripts: sphinx-pre-install: move get_system_release()
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (27 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 28/39] scripts: sphinx-pre-install: add more generic checkers on a class Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 30/39] scripts: sphinx-pre-install: add documentation for the ancillary classes Mauro Carvalho Chehab
` (10 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
The code at get_system_release() is actually a helper function,
independent from the actual Sphinx verification checker. Move
it to MissingCheckers class, where other checkers are present.
With that, the entire distro-specific handler logic, with
all its complexity is confined at SphinxDependencyChecker
class.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 115 +++++++++++++++++-----------------
1 file changed, 59 insertions(+), 56 deletions(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index 9127487bd4d7..593982f350b3 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -456,6 +456,64 @@ class MissingCheckers(AncillaryMethods):
return True
+ def get_system_release(self):
+ """
+ Determine the system type. There's no unique way that would work
+ with all distros with a minimal package install. So, several
+ methods are used here.
+
+ By default, it will use lsb_release function. If not available, it will
+ fail back to reading the known different places where the distro name
+ is stored.
+
+ Several modern distros now have /etc/os-release, which usually have
+ a decent coverage.
+ """
+
+ system_release = ""
+
+ if self.which("lsb_release"):
+ result = self.run(["lsb_release", "-d"], capture_output=True, text=True)
+ system_release = result.stdout.replace("Description:", "").strip()
+
+ release_files = [
+ "/etc/system-release",
+ "/etc/redhat-release",
+ "/etc/lsb-release",
+ "/etc/gentoo-release",
+ ]
+
+ if not system_release:
+ for f in release_files:
+ system_release = self.catcheck(f)
+ if system_release:
+ break
+
+ # This seems more common than LSB these days
+ if not system_release:
+ os_var = {}
+ try:
+ with open("/etc/os-release", "r", encoding="utf-8") as f:
+ for line in f:
+ match = re.match(r"^([\w\d\_]+)=\"?([^\"]*)\"?\n", line)
+ if match:
+ os_var[match.group(1)] = match.group(2)
+
+ system_release = os_var.get("NAME", "")
+ if "VERSION_ID" in os_var:
+ system_release += " " + os_var["VERSION_ID"]
+ elif "VERSION" in os_var:
+ system_release += " " + os_var["VERSION"]
+ except IOError:
+ pass
+
+ if not system_release:
+ system_release = self.catcheck("/etc/issue")
+
+ system_release = system_release.strip()
+
+ return system_release
+
class SphinxDependencyChecker(MissingCheckers):
def __init__(self, args):
@@ -499,7 +557,7 @@ class SphinxDependencyChecker(MissingCheckers):
self.need_pip = 0
self.rec_sphinx_upgrade = 0
- self.system_release = ""
+ self.system_release = self.get_system_release()
self.python_cmd = ""
self.activate_cmd = ""
@@ -1193,7 +1251,6 @@ class SphinxDependencyChecker(MissingCheckers):
"\thttps://github.com/sphinx-doc/sphinx/pull/8313")
def check_needs(self):
- self.get_system_release()
self.python_cmd = sys.executable
# Check if Sphinx is already accessible from current environment
@@ -1270,60 +1327,6 @@ class SphinxDependencyChecker(MissingCheckers):
print("Needed package dependencies are met.")
- def get_system_release(self):
- """
- Determine the system type. There's no unique way that would work
- with all distros with a minimal package install. So, several
- methods are used here.
-
- By default, it will use lsb_release function. If not available, it will
- fail back to reading the known different places where the distro name
- is stored.
-
- Several modern distros now have /etc/os-release, which usually have
- a decent coverage.
- """
-
- if self.which("lsb_release"):
- result = self.run(["lsb_release", "-d"], capture_output=True, text=True)
- self.system_release = result.stdout.replace("Description:", "").strip()
-
- release_files = [
- "/etc/system-release",
- "/etc/redhat-release",
- "/etc/lsb-release",
- "/etc/gentoo-release",
- ]
-
- if not self.system_release:
- for f in release_files:
- self.system_release = self.catcheck(f)
- if self.system_release:
- break
-
- # This seems more common than LSB these days
- if not self.system_release:
- os_var = {}
- try:
- with open("/etc/os-release", "r", encoding="utf-8") as f:
- for line in f:
- match = re.match(r"^([\w\d\_]+)=\"?([^\"]*)\"?\n", line)
- if match:
- os_var[match.group(1)] = match.group(2)
-
- self.system_release = os_var.get("NAME", "")
- if "VERSION_ID" in os_var:
- self.system_release += " " + os_var["VERSION_ID"]
- elif "VERSION" in os_var:
- self.system_release += " " + os_var["VERSION"]
- except IOError:
- pass
-
- if not self.system_release:
- self.system_release = self.catcheck("/etc/issue")
-
- self.system_release = self.system_release.strip()
-
DESCRIPTION = """
Process some flags related to Sphinx installation and documentation build.
"""
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 30/39] scripts: sphinx-pre-install: add documentation for the ancillary classes.
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (28 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 29/39] scripts: sphinx-pre-install: move get_system_release() Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 31/39] scripts: sphinx-pre-install: add docstring documentation Mauro Carvalho Chehab
` (9 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
While here, rename a parameter to have its usage better
documented.
No functional changes.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 162 +++++++++++++++++++++++++++++-----
1 file changed, 140 insertions(+), 22 deletions(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index 593982f350b3..1dc3f19804ab 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -33,35 +33,62 @@ MIN_PYTHON_VERSION = parse_version("3.7")
class DepManager:
+ """
+ Manage package dependencies. There are three types of dependencies:
- # Internal types of dependencies.
+ - System: dependencies required for docs build;
+ - Python: python dependencies for a native distro Sphinx install;
+ - PDF: dependencies needed by PDF builds.
+
+ Each dependency can be mandatory or optional. Not installing an optional
+ dependency won't break the build, but will cause degradation at the
+ docs output.
+ """
+
+ # Internal types of dependencies. Don't use them outside DepManager class.
_SYS_TYPE = 0
_PHY_TYPE = 1
_PDF_TYPE = 2
- # Let's define keys as a tuple with the type and mandatory/optional.
- # This way, checking for optional or type is easy.
+ # Dependencies visible outside the class.
+ # The keys are tuple with: (type, is_mandatory flag).
+ #
+ # Currently we're not using all optional dep types. Yet, we'll keep all
+ # possible combinations here. They're not many, and that makes easier
+ # if later needed and for the name() method below
SYSTEM_MANDATORY = (_SYS_TYPE, True)
PYTHON_MANDATORY = (_PHY_TYPE, True)
PDF_MANDATORY = (_PDF_TYPE, True)
- # Currently we're not using all optional types, but let's keep all
- # combinations here, as we may end needing them in the future. Also,
- # it allows a name() function that handles all possibilities.
SYSTEM_OPTIONAL = (_SYS_TYPE, False)
PYTHON_OPTIONAL = (_PHY_TYPE, False)
PDF_OPTIONAL = (_PDF_TYPE, True)
def __init__(self, pdf):
+ """
+ Initialize internal vars:
+
+ - missing: missing dependencies list, containing a distro-independent
+ name for a missing dependency and its type.
+ - missing_pkg: ancillary dict containing missing dependencies in
+ distro namespace, organized by type.
+ - need: total number of needed dependencies. Never cleaned.
+ - optional: total number of optional dependencies. Never cleaned.
+ - pdf: PDF support is enabled?
+ """
self.missing = {}
- self.run = {}
+ self.missing_pkg = {}
self.need = 0
self.optional = 0
self.pdf = pdf
@staticmethod
def name(dtype):
+ """
+ Ancillary routine to output a warn/error message reporting
+ missing dependencies.
+ """
if dtype[0] == DepManager._SYS_TYPE:
msg = "build"
elif dtype[0] == DepManager._PHY_TYPE:
@@ -76,16 +103,22 @@ class DepManager:
@staticmethod
def is_optional(dtype):
+ """Ancillary routine to report if a dependency is optional"""
return not dtype[1]
@staticmethod
def is_pdf(dtype):
+ """Ancillary routine to report if a dependency is for PDF generation"""
if (dtype[0] == DepManager._PDF_TYPE):
return True
return False
def add_package(self, package, dtype):
+ """
+ Add a package at the self.missing() dictionary.
+ Doesn't update missing_pkg.
+ """
is_optional = DepManager.is_optional(dtype)
self.missing[package] = dtype
if is_optional:
@@ -94,6 +127,10 @@ class DepManager:
self.need += 1
def del_package(self, package):
+ """
+ Remove a package at the self.missing() dictionary.
+ Doesn't update missing_pkg.
+ """
if package in self.missing:
del self.missing[package]
@@ -104,13 +141,22 @@ class DepManager:
This is an ackward way to have a separate section to recommend
a package after system main dependencies.
- TODO: rework the logic to prevent needing it
+ TODO: rework the logic to prevent needing it.
"""
self.missing = {}
+ self.missing_pkg = {}
def check_missing(self, progs):
- self.run = {}
+ """
+ Update self.missing_pkg, using progs dict to convert from the
+ agnostic package name to distro-specific one.
+
+ Returns an string with the packages to be installed, sorted and
+ with eventual duplicates removed.
+ """
+
+ self.missing_pkg = {}
for prog, dtype in sorted(self.missing.items()):
# At least on some LTS distros like CentOS 7, texlive doesn't
@@ -123,23 +169,26 @@ class DepManager:
self.optional -= 1
continue
- if not dtype in self.run:
- self.run[dtype] = []
+ if not dtype in self.missing_pkg:
+ self.missing_pkg[dtype] = []
- self.run[dtype].append(progs.get(prog, prog))
+ self.missing_pkg[dtype].append(progs.get(prog, prog))
install = []
- for dtype in self.run.keys():
- install += self.run[dtype]
+ for dtype in self.missing_pkg.keys():
+ install += self.missing_pkg[dtype]
return " ".join(sorted(set(install)))
def warn_install(self):
+ """
+ Emit warnings/errors related to missing packages.
+ """
output_msg = ""
- for dtype in sorted(self.run.keys()):
- progs = " ".join(sorted(set(self.run[dtype])))
+ for dtype in sorted(self.missing_pkg.keys()):
+ progs = " ".join(sorted(set(self.missing_pkg[dtype])))
try:
name = DepManager.name(dtype)
@@ -158,6 +207,11 @@ class AncillaryMethods:
@staticmethod
def which(prog):
+ """
+ Our own implementation of which(). We could instead use
+ shutil.which(), but this function is simple enough.
+ Probably faster to use this implementation than to import shutil.
+ """
for path in os.environ.get("PATH", "").split(":"):
full_path = os.path.join(path, prog)
if os.access(full_path, os.X_OK):
@@ -167,6 +221,10 @@ class AncillaryMethods:
@staticmethod
def get_python_version(cmd):
+ """
+ Get python version from a Python binary. As we need to detect if
+ are out there newer python binaries, we can't rely on sys.release here.
+ """
result = SphinxDependencyChecker.run([cmd, "--version"],
capture_output=True, text=True)
@@ -181,7 +239,13 @@ class AncillaryMethods:
@staticmethod
def find_python():
+ """
+ Detect if are out there any python 3.xy version newer than the
+ current one.
+ Note: this routine is limited to up to 2 digits for python3. We
+ may need to update it one day, hopefully on a distant future.
+ """
patterns = [
"python3.[0-9]",
"python3.[0-9][0-9]",
@@ -200,7 +264,10 @@ class AncillaryMethods:
@staticmethod
def check_python():
-
+ """
+ Check if the current python binary satisfies our minimal requirement
+ for Sphinx build. If not, re-run with a newer version if found.
+ """
cur_ver = sys.version_info[:3]
if cur_ver >= MIN_PYTHON_VERSION:
ver = ver_str(cur_ver)
@@ -240,7 +307,10 @@ class AncillaryMethods:
@staticmethod
def run(*args, **kwargs):
- """Excecute a command, hiding its output by default"""
+ """
+ Excecute a command, hiding its output by default.
+ Preserve comatibility with older Python versions.
+ """
capture_output = kwargs.pop('capture_output', False)
@@ -262,8 +332,15 @@ class AncillaryMethods:
return subprocess.run(*args, **kwargs)
class MissingCheckers(AncillaryMethods):
+ """
+ Contains some ancillary checkers for different types of binaries and
+ package managers.
+ """
def __init__(self, args, texlive):
+ """
+ Initialize its internal variables
+ """
self.pdf = args.pdf
self.virtualenv = args.virtualenv
self.version_check = args.version_check
@@ -280,17 +357,20 @@ class MissingCheckers(AncillaryMethods):
self.install = ""
- #
- # Methods to check if a feature exists
- #
-
def check_missing_file(self, files, package, dtype):
+ """
+ Does the file exists? If not, add it to missing dependencies.
+ """
for f in files:
if os.path.exists(f):
return
self.deps.add_package(package, dtype)
def check_program(self, prog, dtype):
+ """
+ Does the program exists and it is at the PATH?
+ If not, add it to missing dependencies.
+ """
found = self.which(prog)
if found:
return found
@@ -300,6 +380,18 @@ class MissingCheckers(AncillaryMethods):
return None
def check_perl_module(self, prog, dtype):
+ """
+ Does perl have a dependency? Is it available?
+ If not, add it to missing dependencies.
+
+ Right now, we still need Perl for doc build, as it is required
+ by some tools called at docs or kernel build time, like:
+
+ scripts/documentation-file-ref-check
+
+ Also, checkpatch is on Perl.
+ """
+
# While testing with lxc download template, one of the
# distros (Oracle) didn't have perl - nor even an option to install
# before installing oraclelinux-release-el9 package.
@@ -318,6 +410,10 @@ class MissingCheckers(AncillaryMethods):
self.deps.add_package(prog, dtype)
def check_python_module(self, module, is_optional=False):
+ """
+ Does a python module exists outside venv? If not, add it to missing
+ dependencies.
+ """
if is_optional:
dtype = DepManager.PYTHON_OPTIONAL
else:
@@ -329,6 +425,9 @@ class MissingCheckers(AncillaryMethods):
self.deps.add_package(module, dtype)
def check_rpm_missing(self, pkgs, dtype):
+ """
+ Does a rpm package exists? If not, add it to missing dependencies.
+ """
for prog in pkgs:
try:
self.run(["rpm", "-q", prog], check=True)
@@ -336,6 +435,9 @@ class MissingCheckers(AncillaryMethods):
self.deps.add_package(prog, dtype)
def check_pacman_missing(self, pkgs, dtype):
+ """
+ Does a pacman package exists? If not, add it to missing dependencies.
+ """
for prog in pkgs:
try:
self.run(["pacman", "-Q", prog], check=True)
@@ -343,6 +445,9 @@ class MissingCheckers(AncillaryMethods):
self.deps.add_package(prog, dtype)
def check_missing_tex(self, is_optional=False):
+ """
+ Does a LaTeX package exists? If not, add it to missing dependencies.
+ """
if is_optional:
dtype = DepManager.PDF_OPTIONAL
else:
@@ -371,6 +476,9 @@ class MissingCheckers(AncillaryMethods):
self.deps.add_package(package, dtype)
def get_sphinx_fname(self):
+ """
+ Gets the binary filename for sphinx-build.
+ """
if "SPHINXBUILD" in os.environ:
return os.environ["SPHINXBUILD"]
@@ -386,6 +494,9 @@ class MissingCheckers(AncillaryMethods):
return ""
def get_sphinx_version(self, cmd):
+ """
+ Gets sphinx-build version.
+ """
try:
result = self.run([cmd, "--version"],
stdout=subprocess.PIPE,
@@ -404,6 +515,9 @@ class MissingCheckers(AncillaryMethods):
return parse_version(match.group(1))
def check_sphinx(self, conf):
+ """
+ Checks Sphinx minimal requirements
+ """
try:
with open(conf, "r", encoding="utf-8") as f:
for line in f:
@@ -441,6 +555,10 @@ class MissingCheckers(AncillaryMethods):
sys.exit(0)
def catcheck(self, filename):
+ """
+ Reads a file if it exists, returning as string.
+ If not found, returns an empty string.
+ """
if os.path.exists(filename):
with open(filename, "r", encoding="utf-8") as f:
return f.read().strip()
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 31/39] scripts: sphinx-pre-install: add docstring documentation
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (29 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 30/39] scripts: sphinx-pre-install: add documentation for the ancillary classes Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 32/39] scripts: sphinx-pre-install: fix several codingstyle issues Mauro Carvalho Chehab
` (8 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
This program is somewhat complex. Add some docstring documentation,
explaining what each function and class is supposed to do.
Most of the focus here were to describe the ancillary functions used
to detect dependency needs.
The main SphinxDependencyChecker still requires a lot of care,
and probably need to be reorganized to clearly split the 4 types
of output it produces:
- Need to upgrade Python binary;
- System install needs;
- Virtual env install needs;
- Python install needs via system packages, to run Sphinx
natively.
Yet, for now, I'm happy of having it a lot better documented
than its Perl version.
-
While here, rename a parameter to have its usage better
documented.
No functional changes.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 110 +++++++++++++++++++++++++++++++---
1 file changed, 102 insertions(+), 8 deletions(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index 1dc3f19804ab..1c96f6692e9a 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -9,6 +9,22 @@
# Don't add changes not compatible with it, it is meant to report
# incompatible python versions.
+"""
+Dependency checker for Sphinx documentation Kernel build.
+
+This module provides tools to check for all required dependencies needed to
+build documentation using Sphinx, including system packages, Python modules
+and LaTeX packages for PDF generation.
+
+It detect packages for a subset of Linux distributions used by Kernel
+maintainers, showing hints and missing dependencies.
+
+The main class SphinxDependencyChecker handles the dependency checking logic
+and provides recommendations for installing missing packages. It supports both
+system package installations and Python virtual environments. By default,
+system pacage install is recommended.
+"""
+
import argparse
import os
import re
@@ -75,7 +91,7 @@ class DepManager:
distro namespace, organized by type.
- need: total number of needed dependencies. Never cleaned.
- optional: total number of optional dependencies. Never cleaned.
- - pdf: PDF support is enabled?
+ - pdf: Is PDF support enabled?
"""
self.missing = {}
self.missing_pkg = {}
@@ -565,6 +581,13 @@ class MissingCheckers(AncillaryMethods):
return ""
def check_missing(self, progs):
+ """
+ Check for missing dependencies using the provided program mapping.
+
+ The actual distro-specific programs are mapped via progs argument.
+
+ Returns True if there are missing dependencies, False otherwise.
+ """
self.install += self.deps.check_missing(progs)
if self.verbose_warn_install:
self.deps.warn_install()
@@ -633,8 +656,18 @@ class MissingCheckers(AncillaryMethods):
return system_release
class SphinxDependencyChecker(MissingCheckers):
+ """
+ Main class for checking Sphinx documentation build dependencies.
+ - Check for missing system packages;
+ - Check for missing Python modules;
+ - Check for missing LaTeX packages needed by PDF generation;
+ - Propose Sphinx install via Python Virtual environment;
+ - Propose Sphinx install via distro-specific package install.
+ """
def __init__(self, args):
+ """Initialize checker variables"""
+
# List of required texlive packages on Fedora and OpenSuse
texlive = {
"amsfonts.sty": "texlive-amsfonts",
@@ -705,6 +738,9 @@ class SphinxDependencyChecker(MissingCheckers):
#
def give_debian_hints(self):
+ """
+ Provide package installation hints for Debian-based distros.
+ """
progs = {
"Pod::Usage": "perl-modules",
"convert": "imagemagick",
@@ -745,6 +781,10 @@ class SphinxDependencyChecker(MissingCheckers):
print(f"\n\tsudo apt-get install {self.install}")
def give_redhat_hints(self):
+ """
+ Provide package installation hints for RedHat-based distros
+ (Fedora, RHEL and RHEL-based variants).
+ """
progs = {
"Pod::Usage": "perl-Pod-Usage",
"convert": "ImageMagick",
@@ -837,6 +877,10 @@ class SphinxDependencyChecker(MissingCheckers):
print(f"\n\tsudo dnf install -y {self.install}")
def give_opensuse_hints(self):
+ """
+ Provide package installation hints for openSUSE-based distros
+ (Leap and Tumbleweed).
+ """
progs = {
"Pod::Usage": "perl-Pod-Usage",
"convert": "ImageMagick",
@@ -909,6 +953,9 @@ class SphinxDependencyChecker(MissingCheckers):
print(f"\n\tsudo zypper install --no-recommends {self.install}")
def give_mageia_hints(self):
+ """
+ Provide package installation hints for Mageia and OpenMandriva.
+ """
progs = {
"Pod::Usage": "perl-Pod-Usage",
"convert": "ImageMagick",
@@ -956,6 +1003,9 @@ class SphinxDependencyChecker(MissingCheckers):
print(f"\n\tsudo {packager_cmd} {self.install}")
def give_arch_linux_hints(self):
+ """
+ Provide package installation hints for ArchLinux.
+ """
progs = {
"convert": "imagemagick",
"dot": "graphviz",
@@ -989,6 +1039,9 @@ class SphinxDependencyChecker(MissingCheckers):
print(f"\n\tsudo pacman -S {self.install}")
def give_gentoo_hints(self):
+ """
+ Provide package installation hints for Gentoo.
+ """
progs = {
"convert": "media-gfx/imagemagick",
"dot": "media-gfx/graphviz",
@@ -1107,7 +1160,12 @@ class SphinxDependencyChecker(MissingCheckers):
#
def check_distros(self):
- # OS-specific hints logic
+ """
+ OS-specific hints logic. Seeks for a hinter. If found, provide
+ package-manager-specific install commands.
+
+ Otherwise, just lists the missing dependencies.
+ """
os_hints = {
re.compile("Red Hat Enterprise Linux"): self.give_redhat_hints,
re.compile("Fedora"): self.give_redhat_hints,
@@ -1159,10 +1217,22 @@ class SphinxDependencyChecker(MissingCheckers):
# Common dependencies
#
def deactivate_help(self):
+ """
+ Print a helper message to disable a virtual environment.
+ """
+
print("\n If you want to exit the virtualenv, you can use:")
print("\tdeactivate")
def get_virtenv(self):
+ """
+ Give a hint about how to activate an already-existing virtual
+ environment containing sphinx-build.
+
+ Returns a tuble with (activate_cmd_path, sphinx_version) with
+ the newest available virtual env.
+ """
+
cwd = os.getcwd()
activates = []
@@ -1207,6 +1277,14 @@ class SphinxDependencyChecker(MissingCheckers):
return ("", ver)
def recommend_sphinx_upgrade(self):
+ """
+ Check if Sphinx needs to be upgraded.
+
+ Returns a tuple with the higest available Sphinx version if found.
+ Otherwise, returns None to indicate either that no upgrade is needed
+ or no venv was found.
+ """
+
# Avoid running sphinx-builds from venv if cur_version is good
if self.cur_version and self.cur_version >= RECOMMENDED_VERSION:
self.latest_avail_ver = self.cur_version
@@ -1247,6 +1325,9 @@ class SphinxDependencyChecker(MissingCheckers):
return self.latest_avail_ver
def recommend_package(self):
+ """
+ Recommend installing Sphinx as a distro-specific package.
+ """
print("\n2) As a package with:")
@@ -1270,11 +1351,19 @@ class SphinxDependencyChecker(MissingCheckers):
self.verbose_warn_install = old_verbose
def recommend_sphinx_version(self, virtualenv_cmd):
- # The logic here is complex, as it have to deal with different versions:
- # - minimal supported version;
- # - minimal PDF version;
- # - recommended version.
- # It also needs to work fine with both distro's package and venv/virtualenv
+ """
+ Provide recommendations for installing or upgrading Sphinx based
+ on current version.
+
+ The logic here is complex, as it have to deal with different versions:
+
+ - minimal supported version;
+ - minimal PDF version;
+ - recommended version.
+
+ It also needs to work fine with both distro's package and
+ venv/virtualenv
+ """
if self.recommend_python:
print("\nPython version is incompatible with doc build.\n" \
@@ -1369,6 +1458,10 @@ class SphinxDependencyChecker(MissingCheckers):
"\thttps://github.com/sphinx-doc/sphinx/pull/8313")
def check_needs(self):
+ """
+ Main method that checks needed dependencies and provides
+ recommendations.
+ """
self.python_cmd = sys.executable
# Check if Sphinx is already accessible from current environment
@@ -1451,6 +1544,7 @@ Process some flags related to Sphinx installation and documentation build.
def main():
+ """Main function"""
parser = argparse.ArgumentParser(description=DESCRIPTION)
parser.add_argument(
@@ -1481,6 +1575,6 @@ def main():
checker.check_python()
checker.check_needs()
-
+# Call main if not used as module
if __name__ == "__main__":
main()
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 32/39] scripts: sphinx-pre-install: fix several codingstyle issues
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (30 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 31/39] scripts: sphinx-pre-install: add docstring documentation Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 33/39] scripts: sphinx-pre-install: rework install command logic Mauro Carvalho Chehab
` (7 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Address most pylint issues.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 52 +++++++++++++++++------------------
1 file changed, 26 insertions(+), 26 deletions(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index 1c96f6692e9a..1b11162da9fb 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -2,8 +2,8 @@
# SPDX-License-Identifier: GPL-2.0-or-later
# Copyright (c) 2017-2025 Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
#
-# pylint: disable=C0103,C0114,C0115,C0116,C0301
-# pylint: disable=R0902,R0904,R0912,R0915,R1705,R1710,E1121
+# pylint: disable=C0103,C0114,C0115,C0116,C0301,C0302
+# pylint: disable=R0902,R0904,R0911,R0912,R0914,R0915,R1705,R1710,E1121
# Note: this script requires at least Python 3.6 to run.
# Don't add changes not compatible with it, it is meant to report
@@ -115,7 +115,7 @@ class DepManager:
if dtype[1]:
return f"ERROR: {msg} mandatory deps missing"
else:
- out = f"Warning: {msg} optional deps missing"
+ return f"Warning: {msg} optional deps missing"
@staticmethod
def is_optional(dtype):
@@ -125,7 +125,7 @@ class DepManager:
@staticmethod
def is_pdf(dtype):
"""Ancillary routine to report if a dependency is for PDF generation"""
- if (dtype[0] == DepManager._PDF_TYPE):
+ if dtype[0] == DepManager._PDF_TYPE:
return True
return False
@@ -191,8 +191,8 @@ class DepManager:
self.missing_pkg[dtype].append(progs.get(prog, prog))
install = []
- for dtype in self.missing_pkg.keys():
- install += self.missing_pkg[dtype]
+ for dtype, pkgs in self.missing_pkg.items():
+ install += pkgs
return " ".join(sorted(set(install)))
@@ -267,8 +267,6 @@ class AncillaryMethods:
"python3.[0-9][0-9]",
]
- new_python_cmd = None
-
# Seek for a python binary newer than MIN_PYTHON_VERSION
for path in os.getenv("PATH", "").split(":"):
for pattern in patterns:
@@ -276,7 +274,7 @@ class AncillaryMethods:
if os.path.isfile(cmd) and os.access(cmd, os.X_OK):
version = SphinxDependencyChecker.get_python_version(cmd)
if version >= MIN_PYTHON_VERSION:
- return(cmd)
+ return cmd
@staticmethod
def check_python():
@@ -306,8 +304,8 @@ class AncillaryMethods:
new_python_cmd = SphinxDependencyChecker.find_python()
if not new_python_cmd:
- print(f"ERROR: Python version {python_ver} is not spported anymore")
- print(f" Can't find a new version. This script may fail")
+ print(f"ERROR: Python version {python_ver} is not spported anymore\n")
+ print(" Can't find a new version. This script may fail")
return
# Restart script using the newer version
@@ -362,6 +360,9 @@ class MissingCheckers(AncillaryMethods):
self.version_check = args.version_check
self.texlive = texlive
+ self.min_version = (0, 0, 0)
+ self.cur_version = (0, 0, 0)
+
self.deps = DepManager(self.pdf)
self.need_symlink = 0
@@ -370,8 +371,10 @@ class MissingCheckers(AncillaryMethods):
self.verbose_warn_install = 1
self.virtenv_dir = ""
-
self.install = ""
+ self.python_cmd = ""
+
+ self.virtenv_prefix = ["sphinx_", "Sphinx_" ]
def check_missing_file(self, files, package, dtype):
"""
@@ -542,10 +545,10 @@ class MissingCheckers(AncillaryMethods):
self.min_version = parse_version(match.group(1))
break
except IOError:
- sys.exit(f"Can't open {self.conf}")
+ sys.exit(f"Can't open {conf}")
if not self.min_version:
- sys.exit(f"Can't get needs_sphinx version from {self.conf}")
+ sys.exit(f"Can't get needs_sphinx version from {conf}")
self.virtenv_dir = self.virtenv_prefix[0] + "latest"
@@ -709,7 +712,6 @@ class SphinxDependencyChecker(MissingCheckers):
self.rec_sphinx_upgrade = 0
self.system_release = self.get_system_release()
- self.python_cmd = ""
self.activate_cmd = ""
# Some distros may not have a Sphinx shipped package compatible with
@@ -722,8 +724,6 @@ class SphinxDependencyChecker(MissingCheckers):
# Certain hints are meant to be shown only once
self.first_hint = True
- self.min_version = (0, 0, 0)
- self.cur_version = (0, 0, 0)
self.latest_avail_ver = (0, 0, 0)
self.venv_ver = (0, 0, 0)
@@ -731,7 +731,6 @@ class SphinxDependencyChecker(MissingCheckers):
self.conf = prefix + "Documentation/conf.py"
self.requirement_file = prefix + "Documentation/sphinx/requirements.txt"
- self.virtenv_prefix = ["sphinx_", "Sphinx_" ]
#
# Distro-specific hints methods
@@ -814,6 +813,7 @@ class SphinxDependencyChecker(MissingCheckers):
if not rel:
print("Couldn't identify release number")
+ noto_sans_redhat = None
self.pdf = False
elif re.search("Fedora", self.system_release):
# Fedora 38 and upper use this CJK font
@@ -1111,7 +1111,7 @@ class SphinxDependencyChecker(MissingCheckers):
for fname, portage in portages.items():
install = False
- while install == False:
+ while install is False:
if not files:
# No files under package.usage. Install all
install = True
@@ -1335,7 +1335,7 @@ class SphinxDependencyChecker(MissingCheckers):
old_optional = self.deps.optional
self.pdf = False
- self.optional = 0
+ self.deps.optional = 0
self.install = ""
old_verbose = self.verbose_warn_install
self.verbose_warn_install = 0
@@ -1346,8 +1346,8 @@ class SphinxDependencyChecker(MissingCheckers):
self.check_distros()
- self.need = old_need
- self.optional = old_optional
+ self.deps.need = old_need
+ self.deps.optional = old_optional
self.verbose_warn_install = old_verbose
def recommend_sphinx_version(self, virtualenv_cmd):
@@ -1528,13 +1528,13 @@ class SphinxDependencyChecker(MissingCheckers):
self.recommend_sphinx_version(virtualenv_cmd)
print("")
- if not self.optional:
+ if not self.deps.optional:
print("All optional dependencies are met.")
- if self.need == 1:
+ if self.deps.need == 1:
sys.exit("Can't build as 1 mandatory dependency is missing")
- elif self.need:
- sys.exit(f"Can't build as {self.need} mandatory dependencies are missing")
+ elif self.deps.need:
+ sys.exit(f"Can't build as {self.deps.need} mandatory dependencies are missing")
print("Needed package dependencies are met.")
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 33/39] scripts: sphinx-pre-install: rework install command logic
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (31 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 32/39] scripts: sphinx-pre-install: fix several codingstyle issues Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 34/39] docs: Makefile: switch to the new scripts/sphinx-pre-install.py Mauro Carvalho Chehab
` (6 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Cleanup the code to remove some redundancy and to let it be
clearer about the command install instructions.
Ensure that special instructions will be shown only once,
before the actual install command.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 179 ++++++++++++++++------------------
1 file changed, 82 insertions(+), 97 deletions(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index 1b11162da9fb..7dfe5c2a6cc2 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -213,7 +213,7 @@ class DepManager:
raise KeyError(f"ERROR!!!: invalid dtype for {progs}: {dtype}")
if output_msg:
- print(f"\n{output_msg}\n")
+ print(f"\n{output_msg}")
class AncillaryMethods:
"""
@@ -583,23 +583,6 @@ class MissingCheckers(AncillaryMethods):
return f.read().strip()
return ""
- def check_missing(self, progs):
- """
- Check for missing dependencies using the provided program mapping.
-
- The actual distro-specific programs are mapped via progs argument.
-
- Returns True if there are missing dependencies, False otherwise.
- """
- self.install += self.deps.check_missing(progs)
- if self.verbose_warn_install:
- self.deps.warn_install()
-
- if not self.deps.need and not self.deps.optional:
- return False
-
- return True
-
def get_system_release(self):
"""
Determine the system type. There's no unique way that would work
@@ -722,7 +705,7 @@ class SphinxDependencyChecker(MissingCheckers):
self.recommend_python = None
# Certain hints are meant to be shown only once
- self.first_hint = True
+ self.distro_msg = None
self.latest_avail_ver = (0, 0, 0)
self.venv_ver = (0, 0, 0)
@@ -732,6 +715,33 @@ class SphinxDependencyChecker(MissingCheckers):
self.conf = prefix + "Documentation/conf.py"
self.requirement_file = prefix + "Documentation/sphinx/requirements.txt"
+ def get_install_progs(self, progs, cmd, extra=None):
+ """
+ Check for missing dependencies using the provided program mapping.
+
+ The actual distro-specific programs are mapped via progs argument.
+ """
+ install = self.deps.check_missing(progs)
+
+ if self.verbose_warn_install:
+ self.deps.warn_install()
+
+ if not install:
+ return
+
+ if cmd:
+ if self.verbose_warn_install:
+ msg = "You should run:"
+ else:
+ msg = ""
+
+ if extra:
+ msg += "\n\t" + extra.replace("\n", "\n\t")
+
+ return(msg + "\n\tsudo " + cmd + " " + install)
+
+ return None
+
#
# Distro-specific hints methods
#
@@ -772,12 +782,7 @@ class SphinxDependencyChecker(MissingCheckers):
self.check_program("dvipng", DepManager.PDF_MANDATORY)
- if self.check_missing(progs):
- return
-
- if self.verbose_warn_install:
- print("You should run:")
- print(f"\n\tsudo apt-get install {self.install}")
+ return self.get_install_progs(progs, "apt-get install")
def give_redhat_hints(self):
"""
@@ -838,18 +843,16 @@ class SphinxDependencyChecker(MissingCheckers):
self.deps.add_package("python39", DepManager.SYSTEM_MANDATORY)
self.recommend_python = True
- if self.first_hint:
- print("Note: RHEL-based distros typically require extra repositories.\n" \
- "For most, enabling epel and crb are enough:\n" \
- "\tsudo dnf install -y epel-release", \
- "\tsudo dnf config-manager --set-enabled crb\n" \
- "Yet, some may have other required repositories. Those commands could be useful:\n" \
- "\tsudo dnf repolist all\n" \
- "\tsudo dnf repoquery --available --info <pkgs>\n",
- "\tsudo dnf config-manager --set-enabled '*' # enable all - probably not what you want")
-
- self.first_hint = False
-
+ if not self.distro_msg:
+ self.distro_msg = \
+ "Note: RHEL-based distros typically require extra repositories.\n" \
+ "For most, enabling epel and crb are enough:\n" \
+ "\tsudo dnf install -y epel-release\n" \
+ "\tsudo dnf config-manager --set-enabled crb\n" \
+ "Yet, some may have other required repositories. Those commands could be useful:\n" \
+ "\tsudo dnf repolist all\n" \
+ "\tsudo dnf repoquery --available --info <pkgs>\n" \
+ "\tsudo dnf config-manager --set-enabled '*' # enable all - probably not what you want"
if self.pdf:
pdf_pkgs = [
@@ -868,13 +871,7 @@ class SphinxDependencyChecker(MissingCheckers):
if not fedora and rel == 8:
self.deps.del_package("texlive-ctex")
- if self.check_missing(progs):
- return
-
- if self.verbose_warn_install:
- print("You should run:")
-
- print(f"\n\tsudo dnf install -y {self.install}")
+ return self.get_install_progs(progs, "dnf install")
def give_opensuse_hints(self):
"""
@@ -945,12 +942,7 @@ class SphinxDependencyChecker(MissingCheckers):
if self.pdf:
self.check_missing_tex()
- if self.check_missing(progs):
- return
-
- if self.verbose_warn_install:
- print("You should run:")
- print(f"\n\tsudo zypper install --no-recommends {self.install}")
+ return self.get_install_progs(progs, "zypper install --no-recommends")
def give_mageia_hints(self):
"""
@@ -995,12 +987,7 @@ class SphinxDependencyChecker(MissingCheckers):
self.check_missing_file(pdf_pkgs, noto_sans, DepManager.PDF_MANDATORY)
self.check_rpm_missing(tex_pkgs, DepManager.PDF_MANDATORY)
- if self.check_missing(progs):
- return
-
- if self.verbose_warn_install:
- print("You should run:")
- print(f"\n\tsudo {packager_cmd} {self.install}")
+ return self.get_install_progs(progs, packager_cmd)
def give_arch_linux_hints(self):
"""
@@ -1023,20 +1010,15 @@ class SphinxDependencyChecker(MissingCheckers):
]
if self.pdf:
- self.check_pacman_missing(archlinux_tex_pkgs, DepManager.PDF_MANDATORY)
+ self.check_pacman_missing(archlinux_tex_pkgs,
+ DepManager.PDF_MANDATORY)
- self.check_missing_file(
- ["/usr/share/fonts/noto-cjk/NotoSansCJK-Regular.ttc"],
- "noto-fonts-cjk",
- 2,
- )
+ self.check_missing_file(["/usr/share/fonts/noto-cjk/NotoSansCJK-Regular.ttc"],
+ "noto-fonts-cjk",
+ DepManager.PDF_MANDATORY)
- if self.check_missing(progs):
- return
- if self.verbose_warn_install:
- print("You should run:")
- print(f"\n\tsudo pacman -S {self.install}")
+ return self.get_install_progs(progs, "pacman -S")
def give_gentoo_hints(self):
"""
@@ -1065,13 +1047,6 @@ class SphinxDependencyChecker(MissingCheckers):
for package, files in pdf_pkgs.items():
self.check_missing_file(files, package, DepManager.PDF_MANDATORY)
- if self.check_missing(progs):
- return
-
- if self.verbose_warn_install:
- print("You should run:")
- print("\n")
-
# Handling dependencies is a nightmare, as Gentoo refuses to emerge
# some packages if there's no package.use file describing them.
# To make it worse, compilation flags shall also be present there
@@ -1104,7 +1079,10 @@ class SphinxDependencyChecker(MissingCheckers):
"zziblib": "dev-libs/zziplib sdl",
}
- if self.first_hint:
+ extra_cmds = ""
+ if not self.distro_msg:
+ self.distro_msg = "Note: Gentoo requires package.use to be adjusted before emerging packages"
+
use_base = "/etc/portage/package.use"
files = glob(f"{use_base}/*")
@@ -1148,23 +1126,23 @@ class SphinxDependencyChecker(MissingCheckers):
# emit a code to setup missing USE
if install:
- print(f"\tsudo su -c 'echo \"{portage}\" > {use_base}/{fname}'")
-
- self.first_hint = False
+ extra_cmds += (f"sudo su -c 'echo \"{portage}\" > {use_base}/{fname}'\n")
# Now, we can use emerge and let it respect USE
- print(f"\tsudo emerge --ask --changed-use --binpkg-respect-use=y {self.install}")
+ return self.get_install_progs(progs,
+ "emerge --ask --changed-use --binpkg-respect-use=y",
+ extra_cmds)
- #
- # Dispatch the check to an os_specific hinter
- #
-
- def check_distros(self):
+ def get_install(self):
"""
- OS-specific hints logic. Seeks for a hinter. If found, provide
- package-manager-specific install commands.
+ OS-specific hints logic. Seeks for a hinter. If found, use it to
+ provide package-manager specific install commands.
- Otherwise, just lists the missing dependencies.
+ Otherwise, outputs install instructions for the meta-packages.
+
+ Returns a string with the command to be executed to install the
+ the needed packages, if distro found. Otherwise, return just a
+ list of packages that require installation.
"""
os_hints = {
re.compile("Red Hat Enterprise Linux"): self.give_redhat_hints,
@@ -1195,9 +1173,7 @@ class SphinxDependencyChecker(MissingCheckers):
# If the OS is detected, use per-OS hint logic
for regex, os_hint in os_hints.items():
if regex.search(self.system_release):
- os_hint()
-
- return
+ return os_hint()
#
# Fall-back to generic hint code for other distros
@@ -1207,11 +1183,12 @@ class SphinxDependencyChecker(MissingCheckers):
if self.pdf:
self.check_missing_tex()
- self.check_missing(progs)
+ self.distro_msg = \
+ f"I don't know distro {self.system_release}.\n" \
+ "So, I can't provide you a hint with the install procedure.\n" \
+ "There are likely missing dependencies.\n"
- print(f"I don't know distro {self.system_release}.")
- print("So, I can't provide you a hint with the install procedure.")
- print("There are likely missing dependencies.")
+ return self.get_install_progs(progs, None)
#
# Common dependencies
@@ -1336,7 +1313,6 @@ class SphinxDependencyChecker(MissingCheckers):
self.pdf = False
self.deps.optional = 0
- self.install = ""
old_verbose = self.verbose_warn_install
self.verbose_warn_install = 0
@@ -1344,7 +1320,9 @@ class SphinxDependencyChecker(MissingCheckers):
self.deps.add_package("python-sphinx", DepManager.PYTHON_MANDATORY)
- self.check_distros()
+ cmd = self.get_install()
+ if cmd:
+ print(cmd)
self.deps.need = old_need
self.deps.optional = old_optional
@@ -1511,7 +1489,14 @@ class SphinxDependencyChecker(MissingCheckers):
self.check_program("latexmk", DepManager.PDF_MANDATORY)
# Do distro-specific checks and output distro-install commands
- self.check_distros()
+ cmd = self.get_install()
+ if cmd:
+ print(cmd)
+
+ # If distro requires some special instructions, print here.
+ # Please notice that get_install() needs to be called first.
+ if self.distro_msg:
+ print("\n" + self.distro_msg)
if not self.python_cmd:
if self.need == 1:
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 34/39] docs: Makefile: switch to the new scripts/sphinx-pre-install.py
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (32 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 33/39] scripts: sphinx-pre-install: rework install command logic Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 35/39] scripts: sphinx-pre-install.pl: get rid of the old script Mauro Carvalho Chehab
` (5 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Now that we have a better, improved Python script, use it when
checking for documentation build dependencies.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/Makefile | 14 +++++++-------
.../{sphinx-pre-install.py => sphinx-pre-install} | 0
2 files changed, 7 insertions(+), 7 deletions(-)
rename scripts/{sphinx-pre-install.py => sphinx-pre-install} (100%)
diff --git a/Documentation/Makefile b/Documentation/Makefile
index c486fe3cc5e1..b98477df5ddf 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -46,7 +46,7 @@ ifeq ($(HAVE_SPHINX),0)
.DEFAULT:
$(warning The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed and in PATH, or set the SPHINXBUILD make variable to point to the full path of the '$(SPHINXBUILD)' executable.)
@echo
- @$(srctree)/scripts/sphinx-pre-install.pl
+ @$(srctree)/scripts/sphinx-pre-install
@echo " SKIP Sphinx $@ target."
else # HAVE_SPHINX
@@ -121,7 +121,7 @@ $(YNL_RST_DIR)/%.rst: $(YNL_YAML_DIR)/%.yaml $(YNL_TOOL)
htmldocs texinfodocs latexdocs epubdocs xmldocs: $(YNL_INDEX)
htmldocs:
- @$(srctree)/scripts/sphinx-pre-install.pl --version-check
+ @$(srctree)/scripts/sphinx-pre-install --version-check
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,html,$(var),,$(var)))
# If Rust support is available and .config exists, add rustdoc generated contents.
@@ -135,7 +135,7 @@ endif
endif
texinfodocs:
- @$(srctree)/scripts/sphinx-pre-install.pl --version-check
+ @$(srctree)/scripts/sphinx-pre-install --version-check
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,texinfo,$(var),texinfo,$(var)))
# Note: the 'info' Make target is generated by sphinx itself when
@@ -147,7 +147,7 @@ linkcheckdocs:
@$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,linkcheck,$(var),,$(var)))
latexdocs:
- @$(srctree)/scripts/sphinx-pre-install.pl --version-check
+ @$(srctree)/scripts/sphinx-pre-install --version-check
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,latex,$(var),latex,$(var)))
ifeq ($(HAVE_PDFLATEX),0)
@@ -160,7 +160,7 @@ else # HAVE_PDFLATEX
pdfdocs: DENY_VF = XDG_CONFIG_HOME=$(FONTS_CONF_DENY_VF)
pdfdocs: latexdocs
- @$(srctree)/scripts/sphinx-pre-install.pl --version-check
+ @$(srctree)/scripts/sphinx-pre-install --version-check
$(foreach var,$(SPHINXDIRS), \
$(MAKE) PDFLATEX="$(PDFLATEX)" LATEXOPTS="$(LATEXOPTS)" $(DENY_VF) -C $(BUILDDIR)/$(var)/latex || sh $(srctree)/scripts/check-variable-fonts.sh || exit; \
mkdir -p $(BUILDDIR)/$(var)/pdf; \
@@ -170,11 +170,11 @@ pdfdocs: latexdocs
endif # HAVE_PDFLATEX
epubdocs:
- @$(srctree)/scripts/sphinx-pre-install.pl --version-check
+ @$(srctree)/scripts/sphinx-pre-install --version-check
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,epub,$(var),epub,$(var)))
xmldocs:
- @$(srctree)/scripts/sphinx-pre-install.pl --version-check
+ @$(srctree)/scripts/sphinx-pre-install --version-check
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,xml,$(var),xml,$(var)))
endif # HAVE_SPHINX
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install
similarity index 100%
rename from scripts/sphinx-pre-install.py
rename to scripts/sphinx-pre-install
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 35/39] scripts: sphinx-pre-install.pl: get rid of the old script
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (33 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 34/39] docs: Makefile: switch to the new scripts/sphinx-pre-install.py Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 36/39] scripts: sphinx-pre-install: update mandatory system deps Mauro Carvalho Chehab
` (4 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
All features were ported to the Python version. Plus, it
supports more variants and contain fixes.
So, drop the old version.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.pl | 1056 ---------------------------------
1 file changed, 1056 deletions(-)
delete mode 100755 scripts/sphinx-pre-install.pl
diff --git a/scripts/sphinx-pre-install.pl b/scripts/sphinx-pre-install.pl
deleted file mode 100755
index 07234d482fa8..000000000000
--- a/scripts/sphinx-pre-install.pl
+++ /dev/null
@@ -1,1056 +0,0 @@
-#!/usr/bin/env perl
-# SPDX-License-Identifier: GPL-2.0-or-later
-use strict;
-
-# Copyright (c) 2017-2020 Mauro Carvalho Chehab <mchehab@kernel.org>
-#
-
-my $prefix = "./";
-$prefix = "$ENV{'srctree'}/" if ($ENV{'srctree'});
-
-my $conf = $prefix . "Documentation/conf.py";
-my $requirement_file = $prefix . "Documentation/sphinx/requirements.txt";
-my $virtenv_prefix = "sphinx_";
-
-#
-# Static vars
-#
-
-my %missing;
-my $system_release;
-my $need = 0;
-my $optional = 0;
-my $need_symlink = 0;
-my $need_sphinx = 0;
-my $need_pip = 0;
-my $need_virtualenv = 0;
-my $rec_sphinx_upgrade = 0;
-my $verbose_warn_install = 1;
-my $install = "";
-my $virtenv_dir = "";
-my $python_cmd = "";
-my $activate_cmd;
-my $min_version;
-my $cur_version;
-my $rec_version = "3.4.3";
-my $latest_avail_ver;
-
-#
-# Command line arguments
-#
-
-my $pdf = 1;
-my $virtualenv = 1;
-my $version_check = 0;
-
-#
-# List of required texlive packages on Fedora and OpenSuse
-#
-
-my %texlive = (
- 'amsfonts.sty' => 'texlive-amsfonts',
- 'amsmath.sty' => 'texlive-amsmath',
- 'amssymb.sty' => 'texlive-amsfonts',
- 'amsthm.sty' => 'texlive-amscls',
- 'anyfontsize.sty' => 'texlive-anyfontsize',
- 'atbegshi.sty' => 'texlive-oberdiek',
- 'bm.sty' => 'texlive-tools',
- 'capt-of.sty' => 'texlive-capt-of',
- 'cmap.sty' => 'texlive-cmap',
- 'ecrm1000.tfm' => 'texlive-ec',
- 'eqparbox.sty' => 'texlive-eqparbox',
- 'eu1enc.def' => 'texlive-euenc',
- 'fancybox.sty' => 'texlive-fancybox',
- 'fancyvrb.sty' => 'texlive-fancyvrb',
- 'float.sty' => 'texlive-float',
- 'fncychap.sty' => 'texlive-fncychap',
- 'footnote.sty' => 'texlive-mdwtools',
- 'framed.sty' => 'texlive-framed',
- 'luatex85.sty' => 'texlive-luatex85',
- 'multirow.sty' => 'texlive-multirow',
- 'needspace.sty' => 'texlive-needspace',
- 'palatino.sty' => 'texlive-psnfss',
- 'parskip.sty' => 'texlive-parskip',
- 'polyglossia.sty' => 'texlive-polyglossia',
- 'tabulary.sty' => 'texlive-tabulary',
- 'threeparttable.sty' => 'texlive-threeparttable',
- 'titlesec.sty' => 'texlive-titlesec',
- 'ucs.sty' => 'texlive-ucs',
- 'upquote.sty' => 'texlive-upquote',
- 'wrapfig.sty' => 'texlive-wrapfig',
- 'ctexhook.sty' => 'texlive-ctex',
-);
-
-#
-# Subroutines that checks if a feature exists
-#
-
-sub check_missing(%)
-{
- my %map = %{$_[0]};
-
- foreach my $prog (sort keys %missing) {
- my $is_optional = $missing{$prog};
-
- # At least on some LTS distros like CentOS 7, texlive doesn't
- # provide all packages we need. When such distros are
- # detected, we have to disable PDF output.
- #
- # So, we need to ignore the packages that distros would
- # need for LaTeX to work
- if ($is_optional == 2 && !$pdf) {
- $optional--;
- next;
- }
-
- if ($verbose_warn_install) {
- if ($is_optional) {
- print "Warning: better to also install \"$prog\".\n";
- } else {
- print "ERROR: please install \"$prog\", otherwise, build won't work.\n";
- }
- }
- if (defined($map{$prog})) {
- $install .= " " . $map{$prog};
- } else {
- $install .= " " . $prog;
- }
- }
-
- $install =~ s/^\s//;
-}
-
-sub add_package($$)
-{
- my $package = shift;
- my $is_optional = shift;
-
- $missing{$package} = $is_optional;
- if ($is_optional) {
- $optional++;
- } else {
- $need++;
- }
-}
-
-sub check_missing_file($$$)
-{
- my $files = shift;
- my $package = shift;
- my $is_optional = shift;
-
- for (@$files) {
- return if(-e $_);
- }
-
- add_package($package, $is_optional);
-}
-
-sub findprog($)
-{
- foreach(split(/:/, $ENV{PATH})) {
- return "$_/$_[0]" if(-x "$_/$_[0]");
- }
-}
-
-sub find_python_no_venv()
-{
- my $prog = shift;
-
- my $cur_dir = qx(pwd);
- $cur_dir =~ s/\s+$//;
-
- foreach my $dir (split(/:/, $ENV{PATH})) {
- next if ($dir =~ m,($cur_dir)/sphinx,);
- return "$dir/python3" if(-x "$dir/python3");
- }
- foreach my $dir (split(/:/, $ENV{PATH})) {
- next if ($dir =~ m,($cur_dir)/sphinx,);
- return "$dir/python" if(-x "$dir/python");
- }
- return "python";
-}
-
-sub check_program($$)
-{
- my $prog = shift;
- my $is_optional = shift;
-
- return $prog if findprog($prog);
-
- add_package($prog, $is_optional);
-}
-
-sub check_perl_module($$)
-{
- my $prog = shift;
- my $is_optional = shift;
-
- my $err = system("perl -M$prog -e 1 2>/dev/null /dev/null");
- return if ($err == 0);
-
- add_package($prog, $is_optional);
-}
-
-sub check_python_module($$)
-{
- my $prog = shift;
- my $is_optional = shift;
-
- return if (!$python_cmd);
-
- my $err = system("$python_cmd -c 'import $prog' 2>/dev/null /dev/null");
- return if ($err == 0);
-
- add_package($prog, $is_optional);
-}
-
-sub check_rpm_missing($$)
-{
- my @pkgs = @{$_[0]};
- my $is_optional = $_[1];
-
- foreach my $prog(@pkgs) {
- my $err = system("rpm -q '$prog' 2>/dev/null >/dev/null");
- add_package($prog, $is_optional) if ($err);
- }
-}
-
-sub check_pacman_missing($$)
-{
- my @pkgs = @{$_[0]};
- my $is_optional = $_[1];
-
- foreach my $prog(@pkgs) {
- my $err = system("pacman -Q '$prog' 2>/dev/null >/dev/null");
- add_package($prog, $is_optional) if ($err);
- }
-}
-
-sub check_missing_tex($)
-{
- my $is_optional = shift;
- my $kpsewhich = findprog("kpsewhich");
-
- foreach my $prog(keys %texlive) {
- my $package = $texlive{$prog};
- if (!$kpsewhich) {
- add_package($package, $is_optional);
- next;
- }
- my $file = qx($kpsewhich $prog);
- add_package($package, $is_optional) if ($file =~ /^\s*$/);
- }
-}
-
-sub get_sphinx_fname()
-{
- if ($ENV{'SPHINXBUILD'}) {
- return $ENV{'SPHINXBUILD'};
- }
-
- my $fname = "sphinx-build";
- return $fname if findprog($fname);
-
- $fname = "sphinx-build-3";
- if (findprog($fname)) {
- $need_symlink = 1;
- return $fname;
- }
-
- return "";
-}
-
-sub get_sphinx_version($)
-{
- my $cmd = shift;
- my $ver;
-
- open IN, "$cmd --version 2>&1 |";
- while (<IN>) {
- if (m/^\s*sphinx-build\s+([\d\.]+)((\+\/[\da-f]+)|(b\d+))?$/) {
- $ver=$1;
- last;
- }
- # Sphinx 1.2.x uses a different format
- if (m/^\s*Sphinx.*\s+([\d\.]+)$/) {
- $ver=$1;
- last;
- }
- }
- close IN;
- return $ver;
-}
-
-sub check_sphinx()
-{
- open IN, $conf or die "Can't open $conf";
- while (<IN>) {
- if (m/^\s*needs_sphinx\s*=\s*[\'\"]([\d\.]+)[\'\"]/) {
- $min_version=$1;
- last;
- }
- }
- close IN;
-
- die "Can't get needs_sphinx version from $conf" if (!$min_version);
-
- $virtenv_dir = $virtenv_prefix . "latest";
-
- my $sphinx = get_sphinx_fname();
- if ($sphinx eq "") {
- $need_sphinx = 1;
- return;
- }
-
- $cur_version = get_sphinx_version($sphinx);
- die "$sphinx didn't return its version" if (!$cur_version);
-
- if ($cur_version lt $min_version) {
- printf "ERROR: Sphinx version is %s. It should be >= %s\n",
- $cur_version, $min_version;
- $need_sphinx = 1;
- return;
- }
-
- return if ($cur_version lt $rec_version);
-
- # On version check mode, just assume Sphinx has all mandatory deps
- exit (0) if ($version_check);
-}
-
-#
-# Ancillary subroutines
-#
-
-sub catcheck($)
-{
- my $res = "";
- $res = qx(cat $_[0]) if (-r $_[0]);
- return $res;
-}
-
-sub which($)
-{
- my $file = shift;
- my @path = split ":", $ENV{PATH};
-
- foreach my $dir(@path) {
- my $name = $dir.'/'.$file;
- return $name if (-x $name );
- }
- return undef;
-}
-
-#
-# Subroutines that check distro-specific hints
-#
-
-sub give_debian_hints()
-{
- my %map = (
- "python-sphinx" => "python3-sphinx",
- "yaml" => "python3-yaml",
- "ensurepip" => "python3-venv",
- "virtualenv" => "virtualenv",
- "dot" => "graphviz",
- "convert" => "imagemagick",
- "Pod::Usage" => "perl-modules",
- "xelatex" => "texlive-xetex",
- "rsvg-convert" => "librsvg2-bin",
- );
-
- if ($pdf) {
- check_missing_file(["/usr/share/texlive/texmf-dist/tex/latex/ctex/ctexhook.sty"],
- "texlive-lang-chinese", 2);
-
- check_missing_file(["/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf"],
- "fonts-dejavu", 2);
-
- check_missing_file(["/usr/share/fonts/noto-cjk/NotoSansCJK-Regular.ttc",
- "/usr/share/fonts/opentype/noto/NotoSansCJK-Regular.ttc",
- "/usr/share/fonts/opentype/noto/NotoSerifCJK-Regular.ttc"],
- "fonts-noto-cjk", 2);
- }
-
- check_program("dvipng", 2) if ($pdf);
- check_missing(\%map);
-
- return if (!$need && !$optional);
- printf("You should run:\n") if ($verbose_warn_install);
- printf("\n\tsudo apt-get install $install\n");
-}
-
-sub give_redhat_hints()
-{
- my %map = (
- "python-sphinx" => "python3-sphinx",
- "yaml" => "python3-pyyaml",
- "virtualenv" => "python3-virtualenv",
- "dot" => "graphviz",
- "convert" => "ImageMagick",
- "Pod::Usage" => "perl-Pod-Usage",
- "xelatex" => "texlive-xetex-bin",
- "rsvg-convert" => "librsvg2-tools",
- );
-
- my @fedora26_opt_pkgs = (
- "graphviz-gd", # Fedora 26: needed for PDF support
- );
-
- my @fedora_tex_pkgs = (
- "texlive-collection-fontsrecommended",
- "texlive-collection-latex",
- "texlive-xecjk",
- "dejavu-sans-fonts",
- "dejavu-serif-fonts",
- "dejavu-sans-mono-fonts",
- );
-
- #
- # Checks valid for RHEL/CentOS version 7.x.
- #
- my $old = 0;
- my $rel;
- my $noto_sans_redhat = "google-noto-sans-cjk-ttc-fonts";
- $rel = $2 if ($system_release =~ /(release|Linux)\s+(\d+)/);
-
- if (!($system_release =~ /Fedora/)) {
- $map{"virtualenv"} = "python-virtualenv";
-
- if ($rel && $rel < 8) {
- $old = 1;
- $pdf = 0;
-
- printf("Note: texlive packages on RHEL/CENTOS <= 7 are incomplete. Can't support PDF output\n");
- printf("If you want to build PDF, please read:\n");
- printf("\thttps://www.systutorials.com/241660/how-to-install-tex-live-on-centos-7-linux/\n");
- }
- } else {
- if ($rel && $rel < 26) {
- $old = 1;
- }
- if ($rel && $rel >= 38) {
- $noto_sans_redhat = "google-noto-sans-cjk-fonts";
- }
- }
- if (!$rel) {
- printf("Couldn't identify release number\n");
- $old = 1;
- $pdf = 0;
- }
-
- if ($pdf) {
- check_missing_file(["/usr/share/fonts/google-noto-cjk/NotoSansCJK-Regular.ttc",
- "/usr/share/fonts/google-noto-sans-cjk-fonts/NotoSansCJK-Regular.ttc"],
- $noto_sans_redhat, 2);
- }
-
- check_rpm_missing(\@fedora26_opt_pkgs, 2) if ($pdf && !$old);
- check_rpm_missing(\@fedora_tex_pkgs, 2) if ($pdf);
- check_missing_tex(2) if ($pdf);
- check_missing(\%map);
-
- return if (!$need && !$optional);
-
- if (!$old) {
- # dnf, for Fedora 18+
- printf("You should run:\n") if ($verbose_warn_install);
- printf("\n\tsudo dnf install -y $install\n");
- } else {
- # yum, for RHEL (and clones) or Fedora version < 18
- printf("You should run:\n") if ($verbose_warn_install);
- printf("\n\tsudo yum install -y $install\n");
- }
-}
-
-sub give_opensuse_hints()
-{
- my %map = (
- "python-sphinx" => "python3-sphinx",
- "yaml" => "python3-pyyaml",
- "virtualenv" => "python3-virtualenv",
- "dot" => "graphviz",
- "convert" => "ImageMagick",
- "Pod::Usage" => "perl-Pod-Usage",
- "xelatex" => "texlive-xetex-bin",
- );
-
- # On Tumbleweed, this package is also named rsvg-convert
- $map{"rsvg-convert"} = "rsvg-view" if (!($system_release =~ /Tumbleweed/));
-
- my @suse_tex_pkgs = (
- "texlive-babel-english",
- "texlive-caption",
- "texlive-colortbl",
- "texlive-courier",
- "texlive-dvips",
- "texlive-helvetic",
- "texlive-makeindex",
- "texlive-metafont",
- "texlive-metapost",
- "texlive-palatino",
- "texlive-preview",
- "texlive-times",
- "texlive-zapfchan",
- "texlive-zapfding",
- );
-
- $map{"latexmk"} = "texlive-latexmk-bin";
-
- # FIXME: add support for installing CJK fonts
- #
- # I tried hard, but was unable to find a way to install
- # "Noto Sans CJK SC" on openSUSE
-
- check_rpm_missing(\@suse_tex_pkgs, 2) if ($pdf);
- check_missing_tex(2) if ($pdf);
- check_missing(\%map);
-
- return if (!$need && !$optional);
- printf("You should run:\n") if ($verbose_warn_install);
- printf("\n\tsudo zypper install --no-recommends $install\n");
-}
-
-sub give_mageia_hints()
-{
- my %map = (
- "python-sphinx" => "python3-sphinx",
- "yaml" => "python3-yaml",
- "virtualenv" => "python3-virtualenv",
- "dot" => "graphviz",
- "convert" => "ImageMagick",
- "Pod::Usage" => "perl-Pod-Usage",
- "xelatex" => "texlive",
- "rsvg-convert" => "librsvg2",
- );
-
- my @tex_pkgs = (
- "texlive-fontsextra",
- );
-
- $map{"latexmk"} = "texlive-collection-basic";
-
- my $packager_cmd;
- my $noto_sans;
- if ($system_release =~ /OpenMandriva/) {
- $packager_cmd = "dnf install";
- $noto_sans = "noto-sans-cjk-fonts";
- @tex_pkgs = ( "texlive-collection-fontsextra" );
- } else {
- $packager_cmd = "urpmi";
- $noto_sans = "google-noto-sans-cjk-ttc-fonts";
- }
-
-
- if ($pdf) {
- check_missing_file(["/usr/share/fonts/google-noto-cjk/NotoSansCJK-Regular.ttc",
- "/usr/share/fonts/TTF/NotoSans-Regular.ttf"],
- $noto_sans, 2);
- }
-
- check_rpm_missing(\@tex_pkgs, 2) if ($pdf);
- check_missing(\%map);
-
- return if (!$need && !$optional);
- printf("You should run:\n") if ($verbose_warn_install);
- printf("\n\tsudo $packager_cmd $install\n");
-}
-
-sub give_arch_linux_hints()
-{
- my %map = (
- "yaml" => "python-yaml",
- "virtualenv" => "python-virtualenv",
- "dot" => "graphviz",
- "convert" => "imagemagick",
- "xelatex" => "texlive-xetex",
- "latexmk" => "texlive-core",
- "rsvg-convert" => "extra/librsvg",
- );
-
- my @archlinux_tex_pkgs = (
- "texlive-core",
- "texlive-latexextra",
- "ttf-dejavu",
- );
- check_pacman_missing(\@archlinux_tex_pkgs, 2) if ($pdf);
-
- if ($pdf) {
- check_missing_file(["/usr/share/fonts/noto-cjk/NotoSansCJK-Regular.ttc"],
- "noto-fonts-cjk", 2);
- }
-
- check_missing(\%map);
-
- return if (!$need && !$optional);
- printf("You should run:\n") if ($verbose_warn_install);
- printf("\n\tsudo pacman -S $install\n");
-}
-
-sub give_gentoo_hints()
-{
- my %map = (
- "yaml" => "dev-python/pyyaml",
- "virtualenv" => "dev-python/virtualenv",
- "dot" => "media-gfx/graphviz",
- "convert" => "media-gfx/imagemagick",
- "xelatex" => "dev-texlive/texlive-xetex media-fonts/dejavu",
- "rsvg-convert" => "gnome-base/librsvg",
- );
-
- check_missing_file(["/usr/share/fonts/dejavu/DejaVuSans.ttf"],
- "media-fonts/dejavu", 2) if ($pdf);
-
- if ($pdf) {
- check_missing_file(["/usr/share/fonts/noto-cjk/NotoSansCJKsc-Regular.otf",
- "/usr/share/fonts/noto-cjk/NotoSerifCJK-Regular.ttc"],
- "media-fonts/noto-cjk", 2);
- }
-
- check_missing(\%map);
-
- return if (!$need && !$optional);
-
- printf("You should run:\n") if ($verbose_warn_install);
- printf("\n");
-
- my $imagemagick = "media-gfx/imagemagick svg png";
- my $cairo = "media-gfx/graphviz cairo pdf";
- my $portage_imagemagick = "/etc/portage/package.use/imagemagick";
- my $portage_cairo = "/etc/portage/package.use/graphviz";
-
- if (qx(grep imagemagick $portage_imagemagick 2>/dev/null) eq "") {
- printf("\tsudo su -c 'echo \"$imagemagick\" > $portage_imagemagick'\n")
- }
- if (qx(grep graphviz $portage_cairo 2>/dev/null) eq "") {
- printf("\tsudo su -c 'echo \"$cairo\" > $portage_cairo'\n");
- }
-
- printf("\tsudo emerge --ask $install\n");
-
-}
-
-sub check_distros()
-{
- # Distro-specific hints
- if ($system_release =~ /Red Hat Enterprise Linux/) {
- give_redhat_hints;
- return;
- }
- if ($system_release =~ /CentOS/) {
- give_redhat_hints;
- return;
- }
- if ($system_release =~ /Scientific Linux/) {
- give_redhat_hints;
- return;
- }
- if ($system_release =~ /Oracle Linux Server/) {
- give_redhat_hints;
- return;
- }
- if ($system_release =~ /Fedora/) {
- give_redhat_hints;
- return;
- }
- if ($system_release =~ /Ubuntu/) {
- give_debian_hints;
- return;
- }
- if ($system_release =~ /Debian/) {
- give_debian_hints;
- return;
- }
- if ($system_release =~ /openSUSE/) {
- give_opensuse_hints;
- return;
- }
- if ($system_release =~ /Mageia/) {
- give_mageia_hints;
- return;
- }
- if ($system_release =~ /OpenMandriva/) {
- give_mageia_hints;
- return;
- }
- if ($system_release =~ /Arch Linux/) {
- give_arch_linux_hints;
- return;
- }
- if ($system_release =~ /Gentoo/) {
- give_gentoo_hints;
- return;
- }
-
- #
- # Fall-back to generic hint code for other distros
- # That's far from ideal, specially for LaTeX dependencies.
- #
- my %map = (
- "sphinx-build" => "sphinx"
- );
- check_missing_tex(2) if ($pdf);
- check_missing(\%map);
- print "I don't know distro $system_release.\n";
- print "So, I can't provide you a hint with the install procedure.\n";
- print "There are likely missing dependencies.\n";
-}
-
-#
-# Common dependencies
-#
-
-sub deactivate_help()
-{
- printf "\n If you want to exit the virtualenv, you can use:\n";
- printf "\tdeactivate\n";
-}
-
-sub get_virtenv()
-{
- my $ver;
- my $min_activate = "$ENV{'PWD'}/${virtenv_prefix}${min_version}/bin/activate";
- my @activates = glob "$ENV{'PWD'}/${virtenv_prefix}*/bin/activate";
-
- @activates = sort {$b cmp $a} @activates;
-
- foreach my $f (@activates) {
- next if ($f lt $min_activate);
-
- my $sphinx_cmd = $f;
- $sphinx_cmd =~ s/activate/sphinx-build/;
- next if (! -f $sphinx_cmd);
-
- my $ver = get_sphinx_version($sphinx_cmd);
-
- if (!$ver) {
- $f =~ s#/bin/activate##;
- print("Warning: virtual environment $f is not working.\nPython version upgrade? Remove it with:\n\n\trm -rf $f\n\n");
- }
-
- if ($need_sphinx && ($ver ge $min_version)) {
- return ($f, $ver);
- } elsif ($ver gt $cur_version) {
- return ($f, $ver);
- }
- }
- return ("", "");
-}
-
-sub recommend_sphinx_upgrade()
-{
- my $venv_ver;
-
- # Avoid running sphinx-builds from venv if $cur_version is good
- if ($cur_version && ($cur_version ge $rec_version)) {
- $latest_avail_ver = $cur_version;
- return;
- }
-
- # Get the highest version from sphinx_*/bin/sphinx-build and the
- # corresponding command to activate the venv/virtenv
- ($activate_cmd, $venv_ver) = get_virtenv();
-
- # Store the highest version from Sphinx existing virtualenvs
- if (($activate_cmd ne "") && ($venv_ver gt $cur_version)) {
- $latest_avail_ver = $venv_ver;
- } else {
- $latest_avail_ver = $cur_version if ($cur_version);
- }
-
- # As we don't know package version of Sphinx, and there's no
- # virtual environments, don't check if upgrades are needed
- if (!$virtualenv) {
- return if (!$latest_avail_ver);
- }
-
- # Either there are already a virtual env or a new one should be created
- $need_pip = 1;
-
- return if (!$latest_avail_ver);
-
- # Return if the reason is due to an upgrade or not
- if ($latest_avail_ver lt $rec_version) {
- $rec_sphinx_upgrade = 1;
- }
-
- return $latest_avail_ver;
-}
-
-#
-# The logic here is complex, as it have to deal with different versions:
-# - minimal supported version;
-# - minimal PDF version;
-# - recommended version.
-# It also needs to work fine with both distro's package and venv/virtualenv
-sub recommend_sphinx_version($)
-{
- my $virtualenv_cmd = shift;
-
- # Version is OK. Nothing to do.
- if ($cur_version && ($cur_version ge $rec_version)) {
- return;
- };
-
- if (!$need_sphinx) {
- # sphinx-build is present and its version is >= $min_version
-
- #only recommend enabling a newer virtenv version if makes sense.
- if ($latest_avail_ver gt $cur_version) {
- printf "\nYou may also use the newer Sphinx version $latest_avail_ver with:\n";
- printf "\tdeactivate\n" if ($ENV{'PWD'} =~ /${virtenv_prefix}/);
- printf "\t. $activate_cmd\n";
- deactivate_help();
-
- return;
- }
- return if ($latest_avail_ver ge $rec_version);
- }
-
- if (!$virtualenv) {
- # No sphinx either via package or via virtenv. As we can't
- # Compare the versions here, just return, recommending the
- # user to install it from the package distro.
- return if (!$latest_avail_ver);
-
- # User doesn't want a virtenv recommendation, but he already
- # installed one via virtenv with a newer version.
- # So, print commands to enable it
- if ($latest_avail_ver gt $cur_version) {
- printf "\nYou may also use the Sphinx virtualenv version $latest_avail_ver with:\n";
- printf "\tdeactivate\n" if ($ENV{'PWD'} =~ /${virtenv_prefix}/);
- printf "\t. $activate_cmd\n";
- deactivate_help();
-
- return;
- }
- print "\n";
- } else {
- $need++ if ($need_sphinx);
- }
-
- # Suggest newer versions if current ones are too old
- if ($latest_avail_ver && $latest_avail_ver ge $min_version) {
- # If there's a good enough version, ask the user to enable it
- if ($latest_avail_ver ge $rec_version) {
- printf "\nNeed to activate Sphinx (version $latest_avail_ver) on virtualenv with:\n";
- printf "\t. $activate_cmd\n";
- deactivate_help();
-
- return;
- }
-
- # Version is above the minimal required one, but may be
- # below the recommended one. So, print warnings/notes
-
- if ($latest_avail_ver lt $rec_version) {
- print "Warning: It is recommended at least Sphinx version $rec_version.\n";
- }
- }
-
- # At this point, either it needs Sphinx or upgrade is recommended,
- # both via pip
-
- if ($rec_sphinx_upgrade) {
- if (!$virtualenv) {
- print "Instead of install/upgrade Python Sphinx pkg, you could use pip/pypi with:\n\n";
- } else {
- print "To upgrade Sphinx, use:\n\n";
- }
- } else {
- print "\nSphinx needs to be installed either:\n1) via pip/pypi with:\n\n";
- }
-
- $python_cmd = find_python_no_venv();
-
- printf "\t$virtualenv_cmd $virtenv_dir\n";
-
- printf "\t. $virtenv_dir/bin/activate\n";
- printf "\tpip install -r $requirement_file\n";
- deactivate_help();
-
- printf "\n2) As a package with:\n";
-
- my $old_need = $need;
- my $old_optional = $optional;
- %missing = ();
- $pdf = 0;
- $optional = 0;
- $install = "";
- $verbose_warn_install = 0;
-
- add_package("python-sphinx", 0);
-
- check_distros();
-
- $need = $old_need;
- $optional = $old_optional;
-
- printf "\n Please note that Sphinx >= 3.0 will currently produce false-positive\n";
- printf " warning when the same name is used for more than one type (functions,\n";
- printf " structs, enums,...). This is known Sphinx bug. For more details, see:\n";
- printf "\thttps://github.com/sphinx-doc/sphinx/pull/8313\n";
-}
-
-sub check_needs()
-{
- # Check if Sphinx is already accessible from current environment
- check_sphinx();
-
- if ($system_release) {
- print "Detected OS: $system_release.\n";
- } else {
- print "Unknown OS\n";
- }
- printf "Sphinx version: %s\n\n", $cur_version if ($cur_version);
-
- # Check python command line, trying first python3
- $python_cmd = findprog("python3");
- $python_cmd = check_program("python", 0) if (!$python_cmd);
-
- # Check the type of virtual env, depending on Python version
- if ($python_cmd) {
- if ($virtualenv) {
- my $tmp = qx($python_cmd --version 2>&1);
- if ($tmp =~ m/(\d+\.)(\d+\.)/) {
- if ($1 < 3) {
- # Fail if it finds python2 (or worse)
- die "Python 3 is required to build the kernel docs\n";
- }
- if ($1 == 3 && $2 < 3) {
- # Need Python 3.3 or upper for venv
- $need_virtualenv = 1;
- }
- } else {
- die "Warning: couldn't identify $python_cmd version!";
- }
- } else {
- add_package("python-sphinx", 0);
- }
- }
-
- my $venv_ver = recommend_sphinx_upgrade();
-
- my $virtualenv_cmd;
-
- if ($need_pip) {
- # Set virtualenv command line, if python < 3.3
- if ($need_virtualenv) {
- $virtualenv_cmd = findprog("virtualenv-3");
- $virtualenv_cmd = findprog("virtualenv-3.5") if (!$virtualenv_cmd);
- if (!$virtualenv_cmd) {
- check_program("virtualenv", 0);
- $virtualenv_cmd = "virtualenv";
- }
- } else {
- $virtualenv_cmd = "$python_cmd -m venv";
- check_python_module("ensurepip", 0);
- }
- }
-
- # Check for needed programs/tools
- check_perl_module("Pod::Usage", 0);
- check_python_module("yaml", 0);
- check_program("make", 0);
- check_program("gcc", 0);
- check_program("dot", 1);
- check_program("convert", 1);
-
- # Extra PDF files - should use 2 for is_optional
- check_program("xelatex", 2) if ($pdf);
- check_program("rsvg-convert", 2) if ($pdf);
- check_program("latexmk", 2) if ($pdf);
-
- # Do distro-specific checks and output distro-install commands
- check_distros();
-
- if (!$python_cmd) {
- if ($need == 1) {
- die "Can't build as $need mandatory dependency is missing";
- } elsif ($need) {
- die "Can't build as $need mandatory dependencies are missing";
- }
- }
-
- # Check if sphinx-build is called sphinx-build-3
- if ($need_symlink) {
- printf "\tsudo ln -sf %s /usr/bin/sphinx-build\n\n",
- which("sphinx-build-3");
- }
-
- recommend_sphinx_version($virtualenv_cmd);
- printf "\n";
-
- print "All optional dependencies are met.\n" if (!$optional);
-
- if ($need == 1) {
- die "Can't build as $need mandatory dependency is missing";
- } elsif ($need) {
- die "Can't build as $need mandatory dependencies are missing";
- }
-
- print "Needed package dependencies are met.\n";
-}
-
-#
-# Main
-#
-
-while (@ARGV) {
- my $arg = shift(@ARGV);
-
- if ($arg eq "--no-virtualenv") {
- $virtualenv = 0;
- } elsif ($arg eq "--no-pdf"){
- $pdf = 0;
- } elsif ($arg eq "--version-check"){
- $version_check = 1;
- } else {
- print "Usage:\n\t$0 <--no-virtualenv> <--no-pdf> <--version-check>\n\n";
- print "Where:\n";
- print "\t--no-virtualenv\t- Recommend installing Sphinx instead of using a virtualenv\n";
- print "\t--version-check\t- if version is compatible, don't check for missing dependencies\n";
- print "\t--no-pdf\t- don't check for dependencies required to build PDF docs\n\n";
- exit -1;
- }
-}
-
-#
-# Determine the system type. There's no standard unique way that would
-# work with all distros with a minimal package install. So, several
-# methods are used here.
-#
-# By default, it will use lsb_release function. If not available, it will
-# fail back to reading the known different places where the distro name
-# is stored
-#
-
-$system_release = qx(lsb_release -d) if which("lsb_release");
-$system_release =~ s/Description:\s*// if ($system_release);
-$system_release = catcheck("/etc/system-release") if !$system_release;
-$system_release = catcheck("/etc/redhat-release") if !$system_release;
-$system_release = catcheck("/etc/lsb-release") if !$system_release;
-$system_release = catcheck("/etc/gentoo-release") if !$system_release;
-
-# This seems more common than LSB these days
-if (!$system_release) {
- my %os_var;
- if (open IN, "cat /etc/os-release|") {
- while (<IN>) {
- if (m/^([\w\d\_]+)=\"?([^\"]*)\"?\n/) {
- $os_var{$1}=$2;
- }
- }
- $system_release = $os_var{"NAME"};
- if (defined($os_var{"VERSION_ID"})) {
- $system_release .= " " . $os_var{"VERSION_ID"} if (defined($os_var{"VERSION_ID"}));
- } else {
- $system_release .= " " . $os_var{"VERSION"};
- }
- }
-}
-$system_release = catcheck("/etc/issue") if !$system_release;
-$system_release =~ s/\s+$//;
-
-check_needs;
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 36/39] scripts: sphinx-pre-install: update mandatory system deps
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (34 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 35/39] scripts: sphinx-pre-install.pl: get rid of the old script Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 37/39] scripts: sphinx-pre-install: add support for RHEL8-based distros Mauro Carvalho Chehab
` (3 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
Bill Wendling, Justin Stitt, Nathan Chancellor, Nick Desaulniers,
linux-kernel, llvm
To build docs, gcc is not needed. Also, Kernel can be built
nowadays with clang. So, drop it.
On the other hand, which is needed. Add a system dependency
for it.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index 7dfe5c2a6cc2..fc9dc45054d7 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -1476,7 +1476,7 @@ class SphinxDependencyChecker(MissingCheckers):
self.check_perl_module("Pod::Usage", DepManager.SYSTEM_MANDATORY)
self.check_program("make", DepManager.SYSTEM_MANDATORY)
- self.check_program("gcc", DepManager.SYSTEM_MANDATORY)
+ self.check_program("which", DepManager.SYSTEM_MANDATORY)
self.check_program("dot", DepManager.SYSTEM_OPTIONAL)
self.check_program("convert", DepManager.SYSTEM_OPTIONAL)
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 37/39] scripts: sphinx-pre-install: add support for RHEL8-based distros
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (35 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 36/39] scripts: sphinx-pre-install: update mandatory system deps Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 38/39] scripts: sphinx-pre-install: add a warning for Debian-based distros Mauro Carvalho Chehab
` (2 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
On RHEL8, only installing with a venv is supported, as there's
no Sphinx package using Python 3.7 or upper.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install | 25 ++++++++++++++++---------
1 file changed, 16 insertions(+), 9 deletions(-)
diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index fc9dc45054d7..324baa98a395 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -691,7 +691,7 @@ class SphinxDependencyChecker(MissingCheckers):
super().__init__(args, texlive)
- self.need_pip = 0
+ self.need_pip = False
self.rec_sphinx_upgrade = 0
self.system_release = self.get_system_release()
@@ -840,9 +840,15 @@ class SphinxDependencyChecker(MissingCheckers):
# RHEL 8 uses Python 3.6, which is not compatible with
# the build system anymore. Suggest Python 3.11
if rel == 8:
- self.deps.add_package("python39", DepManager.SYSTEM_MANDATORY)
+ self.check_program("python3.9", DepManager.SYSTEM_MANDATORY)
+ progs["python3.9"] = "python39"
+ progs["yaml"] = "python39-pyyaml"
+
self.recommend_python = True
+ # There's no python39-sphinx package. Only pip is supported
+ self.package_supported = False
+
if not self.distro_msg:
self.distro_msg = \
"Note: RHEL-based distros typically require extra repositories.\n" \
@@ -915,8 +921,9 @@ class SphinxDependencyChecker(MissingCheckers):
# the build system anymore. Suggest Python 3.11
if rel == 15:
if not self.which(self.python_cmd):
+ self.check_program("python3.11", DepManager.SYSTEM_MANDATORY)
+ progs["python3.11"] = "python311"
self.recommend_python = True
- self.deps.add_package(self.python_cmd, DepManager.SYSTEM_MANDATORY)
progs.update({
"python-sphinx": "python311-Sphinx",
@@ -1289,7 +1296,7 @@ class SphinxDependencyChecker(MissingCheckers):
return self.latest_avail_ver
# Either there are already a virtual env or a new one should be created
- self.need_pip = 1
+ self.need_pip = True
if not self.latest_avail_ver:
return None
@@ -1344,10 +1351,11 @@ class SphinxDependencyChecker(MissingCheckers):
"""
if self.recommend_python:
- print("\nPython version is incompatible with doc build.\n" \
- "Please upgrade it and re-run.\n")
- return
-
+ cur_ver = sys.version_info[:3]
+ if cur_ver < MIN_PYTHON_VERSION:
+ print(f"\nPython version {cur_ver} is incompatible with doc build.\n" \
+ "Please upgrade it and re-run.\n")
+ return
# Version is OK. Nothing to do.
if self.cur_version != (0, 0, 0) and self.cur_version >= RECOMMENDED_VERSION:
@@ -1467,7 +1475,6 @@ class SphinxDependencyChecker(MissingCheckers):
if sys.version_info < MIN_PYTHON_VERSION:
self.need_pip = False
print("Warning: python version is not supported.")
-
else:
virtualenv_cmd = f"{self.python_cmd} -m venv"
self.check_python_module("ensurepip")
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 38/39] scripts: sphinx-pre-install: add a warning for Debian-based distros
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (36 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 37/39] scripts: sphinx-pre-install: add support for RHEL8-based distros Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 39/39] scripts: sphinx-pre-install: some adjustments related to venv Mauro Carvalho Chehab
2025-08-13 16:23 ` [PATCH v2 00/39] Translate sphinx-pre-install to Python Jonathan Corbet
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
On Some Debian-based distros, ImageMagick package has a broken
policy that causes LaTeX to fail while building docs.
Add a note about that.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index 324baa98a395..09a337509b23 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -782,6 +782,11 @@ class SphinxDependencyChecker(MissingCheckers):
self.check_program("dvipng", DepManager.PDF_MANDATORY)
+ if not self.distro_msg:
+ self.distro_msg = \
+ "Note: ImageMagick is broken on some distros, affecting PDF output. For more details:\n" \
+ "\thttps://askubuntu.com/questions/1158894/imagemagick-still-broken-using-with-usr-bin-convert"
+
return self.get_install_progs(progs, "apt-get install")
def give_redhat_hints(self):
@@ -1193,7 +1198,7 @@ class SphinxDependencyChecker(MissingCheckers):
self.distro_msg = \
f"I don't know distro {self.system_release}.\n" \
"So, I can't provide you a hint with the install procedure.\n" \
- "There are likely missing dependencies.\n"
+ "There are likely missing dependencies."
return self.get_install_progs(progs, None)
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 39/39] scripts: sphinx-pre-install: some adjustments related to venv
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (37 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 38/39] scripts: sphinx-pre-install: add a warning for Debian-based distros Mauro Carvalho Chehab
@ 2025-08-12 15:52 ` Mauro Carvalho Chehab
2025-08-13 16:23 ` [PATCH v2 00/39] Translate sphinx-pre-install to Python Jonathan Corbet
39 siblings, 0 replies; 42+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-12 15:52 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
While nothing was really needed for virtualenv to work on most
distros, we had an issue with OpenMandriva.
While checking for it, it was noticed that there was no check if
python-virtualenv was installed.
This didn't solve the issues we faced there: at least with
the half-broken OpenMandriva Lx 4.0 docker container we used,
ensurepip was not available anywhere, causing venv to fail.
Add a distro-specific note about that.
Note: at least at the time we did our tests, OpenMandriva Lx 4.0
docker was shipped with wrong dnf repositories. Also, there
was no repos available for it anymore. So, we had to do some
hacks to upgrade to 4.3 before being able to run any tests.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index 09a337509b23..b8474848df4e 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -983,6 +983,22 @@ class SphinxDependencyChecker(MissingCheckers):
# Tested on OpenMandriva Lx 4.3
progs["convert"] = "imagemagick"
progs["yaml"] = "python-pyyaml"
+ progs["python-virtualenv"] = "python-virtualenv"
+ progs["python-sphinx"] = "python-sphinx"
+
+ self.check_program("python-virtualenv", DepManager.PYTHON_MANDATORY)
+
+ # On my tests with openMandriva LX 4.0 docker image, upgraded
+ # to 4.3, python-virtualenv package is broken: it is missing
+ # ensurepip. Without it, the alternative would be to run:
+ # python3 -m venv --without-pip ~/sphinx_latest, but running
+ # pip there won't install sphinx at venv.
+ #
+ # Add a note about that.
+
+ if not self.distro_msg:
+ self.distro_msg = \
+ "Note: for venv, ensurepip could be broken, preventing its install method."
else:
packager_cmd = "urpmi"
--
2.50.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* Re: [PATCH v2 00/39] Translate sphinx-pre-install to Python
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
` (38 preceding siblings ...)
2025-08-12 15:52 ` [PATCH v2 39/39] scripts: sphinx-pre-install: some adjustments related to venv Mauro Carvalho Chehab
@ 2025-08-13 16:23 ` Jonathan Corbet
39 siblings, 0 replies; 42+ messages in thread
From: Jonathan Corbet @ 2025-08-13 16:23 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Bill Wendling, Justin Stitt, Nathan Chancellor, Nick Desaulniers,
llvm
Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
> Hi Jon,
>
> that's the second version of the patch series which converts
> sphinx-pre-install to Python.
>
> The core patches are basically the same as on v1, but it has lots of
> fixes over the original script after testing sphinx-install on 22
> distros.
Whee ... I've applied the set, thanks.
jon
^ permalink raw reply [flat|nested] 42+ messages in thread