From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
To: Linux Doc Mailing List <linux-doc@vger.kernel.org>
Cc: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>,
"Jonathan Corbet" <corbet@lwn.net>,
"Mauro Carvalho Chehab" <mchehab+huawei@kernel.org>,
linux-kernel@vger.kernel.org
Subject: [PATCH v2 19/39] scripts: sphinx-pre-install: improve Gentoo package deps logic
Date: Tue, 12 Aug 2025 17:52:36 +0200 [thread overview]
Message-ID: <365fe5e7d568da932dcffde65f48f2c1256cb773.1754992972.git.mchehab+huawei@kernel.org> (raw)
In-Reply-To: <cover.1754992972.git.mchehab+huawei@kernel.org>
It took me a lot of time, but I guess understand now what it
takes to install a package on Gentoo.
Handling dependencies is a nightmare, as Gentoo refuses to emerge
some packages if there's no package.use file describing them.
To make it worse, compilation flags shall also be present there
for some packages. If USE is not perfect, error/warning messages
like those are shown:
gnome-base/librsvg dev-texlive/texlive-xetex media-fonts/dejavu dev-python/pyyaml
...
!!! The following binary packages have been ignored due to non matching USE:
=media-gfx/graphviz-12.2.1-r1 X pdf -python_single_target_python3_13 qt6 svg
=media-gfx/graphviz-12.2.1-r1 X pdf python_single_target_python3_12 -python_single_target_python3_13 qt6 svg
=media-gfx/graphviz-12.2.1-r1 X pdf qt6 svg
=media-gfx/graphviz-12.2.1-r1 X pdf -python_single_target_python3_10 qt6 svg
=media-gfx/graphviz-12.2.1-r1 X pdf -python_single_target_python3_10 python_single_target_python3_12 -python_single_target_python3_13 qt6 svg
=media-fonts/noto-cjk-20190416 X
=app-text/texlive-core-2024-r1 X cjk -xetex
=app-text/texlive-core-2024-r1 X -xetex
=app-text/texlive-core-2024-r1 -xetex
=dev-libs/zziplib-0.13.79-r1 sdl
If emerge is allowed, it will simply ignore the above packages,
creating an incomplete installation, which will later fail when
one tries to build docs with images or build PDFs.
After the fix, command line commands to produce the needed USE
chain will be emitted, if they don't exist yet.
sudo su -c 'echo "media-gfx/graphviz" > /etc/portage/package.use/graphviz'
sudo su -c 'echo "media-gfx/imagemagick" > /etc/portage/package.use/imagemagick'
sudo su -c 'echo "media-libs/harfbuzz icu" > /etc/portage/package.use/media-libs'
sudo su -c 'echo "media-fonts/noto-cjk" > /etc/portage/package.use/media-fonts'
sudo su -c 'echo "app-text/texlive-core xetex" > /etc/portage/package.use/texlive'
sudo su -c 'echo "dev-libs/zziplib sdl" > /etc/portage/package.use/zziblib'
The new logic tries to be smart enough to detect for missing files
and missing arguments. Yet, as Gentoo seems to require users to
manage those package.use files by hand, the logic isn't perfect:
users may still need to verify for conflicts on different use
files.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/sphinx-pre-install.py | 86 +++++++++++++++++++++++++++++++----
1 file changed, 76 insertions(+), 10 deletions(-)
diff --git a/scripts/sphinx-pre-install.py b/scripts/sphinx-pre-install.py
index cd9c83b17971..94f3d2e32fd6 100755
--- a/scripts/sphinx-pre-install.py
+++ b/scripts/sphinx-pre-install.py
@@ -737,22 +737,88 @@ class SphinxDependencyChecker:
print("You should run:")
print("\n")
+ # Handling dependencies is a nightmare, as Gentoo refuses to emerge
+ # some packages if there's no package.use file describing them.
+ # To make it worse, compilation flags shall also be present there
+ # for some packages. If USE is not perfect, error/warning messages
+ # like those are shown:
+ #
+ # !!! The following binary packages have been ignored due to non matching USE:
+ #
+ # =media-gfx/graphviz-12.2.1-r1 X pdf -python_single_target_python3_13 qt6 svg
+ # =media-gfx/graphviz-12.2.1-r1 X pdf python_single_target_python3_12 -python_single_target_python3_13 qt6 svg
+ # =media-gfx/graphviz-12.2.1-r1 X pdf qt6 svg
+ # =media-gfx/graphviz-12.2.1-r1 X pdf -python_single_target_python3_10 qt6 svg
+ # =media-gfx/graphviz-12.2.1-r1 X pdf -python_single_target_python3_10 python_single_target_python3_12 -python_single_target_python3_13 qt6 svg
+ # =media-fonts/noto-cjk-20190416 X
+ # =app-text/texlive-core-2024-r1 X cjk -xetex
+ # =app-text/texlive-core-2024-r1 X -xetex
+ # =app-text/texlive-core-2024-r1 -xetex
+ # =dev-libs/zziplib-0.13.79-r1 sdl
+ #
+ # And will ignore such packages, installing the remaining ones. That
+ # affects mostly the image extension and PDF generation.
- portages = [
- "media-gfx/imagemagick",
- "media-gfx/graphviz",
- ]
+ # Package dependencies and the minimal needed args:
+ portages = {
+ "graphviz": "media-gfx/graphviz",
+ "imagemagick": "media-gfx/imagemagick",
+ "media-libs": "media-libs/harfbuzz icu",
+ "media-fonts": "media-fonts/noto-cjk",
+ "texlive": "app-text/texlive-core xetex",
+ "zziblib": "dev-libs/zziplib sdl",
+ }
if self.first_hint:
- for p in portages:
- result = self.run(["grep", p, "/etc/portage/package.use/*"],
- stdout=subprocess.PIPE, text=True)
- if not result.stdout.strip():
- print(f"\tsudo emerge -av1 {p}")
+ use_base = "/etc/portage/package.use"
+ files = glob(f"{use_base}/*")
+
+ for fname, portage in portages.items():
+ install = False
+
+ while install == False:
+ if not files:
+ # No files under package.usage. Install all
+ install = True
+ break
+
+ args = portage.split(" ")
+
+ name = args.pop(0)
+
+ cmd = ["grep", "-l", "-E", rf"^{name}\b" ] + files
+ result = self.run(cmd, stdout=subprocess.PIPE, text=True)
+ if result.returncode or not result.stdout.strip():
+ # File containing portage name not found
+ install = True
+ break
+
+ # Ensure that needed USE flags are present
+ if args:
+ match_fname = result.stdout.strip()
+ with open(match_fname, 'r', encoding='utf8',
+ errors='backslashreplace') as fp:
+ for line in fp:
+ for arg in args:
+ if arg.startswith("-"):
+ continue
+
+ if not re.search(rf"\s*{arg}\b", line):
+ # Needed file argument not found
+ install = True
+ break
+
+ # Everything looks ok, don't install
+ break
+
+ # emit a code to setup missing USE
+ if install:
+ print(f"\tsudo su -c 'echo \"{portage}\" > {use_base}/{fname}'")
self.first_hint = False
- print(f"\tsudo emerge --ask {self.install}")
+ # Now, we can use emerge and let it respect USE
+ print(f"\tsudo emerge --ask --changed-use --binpkg-respect-use=y {self.install}")
#
# Dispatch the check to an os_specific hinter
--
2.50.1
next prev parent reply other threads:[~2025-08-12 15:53 UTC|newest]
Thread overview: 42+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-08-12 15:52 [PATCH v2 00/39] Translate sphinx-pre-install to Python Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 01/39] scripts: sphinx-pre-install: fix version check for Fedora Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 02/39] scripts: sphinx-pre-install: rename it to scripts/sphinx-pre-install.pl Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 03/39] scripts: sphinx-pre-install: Convert script to Python Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 04/39] scripts: sphinx-pre-install: Make it compatible with Python 3.6 Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 05/39] scripts: sphinx-pre-install: run on a supported version Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 06/39] scripts: sphinx-pre-install: drop obsolete routines Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 07/39] scripts: sphinx-pre-install: drop support for old virtualenv Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 08/39] scripts: sphinx-pre-install: Address issues with OpenSUSE Leap 15.x Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 09/39] scripts: sphinx-pre-install: fix opensuse Leap hint for PyYAML Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 10/39] scripts: sphinx-pre-install: fix support for gentoo Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 11/39] scripts: sphinx-pre-install: Address issues with OpenSUSE Tumbleweed Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 12/39] scripts: sphinx-pre-install: only show portage hints once Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 13/39] scripts: sphinx-pre-install: cleanup rhel support Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 14/39] scripts: sphinx-pre-install: output Python and docutils version Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 15/39] scripts: sphinx-pre-install: add a missing f-string marker Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 16/39] scripts: sphinx-pre-install: fix Leap support for rsvg-convert Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 17/39] scripts: sphinx-pre-install: fix rhel recomendations Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 18/39] scripts: sphinx-pre-install: remove Scientific Linux Mauro Carvalho Chehab
2025-08-12 15:52 ` Mauro Carvalho Chehab [this message]
2025-08-12 15:52 ` [PATCH v2 20/39] scripts: sphinx-pre-install: fix OpenMandriva support Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 21/39] scripts: sphinx-pre-install: move package instructions to a new func Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 22/39] scripts: sphinx-pre-install: adjust a warning message Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 23/39] scripts: sphinx-pre-install: better handle Python min version Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 24/39] scripts: sphinx-pre-install: convert is_optional to a class Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 25/39] scripts: sphinx-pre-install: better handle RHEL-based distros Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 26/39] scripts: sphinx-pre-install: move missing logic to a separate class Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 27/39] scripts: sphinx-pre-install: move ancillary checkers " Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 28/39] scripts: sphinx-pre-install: add more generic checkers on a class Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 29/39] scripts: sphinx-pre-install: move get_system_release() Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 30/39] scripts: sphinx-pre-install: add documentation for the ancillary classes Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 31/39] scripts: sphinx-pre-install: add docstring documentation Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 32/39] scripts: sphinx-pre-install: fix several codingstyle issues Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 33/39] scripts: sphinx-pre-install: rework install command logic Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 34/39] docs: Makefile: switch to the new scripts/sphinx-pre-install.py Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 35/39] scripts: sphinx-pre-install.pl: get rid of the old script Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 36/39] scripts: sphinx-pre-install: update mandatory system deps Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 37/39] scripts: sphinx-pre-install: add support for RHEL8-based distros Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 38/39] scripts: sphinx-pre-install: add a warning for Debian-based distros Mauro Carvalho Chehab
2025-08-12 15:52 ` [PATCH v2 39/39] scripts: sphinx-pre-install: some adjustments related to venv Mauro Carvalho Chehab
2025-08-13 16:23 ` [PATCH v2 00/39] Translate sphinx-pre-install to Python Jonathan Corbet
-- strict thread matches above, loose matches on Subject: below --
2025-07-09 13:51 Mauro Carvalho Chehab
2025-07-09 13:51 ` [PATCH v2 19/39] scripts: sphinx-pre-install: improve Gentoo package deps logic Mauro Carvalho Chehab
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=365fe5e7d568da932dcffde65f48f2c1256cb773.1754992972.git.mchehab+huawei@kernel.org \
--to=mchehab+huawei@kernel.org \
--cc=corbet@lwn.net \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).