Re: [PATCH 00/11] Fix PDF doc builds on major distros

[Mauro Carvalho Chehab <mchehab+huawei@kernel.org>]

Em Tue, 19 Aug 2025 11:16:29 +0900
Akira Yokosawa <akiyks@gmail.com> escreveu:

> Hi Mauro,
> 
> Please calm down.
> Apologizes if my wording has annoyed you.
> 
> On Tue, 19 Aug 2025 03:02:39 +0200, Mauro Carvalho Chehab wrote:
> > Em Tue, 19 Aug 2025 08:26:31 +0900
> > Akira Yokosawa <akiyks@gmail.com> escreveu:  
> 
> [...]
> 
> >> Mauro, I can't review on 5/11 unless you provide me exact steps to reproduce
> >> the font discovery issue you said you have observed under debian at 4/11 of
> >> this series.  That is, without assuming your other series of build-wrapper.  
> > 
> > See below.
> >   
> >> The build-wrapper should be upper compatible with the current way of
> >> running sub-make, without any change in conf.py.  
> > 
> > The build-wrapper series doesn't make any changes on conf.py:  
> 
> Of course not, I'm well aware of the fact from the cover letter.
> 
> [...]
> 
> > 4. It prevents the usage of T1 fontenc fonts, which could be
> >    caused by either one of those two reasons:
> > 
> > 	a) \sphinxhyphen{}
> > 	b) index build
> > 
> > I got those when checking what packages were required on some
> > distros (Debian, Ubuntu, Mageia, openMandriva, Gentoo).  
> 
> I couldn't reproduce it at PATCH 4/11 under a Ubnutu-based container
> I've been using, ignoring your sphinx-build-warpper series.

I'm assuming you're referring to patch 5/11, e.g.:

	ceaa95123364 ("docs: conf.py: fix some troubles for LaTeX output")

> I'd really like to see the issue with my eyes on top current way of
> doc building!
> 
> That's all I am asking.
> 
> I have no particular interest in the way you are testing all those
> variety of distros, and have no idea about it.
> 
> Please provide steps to reproduce.
> I'd like to see a Dockerfile based "FROM ubuntu:latest" as the testing
> environment.

Heh, that's why you can't reproduce... you're using a Docker container
and a different Ubuntu version than I.

What I did here was to use a lxc container with 25.04, and used
a script to pick the exact recommendations given by sphinx-pre-install.

The container was created with:

<script>
sudo lxc-create -n ubuntu-test -t download -- --dist ubuntu --release plucky --arch amd64
sudo mkdir -p /var/lib/lxc/ubuntu-test
sudo sh -c cat > /var/lib/lxc/ubuntu-test/config <<EOF
lxc.arch = x86_64
lxc.pty.max = 1024
lxc.tty.max = 4
lxc.net.0.type = veth
lxc.net.0.link = lxcbr0
lxc.net.0.flags = up
lxc.net.0.hwaddr = 00:16:3e:6f:66:8c
lxc.cap.drop = mac_admin mac_override sys_time sys_module sys_rawio
lxc.cgroup.devices.allow = a
lxc.cgroup2.devices.allow = a
lxc.mount.auto = proc sys cgroup
lxc.rootfs.path = /var/lib/lxc/ubuntu-test/rootfs
EOF
</script>

once you get what's needed for make htmldocs work, installing
only the dependencies recommended by sphinx-pre-install, you can
start trying to reproduce the build issues for pdf.

For the steps below, I installed Sphinx latest using venv:

	$ make SPHINXDIRS=peci pdfdocs
	...
	This is XeTeX, Version 3.141592653-2.6-0.999996 (TeX Live 2025/dev/Debian) (preloaded format=xelatex)
	entering extended mode
	Latexmk: Getting log file 'peci.log'
	Latexmk: Examining 'peci.fls'
	Latexmk: Examining 'peci.log'
	Latexmk: Missing input file 'tgtermes.sty' (or dependence on it) from following:
	  ! LaTeX Error: File `tgtermes.sty' not found.
	Latexmk: Log file says no output from latex
	Latexmk: For rule 'xelatex', no output was made
	Latexmk: Errors, so I did not complete making targets
	Collected error summary (may duplicate other messages):
	  xelatex: Command for 'xelatex' gave return code 1
	      Refer to 'peci.log' and/or above output for details

	Latexmk: Sometimes, the -f option can be used to get latexmk
	  to try to force complete processing.
	  But normally, you will need to correct the file(s) that caused the
	  error, and then rerun latexmk.
	  In some cases, it is best to clean out generated files before rerunning
	  latexmk after you've corrected the files.
	make[3]: *** [Makefile:29: peci.pdf] Error 12
	make[2]: *** [Documentation/Makefile:148: pdfdocs] Error 1
	make[1]: *** [/root/Makefile:1806: pdfdocs] Error 2
	make: *** [Makefile:248: __sub-make] Error 2

Fix the broken font dependencies require installing other packages:

	# apt install tex-gyre texlive-fonts-recommended texlive-lang-chinese
	texlive-lang-chinese is already the newest version (2024.20250309-1).
	Installing:
	  tex-gyre  texlive-fonts-recommended
	
	Installing dependencies:
	  fonts-texgyre  fonts-texgyre-math

As described at sphinx-pre-install patch:

  commit a888d353ac1eb0d558fda5482c6955d6c4a29f23
  Author: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
  Date:   Mon Jul 28 15:54:09 2025 +0200

    scripts: sphinx-pre-install: fix PDF build issues on Ubuntu
    
    PDF output with current Debian-based distros require other
    packages for it to work. The main one is pzdr.tfm. Without
    that, \sphinxhyphen{} won't work, affecting multiple docs.
    
    For CJK, tex-gyre is required.
    
    Add the missing packages to the list.

Now, here's the thing: to get the above dependencies, I played with 
installing/uninstalling multiple packages, until I discovered where
each font dependency was, and where CJK fonts were placed.

Still, the above is not enough:

	$ make cleandocs; LANG=C LC_ALL=C make SPHINXDIRS=peci pdfdocs
	...
	entering extended mode
	Latexmk: Getting log file 'peci.log'
	Latexmk: Examining 'peci.fls'
	Latexmk: Examining 'peci.log'
	Latexmk: Index file 'peci.idx' was written
	Latexmk: References changed.
	Latexmk: Missing input file 'peci.toc' (or dependence on it) from following:
	  No file peci.toc.
	Latexmk: Missing input file 'peci.ind' (or dependence on it) from following:
	  No file peci.ind.
	Latexmk: References changed.
	Latexmk: References changed.
	Latexmk: Log file says output to 'peci.xdv'
	Have index file 'peci.idx', peci.ind peci
	Latexmk: Errors, so I did not complete making targets
	Collected error summary (may duplicate other messages):
	  xelatex: Command for 'xelatex' gave return code 1
      Refer to 'peci.log' and/or above output for details
	
	Latexmk: Sometimes, the -f option can be used to get latexmk
	  to try to force complete processing.
	  But normally, you will need to correct the file(s) that caused the
	  error, and then rerun latexmk.
	  In some cases, it is best to clean out generated files before rerunning
	  latexmk after you've corrected the files.
	make[3]: *** [Makefile:29: peci.pdf] Error 12
	make[2]: *** [Documentation/Makefile:148: pdfdocs] Error 1
	make[1]: *** [/root/Makefile:1806: pdfdocs] Error 2
	make: *** [Makefile:248: __sub-make] Error 2

	$ ls Documentation/output/peci/latex/|grep pdf
	<none>

PS.: such error is intermitent: re-creating containers sometimes
fix the issues; sometimes not. It sounds to be that this is due
to some race condition somewhere.

Also, if you look at the *.tex output, you'll see it is messing with
fonts:

	# grep usepackage Documentation/output/peci/latex/peci.tex 
	\usepackage{cmap}
	\usepackage[T1]{fontenc}
	\usepackage{amsmath,amssymb,amstext}
	\usepackage{babel}
	\usepackage{tgtermes}
	\usepackage{tgheros}
	\usepackage[Bjarne]{fncychap}
	\usepackage[,maxlistdepth=10]{sphinx}
	\usepackage{geometry}
	\usepackage{setspace}
	\usepackage{hyperref}
	\usepackage{hypcap}% it must be loaded after hyperref.
	\usepackage{sphinxmessages}
	        \usepackage{fontspec}

e.g.:

- The tex file is trying to use T1 fontenc and fontspec;

- The fontspec usage came from conf.py preamble, which is not indented
  well:

	# grep -A7 fontspec Documentation/output/peci/latex/peci.tex
	        \usepackage{fontspec}
	        \setsansfont{DejaVu Sans}
	        \setromanfont{DejaVu Serif}
	        \setmonofont{DejaVu Sans Mono}
    
	        % Load kerneldoc specific LaTeX settings
	        \input{kerneldoc-preamble.sty}

On several of my attempts, I got those errors:

	Package: fontenc 2021/04/29 v2.0v Standard LaTeX package
	LaTeX Font Info:    Trying to load font information for T1+lmr on input line 11
	6.
	LaTeX Font Info:    No file T1lmr.fd. on input line 116.

	LaTeX Font Warning: Font shape `T1/lmr/m/n' undefined
	(Font)              using `T1/lmr/m/n' instead on input line 116.

	! Corrupted NFSS tables.
	wrong@fontshape ...message {Corrupted NFSS tables}
                                                  error@fontshape else let f...
	l.116 ...\familydefault\seriesdefault\shapedefault

If you search for it, you would get references like:
	https://tex.stackexchange.com/questions/740050/corrupted-nfss-tables

explaining that using T1 together with fontenc is a very bad idea,
recommending to use instead:

	\usepackage{fontspec}
	\newfontfamily<sometthing>

and not use T1 fontenc.

Unfortunately, I was not able to reproduce this exact error anymore,
at least on Ubuntu. Perhaps I got this one on Gentoo or on some other
distro, but I can't remember anymore how to reproduce this specific
error message.

-

Yet, using my today's container, this specific scenario can be fixed by moving
the font config data from preamble to "fontenc":

<patch>
+from textwrap import dedent
+
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
@@ -441,20 +443,18 @@ latex_elements = {
     # For CJK One-half spacing, need to be in front of hyperref
     "extrapackages": r"\usepackage{setspace}",
     # Additional stuff for the LaTeX preamble.
-    "preamble": """
+    "fontenc": dedent("""
         % Use some font with UTF-8 support with XeLaTeX
         \\usepackage{fontspec}
         \\setsansfont{DejaVu Sans}
         \\setromanfont{DejaVu Serif}
         \\setmonofont{DejaVu Sans Mono}
-    """,
-}
-
-# Load kerneldoc specific LaTeX settings
-latex_elements["preamble"] += """
+    """),
+    "preamble": dedent("""
         % Load kerneldoc specific LaTeX settings
         \\input{kerneldoc-preamble.sty}
-"""
+    """)
+}
</patch> 

which is one of the main changes I made. At the actual change, I opted to
use raw strings, to avoid "\\" e.g.:

	"fontenc": dedent(r"""
             % Use some font with UTF-8 support with XeLaTeX
             \usepackage{fontspec}
             \setsansfont{DejaVu Sans}
             \setromanfont{DejaVu Serif}
             \setmonofont{DejaVu Sans Mono}
	"""),

With that, we have now:

	# grep usepackage Documentation/output/peci/latex/*.tex
	\usepackage{cmap}
	\usepackage{fontspec}
	\usepackage{amsmath,amssymb,amstext}
	\usepackage{babel}
	\usepackage{tgtermes}
	\usepackage{tgheros}
	\usepackage[Bjarne]{fncychap}
	\usepackage[,maxlistdepth=10]{sphinx}
	\usepackage{geometry}
	\usepackage{setspace}
	\usepackage{hyperref}
	\usepackage{hypcap}% it must be loaded after hyperref.
	\usepackage{sphinxmessages}

However, on other tests, including other distros, I found that this
is not enough, because:

1) the *.ind creation may use xindy on some distros and xindy could try
   to use T1 fontenc. So, I had to add this line:

	\PassOptionsToPackage{xindy}{language=english,codepage=utf8,noautomatic}

  to ensure that, if xindy is used, it will get codepage=utf8. 

2) there were some other corner cases that required to add 

	\newfontfamily\headingfont{DejaVu Serif}

Again related to T1 fontenc.

Thanks,
Mauro