PDF misery

PDF misery

[Jonathan Corbet]

So, FYI, I have Vegard's PDF-generation fix applied here, but I've not
pushed it yet (even though I think it's fine) because I wanted to be
sure that all was well with PDF generation.  Bear with me as I ramble
for a bit...

I'll confess to not paying as much attention to PDF output as I should.
Partly that's from a perception that not many people care about it
(though a few really do), and partly because PDF generation has always
been a bit of a house of cards that turns into a real pain whenever
anybody looks at it hard.

Somebody looked at it hard.

My fairly standard Fedora 39 system can't do a pdfdocs build at all.
There are a couple of untoward things that get in the way.

One is the dreaded Latex "too deeply nested" message that causes
everything to explode while generating filesystems.pdf.  The problem,
specifically, is .../filesystems/xfs/xfs-online-fsck-design.rst, which
blows the limits in two ways.  This patch "fixes" it - this clearly
isn't something to apply, it's just a demonstration of the problem:

-----
diff --git a/Documentation/filesystems/xfs/xfs-online-fsck-design.rst b/Documentation/filesystems/xfs/xfs-online-fsck-design.rst
index 352516feef6f..1762a9415ec1 100644
--- a/Documentation/filesystems/xfs/xfs-online-fsck-design.rst
+++ b/Documentation/filesystems/xfs/xfs-online-fsck-design.rst
@@ -739,15 +739,6 @@ For a given fstests filesystem configuration:
         7. Subtract a small quantity
         8. Randomize the contents
 
-        * ...test the reactions of:
-
-          1. The kernel verifiers to stop obviously bad metadata
-          2. Offline checking (``xfs_repair -n``)
-          3. Offline repair (``xfs_repair``)
-          4. Online checking (``xfs_scrub -n``)
-          5. Online repair (``xfs_scrub``)
-          6. Both repair tools (``xfs_scrub`` and then ``xfs_repair`` if online repair doesn't succeed)
-
 This is quite the combinatoric explosion!
 
 Fortunately, having this much test coverage makes it easy for XFS developers to
@@ -2093,7 +2084,6 @@ of the records, so naturally the xfarray must also support sorting the record
 set prior to bulk loading.
 
 Case Study: Sorting xfarrays
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 The sorting algorithm used in the xfarray is actually a combination of adaptive
 quicksort and a heapsort subalgorithm in the spirit of

-----
The first change removes the deepest level of nesting in an impressive
set of enumerated lists; the second removes the nth-level subsection
heading markup for "Sorting xfarrays".  Having either of those present
will cause Latex to blow a fuse.

I'm surprised that nobody else is reporting this problem.  I honestly
don't see a fix other than changing the organization of the document to
avoid both deeply nested itemized lists and section structure, which is
less than entirely desirable.  I think there are good reasons for
avoiding structures that deep, but limitations in the tooling aren't one
of them.

Incidentally, on F38, I also run into a similar Latex error in
.../userspace-api/media/v4l/pixfmt-rgb.html, which has a set of tables
from hell.  On F39, that somehow works.  Weird.

The *other* problem is that PDF generation of the Chinese, Korean, or
Japanese translations fails with:

  xdvipdfmx:fatal: Invalid TTC index number

This, I am thinking, is likely some sort of F39 bug.  xdvipdfmx is
finding the CJK fonts just fine, but then something clearly goes wrong.
I'll try to find the time to narrow that down and perhaps put in a
proper bug report.

Meanwhile, I wish I knew what to do to make the PDF build a bit more
robust.  I wish that rst2pdf or rinohtype would progress to the point
where they could be used, and Latex could be taken out of the picture,
but no such luck.

Oh well...sorry for rambling at you...I wish I had a good plan for
making this whole thing better.

jon

Re: PDF misery

[Mauro Carvalho Chehab]

Em Sat, 17 Feb 2024 14:29:24 -0700
Jonathan Corbet <corbet@lwn.net> escreveu:

> So, FYI, I have Vegard's PDF-generation fix applied here, but I've not
> pushed it yet (even though I think it's fine) because I wanted to be
> sure that all was well with PDF generation.

Yeah, LaTeX Sphinx output is really tricky, requiring workarounds from
time to time to address some issues.

> The first change removes the deepest level of nesting in an impressive
> set of enumerated lists; the second removes the nth-level subsection
> heading markup for "Sorting xfarrays".  Having either of those present
> will cause Latex to blow a fuse.

...

> I'm surprised that nobody else is reporting this problem.  I honestly
> don't see a fix other than changing the organization of the document to
> avoid both deeply nested itemized lists and section structure, which is
> less than entirely desirable.  I think there are good reasons for
> avoiding structures that deep, but limitations in the tooling aren't one
> of them.

The maximum indentation level seems easily fixable. See my PoC at the
end of this e-mail.

From:
	- https://tex.stackexchange.com/questions/571826/too-deeply-nested
	- https://stackoverflow.com/questions/1935952/maximum-nesting-level-of-lists-in-latex

And some local tests I did on Fedora 39, the default enum indentation
limit is 7.

This is configurable by adding \setlistdepth after using enum item,
e. g.:

	\usepackage{enumitem}
	\setlistdepth{9}

I guess we can add something like that to latex_elements at conf.py to 
increase the maximum limit.

> Incidentally, on F38, I also run into a similar Latex error in
> .../userspace-api/media/v4l/pixfmt-rgb.html, which has a set of tables
> from hell.  On F39, that somehow works.  Weird.

I remember in the past I had to increase some LaTeX memory limits,
but it seems that this is not required anymore.

> 
> The *other* problem is that PDF generation of the Chinese, Korean, or
> Japanese translations fails with:
> 
>   xdvipdfmx:fatal: Invalid TTC index number
> 
> This, I am thinking, is likely some sort of F39 bug.  xdvipdfmx is
> finding the CJK fonts just fine, but then something clearly goes wrong.
> I'll try to find the time to narrow that down and perhaps put in a
> proper bug report.
> 
> Meanwhile, I wish I knew what to do to make the PDF build a bit more
> robust.  I wish that rst2pdf or rinohtype would progress to the point
> where they could be used, and Latex could be taken out of the picture,
> but no such luck.

I haven't tested rst2pdf for a while on Kernel builds. It works fine
when producing IGT documentation, but the docs there are too trivial
to cause issues.

Didn't try rinohtype yet.

Ideally, if one of those would work, that would be a lot better, as
LaTeX makes it a lot more complex than it should be.

> Oh well...sorry for rambling at you...I wish I had a good plan for
> making this whole thing better.
> 
> jon

Regards,
Mauro

---

The way the .tex file below is, it builds fine with:

	$ xelatex  -halt-on-error /tmp/test.tex
	This is XeTeX, Version 3.141592653-2.6-0.999995 (TeX Live 2023) (preloaded format=xelatex)
	...
	Output written on test2.pdf (1 page).
	Transcript written on test2.log.

If you remove this line:

	\setlistdepth{9}

You'll see the error:

	$ xelatex  -halt-on-error /tmp/test_without_setlistdepth.tex
	This is XeTeX, Version 3.141592653-2.6-0.999995 (TeX Live 2023) (preloaded format=xelatex)
	...

	! LaTeX Error: Too deeply nested.

	See the LaTeX manual or LaTeX Companion for explanation.
	Type  H <return>  for immediate help.
	 ...                                              
	                                                  
	l.60                 \item
	                           item level 7
	No pages of output.
	Transcript written on test_without_setlistdepth.log.

And it indicates that it fails when level == 7.


-


\documentclass{article}

\usepackage{enumitem}
\RequirePackage{enumitem}

\setlistdepth{9}

\setlist{nolistsep,
         leftmargin = *}
%
%
\newlist{myEnumerate}{enumerate}{9}
\setlist[myEnumerate]{
    font        = {\bfseries} ,
    topsep      = 0pt }
\setlist[myEnumerate,1]{label=\arabic* ---,ref=\arabic*}
\setlist[myEnumerate,2]{label=\arabic{myEnumeratei}.\arabic*),ref=\themyEnumeratei.\arabic*}
\setlist[myEnumerate,3]{label=\roman*),ref=\themyEnumerateii.\roman*}
\setlist[myEnumerate,4]{label=\Roman*.,ref=\themyEnumerateiii.\Roman*}
\setlist[myEnumerate,5]{label=\themyEnumerateiiii.\roman*.,ref = \themyEnumerateiiii.\roman*}
\setlist[myEnumerate,6]{label=(\arabic*)}
\setlist[myEnumerate,7]{label=(\Roman*)}
\setlist[myEnumerate,8]{label=(\Alph*)}
\setlist[myEnumerate,9]{label=(\roman*)}
\newcommand{\ben}{\begin{myEnumerate}}
\newcommand{\een}{\end{myEnumerate}}
%
\newlist{myItemize}{itemize}{9}
\setlist[myItemize]{
    topsep      = 0pt }
\setlist[myItemize,1]{label=\textbullet}
\setlist[myItemize,2]{label=---}
\setlist[myItemize,3]{label=---}
\setlist[myItemize,4]{label=---}
\setlist[myItemize,5]{label=---}
\setlist[myItemize,6]{label=---}
\setlist[myItemize,7]{label=---}
\setlist[myItemize,8]{label=---}
\setlist[myItemize,9]{label=---}
\newcommand{\bit}{\begin{myItemize}}
\newcommand{\eit}{\end{myItemize}}


\begin{document}

\section{Main section}

\section{Subsection}
\ben 
  \item item level 1
  \ben
  \item item level 2
    \bit 
    \item item level 3
      \bit
	\item item level 4
	  \bit
          \item item level 5
            \bit
              \item item level 6
              \bit
                \item item level 7
                  \bit
                  \item item level 8
              \eit
            \eit
          \eit
        \eit
      \eit
    \eit
  \item item level 2
  \een
\item item level 1
\een

\end{document}

Re: PDF misery

[Akira Yokosawa]

Hi Jon,

A few comments on PDF of CJK docs. 

On Sat, 17 Feb 2024 14:29:24 -0700, Jonathan Corbet wrote:
> [...]
> 
> The *other* problem is that PDF generation of the Chinese, Korean, or
> Japanese translations fails with:
> 
>   xdvipdfmx:fatal: Invalid TTC index number
> 
> This, I am thinking, is likely some sort of F39 bug.  xdvipdfmx is
> finding the CJK fonts just fine, but then something clearly goes wrong.
> I'll try to find the time to narrow that down and perhaps put in a
> proper bug report.

I think this is because xdvipdfmx accesses NotoSansCJK-VF.ttc, which is
a variable font.  xdvipdfmx/xetex can't work with such fonts (yet).
See note at the bottom for more info on variable fonts support.

It sounds like you have google-noto-sans-cjk-vf-fonts installed on your
system besides google-noto-sans-cjk-fonts.

What does

    fc-list | grep NotoSansCJK-VF.ttc

say?

I'm wondering why xdvipdfmx behaves that way despite the fontconfig
setting with:

        fc-match "Noto Sans CJK SC"

    returning:

        NotoSansCJK-Regular.ttc: "Noto Sans CJK SC" "Regular"

This might be a bug in xdvipdfmx worth reporting.  Or there might
be glitches in the fontconfig setting.


Uninstalling google-noto-sans-cjk-vf-fonts (and
google-noto-serif-cjk-vf-fonts in case you have it) should resolve
the issue for you, that is if you can safely uninstall it/them.


* Note on variable fonts support:

In the announcement of LaTeX fontspec package v2.9a release [1], there is
an entry:

    - Support variable fonts under LuaLaTeX.

It is not explicitly mentioned there, but in the opening paragraph
of Part III "Selecting font features" Chapter 7 "Variable fonts" of
fontspec's package documentation, there is a sentence which reads:

    Currently OpenType variable fonts are only supported in LuaTEX, while
    Multiple Master fonts only work with XETEX.

As the xeCJK package requires xelatex, variable fonts are not in
our options, I guess.

[1]: https://ctan.org/ctan-ann/id/mailman.6477.1707925238.3764.ctan-ann@ctan.org

HTH, Akira

Re: PDF misery

[Mauro Carvalho Chehab]

Em Sun, 18 Feb 2024 18:58:47 +0900
Akira Yokosawa <akiyks@gmail.com> escreveu:

> Hi Jon,
> 
> A few comments on PDF of CJK docs. 
> 
> On Sat, 17 Feb 2024 14:29:24 -0700, Jonathan Corbet wrote:
> > [...]
> > 
> > The *other* problem is that PDF generation of the Chinese, Korean, or
> > Japanese translations fails with:
> > 
> >   xdvipdfmx:fatal: Invalid TTC index number
> > 
> > This, I am thinking, is likely some sort of F39 bug.  xdvipdfmx is
> > finding the CJK fonts just fine, but then something clearly goes wrong.
> > I'll try to find the time to narrow that down and perhaps put in a
> > proper bug report.  
> 
> I think this is because xdvipdfmx accesses NotoSansCJK-VF.ttc, which is
> a variable font.  xdvipdfmx/xetex can't work with such fonts (yet).
> See note at the bottom for more info on variable fonts support.
> 
> It sounds like you have google-noto-sans-cjk-vf-fonts installed on your
> system besides google-noto-sans-cjk-fonts.
> 
> What does
> 
>     fc-list | grep NotoSansCJK-VF.ttc
> 
> say?
> 
> I'm wondering why xdvipdfmx behaves that way despite the fontconfig
> setting with:
> 
>         fc-match "Noto Sans CJK SC"
> 
>     returning:
> 
>         NotoSansCJK-Regular.ttc: "Noto Sans CJK SC" "Regular"
> 
> This might be a bug in xdvipdfmx worth reporting.  Or there might
> be glitches in the fontconfig setting.
> 
> 
> Uninstalling google-noto-sans-cjk-vf-fonts (and
> google-noto-serif-cjk-vf-fonts in case you have it) should resolve
> the issue for you, that is if you can safely uninstall it/them.

Tried the above on my desktop with Fedora 39:

<snip>
$ fc-list | grep NotoSansCJK-VF.ttc
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK TC:style=Bold
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK SC:style=Bold
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK HK:style=Bold
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK KR:style=Bold
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK JP:style=Bold
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK TC:style=Black
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK SC:style=Black
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK KR:style=Black
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK HK:style=Black
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK JP:style=Black
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK KR
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK HK
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK JP
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK TC
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK SC
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK JP:style=Thin,Regular
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK HK:style=Thin,Regular
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK KR:style=Thin,Regular
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK TC:style=Thin,Regular
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK SC:style=Thin,Regular
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK SC:style=Regular
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK TC:style=Regular
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK JP:style=Regular
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK HK:style=Regular
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK KR:style=Regular
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK TC:style=DemiLight
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK SC:style=DemiLight
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK TC:style=Light
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK SC:style=Light
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK JP:style=DemiLight
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK TC:style=Medium
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK HK:style=DemiLight
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK KR:style=DemiLight
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK JP:style=Light
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK SC:style=Medium
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK KR:style=Light
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK HK:style=Light
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK HK:style=Medium
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK KR:style=Medium
/usr/share/fonts/google-noto-sans-cjk-vf-fonts/NotoSansCJK-VF.ttc: Noto Sans CJK JP:style=Medium
$ fc-match "Noto Sans CJK SC"
NotoSansCJK-VF.ttc: "Noto Sans CJK SC" "Regular"
$ sudo dnf remove google-noto-sans-cjk-vf-fonts google-noto-serif-cjk-vf-fonts -y
...
Removed:
  default-fonts-cjk-sans-4.0-9.fc39.noarch              default-fonts-cjk-serif-4.0-9.fc39.noarch  google-noto-sans-cjk-vf-fonts-1:2.004-5.fc39.noarch 
  google-noto-serif-cjk-vf-fonts-1:2.002-2.fc39.noarch 

Done!
$ fc-list | grep NotoSansCJK-VF.ttc
$ fc-match "Noto Sans CJK SC"
Vera.ttf: "Bitstream Vera Sans" "Regular"
</snip>

Thanks,
Mauro

Re: PDF misery

[Jonathan Corbet]

Akira Yokosawa <akiyks@gmail.com> writes:

> It sounds like you have google-noto-sans-cjk-vf-fonts installed on your
> system besides google-noto-sans-cjk-fonts.
>
> What does
>
>     fc-list | grep NotoSansCJK-VF.ttc
>
> say?

No output at all, even though I had google-noto-sans-cjk-vf-fonts
installed.

> Uninstalling google-noto-sans-cjk-vf-fonts (and
> google-noto-serif-cjk-vf-fonts in case you have it) should resolve
> the issue for you, that is if you can safely uninstall it/them.

Getting rid of google-noto-sans-cjk-vf-fonts did indeed make the problem
go away.

> As the xeCJK package requires xelatex, variable fonts are not in
> our options, I guess.

So it seems.  I guess $WE should probably change
scripts/sphinx-pre-install to give a warning; I wonder if some defensive
changes to Documentation/sphinx/kerneldoc-preamble.sty would also make
sense?  My TeXpert days (such as they were) are decades behind me at
this point, so I'll be slow to do the latter part if it comes to that.

Thanks for your help on this!

jon

Re: PDF misery

[Vegard Nossum]

On 17/02/2024 22:29, Jonathan Corbet wrote:
> So, FYI, I have Vegard's PDF-generation fix applied here, but I've not
> pushed it yet (even though I think it's fine) because I wanted to be
> sure that all was well with PDF generation.  Bear with me as I ramble
> for a bit...

This won't help with the immediate issue at hand, but I am trying to put
together something of a "docs CI" system that can pick up mailing list
patches and do builds against all the various OS/Sphinx/target combinations.

At least that was my takeaway from the recent couple of regressions I
introduced: I need (much) better tests than just running 'make htmldocs'
once or twice in my native environment.

> I'm surprised that nobody else is reporting this problem.  I honestly
> don't see a fix other than changing the organization of the document to
> avoid both deeply nested itemized lists and section structure, which is
> less than entirely desirable.  I think there are good reasons for
> avoiding structures that deep, but limitations in the tooling aren't one
> of them.

I don't know. Tooling limitations notwithstanding, the good reasons are
still there. HTML has no <h7> and that was clearly a "human" decision,
not a technological limit.

I think the "bug" here is really that Sphinx generated LaTeX code that
doesn't build -- and that it didn't warn about the problem in other
modes (maybe it does in newer versions, or maybe we overlooked the
warnings?).

If we'd had those warnings then we could have stopped those constructs
from entering the documentation sources to start with.

> Meanwhile, I wish I knew what to do to make the PDF build a bit more
> robust.  I wish that rst2pdf or rinohtype would progress to the point
> where they could be used, and Latex could be taken out of the picture,
> but no such luck.

We could maybe use the ".. only::" directive to "#ifdef out" the parts
that don't work with various output formats and then incrementally work
towards removing them while ensuring that the whole build still succeeds.


Vegard

Re: PDF misery

[Vegard Nossum]

[-- Attachment #1: Type: text/plain, Size: 866 bytes --]


On 18/02/2024 11:08, Vegard Nossum wrote:
> I think the "bug" here is really that Sphinx generated LaTeX code that
> doesn't build -- and that it didn't warn about the problem in other
> modes (maybe it does in newer versions, or maybe we overlooked the
> warnings?).
> 
> If we'd had those warnings then we could have stopped those constructs
> from entering the documentation sources to start with.

With the attached patch I get:

$ make htmldocs SPHINXDIRS=filesystems/xfs
make[3]: Nothing to be done for 'html'.
Using alabaster theme
source directory: filesystems/xfs
WARNING: xfs-online-fsck-design.rst:2096: heading "Case Study: Sorting 
xfarrays" has nesting level 7; consider restructuring

Maybe we can do the same for itemize lists.

I didn't check what impact this potentially has on running time. Also
only tested with Sphinx 4.3.2.

Thoughts?


Vegard

[-- Attachment #2: kvalidate.patch --]
[-- Type: text/x-patch, Size: 1958 bytes --]

diff --git a/Documentation/conf.py b/Documentation/conf.py
index 5830b01c5642..1cfcea4e487d 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -55,7 +55,8 @@ needs_sphinx = '2.4.4'
 extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include',
               'kfigure', 'sphinx.ext.ifconfig', 'automarkup',
               'maintainers_include', 'sphinx.ext.autosectionlabel',
-              'kernel_abi', 'kernel_feat', 'translations']
+              'kernel_abi', 'kernel_feat', 'translations',
+              'kvalidate']
 
 if major >= 3:
     if (major > 3) or (minor > 0 or patch >= 2):
diff --git a/Documentation/sphinx/kvalidate.py b/Documentation/sphinx/kvalidate.py
new file mode 100644
index 000000000000..3302e936db77
--- /dev/null
+++ b/Documentation/sphinx/kvalidate.py
@@ -0,0 +1,42 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright © 2024, Oracle and/or its affiliates.
+# Author: Vegard Nossum <vegard.nossum@oracle.com>
+#
+# Validate doc tree against kernel-specific rules.
+#
+
+import os
+
+import docutils
+from sphinx.util import logging
+
+logger = logging.getLogger('kerneldoc')
+
+max_heading_levels = 5
+
+def process_doctree(app, doctree):
+    env = app.env
+    srcdir = env.srcdir
+
+    for node in doctree.traverse(docutils.nodes.title):
+        heading_level = 0
+
+        ancestor = node.parent
+        while ancestor is not None:
+            if isinstance(ancestor, docutils.nodes.section):
+                heading_level = heading_level + 1
+
+            ancestor = ancestor.parent
+
+        if heading_level > max_heading_levels:
+            logger.warning("%s:%u: heading \"%s\" has nesting level %u; consider restructuring",
+                os.path.relpath(node.source, srcdir), node.line, node.astext(), heading_level + 1)
+
+def setup(app):
+    app.connect('doctree-read', process_doctree)
+
+    return {
+        'parallel_read_safe': True,
+        'parallel_write_safe': True,
+    }

Re: PDF misery

[Akira Yokosawa]

Hi,

On 2024/02/18 6:29, Jonathan Corbet wrote:
...
> One is the dreaded Latex "too deeply nested" message that causes
> everything to explode while generating filesystems.pdf.  The problem,
> specifically, is .../filesystems/xfs/xfs-online-fsck-design.rst, which
> blows the limits in two ways.  This patch "fixes" it - this clearly
> isn't something to apply, it's just a demonstration of the problem:

So, there is a handy answer in the Sphinx documentation.

Just look at https://www.sphinx-doc.org/en/master/latex.html
and search 'maxlistdepth'.

You can set this variable in conf.py, say,

    'maxlistdepth': '9',

Now the deep lists in xfs-online-fsck-design.rst can be built
into PDF.

Problem solved.

It's getting late here, so can anybody prepare a proper patch
with my

Suggested-by: Akira Yokosawa <akiyks@gmail.com>

?

        Thanks, Akira

Re: PDF misery

[Mauro Carvalho Chehab]

Em Mon, 19 Feb 2024 00:49:08 +0900
Akira Yokosawa <akiyks@gmail.com> escreveu:

> Hi,
> 
> On 2024/02/18 6:29, Jonathan Corbet wrote:
> ...
> > One is the dreaded Latex "too deeply nested" message that causes
> > everything to explode while generating filesystems.pdf.  The problem,
> > specifically, is .../filesystems/xfs/xfs-online-fsck-design.rst, which
> > blows the limits in two ways.  This patch "fixes" it - this clearly
> > isn't something to apply, it's just a demonstration of the problem:  
> 
> So, there is a handy answer in the Sphinx documentation.
> 
> Just look at https://www.sphinx-doc.org/en/master/latex.html
> and search 'maxlistdepth'.

Hmm... there are some other interesting options there, like the
verbatim ones. One of the problems that required lots of changes at
the media uAPI part of the docs is because code blocks are reproduced
as-is, without wrapping, making them to either be truncated or going
past the margin. That's specially painful on code blocks inside tables.

The way it was solved was this this patch:

	3b4c963243b1 ("docs: conf.py: adjust the LaTeX document output")

plus several changes on media docs with redesign changes (before and after
the above patch), plus several and several macros at the .rst files to
change font size and carefully adjusting columns sizes on tables.

It sounds that since then other verbatim options got added. So, perhaps
the PDF output could be improved on newer Sphinx versions.

> 
> You can set this variable in conf.py, say,
> 
>     'maxlistdepth': '9',

Good catch!

> 
> Now the deep lists in xfs-online-fsck-design.rst can be built
> into PDF.
> 
> Problem solved.
> 
> It's getting late here, so can anybody prepare a proper patch
> with my
> 
> Suggested-by: Akira Yokosawa <akiyks@gmail.com>
> 
> ?
> 
>         Thanks, Akira



Thanks,
Mauro

Re: PDF misery

[Jonathan Corbet]

Akira Yokosawa <akiyks@gmail.com> writes:

> Just look at https://www.sphinx-doc.org/en/master/latex.html
> and search 'maxlistdepth'.
>
> You can set this variable in conf.py, say,
>
>     'maxlistdepth': '9',
>
> Now the deep lists in xfs-online-fsck-design.rst can be built
> into PDF.
>
> Problem solved.

So I had actually tried this, and it failed miserably for me.  Now it
works.

As far as I can tell, my failing was that I tossed in that change and
the \usepackage{enumitem} change at the same time; doing both doesn't
work.  I know better than to change multiple things at a time, but given
how long a pdfdocs build attempt takes, one wants to cover all the
bases... 

> It's getting late here, so can anybody prepare a proper patch
> with my
>
> Suggested-by: Akira Yokosawa <akiyks@gmail.com>

I'll put something in, yes.

Thanks,

jon