Error at www.kernel.org/doc/html/next/ since next-20250610

Error at www.kernel.org/doc/html/next/ since next-20250610

[Akira Yokosawa]

Hi Konstantin,

As you might well be aware of, latest successful documentation build against
linux-next at https://www.kernel.org/doc/html/next/ was that of next-20250606.

I managed to reproduce the build error using debian bookworm's distro packages
which can be close enough to Sphinx 5.3.0 used there.

This is the error I got from "make htmldocs":

----------------------------------------------------------------
Extension error (automarkup):
Handler <function auto_markup at 0x796307745bc0> for event 'doctree-resolved' threw an exception (exception: argument for warn() given by name ('stacklevel') and position (3))
----------------------------------------------------------------

Bisection points to commit d6d1df92c25f ("docs: automarkup: Mark up
undocumented entities too") queued in Jon's docs-next branch.

Reverting it resolves the error, but honestly speaking, I don't have
any plausible theory on how that change could cause such an issue.

Actually, when I install Sphinx 5.3.0 by using venv and saying

    pip install "Sphinx<6.0" pyyaml

on Ubuntu 24.04 today, said error doesn't reproduce.

I'm not sure but there might have been some minor incompatibility
between past versions of dependencies.

I'd rather upgrade Sphinx to 8.2.3 (latest).

Konstantin, could you try it?

        Thanks, Akira

Re: Error at www.kernel.org/doc/html/next/ since next-20250610

[Mauro Carvalho Chehab]

Em Mon, 16 Jun 2025 20:18:22 +0900
Akira Yokosawa <akiyks@gmail.com> escreveu:

> Hi Konstantin,
> 
> As you might well be aware of, latest successful documentation build against
> linux-next at https://www.kernel.org/doc/html/next/ was that of next-20250606.
> 
> I managed to reproduce the build error using debian bookworm's distro packages
> which can be close enough to Sphinx 5.3.0 used there.
> 
> This is the error I got from "make htmldocs":
> 
> ----------------------------------------------------------------
> Extension error (automarkup):
> Handler <function auto_markup at 0x796307745bc0> for event 'doctree-resolved' threw an exception (exception: argument for warn() given by name ('stacklevel') and position (3))
> ----------------------------------------------------------------

Sphinx crashes generally generate a python dump. Maybe you could take
a look on it.

> Bisection points to commit d6d1df92c25f ("docs: automarkup: Mark up
> undocumented entities too") queued in Jon's docs-next branch.
> 
> Reverting it resolves the error, but honestly speaking, I don't have
> any plausible theory on how that change could cause such an issue.
> 
> Actually, when I install Sphinx 5.3.0 by using venv and saying
> 
>     pip install "Sphinx<6.0" pyyaml
> 
> on Ubuntu 24.04 today, said error doesn't reproduce.

Perhaps it is related to docutils version.

Assuming that it doesn't work with certain versions of sphinx or
docutils, instead of reverting the entire patch, IMO the best would
be to add a check for Sphinx version, disabling the broken refs
only for too old versions.

> I'm not sure but there might have been some minor incompatibility
> between past versions of dependencies.
> 
> I'd rather upgrade Sphinx to 8.2.3 (latest).
> 
> Konstantin, could you try it?
> 
>         Thanks, Akira

Regards

Thanks,
Mauro

Re: Error at www.kernel.org/doc/html/next/ since next-20250610

[Akira Yokosawa]

Hi Mauro,

On Mon, 16 Jun 2025 14:05:36 +0200, Mauro Carvalho Chehab wrote:
> Em Mon, 16 Jun 2025 20:18:22 +0900
> Akira Yokosawa <akiyks@gmail.com> escreveu:
[...]

> > I managed to reproduce the build error using debian bookworm's distro packages
> > which can be close enough to Sphinx 5.3.0 used there.
> >
> > This is the error I got from "make htmldocs":
> >
> > ----------------------------------------------------------------
> > Extension error (automarkup):
> > Handler <function auto_markup at 0x796307745bc0> for event 'doctree-resolved' threw an exception (exception: argument for warn() given by name ('stacklevel') and position (3))
> > ----------------------------------------------------------------

> Perhaps it is related to docutils version.

Looks like you are right.

debian:12 (Sphinx 5.3.0) and mageia:9 (Sphinx 6.1.3) are distro releases
whose docutils is 0.19.  They both show the same symptom.

Furthermore, if I install Sphinx by using venv and saying:

    pip install "docutils==0.19" Sphinx pyyaml

, I have Sphinx 7.3.6 and it shows the same symptom.

So you need to say "docutils!=0.19" when you install specific versions
of Sphinx.

> 
> Assuming that it doesn't work with certain versions of sphinx or
> docutils, instead of reverting the entire patch, IMO the best would
> be to add a check for Sphinx version, disabling the broken refs
> only for too old versions.
> 

Can you do so against docutils 0.19 only?

       Thanks, Akira

Re: Error at www.kernel.org/doc/html/next/ since next-20250610

[Mauro Carvalho Chehab]

Em Thu, 19 Jun 2025 11:22:19 +0900
Akira Yokosawa <akiyks@gmail.com> escreveu:

> Hi Mauro,
> 
> On Mon, 16 Jun 2025 14:05:36 +0200, Mauro Carvalho Chehab wrote:
> > Em Mon, 16 Jun 2025 20:18:22 +0900
> > Akira Yokosawa <akiyks@gmail.com> escreveu:  
> [...]
> 
> > > I managed to reproduce the build error using debian bookworm's distro packages
> > > which can be close enough to Sphinx 5.3.0 used there.
> > >
> > > This is the error I got from "make htmldocs":
> > >
> > > ----------------------------------------------------------------
> > > Extension error (automarkup):
> > > Handler <function auto_markup at 0x796307745bc0> for event 'doctree-resolved' threw an exception (exception: argument for warn() given by name ('stacklevel') and position (3))
> > > ----------------------------------------------------------------  
> 
> > Perhaps it is related to docutils version.  
> 
> Looks like you are right.
> 
> debian:12 (Sphinx 5.3.0) and mageia:9 (Sphinx 6.1.3) are distro releases
> whose docutils is 0.19.  They both show the same symptom.
> 
> Furthermore, if I install Sphinx by using venv and saying:
> 
>     pip install "docutils==0.19" Sphinx pyyaml
> 
> , I have Sphinx 7.3.6 and it shows the same symptom.
> 
> So you need to say "docutils!=0.19" when you install specific versions
> of Sphinx.
> 
> > 
> > Assuming that it doesn't work with certain versions of sphinx or
> > docutils, instead of reverting the entire patch, IMO the best would
> > be to add a check for Sphinx version, disabling the broken refs
> > only for too old versions.
> >   
> 
> Can you do so against docutils 0.19 only?

If we're willing to do that, IMO we need to do a more generic solution
that will compare both versions and warn if incompatibilities are
detected.

Something like the enclosed patch (it is against my latest conf.py
patch).

Thanks,
Mauro

---

[PATCH] docs: conf.py: Check Sphinx and docutils version

As reported by Akira, there are incompatibility issues with
Sphinx and docutils.

I manually checked that before docutils 0.17.1, yaml generation
doesn't work properly. Akira checked that 0.19 is problematic too.

After check Sphinx release notes, it seems that the
versions that are supposed to cope well together are:

	========  ============  ============
	Sphinx    Min Docutils  Max Docutils
	Version   Version       Version
	--------  ------------  ------------
	< 4.0.0	  0.17.1        0.17.1
	< 6.0.0	  0.17.1        0.18.1
	< 7.0.0   0.18.0        0.18.1
	>= 7.0.0  0.20.0        0.21.2
	========  ============  ============

Add a logic inside conf.py to check the above, emitting warnings
if the docutils version don't match what is known to be supported.

Reported-by: Akira Yokosawa <akiyks@gmail.com>
Closes: https://lore.kernel.org/linux-doc/6fcb75ee-61db-4fb3-9c5f-2029a7fea4ee@gmail.com/
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

diff --git a/Documentation/conf.py b/Documentation/conf.py
index 5eddf5885f77..6047ec85add1 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -9,7 +9,11 @@ import os
 import shutil
 import sys
 
+import docutils
 import sphinx
+from sphinx.util import logging
+
+logger = logging.getLogger(__name__)
 
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
@@ -21,11 +25,34 @@ from load_config import loadConfig               # pylint: disable=C0413,E0401
 # Minimal supported version
 needs_sphinx = "3.4.3"
 
-# Get Sphinx version
-major, minor, patch = sphinx.version_info[:3]          # pylint: disable=I1101
+# Get Sphinx and docutils versions
+sphinx_ver = sphinx.version_info[:3]          # pylint: disable=I1101
+docutils_ver = docutils.__version_info__[:3]
+
+#
+if sphinx_ver < (4, 0, 0):
+    min_docutils = (0, 16, 0)
+    max_docutils = (0, 17, 1)
+elif sphinx_ver < (6, 0, 0):
+    min_docutils = (0, 17, 0)
+    max_docutils = (0, 18, 1)
+elif sphinx_ver < (7, 0, 0):
+    min_docutils = (0, 18, 0)
+    max_docutils = (0, 18, 1)
+else:
+    min_docutils = (0, 20, 0)
+    max_docutils = (0, 21, 2)
+
+sphinx_ver_str = ".".join([str(x) for x in sphinx_ver])
+docutils_ver_str = ".".join([str(x) for x in docutils_ver])
+
+if docutils_ver < min_docutils:
+    logger.warning(f"Docutils {docutils_ver_str} is too old for Sphinx {sphinx_ver_str}. Doc generation may fail")
+elif docutils_ver > max_docutils:
+    logger.warning(f"Docutils {docutils_ver_str} could be too new for Sphinx {sphinx_ver_str}. Doc generation may fail")
 
 # Include_patterns were added on Sphinx 5.1
-if (major < 5) or (major == 5 and minor < 1):
+if sphinx_ver < (5, 1, 0):
     has_include_patterns = False
 else:
     has_include_patterns = True

Re: Error at www.kernel.org/doc/html/next/ since next-20250610

[Randy Dunlap]

On 6/18/25 11:16 PM, Mauro Carvalho Chehab wrote:
> Em Thu, 19 Jun 2025 11:22:19 +0900
> Akira Yokosawa <akiyks@gmail.com> escreveu:
> 

>>
>> Can you do so against docutils 0.19 only?
> 
> If we're willing to do that, IMO we need to do a more generic solution
> that will compare both versions and warn if incompatibilities are
> detected.
> 
> Something like the enclosed patch (it is against my latest conf.py
> patch).
> 
> Thanks,
> Mauro
> 
> ---
> 
> [PATCH] docs: conf.py: Check Sphinx and docutils version
> 
> As reported by Akira, there are incompatibility issues with
> Sphinx and docutils.
> 
> I manually checked that before docutils 0.17.1, yaml generation
> doesn't work properly. Akira checked that 0.19 is problematic too.
> 
> After check Sphinx release notes, it seems that the
> versions that are supposed to cope well together are:
> 
> 	========  ============  ============
> 	Sphinx    Min Docutils  Max Docutils
> 	Version   Version       Version
> 	--------  ------------  ------------
> 	< 4.0.0	  0.17.1        0.17.1
> 	< 6.0.0	  0.17.1        0.18.1
> 	< 7.0.0   0.18.0        0.18.1
> 	>= 7.0.0  0.20.0        0.21.2
> 	========  ============  ============
> 
> Add a logic inside conf.py to check the above, emitting warnings
> if the docutils version don't match what is known to be supported.
> 
> Reported-by: Akira Yokosawa <akiyks@gmail.com>
> Closes: https://lore.kernel.org/linux-doc/6fcb75ee-61db-4fb3-9c5f-2029a7fea4ee@gmail.com/
> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> 
> diff --git a/Documentation/conf.py b/Documentation/conf.py
> index 5eddf5885f77..6047ec85add1 100644
> --- a/Documentation/conf.py
> +++ b/Documentation/conf.py
> @@ -9,7 +9,11 @@ import os
>  import shutil
>  import sys
>  
> +import docutils
>  import sphinx
> +from sphinx.util import logging
> +
> +logger = logging.getLogger(__name__)
>  
>  # If extensions (or modules to document with autodoc) are in another directory,
>  # add these directories to sys.path here. If the directory is relative to the
> @@ -21,11 +25,34 @@ from load_config import loadConfig               # pylint: disable=C0413,E0401
>  # Minimal supported version
>  needs_sphinx = "3.4.3"
>  
> -# Get Sphinx version
> -major, minor, patch = sphinx.version_info[:3]          # pylint: disable=I1101
> +# Get Sphinx and docutils versions
> +sphinx_ver = sphinx.version_info[:3]          # pylint: disable=I1101
> +docutils_ver = docutils.__version_info__[:3]
> +
> +#
> +if sphinx_ver < (4, 0, 0):
> +    min_docutils = (0, 16, 0)
> +    max_docutils = (0, 17, 1)
> +elif sphinx_ver < (6, 0, 0):
> +    min_docutils = (0, 17, 0)
> +    max_docutils = (0, 18, 1)
> +elif sphinx_ver < (7, 0, 0):
> +    min_docutils = (0, 18, 0)
> +    max_docutils = (0, 18, 1)
> +else:
> +    min_docutils = (0, 20, 0)
> +    max_docutils = (0, 21, 2)
> +
> +sphinx_ver_str = ".".join([str(x) for x in sphinx_ver])
> +docutils_ver_str = ".".join([str(x) for x in docutils_ver])


That ".". (2 times) is ugly. ;)  (needs spaces added IMO)
Is that compliant with PEP8?
I can't see that PEP8 addresses usage of . directly.  For some binary operators:
  Always surround these binary operators with a single space on either side: assignment (=),
  augmented assignment (+=, -= etc.), comparisons (==, <, >, !=, <=, >=, in, not in, is, is not),
  Booleans (and, or, not).
[https://peps.python.org/pep-0008/#whitespace-in-expressions-and-statements]

Thanks.

> +
> +if docutils_ver < min_docutils:
> +    logger.warning(f"Docutils {docutils_ver_str} is too old for Sphinx {sphinx_ver_str}. Doc generation may fail")
> +elif docutils_ver > max_docutils:
> +    logger.warning(f"Docutils {docutils_ver_str} could be too new for Sphinx {sphinx_ver_str}. Doc generation may fail")
>  
>  # Include_patterns were added on Sphinx 5.1
> -if (major < 5) or (major == 5 and minor < 1):
> +if sphinx_ver < (5, 1, 0):
>      has_include_patterns = False
>  else:
>      has_include_patterns = True
> 
> 

-- 
~Randy

Re: Error at www.kernel.org/doc/html/next/ since next-20250610

[Konstantin Ryabitsev]

On Mon, Jun 16, 2025 at 08:18:22PM +0900, Akira Yokosawa wrote:
> I'm not sure but there might have been some minor incompatibility
> between past versions of dependencies.
> 
> I'd rather upgrade Sphinx to 8.2.3 (latest).
> 
> Konstantin, could you try it?

Looks like 8.2.3 requires python-3.11, which I don't have readily available on
the system that builds the documentation. However, upgrading to 7.4 also fixed
the problem, so the latest next tree is now built and should be propagating
shortly.

Thanks,
-K

Re: Error at www.kernel.org/doc/html/next/ since next-20250610

[Jonathan Corbet]

Konstantin Ryabitsev <konstantin@linuxfoundation.org> writes:

> On Mon, Jun 16, 2025 at 08:18:22PM +0900, Akira Yokosawa wrote:
>> I'm not sure but there might have been some minor incompatibility
>> between past versions of dependencies.
>> 
>> I'd rather upgrade Sphinx to 8.2.3 (latest).
>> 
>> Konstantin, could you try it?
>
> Looks like 8.2.3 requires python-3.11, which I don't have readily
> available on the system that builds the documentation. However,
> upgrading to 7.4 also fixed the problem, so the latest next tree is
> now built and should be propagating shortly.

[Catching up after some much-needed time in the wilderness...]

Konstantin, is there any easy way to get a notification when the build
there fails?  I was entirely unaware of the problem, needless to say.

Once I've dug out a little further, I'll try to figure out why my change
makes things explode.  It really shouldn't...so it should be possible to
find a way to work around the problem and avoid putting in version
requirements (and tests) that we haven't needed so far.

Thanks,

jon