linux-next: build failure after merge all the trees

linux-next: build failure after merge all the trees

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 1224 bytes --]

Hi all,

In my after merge build tests, today's linux-next build (htmldocs)
failed like this:

make[1]: Entering directory '/home/sfr/next/htmldocs'
  PARSE   include/uapi/linux/dvb/ca.h
  PARSE   include/uapi/linux/dvb/dmx.h
  PARSE   include/uapi/linux/dvb/frontend.h
  PARSE   include/uapi/linux/dvb/net.h
  PARSE   include/uapi/linux/videodev2.h
  PARSE   include/uapi/linux/media.h
  PARSE   include/uapi/linux/cec.h
  PARSE   include/uapi/linux/lirc.h
Using alabaster theme
Using Python kernel-doc
/home/sfr/next/next/Documentation/virt/kvm/x86/intel-tdx.rst:255: WARNING: Footnote [1] is not referenced. [ref.footnote]

Sphinx error:
Sphinx is unable to load the master document (/home/sfr/next/next/Documentation/index.rst). The master document must be within the source directory or a subdirectory of it.
make[3]: *** [/home/sfr/next/next/Documentation/Makefile:123: htmldocs] Error 2
make[2]: *** [/home/sfr/next/next/Makefile:1804: htmldocs] Error 2
make[1]: *** [/home/sfr/next/next/Makefile:248: __sub-make] Error 2
make[1]: Leaving directory '/home/sfr/next/htmldocs'
make: *** [Makefile:248: __sub-make] Error 2

I have no idea what caued this :-(

-- 
Cheers,
Stephen Rothwell

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: linux-next: build failure after merge all the trees

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 1585 bytes --]

Hi all,

On Thu, 8 May 2025 18:25:04 +1000 Stephen Rothwell <sfr@canb.auug.org.au> wrote:
>
> In my after merge build tests, today's linux-next build (htmldocs)
> failed like this:
> 
> make[1]: Entering directory '/home/sfr/next/htmldocs'
>   PARSE   include/uapi/linux/dvb/ca.h
>   PARSE   include/uapi/linux/dvb/dmx.h
>   PARSE   include/uapi/linux/dvb/frontend.h
>   PARSE   include/uapi/linux/dvb/net.h
>   PARSE   include/uapi/linux/videodev2.h
>   PARSE   include/uapi/linux/media.h
>   PARSE   include/uapi/linux/cec.h
>   PARSE   include/uapi/linux/lirc.h
> Using alabaster theme
> Using Python kernel-doc
> /home/sfr/next/next/Documentation/virt/kvm/x86/intel-tdx.rst:255: WARNING: Footnote [1] is not referenced. [ref.footnote]
> 
> Sphinx error:
> Sphinx is unable to load the master document (/home/sfr/next/next/Documentation/index.rst). The master document must be within the source directory or a subdirectory of it.
> make[3]: *** [/home/sfr/next/next/Documentation/Makefile:123: htmldocs] Error 2
> make[2]: *** [/home/sfr/next/next/Makefile:1804: htmldocs] Error 2
> make[1]: *** [/home/sfr/next/next/Makefile:248: __sub-make] Error 2
> make[1]: Leaving directory '/home/sfr/next/htmldocs'
> make: *** [Makefile:248: __sub-make] Error 2
> 
> I have no idea what caued this :-(

$ ls -l $HOME/next/next/Documentation/index.rst 
-rw-r--r-- 1 sfr users 3274 May  8 10:55 /home/sfr/next/next/Documentation/index.rst

The commands I use are:

cd $HOME/next/next
make O="$HOME/next/htmldocs" htmldocs

-- 
Cheers,
Stephen Rothwell

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: linux-next: build failure after merge all the trees

[Ozgur Kara]

Stephen Rothwell <sfr@canb.auug.org.au>, 8 May 2025 Per, 11:26
tarihinde şunu yazdı:
>
> Hi all,
>
> In my after merge build tests, today's linux-next build (htmldocs)
> failed like this:
>
> make[1]: Entering directory '/home/sfr/next/htmldocs'
>   PARSE   include/uapi/linux/dvb/ca.h
>   PARSE   include/uapi/linux/dvb/dmx.h
>   PARSE   include/uapi/linux/dvb/frontend.h
>   PARSE   include/uapi/linux/dvb/net.h
>   PARSE   include/uapi/linux/videodev2.h
>   PARSE   include/uapi/linux/media.h
>   PARSE   include/uapi/linux/cec.h
>   PARSE   include/uapi/linux/lirc.h
> Using alabaster theme
> Using Python kernel-doc
> /home/sfr/next/next/Documentation/virt/kvm/x86/intel-tdx.rst:255: WARNING: Footnote [1] is not referenced. [ref.footnote]
>
> Sphinx error:
> Sphinx is unable to load the master document (/home/sfr/next/next/Documentation/index.rst). The master document must be within the source directory or a subdirectory of it.
> make[3]: *** [/home/sfr/next/next/Documentation/Makefile:123: htmldocs] Error 2
> make[2]: *** [/home/sfr/next/next/Makefile:1804: htmldocs] Error 2
> make[1]: *** [/home/sfr/next/next/Makefile:248: __sub-make] Error 2
> make[1]: Leaving directory '/home/sfr/next/htmldocs'
> make: *** [Makefile:248: __sub-make] Error 2
>
> I have no idea what caued this :-(

Hello,

Did you run this command before you got this error?

$ make htmldocs

but the error shows itself because index.rst is not in need requested
directory, please run:

$ git restore Documentation/index.rst

Can this solve it?
Regards,

Ozgur

>
> --
> Cheers,
> Stephen Rothwell

Re: linux-next: build failure after merge all the trees

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 405 bytes --]

Hi Ozgur,

On Thu, 8 May 2025 09:20:26 +0000 Ozgur Kara <ozgur@goosey.org> wrote:
>
> Did you run this command before you got this error?
> 
> $ make htmldocs
> 
> but the error shows itself because index.rst is not in need requested
> directory, please run:
> 
> $ git restore Documentation/index.rst
> 
> Can this solve it?

Please see my second email.

-- 
Cheers,
Stephen Rothwell

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: linux-next: build failure after merge all the trees

[Ozgur Kara]

Stephen Rothwell <sfr@canb.auug.org.au>, 8 May 2025 Per, 12:27
tarihinde şunu yazdı:
>
> Hi Ozgur,
>
> On Thu, 8 May 2025 09:20:26 +0000 Ozgur Kara <ozgur@goosey.org> wrote:
> >
> > Did you run this command before you got this error?
> >
> > $ make htmldocs
> >
> > but the error shows itself because index.rst is not in need requested
> > directory, please run:
> >
> > $ git restore Documentation/index.rst
> >
> > Can this solve it?
>
> Please see my second email.

Hello Stephen,

ah okay, i see it now and i'm sorry for sending you an e-mail without
seeing it.
and the error there also caught my attention.

Regards,

Ozgur


>
> --
> Cheers,
> Stephen Rothwell

Re: linux-next: build failure after merge all the trees

[Akira Yokosawa]

[+CC: Mauro & Dan]

Hi,

Stephen Rothwell wrote:
> The commands I use are:
> 
> cd $HOME/next/next
> make O="$HOME/next/htmldocs" htmldocs

Please try:

  make O="$HOME/next/htmldocs" KERNELDOC=scripts/kernel-doc.pl htmldocs

, assuming your $HOME/next/next is the top of kernel source.

I'm suspecting that the conflict resolution done in
c84724f2137f ("Merge branch 'for-6.16/tsm-mr' into tsm-next")
ended up in mismatching path names given to "kernel-doc::" somewhere.

Looks like recent conversion of the kernel-doc script into python
has changed the behavior in such error conditions.
With the perl version, you'll see a couple of:

    Error: Cannot open file <...>/linux/drivers/virt/coco/tsm-mr.c

, but the doc build should complete.

HTH,
Akira

Re: linux-next: build failure after merge all the trees

[Mauro Carvalho Chehab]

Em Thu, 8 May 2025 19:27:13 +1000
Stephen Rothwell <sfr@canb.auug.org.au> escreveu:

> Hi Ozgur,
> 
> On Thu, 8 May 2025 09:20:26 +0000 Ozgur Kara <ozgur@goosey.org> wrote:
> >
> > Did you run this command before you got this error?
> > 
> > $ make htmldocs
> > 
> > but the error shows itself because index.rst is not in need requested
> > directory, please run:
> > 
> > $ git restore Documentation/index.rst
> > 
> > Can this solve it?  
> 
> Please see my second email.
> 

On a quick look, I noticed two build issues. See the enclosed diff.
Feel free to split them on separate patches and/or merge hunks with
the offending code from the original tree.

Btw, if you're in doubt about what's causing doc build issues, you
can now use V=1, as it will tell the kernel-doc command-line equivalent
command:

	$ make V=1 htmldocs
	...
	./scripts/kernel-doc.py -rst -enable-lineno -no-doc-sections ./drivers/cxl/core/pci.c
	./scripts/kernel-doc.py -rst -enable-lineno -function 'cxl pmem' ./drivers/cxl/core/pmem.c
	./scripts/kernel-doc.py -rst -enable-lineno -function 'cxl registers' ./drivers/cxl/core/regs.c
	./scripts/kernel-doc.py -rst -enable-lineno ./kernel/irq/manage.c
	...

(the execution itself is done via a Kernel-doc class, instead of via shell,
 but you can reproduce the results by calling kernel-doc with the shown syntax)

Thanks,
Mauro

[PATCH] Fix build issues with linux-next

Address two issues causing issues during docs build on next:
- a broken kernel-doc reference;
- a non-reference listed as if it was a reference.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

diff --git a/Documentation/driver-api/coco/measurement-registers.rst b/Documentation/driver-api/coco/measurement-registers.rst
index cef85945a9a7..962a44efa2c0 100644
--- a/Documentation/driver-api/coco/measurement-registers.rst
+++ b/Documentation/driver-api/coco/measurement-registers.rst
@@ -8,5 +8,5 @@ Measurement Registers
 .. kernel-doc:: include/linux/tsm-mr.h
    :internal:
 
-.. kernel-doc:: drivers/virt/coco/tsm-mr.c
+.. kernel-doc:: drivers/virt/coco/guest/tsm-mr.c
    :export:
diff --git a/Documentation/virt/kvm/x86/intel-tdx.rst b/Documentation/virt/kvm/x86/intel-tdx.rst
index de41d4c01e5c..fa5efd970146 100644
--- a/Documentation/virt/kvm/x86/intel-tdx.rst
+++ b/Documentation/virt/kvm/x86/intel-tdx.rst
@@ -252,4 +252,4 @@ control flow is as follows:
 References
 ==========
 
-.. [1] https://www.intel.com/content/www/us/en/developer/tools/trust-domain-extensions/documentation.html
+[1] https://www.intel.com/content/www/us/en/developer/tools/trust-domain-extensions/documentation.html

Re: linux-next: build failure after merge all the trees

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 1586 bytes --]

Hi Akira,

On Thu, 8 May 2025 19:54:08 +0900 Akira Yokosawa <akiyks@gmail.com> wrote:
>
> Please try:
> 
>   make O="$HOME/next/htmldocs" KERNELDOC=scripts/kernel-doc.pl htmldocs
> 
> , assuming your $HOME/next/next is the top of kernel source.
> 
> I'm suspecting that the conflict resolution done in
> c84724f2137f ("Merge branch 'for-6.16/tsm-mr' into tsm-next")
> ended up in mismatching path names given to "kernel-doc::" somewhere.
> 
> Looks like recent conversion of the kernel-doc script into python
> has changed the behavior in such error conditions.
> With the perl version, you'll see a couple of:
> 
>     Error: Cannot open file <...>/linux/drivers/virt/coco/tsm-mr.c
> 
> , but the doc build should complete.

OK, so, yes, the build completes.  I get the following message
(multiple similar ones):

WARNING: kernel-doc 'scripts/kernel-doc.pl -rst -enable-lineno -export -export-file drivers/misc/mei/bus.c drivers/misc/mei/bus.c' processing failed with: [Errno 2] No such file or directory: 'scripts/kernel-doc.pl'

So, I used "KERNELDOC=$(pwd)/scripts/kernel-doc.pl" and tried again.

I got these (new) messages:

Error: Cannot open file drivers/virt/coco/tsm-mr.c
Error: Cannot open file drivers/virt/coco/tsm-mr.c
WARNING: kernel-doc 'scripts/kernel-doc.pl -rst -enable-lineno -export drivers/virt/coco/tsm-mr.c' failed with return code 2

(and a few other innocuous ones)

So your guess is good.

It would be nice to have the Python kernel-doc fixed as well as the
devsec-tsm tree.

Thanks.
-- 
Cheers,
Stephen Rothwell

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: linux-next: build failure after merge all the trees

[Mauro Carvalho Chehab]

Em Thu, 8 May 2025 22:25:31 +1000
Stephen Rothwell <sfr@canb.auug.org.au> escreveu:

> Hi Akira,
> 
> On Thu, 8 May 2025 19:54:08 +0900 Akira Yokosawa <akiyks@gmail.com> wrote:
> >
> > Please try:
> > 
> >   make O="$HOME/next/htmldocs" KERNELDOC=scripts/kernel-doc.pl htmldocs
> > 
> > , assuming your $HOME/next/next is the top of kernel source.
> > 
> > I'm suspecting that the conflict resolution done in
> > c84724f2137f ("Merge branch 'for-6.16/tsm-mr' into tsm-next")
> > ended up in mismatching path names given to "kernel-doc::" somewhere.
> > 
> > Looks like recent conversion of the kernel-doc script into python
> > has changed the behavior in such error conditions.
> > With the perl version, you'll see a couple of:
> > 
> >     Error: Cannot open file <...>/linux/drivers/virt/coco/tsm-mr.c
> > 
> > , but the doc build should complete.  
> 
> OK, so, yes, the build completes.  I get the following message
> (multiple similar ones):
> 
> WARNING: kernel-doc 'scripts/kernel-doc.pl -rst -enable-lineno -export -export-file drivers/misc/mei/bus.c drivers/misc/mei/bus.c' processing failed with: [Errno 2] No such file or directory: 'scripts/kernel-doc.pl'
> 
> So, I used "KERNELDOC=$(pwd)/scripts/kernel-doc.pl" and tried again.
> 
> I got these (new) messages:
> 
> Error: Cannot open file drivers/virt/coco/tsm-mr.c
> Error: Cannot open file drivers/virt/coco/tsm-mr.c
> WARNING: kernel-doc 'scripts/kernel-doc.pl -rst -enable-lineno -export drivers/virt/coco/tsm-mr.c' failed with return code 2
> 
> (and a few other innocuous ones)
> 
> So your guess is good.
> 
> It would be nice to have the Python kernel-doc fixed as well as the
> devsec-tsm tree.

With regards to kernel-doc, failing to build if a file is missing
is the right thing to do. As kernel-doc is now fully an Sphinx extension,
it now signalizes to Sphinx that the build has failed. 

Yet, it should have produced some warnings. Maybe we need to specify a
different log level with Sphinx to make it happen. I'll double check it
and send a fix later on to kernel-doc.

Thanks,
Mauro

Re: linux-next: build failure after merge all the trees

[Dan Williams]

Stephen Rothwell wrote:
> Hi Akira,
> 
> On Thu, 8 May 2025 19:54:08 +0900 Akira Yokosawa <akiyks@gmail.com> wrote:
> >
> > Please try:
> > 
> >   make O="$HOME/next/htmldocs" KERNELDOC=scripts/kernel-doc.pl htmldocs
> > 
> > , assuming your $HOME/next/next is the top of kernel source.
> > 
> > I'm suspecting that the conflict resolution done in
> > c84724f2137f ("Merge branch 'for-6.16/tsm-mr' into tsm-next")
> > ended up in mismatching path names given to "kernel-doc::" somewhere.
> > 
> > Looks like recent conversion of the kernel-doc script into python
> > has changed the behavior in such error conditions.
> > With the perl version, you'll see a couple of:
> > 
> >     Error: Cannot open file <...>/linux/drivers/virt/coco/tsm-mr.c
> > 
> > , but the doc build should complete.
> 
> OK, so, yes, the build completes.  I get the following message
> (multiple similar ones):
> 
> WARNING: kernel-doc 'scripts/kernel-doc.pl -rst -enable-lineno -export -export-file drivers/misc/mei/bus.c drivers/misc/mei/bus.c' processing failed with: [Errno 2] No such file or directory: 'scripts/kernel-doc.pl'
> 
> So, I used "KERNELDOC=$(pwd)/scripts/kernel-doc.pl" and tried again.
> 
> I got these (new) messages:
> 
> Error: Cannot open file drivers/virt/coco/tsm-mr.c
> Error: Cannot open file drivers/virt/coco/tsm-mr.c
> WARNING: kernel-doc 'scripts/kernel-doc.pl -rst -enable-lineno -export drivers/virt/coco/tsm-mr.c' failed with return code 2
> 
> (and a few other innocuous ones)
> 
> So your guess is good.
> 
> It would be nice to have the Python kernel-doc fixed as well as the
> devsec-tsm tree.

Will fix.

Re: linux-next: build failure after merge all the trees

[Mauro Carvalho Chehab]

Em Thu, 8 May 2025 17:46:03 -0700
Dan Williams <dan.j.williams@intel.com> escreveu:

> Stephen Rothwell wrote:
> > Hi Akira,
> > 
> > On Thu, 8 May 2025 19:54:08 +0900 Akira Yokosawa <akiyks@gmail.com> wrote:  
> > >
> > > Please try:
> > > 
> > >   make O="$HOME/next/htmldocs" KERNELDOC=scripts/kernel-doc.pl htmldocs
> > > 
> > > , assuming your $HOME/next/next is the top of kernel source.
> > > 
> > > I'm suspecting that the conflict resolution done in
> > > c84724f2137f ("Merge branch 'for-6.16/tsm-mr' into tsm-next")
> > > ended up in mismatching path names given to "kernel-doc::" somewhere.
> > > 
> > > Looks like recent conversion of the kernel-doc script into python
> > > has changed the behavior in such error conditions.
> > > With the perl version, you'll see a couple of:
> > > 
> > >     Error: Cannot open file <...>/linux/drivers/virt/coco/tsm-mr.c
> > > 
> > > , but the doc build should complete.  
> > 
> > OK, so, yes, the build completes.  I get the following message
> > (multiple similar ones):
> > 
> > WARNING: kernel-doc 'scripts/kernel-doc.pl -rst -enable-lineno -export -export-file drivers/misc/mei/bus.c drivers/misc/mei/bus.c' processing failed with: [Errno 2] No such file or directory: 'scripts/kernel-doc.pl'
> > 
> > So, I used "KERNELDOC=$(pwd)/scripts/kernel-doc.pl" and tried again.
> > 
> > I got these (new) messages:
> > 
> > Error: Cannot open file drivers/virt/coco/tsm-mr.c
> > Error: Cannot open file drivers/virt/coco/tsm-mr.c
> > WARNING: kernel-doc 'scripts/kernel-doc.pl -rst -enable-lineno -export drivers/virt/coco/tsm-mr.c' failed with return code 2
> > 
> > (and a few other innocuous ones)

The same messages can be seen if we don't use Sphinx logger facility,
using the default Python logging class instead:

diff --git a/Documentation/sphinx/kerneldoc.py b/Documentation/sphinx/kerneldoc.py
index b713a2c4a615..914ebfe2080b 100644
--- a/Documentation/sphinx/kerneldoc.py
+++ b/Documentation/sphinx/kerneldoc.py
@@ -311,7 +311,7 @@ def setup_kfiles(app):
     if kerneldoc_bin and kerneldoc_bin.endswith("kernel-doc.py"):
         print("Using Python kernel-doc")
         out_style = RestFormat()
-        kfiles = KernelFiles(out_style=out_style, logger=logger)
+        kfiles = KernelFiles(out_style=out_style)
     else:
         print(f"Using {kerneldoc_bin}")

Somehow, Sphinx is sending such messages to /dev/null if we don't use V=1.

I'll see if I can discover why Sphinx is doing that.

Thanks,
Mauro

Re: linux-next: build failure after merge all the trees

[Akira Yokosawa]

[+CC linux-doc]

Hi,

On Thu, 8 May 2025 14:39:11 +0200, Mauro Carvalho Chehab wrote:
> Em Thu, 8 May 2025 22:25:31 +1000
> Stephen Rothwell <sfr@canb.auug.org.au> escreveu:
[...]

>>
>> So, I used "KERNELDOC=$(pwd)/scripts/kernel-doc.pl" and tried again.
>>
>> I got these (new) messages:
>>
>> Error: Cannot open file drivers/virt/coco/tsm-mr.c
>> Error: Cannot open file drivers/virt/coco/tsm-mr.c
>> WARNING: kernel-doc 'scripts/kernel-doc.pl -rst -enable-lineno -export drivers/virt/coco/tsm-mr.c' failed with return code 2
>>
>> (and a few other innocuous ones)
>>
>> So your guess is good.
>>
>> It would be nice to have the Python kernel-doc fixed as well as the
>> devsec-tsm tree.
> 
> With regards to kernel-doc, failing to build if a file is missing
> is the right thing to do.

Mauro, I don't agree here.

With the perl version of kernel-doc, a typo in a file path doesn't cause
a fatal error of docs build.

kernel-doc as python class libs ends up in a fatal error.

Here is a log of such a fatal error (on top of current docs-next with
intentional typo made in a pathname in one of .. kernel-doc::

-----------------------------------------------------------------
Sphinx parallel build error!

Versions
========

* Platform:         linux; (Linux-6.8.0-59-generic-x86_64-with-glibc2.39)
* Python version:   3.12.3 (CPython)
* Sphinx version:   8.2.3
* Docutils version: 0.21.2
* Jinja2 version:   3.1.6
* Pygments version: 2.19.1

Last Messages
=============

    userspace-api/gpio/gpio-get-chipinfo-ioctl .. userspace-api/media/dvb/dmx-fclose


    reading sources... [ 90%]
    userspace-api/media/dvb/dmx-fopen .. userspace-api/media/mediactl/media-controller-model


    reading sources... [ 92%]
    userspace-api/media/mediactl/media-func-close .. userspace-api/media/v4l/diff-v4l

Loaded Extensions
=================

* sphinx.ext.mathjax (8.2.3)
* alabaster (1.0.0)
* sphinxcontrib.applehelp (2.0.0)
* sphinxcontrib.devhelp (2.0.0)
* sphinxcontrib.htmlhelp (2.1.0)
* sphinxcontrib.serializinghtml (2.0.0)
* sphinxcontrib.qthelp (2.0.0)
* kerneldoc (1.0)
* rstFlatTable (1.0)
* kernel_include (1.0)
* kfigure (1.0.0)
* sphinx.ext.ifconfig (8.2.3)
* automarkup (unknown version)
* maintainers_include (1.0)
* sphinx.ext.autosectionlabel (8.2.3)
* kernel_abi (1.0)
* kernel_feat (1.0)
* translations (unknown version)
* sphinx.ext.imgmath (8.2.3)

Traceback
=========

      File "/<...>/sphinx-8.2.3/lib/python3.12/site-packages/sphinx/util/parallel.py", line 137, in _join_one
        raise SphinxParallelError(*result)
    sphinx.errors.SphinxParallelError: KeyError: '/<...>/lib/bitmap-bad.c'


The full traceback has been saved in:
/tmp/sphinx-err-8jzxndsr.log

To report this error to the developers, please open an issue at <https://github.com/sphinx-doc/sphinx/issues/>. Thanks!
Please also report this if it was a user error, so that a better error message can be provided next time.
make[3]: *** [/<...>/Documentation/Makefile:123: htmldocs] Error 2
make[2]: *** [/<...>/Makefile:1806: htmldocs] Error 2
make[1]: *** [/<...>/Makefile:248: __sub-make] Error 2
make[1]: Leaving directory '/<...>/my-output'
make: *** [Makefile:248: __sub-make] Error 2

-----------------------------------------------------------------

This would surprise innocent devs who are kindly willing to test docs build.

I think you need to tame its behavior and make it emit a proper warning and
continue building docs in case of such predictable user errors.

Am I asking you something unreasonable?

Thanks,
Akira

>                           As kernel-doc is now fully an Sphinx extension,
> it now signalizes to Sphinx that the build has failed.
> 
> Yet, it should have produced some warnings. Maybe we need to specify a
> different log level with Sphinx to make it happen. I'll double check it
> and send a fix later on to kernel-doc.
> 
> Thanks,
> Mauro

Re: linux-next: build failure after merge all the trees

[Randy Dunlap]

On 5/14/25 7:33 PM, Akira Yokosawa wrote:
> [+CC linux-doc]
> 
> Hi,
> 
> On Thu, 8 May 2025 14:39:11 +0200, Mauro Carvalho Chehab wrote:
>> Em Thu, 8 May 2025 22:25:31 +1000
>> Stephen Rothwell <sfr@canb.auug.org.au> escreveu:
> [...]
> 
>>>
>>> So, I used "KERNELDOC=$(pwd)/scripts/kernel-doc.pl" and tried again.
>>>
>>> I got these (new) messages:
>>>
>>> Error: Cannot open file drivers/virt/coco/tsm-mr.c
>>> Error: Cannot open file drivers/virt/coco/tsm-mr.c
>>> WARNING: kernel-doc 'scripts/kernel-doc.pl -rst -enable-lineno -export drivers/virt/coco/tsm-mr.c' failed with return code 2
>>>
>>> (and a few other innocuous ones)
>>>
>>> So your guess is good.
>>>
>>> It would be nice to have the Python kernel-doc fixed as well as the
>>> devsec-tsm tree.
>>
>> With regards to kernel-doc, failing to build if a file is missing
>> is the right thing to do.
> 
> Mauro, I don't agree here.
> 
> With the perl version of kernel-doc, a typo in a file path doesn't cause
> a fatal error of docs build.
> 
> kernel-doc as python class libs ends up in a fatal error.
> 
> Here is a log of such a fatal error (on top of current docs-next with
> intentional typo made in a pathname in one of .. kernel-doc::
> 
> -----------------------------------------------------------------
> Sphinx parallel build error!
> 
> Versions
> ========
> 
> * Platform:         linux; (Linux-6.8.0-59-generic-x86_64-with-glibc2.39)
> * Python version:   3.12.3 (CPython)
> * Sphinx version:   8.2.3
> * Docutils version: 0.21.2
> * Jinja2 version:   3.1.6
> * Pygments version: 2.19.1
> 
> Last Messages
> =============
> 
>     userspace-api/gpio/gpio-get-chipinfo-ioctl .. userspace-api/media/dvb/dmx-fclose
> 
> 
>     reading sources... [ 90%]
>     userspace-api/media/dvb/dmx-fopen .. userspace-api/media/mediactl/media-controller-model
> 
> 
>     reading sources... [ 92%]
>     userspace-api/media/mediactl/media-func-close .. userspace-api/media/v4l/diff-v4l
> 
> Loaded Extensions
> =================
> 
> * sphinx.ext.mathjax (8.2.3)
> * alabaster (1.0.0)
> * sphinxcontrib.applehelp (2.0.0)
> * sphinxcontrib.devhelp (2.0.0)
> * sphinxcontrib.htmlhelp (2.1.0)
> * sphinxcontrib.serializinghtml (2.0.0)
> * sphinxcontrib.qthelp (2.0.0)
> * kerneldoc (1.0)
> * rstFlatTable (1.0)
> * kernel_include (1.0)
> * kfigure (1.0.0)
> * sphinx.ext.ifconfig (8.2.3)
> * automarkup (unknown version)
> * maintainers_include (1.0)
> * sphinx.ext.autosectionlabel (8.2.3)
> * kernel_abi (1.0)
> * kernel_feat (1.0)
> * translations (unknown version)
> * sphinx.ext.imgmath (8.2.3)
> 
> Traceback
> =========
> 
>       File "/<...>/sphinx-8.2.3/lib/python3.12/site-packages/sphinx/util/parallel.py", line 137, in _join_one
>         raise SphinxParallelError(*result)
>     sphinx.errors.SphinxParallelError: KeyError: '/<...>/lib/bitmap-bad.c'
> 
> 
> The full traceback has been saved in:
> /tmp/sphinx-err-8jzxndsr.log
> 
> To report this error to the developers, please open an issue at <https://github.com/sphinx-doc/sphinx/issues/>. Thanks!
> Please also report this if it was a user error, so that a better error message can be provided next time.
> make[3]: *** [/<...>/Documentation/Makefile:123: htmldocs] Error 2
> make[2]: *** [/<...>/Makefile:1806: htmldocs] Error 2
> make[1]: *** [/<...>/Makefile:248: __sub-make] Error 2
> make[1]: Leaving directory '/<...>/my-output'
> make: *** [Makefile:248: __sub-make] Error 2
> 
> -----------------------------------------------------------------
> 
> This would surprise innocent devs who are kindly willing to test docs build.
> 
> I think you need to tame its behavior and make it emit a proper warning and
> continue building docs in case of such predictable user errors.

Totally agree.
Thanks.

> Am I asking you something unreasonable?
> 
> Thanks,
> Akira
> 
>>                           As kernel-doc is now fully an Sphinx extension,
>> it now signalizes to Sphinx that the build has failed.
>>
>> Yet, it should have produced some warnings. Maybe we need to specify a
>> different log level with Sphinx to make it happen. I'll double check it
>> and send a fix later on to kernel-doc.
>>
>> Thanks,
>> Mauro
> 
> 

-- 
~Randy

Re: linux-next: build failure after merge all the trees

[Mauro Carvalho Chehab]

Em Mon, 19 May 2025 12:33:03 -0700
Randy Dunlap <rdunlap@infradead.org> escreveu:

> On 5/14/25 7:33 PM, Akira Yokosawa wrote:
> > [+CC linux-doc]
> > 
> > Hi,
> > 
> > On Thu, 8 May 2025 14:39:11 +0200, Mauro Carvalho Chehab wrote:  
> >> Em Thu, 8 May 2025 22:25:31 +1000
> >> Stephen Rothwell <sfr@canb.auug.org.au> escreveu:  
> > [...]
> >   
> >>>
> >>> So, I used "KERNELDOC=$(pwd)/scripts/kernel-doc.pl" and tried again.
> >>>
> >>> I got these (new) messages:
> >>>
> >>> Error: Cannot open file drivers/virt/coco/tsm-mr.c
> >>> Error: Cannot open file drivers/virt/coco/tsm-mr.c
> >>> WARNING: kernel-doc 'scripts/kernel-doc.pl -rst -enable-lineno -export drivers/virt/coco/tsm-mr.c' failed with return code 2
> >>>
> >>> (and a few other innocuous ones)
> >>>
> >>> So your guess is good.
> >>>
> >>> It would be nice to have the Python kernel-doc fixed as well as the
> >>> devsec-tsm tree.  
> >>
> >> With regards to kernel-doc, failing to build if a file is missing
> >> is the right thing to do.  
> > 
> > Mauro, I don't agree here.
> > 
> > With the perl version of kernel-doc, a typo in a file path doesn't cause
> > a fatal error of docs build.
> > 
> > kernel-doc as python class libs ends up in a fatal error.
> > 
> > Here is a log of such a fatal error (on top of current docs-next with
> > intentional typo made in a pathname in one of .. kernel-doc::
> > 
> > -----------------------------------------------------------------
> > Sphinx parallel build error!
> > 
> > Versions
> > ========
> > 
> > * Platform:         linux; (Linux-6.8.0-59-generic-x86_64-with-glibc2.39)
> > * Python version:   3.12.3 (CPython)
> > * Sphinx version:   8.2.3
> > * Docutils version: 0.21.2
> > * Jinja2 version:   3.1.6
> > * Pygments version: 2.19.1
> > 
> > Last Messages
> > =============
> > 
> >     userspace-api/gpio/gpio-get-chipinfo-ioctl .. userspace-api/media/dvb/dmx-fclose
> > 
> > 
> >     reading sources... [ 90%]
> >     userspace-api/media/dvb/dmx-fopen .. userspace-api/media/mediactl/media-controller-model
> > 
> > 
> >     reading sources... [ 92%]
> >     userspace-api/media/mediactl/media-func-close .. userspace-api/media/v4l/diff-v4l
> > 
> > Loaded Extensions
> > =================
> > 
> > * sphinx.ext.mathjax (8.2.3)
> > * alabaster (1.0.0)
> > * sphinxcontrib.applehelp (2.0.0)
> > * sphinxcontrib.devhelp (2.0.0)
> > * sphinxcontrib.htmlhelp (2.1.0)
> > * sphinxcontrib.serializinghtml (2.0.0)
> > * sphinxcontrib.qthelp (2.0.0)
> > * kerneldoc (1.0)
> > * rstFlatTable (1.0)
> > * kernel_include (1.0)
> > * kfigure (1.0.0)
> > * sphinx.ext.ifconfig (8.2.3)
> > * automarkup (unknown version)
> > * maintainers_include (1.0)
> > * sphinx.ext.autosectionlabel (8.2.3)
> > * kernel_abi (1.0)
> > * kernel_feat (1.0)
> > * translations (unknown version)
> > * sphinx.ext.imgmath (8.2.3)
> > 
> > Traceback
> > =========
> > 
> >       File "/<...>/sphinx-8.2.3/lib/python3.12/site-packages/sphinx/util/parallel.py", line 137, in _join_one
> >         raise SphinxParallelError(*result)
> >     sphinx.errors.SphinxParallelError: KeyError: '/<...>/lib/bitmap-bad.c'
> > 
> > 
> > The full traceback has been saved in:
> > /tmp/sphinx-err-8jzxndsr.log
> > 
> > To report this error to the developers, please open an issue at <https://github.com/sphinx-doc/sphinx/issues/>. Thanks!
> > Please also report this if it was a user error, so that a better error message can be provided next time.
> > make[3]: *** [/<...>/Documentation/Makefile:123: htmldocs] Error 2
> > make[2]: *** [/<...>/Makefile:1806: htmldocs] Error 2
> > make[1]: *** [/<...>/Makefile:248: __sub-make] Error 2
> > make[1]: Leaving directory '/<...>/my-output'
> > make: *** [Makefile:248: __sub-make] Error 2
> > 
> > -----------------------------------------------------------------
> > 
> > This would surprise innocent devs who are kindly willing to test docs build.
> > 
> > I think you need to tame its behavior and make it emit a proper warning and
> > continue building docs in case of such predictable user errors.  
> 
> Totally agree.

I also agree. 

The main difference between calling kernel-doc via a shell script or
via a Python class is that now errors flow via Sphinx logger class,
so they are subject to Sphinx filtering rules.

I double-checked: the logs are produced, and you can see them with "V=1",
but Sphinx is hiding them, perhaps because of some options passed through
sphinx-build call, or because they require them to have certain types.

A quick workaround would be to not use Sphinx logger anymore (see
enclosed). It has a side effect, though: we lose control of setting
it via V= variable, which is not good.

I'd like to test some other approaches before, but I was at Embedded
Recipes last week, without enough time to work on a proper solution.

Jon,

If you think we need something in place quickly, feel free to merge
it. Otherwise, I'll seek for another approach along the week.

Regards,
Mauro

[PATCH] docs: kerneldoc.py: don't use Sphinx logger

Unfortunately, currently Sphinx logger is suppressing too much, not
allowing warnings to be displayed. Disable it.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

diff --git a/Documentation/sphinx/kerneldoc.py b/Documentation/sphinx/kerneldoc.py
index b713a2c4a615..687121300291 100644
--- a/Documentation/sphinx/kerneldoc.py
+++ b/Documentation/sphinx/kerneldoc.py
@@ -311,7 +311,7 @@ def setup_kfiles(app):
     if kerneldoc_bin and kerneldoc_bin.endswith("kernel-doc.py"):
         print("Using Python kernel-doc")
         out_style = RestFormat()
-        kfiles = KernelFiles(out_style=out_style, logger=logger)
+        kfiles = KernelFiles(out_style=out_style)
     else:
         print(f"Using {kerneldoc_bin}")
 

Re: linux-next: build failure after merge all the trees

[Mauro Carvalho Chehab]

Em Tue, 20 May 2025 07:18:02 +0200
Mauro Carvalho Chehab <mchehab+huawei@kernel.org> escreveu:

> Em Mon, 19 May 2025 12:33:03 -0700
> Randy Dunlap <rdunlap@infradead.org> escreveu:
> 
> > On 5/14/25 7:33 PM, Akira Yokosawa wrote:  
> > > [+CC linux-doc]
> > > 
> > > Hi,
> > > 
> > > On Thu, 8 May 2025 14:39:11 +0200, Mauro Carvalho Chehab wrote:    
> > >> Em Thu, 8 May 2025 22:25:31 +1000
> > >> Stephen Rothwell <sfr@canb.auug.org.au> escreveu:    
> > > [...]
> > >     
> > >>>
> > >>> So, I used "KERNELDOC=$(pwd)/scripts/kernel-doc.pl" and tried again.
> > >>>
> > >>> I got these (new) messages:
> > >>>
> > >>> Error: Cannot open file drivers/virt/coco/tsm-mr.c
> > >>> Error: Cannot open file drivers/virt/coco/tsm-mr.c
> > >>> WARNING: kernel-doc 'scripts/kernel-doc.pl -rst -enable-lineno -export drivers/virt/coco/tsm-mr.c' failed with return code 2
> > >>>
> > >>> (and a few other innocuous ones)
> > >>>
> > >>> So your guess is good.
> > >>>
> > >>> It would be nice to have the Python kernel-doc fixed as well as the
> > >>> devsec-tsm tree.    
> > >>
> > >> With regards to kernel-doc, failing to build if a file is missing
> > >> is the right thing to do.    
> > > 
> > > Mauro, I don't agree here.
> > > 
> > > With the perl version of kernel-doc, a typo in a file path doesn't cause
> > > a fatal error of docs build.
> > > 
> > > kernel-doc as python class libs ends up in a fatal error.
> > > 
> > > Here is a log of such a fatal error (on top of current docs-next with
> > > intentional typo made in a pathname in one of .. kernel-doc::
> > > 
> > > -----------------------------------------------------------------
> > > Sphinx parallel build error!
> > > 
> > > Versions
> > > ========
> > > 
> > > * Platform:         linux; (Linux-6.8.0-59-generic-x86_64-with-glibc2.39)
> > > * Python version:   3.12.3 (CPython)
> > > * Sphinx version:   8.2.3
> > > * Docutils version: 0.21.2
> > > * Jinja2 version:   3.1.6
> > > * Pygments version: 2.19.1
> > > 
> > > Last Messages
> > > =============
> > > 
> > >     userspace-api/gpio/gpio-get-chipinfo-ioctl .. userspace-api/media/dvb/dmx-fclose
> > > 
> > > 
> > >     reading sources... [ 90%]
> > >     userspace-api/media/dvb/dmx-fopen .. userspace-api/media/mediactl/media-controller-model
> > > 
> > > 
> > >     reading sources... [ 92%]
> > >     userspace-api/media/mediactl/media-func-close .. userspace-api/media/v4l/diff-v4l
> > > 
> > > Loaded Extensions
> > > =================
> > > 
> > > * sphinx.ext.mathjax (8.2.3)
> > > * alabaster (1.0.0)
> > > * sphinxcontrib.applehelp (2.0.0)
> > > * sphinxcontrib.devhelp (2.0.0)
> > > * sphinxcontrib.htmlhelp (2.1.0)
> > > * sphinxcontrib.serializinghtml (2.0.0)
> > > * sphinxcontrib.qthelp (2.0.0)
> > > * kerneldoc (1.0)
> > > * rstFlatTable (1.0)
> > > * kernel_include (1.0)
> > > * kfigure (1.0.0)
> > > * sphinx.ext.ifconfig (8.2.3)
> > > * automarkup (unknown version)
> > > * maintainers_include (1.0)
> > > * sphinx.ext.autosectionlabel (8.2.3)
> > > * kernel_abi (1.0)
> > > * kernel_feat (1.0)
> > > * translations (unknown version)
> > > * sphinx.ext.imgmath (8.2.3)
> > > 
> > > Traceback
> > > =========
> > > 
> > >       File "/<...>/sphinx-8.2.3/lib/python3.12/site-packages/sphinx/util/parallel.py", line 137, in _join_one
> > >         raise SphinxParallelError(*result)
> > >     sphinx.errors.SphinxParallelError: KeyError: '/<...>/lib/bitmap-bad.c'
> > > 
> > > 
> > > The full traceback has been saved in:
> > > /tmp/sphinx-err-8jzxndsr.log
> > > 
> > > To report this error to the developers, please open an issue at <https://github.com/sphinx-doc/sphinx/issues/>. Thanks!
> > > Please also report this if it was a user error, so that a better error message can be provided next time.
> > > make[3]: *** [/<...>/Documentation/Makefile:123: htmldocs] Error 2
> > > make[2]: *** [/<...>/Makefile:1806: htmldocs] Error 2
> > > make[1]: *** [/<...>/Makefile:248: __sub-make] Error 2
> > > make[1]: Leaving directory '/<...>/my-output'
> > > make: *** [Makefile:248: __sub-make] Error 2
> > > 
> > > -----------------------------------------------------------------
> > > 
> > > This would surprise innocent devs who are kindly willing to test docs build.
> > > 
> > > I think you need to tame its behavior and make it emit a proper warning and
> > > continue building docs in case of such predictable user errors.    
> > 
> > Totally agree.  
> 
> I also agree. 
> 
> The main difference between calling kernel-doc via a shell script or
> via a Python class is that now errors flow via Sphinx logger class,
> so they are subject to Sphinx filtering rules.
> 
> I double-checked: the logs are produced, and you can see them with "V=1",
> but Sphinx is hiding them, perhaps because of some options passed through
> sphinx-build call, or because they require them to have certain types.
> 
> A quick workaround would be to not use Sphinx logger anymore (see
> enclosed). It has a side effect, though: we lose control of setting
> it via V= variable, which is not good.

Heh, V=1 is not actually affected by not using Sphinx logger inside
the class. So, I can't see a side effect of letting the kernel-doc
use directly Python logger instead of the Sphinx variant.

So, I submitted the fix at:
	https://lore.kernel.org/linux-doc/cover.1747719873.git.mchehab+huawei@kernel.org/T/#t

As we may want to revisit it later in the future, in case Sphinx makes
something more fancy there, I added a comment at the patch.

Thanks,
Mauro