Re: linux-next: Tree for Sep 19 (make htmldocs problem)

Re: linux-next: Tree for Sep 19 (make htmldocs problem)

[Randy Dunlap]

Hi,

On 9/19/25 8:05 AM, Mark Brown wrote:

With today's linux-next, when I do 'make O=DOC1 htmldocs', I get:

make[1]: Entering directory '/home/rdunlap/lnx/repo/linux-next/DOC1'
../Documentation/Makefile:71: warning: overriding recipe for target 'pdfdocs'
../Documentation/Makefile:62: warning: ignoring old recipe for target 'pdfdocs'
  File "/usr/bin/sphinx-build", line 1
    ELF
SyntaxError: source code cannot contain null bytes
make[1]: Leaving directory '/home/rdunlap/lnx/repo/linux-next/DOC1'

where the "ELF" line contains some binary bytes that are not shown
via copy/paste. Here they are in hex in case that might help:

7f 45 4c 46 02 01 01 0a              .ELF....


I don't see what is causing this, so I am using the previous day's
linux-next for Documentation testing etc...

Any ideas/suggestions appreciated.

-- 
~Randy

Re: linux-next: Tree for Sep 19 (make htmldocs problem)

[Akira Yokosawa]

Hi Randy,

On Sat, 20 Sep 2025 17:17:56 -0700, Randy Dunlap wrote:
> Hi,
> 
> On 9/19/25 8:05 AM, Mark Brown wrote:
> 
> With today's linux-next, when I do 'make O=DOC1 htmldocs', I get:
> 
> make[1]: Entering directory '/home/rdunlap/lnx/repo/linux-next/DOC1'
> ../Documentation/Makefile:71: warning: overriding recipe for target 'pdfdocs'
> ../Documentation/Makefile:62: warning: ignoring old recipe for target 'pdfdocs'
>   File "/usr/bin/sphinx-build", line 1
>     ELF
> SyntaxError: source code cannot contain null bytes
> make[1]: Leaving directory '/home/rdunlap/lnx/repo/linux-next/DOC1'
> 
> where the "ELF" line contains some binary bytes that are not shown
> via copy/paste. Here they are in hex in case that might help:
> 
> 7f 45 4c 46 02 01 01 0a              .ELF....
> 

Hmm, my /usr/bin/sphinx-build (under Ubuntu 24.04) is a python script.

Running "file /usr/bin/sphinx-build" says:

    /usr/bin/sphinx-build: Python script, ASCII text executable

Randy, how did you install the sphinx-build?  And which distro are you
on?

> 
> I don't see what is causing this, so I am using the previous day's
> linux-next for Documentation testing etc...

Another question ...

Does going back to a past linux-next make sphinx-build work again?

If that's the case ..., I'm not sure, but it might be related to
Mauro's sphinx-build-wrapper, which is now in docs-next (not in docs-mw
for v6.18 at the moment).

Randy, what happens if you revert the latter part of Mauro's series, namely,

  819667bc3ccd ("tools/docs: sphinx-build-wrapper: add a wrapper for sphinx-build")
  2e1760999e58 ("docs: parallel-wrapper.sh: remove script")

  ... up to ...

  42180ada39da ("tools/docs: sphinx-build-wrapper: add support to run inside venv")

on top of next-20250919 ?

Regards,
Akira

> 
> Any ideas/suggestions appreciated.
> 
> -- 
> ~Randy
>

Re: linux-next: Tree for Sep 19 (make htmldocs problem)

[Jonathan Corbet]

Mauro, have you seen this ... any ideas ... ?  Randy, what can you say
about the environment you're running when you hit this problem?

(It doesn't reproduce here).

jon

Randy Dunlap <rdunlap@infradead.org> writes:

> Hi,
>
> On 9/19/25 8:05 AM, Mark Brown wrote:
>
> With today's linux-next, when I do 'make O=DOC1 htmldocs', I get:
>
> make[1]: Entering directory '/home/rdunlap/lnx/repo/linux-next/DOC1'
> ../Documentation/Makefile:71: warning: overriding recipe for target 'pdfdocs'
> ../Documentation/Makefile:62: warning: ignoring old recipe for target 'pdfdocs'
>   File "/usr/bin/sphinx-build", line 1
>     ELF
> SyntaxError: source code cannot contain null bytes
> make[1]: Leaving directory '/home/rdunlap/lnx/repo/linux-next/DOC1'
>
> where the "ELF" line contains some binary bytes that are not shown
> via copy/paste. Here they are in hex in case that might help:
>
> 7f 45 4c 46 02 01 01 0a              .ELF....
>
>
> I don't see what is causing this, so I am using the previous day's
> linux-next for Documentation testing etc...
>
> Any ideas/suggestions appreciated.
>
> -- 
> ~Randy

Re: linux-next: Tree for Sep 19 (make htmldocs problem)

[Randy Dunlap]

On 9/21/25 7:03 AM, Jonathan Corbet wrote:
> Mauro, have you seen this ... any ideas ... ?  Randy, what can you say
> about the environment you're running when you hit this problem?
> 
> (It doesn't reproduce here).
> 

openSUSE/tumbleweed current.

Python version: 3.13.7
Docutils version: 0.21.2
Using alabaster theme
Using Python kernel-doc
sphinx-build 8.2.3

lrwxrwxrwx 1 root root 4 Sep 11 01:42 /usr/bin/sphinx-build -> alts*

$ file  /usr/bin/alts
/usr/bin/alts: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 4.3.0, BuildID[sha1]=17681640c9985eb36ae6d9eca0f08159509386c4, stripped


> 
> Randy Dunlap <rdunlap@infradead.org> writes:
> 
>> Hi,
>>
>> On 9/19/25 8:05 AM, Mark Brown wrote:
>>
>> With today's linux-next, when I do 'make O=DOC1 htmldocs', I get:
>>
>> make[1]: Entering directory '/home/rdunlap/lnx/repo/linux-next/DOC1'
>> ../Documentation/Makefile:71: warning: overriding recipe for target 'pdfdocs'
>> ../Documentation/Makefile:62: warning: ignoring old recipe for target 'pdfdocs'
>>   File "/usr/bin/sphinx-build", line 1
>>     ELF
>> SyntaxError: source code cannot contain null bytes
>> make[1]: Leaving directory '/home/rdunlap/lnx/repo/linux-next/DOC1'
>>
>> where the "ELF" line contains some binary bytes that are not shown
>> via copy/paste. Here they are in hex in case that might help:
>>
>> 7f 45 4c 46 02 01 01 0a              .ELF....
>>
>>
>> I don't see what is causing this, so I am using the previous day's
>> linux-next for Documentation testing etc...
>>
>> Any ideas/suggestions appreciated.
>>
>> -- 
>> ~Randy

-- 
~Randy

Re: linux-next: Tree for Sep 19 (make htmldocs problem)

[Jonathan Corbet]

Randy Dunlap <rdunlap@infradead.org> writes:

> lrwxrwxrwx 1 root root 4 Sep 11 01:42 /usr/bin/sphinx-build -> alts*
>
> $ file  /usr/bin/alts
> /usr/bin/alts: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 4.3.0, BuildID[sha1]=17681640c9985eb36ae6d9eca0f08159509386c4, stripped

That is clearly the problem, when combined with this code in
sphinx-build-wrapper:

>             if self.venv:
>                 cmd = ["python"]
>             else:
>                 cmd = [sys.executable,]
> 
>             cmd += [sphinx_build]

Mauro, what is the reason for explicitly interposing the interpreter
there rather than just invoking the sphinx-build binary directly?  It
seems like we could take that out and make this problem go away?

Thanks,

jon

Re: linux-next: Tree for Sep 19 (make htmldocs problem)

[Mauro Carvalho Chehab]

Em Sun, 21 Sep 2025 13:27:48 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> Randy Dunlap <rdunlap@infradead.org> writes:
> 
> > lrwxrwxrwx 1 root root 4 Sep 11 01:42 /usr/bin/sphinx-build -> alts*
> >
> > $ file  /usr/bin/alts
> > /usr/bin/alts: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 4.3.0, BuildID[sha1]=17681640c9985eb36ae6d9eca0f08159509386c4, stripped  

Such setup looks weird on my eyes... why is it pointing to some other
exec?

> 
> That is clearly the problem, when combined with this code in
> sphinx-build-wrapper:
> 
> >             if self.venv:
> >                 cmd = ["python"]
> >             else:
> >                 cmd = [sys.executable,]

It ended that an extra comma is here, added on one of my final
rebases. I noticed it already and one of the patches I sent during
the weekend fixes it.

> > 
> >             cmd += [sphinx_build]  
> 
> Mauro, what is the reason for explicitly interposing the interpreter
> there rather than just invoking the sphinx-build binary directly?  It
> seems like we could take that out and make this problem go away?

I added it to ensure that it would be using exactly the same python
version that was called via makefile, which is called with:

	Documentation/Makefile: +$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ \

See, the other alternatives:

	# This may not run the same Python version, as it doesn't
	# take PYTHON3 var into account
	cmd = [sphinx_build]

and:
	# This would require some extra logic for it to work 
	# when calling shinx-build-wrapper directly from command line
	cmd = [PYTHON3, sphinx_build]

are, IMO, more problematic. The actual problem here seems to be that
/usr/bin/sphinx-build is not the actual sphinx-build, but something
else.

Randy,

do you know what is this "alts" file? Is it a custom script or
binary?


Thanks,
Mauro

Re: linux-next: Tree for Sep 19 (make htmldocs problem)

[Jonathan Corbet]

Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:

> do you know what is this "alts" file? Is it a custom script or
> binary?

It's the alternatives mechanism that openSUSE uses.  I have no idea why
they felt the need to employ it for Sphinx, but they did, so we need to
work with that kind of configuration.  Meaning, really, I think we have
to just invoke sphinx-build directly rather than trying to control which
version of Python it may ultimately get.

jon

Re: linux-next: Tree for Sep 19 (make htmldocs problem)

[Mauro Carvalho Chehab]

Em Sun, 21 Sep 2025 14:44:48 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
> 
> > do you know what is this "alts" file? Is it a custom script or
> > binary?  
> 
> It's the alternatives mechanism that openSUSE uses. 

See my RFC patch. It fixes the issue when PYTHON3 is not explicitly set.

> I have no idea why
> they felt the need to employ it for Sphinx, but they did, so we need to
> work with that kind of configuration.  Meaning, really, I think we have
> to just invoke sphinx-build directly rather than trying to control which
> version of Python it may ultimately get.

True, but if we ignore PYTHON3 env completely, this will break on
other setups where sphinx is installed with a different python version
(with includes OpenSUSE non-thumbleweed).

Thanks,
Mauro

Re: linux-next: Tree for Sep 19 (make htmldocs problem)

[Jonathan Corbet]

Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:

>> I have no idea why
>> they felt the need to employ it for Sphinx, but they did, so we need to
>> work with that kind of configuration.  Meaning, really, I think we have
>> to just invoke sphinx-build directly rather than trying to control which
>> version of Python it may ultimately get.
>
> True, but if we ignore PYTHON3 env completely, this will break on
> other setups where sphinx is installed with a different python version
> (with includes OpenSUSE non-thumbleweed).

How do those setups work prior to your changes?  We didn't have to do
that dance before, right?  What makes it necessary now?

Thanks,

jon

Re: linux-next: Tree for Sep 19 (make htmldocs problem)

[Mauro Carvalho Chehab]

Em Sun, 21 Sep 2025 15:56:37 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
> 
> >> I have no idea why
> >> they felt the need to employ it for Sphinx, but they did, so we need to
> >> work with that kind of configuration.  Meaning, really, I think we have
> >> to just invoke sphinx-build directly rather than trying to control which
> >> version of Python it may ultimately get.  
> >
> > True, but if we ignore PYTHON3 env completely, this will break on
> > other setups where sphinx is installed with a different python version
> > (with includes OpenSUSE non-thumbleweed).  
> 
> How do those setups work prior to your changes?  We didn't have to do
> that dance before, right?  What makes it necessary now?

We didn't, but in recent past minimal python version was below 3.6.

I added this after a feedback during patch review that it should be 
using PYTHON3 env var to better support Leap and other distros with
native python version < 3.7 and would require a pyhton version override 
to build docs.

Thanks,
Mauro

Re: linux-next: Tree for Sep 19 (make htmldocs problem)

[Mauro Carvalho Chehab]

Em Sun, 21 Sep 2025 22:32:50 +0200
Mauro Carvalho Chehab <mchehab+huawei@kernel.org> escreveu:

> Em Sun, 21 Sep 2025 13:27:48 -0600
> Jonathan Corbet <corbet@lwn.net> escreveu:
> 
> > Randy Dunlap <rdunlap@infradead.org> writes:
> > 
> > > lrwxrwxrwx 1 root root 4 Sep 11 01:42 /usr/bin/sphinx-build -> alts*
> > >
> > > $ file  /usr/bin/alts
> > > /usr/bin/alts: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 4.3.0, BuildID[sha1]=17681640c9985eb36ae6d9eca0f08159509386c4, stripped  
> 
> Such setup looks weird on my eyes... why is it pointing to some other
> exec?
> 
> > 
> > That is clearly the problem, when combined with this code in
> > sphinx-build-wrapper:
> > 
> > >             if self.venv:
> > >                 cmd = ["python"]
> > >             else:
> > >                 cmd = [sys.executable,]
> 
> It ended that an extra comma is here, added on one of my final
> rebases. I noticed it already and one of the patches I sent during
> the weekend fixes it.
> 
> > > 
> > >             cmd += [sphinx_build]  
> > 
> > Mauro, what is the reason for explicitly interposing the interpreter
> > there rather than just invoking the sphinx-build binary directly?  It
> > seems like we could take that out and make this problem go away?
> 
> I added it to ensure that it would be using exactly the same python
> version that was called via makefile, which is called with:
> 
> 	Documentation/Makefile: +$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ \
> 
> See, the other alternatives:
> 
> 	# This may not run the same Python version, as it doesn't
> 	# take PYTHON3 var into account
> 	cmd = [sphinx_build]

This breaks when one is using PYTHON3 env with something different
than python3...
> 
> and:
> 	# This would require some extra logic for it to work 
> 	# when calling shinx-build-wrapper directly from command line
> 	cmd = [PYTHON3, sphinx_build]

Heh, this would require a change at Kernel's main Makefile,
and will likely break Kernel compilation, or an explicit check
if PYTHON3 != "python3". See suggested fix below.

Thanks,
Mauro

---

[PATCH] [RFC] tools/docs: sphinx-build-wrapper: accept non-python sphinx-build

At least on some environments, sphinx-build is an ELF executable,
with some weird alternate logic. This causes the logic which
honours PYTHON3 env var to break.

Change the logic to pick PYTHON3 directly from the environment,
using it only when explicitly set with a value different than python3.13.

Reported-by: Randy Dunlap <rdunlap@infradead.org>
Link: https://lore.kernel.org/all/883df949-0281-4a39-8745-bcdcce3a5594@infradead.org/
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

diff --git a/tools/docs/sphinx-build-wrapper b/tools/docs/sphinx-build-wrapper
index 1f9c66ab33fe..ccffc51d9caf 100755
--- a/tools/docs/sphinx-build-wrapper
+++ b/tools/docs/sphinx-build-wrapper
@@ -274,8 +274,10 @@ class SphinxBuilder:
 
             if self.venv:
                 cmd = ["python"]
+            elif self.env.get("PYTHON3", "python3") != "python3":
+                cmd = [self.env["PYTHON3"]]
             else:
-                cmd = [sys.executable]
+                cmd = []
 
             cmd += [sphinx_build]
             cmd += [f"-j{n_jobs}"]