[PATCH v2 3/2] docs: changes: better document Python needs

[PATCH v2 3/2] docs: changes: better document Python needs

[Mauro Carvalho Chehab]

Python is listed as an optional dependency, but this is not
true, as kernel-doc is called during compilation when DRM is
enabled. Better document that.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 Documentation/process/changes.rst | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
index bccfa19b45df..6a7d7c1ee274 100644
--- a/Documentation/process/changes.rst
+++ b/Documentation/process/changes.rst
@@ -61,7 +61,7 @@ Sphinx\ [#f1]_         3.4.3            sphinx-build --version
 GNU tar                1.28             tar --version
 gtags (optional)       6.6.5            gtags --version
 mkimage (optional)     2017.01          mkimage --version
-Python (optional)      3.9.x            python3 --version
+Python                 3.9.x            python3 --version
 GNU AWK (optional)     5.1.0            gawk --version
 ====================== ===============  ========================================
 
@@ -154,6 +154,13 @@ Perl
 You will need perl 5 and the following modules: ``Getopt::Long``,
 ``Getopt::Std``, ``File::Basename``, and ``File::Find`` to build the kernel.
 
+Python
+------
+
+At least Python 2.7 or 3.4 is required if CONFIG_DRM is selected to avoid
+breaking compilation. Documentation build and kernel-doc won't produce
+valid results if version is below 3.7.
+
 BC
 --
 
-- 
2.50.0

Re: [PATCH v2 3/2] docs: changes: better document Python needs

[Laurent Pinchart]

Hi Mauro,

Thank you for the patch.

On Sat, Jul 12, 2025 at 10:32:38AM +0200, Mauro Carvalho Chehab wrote:
> Python is listed as an optional dependency, but this is not
> true, as kernel-doc is called during compilation when DRM is
> enabled. Better document that.
> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

Isn't it only when CONFIG_DRM_HEADER_TEST is enabled ? That option
depends on EXPERT && BROKEN, so I wouldn't expect it to be widely
enabled. A quick grep shows that CONFIG_DRM_I915_WERROR does the same
(with a dependency on EXPERT but not BROKEN though).

Is there something else in DRM that invokes kernel-doc ?

> ---
>  Documentation/process/changes.rst | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
> index bccfa19b45df..6a7d7c1ee274 100644
> --- a/Documentation/process/changes.rst
> +++ b/Documentation/process/changes.rst
> @@ -61,7 +61,7 @@ Sphinx\ [#f1]_         3.4.3            sphinx-build --version
>  GNU tar                1.28             tar --version
>  gtags (optional)       6.6.5            gtags --version
>  mkimage (optional)     2017.01          mkimage --version
> -Python (optional)      3.9.x            python3 --version
> +Python                 3.9.x            python3 --version
>  GNU AWK (optional)     5.1.0            gawk --version
>  ====================== ===============  ========================================
>  
> @@ -154,6 +154,13 @@ Perl
>  You will need perl 5 and the following modules: ``Getopt::Long``,
>  ``Getopt::Std``, ``File::Basename``, and ``File::Find`` to build the kernel.
>  
> +Python
> +------
> +
> +At least Python 2.7 or 3.4 is required if CONFIG_DRM is selected to avoid
> +breaking compilation. Documentation build and kernel-doc won't produce
> +valid results if version is below 3.7.
> +
>  BC
>  --
>  

-- 
Regards,

Laurent Pinchart

Re: [PATCH v2 3/2] docs: changes: better document Python needs

[Mauro Carvalho Chehab]

Em Sat, 12 Jul 2025 19:31:55 +0300
Laurent Pinchart <laurent.pinchart@ideasonboard.com> escreveu:

> Hi Mauro,
> 
> Thank you for the patch.
> 
> On Sat, Jul 12, 2025 at 10:32:38AM +0200, Mauro Carvalho Chehab wrote:
> > Python is listed as an optional dependency, but this is not
> > true, as kernel-doc is called during compilation when DRM is
> > enabled. Better document that.
> > 
> > Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>  
> 
> Isn't it only when CONFIG_DRM_HEADER_TEST is enabled ? That option
> depends on EXPERT && BROKEN, so I wouldn't expect it to be widely
> enabled. A quick grep shows that CONFIG_DRM_I915_WERROR does the same
> (with a dependency on EXPERT but not BROKEN though).

Well, EXPERT is currently enabled on several distros. The three ones I have
it handy all have it:

Fedora 42:
	$ grep CONFIG_EXPERT /boot/config*
	/boot/config-6.14.9-300.fc42.x86_64:CONFIG_EXPERT=y
	/boot/config-6.15.3-200.fc42.x86_64:CONFIG_EXPERT=y
	/boot/config-6.15.4-200.fc42.x86_64:CONFIG_EXPERT=y

Ubuntu 24.10:
	$ grep CONFIG_EXPERT /boot/config*
	/boot/config-6.11.0-26-generic:CONFIG_EXPERT=y
	/boot/config-6.8.0-60-generic:CONFIG_EXPERT=y

Debian 12:
	$ grep CONFIG_EXPERT /boot/config*
	/boot/config-6.1.0-34-amd64:CONFIG_EXPERT=y
	/boot/config-6.1.0-37-amd64:CONFIG_EXPERT=y
	/boot/config-6.1.0-37-rt-amd64:CONFIG_EXPERT=y

So, expert on distros seem quite common those days.

Fedora has it enabled for a long time. On Fedora 42:

	$ grep CONFIG_DRM_WERROR /boot/config*
	/boot/config-6.14.9-300.fc42.x86_64:CONFIG_DRM_WERROR=y
	/boot/config-6.15.3-200.fc42.x86_64:CONFIG_DRM_WERROR=y
	/boot/config-6.15.4-200.fc42.x86_64:CONFIG_DRM_WERROR=y

I would expect to have it enabled on other distros as well.

> Is there something else in DRM that invokes kernel-doc ?
> 
> > ---
> >  Documentation/process/changes.rst | 9 ++++++++-
> >  1 file changed, 8 insertions(+), 1 deletion(-)
> > 
> > diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
> > index bccfa19b45df..6a7d7c1ee274 100644
> > --- a/Documentation/process/changes.rst
> > +++ b/Documentation/process/changes.rst
> > @@ -61,7 +61,7 @@ Sphinx\ [#f1]_         3.4.3            sphinx-build --version
> >  GNU tar                1.28             tar --version
> >  gtags (optional)       6.6.5            gtags --version
> >  mkimage (optional)     2017.01          mkimage --version
> > -Python (optional)      3.9.x            python3 --version
> > +Python                 3.9.x            python3 --version
> >  GNU AWK (optional)     5.1.0            gawk --version
> >  ====================== ===============  ========================================
> >  
> > @@ -154,6 +154,13 @@ Perl
> >  You will need perl 5 and the following modules: ``Getopt::Long``,
> >  ``Getopt::Std``, ``File::Basename``, and ``File::Find`` to build the kernel.
> >  
> > +Python
> > +------
> > +
> > +At least Python 2.7 or 3.4 is required if CONFIG_DRM is selected to avoid
> > +breaking compilation. Documentation build and kernel-doc won't produce
> > +valid results if version is below 3.7.

Maybe I can place instead CONFIG_DRM_I915_WERROR.


Thanks,
Mauro

Re: [PATCH v2 3/2] docs: changes: better document Python needs

[Laurent Pinchart]

On Sun, Jul 13, 2025 at 12:25:17AM +0200, Mauro Carvalho Chehab wrote:
> Em Sat, 12 Jul 2025 19:31:55 +0300 Laurent Pinchart escreveu:
> 
> > Hi Mauro,
> > 
> > Thank you for the patch.
> > 
> > On Sat, Jul 12, 2025 at 10:32:38AM +0200, Mauro Carvalho Chehab wrote:
> > > Python is listed as an optional dependency, but this is not
> > > true, as kernel-doc is called during compilation when DRM is
> > > enabled. Better document that.
> > > 
> > > Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>  
> > 
> > Isn't it only when CONFIG_DRM_HEADER_TEST is enabled ? That option
> > depends on EXPERT && BROKEN, so I wouldn't expect it to be widely
> > enabled. A quick grep shows that CONFIG_DRM_I915_WERROR does the same
> > (with a dependency on EXPERT but not BROKEN though).
> 
> Well, EXPERT is currently enabled on several distros. The three ones I have
> it handy all have it:
> 
> Fedora 42:
> 	$ grep CONFIG_EXPERT /boot/config*
> 	/boot/config-6.14.9-300.fc42.x86_64:CONFIG_EXPERT=y
> 	/boot/config-6.15.3-200.fc42.x86_64:CONFIG_EXPERT=y
> 	/boot/config-6.15.4-200.fc42.x86_64:CONFIG_EXPERT=y
> 
> Ubuntu 24.10:
> 	$ grep CONFIG_EXPERT /boot/config*
> 	/boot/config-6.11.0-26-generic:CONFIG_EXPERT=y
> 	/boot/config-6.8.0-60-generic:CONFIG_EXPERT=y
> 
> Debian 12:
> 	$ grep CONFIG_EXPERT /boot/config*
> 	/boot/config-6.1.0-34-amd64:CONFIG_EXPERT=y
> 	/boot/config-6.1.0-37-amd64:CONFIG_EXPERT=y
> 	/boot/config-6.1.0-37-rt-amd64:CONFIG_EXPERT=y
> 
> So, expert on distros seem quite common those days.

But not CONFIG_BROKEN, right ? That would leave only
CONFIG_DRM_I915_WERROR.

> Fedora has it enabled for a long time. On Fedora 42:
> 
> 	$ grep CONFIG_DRM_WERROR /boot/config*
> 	/boot/config-6.14.9-300.fc42.x86_64:CONFIG_DRM_WERROR=y
> 	/boot/config-6.15.3-200.fc42.x86_64:CONFIG_DRM_WERROR=y
> 	/boot/config-6.15.4-200.fc42.x86_64:CONFIG_DRM_WERROR=y
> 
> I would expect to have it enabled on other distros as well.

CONFIG_DRM_WERROR doesn't seem related to running kernel-doc though, is
it ?

> > Is there something else in DRM that invokes kernel-doc ?
> > 
> > > ---
> > >  Documentation/process/changes.rst | 9 ++++++++-
> > >  1 file changed, 8 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
> > > index bccfa19b45df..6a7d7c1ee274 100644
> > > --- a/Documentation/process/changes.rst
> > > +++ b/Documentation/process/changes.rst
> > > @@ -61,7 +61,7 @@ Sphinx\ [#f1]_         3.4.3            sphinx-build --version
> > >  GNU tar                1.28             tar --version
> > >  gtags (optional)       6.6.5            gtags --version
> > >  mkimage (optional)     2017.01          mkimage --version
> > > -Python (optional)      3.9.x            python3 --version
> > > +Python                 3.9.x            python3 --version
> > >  GNU AWK (optional)     5.1.0            gawk --version
> > >  ====================== ===============  ========================================
> > >  
> > > @@ -154,6 +154,13 @@ Perl
> > >  You will need perl 5 and the following modules: ``Getopt::Long``,
> > >  ``Getopt::Std``, ``File::Basename``, and ``File::Find`` to build the kernel.
> > >  
> > > +Python
> > > +------
> > > +
> > > +At least Python 2.7 or 3.4 is required if CONFIG_DRM is selected to avoid
> > > +breaking compilation. Documentation build and kernel-doc won't produce
> > > +valid results if version is below 3.7.
> 
> Maybe I can place instead CONFIG_DRM_I915_WERROR.

I think that's better. I also wouldn't consider CONFIG_DRM_I915_WERROR
to make Python "required".

-- 
Regards,

Laurent Pinchart

Re: [PATCH v2 3/2] docs: changes: better document Python needs

[Jonathan Corbet]

Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:

> Maybe I can place instead CONFIG_DRM_I915_WERROR.

I've held off on this series on the expectation that a new version would
come.  I guess, at this point, it will be a post-merge-window thing?

Thanks,

jon

Re: [PATCH v2 3/2] docs: changes: better document Python needs

[Mauro Carvalho Chehab]

Em Thu, 24 Jul 2025 08:42:59 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
> 
> > Maybe I can place instead CONFIG_DRM_I915_WERROR.
> 
> I've held off on this series on the expectation that a new version would
> come.  I guess, at this point, it will be a post-merge-window thing?

Feel free to postpone. I have already a new version of it here somewhere on
my branches, but I had to take some days off. So, I ended not sending you
the (probably) final version.

I intend to send what I have here during the merge window for you to
review and apply post-merge-window.

Thanks,
Mauro

Re: [PATCH v2 3/2] docs: changes: better document Python needs

[Jani Nikula]

On Thu, 24 Jul 2025, Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote:
> Em Thu, 24 Jul 2025 08:42:59 -0600
> Jonathan Corbet <corbet@lwn.net> escreveu:
>
>> Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
>> 
>> > Maybe I can place instead CONFIG_DRM_I915_WERROR.
>> 
>> I've held off on this series on the expectation that a new version would
>> come.  I guess, at this point, it will be a post-merge-window thing?
>
> Feel free to postpone. I have already a new version of it here somewhere on
> my branches, but I had to take some days off. So, I ended not sending you
> the (probably) final version.
>
> I intend to send what I have here during the merge window for you to
> review and apply post-merge-window.

I think the main questions here are 1) how to handle optional build tool
dependencies, and 2) whether Python is an optional or required
dependency.

It might be nice to be able to have an actual Kconfig and dependency for
optional tools. "depends on TOOL_PYTHON" or something. Enable the
option, and you should have Python.

This in turn raises the question for allyesconfig. It's cumbersome
(though not impossible) to add config options that you actually have to
enable manually.

The header test stuff really isn't required to actually build the kernel
or drm, however DRM_MSM does depend on Python for building the driver.


BR,
Jani.

-- 
Jani Nikula, Intel

Re: [PATCH v2 3/2] docs: changes: better document Python needs

[Mauro Carvalho Chehab]

Em Mon, 28 Jul 2025 12:28:45 +0300
Jani Nikula <jani.nikula@intel.com> escreveu:

> On Thu, 24 Jul 2025, Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote:
> > Em Thu, 24 Jul 2025 08:42:59 -0600
> > Jonathan Corbet <corbet@lwn.net> escreveu:
> >
> >> Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
> >> 
> >> > Maybe I can place instead CONFIG_DRM_I915_WERROR.
> >> 
> >> I've held off on this series on the expectation that a new version would
> >> come.  I guess, at this point, it will be a post-merge-window thing?
> >
> > Feel free to postpone. I have already a new version of it here somewhere on
> > my branches, but I had to take some days off. So, I ended not sending you
> > the (probably) final version.
> >
> > I intend to send what I have here during the merge window for you to
> > review and apply post-merge-window.
> 
> I think the main questions here are 1) how to handle optional build tool
> dependencies, and 2) whether Python is an optional or required
> dependency.

with regards to (2), besides doc build and kernel-doc --none, there is
at least another place at the building system requiring Python: 
scripts/Makefile.vmlinux_o: There, it has:

	quiet_cmd_gen_initcalls_lds = GEN     $@
	      cmd_gen_initcalls_lds = \
	        $(PYTHON3) $(srctree)/scripts/jobserver-exec \
	        $(PERL) $(real-prereqs) > $@

	.tmp_initcalls.lds: $(srctree)/scripts/generate_initcall_order.pl \
	                vmlinux.a $(KBUILD_VMLINUX_LIBS) FORCE
	        $(call if_changed,gen_initcalls_lds)

	targets := .tmp_initcalls.lds

	ifdef CONFIG_LTO_CLANG
	initcalls-lds := .tmp_initcalls.lds
	endif

Now, I didn't check exactly what conditions trigger
.tmp_initcalls.lds, but there are some places that use
select:

	arch/Kconfig:   select LTO_CLANG

On a quick look, it sounds that some archs will select this
automatically when built with clang.

I didn't check if other parts of the building system requires it.

In any case, on its current state, I'd say that currently this
is not optional.

> It might be nice to be able to have an actual Kconfig and dependency for
> optional tools. "depends on TOOL_PYTHON" or something. Enable the
> option, and you should have Python.

That would be an option. The question is: is it worth spending
time on it?

> This in turn raises the question for allyesconfig. It's cumbersome
> (though not impossible) to add config options that you actually have to
> enable manually.

IMO it doesn't make sense to manually enable something with *config.
If they depend on Python, be it: for such targets, Python is
mandatory.

> 
> The header test stuff really isn't required to actually build the kernel
> or drm, however DRM_MSM does depend on Python for building the driver.

Good to know. It means that, for those *config targets:

	arch/arm/configs/imx_v6_v7_defconfig:CONFIG_DRM_MSM=y
	arch/arm/configs/multi_v7_defconfig:CONFIG_DRM_MSM=m
	arch/arm/configs/qcom_defconfig:CONFIG_DRM_MSM=m

(plus all{mod|yes}config)

Python is mandatory. As multi_v7_defconfig is one of them, we
may assume, that, o practical cases, Python 2.7 or 3.2+ is 
mandatory for arm support. The current requirement is:

	$ vermin -v --no-tips ./drivers/gpu/drm/msm/registers/gen_header.py
	Detecting python files..
	Analyzing using 8 processes..
	2.7, 3.2     /new_devel/v4l/docs/drivers/gpu/drm/msm/registers/gen_header.py
	Minimum required versions: 2.7, 3.2

Heh, looking for files that end with .py at Makefile (not all
Python scripts at the Kernel end with such extension), it seems
that there are more:

	grep \\\.py $(git ls-files|grep Makefile|grep -v tools)


	Makefile:KERNELDOC       = $(srctree)/scripts/kernel-doc.py
	Makefile:		  vmlinux-gdb.py \
	Makefile:	$(Q)ln -fsn $(abspath $(srctree)/scripts/gdb/vmlinux-gdb.py)
	Makefile:compile_commands.json: $(srctree)/scripts/clang-tools/gen_compile_commands.py \
	Makefile:      cmd_clang_tools = $(PYTHON3) $(srctree)/scripts/clang-tools/run-clang-tools.py $@ $<
	drivers/gpu/drm/msm/Makefile:      cmd_headergen = mkdir -p $(obj)/generated && $(PYTHON3) $(src)/registers/gen_header.py \
	drivers/gpu/drm/msm/Makefile:		$(src)/registers/gen_header.py \
	drivers/gpu/drm/msm/Makefile:		$(src)/registers/gen_header.py \
	drivers/tty/vt/Makefile:#GENERATE_UCS_TABLES := 2  # invokes gen_ucs_recompose_table.py with --full
	drivers/tty/vt/Makefile:$(obj)/ucs_width_table.h: $(src)/gen_ucs_width_table.py
	drivers/tty/vt/Makefile:$(obj)/ucs_recompose_table.h: $(src)/gen_ucs_recompose_table.py
	drivers/tty/vt/Makefile:$(obj)/ucs_fallback_table.h: $(src)/gen_ucs_fallback_table.py
	rust/Makefile:	$(Q)MAKEFLAGS= $(srctree)/scripts/generate_rust_analyzer.py \
	scripts/Makefile.lib:MAKE_FIT := $(srctree)/scripts/make_fit.py
	scripts/gdb/linux/Makefile:symlinks := $(patsubst $(src)/%,%,$(wildcard $(src)/*.py))
	scripts/gdb/linux/Makefile:always-y += constants.py
	scripts/gdb/linux/Makefile:$(obj)/constants.py: $(src)/constants.py.in FORCE
	scripts/gdb/linux/Makefile:clean-files := *.pyc *.pyo

(that not including tools/*)

Some seem false positives, but on the other hand, looking for tools,
several scripts seem to be executed by non-tools Makefiles. I didn't
check if any of them are written in python, though.

Considering the above, for me it seems that the bus already 
departed: there are several cases where Python is required during
build time. So, adding a "depends on TOOL_PYTHON" doesn't seem
to be trivial.

From my side, I don't mind much about that, as this is something
that affects only the Kernel build. I would very much prefer to
have things like config BPF optional.

Regards,
Mauro

Re: [PATCH v2 3/2] docs: changes: better document Python needs

[Jani Nikula]

On Mon, 28 Jul 2025, Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote:
> Considering the above, for me it seems that the bus already departed:
> there are several cases where Python is required during build time.

FWIW, if it was up to me, I'd make Python 3+ a non-optional build
dependency. I'd also forget about any Python 2 backward compat stuff.

I would find it very useful for code/header generation during build
time, instead of having to resort to C hostprogs. Similar to what MSM is
doing.

That said, I know there's going to be people vehemently opposed.

> So, adding a "depends on TOOL_PYTHON" doesn't seem to be trivial.

Agreed. Forget about that idea.


BR,
Jani.


-- 
Jani Nikula, Intel

Re: [PATCH v2 3/2] docs: changes: better document Python needs

[Mauro Carvalho Chehab]

Em Tue, 29 Jul 2025 13:45:26 +0300
Jani Nikula <jani.nikula@intel.com> escreveu:

> On Mon, 28 Jul 2025, Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote:
> > Considering the above, for me it seems that the bus already departed:
> > there are several cases where Python is required during build time.  
> 
> FWIW, if it was up to me, I'd make Python 3+ a non-optional build
> dependency. I'd also forget about any Python 2 backward compat stuff.

I don't think we should do much effort to support Python 2, but it comes
almost for free: only shebang needs to be different, and, if the comments
inside the doc contains non-utf8 chars, an encoding line.

The current tools during Kernel build currently supports it (again,
except for shebang).

Anyway, from my side I'm happy either way.

> I would find it very useful for code/header generation during build
> time, instead of having to resort to C hostprogs. Similar to what MSM is
> doing.
> 
> That said, I know there's going to be people vehemently opposed.

Fine from my side ;-) There are some precedents here, so I guess it
should be up to each subsystem to decide using it or not.

> > So, adding a "depends on TOOL_PYTHON" doesn't seem to be trivial.  
> 
> Agreed. Forget about that idea.
> 
> 
> BR,
> Jani.
> 
> 

Re: [PATCH v2 3/2] docs: changes: better document Python needs

[Jani Nikula]

On Tue, 29 Jul 2025, Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote:
> I don't think we should do much effort to support Python 2, but it comes
> almost for free: only shebang needs to be different, and, if the comments
> inside the doc contains non-utf8 chars, an encoding line.

As said in the other thread, changing the shebang is not a trivial
thing. I don't have 'python' installed, haven't had for years, just
'python3'.

BR,
Jani.


-- 
Jani Nikula, Intel