[PATCH 0/2] Better handle and document Python needs for Kernel build

[PATCH 0/2] Better handle and document Python needs for Kernel build

[Mauro Carvalho Chehab]

Hi Jon,

As promised, I'm sending you the missing patches from the
python backward-compatibility series. They're not urgent,
IMO it should be OK to have them merged for 6.17.

The first patch fix process/changes.rst. Currently, it makes one 
think that Python is optional. While not having python may work
on some environments, there are some python scripts called 
during Kernel build time that require python.

Document them. 

Also, in order to avoid breaking compilation, make kernel-doc's
main script executable with elder Python versions. 

Backward-compatibility tested with Python 2.7 and 3.4 (although
it may work since 3.2).  On elder versions, it would emit a 
warning and do nothing, as the actual code depends on features
at 3.6 (f-strings) and 3.7 (ordered dict).

Mauro Carvalho Chehab (2):
  docs: kernel-doc: avoid script crash on ancient Python
  docs: changes: better document Python needs

 Documentation/process/changes.rst | 13 ++++++++++-
 scripts/kernel-doc.py             | 37 ++++++++++++++++++++++---------
 2 files changed, 38 insertions(+), 12 deletions(-)

-- 
2.49.0

[PATCH 1/2] docs: kernel-doc: avoid script crash on ancient Python

[Mauro Carvalho Chehab]

While we do need at least 3.6 for kernel-doc to work, and at least
3.7 for it to output functions and structs with parameters at the
right order, let the python binary be compatible with legacy
versions.

The rationale is that the Kernel build nowadays calls kernel-doc
with -none on some places. Better not to bail out when older
versions are found.

With that, potentially this will run with python 2.7 and 3.2+,
according with vermin:

	$ vermin --no-tips -v ./scripts/kernel-doc
	Detecting python files..
	Analyzing using 24 processes..
	2.7, 3.2     /new_devel/v4l/docs/scripts/kernel-doc
	Minimum required versions: 2.7, 3.2

I actually tested it with versions 2.7 and 3.4 using anaconda
(version 3 < 3.4 was not available there). For both, the
script didn't crash and emitted the right warning:

	$ conda create -n py27 python=2.7
	$ conda activate py27
	$ python ./scripts/kernel-doc --none include/media
	Error: Python 3.6 or later is required by kernel-doc
	$ conda deactivate

	$ conda create -n py34 python=3.4
	$ conda activate py34
	python --version
        Python 3.4.5
        $ python ./scripts/kernel-doc --none include/media
	Error: Python 3.6 or later is required by kernel-doc
	$ conda deactivate

	$ python --version
	Python 3.13.5
        $ python ./scripts/kernel-doc --none include/media
	(no warnings and script ran properly)

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 scripts/kernel-doc.py | 37 ++++++++++++++++++++++++++-----------
 1 file changed, 26 insertions(+), 11 deletions(-)

diff --git a/scripts/kernel-doc.py b/scripts/kernel-doc.py
index fc3d46ef519f..d4f9188d6a19 100755
--- a/scripts/kernel-doc.py
+++ b/scripts/kernel-doc.py
@@ -1,9 +1,19 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
 # SPDX-License-Identifier: GPL-2.0
 # Copyright(c) 2025: Mauro Carvalho Chehab <mchehab@kernel.org>.
 #
-# pylint: disable=C0103,R0915
-#
+# pylint: disable=C0103,R0912,R0914,R0915
+
+# NOTE: While kernel-doc requires at least version 3.6 to run, the
+#       command line explicitly supports Python 2,7 and 3.4+.
+#       The rationale is that it shall fail gracefully during Kernel
+#       compilation with older Kernel versions. Due to that:
+#       - encoding line is needed here;
+#       - no f-strings can be used on this file.
+#       - the libraries that require newer versions can only be included
+#         after Python version is checked.
+
 # Converted from the kernel-doc script originally written in Perl
 # under GPLv2, copyrighted since 1998 by the following authors:
 #
@@ -107,9 +117,6 @@ SRC_DIR = os.path.dirname(os.path.realpath(__file__))
 
 sys.path.insert(0, os.path.join(SRC_DIR, LIB_DIR))
 
-from kdoc_files import KernelFiles                      # pylint: disable=C0413
-from kdoc_output import RestFormat, ManFormat           # pylint: disable=C0413
-
 DESC = """
 Read C language source or header FILEs, extract embedded documentation comments,
 and print formatted documentation to standard output.
@@ -273,14 +280,22 @@ def main():
 
     python_ver = sys.version_info[:2]
     if python_ver < (3,6):
-        logger.warning("Python 3.6 or later is required by kernel-doc")
+        # Depending on Kernel configuration, kernel-doc --none is called at
+        # build time. As we don't want to break compilation due to the
+        # usage of an old Python version, return 0 here.
+        if args.none:
+            logger.error("Python 3.6 or later is required by kernel-doc. skipping checks")
+            sys.exit(0)
 
-        # Return 0 here to avoid breaking compilation
-        sys.exit(0)
+        sys.exit("Python 3.6 or later is required by kernel-doc. Aborting.")
 
     if python_ver < (3,7):
         logger.warning("Python 3.7 or later is required for correct results")
 
+    # Import kernel-doc libraries only after checking Python version
+    from kdoc_files import KernelFiles                  # pylint: disable=C0415
+    from kdoc_output import RestFormat, ManFormat       # pylint: disable=C0415
+
     if args.man:
         out_style = ManFormat(modulename=args.modulename)
     elif args.none:
@@ -308,11 +323,11 @@ def main():
         sys.exit(0)
 
     if args.werror:
-        print(f"{error_count} warnings as errors")
+        print("%s warnings as errors" % error_count)    # pylint: disable=C0209
         sys.exit(error_count)
 
     if args.verbose:
-        print(f"{error_count} errors")
+        print("%s errors" % error_count)                # pylint: disable=C0209
 
     if args.none:
         sys.exit(0)
-- 
2.49.0

Re: [PATCH 1/2] docs: kernel-doc: avoid script crash on ancient Python

[Akira Yokosawa]

Hi,

On Mon, 28 Jul 2025 16:54:28 +0200, Mauro Carvalho Chehab wrote:
> While we do need at least 3.6 for kernel-doc to work, and at least
> 3.7 for it to output functions and structs with parameters at the
> right order, let the python binary be compatible with legacy
                              code?

> versions.
> 
[...]
> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> ---
>  scripts/kernel-doc.py | 37 ++++++++++++++++++++++++++-----------
>  1 file changed, 26 insertions(+), 11 deletions(-)
> 
> diff --git a/scripts/kernel-doc.py b/scripts/kernel-doc.py
> index fc3d46ef519f..d4f9188d6a19 100755
> --- a/scripts/kernel-doc.py
> +++ b/scripts/kernel-doc.py
> @@ -1,9 +1,19 @@
> -#!/usr/bin/env python3
> +#!/usr/bin/env python

This would conflict with my existing setup which has
/usr/bin/python3 only.

Please don't do this!

        Thanks, Akira

Re: [PATCH 1/2] docs: kernel-doc: avoid script crash on ancient Python

[Jani Nikula]

On Tue, 29 Jul 2025, Akira Yokosawa <akiyks@gmail.com> wrote:
> Hi,
>
> On Mon, 28 Jul 2025 16:54:28 +0200, Mauro Carvalho Chehab wrote:
>> While we do need at least 3.6 for kernel-doc to work, and at least
>> 3.7 for it to output functions and structs with parameters at the
>> right order, let the python binary be compatible with legacy
>                               code?
>
>> versions.
>> 
> [...]
>> 
>> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
>> ---
>>  scripts/kernel-doc.py | 37 ++++++++++++++++++++++++++-----------
>>  1 file changed, 26 insertions(+), 11 deletions(-)
>> 
>> diff --git a/scripts/kernel-doc.py b/scripts/kernel-doc.py
>> index fc3d46ef519f..d4f9188d6a19 100755
>> --- a/scripts/kernel-doc.py
>> +++ b/scripts/kernel-doc.py
>> @@ -1,9 +1,19 @@
>> -#!/usr/bin/env python3
>> +#!/usr/bin/env python
>
> This would conflict with my existing setup which has
> /usr/bin/python3 only.
>
> Please don't do this!

Agreed, this breaks more than it fixes.

Python 2.7 reached end-of-life over five years ago. Do we really have to
cater for ancient stuff? Which actual real world cases do not have
Python 3+ available? Please just let it go, and see if anyone ever
notices?

BR,
Jani.


-- 
Jani Nikula, Intel

Re: [PATCH 1/2] docs: kernel-doc: avoid script crash on ancient Python

[Jonathan Corbet]

Jani Nikula <jani.nikula@linux.intel.com> writes:

> Agreed, this breaks more than it fixes.
>
> Python 2.7 reached end-of-life over five years ago. Do we really have to
> cater for ancient stuff? Which actual real world cases do not have
> Python 3+ available? Please just let it go, and see if anyone ever
> notices?

I kind of have to agree.  The only real Python 2 user I know about is
OpenOffice.org, which promises to get off any year now.  Meanwhile, as I
recall, the advice from the Python project is to say "python3" and not
count on bare "python" being available.

jon

[PATCH 2/2] docs: changes: better document Python needs

[Mauro Carvalho Chehab]

Python is listed as an optional dependency, but this is not
true, as:

1) CONFIG_LTO_CLANG runs a python script at scripts/Makefile.vmlinux_o;

2) kernel-doc is called during compilation when some DRM options
   like CONFIG_DRM_I915_WERROR are enabled;

3) allyesconfig/allmodconfig will enable both.

So, better document that.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 Documentation/process/changes.rst | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
index bccfa19b45df..f6abecf6da86 100644
--- a/Documentation/process/changes.rst
+++ b/Documentation/process/changes.rst
@@ -61,7 +61,7 @@ Sphinx\ [#f1]_         3.4.3            sphinx-build --version
 GNU tar                1.28             tar --version
 gtags (optional)       6.6.5            gtags --version
 mkimage (optional)     2017.01          mkimage --version
-Python (optional)      3.9.x            python3 --version
+Python                 3.9.x            python3 --version
 GNU AWK (optional)     5.1.0            gawk --version
 ====================== ===============  ========================================
 
@@ -154,6 +154,17 @@ Perl
 You will need perl 5 and the following modules: ``Getopt::Long``,
 ``Getopt::Std``, ``File::Basename``, and ``File::Find`` to build the kernel.
 
+Python
+------
+
+CONFIG_LTO_CLANG requires python 2.7 or 3.0+; some DRM config options like
+CONFIG_DRM_I915_WERROR require at least Python 2.7 or 3.4+.
+
+The kernel-doc tool and docs build require at least 3.6, but they depend on
+dict changes that happened on 3.7 to produce valid results.
+
+Other tools within the Kernel tree require newer versions.
+
 BC
 --
 
-- 
2.49.0

Re: [PATCH 2/2] docs: changes: better document Python needs

[Akira Yokosawa]

[+CC Laurent and Jani]

Hi,

On Mon, 28 Jul 2025 16:54:29 +0200, Mauro Carvalho Chehab wrote:
> Python is listed as an optional dependency, but this is not
> true, as:
> 
> 1) CONFIG_LTO_CLANG runs a python script at scripts/Makefile.vmlinux_o;
> 
> 2) kernel-doc is called during compilation when some DRM options
>    like CONFIG_DRM_I915_WERROR are enabled;
> 
> 3) allyesconfig/allmodconfig will enable both.

Well, these conditions still sound to me optional.

> 
> So, better document that.
> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> ---
>  Documentation/process/changes.rst | 13 ++++++++++++-
>  1 file changed, 12 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
> index bccfa19b45df..f6abecf6da86 100644
> --- a/Documentation/process/changes.rst
> +++ b/Documentation/process/changes.rst
> @@ -61,7 +61,7 @@ Sphinx\ [#f1]_         3.4.3            sphinx-build --version
>  GNU tar                1.28             tar --version
>  gtags (optional)       6.6.5            gtags --version
>  mkimage (optional)     2017.01          mkimage --version
> -Python (optional)      3.9.x            python3 --version
> +Python                 3.9.x            python3 --version
>  GNU AWK (optional)     5.1.0            gawk --version
>  ====================== ===============  ========================================
>  
> @@ -154,6 +154,17 @@ Perl
>  You will need perl 5 and the following modules: ``Getopt::Long``,
>  ``Getopt::Std``, ``File::Basename``, and ``File::Find`` to build the kernel.
>  
> +Python
> +------
> +
> +CONFIG_LTO_CLANG requires python 2.7 or 3.0+; some DRM config options like
> +CONFIG_DRM_I915_WERROR require at least Python 2.7 or 3.4+.
> +
> +The kernel-doc tool and docs build require at least 3.6, but they depend on
> +dict changes that happened on 3.7 to produce valid results.
> +
> +Other tools within the Kernel tree require newer versions.
> +

These details look confusing in changes.rst.  The table above says
python >=3.9.x.  All you need to say here would just something lile:

+Python
+------
+
+Required with CONFIG_LTO_CLANG, some DRM config options like
+CONFIG_DRM_I915_WERROR, the kernel-doc tool, and docs build (Sphinx),
+among others.

Other details can go to comments in each script or other docs if
necessary.

        Thanks, Akira

Re: [PATCH 2/2] docs: changes: better document Python needs

[Mauro Carvalho Chehab]

Em Tue, 29 Jul 2025 19:35:57 +0900
Akira Yokosawa <akiyks@gmail.com> escreveu:

> [+CC Laurent and Jani]
> 
> Hi,
> 
> On Mon, 28 Jul 2025 16:54:29 +0200, Mauro Carvalho Chehab wrote:
> > Python is listed as an optional dependency, but this is not
> > true, as:
> > 
> > 1) CONFIG_LTO_CLANG runs a python script at scripts/Makefile.vmlinux_o;
> > 
> > 2) kernel-doc is called during compilation when some DRM options
> >    like CONFIG_DRM_I915_WERROR are enabled;
> > 
> > 3) allyesconfig/allmodconfig will enable both.  
> 
> Well, these conditions still sound to me optional.

Then we agree to disagree: if a missing package prevents building
the kernel with allyesconfig/allmodconfig, IMO it is mandatory.

In any case, as Jani pointed out, DRM_MSM, which is part of arm
default configs also require Python. Building with clang also
requires it on several archs. There are other parts that
seem to require it, but I didn't dig into it.

So, IMHO, even if you consider allmodconfig/allyesconfig as
"optional", and you also consider clang as "optional" (it is marked
as optional at changes), it is still mandatory for arm 32-bit most
common  config (multi_v7_defconfig) and for arm64 defconfig.

It sounds pretty much mandatory to me: if a distro doesn't have
it, it can't support arm architectures.

IMO, it should not be tagged as optional there. At max, a note
under Python description could mention what scenarios one still
may not have it.

Now, adding a note like that means that someone would need to be
periodically updating it when people add more auto-generated
stuff that may require it.

> > So, better document that.
> > 
> > Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> > ---
> >  Documentation/process/changes.rst | 13 ++++++++++++-
> >  1 file changed, 12 insertions(+), 1 deletion(-)
> > 
> > diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
> > index bccfa19b45df..f6abecf6da86 100644
> > --- a/Documentation/process/changes.rst
> > +++ b/Documentation/process/changes.rst
> > @@ -61,7 +61,7 @@ Sphinx\ [#f1]_         3.4.3            sphinx-build --version
> >  GNU tar                1.28             tar --version
> >  gtags (optional)       6.6.5            gtags --version
> >  mkimage (optional)     2017.01          mkimage --version
> > -Python (optional)      3.9.x            python3 --version
> > +Python                 3.9.x            python3 --version
> >  GNU AWK (optional)     5.1.0            gawk --version
> >  ====================== ===============  ========================================
> >  
> > @@ -154,6 +154,17 @@ Perl
> >  You will need perl 5 and the following modules: ``Getopt::Long``,
> >  ``Getopt::Std``, ``File::Basename``, and ``File::Find`` to build the kernel.
> >  
> > +Python
> > +------
> > +
> > +CONFIG_LTO_CLANG requires python 2.7 or 3.0+; some DRM config options like
> > +CONFIG_DRM_I915_WERROR require at least Python 2.7 or 3.4+.
> > +
> > +The kernel-doc tool and docs build require at least 3.6, but they depend on
> > +dict changes that happened on 3.7 to produce valid results.
> > +
> > +Other tools within the Kernel tree require newer versions.
> > +  
> 
> These details look confusing in changes.rst.  The table above says
> python >=3.9.x.  All you need to say here would just something lile:
> 
> +Python
> +------
> +
> +Required with CONFIG_LTO_CLANG, some DRM config options like
> +CONFIG_DRM_I915_WERROR, the kernel-doc tool, and docs build (Sphinx),
> +among others.
> 
> Other details can go to comments in each script or other docs if
> necessary.

Maybe something like:

Python
------

Several config options require it: it is required for arm/arm64
default configs, CONFIG_LTO_CLANG, some DRM optional configs,
the kernel-doc tool, and docs build (Sphinx), among others.

Regards,
Mauro