[PATCH v2 0/3] Some kernel-doc fixes

[PATCH v2 0/3] Some kernel-doc fixes

[Mauro Carvalho Chehab]

Hi Jon,

That's the second version of the kernel-doc fixup patch series.

I discovered the root cause why Sphinx logger was not working: there
was a call there for logger.verbose(). According with:

	https://www.sphinx-doc.org/en/master/extdev/logging.html

This is a valid call, but it doesn't verbose messages. Instead, it is
crashing with modern Sphinx versions, causing the log to not work.

I got rid of it, replacing by logger.info().  I took the time to also
address an issue pointed by Andy: not having the same log message
placed everywhere. With such change, we can keep using Sphinx
logger (which produces colored messages) inside kernel-doc
classes.

With that, we have:

Patch 1:	makes Lore and kernel-doc ML receive patches related
	to kernel-doc.py and get_abi.py.
Patch 2:	cleanup try/except logic and get rid of logger.verbose();
Patch 3:	fix a KeyError when trying to acess data from non-existing files;

If you test just patch 1 on the top of next-20250516, you'll see the
keyerror message (in red):

	ERROR: Cannot find file ./drivers/gpio/gpiolib-acpi.c
	ERROR: Cannot find file ./drivers/gpio/gpiolib-acpi.c
	WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno -export ./drivers/gpio/gpiolib-acpi.c' processing failed with: KeyError('./drivers/gpio/gpiolib-acpi.c')

And the script doesn't crash anymore. After patch 2, the try/except
warning gets replaced by a proper message:

	ERROR: Cannot find file ./drivers/gpio/gpiolib-acpi.c
	ERROR: Cannot find file ./drivers/gpio/gpiolib-acpi.c
	WARNING: No kernel-doc for file ./drivers/gpio/gpiolib-acpi.c

Please apply it to solve the issues that are reported by Stephen and Akira.

Regards,
Mauro
Mauro Carvalho Chehab (3):
  MAINTAINERS: update linux-doc entry to cover new Python scripts
  docs: kerneldoc.py: simplify exception handling logic
  scripts: kernel-doc: prevent a KeyError when checking output

 Documentation/sphinx/kerneldoc.py | 108 +++++++++++++-----------------
 MAINTAINERS                       |   5 +-
 scripts/lib/kdoc/kdoc_files.py    |   4 ++
 3 files changed, 53 insertions(+), 64 deletions(-)

-- 
2.49.0

[PATCH v2 1/3] MAINTAINERS: update linux-doc entry to cover new Python scripts

[Mauro Carvalho Chehab]

Changes to ABI and kernel-doc need to be c/c linux-doc. Update
the maintainer's entry to cover those files.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 MAINTAINERS | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index 20e07e61a148..a668808769b6 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -7173,7 +7173,10 @@ T:	git git://git.lwn.net/linux.git docs-next
 F:	Documentation/
 F:	scripts/check-variable-fonts.sh
 F:	scripts/documentation-file-ref-check
-F:	scripts/kernel-doc
+F:	scripts/get_abi.py
+F:	scripts/kernel-doc*
+F	scripts/lib/abi/*
+F	scripts/lib/kdoc/*
 F:	scripts/sphinx-pre-install
 X:	Documentation/ABI/
 X:	Documentation/admin-guide/media/
-- 
2.49.0

[PATCH v2 2/3] docs: kerneldoc.py: simplify exception handling logic

[Mauro Carvalho Chehab]

Get rid of logger.verbose() which is causing the logger to not
work.

Also, instead of having try/except everywhere, place them on a
common place.

While here, get rid of some bogus logs.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 Documentation/sphinx/kerneldoc.py | 108 +++++++++++++-----------------
 1 file changed, 45 insertions(+), 63 deletions(-)

diff --git a/Documentation/sphinx/kerneldoc.py b/Documentation/sphinx/kerneldoc.py
index b713a2c4a615..b818d4c77924 100644
--- a/Documentation/sphinx/kerneldoc.py
+++ b/Documentation/sphinx/kerneldoc.py
@@ -40,6 +40,7 @@ from docutils.parsers.rst import directives, Directive
 import sphinx
 from sphinx.util.docutils import switch_source_input
 from sphinx.util import logging
+from pprint import pformat
 
 srctree = os.path.abspath(os.environ["srctree"])
 sys.path.insert(0, os.path.join(srctree, "scripts/lib/kdoc"))
@@ -49,7 +50,7 @@ from kdoc_output import RestFormat
 
 __version__  = '1.0'
 kfiles = None
-logger = logging.getLogger('kerneldoc')
+logger = logging.getLogger(__name__)
 
 def cmd_str(cmd):
     """
@@ -190,46 +191,31 @@ class KernelDocDirective(Directive):
 
         return cmd
 
-    def run_cmd(self):
+    def run_cmd(self, cmd):
         """
         Execute an external kernel-doc command.
         """
 
         env = self.state.document.settings.env
-        cmd = self.handle_args()
-
-        if self.verbose >= 1:
-            print(cmd_str(cmd))
-
         node = nodes.section()
 
-        try:
-            logger.verbose("calling kernel-doc '%s'" % (" ".join(cmd)))
+        p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+        out, err = p.communicate()
 
-            p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
-            out, err = p.communicate()
+        out, err = codecs.decode(out, 'utf-8'), codecs.decode(err, 'utf-8')
 
-            out, err = codecs.decode(out, 'utf-8'), codecs.decode(err, 'utf-8')
+        if p.returncode != 0:
+            sys.stderr.write(err)
 
-            if p.returncode != 0:
-                sys.stderr.write(err)
-
-                logger.warning("kernel-doc '%s' failed with return code %d"
-                                    % (" ".join(cmd), p.returncode))
-                return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))]
-            elif env.config.kerneldoc_verbosity > 0:
-                sys.stderr.write(err)
-
-        except Exception as e:  # pylint: disable=W0703
-            logger.warning("kernel-doc '%s' processing failed with: %s" %
-                                (" ".join(cmd), str(e)))
+            logger.warning("kernel-doc '%s' failed with return code %d"
+                                % (" ".join(cmd), p.returncode))
             return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))]
+        elif env.config.kerneldoc_verbosity > 0:
+            sys.stderr.write(err)
 
         filenames = self.parse_args["file_list"]
         for filename in filenames:
-            ret = self.parse_msg(filename, node, out, cmd)
-            if ret:
-                return ret
+            self.parse_msg(filename, node, out, cmd)
 
         return node.children
 
@@ -240,40 +226,31 @@ class KernelDocDirective(Directive):
 
         env = self.state.document.settings.env
 
-        try:
-            lines = statemachine.string2lines(out, self.tab_width,
-                                              convert_whitespace=True)
-            result = ViewList()
+        lines = statemachine.string2lines(out, self.tab_width,
+                                            convert_whitespace=True)
+        result = ViewList()
 
-            lineoffset = 0;
-            line_regex = re.compile(r"^\.\. LINENO ([0-9]+)$")
-            for line in lines:
-                match = line_regex.search(line)
-                if match:
-                    # sphinx counts lines from 0
-                    lineoffset = int(match.group(1)) - 1
-                    # we must eat our comments since the upset the markup
-                else:
-                    doc = str(env.srcdir) + "/" + env.docname + ":" + str(self.lineno)
-                    result.append(line, doc + ": " + filename, lineoffset)
-                    lineoffset += 1
+        lineoffset = 0;
+        line_regex = re.compile(r"^\.\. LINENO ([0-9]+)$")
+        for line in lines:
+            match = line_regex.search(line)
+            if match:
+                # sphinx counts lines from 0
+                lineoffset = int(match.group(1)) - 1
+                # we must eat our comments since the upset the markup
+            else:
+                doc = str(env.srcdir) + "/" + env.docname + ":" + str(self.lineno)
+                result.append(line, doc + ": " + filename, lineoffset)
+                lineoffset += 1
 
-            self.do_parse(result, node)
+        self.do_parse(result, node)
 
-        except Exception as e:  # pylint: disable=W0703
-            logger.warning("kernel-doc '%s' processing failed with: %s" %
-                                (cmd_str(cmd), str(e)))
-            return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))]
-
-        return None
-
-    def run_kdoc(self, kfiles):
+    def run_kdoc(self, cmd, kfiles):
         """
         Execute kernel-doc classes directly instead of running as a separate
         command.
         """
 
-        cmd = self.handle_args()
         env = self.state.document.settings.env
 
         node = nodes.section()
@@ -282,22 +259,27 @@ class KernelDocDirective(Directive):
         filenames = self.parse_args["file_list"]
 
         for filename, out in kfiles.msg(**self.msg_args, filenames=filenames):
-            if self.verbose >= 1:
-                print(cmd_str(cmd))
-
-            ret = self.parse_msg(filename, node, out, cmd)
-            if ret:
-                return ret
+            self.parse_msg(filename, node, out, cmd)
 
         return node.children
 
     def run(self):
         global kfiles
 
-        if kfiles:
-            return self.run_kdoc(kfiles)
-        else:
-            return self.run_cmd()
+        cmd = self.handle_args()
+        if self.verbose >= 1:
+            logger.info(cmd_str(cmd))
+
+        try:
+            if kfiles:
+                return self.run_kdoc(cmd, kfiles)
+            else:
+                return self.run_cmd(cmd)
+
+        except Exception as e:  # pylint: disable=W0703
+            logger.warning("kernel-doc '%s' processing failed with: %s" %
+                           (cmd_str(cmd), pformat(e)))
+            return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))]
 
     def do_parse(self, result, node):
         with switch_source_input(self.state, result):
-- 
2.49.0

[PATCH v2 3/3] scripts: kernel-doc: prevent a KeyError when checking output

[Mauro Carvalho Chehab]

If a file sent to KernelFiles.msg() method doesn't exist, instead
of producing a KeyError, output an error message.

Reported-by: Randy Dunlap <rdunlap@infradead.org>
Closes: https://lore.kernel.org/linux-doc/cover.1747719873.git.mchehab+huawei@kernel.org/T/#ma43ae9d8d0995b535cf5099e5381dace0410de04
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 scripts/lib/kdoc/kdoc_files.py | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/scripts/lib/kdoc/kdoc_files.py b/scripts/lib/kdoc/kdoc_files.py
index 630aa5ca6460..9be4a64df71d 100644
--- a/scripts/lib/kdoc/kdoc_files.py
+++ b/scripts/lib/kdoc/kdoc_files.py
@@ -271,6 +271,10 @@ class KernelFiles():
                                       no_doc_sections)
 
             msg = ""
+            if fname not in self.results:
+                self.config.log.warning("No kernel-doc for file %s", fname)
+                continue
+
             for name, arg in self.results[fname]:
                 m = self.out_msg(fname, name, arg)
 
-- 
2.49.0

Re: [PATCH v2 1/3] MAINTAINERS: update linux-doc entry to cover new Python scripts

[Akira Yokosawa]

Hi Mauro,

On Wed, 21 May 2025 08:59:31 +0200, Mauro Carvalho Chehab wrote:
> Changes to ABI and kernel-doc need to be c/c linux-doc. Update
> the maintainer's entry to cover those files.

See below for a couple of trivial typo. 

> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> ---
>  MAINTAINERS | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 20e07e61a148..a668808769b6 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -7173,7 +7173,10 @@ T:	git git://git.lwn.net/linux.git docs-next
>  F:	Documentation/
>  F:	scripts/check-variable-fonts.sh
>  F:	scripts/documentation-file-ref-check
> -F:	scripts/kernel-doc
> +F:	scripts/get_abi.py
> +F:	scripts/kernel-doc*
> +F	scripts/lib/abi/*
   F:	scripts/lib/abi/*

> +F	scripts/lib/kdoc/*
   F:	scripts/lib/kdoc/*

>  F:	scripts/sphinx-pre-install
>  X:	Documentation/ABI/
>  X:	Documentation/admin-guide/media/

Regards,
Akira

Re: [PATCH v2 0/3] Some kernel-doc fixes

[Akira Yokosawa]

Hi,

On Wed, 21 May 2025 08:59:30 +0200, Mauro Carvalho Chehab wrote:
> Hi Jon,
> 
> That's the second version of the kernel-doc fixup patch series.
> 
> I discovered the root cause why Sphinx logger was not working: there
> was a call there for logger.verbose(). According with:
> 
> 	https://www.sphinx-doc.org/en/master/extdev/logging.html
> 
> This is a valid call, but it doesn't verbose messages. Instead, it is
> crashing with modern Sphinx versions, causing the log to not work.
> 
> I got rid of it, replacing by logger.info().  I took the time to also
> address an issue pointed by Andy: not having the same log message
> placed everywhere. With such change, we can keep using Sphinx
> logger (which produces colored messages) inside kernel-doc
> classes.
> 
> With that, we have:
> 
> Patch 1:	makes Lore and kernel-doc ML receive patches related
> 	to kernel-doc.py and get_abi.py.
> Patch 2:	cleanup try/except logic and get rid of logger.verbose();
> Patch 3:	fix a KeyError when trying to acess data from non-existing files;
> 
> If you test just patch 1 on the top of next-20250516, you'll see the
> keyerror message (in red):
> 
> 	ERROR: Cannot find file ./drivers/gpio/gpiolib-acpi.c
> 	ERROR: Cannot find file ./drivers/gpio/gpiolib-acpi.c
> 	WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno -export ./drivers/gpio/gpiolib-acpi.c' processing failed with: KeyError('./drivers/gpio/gpiolib-acpi.c')
> 
> And the script doesn't crash anymore. After patch 2, the try/except
> warning gets replaced by a proper message:
> 
> 	ERROR: Cannot find file ./drivers/gpio/gpiolib-acpi.c
> 	ERROR: Cannot find file ./drivers/gpio/gpiolib-acpi.c
> 	WARNING: No kernel-doc for file ./drivers/gpio/gpiolib-acpi.c
> 
> Please apply it to solve the issues that are reported by Stephen and Akira.

Other than the small typo in 1/3, this looks promising.

For the series,
Acked-by: Akira Yokosawa <akiyks@gmail.com>

Regards,
Akira

> 
> Regards,
> Mauro
> Mauro Carvalho Chehab (3):
>   MAINTAINERS: update linux-doc entry to cover new Python scripts
>   docs: kerneldoc.py: simplify exception handling logic
>   scripts: kernel-doc: prevent a KeyError when checking output
> 
>  Documentation/sphinx/kerneldoc.py | 108 +++++++++++++-----------------
>  MAINTAINERS                       |   5 +-
>  scripts/lib/kdoc/kdoc_files.py    |   4 ++
>  3 files changed, 53 insertions(+), 64 deletions(-)
>