[PATCH 0/3] Some kernel-doc fixes

[PATCH 0/3] Some kernel-doc fixes

[Mauro Carvalho Chehab]

Hi Jon,

This small series contain 3 patches:
- patch 1 fixes PDF docs build, as reported by Akira;
  (I'm resending this one as-is from its v2)
- patch 2 addresses a complain from Jani about not being able
  of disabling "-q" flag when building docs with V=0;
- patch 3 addresses an issue indirectly reported by Jani that
  it the env vars that affects the wrapper aren't documented.

With regards to patch 2, docs build honours V=0 by adding a
"-q" flag.

When V=1 is set, there are two effects in place:

1. sphix-build will be called without "-q";
2. Sphinx extensions will increase their verbosity levels.

Sometimes, it is desired to just remove "-q" without increasing
extensions verbosity. That's what patch 2 does.

IMO, at least patch 1 should be merged during Kernel v6.21
development cycle.

Mauro Carvalho Chehab (3):
  docs: kdoc: Fix pdfdocs build for tools
  docs: sphinx-build-wrapper: allow -v override -q
  tools: sphinx-build-wrapper: improve its help message

 tools/docs/sphinx-build-wrapper  | 42 +++++++++++++++++++++++++++-----
 tools/lib/python/kdoc/kdoc_re.py | 10 +++++---
 2 files changed, 43 insertions(+), 9 deletions(-)

-- 
2.52.0

[PATCH 1/3] docs: kdoc: Fix pdfdocs build for tools

[Mauro Carvalho Chehab]

the "\1" inside a docstring requires proper scaping to not be
considered a hex character and break the build.

Reported-by: Akira Yokosawa <akiyks@gmail.com>
Closes: https://lore.kernel.org/linux-doc/63e99049-cc72-4156-83af-414fdde34312@gmail.com/
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 tools/lib/python/kdoc/kdoc_re.py | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/tools/lib/python/kdoc/kdoc_re.py b/tools/lib/python/kdoc/kdoc_re.py
index 2816bd9f90f8..0bf9e01cdc57 100644
--- a/tools/lib/python/kdoc/kdoc_re.py
+++ b/tools/lib/python/kdoc/kdoc_re.py
@@ -228,14 +228,18 @@ class NestedMatch:
             yield line[t[0]:t[2]]
 
     def sub(self, regex, sub, line, count=0):
-        """
+        r"""
         This is similar to re.sub:
 
         It matches a regex that it is followed by a delimiter,
         replacing occurrences only if all delimiters are paired.
 
-        if r'\1' is used, it works just like re: it places there the
-        matched paired data with the delimiter stripped.
+        if the sub argument contains::
+
+            r'\1'
+
+        it will work just like re: it places there the matched paired data
+        with the delimiter stripped.
 
         If count is different than zero, it will replace at most count
         items.
-- 
2.52.0

[PATCH 2/3] docs: sphinx-build-wrapper: allow -v override -q

[Mauro Carvalho Chehab]

Documentation builds were using "-q" for a long time, but sometimes
it is nice to see the Sphinx progress, without increasing build
verbosity - which would also turn on kernel-doc verbosity.

Instead of doing that, let's parse the sphinx-build already-existing
-v: each time it is used, it increases the verbosity level.

With that, if the default is to use -q, a single -v will disable
quiet mode. Passing more -v will keep increasing its verbosity.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 tools/docs/sphinx-build-wrapper | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/tools/docs/sphinx-build-wrapper b/tools/docs/sphinx-build-wrapper
index 78ff7ac202ef..8080ace60680 100755
--- a/tools/docs/sphinx-build-wrapper
+++ b/tools/docs/sphinx-build-wrapper
@@ -168,6 +168,7 @@ class SphinxBuilder:
         parser = argparse.ArgumentParser()
         parser.add_argument('-j', '--jobs', type=int)
         parser.add_argument('-q', '--quiet', action='store_true')
+        parser.add_argument('-v', '--verbose', default=0, action='count')
 
         #
         # Other sphinx-build arguments go as-is, so place them
@@ -179,10 +180,14 @@ class SphinxBuilder:
         # Build a list of sphinx args, honoring verbosity here if specified
         #
 
-        verbose = self.verbose
         sphinx_args, self.sphinxopts = parser.parse_known_args(sphinxopts)
+
+        verbose = sphinx_args.verbose
+        if self.verbose:
+            verbose += 1
+
         if sphinx_args.quiet is True:
-            verbose = False
+            verbose = 0
 
         #
         # If the user explicitly sets "-j" at command line, use it.
@@ -195,8 +200,11 @@ class SphinxBuilder:
         else:
             self.n_jobs = None
 
-        if not verbose:
+        if verbose < 1:
             self.sphinxopts += ["-q"]
+        else:
+            for i in range(1, sphinx_args.verbose):
+                self.sphinxopts += ["-v"]
 
     def __init__(self, builddir, venv=None, verbose=False, n_jobs=None,
                  interactive=None):
-- 
2.52.0

[PATCH 3/3] tools: sphinx-build-wrapper: improve its help message

[Mauro Carvalho Chehab]

Besides the parameters that are passed via command line arguments,
the wrapper's behavior is affected by several environment variables.

Document that. While here, use __doc__ for its description.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 tools/docs/sphinx-build-wrapper | 28 +++++++++++++++++++++++++---
 1 file changed, 25 insertions(+), 3 deletions(-)

diff --git a/tools/docs/sphinx-build-wrapper b/tools/docs/sphinx-build-wrapper
index 8080ace60680..b7c149dff06b 100755
--- a/tools/docs/sphinx-build-wrapper
+++ b/tools/docs/sphinx-build-wrapper
@@ -814,20 +814,42 @@ def jobs_type(value):
     except ValueError:
         raise argparse.ArgumentTypeError(f"Must be 'auto' or positive integer, got {value}")  # pylint: disable=W0707
 
+EPILOG="""
+Besides the command line arguments, several environment variables affect its
+default behavior, meant to be used when called via Kernel Makefile:
+
+- KERNELVERSION:  Kernel major version
+- KERNELRELEASE:  Kernel release
+- KBUILD_VERBOSE: Contains the value of "make V=[0|1] variable.
+                  When V=0 (KBUILD_VERBOSE=0), sets verbose level to "-q".
+- SPHINXBUILD:    Documentation build tool (default: "sphinx-build").
+- SPHINXOPTS:     Extra options pased to SPHINXBUILD
+                  (default: "-j auto" and "-q" if KBUILD_VERBOSE=0).
+                  The "-v" flag can be used to increase verbosity.
+                  If V=0, the first "-v" will drop "-q".
+- PYTHON3:        Python command to run SPHINXBUILD
+- PDFLATEX:       LaTeX PDF engine. (default: "xelatex")
+- LATEXOPTS:      Optional set of command line arguments to the LaTeX engine
+- srctree:        Location of the Kernel root directory (default: ".").
+
+"""
+
 def main():
     """
     Main function. The only mandatory argument is the target. If not
     specified, the other arguments will use default values if not
     specified at os.environ.
     """
-    parser = argparse.ArgumentParser(description="Kernel documentation builder")
+    parser = argparse.ArgumentParser(formatter_class=argparse.RawTextHelpFormatter,
+                                     description=__doc__,
+                                     epilog=EPILOG)
 
     parser.add_argument("target", choices=list(TARGETS.keys()),
                         help="Documentation target to build")
     parser.add_argument("--sphinxdirs", nargs="+",
                         help="Specific directories to build")
     parser.add_argument("--builddir", default="output",
-                        help="Sphinx configuration file")
+                        help="Sphinx configuration file (default: %(default)s)")
 
     parser.add_argument("--theme", help="Sphinx theme to use")
 
@@ -843,7 +865,7 @@ def main():
                         help="place build in verbose mode")
 
     parser.add_argument('-j', '--jobs', type=jobs_type,
-                        help="Sets number of jobs to use with sphinx-build")
+                        help="Sets number of jobs to use with sphinx-build(default: auto)")
 
     parser.add_argument('-i', '--interactive', action='store_true',
                         help="Change latex default to run in interactive mode")
-- 
2.52.0

Re: [PATCH 0/3] Some kernel-doc fixes

[Jonathan Corbet]

Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:

> Hi Jon,
>
> This small series contain 3 patches:
> - patch 1 fixes PDF docs build, as reported by Akira;
>   (I'm resending this one as-is from its v2)
> - patch 2 addresses a complain from Jani about not being able
>   of disabling "-q" flag when building docs with V=0;
> - patch 3 addresses an issue indirectly reported by Jani that
>   it the env vars that affects the wrapper aren't documented.
>
> With regards to patch 2, docs build honours V=0 by adding a
> "-q" flag.
>
> When V=1 is set, there are two effects in place:
>
> 1. sphix-build will be called without "-q";
> 2. Sphinx extensions will increase their verbosity levels.
>
> Sometimes, it is desired to just remove "-q" without increasing
> extensions verbosity. That's what patch 2 does.
>
> IMO, at least patch 1 should be merged during Kernel v6.21
> development cycle.

6.21?  I'm kind of assuming you mean the *next* cycle, which will be
7.0?

Thanks,

jon

Re: [PATCH 0/3] Some kernel-doc fixes

[Jonathan Corbet]

Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:

> Hi Jon,
>
> This small series contain 3 patches:
> - patch 1 fixes PDF docs build, as reported by Akira;
>   (I'm resending this one as-is from its v2)
> - patch 2 addresses a complain from Jani about not being able
>   of disabling "-q" flag when building docs with V=0;
> - patch 3 addresses an issue indirectly reported by Jani that
>   it the env vars that affects the wrapper aren't documented.
>
> With regards to patch 2, docs build honours V=0 by adding a
> "-q" flag.
>
> When V=1 is set, there are two effects in place:
>
> 1. sphix-build will be called without "-q";
> 2. Sphinx extensions will increase their verbosity levels.
>
> Sometimes, it is desired to just remove "-q" without increasing
> extensions verbosity. That's what patch 2 does.
>
> IMO, at least patch 1 should be merged during Kernel v6.21
> development cycle.
>
> Mauro Carvalho Chehab (3):
>   docs: kdoc: Fix pdfdocs build for tools
>   docs: sphinx-build-wrapper: allow -v override -q
>   tools: sphinx-build-wrapper: improve its help message
>
>  tools/docs/sphinx-build-wrapper  | 42 +++++++++++++++++++++++++++-----
>  tools/lib/python/kdoc/kdoc_re.py | 10 +++++---
>  2 files changed, 43 insertions(+), 9 deletions(-)

I've applied this set, thanks.

jon

Re: [PATCH 0/3] Some kernel-doc fixes

[Mauro Carvalho Chehab]

Hi Jon,

On Mon, 02 Feb 2026 10:01:14 -0700
Jonathan Corbet <corbet@lwn.net> wrote:

> Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
> 
> > Hi Jon,
> >
> > This small series contain 3 patches:
> > - patch 1 fixes PDF docs build, as reported by Akira;
> >   (I'm resending this one as-is from its v2)
> > - patch 2 addresses a complain from Jani about not being able
> >   of disabling "-q" flag when building docs with V=0;
> > - patch 3 addresses an issue indirectly reported by Jani that
> >   it the env vars that affects the wrapper aren't documented.
> >
> > With regards to patch 2, docs build honours V=0 by adding a
> > "-q" flag.
> >
> > When V=1 is set, there are two effects in place:
> >
> > 1. sphix-build will be called without "-q";
> > 2. Sphinx extensions will increase their verbosity levels.
> >
> > Sometimes, it is desired to just remove "-q" without increasing
> > extensions verbosity. That's what patch 2 does.
> >
> > IMO, at least patch 1 should be merged during Kernel v6.21
> > development cycle.
> >
> > Mauro Carvalho Chehab (3):
> >   docs: kdoc: Fix pdfdocs build for tools
> >   docs: sphinx-build-wrapper: allow -v override -q
> >   tools: sphinx-build-wrapper: improve its help message
> >
> >  tools/docs/sphinx-build-wrapper  | 42 +++++++++++++++++++++++++++-----
> >  tools/lib/python/kdoc/kdoc_re.py | 10 +++++---
> >  2 files changed, 43 insertions(+), 9 deletions(-)  
> 
> I've applied this set, thanks.

Thanks!

-

Jon,

I ended placing this series near the end of the /41 one mainly because
the first patch conflicts there.

The conflict is trivial, to solve though, if you opt to merge
from it.

IMO, it is worth merging this series before the merge window,
as it will solve some troubles that will appear after Linus
picks stuff from -next, related to some annotations Peter
Zijlstra added on his tree, but it is up to you.

Yeah, it is a lot more complex than I would expect for a late
-rc, so we could end just postpone it.

---

As a heads up, I'm working on a separate set of patches that, if 
things go well, we may end having a regression test for kernel-doc.
When done, I'll be submitting in separate.

The idea is to have a YAML file with source code, KdocItem,
man output and rst output, and a dynamic unit test to run
them. I finished today to write a skeleton, but still requires
polishing (*).

I'm thinking on modifying kernel-doc executable to be able
to generate the content for such YAML file.

If things go well, we could have something like:

	$ tools/docs/kernel-doc --gen-yaml all_kdoc_tests.yaml .

To place all tests there. As we modify kernel-doc, we can
run the code I'm currently writing. It would identify the
differences for both ReST and man and report.
	
(*) I started working on it today.

The unittest generation part from YAML already works:

	$ tools/unittests/test_kdoc_from_yaml.py 
	Ran 3 tests in 0.004s

	FAILED (failures=1)
	test_kdoc_from_yaml:
	    KernelDocParser:
	        test_gen_function3:  FAIL
	    KernelManOutput:
	        test_man_function3:  OK
	    KernelRestOutput:
	        test_rst_function3:  OK

Those 3 classes were auto-populated from yaml. Right now,
only KernelDocParser test exectution was implemented.

Those three classes contain just the test functions (two are just
stubs):

	class KernelDocParser(KdocParser):
	    def run_parser_test(self, source, kdoc_item, exports, fname):
	        if isinstance(kdoc_item, dict):
	            kdoc_item = [kdoc_item]

        	if isinstance(exports, str):
        	    exports=set([exports])
        	elif isinstance(exports, list):
        	    exports=set(exports)

		# For now, it is using here a function I wrote on a non-dynamic unit test
	        self.run_test(source, kdoc_item, exports=exports, fname=fname)

	class KernelManOutput(unittest.TestCase):
	    # TODO: for now, this is just a stub
	    def run_out_test(self, kdoc_item, out_type, data):
	        pass

	class KernelRestOutput(unittest.TestCase):
	    # TODO: for now, this is just a stub
	    def run_out_test(self, kdoc_item, out_type, data):
	        pass

And It contains a separate class to add dynamic tests on those.

The only issue I need to handle on its current implementation for
KernelDocParser (with the simple YAML test) is to solve whitespace
issues:

	FAIL: test_gen_function3 (test_kdoc_from_yaml.KernelDocParser.test_gen_function3)
	Lambda-like function to run tests with provided vars
	----------------------------------------------------------------------
	Traceback (most recent call last):
	  File "/new_devel/docs/tools/unittests/test_kdoc_from_yaml.py", line 79, in test_method
	    self.run_parser_test(source, kdoc_item, exports, fname)
	    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	  File "/new_devel/docs/tools/unittests/test_kdoc_from_yaml.py", line 54, in run_parser_test
	    self.run_test(source, kdoc_item, exports=exports, fname=fname)
	    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	  File "/new_devel/docs/tools/unittests/test_kdoc_parser.py", line 84, in run_test
	    self.assertEqual(d[key], value, msg=f"at {key}")
	    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	AssertionError: {'Description': 'Does nothing\n\n', 'Return': '\nalways return 0.\n'} != {'Description': 'Does nothing\n', 'Return': 'always return 0.\n'}
	- {'Description': 'Does nothing\n\n', 'Return': '\nalways return 0.\n'}
	?                                --              --
	
	+ {'Description': 'Does nothing\n', 'Return': 'always return 0.\n'} : at sections

I'll probably add a regex to replace all whitespaces/new lines by a
single one.

For man pages, I need to handle timestamps, but except for that and
whitespace differences, both rst and man page functions should be
straight forward: just call the output class passing the list of
KdocItem objects.

I guess tomorrow I may have this fixed and the test functions for
rst and man implemented.

Thanks,
Mauro

---

If you're curious enough, this is the test YAML file it is using to
generate the 3 dynamic unit tests:

tests:
  - name: function3
    fname: function3.c
    description: Just a simple example

    source: |
      /**
      * function3: Exported function
      * @arg1: @arg1 does nothing
      *
      * Does nothing
      *
      * return:
      *    always return 0.
      */
      int function3(char *arg1) { return 0; };
      EXPORT_SYMBOL(function3);

    exports: function3
    expected:
      - kdoc_item:
          name: function3
          type: function
          declaration_start_line: 1

          sections:
            Description: |
              Does nothing

            Return: |
              always return 0.

          parameterdescs:
            arg1: |
              @arg1 does nothing
          parameterlist:
            - arg1
          parameterdesc_start_lines:
            arg1: 2
          parametertypes:
            arg1: char *arg1

          other_stuff:
            func_macro: false
            functiontype: int
            purpose: Exported function
            typedef: false

        rst: |
          .. c:function:: int function3 (char *arg1)

            Exported function

          .. container:: kernelindent

            **Parameters**

            ``char *arg1``
              **arg1** does nothing

            **Description**

            Does nothing

            **Return**

            always return 0.

        man: |
          .TH "function3" 9 "February 2026"  "" "Kernel API Manual"
          .SH NAME
          function3 \- Exported function
          .SH SYNOPSIS
          .B "int" function3
          .BI "(char *arg1 "  ");"
          .SH ARGUMENTS
          .IP "arg1" 12
          \fIarg1\fP does nothing
          .SH "DESCRIPTION"
          Does nothing
          .SH "RETURN"
          always return 0.
          .SH "SEE ALSO"
          .PP
          Kernel file \fBtest.c\fR

Re: [PATCH 0/3] Some kernel-doc fixes

[Jani Nikula]

On Mon, 02 Feb 2026, Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote:
> As a heads up, I'm working on a separate set of patches that, if 
> things go well, we may end having a regression test for kernel-doc.
> When done, I'll be submitting in separate.
>
> The idea is to have a YAML file with source code, KdocItem,
> man output and rst output, and a dynamic unit test to run
> them. I finished today to write a skeleton, but still requires
> polishing (*).

...

> If you're curious enough, this is the test YAML file it is using to
> generate the 3 dynamic unit tests:

FWIW, I think it'll be painful to have the source and the expected
result in the same YAML file, simply because all parts of this are fussy
about whitespace and indentation. I'd put them all in separate files,
with the YAML tying them together. Then you can also reuse a single
source file with multiple tests with different parameters and different
outputs. And you get editor syntax higlighting and other help for the
individual files. And you can easily debug and compare the outputs with
direct kernel-doc invocation. Etc.

In fact, this is exactly what I've done with Hawkmoth tests
[1][2]. There's years of experience poured into this. I test everything
through the parser directly, through the command-line, and through
Sphinx.


BR,
Jani.


[1] https://github.com/jnikula/hawkmoth/blob/master/doc/developer/testing.rst
[2] https://github.com/jnikula/hawkmoth/tree/master/test


-- 
Jani Nikula, Intel

Re: [PATCH 0/3] Some kernel-doc fixes

[Mauro Carvalho Chehab]

On Tue, 03 Feb 2026 12:52:22 +0200
Jani Nikula <jani.nikula@linux.intel.com> wrote:

> On Mon, 02 Feb 2026, Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote:
> > As a heads up, I'm working on a separate set of patches that, if 
> > things go well, we may end having a regression test for kernel-doc.
> > When done, I'll be submitting in separate.
> >
> > The idea is to have a YAML file with source code, KdocItem,
> > man output and rst output, and a dynamic unit test to run
> > them. I finished today to write a skeleton, but still requires
> > polishing (*).  
> 
> ...
> 
> > If you're curious enough, this is the test YAML file it is using to
> > generate the 3 dynamic unit tests:  
> 
> FWIW, I think it'll be painful to have the source and the expected
> result in the same YAML file, simply because all parts of this are fussy
> about whitespace and indentation. I'd put them all in separate files,
> with the YAML tying them together. Then you can also reuse a single
> source file with multiple tests with different parameters and different
> outputs. And you get editor syntax higlighting and other help for the
> individual files.

I considered that, but on the other hand, if one wants to test
the intermediate internal KdocItem representation, having them
altogether helps to see what happened.

> And you can easily debug and compare the outputs with
> direct kernel-doc invocation. Etc.

True, but as I added support at kernel-doc to generate a yaml
format, it should be easy to diff from what someone wrote and
what kernel-doc actually produced - heh - sort of...

... pyyaml output is not user-friendly. Maybe there are ways
to make it use a more compact/nicer notation, but digging
into its documentation is not easy:

	https://pyyaml.org/wiki/PyYAMLDocumentation

and it is not complete.

> In fact, this is exactly what I've done with Hawkmoth tests
> [1][2]. There's years of experience poured into this. I test everything
> through the parser directly, through the command-line, and through
> Sphinx.

I'll take a look on it. Thanks for the hints!

> BR,
> Jani.
> 
> 
> [1] https://github.com/jnikula/hawkmoth/blob/master/doc/developer/testing.rst
> [2] https://github.com/jnikula/hawkmoth/tree/master/test
> 
> 



Thanks,
Mauro

Re: [PATCH 0/3] Some kernel-doc fixes

[Jani Nikula]

On Tue, 03 Feb 2026, Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote:
> ... pyyaml output is not user-friendly. Maybe there are ways
> to make it use a more compact/nicer notation, but digging
> into its documentation is not easy:
>
> 	https://pyyaml.org/wiki/PyYAMLDocumentation
>
> and it is not complete.

Side note, I really like using StrictYAML [1] with a schema for parsing
YAML. It's maybe slower, and probably won't help with the output either,
but the YAML type safety and validation are such nice features. (See the
Norway Problem.)

BR,
Jani.


[1] https://hitchdev.com/strictyaml/

-- 
Jani Nikula, Intel