Re: [PATCH 00/13] Add kernel-doc modules to Documentation/tools

[Mauro Carvalho Chehab <mchehab+huawei@kernel.org>]

Em Wed, 14 Jan 2026 13:46:31 -0700
Jonathan Corbet <corbet@lwn.net> escreveu:

> Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
> 
> >> We could certainly rename it to something different.
> >> But I really dislike having language extensions on files meant to be
> >> executed as commands; you shouldn't care what language it's written in
> >> when you run it.  
> >
> > I don't like it either, but Python is really picky on some things.
> >
> > The problem here is that this is a Python policy violation. To change
> > that, one needs to write a PEP and convince Python maintainers to merge
> > it, together with changes on python "import" directive.  
> 
> ...or just ignore it. 

Indeed this is an option.

> There is a reason that "pip" is called "pip"
> rather than "pip.py" - the Python folks don't keep those extensions on
> commands either.

True, but see what pip has:

	$ more /usr/bin/pip
	#! /usr/bin/python3 -P
	# -*- coding: utf-8 -*-
	import re
	import sys
	from pip._internal.cli.main import main
	if __name__ == '__main__':
	    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
	    sys.exit(main())
	/usr/bin/pip (END)

Everything, including main are outside it. Btw, this code is almost
identical to sphinx-build:

	$ more /usr/bin/sphinx-build
	#! /usr/bin/python3 -P
	# -*- coding: utf-8 -*-
	import re
	import sys
	from sphinx.cmd.build import main
	if __name__ == '__main__':
	    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
	    sys.exit(main())

This would be equivalent of having a tools/docs/kernel-doc like this
(untested):

	#!/usr/bin/env python3
	from .kdoc.main import main
	if __name__ == '__main__':
	    sys.exit(main())

where the actual argparse code would be inside tools/docs/kdoc/main.py

> > Alternatively, assuming that some magic words would be enough to
> > convince importlib to load a name without ".py" and with "-", it could be
> > easier to convince Sphinx autodoc maintainers to take a patch, as they're 
> > probably using importlib somewhere to dynamically import a file based 
> > at the string inside "automodule" directive. On a quick grep,
> > this seems to be the case, and such logic is inside:
> >
> > 	sphinx/ext/autodoc/importer.py  
> 
> No doubt we could do that.  But is it really worth the trouble?  There
> is not much in kernel-doc that needs documenting, especially since you
> did the work to move the actual functionality into separate modules. 

I'm not particularly concerned about kernel-doc here. I'm more
concerned on defining how things like that are expected to be
documented.

Yet, if you add this:

	.. automodule:: docs.kernel_doc
	   :members:
	   :show-inheritance:
	   :undoc-members:

The generated documentation sounds somewhat relevant to me - specially
if placed together with the kernel-doc module API documentation:

	kernel-doc module documentation
	===============================

	kernel_doc

	Print formatted kernel documentation to stdout

	Read C language source or header FILEs, extract embedded documentation comments,
	and print formatted documentation to standard output.

	The documentation comments are identified by the /** opening comment mark.

	See Documentation/doc-guide/kernel-doc.rst for the documentation comment syntax.

	class docs.kernel_doc.MsgFormatter(fmt=None, datefmt=None, style='%', validate=True, *, defaults=None)

	    Bases: Formatter

	    Helper class to format warnings in a similar way to kernel-doc.pl.

	    format(record)

	        Format the specified record as text.

	        The record’s attribute dictionary is used as the operand to a 
		string formatting operation which yields the returned string. 
		Before formatting the dictionary, a couple of preparatory
		steps are carried out. The message attribute of the record
		is computed using LogRecord.getMessage(). If the formatting
		string uses the time (as determined by a call to usesTime(),
		formatTime() is called to format the event time. If there is 
		exception information, it is formatted using formatException()
		and appended to the message.

	docs.kernel_doc.main()

	    Main program By default, the return value is:

	        0: success or Python version is not compatible with
		   kernel-doc. If -Werror is not used, it will also
        	   return 0 if there are issues at kernel-doc markups;

	        1: an abnormal condition happened;

	        2: argparse issued an error;

	        3: -Werror is used, and one or more unfiltered parse warnings
	           happened.

> > So, even if we don't actually add kernel-doc docstrings and
> > functions via autodoc, I think it is still worth having a
> > name convention that would allow that.  
> 
> Instead, I think you're trying to take a functionality meant to describe
> APIs and use it to document command-line stuff.  I'm happy to live by
> the import rules for stuff that is actually imported; I think it makes
> less sense to let them drive the naming of files that are outside of
> their intended scope.

Yeah, `MsgFormatter` doesn't belong to doc-guide. main return values
does, however. IMO it is important to keep it documented together with
the code.

It should be noticed that autodoc has support for selecting symbols.
So:

	.. automodule:: docs.kernel_doc
	   :members: main

would pick only module description + main

and if we add, instead:

	.. autofunction:: docs.kernel_doc.main

it would just pick docstrings for main, e.g. it would place just
this:

	docs.kernel_doc.main()

	    Main program By default, the return value is:

	        0: success or Python version is not compatible with
		   kernel-doc. If -Werror is not used, it will also
        	   return 0 if there are issues at kernel-doc markups;

	        1: an abnormal condition happened;

	        2: argparse issued an error;

	        3: -Werror is used, and one or more unfiltered parse warnings
	           happened.

---

In summary, all I'm saying is that, if we stick to PEP8 names, we
can opt to import a documentation directly from the script instead
of writing it twice: at the code and on a rst file.

Btw, if you want to test it, you need just one patch to enable
it:

	https://lore.kernel.org/linux-doc/6aa5a5b4a686f07c8f3e6cb04fe4c07ed9c1d071.1768396023.git.mchehab+huawei@kernel.org/T/#u

this basically allows using tools/ and scripts/ as the base for
documentation.

You may also want this css patch, as default format with alabaster
is very ugly:

	https://lore.kernel.org/linux-doc/8d66988f05d06d10938e062ed4465bf303c51441.1768396023.git.mchehab+huawei@kernel.org/T/#u
	
with that, you can experiment inserting autodoc stuff using:	

	.. automodule:: docs.name
	   :modules:

to document all public methods from "docs/name.py" file, or:

	.. autofunction:: docs.name.function

and/or:

	.. autoclass:: docs.name.class

for a single "function" (or "class") inside "docs/name.py".

Again, the limitation is that "name" ends with ".py" and only
have (lower case?) letters, numbers and underscores - e.g. it
shall be something that "import" and "from ... import" supports.


Thanks,
Mauro