* [PATCH v3 0/2] add a make target to generate man pages from kernel-doc
@ 2025-08-27 8:26 Mauro Carvalho Chehab
2025-08-27 8:26 ` [PATCH v3 1/2] docs: add support to build manpages from kerneldoc output Mauro Carvalho Chehab
0 siblings, 1 reply; 4+ messages in thread
From: Mauro Carvalho Chehab @ 2025-08-27 8:26 UTC (permalink / raw)
To: Linux Doc Mailing List, Jonathan Corbet
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
linux-kbuild
Hi Jon,
Currently, generation of manpages is hacky: one needs to run
kernel-doc over c source files and then use an extra script to
split results.
The new kernel-doc tool supports multiple files and even dirs,
so there's no need to use git ls-files anymore.
Yet, it produces a single output. Change the logic to add, instead
a target to docs Makefile to produce them, moving the split
and build logic to sphinx-build-wrapper.
That allows honoring SPHINXDIRS when they point to a subdir,
while scanning all files by default.
This series comes after:
https://lore.kernel.org/linux-doc/cover.1756138805.git.mchehab+huawei@kernel.org/T/#t
-
It should be noticed that, while Sphinx does have a man pages
builder, it requires a completely different set of ReST entries
that would need to be designed specially for that:
- for each man page, an entry at conf.py is required;
- the ReST page should be in a format closer to a real man page;
- exactly one .rst file for each man page.
This is completely different than the approach we take where
we have, where kernel-doc direcly generates groff output
formated data.
So, yeah, this is still a little bit hackish, but to avoid
that we would need a completely different setup with, frankly,
would be a headache to setup and maintain.
With that in mind, there are a couple of possible alternatives:
1. change kernel-doc to produce one file per man page.
A change like that would require adding file directory
parameter for output, and/or teaching it about
Documentation/output and O=;
2. make sphinx-build-wrapper do that for us. That's IMO
the best alternative, which is what I actually implemented;
3. add a man pages Sphinx extension that would be tricking
sphinx, making it produce one man page .rst file for each
kernel-doc markup entry.
This is probably doable, and could eventually bring some
advantages, as we could also have man pages for pages
that are only in ReST format, but it will require some extra
development, and likely require several changes at the
existing .rst files. Also, generating extra output files out of
the blue is something that Sphinx doesn't like to do, as it
likes to have a 1:1 mapping between source files and output
ones.
Not sure if it worth the efforts, and will still be hackish.
I opted for (2), as it means that ones needing to generate
docs can benefit from SPHINXDIRS, and we don't need anything
extra at docs Makefile. By doing that, we can later improve the
man pages logic without needing to touch anything else but
kernel-doc and/or the wrapper tool.
There are two points to consider:
- when SPHINXDIRS=. (the default), it searches fror all files
treewide, even the ones without kernel-doc entries at docs,
which is aligned with the procedure used by the current
documentation about how to generate man pages;
- when SHINXDIRS=subdirs, it searches only for the files that
come just after kernel-doc markups. It doesn't take into
account any kernel-doc markup parameters to filter out
internal/external functions or to point to an extra header
file.
---
v3:
- fixed a badly-solved conflict on v2;
- improved documentation about manpages generation;
- added SEE ALSO at the man pages.
v2:
- no actual changes, but it was rebased on the top of the actual
pacth series, in order to apply cleanly.
*** BLURB HERE ***
Mauro Carvalho Chehab (2):
docs: add support to build manpages from kerneldoc output
tools: kernel-doc: add a see also section at man pages
Documentation/Makefile | 3 +-
Documentation/doc-guide/kernel-doc.rst | 29 +++++----
Makefile | 2 +-
scripts/lib/kdoc/kdoc_files.py | 5 +-
scripts/lib/kdoc/kdoc_output.py | 84 ++++++++++++++++++++++++--
scripts/split-man.pl | 28 ---------
tools/docs/sphinx-build-wrapper | 81 +++++++++++++++++++++++--
7 files changed, 178 insertions(+), 54 deletions(-)
delete mode 100755 scripts/split-man.pl
--
2.51.0
^ permalink raw reply [flat|nested] 4+ messages in thread* [PATCH v3 1/2] docs: add support to build manpages from kerneldoc output 2025-08-27 8:26 [PATCH v3 0/2] add a make target to generate man pages from kernel-doc Mauro Carvalho Chehab @ 2025-08-27 8:26 ` Mauro Carvalho Chehab 2025-08-27 22:04 ` Nicolas Schier 0 siblings, 1 reply; 4+ messages in thread From: Mauro Carvalho Chehab @ 2025-08-27 8:26 UTC (permalink / raw) To: Linux Doc Mailing List, Jonathan Corbet Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, Thomas Weißschuh, Alice Ryhl, Masahiro Yamada, Miguel Ojeda, Nathan Chancellor, Nicolas Schier, Randy Dunlap, Tamir Duberstein, linux-kbuild, linux-kernel Generating man files currently requires running a separate script. The target also doesn't appear at the docs Makefile. Add support for mandocs at the Makefile, adding the build logic inside sphinx-build-wrapper, updating documentation and dropping the ancillary script. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> --- Documentation/Makefile | 3 +- Documentation/doc-guide/kernel-doc.rst | 29 ++++----- Makefile | 2 +- scripts/split-man.pl | 28 --------- tools/docs/sphinx-build-wrapper | 81 ++++++++++++++++++++++++-- 5 files changed, 95 insertions(+), 48 deletions(-) delete mode 100755 scripts/split-man.pl diff --git a/Documentation/Makefile b/Documentation/Makefile index 3e1cb44a5fbb..22e39e5ed07d 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -53,7 +53,7 @@ ifeq ($(HAVE_SPHINX),0) else # HAVE_SPHINX # Common documentation targets -infodocs texinfodocs latexdocs epubdocs xmldocs pdfdocs linkcheckdocs: +mandocs infodocs texinfodocs latexdocs epubdocs xmldocs pdfdocs linkcheckdocs: $(Q)@$(srctree)/tools/docs/sphinx-pre-install --version-check +$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ \ --sphinxdirs="$(SPHINXDIRS)" --conf=$(SPHINX_CONF) \ @@ -104,6 +104,7 @@ dochelp: @echo ' htmldocs - HTML' @echo ' texinfodocs - Texinfo' @echo ' infodocs - Info' + @echo ' mandocs - Man pages' @echo ' latexdocs - LaTeX' @echo ' pdfdocs - PDF' @echo ' epubdocs - EPUB' diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst index af9697e60165..4370cc8fbcf5 100644 --- a/Documentation/doc-guide/kernel-doc.rst +++ b/Documentation/doc-guide/kernel-doc.rst @@ -579,20 +579,23 @@ source. How to use kernel-doc to generate man pages ------------------------------------------- -If you just want to use kernel-doc to generate man pages you can do this -from the kernel git tree:: +To generate man pages for all files that contain kernel-doc markups, run:: - $ scripts/kernel-doc -man \ - $(git grep -l '/\*\*' -- :^Documentation :^tools) \ - | scripts/split-man.pl /tmp/man + $ make mandocs -Some older versions of git do not support some of the variants of syntax for -path exclusion. One of the following commands may work for those versions:: +Or calling ``script-build-wrapper`` directly:: - $ scripts/kernel-doc -man \ - $(git grep -l '/\*\*' -- . ':!Documentation' ':!tools') \ - | scripts/split-man.pl /tmp/man + $ ./tools/docs/sphinx-build-wrapper mandocs - $ scripts/kernel-doc -man \ - $(git grep -l '/\*\*' -- . ":(exclude)Documentation" ":(exclude)tools") \ - | scripts/split-man.pl /tmp/man +The output will be at ``/man`` directory inside the output directory +(by default: ``Documentation/output``). + +Optionally, it is possible to generate a partial set of man pages by +using SPHINXDIRS: + + $ make SPHINXDIRS=driver-api/media mandocs + +.. note:: + + When SPHINXDIRS={subdir} is used, it will only generate man pages for + the files explicitly inside a ``Documentation/{subdir}/.../*.rst`` file. diff --git a/Makefile b/Makefile index 6bfe776bf3c5..9bd44afeda26 100644 --- a/Makefile +++ b/Makefile @@ -1800,7 +1800,7 @@ $(help-board-dirs): help-%: # Documentation targets # --------------------------------------------------------------------------- DOC_TARGETS := xmldocs latexdocs pdfdocs htmldocs epubdocs cleandocs \ - linkcheckdocs dochelp refcheckdocs texinfodocs infodocs + linkcheckdocs dochelp refcheckdocs texinfodocs infodocs mandocs PHONY += $(DOC_TARGETS) $(DOC_TARGETS): $(Q)$(MAKE) $(build)=Documentation $@ diff --git a/scripts/split-man.pl b/scripts/split-man.pl deleted file mode 100755 index 96bd99dc977a..000000000000 --- a/scripts/split-man.pl +++ /dev/null @@ -1,28 +0,0 @@ -#!/usr/bin/env perl -# SPDX-License-Identifier: GPL-2.0 -# -# Author: Mauro Carvalho Chehab <mchehab+samsung@kernel.org> -# -# Produce manpages from kernel-doc. -# See Documentation/doc-guide/kernel-doc.rst for instructions - -if ($#ARGV < 0) { - die "where do I put the results?\n"; -} - -mkdir $ARGV[0],0777; -$state = 0; -while (<STDIN>) { - if (/^\.TH \"[^\"]*\" 9 \"([^\"]*)\"/) { - if ($state == 1) { close OUT } - $state = 1; - $fn = "$ARGV[0]/$1.9"; - print STDERR "Creating $fn\n"; - open OUT, ">$fn" or die "can't open $fn: $!\n"; - print OUT $_; - } elsif ($state != 0) { - print OUT $_; - } -} - -close OUT; diff --git a/tools/docs/sphinx-build-wrapper b/tools/docs/sphinx-build-wrapper index c884022ad733..932b1b675274 100755 --- a/tools/docs/sphinx-build-wrapper +++ b/tools/docs/sphinx-build-wrapper @@ -47,6 +47,7 @@ the newer version. import argparse import locale import os +import re import shlex import shutil import subprocess @@ -55,6 +56,7 @@ import sys from concurrent import futures from lib.python_version import PythonVersion +from glob import glob LIB_DIR = "../../scripts/lib" SRC_DIR = os.path.dirname(os.path.realpath(__file__)) @@ -77,6 +79,7 @@ TARGETS = { "epubdocs": { "builder": "epub", "out_dir": "epub" }, "texinfodocs": { "builder": "texinfo", "out_dir": "texinfo" }, "infodocs": { "builder": "texinfo", "out_dir": "texinfo" }, + "mandocs": { "builder": "man", "out_dir": "man" }, "latexdocs": { "builder": "latex", "out_dir": "latex" }, "pdfdocs": { "builder": "latex", "out_dir": "latex" }, "xmldocs": { "builder": "xml", "out_dir": "xml" }, @@ -455,6 +458,71 @@ class SphinxBuilder: except subprocess.CalledProcessError as e: sys.exit(f"Error generating info docs: {e}") + def handle_man(self, kerneldoc, docs_dir, src_dir, output_dir): + """ + Create man pages from kernel-doc output + """ + + re_kernel_doc = re.compile(r"^\.\.\s+kernel-doc::\s*(\S+)") + re_man = re.compile(r'^\.TH "[^"]*" (\d+) "([^"]*)"') + + if docs_dir == src_dir: + # + # Pick the entire set of kernel-doc markups from the entire tree + # + kdoc_files = set([self.srctree]) + else: + kdoc_files = set() + + for fname in glob(os.path.join(src_dir, "**"), recursive=True): + if os.path.isfile(fname) and fname.endswith(".rst"): + with open(fname, "r", encoding="utf-8") as in_fp: + data = in_fp.read() + + for line in data.split("\n"): + match = re_kernel_doc.match(line) + if match: + if os.path.isfile(match.group(1)): + kdoc_files.add(match.group(1)) + + if not kdoc_files: + sys.exit(f"Directory {src_dir} doesn't contain kernel-doc tags") + + cmd = [ kerneldoc, "-m" ] + sorted(kdoc_files) + try: + if self.verbose: + print(" ".join(cmd)) + + result = subprocess.run(cmd, stdout=subprocess.PIPE, text= True) + + if result.returncode: + print(f"Warning: kernel-doc returned {result.returncode} warnings") + + except (OSError, ValueError, subprocess.SubprocessError) as e: + sys.exit(f"Failed to create man pages for {src_dir}: {repr(e)}") + + fp = None + try: + for line in result.stdout.split("\n"): + match = re_man.match(line) + if not match: + if fp: + fp.write(line + '\n') + continue + + if fp: + fp.close() + + fname = f"{output_dir}/{match.group(2)}.{match.group(1)}" + + if self.verbose: + print(f"Creating {fname}") + fp = open(fname, "w", encoding="utf-8") + fp.write(line + '\n') + finally: + if fp: + fp.close() + def cleandocs(self, builder): # pylint: disable=W0613 """Remove documentation output directory""" shutil.rmtree(self.builddir, ignore_errors=True) @@ -483,7 +551,7 @@ class SphinxBuilder: # Other targets require sphinx-build, so check if it exists # sphinxbuild = shutil.which(self.sphinxbuild, path=self.env["PATH"]) - if not sphinxbuild: + if not sphinxbuild and target != "mandocs": sys.exit(f"Error: {self.sphinxbuild} not found in PATH.\n") if builder == "latex": @@ -572,10 +640,13 @@ class SphinxBuilder: output_dir, ] - try: - self.run_sphinx(sphinxbuild, build_args, env=self.env) - except (OSError, ValueError, subprocess.SubprocessError) as e: - sys.exit(f"Build failed: {repr(e)}") + if target == "mandocs": + self.handle_man(kerneldoc, docs_dir, src_dir, output_dir) + else: + try: + self.run_sphinx(sphinxbuild, build_args, env=self.env) + except (OSError, ValueError, subprocess.SubprocessError) as e: + sys.exit(f"Build failed: {repr(e)}") # # Ensure that each html/epub output will have needed static files -- 2.51.0 ^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH v3 1/2] docs: add support to build manpages from kerneldoc output 2025-08-27 8:26 ` [PATCH v3 1/2] docs: add support to build manpages from kerneldoc output Mauro Carvalho Chehab @ 2025-08-27 22:04 ` Nicolas Schier 2025-08-28 21:07 ` Mauro Carvalho Chehab 0 siblings, 1 reply; 4+ messages in thread From: Nicolas Schier @ 2025-08-27 22:04 UTC (permalink / raw) To: Mauro Carvalho Chehab Cc: Linux Doc Mailing List, Jonathan Corbet, Thomas Weißschuh, Alice Ryhl, Randy Dunlap, Tamir Duberstein, linux-kbuild, linux-kernel Please check your mail tooling: something strange happened to all @kernel.org adresses: > Cc: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>, > Mauro Carvalho Chehab <mchehab+huawei@kernel.org>, > Thomas Weißschuh <thomas.weissschuh@linutronix.de>, > Alice Ryhl <aliceryhl@google.com>, > Masahiro Yamada <mchehab+huawei@kernel.org>, > Miguel Ojeda <mchehab+huawei@kernel.org>, > Nathan Chancellor <mchehab+huawei@kernel.org>, > Nicolas Schier <nicolas.schier@linux.dev>, > Randy Dunlap <rdunlap@infradead.org>, > Tamir Duberstein <tamird@gmail.com>, > linux-kbuild@vger.kernel.org, > linux-kernel@vger.kernel.org On Wed, Aug 27, 2025 at 10:26:37AM +0200, Mauro Carvalho Chehab wrote: > Generating man files currently requires running a separate > script. The target also doesn't appear at the docs Makefile. > > Add support for mandocs at the Makefile, adding the build > logic inside sphinx-build-wrapper, updating documentation > and dropping the ancillary script. > > Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> > --- > Documentation/Makefile | 3 +- > Documentation/doc-guide/kernel-doc.rst | 29 ++++----- > Makefile | 2 +- > scripts/split-man.pl | 28 --------- > tools/docs/sphinx-build-wrapper | 81 ++++++++++++++++++++++++-- > 5 files changed, 95 insertions(+), 48 deletions(-) > delete mode 100755 scripts/split-man.pl > > diff --git a/Documentation/Makefile b/Documentation/Makefile > index 3e1cb44a5fbb..22e39e5ed07d 100644 > --- a/Documentation/Makefile > +++ b/Documentation/Makefile > @@ -53,7 +53,7 @@ ifeq ($(HAVE_SPHINX),0) > else # HAVE_SPHINX > > # Common documentation targets > -infodocs texinfodocs latexdocs epubdocs xmldocs pdfdocs linkcheckdocs: > +mandocs infodocs texinfodocs latexdocs epubdocs xmldocs pdfdocs linkcheckdocs: > $(Q)@$(srctree)/tools/docs/sphinx-pre-install --version-check > +$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ \ > --sphinxdirs="$(SPHINXDIRS)" --conf=$(SPHINX_CONF) \ > @@ -104,6 +104,7 @@ dochelp: > @echo ' htmldocs - HTML' > @echo ' texinfodocs - Texinfo' > @echo ' infodocs - Info' > + @echo ' mandocs - Man pages' > @echo ' latexdocs - LaTeX' > @echo ' pdfdocs - PDF' > @echo ' epubdocs - EPUB' > diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst > index af9697e60165..4370cc8fbcf5 100644 > --- a/Documentation/doc-guide/kernel-doc.rst > +++ b/Documentation/doc-guide/kernel-doc.rst > @@ -579,20 +579,23 @@ source. > How to use kernel-doc to generate man pages > ------------------------------------------- > > -If you just want to use kernel-doc to generate man pages you can do this > -from the kernel git tree:: > +To generate man pages for all files that contain kernel-doc markups, run:: > > - $ scripts/kernel-doc -man \ > - $(git grep -l '/\*\*' -- :^Documentation :^tools) \ > - | scripts/split-man.pl /tmp/man > + $ make mandocs > > -Some older versions of git do not support some of the variants of syntax for > -path exclusion. One of the following commands may work for those versions:: > +Or calling ``script-build-wrapper`` directly:: ^^^^^^ sphinx-build-wrapper ? Kind regards Nicolas ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH v3 1/2] docs: add support to build manpages from kerneldoc output 2025-08-27 22:04 ` Nicolas Schier @ 2025-08-28 21:07 ` Mauro Carvalho Chehab 0 siblings, 0 replies; 4+ messages in thread From: Mauro Carvalho Chehab @ 2025-08-28 21:07 UTC (permalink / raw) To: Nicolas Schier Cc: Linux Doc Mailing List, Jonathan Corbet, Thomas Weißschuh, Alice Ryhl, Randy Dunlap, Tamir Duberstein, linux-kbuild, linux-kernel Em Thu, 28 Aug 2025 00:04:08 +0200 Nicolas Schier <nsc@kernel.org> escreveu: > Please check your mail tooling: something strange happened to all > @kernel.org adresses: > > > Cc: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>, > > Mauro Carvalho Chehab <mchehab+huawei@kernel.org>, > > Thomas Weißschuh <thomas.weissschuh@linutronix.de>, > > Alice Ryhl <aliceryhl@google.com>, > > Masahiro Yamada <mchehab+huawei@kernel.org>, > > Miguel Ojeda <mchehab+huawei@kernel.org>, > > Nathan Chancellor <mchehab+huawei@kernel.org>, > > Nicolas Schier <nicolas.schier@linux.dev>, > > Randy Dunlap <rdunlap@infradead.org>, > > Tamir Duberstein <tamird@gmail.com>, > > linux-kbuild@vger.kernel.org, > > linux-kernel@vger.kernel.org That's really weird! Thanks for letting me know. It was not supposed to do any e-mail changes. > > On Wed, Aug 27, 2025 at 10:26:37AM +0200, Mauro Carvalho Chehab wrote: > > Generating man files currently requires running a separate > > script. The target also doesn't appear at the docs Makefile. > > > > Add support for mandocs at the Makefile, adding the build > > logic inside sphinx-build-wrapper, updating documentation > > and dropping the ancillary script. > > > > Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> > > --- > > Documentation/Makefile | 3 +- > > Documentation/doc-guide/kernel-doc.rst | 29 ++++----- > > Makefile | 2 +- > > scripts/split-man.pl | 28 --------- > > tools/docs/sphinx-build-wrapper | 81 ++++++++++++++++++++++++-- > > 5 files changed, 95 insertions(+), 48 deletions(-) > > delete mode 100755 scripts/split-man.pl > > > > diff --git a/Documentation/Makefile b/Documentation/Makefile > > index 3e1cb44a5fbb..22e39e5ed07d 100644 > > --- a/Documentation/Makefile > > +++ b/Documentation/Makefile > > @@ -53,7 +53,7 @@ ifeq ($(HAVE_SPHINX),0) > > else # HAVE_SPHINX > > > > # Common documentation targets > > -infodocs texinfodocs latexdocs epubdocs xmldocs pdfdocs linkcheckdocs: > > +mandocs infodocs texinfodocs latexdocs epubdocs xmldocs pdfdocs linkcheckdocs: > > $(Q)@$(srctree)/tools/docs/sphinx-pre-install --version-check > > +$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ \ > > --sphinxdirs="$(SPHINXDIRS)" --conf=$(SPHINX_CONF) \ > > @@ -104,6 +104,7 @@ dochelp: > > @echo ' htmldocs - HTML' > > @echo ' texinfodocs - Texinfo' > > @echo ' infodocs - Info' > > + @echo ' mandocs - Man pages' > > @echo ' latexdocs - LaTeX' > > @echo ' pdfdocs - PDF' > > @echo ' epubdocs - EPUB' > > diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst > > index af9697e60165..4370cc8fbcf5 100644 > > --- a/Documentation/doc-guide/kernel-doc.rst > > +++ b/Documentation/doc-guide/kernel-doc.rst > > @@ -579,20 +579,23 @@ source. > > How to use kernel-doc to generate man pages > > ------------------------------------------- > > > > -If you just want to use kernel-doc to generate man pages you can do this > > -from the kernel git tree:: > > +To generate man pages for all files that contain kernel-doc markups, run:: > > > > - $ scripts/kernel-doc -man \ > > - $(git grep -l '/\*\*' -- :^Documentation :^tools) \ > > - | scripts/split-man.pl /tmp/man > > + $ make mandocs > > > > -Some older versions of git do not support some of the variants of syntax for > > -path exclusion. One of the following commands may work for those versions:: > > +Or calling ``script-build-wrapper`` directly:: > > ^^^^^^ sphinx-build-wrapper ? This is on a separate patch series: https://lore.kernel.org/linux-doc/cover.1756138805.git.mchehab+huawei@kernel.org/ It basically moves the complex logic around docs Makefile into a script that can either be called via Makefile or directly. Thanks, Mauro ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2025-08-28 21:07 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2025-08-27 8:26 [PATCH v3 0/2] add a make target to generate man pages from kernel-doc Mauro Carvalho Chehab 2025-08-27 8:26 ` [PATCH v3 1/2] docs: add support to build manpages from kerneldoc output Mauro Carvalho Chehab 2025-08-27 22:04 ` Nicolas Schier 2025-08-28 21:07 ` Mauro Carvalho Chehab
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).