* [PATCH v8 19/24] docs: add support to build manpages from kerneldoc output [not found] <cover.1758196090.git.mchehab+huawei@kernel.org> @ 2025-09-18 11:54 ` Mauro Carvalho Chehab 2025-09-27 20:35 ` Nicolas Schier 0 siblings, 1 reply; 3+ messages in thread From: Mauro Carvalho Chehab @ 2025-09-18 11:54 UTC (permalink / raw) To: Linux Doc Mailing List, Jonathan Corbet Cc: Mauro Carvalho Chehab, Akira Yokosawa, Mauro Carvalho Chehab, Thomas Weißschuh, Alice Ryhl, Masahiro Yamada, Miguel Ojeda, Nathan Chancellor, Nicolas Schier, Randy Dunlap, Tamir Duberstein, linux-kbuild, linux-kernel Generating man files currently requires running a separate script. The target also doesn't appear at the docs Makefile. Add support for mandocs at the Makefile, adding the build logic inside sphinx-build-wrapper, updating documentation and dropping the ancillary script. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> --- Documentation/Makefile | 3 +- Documentation/doc-guide/kernel-doc.rst | 29 ++++----- Makefile | 2 +- scripts/split-man.pl | 28 --------- tools/docs/sphinx-build-wrapper | 81 ++++++++++++++++++++++++-- 5 files changed, 95 insertions(+), 48 deletions(-) delete mode 100755 scripts/split-man.pl diff --git a/Documentation/Makefile b/Documentation/Makefile index 05397b9d844d..aa42b2cb7030 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -53,7 +53,7 @@ ifeq ($(HAVE_SPHINX),0) else # HAVE_SPHINX # Common documentation targets -infodocs texinfodocs latexdocs epubdocs xmldocs pdfdocs linkcheckdocs: +mandocs infodocs texinfodocs latexdocs epubdocs xmldocs pdfdocs linkcheckdocs: $(Q)@$(srctree)/tools/docs/sphinx-pre-install --version-check +$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ \ --sphinxdirs="$(SPHINXDIRS)" --conf="$(SPHINX_CONF)" \ @@ -104,6 +104,7 @@ dochelp: @echo ' htmldocs - HTML' @echo ' texinfodocs - Texinfo' @echo ' infodocs - Info' + @echo ' mandocs - Man pages' @echo ' latexdocs - LaTeX' @echo ' pdfdocs - PDF' @echo ' epubdocs - EPUB' diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst index af9697e60165..4370cc8fbcf5 100644 --- a/Documentation/doc-guide/kernel-doc.rst +++ b/Documentation/doc-guide/kernel-doc.rst @@ -579,20 +579,23 @@ source. How to use kernel-doc to generate man pages ------------------------------------------- -If you just want to use kernel-doc to generate man pages you can do this -from the kernel git tree:: +To generate man pages for all files that contain kernel-doc markups, run:: - $ scripts/kernel-doc -man \ - $(git grep -l '/\*\*' -- :^Documentation :^tools) \ - | scripts/split-man.pl /tmp/man + $ make mandocs -Some older versions of git do not support some of the variants of syntax for -path exclusion. One of the following commands may work for those versions:: +Or calling ``script-build-wrapper`` directly:: - $ scripts/kernel-doc -man \ - $(git grep -l '/\*\*' -- . ':!Documentation' ':!tools') \ - | scripts/split-man.pl /tmp/man + $ ./tools/docs/sphinx-build-wrapper mandocs - $ scripts/kernel-doc -man \ - $(git grep -l '/\*\*' -- . ":(exclude)Documentation" ":(exclude)tools") \ - | scripts/split-man.pl /tmp/man +The output will be at ``/man`` directory inside the output directory +(by default: ``Documentation/output``). + +Optionally, it is possible to generate a partial set of man pages by +using SPHINXDIRS: + + $ make SPHINXDIRS=driver-api/media mandocs + +.. note:: + + When SPHINXDIRS={subdir} is used, it will only generate man pages for + the files explicitly inside a ``Documentation/{subdir}/.../*.rst`` file. diff --git a/Makefile b/Makefile index 6bfe776bf3c5..9bd44afeda26 100644 --- a/Makefile +++ b/Makefile @@ -1800,7 +1800,7 @@ $(help-board-dirs): help-%: # Documentation targets # --------------------------------------------------------------------------- DOC_TARGETS := xmldocs latexdocs pdfdocs htmldocs epubdocs cleandocs \ - linkcheckdocs dochelp refcheckdocs texinfodocs infodocs + linkcheckdocs dochelp refcheckdocs texinfodocs infodocs mandocs PHONY += $(DOC_TARGETS) $(DOC_TARGETS): $(Q)$(MAKE) $(build)=Documentation $@ diff --git a/scripts/split-man.pl b/scripts/split-man.pl deleted file mode 100755 index 96bd99dc977a..000000000000 --- a/scripts/split-man.pl +++ /dev/null @@ -1,28 +0,0 @@ -#!/usr/bin/env perl -# SPDX-License-Identifier: GPL-2.0 -# -# Author: Mauro Carvalho Chehab <mchehab+samsung@kernel.org> -# -# Produce manpages from kernel-doc. -# See Documentation/doc-guide/kernel-doc.rst for instructions - -if ($#ARGV < 0) { - die "where do I put the results?\n"; -} - -mkdir $ARGV[0],0777; -$state = 0; -while (<STDIN>) { - if (/^\.TH \"[^\"]*\" 9 \"([^\"]*)\"/) { - if ($state == 1) { close OUT } - $state = 1; - $fn = "$ARGV[0]/$1.9"; - print STDERR "Creating $fn\n"; - open OUT, ">$fn" or die "can't open $fn: $!\n"; - print OUT $_; - } elsif ($state != 0) { - print OUT $_; - } -} - -close OUT; diff --git a/tools/docs/sphinx-build-wrapper b/tools/docs/sphinx-build-wrapper index 8d1f77c4a880..7a6eb41837e6 100755 --- a/tools/docs/sphinx-build-wrapper +++ b/tools/docs/sphinx-build-wrapper @@ -47,12 +47,14 @@ the newer version. import argparse import locale import os +import re import shlex import shutil import subprocess import sys from concurrent import futures +from glob import glob from lib.python_version import PythonVersion from lib.latex_fonts import LatexFontChecker @@ -77,6 +79,7 @@ TARGETS = { "epubdocs": { "builder": "epub", "out_dir": "epub" }, "texinfodocs": { "builder": "texinfo", "out_dir": "texinfo" }, "infodocs": { "builder": "texinfo", "out_dir": "texinfo" }, + "mandocs": { "builder": "man", "out_dir": "man" }, "latexdocs": { "builder": "latex", "out_dir": "latex" }, "pdfdocs": { "builder": "latex", "out_dir": "latex" }, "xmldocs": { "builder": "xml", "out_dir": "xml" }, @@ -503,6 +506,71 @@ class SphinxBuilder: except subprocess.CalledProcessError as e: sys.exit(f"Error generating info docs: {e}") + def handle_man(self, kerneldoc, docs_dir, src_dir, output_dir): + """ + Create man pages from kernel-doc output + """ + + re_kernel_doc = re.compile(r"^\.\.\s+kernel-doc::\s*(\S+)") + re_man = re.compile(r'^\.TH "[^"]*" (\d+) "([^"]*)"') + + if docs_dir == src_dir: + # + # Pick the entire set of kernel-doc markups from the entire tree + # + kdoc_files = set([self.srctree]) + else: + kdoc_files = set() + + for fname in glob(os.path.join(src_dir, "**"), recursive=True): + if os.path.isfile(fname) and fname.endswith(".rst"): + with open(fname, "r", encoding="utf-8") as in_fp: + data = in_fp.read() + + for line in data.split("\n"): + match = re_kernel_doc.match(line) + if match: + if os.path.isfile(match.group(1)): + kdoc_files.add(match.group(1)) + + if not kdoc_files: + sys.exit(f"Directory {src_dir} doesn't contain kernel-doc tags") + + cmd = [ kerneldoc, "-m" ] + sorted(kdoc_files) + try: + if self.verbose: + print(" ".join(cmd)) + + result = subprocess.run(cmd, stdout=subprocess.PIPE, text= True) + + if result.returncode: + print(f"Warning: kernel-doc returned {result.returncode} warnings") + + except (OSError, ValueError, subprocess.SubprocessError) as e: + sys.exit(f"Failed to create man pages for {src_dir}: {repr(e)}") + + fp = None + try: + for line in result.stdout.split("\n"): + match = re_man.match(line) + if not match: + if fp: + fp.write(line + '\n') + continue + + if fp: + fp.close() + + fname = f"{output_dir}/{match.group(2)}.{match.group(1)}" + + if self.verbose: + print(f"Creating {fname}") + fp = open(fname, "w", encoding="utf-8") + fp.write(line + '\n') + finally: + if fp: + fp.close() + def cleandocs(self, builder): # pylint: disable=W0613 """Remove documentation output directory""" shutil.rmtree(self.builddir, ignore_errors=True) @@ -531,7 +599,7 @@ class SphinxBuilder: # Other targets require sphinx-build, so check if it exists # sphinxbuild = shutil.which(self.sphinxbuild, path=self.env["PATH"]) - if not sphinxbuild: + if not sphinxbuild and target != "mandocs": sys.exit(f"Error: {self.sphinxbuild} not found in PATH.\n") if builder == "latex": @@ -619,10 +687,13 @@ class SphinxBuilder: output_dir, ] - try: - self.run_sphinx(sphinxbuild, build_args, env=self.env) - except (OSError, ValueError, subprocess.SubprocessError) as e: - sys.exit(f"Build failed: {repr(e)}") + if target == "mandocs": + self.handle_man(kerneldoc, docs_dir, src_dir, output_dir) + else: + try: + self.run_sphinx(sphinxbuild, build_args, env=self.env) + except (OSError, ValueError, subprocess.SubprocessError) as e: + sys.exit(f"Build failed: {repr(e)}") # # Ensure that each html/epub output will have needed static files -- 2.51.0 ^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH v8 19/24] docs: add support to build manpages from kerneldoc output 2025-09-18 11:54 ` [PATCH v8 19/24] docs: add support to build manpages from kerneldoc output Mauro Carvalho Chehab @ 2025-09-27 20:35 ` Nicolas Schier 2025-09-29 0:44 ` Mauro Carvalho Chehab 0 siblings, 1 reply; 3+ messages in thread From: Nicolas Schier @ 2025-09-27 20:35 UTC (permalink / raw) To: Mauro Carvalho Chehab Cc: Linux Doc Mailing List, Jonathan Corbet, Akira Yokosawa, Thomas Weißschuh, Alice Ryhl, Randy Dunlap, Tamir Duberstein, linux-kbuild, linux-kernel Please note that @kernel.org receipients are still not addressed correctly: > Date: Thu, 18 Sep 2025 13:54:53 +0200 > From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> > To: Linux Doc Mailing List <linux-doc@vger.kernel.org>, Jonathan Corbet <corbet@lwn.net> > Cc: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>, Akira Yokosawa <akiyks@gmail.com>, > Mauro Carvalho Chehab <mchehab+huawei@kernel.org>, Thomas Weißschuh <thomas.weissschuh@linutronix.de>, > Alice Ryhl <aliceryhl@google.com>, Masahiro Yamada <mchehab+huawei@kernel.org>, > Miguel Ojeda <mchehab+huawei@kernel.org>, Nathan Chancellor <mchehab+huawei@kernel.org>, > Nicolas Schier <nicolas.schier@linux.dev>, Randy Dunlap <rdunlap@infradead.org>, > Tamir Duberstein <tamird@gmail.com>, linux-kbuild@vger.kernel.org, > linux-kernel@vger.kernel.org On Thu, Sep 18, 2025 at 01:54:53PM +0200, Mauro Carvalho Chehab wrote: > Generating man files currently requires running a separate > script. The target also doesn't appear at the docs Makefile. > > Add support for mandocs at the Makefile, adding the build > logic inside sphinx-build-wrapper, updating documentation > and dropping the ancillary script. > > Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> > --- > Documentation/Makefile | 3 +- > Documentation/doc-guide/kernel-doc.rst | 29 ++++----- > Makefile | 2 +- > scripts/split-man.pl | 28 --------- > tools/docs/sphinx-build-wrapper | 81 ++++++++++++++++++++++++-- > 5 files changed, 95 insertions(+), 48 deletions(-) > delete mode 100755 scripts/split-man.pl > > diff --git a/Documentation/Makefile b/Documentation/Makefile > index 05397b9d844d..aa42b2cb7030 100644 > --- a/Documentation/Makefile > +++ b/Documentation/Makefile > @@ -53,7 +53,7 @@ ifeq ($(HAVE_SPHINX),0) > else # HAVE_SPHINX > > # Common documentation targets > -infodocs texinfodocs latexdocs epubdocs xmldocs pdfdocs linkcheckdocs: > +mandocs infodocs texinfodocs latexdocs epubdocs xmldocs pdfdocs linkcheckdocs: > $(Q)@$(srctree)/tools/docs/sphinx-pre-install --version-check > +$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ \ > --sphinxdirs="$(SPHINXDIRS)" --conf="$(SPHINX_CONF)" \ > @@ -104,6 +104,7 @@ dochelp: > @echo ' htmldocs - HTML' > @echo ' texinfodocs - Texinfo' > @echo ' infodocs - Info' > + @echo ' mandocs - Man pages' > @echo ' latexdocs - LaTeX' > @echo ' pdfdocs - PDF' > @echo ' epubdocs - EPUB' > diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst > index af9697e60165..4370cc8fbcf5 100644 > --- a/Documentation/doc-guide/kernel-doc.rst > +++ b/Documentation/doc-guide/kernel-doc.rst > @@ -579,20 +579,23 @@ source. > How to use kernel-doc to generate man pages > ------------------------------------------- > > -If you just want to use kernel-doc to generate man pages you can do this > -from the kernel git tree:: > +To generate man pages for all files that contain kernel-doc markups, run:: > > - $ scripts/kernel-doc -man \ > - $(git grep -l '/\*\*' -- :^Documentation :^tools) \ > - | scripts/split-man.pl /tmp/man > + $ make mandocs > > -Some older versions of git do not support some of the variants of syntax for > -path exclusion. One of the following commands may work for those versions:: > +Or calling ``script-build-wrapper`` directly:: For this very patch, there is no 'scripts-build-wrapper' but 'sphinx-build-wrapper'. Or am I missing something? > > - $ scripts/kernel-doc -man \ > - $(git grep -l '/\*\*' -- . ':!Documentation' ':!tools') \ > - | scripts/split-man.pl /tmp/man > + $ ./tools/docs/sphinx-build-wrapper mandocs > > - $ scripts/kernel-doc -man \ > - $(git grep -l '/\*\*' -- . ":(exclude)Documentation" ":(exclude)tools") \ > - | scripts/split-man.pl /tmp/man > +The output will be at ``/man`` directory inside the output directory > +(by default: ``Documentation/output``). > + > +Optionally, it is possible to generate a partial set of man pages by > +using SPHINXDIRS: > + > + $ make SPHINXDIRS=driver-api/media mandocs > + > +.. note:: > + > + When SPHINXDIRS={subdir} is used, it will only generate man pages for > + the files explicitly inside a ``Documentation/{subdir}/.../*.rst`` file. > diff --git a/Makefile b/Makefile > index 6bfe776bf3c5..9bd44afeda26 100644 > --- a/Makefile > +++ b/Makefile > @@ -1800,7 +1800,7 @@ $(help-board-dirs): help-%: > # Documentation targets > # --------------------------------------------------------------------------- > DOC_TARGETS := xmldocs latexdocs pdfdocs htmldocs epubdocs cleandocs \ > - linkcheckdocs dochelp refcheckdocs texinfodocs infodocs > + linkcheckdocs dochelp refcheckdocs texinfodocs infodocs mandocs > PHONY += $(DOC_TARGETS) > $(DOC_TARGETS): > $(Q)$(MAKE) $(build)=Documentation $@ Acked-by: Nicolas Schier <nsc@kernel.org> ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH v8 19/24] docs: add support to build manpages from kerneldoc output 2025-09-27 20:35 ` Nicolas Schier @ 2025-09-29 0:44 ` Mauro Carvalho Chehab 0 siblings, 0 replies; 3+ messages in thread From: Mauro Carvalho Chehab @ 2025-09-29 0:44 UTC (permalink / raw) To: Nicolas Schier Cc: Linux Doc Mailing List, Jonathan Corbet, Akira Yokosawa, Thomas Weißschuh, Alice Ryhl, Randy Dunlap, Tamir Duberstein, linux-kbuild, linux-kernel Em Sat, 27 Sep 2025 22:35:00 +0200 Nicolas Schier <nsc@kernel.org> escreveu: > Please note that @kernel.org receipients are still not addressed > correctly: > > > Date: Thu, 18 Sep 2025 13:54:53 +0200 > > From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> > > To: Linux Doc Mailing List <linux-doc@vger.kernel.org>, Jonathan Corbet <corbet@lwn.net> > > Cc: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>, Akira Yokosawa <akiyks@gmail.com>, > > Mauro Carvalho Chehab <mchehab+huawei@kernel.org>, Thomas Weißschuh <thomas.weissschuh@linutronix.de>, > > Alice Ryhl <aliceryhl@google.com>, Masahiro Yamada <mchehab+huawei@kernel.org>, > > Miguel Ojeda <mchehab+huawei@kernel.org>, Nathan Chancellor <mchehab+huawei@kernel.org>, > > Nicolas Schier <nicolas.schier@linux.dev>, Randy Dunlap <rdunlap@infradead.org>, > > Tamir Duberstein <tamird@gmail.com>, linux-kbuild@vger.kernel.org, > > linux-kernel@vger.kernel.org Just Found the issue: the problem were on a rewrite rule on my emailer which I added ages ago. Dropped it. It should be ok next time. > On Thu, Sep 18, 2025 at 01:54:53PM +0200, Mauro Carvalho Chehab wrote: > > Generating man files currently requires running a separate > > script. The target also doesn't appear at the docs Makefile. > > > > Add support for mandocs at the Makefile, adding the build > > logic inside sphinx-build-wrapper, updating documentation > > and dropping the ancillary script. > > > > Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> > > --- > > Documentation/Makefile | 3 +- > > Documentation/doc-guide/kernel-doc.rst | 29 ++++----- > > Makefile | 2 +- > > scripts/split-man.pl | 28 --------- > > tools/docs/sphinx-build-wrapper | 81 ++++++++++++++++++++++++-- > > 5 files changed, 95 insertions(+), 48 deletions(-) > > delete mode 100755 scripts/split-man.pl > > > > diff --git a/Documentation/Makefile b/Documentation/Makefile > > index 05397b9d844d..aa42b2cb7030 100644 > > --- a/Documentation/Makefile > > +++ b/Documentation/Makefile > > @@ -53,7 +53,7 @@ ifeq ($(HAVE_SPHINX),0) > > else # HAVE_SPHINX > > > > # Common documentation targets > > -infodocs texinfodocs latexdocs epubdocs xmldocs pdfdocs linkcheckdocs: > > +mandocs infodocs texinfodocs latexdocs epubdocs xmldocs pdfdocs linkcheckdocs: > > $(Q)@$(srctree)/tools/docs/sphinx-pre-install --version-check > > +$(Q)$(PYTHON3) $(BUILD_WRAPPER) $@ \ > > --sphinxdirs="$(SPHINXDIRS)" --conf="$(SPHINX_CONF)" \ > > @@ -104,6 +104,7 @@ dochelp: > > @echo ' htmldocs - HTML' > > @echo ' texinfodocs - Texinfo' > > @echo ' infodocs - Info' > > + @echo ' mandocs - Man pages' > > @echo ' latexdocs - LaTeX' > > @echo ' pdfdocs - PDF' > > @echo ' epubdocs - EPUB' > > diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst > > index af9697e60165..4370cc8fbcf5 100644 > > --- a/Documentation/doc-guide/kernel-doc.rst > > +++ b/Documentation/doc-guide/kernel-doc.rst > > @@ -579,20 +579,23 @@ source. > > How to use kernel-doc to generate man pages > > ------------------------------------------- > > > > -If you just want to use kernel-doc to generate man pages you can do this > > -from the kernel git tree:: > > +To generate man pages for all files that contain kernel-doc markups, run:: > > > > - $ scripts/kernel-doc -man \ > > - $(git grep -l '/\*\*' -- :^Documentation :^tools) \ > > - | scripts/split-man.pl /tmp/man > > + $ make mandocs > > > > -Some older versions of git do not support some of the variants of syntax for > > -path exclusion. One of the following commands may work for those versions:: > > +Or calling ``script-build-wrapper`` directly:: > > For this very patch, there is no 'scripts-build-wrapper' but > 'sphinx-build-wrapper'. Or am I missing something? Sorry! yes, sphinx-build-wrapper is the right name. > > > > > - $ scripts/kernel-doc -man \ > > - $(git grep -l '/\*\*' -- . ':!Documentation' ':!tools') \ > > - | scripts/split-man.pl /tmp/man > > + $ ./tools/docs/sphinx-build-wrapper mandocs > > > > - $ scripts/kernel-doc -man \ > > - $(git grep -l '/\*\*' -- . ":(exclude)Documentation" ":(exclude)tools") \ > > - | scripts/split-man.pl /tmp/man > > +The output will be at ``/man`` directory inside the output directory > > +(by default: ``Documentation/output``). > > + > > +Optionally, it is possible to generate a partial set of man pages by > > +using SPHINXDIRS: > > + > > + $ make SPHINXDIRS=driver-api/media mandocs > > + > > +.. note:: > > + > > + When SPHINXDIRS={subdir} is used, it will only generate man pages for > > + the files explicitly inside a ``Documentation/{subdir}/.../*.rst`` file. > > diff --git a/Makefile b/Makefile > > index 6bfe776bf3c5..9bd44afeda26 100644 > > --- a/Makefile > > +++ b/Makefile > > @@ -1800,7 +1800,7 @@ $(help-board-dirs): help-%: > > # Documentation targets > > # --------------------------------------------------------------------------- > > DOC_TARGETS := xmldocs latexdocs pdfdocs htmldocs epubdocs cleandocs \ > > - linkcheckdocs dochelp refcheckdocs texinfodocs infodocs > > + linkcheckdocs dochelp refcheckdocs texinfodocs infodocs mandocs > > PHONY += $(DOC_TARGETS) > > $(DOC_TARGETS): > > $(Q)$(MAKE) $(build)=Documentation $@ > > Acked-by: Nicolas Schier <nsc@kernel.org> Thanks! Mauro ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2025-09-29 0:44 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <cover.1758196090.git.mchehab+huawei@kernel.org>
2025-09-18 11:54 ` [PATCH v8 19/24] docs: add support to build manpages from kerneldoc output Mauro Carvalho Chehab
2025-09-27 20:35 ` Nicolas Schier
2025-09-29 0:44 ` Mauro Carvalho Chehab
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox