From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
To: Linux Doc Mailing List <linux-doc@vger.kernel.org>,
Jonathan Corbet <corbet@lwn.net>
Cc: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>,
Donald Hunter <donald.hunter@gmail.com>,
Jakub Kicinski <mchehab+huawei@kernel.org>,
Jan Stancek <jstancek@redhat.com>,
linux-kernel@vger.kernel.org
Subject: [PATCH 06/24] tools: docs: parse-headers.py: move it from sphinx dir
Date: Thu, 21 Aug 2025 16:21:12 +0200 [thread overview]
Message-ID: <bef6a26663a9c7f7e515f36078432ee7a2a4330e.1755784930.git.mchehab+huawei@kernel.org> (raw)
In-Reply-To: <cover.1755784929.git.mchehab+huawei@kernel.org>
As suggested by Jon, we should start having a tools/docs
directory, instead of placing everything under scripts.
In the specific case of parse-headers.py, the previous
location is where we're placing Sphinx extensions, which is
not the right place for execs.
Move it to tools/docs/parse-headers.py.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
.pylintrc | 2 +-
tools/docs/lib/__init__.py | 0
tools/docs/lib/enrich_formatter.py | 70 ++++++++++++++
.../docs/lib/parse_data_structs.py | 95 ++-----------------
| 57 +++++++++++
5 files changed, 135 insertions(+), 89 deletions(-)
create mode 100644 tools/docs/lib/__init__.py
create mode 100644 tools/docs/lib/enrich_formatter.py
rename Documentation/sphinx/parse-headers.py => tools/docs/lib/parse_data_structs.py (80%)
create mode 100755 tools/docs/parse-headers.py
diff --git a/.pylintrc b/.pylintrc
index f1d21379254b..ad2476751f80 100644
--- a/.pylintrc
+++ b/.pylintrc
@@ -1,2 +1,2 @@
[MASTER]
-init-hook='import sys; sys.path += ["scripts/lib", "scripts/lib/kdoc", "scripts/lib/abi"]'
+init-hook='import sys; sys.path += ["scripts/lib", "scripts/lib/kdoc", "scripts/lib/abi", "tools/docs/lib"]'
diff --git a/tools/docs/lib/__init__.py b/tools/docs/lib/__init__.py
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/tools/docs/lib/enrich_formatter.py b/tools/docs/lib/enrich_formatter.py
new file mode 100644
index 000000000000..bb171567a4ca
--- /dev/null
+++ b/tools/docs/lib/enrich_formatter.py
@@ -0,0 +1,70 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 by Mauro Carvalho Chehab <mchehab@kernel.org>.
+
+"""
+Ancillary argparse HelpFormatter class that works on a similar way as
+argparse.RawDescriptionHelpFormatter, e.g. description maintains line
+breaks, but it also implement transformations to the help text. The
+actual transformations ar given by enrich_text(), if the output is tty.
+
+Currently, the follow transformations are done:
+
+ - Positional arguments are shown in upper cases;
+ - if output is TTY, ``var`` and positional arguments are shown prepended
+ by an ANSI SGR code. This is usually translated to bold. On some
+ terminals, like, konsole, this is translated into a colored bold text.
+"""
+
+import argparse
+import re
+import sys
+
+class EnrichFormatter(argparse.HelpFormatter):
+ """
+ Better format the output, making easier to identify the positional args
+ and how they're used at the __doc__ description.
+ """
+ def __init__(self, *args, **kwargs):
+ """Initialize class and check if is TTY"""
+ super().__init__(*args, **kwargs)
+ self._tty = sys.stdout.isatty()
+
+ def enrich_text(self, text):
+ """Handle ReST markups (currently, only ``foo``)"""
+ if self._tty and text:
+ # Replace ``text`` with ANSI SGR (bold)
+ return re.sub(r'\`\`(.+?)\`\`',
+ lambda m: f'\033[1m{m.group(1)}\033[0m', text)
+ return text
+
+ def _fill_text(self, text, width, indent):
+ """Enrich descriptions with markups on it"""
+ enriched = self.enrich_text(text)
+ return "\n".join(indent + line for line in enriched.splitlines())
+
+ def _format_usage(self, usage, actions, groups, prefix):
+ """Enrich positional arguments at usage: line"""
+
+ prog = self._prog
+ parts = []
+
+ for action in actions:
+ if action.option_strings:
+ opt = action.option_strings[0]
+ if action.nargs != 0:
+ opt += f" {action.dest.upper()}"
+ parts.append(f"[{opt}]")
+ else:
+ # Positional argument
+ parts.append(self.enrich_text(f"``{action.dest.upper()}``"))
+
+ usage_text = f"{prefix or 'usage: '} {prog} {' '.join(parts)}\n"
+ return usage_text
+
+ def _format_action_invocation(self, action):
+ """Enrich argument names"""
+ if not action.option_strings:
+ return self.enrich_text(f"``{action.dest.upper()}``")
+
+ return ", ".join(action.option_strings)
diff --git a/Documentation/sphinx/parse-headers.py b/tools/docs/lib/parse_data_structs.py
similarity index 80%
rename from Documentation/sphinx/parse-headers.py
rename to tools/docs/lib/parse_data_structs.py
index 344090ef259c..2b7fa6bd8321 100755
--- a/Documentation/sphinx/parse-headers.py
+++ b/tools/docs/lib/parse_data_structs.py
@@ -1,36 +1,32 @@
#!/usr/bin/env python3
# SPDX-License-Identifier: GPL-2.0
-# Copyright (c) 2016 by Mauro Carvalho Chehab <mchehab@kernel.org>.
-# pylint: disable=C0103,R0902,R0912,R0914,R0915
+# Copyright (c) 2016-2025 by Mauro Carvalho Chehab <mchehab@kernel.org>.
+# pylint: disable=R0912,R0915
"""
-Convert a C header or source file ``FILE_IN``, into a ReStructured Text
-included via ..parsed-literal block with cross-references for the
-documentation files that describe the API. It accepts an optional
-``FILE_RULES`` file to describes what elements will be either ignored or
-be pointed to a non-default reference type/name.
+Parse a source file or header, creating ReStructured Text cross references.
-The output is written at ``FILE_OUT``.
+It accepts an optional file to change the default symbol reference or to
+suppress symbols from the output.
It is capable of identifying defines, functions, structs, typedefs,
enums and enum symbols and create cross-references for all of them.
It is also capable of distinguish #define used for specifying a Linux
ioctl.
-The optional ``FILE_RULES`` contains a set of rules like:
+The optional rules file contains a set of rules like:
ignore ioctl VIDIOC_ENUM_FMT
replace ioctl VIDIOC_DQBUF vidioc_qbuf
replace define V4L2_EVENT_MD_FL_HAVE_FRAME_SEQ :c:type:`v4l2_event_motion_det`
"""
-import argparse
import os
import re
import sys
-class ParseHeader:
+class ParseDataStructs:
"""
Creates an enriched version of a Kernel header file with cross-links
to each C data structure type.
@@ -400,80 +396,3 @@ class ParseHeader:
f.write("=" * len(title))
f.write("\n\n.. parsed-literal::\n\n")
f.write(text)
-
-class EnrichFormatter(argparse.HelpFormatter):
- """
- Better format the output, making easier to identify the positional args
- and how they're used at the __doc__ description.
- """
- def __init__(self, *args, **kwargs):
- """Initialize class and check if is TTY"""
- super().__init__(*args, **kwargs)
- self._tty = sys.stdout.isatty()
-
- def enrich_text(self, text):
- """Handle ReST markups (currently, only ``foo``)"""
- if self._tty and text:
- # Replace ``text`` with ANSI bold
- return re.sub(r'\`\`(.+?)\`\`',
- lambda m: f'\033[1m{m.group(1)}\033[0m', text)
- return text
-
- def _fill_text(self, text, width, indent):
- """Enrich descriptions with markups on it"""
- enriched = self.enrich_text(text)
- return "\n".join(indent + line for line in enriched.splitlines())
-
- def _format_usage(self, usage, actions, groups, prefix):
- """Enrich positional arguments at usage: line"""
-
- prog = self._prog
- parts = []
-
- for action in actions:
- if action.option_strings:
- opt = action.option_strings[0]
- if action.nargs != 0:
- opt += f" {action.dest.upper()}"
- parts.append(f"[{opt}]")
- else:
- # Positional argument
- parts.append(self.enrich_text(f"``{action.dest.upper()}``"))
-
- usage_text = f"{prefix or 'usage: '} {prog} {' '.join(parts)}\n"
- return usage_text
-
- def _format_action_invocation(self, action):
- """Enrich argument names"""
- if not action.option_strings:
- return self.enrich_text(f"``{action.dest.upper()}``")
- else:
- return ", ".join(action.option_strings)
-
-
-def main():
- """Main function"""
- parser = argparse.ArgumentParser(description=__doc__,
- formatter_class=EnrichFormatter)
-
- parser.add_argument("-d", "--debug", action="count", default=0,
- help="Increase debug level. Can be used multiple times")
- parser.add_argument("file_in", help="Input C file")
- parser.add_argument("file_out", help="Output RST file")
- parser.add_argument("file_rules", nargs="?",
- help="Exceptions file (optional)")
-
- args = parser.parse_args()
-
- parser = ParseHeader(debug=args.debug)
- parser.parse_file(args.file_in)
-
- if args.file_rules:
- parser.process_exceptions(args.file_rules)
-
- parser.debug_print()
- parser.write_output(args.file_in, args.file_out)
-
-
-if __name__ == "__main__":
- main()
--git a/tools/docs/parse-headers.py b/tools/docs/parse-headers.py
new file mode 100755
index 000000000000..07d3b47c4834
--- /dev/null
+++ b/tools/docs/parse-headers.py
@@ -0,0 +1,57 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2016, 2025 by Mauro Carvalho Chehab <mchehab@kernel.org>.
+# pylint: disable=C0103
+
+"""
+Convert a C header or source file ``FILE_IN``, into a ReStructured Text
+included via ..parsed-literal block with cross-references for the
+documentation files that describe the API. It accepts an optional
+``FILE_RULES`` file to describes what elements will be either ignored or
+be pointed to a non-default reference type/name.
+
+The output is written at ``FILE_OUT``.
+
+It is capable of identifying defines, functions, structs, typedefs,
+enums and enum symbols and create cross-references for all of them.
+It is also capable of distinguish #define used for specifying a Linux
+ioctl.
+
+The optional ``FILE_RULES`` contains a set of rules like:
+
+ ignore ioctl VIDIOC_ENUM_FMT
+ replace ioctl VIDIOC_DQBUF vidioc_qbuf
+ replace define V4L2_EVENT_MD_FL_HAVE_FRAME_SEQ :c:type:`v4l2_event_motion_det`
+"""
+
+import argparse
+
+from lib.parse_data_structs import ParseDataStructs
+from lib.enrich_formatter import EnrichFormatter
+
+def main():
+ """Main function"""
+ parser = argparse.ArgumentParser(description=__doc__,
+ formatter_class=EnrichFormatter)
+
+ parser.add_argument("-d", "--debug", action="count", default=0,
+ help="Increase debug level. Can be used multiple times")
+ parser.add_argument("file_in", help="Input C file")
+ parser.add_argument("file_out", help="Output RST file")
+ parser.add_argument("file_rules", nargs="?",
+ help="Exceptions file (optional)")
+
+ args = parser.parse_args()
+
+ parser = ParseDataStructs(debug=args.debug)
+ parser.parse_file(args.file_in)
+
+ if args.file_rules:
+ parser.process_exceptions(args.file_rules)
+
+ parser.debug_print()
+ parser.write_output(args.file_in, args.file_out)
+
+
+if __name__ == "__main__":
+ main()
--
2.50.1
next prev parent reply other threads:[~2025-08-21 14:21 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-08-21 14:21 [PATCH 00/24] better handle media headers Mauro Carvalho Chehab
2025-08-21 14:21 ` [PATCH 01/24] docs: parse-headers.pl: improve its debug output format Mauro Carvalho Chehab
2025-08-21 14:21 ` [PATCH 02/24] docs: parse-headers.py: convert parse-headers.pl Mauro Carvalho Chehab
2025-08-21 14:21 ` [PATCH 03/24] docs: parse-headers.py: improve --help logic Mauro Carvalho Chehab
2025-08-21 14:21 ` [PATCH 04/24] docs: parse-headers.py: better handle @var arguments Mauro Carvalho Chehab
2025-08-21 14:21 ` [PATCH 05/24] docs: parse-headers.py: simplify the rules for hashes Mauro Carvalho Chehab
2025-08-21 14:21 ` Mauro Carvalho Chehab [this message]
2025-08-21 14:21 ` [PATCH 07/24] tools: docs: parse_data_structs.py: add methods to return output Mauro Carvalho Chehab
2025-08-21 14:21 ` [PATCH 08/24] MAINTAINERS: add files from tools/docs to documentation entry Mauro Carvalho Chehab
2025-08-21 14:21 ` [PATCH 09/24] docs: uapi: media: Makefile: use parse-headers.py Mauro Carvalho Chehab
2025-08-21 14:21 ` [PATCH 10/24] docs: kernel_include.py: Update its coding style Mauro Carvalho Chehab
2025-08-21 14:21 ` [PATCH 11/24] docs: kernel_include.py: allow cross-reference generation Mauro Carvalho Chehab
2025-08-21 14:21 ` [PATCH 12/24] docs: kernel_include.py: generate warnings for broken refs Mauro Carvalho Chehab
2025-08-21 14:21 ` [PATCH 13/24] docs: kernel_include.py: move rawtext logic to separate functions Mauro Carvalho Chehab
2025-08-21 14:21 ` [PATCH 14/24] docs: kernel_include.py: move range logic to a separate function Mauro Carvalho Chehab
2025-08-21 14:21 ` [PATCH 15/24] docs: kernel_include.py: remove range restriction for gen docs Mauro Carvalho Chehab
2025-08-21 14:21 ` [PATCH 16/24] docs: kernel_include.py: move code and literal functions Mauro Carvalho Chehab
2025-08-21 14:21 ` [PATCH 17/24] docs: kernel_include.py: add support to generate a TOC table Mauro Carvalho Chehab
2025-08-21 14:21 ` [PATCH 18/24] docs: kernel_include.py: append line numbers to better report errors Mauro Carvalho Chehab
2025-08-21 14:21 ` [PATCH 19/24] docs: kernel_include.py: move apply_range() and add a docstring Mauro Carvalho Chehab
2025-08-21 14:21 ` [PATCH 20/24] docs: kernel_include.py: remove line numbers from parsed-literal Mauro Carvalho Chehab
2025-08-21 14:21 ` [PATCH 21/24] docs: kernel_include.py: remove Include class inheritance Mauro Carvalho Chehab
2025-08-21 14:21 ` [PATCH 22/24] docs: kernel_include.py: document all supported parameters Mauro Carvalho Chehab
2025-08-21 14:21 ` [PATCH 23/24] scripts: sphinx-build-wrapper: get rid of uapi/media Makefile Mauro Carvalho Chehab
2025-08-21 14:21 ` [PATCH 24/24] docs: sphinx: drop parse-headers.pl Mauro Carvalho Chehab
2025-08-25 7:48 ` [PATCH 00/24] better handle media headers Jani Nikula
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=bef6a26663a9c7f7e515f36078432ee7a2a4330e.1755784930.git.mchehab+huawei@kernel.org \
--to=mchehab+huawei@kernel.org \
--cc=corbet@lwn.net \
--cc=donald.hunter@gmail.com \
--cc=jstancek@redhat.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).