* [PATCH 00/27] Improve ABI documentation generation
@ 2025-02-10 10:17 Mauro Carvalho Chehab
2025-02-10 10:17 ` [PATCH 01/27] docs: sphinx: remove kernellog.py file Mauro Carvalho Chehab
` (28 more replies)
0 siblings, 29 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:17 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Jonathan Corbet, Guilherme G. Piccoli, Chao Yu, Jaegeuk Kim,
James Clark, Johannes Berg, Mike Leach, Suzuki K Poulose,
Tony Luck, bpf, coresight, linux-arm-kernel, linux-f2fs-devel,
linux-hardening, linux-wireless
Hi Jon/Greg,
This series replace get_abi.pl with a Python version.
I originally started it due to some issues I noticed when searching for
ABI symbols. While I could just go ahead and fix the already existing
script, I noticed that the script maintainance didn't have much care over
all those years, probably because it is easier to find Python programmers
those days.
Also, the code is complex and was not using modules or classes and
were using lots of global variables.
So, I decided to rewrite it in Python. I started with a manual conversion
for each function. Yet, to avoid future maintainership issues, I opted to
divide the main code on three classes, each on a sepaparate file.
Just like the original RFC, I opted to keep the Sphinx kernel-abi module
on three different phases:
- call get_abi.py as an exec file;
- import AbiParser on a minimal integration scenario;
- cleanup the code to avoid needing to parse line numbers from the text.
This way, if something goes wrong, it would be easier to just revert any
offending patches, It also provides a better rationale about what each
logical change is doing.
The initial patches on this series do some preparation work and
cleans some ABI symbol bugs that lack ":" delimiter.
I opted to place on this series the Sphinx and Python version updates.
I still have ~10 patches here with additional cleanups, from the original
series I sent as RFC but let's get the main changes merged first.
Mauro Carvalho Chehab (27):
docs: sphinx: remove kernellog.py file
docs: sphinx/kernel_abi: adjust coding style
docs: admin-guide: abi: add SPDX tags to ABI files
ABI: sysfs-class-rfkill: fix kernelversion tags
ABI: sysfs-bus-coresight-*: fix kernelversion tags
ABI: sysfs-driver-dma-idxd: fix date tags
ABI: sysfs-fs-f2fs: fix date tags
ABI: sysfs-power: fix a what tag
scripts/documentation-file-ref-check: don't check perl/python scripts
scripts/get_abi.py: add a Python tool to generate ReST output
scripts/get_abi.py: add support for symbol search
docs: use get_abi.py for ABI generation
scripts/lib/abi/abi_parser.py: optimize parse_abi() function
scripts/lib/abi/abi_parser.py: use an interactor for ReST output
docs: sphinx/kernel_abi: use AbiParser directly
docs: sphinx/kernel_abi: reduce buffer usage for ABI messages
docs: sphinx/kernel_abi: properly split lines
scripts/get_abi.pl: Add filtering capabilities to rest output
scripts/get_abi.pl: add support to parse ABI README file
docs: sphinx/kernel_abi: parse ABI files only once
docs: admin-guide/abi: split files from symbols
docs: sphinx/automarkup: add cross-references for ABI
docs: sphinx/kernel_abi: avoid warnings during Sphinx module init
scripts/lib/abi/abi_parser.py: Rename title name for ABI files
scripts/lib/abi/abi_parser.py: make it backward-compatible with Python
3.6
scripts/get_abi.py: add support for undefined ABIs
scripts/get_abi.pl: drop now obsoleted script
Documentation/ABI/removed/sysfs-class-rfkill | 2 +-
Documentation/ABI/stable/sysfs-class-rfkill | 12 +-
.../ABI/stable/sysfs-driver-dma-idxd | 4 +-
.../testing/sysfs-bus-coresight-devices-cti | 78 +-
.../testing/sysfs-bus-coresight-devices-tpdm | 52 +-
Documentation/ABI/testing/sysfs-fs-f2fs | 4 +-
Documentation/ABI/testing/sysfs-power | 2 +-
.../admin-guide/abi-obsolete-files.rst | 7 +
Documentation/admin-guide/abi-obsolete.rst | 6 +-
Documentation/admin-guide/abi-readme-file.rst | 6 +
.../admin-guide/abi-removed-files.rst | 7 +
Documentation/admin-guide/abi-removed.rst | 6 +-
.../admin-guide/abi-stable-files.rst | 7 +
Documentation/admin-guide/abi-stable.rst | 6 +-
.../admin-guide/abi-testing-files.rst | 7 +
Documentation/admin-guide/abi-testing.rst | 6 +-
Documentation/admin-guide/abi.rst | 17 +
Documentation/sphinx/automarkup.py | 47 +
Documentation/sphinx/kernel_abi.py | 162 ++-
Documentation/sphinx/kerneldoc.py | 14 +-
Documentation/sphinx/kernellog.py | 22 -
Documentation/sphinx/kfigure.py | 81 +-
scripts/documentation-file-ref-check | 2 +-
scripts/get_abi.pl | 1103 -----------------
scripts/get_abi.py | 214 ++++
scripts/lib/abi/abi_parser.py | 628 ++++++++++
scripts/lib/abi/abi_regex.py | 234 ++++
scripts/lib/abi/helpers.py | 38 +
scripts/lib/abi/system_symbols.py | 378 ++++++
29 files changed, 1830 insertions(+), 1322 deletions(-)
create mode 100644 Documentation/admin-guide/abi-obsolete-files.rst
create mode 100644 Documentation/admin-guide/abi-readme-file.rst
create mode 100644 Documentation/admin-guide/abi-removed-files.rst
create mode 100644 Documentation/admin-guide/abi-stable-files.rst
create mode 100644 Documentation/admin-guide/abi-testing-files.rst
delete mode 100644 Documentation/sphinx/kernellog.py
delete mode 100755 scripts/get_abi.pl
create mode 100755 scripts/get_abi.py
create mode 100644 scripts/lib/abi/abi_parser.py
create mode 100644 scripts/lib/abi/abi_regex.py
create mode 100644 scripts/lib/abi/helpers.py
create mode 100644 scripts/lib/abi/system_symbols.py
--
2.48.1
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH 01/27] docs: sphinx: remove kernellog.py file
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
@ 2025-02-10 10:17 ` Mauro Carvalho Chehab
2025-02-10 10:17 ` [PATCH 02/27] docs: sphinx/kernel_abi: adjust coding style Mauro Carvalho Chehab
` (27 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:17 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
Randy Dunlap, Vegard Nossum, linux-kernel
In the past, there was a need for a wrapper due to different
Sphinx versions support (before Sphinx 1.6). This is long gone,
and now it is just a wrapper. Get rig of it to simplify
the code.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/sphinx/kernel_abi.py | 5 +-
Documentation/sphinx/kerneldoc.py | 14 +++---
Documentation/sphinx/kernellog.py | 22 --------
Documentation/sphinx/kfigure.py | 81 ++++++++++++++++--------------
4 files changed, 53 insertions(+), 69 deletions(-)
delete mode 100644 Documentation/sphinx/kernellog.py
diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
index 5911bd0d7965..8401562cb5d9 100644
--- a/Documentation/sphinx/kernel_abi.py
+++ b/Documentation/sphinx/kernel_abi.py
@@ -37,13 +37,13 @@ import os
import subprocess
import sys
import re
-import kernellog
from docutils import nodes, statemachine
from docutils.statemachine import ViewList
from docutils.parsers.rst import directives, Directive
from docutils.utils.error_reporting import ErrorString
from sphinx.util.docutils import switch_source_input
+from sphinx.util import logging
__version__ = '1.0'
@@ -64,6 +64,7 @@ class KernelCmd(Directive):
optional_arguments = 2
has_content = False
final_argument_whitespace = True
+ logger = logging.getLogger('kernel_abi')
option_spec = {
"debug" : directives.flag,
@@ -129,7 +130,7 @@ class KernelCmd(Directive):
else:
content.append(line, f, ln)
- kernellog.info(self.state.document.settings.env.app, "%s: parsed %i lines" % (fname, n))
+ self.logger.info("%s: parsed %i lines" % (fname, n))
if content:
self.do_parse(content, node)
diff --git a/Documentation/sphinx/kerneldoc.py b/Documentation/sphinx/kerneldoc.py
index ec1ddfff1863..be5b8fbf373f 100644
--- a/Documentation/sphinx/kerneldoc.py
+++ b/Documentation/sphinx/kerneldoc.py
@@ -39,7 +39,7 @@ from docutils.statemachine import ViewList
from docutils.parsers.rst import directives, Directive
import sphinx
from sphinx.util.docutils import switch_source_input
-import kernellog
+from sphinx.util import logging
__version__ = '1.0'
@@ -56,6 +56,7 @@ class KernelDocDirective(Directive):
'functions': directives.unchanged,
}
has_content = False
+ logger = logging.getLogger('kerneldoc')
def run(self):
env = self.state.document.settings.env
@@ -109,8 +110,7 @@ class KernelDocDirective(Directive):
cmd += [filename]
try:
- kernellog.verbose(env.app,
- 'calling kernel-doc \'%s\'' % (" ".join(cmd)))
+ self.logger.verbose("calling kernel-doc '%s'" % (" ".join(cmd)))
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
@@ -120,8 +120,8 @@ class KernelDocDirective(Directive):
if p.returncode != 0:
sys.stderr.write(err)
- kernellog.warn(env.app,
- 'kernel-doc \'%s\' failed with return code %d' % (" ".join(cmd), p.returncode))
+ self.logger.warning("kernel-doc '%s' failed with return code %d"
+ % (" ".join(cmd), p.returncode))
return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))]
elif env.config.kerneldoc_verbosity > 0:
sys.stderr.write(err)
@@ -148,8 +148,8 @@ class KernelDocDirective(Directive):
return node.children
except Exception as e: # pylint: disable=W0703
- kernellog.warn(env.app, 'kernel-doc \'%s\' processing failed with: %s' %
- (" ".join(cmd), str(e)))
+ self.logger.warning("kernel-doc '%s' processing failed with: %s" %
+ (" ".join(cmd), str(e)))
return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))]
def do_parse(self, result, node):
diff --git a/Documentation/sphinx/kernellog.py b/Documentation/sphinx/kernellog.py
deleted file mode 100644
index 0bc00c138cad..000000000000
--- a/Documentation/sphinx/kernellog.py
+++ /dev/null
@@ -1,22 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-#
-# Sphinx has deprecated its older logging interface, but the replacement
-# only goes back to 1.6. So here's a wrapper layer to keep around for
-# as long as we support 1.4.
-#
-# We don't support 1.4 anymore, but we'll keep the wrappers around until
-# we change all the code to not use them anymore :)
-#
-import sphinx
-from sphinx.util import logging
-
-logger = logging.getLogger('kerneldoc')
-
-def warn(app, message):
- logger.warning(message)
-
-def verbose(app, message):
- logger.verbose(message)
-
-def info(app, message):
- logger.info(message)
diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
index 97166333b727..383f9a695b08 100644
--- a/Documentation/sphinx/kfigure.py
+++ b/Documentation/sphinx/kfigure.py
@@ -59,12 +59,14 @@ from docutils.parsers.rst import directives
from docutils.parsers.rst.directives import images
import sphinx
from sphinx.util.nodes import clean_astext
-import kernellog
+from sphinx.util import logging
Figure = images.Figure
__version__ = '1.0.0'
+logger = logging.getLogger('kfigure')
+
# simple helper
# -------------
@@ -170,7 +172,7 @@ def setupTools(app):
"""
global dot_cmd, dot_Tpdf, convert_cmd, rsvg_convert_cmd # pylint: disable=W0603
global inkscape_cmd, inkscape_ver_one # pylint: disable=W0603
- kernellog.verbose(app, "kfigure: check installed tools ...")
+ logger.verbose("kfigure: check installed tools ...")
dot_cmd = which('dot')
convert_cmd = which('convert')
@@ -178,7 +180,7 @@ def setupTools(app):
inkscape_cmd = which('inkscape')
if dot_cmd:
- kernellog.verbose(app, "use dot(1) from: " + dot_cmd)
+ logger.verbose("use dot(1) from: " + dot_cmd)
try:
dot_Thelp_list = subprocess.check_output([dot_cmd, '-Thelp'],
@@ -190,10 +192,11 @@ def setupTools(app):
dot_Tpdf_ptn = b'pdf'
dot_Tpdf = re.search(dot_Tpdf_ptn, dot_Thelp_list)
else:
- kernellog.warn(app, "dot(1) not found, for better output quality install "
- "graphviz from https://www.graphviz.org")
+ logger.warning(
+ "dot(1) not found, for better output quality install graphviz from https://www.graphviz.org"
+ )
if inkscape_cmd:
- kernellog.verbose(app, "use inkscape(1) from: " + inkscape_cmd)
+ logger.verbose("use inkscape(1) from: " + inkscape_cmd)
inkscape_ver = subprocess.check_output([inkscape_cmd, '--version'],
stderr=subprocess.DEVNULL)
ver_one_ptn = b'Inkscape 1'
@@ -204,26 +207,27 @@ def setupTools(app):
else:
if convert_cmd:
- kernellog.verbose(app, "use convert(1) from: " + convert_cmd)
+ logger.verbose("use convert(1) from: " + convert_cmd)
else:
- kernellog.verbose(app,
+ logger.verbose(
"Neither inkscape(1) nor convert(1) found.\n"
- "For SVG to PDF conversion, "
- "install either Inkscape (https://inkscape.org/) (preferred) or\n"
- "ImageMagick (https://www.imagemagick.org)")
+ "For SVG to PDF conversion, install either Inkscape (https://inkscape.org/) (preferred) or\n"
+ "ImageMagick (https://www.imagemagick.org)"
+ )
if rsvg_convert_cmd:
- kernellog.verbose(app, "use rsvg-convert(1) from: " + rsvg_convert_cmd)
- kernellog.verbose(app, "use 'dot -Tsvg' and rsvg-convert(1) for DOT -> PDF conversion")
+ logger.verbose("use rsvg-convert(1) from: " + rsvg_convert_cmd)
+ logger.verbose("use 'dot -Tsvg' and rsvg-convert(1) for DOT -> PDF conversion")
dot_Tpdf = False
else:
- kernellog.verbose(app,
+ logger.verbose(
"rsvg-convert(1) not found.\n"
- " SVG rendering of convert(1) is done by ImageMagick-native renderer.")
+ " SVG rendering of convert(1) is done by ImageMagick-native renderer."
+ )
if dot_Tpdf:
- kernellog.verbose(app, "use 'dot -Tpdf' for DOT -> PDF conversion")
+ logger.verbose("use 'dot -Tpdf' for DOT -> PDF conversion")
else:
- kernellog.verbose(app, "use 'dot -Tsvg' and convert(1) for DOT -> PDF conversion")
+ logger.verbose("use 'dot -Tsvg' and convert(1) for DOT -> PDF conversion")
# integrate conversion tools
@@ -257,13 +261,12 @@ def convert_image(img_node, translator, src_fname=None):
# in kernel builds, use 'make SPHINXOPTS=-v' to see verbose messages
- kernellog.verbose(app, 'assert best format for: ' + img_node['uri'])
+ logger.verbose('assert best format for: ' + img_node['uri'])
if in_ext == '.dot':
if not dot_cmd:
- kernellog.verbose(app,
- "dot from graphviz not available / include DOT raw.")
+ logger.verbose("dot from graphviz not available / include DOT raw.")
img_node.replace_self(file2literal(src_fname))
elif translator.builder.format == 'latex':
@@ -290,10 +293,11 @@ def convert_image(img_node, translator, src_fname=None):
if translator.builder.format == 'latex':
if not inkscape_cmd and convert_cmd is None:
- kernellog.warn(app,
- "no SVG to PDF conversion available / include SVG raw."
- "\nIncluding large raw SVGs can cause xelatex error."
- "\nInstall Inkscape (preferred) or ImageMagick.")
+ logger.warning(
+ "no SVG to PDF conversion available / include SVG raw.\n"
+ "Including large raw SVGs can cause xelatex error.\n"
+ "Install Inkscape (preferred) or ImageMagick."
+ )
img_node.replace_self(file2literal(src_fname))
else:
dst_fname = path.join(translator.builder.outdir, fname + '.pdf')
@@ -306,15 +310,14 @@ def convert_image(img_node, translator, src_fname=None):
_name = dst_fname[len(str(translator.builder.outdir)) + 1:]
if isNewer(dst_fname, src_fname):
- kernellog.verbose(app,
- "convert: {out}/%s already exists and is newer" % _name)
+ logger.verbose("convert: {out}/%s already exists and is newer" % _name)
else:
ok = False
mkdir(path.dirname(dst_fname))
if in_ext == '.dot':
- kernellog.verbose(app, 'convert DOT to: {out}/' + _name)
+ logger.verbose('convert DOT to: {out}/' + _name)
if translator.builder.format == 'latex' and not dot_Tpdf:
svg_fname = path.join(translator.builder.outdir, fname + '.svg')
ok1 = dot2format(app, src_fname, svg_fname)
@@ -325,7 +328,7 @@ def convert_image(img_node, translator, src_fname=None):
ok = dot2format(app, src_fname, dst_fname)
elif in_ext == '.svg':
- kernellog.verbose(app, 'convert SVG to: {out}/' + _name)
+ logger.verbose('convert SVG to: {out}/' + _name)
ok = svg2pdf(app, src_fname, dst_fname)
if not ok:
@@ -354,7 +357,7 @@ def dot2format(app, dot_fname, out_fname):
with open(out_fname, "w") as out:
exit_code = subprocess.call(cmd, stdout = out)
if exit_code != 0:
- kernellog.warn(app,
+ logger.warning(
"Error #%d when calling: %s" % (exit_code, " ".join(cmd)))
return bool(exit_code == 0)
@@ -388,13 +391,14 @@ def svg2pdf(app, svg_fname, pdf_fname):
pass
if exit_code != 0:
- kernellog.warn(app, "Error #%d when calling: %s" % (exit_code, " ".join(cmd)))
+ logger.warning("Error #%d when calling: %s" %
+ (exit_code, " ".join(cmd)))
if warning_msg:
- kernellog.warn(app, "Warning msg from %s: %s"
- % (cmd_name, str(warning_msg, 'utf-8')))
+ logger.warning( "Warning msg from %s: %s" %
+ (cmd_name, str(warning_msg, 'utf-8')))
elif warning_msg:
- kernellog.verbose(app, "Warning msg from %s (likely harmless):\n%s"
- % (cmd_name, str(warning_msg, 'utf-8')))
+ logger.verbose("Warning msg from %s (likely harmless):\n%s" %
+ (cmd_name, str(warning_msg, 'utf-8')))
return bool(exit_code == 0)
@@ -418,7 +422,8 @@ def svg2pdf_by_rsvg(app, svg_fname, pdf_fname):
# use stdout and stderr from parent
exit_code = subprocess.call(cmd)
if exit_code != 0:
- kernellog.warn(app, "Error #%d when calling: %s" % (exit_code, " ".join(cmd)))
+ logger.warning("Error #%d when calling: %s" %
+ (exit_code, " ".join(cmd)))
ok = bool(exit_code == 0)
return ok
@@ -513,15 +518,15 @@ def visit_kernel_render(self, node):
app = self.builder.app
srclang = node.get('srclang')
- kernellog.verbose(app, 'visit kernel-render node lang: "%s"' % (srclang))
+ logger.verbose('visit kernel-render node lang: "%s"' % srclang)
tmp_ext = RENDER_MARKUP_EXT.get(srclang, None)
if tmp_ext is None:
- kernellog.warn(app, 'kernel-render: "%s" unknown / include raw.' % (srclang))
+ logger.warning( 'kernel-render: "%s" unknown / include raw.' % srclang)
return
if not dot_cmd and tmp_ext == '.dot':
- kernellog.verbose(app, "dot from graphviz not available / include raw.")
+ logger.verbose("dot from graphviz not available / include raw.")
return
literal_block = node[0]
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 02/27] docs: sphinx/kernel_abi: adjust coding style
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
2025-02-10 10:17 ` [PATCH 01/27] docs: sphinx: remove kernellog.py file Mauro Carvalho Chehab
@ 2025-02-10 10:17 ` Mauro Carvalho Chehab
2025-02-10 10:17 ` [PATCH 03/27] docs: admin-guide: abi: add SPDX tags to ABI files Mauro Carvalho Chehab
` (26 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:17 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Make pylint and flake8 happier with this module's coding style
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/sphinx/kernel_abi.py | 25 ++++++++++++-------------
1 file changed, 12 insertions(+), 13 deletions(-)
diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
index 8401562cb5d9..38653f5706c0 100644
--- a/Documentation/sphinx/kernel_abi.py
+++ b/Documentation/sphinx/kernel_abi.py
@@ -32,32 +32,31 @@ u"""
"""
-import codecs
import os
+import re
import subprocess
import sys
-import re
-from docutils import nodes, statemachine
+from docutils import nodes
from docutils.statemachine import ViewList
from docutils.parsers.rst import directives, Directive
-from docutils.utils.error_reporting import ErrorString
from sphinx.util.docutils import switch_source_input
from sphinx.util import logging
-__version__ = '1.0'
+__version__ = "1.0"
+
def setup(app):
app.add_directive("kernel-abi", KernelCmd)
- return dict(
- version = __version__
- , parallel_read_safe = True
- , parallel_write_safe = True
- )
+ return {
+ "version": __version__,
+ "parallel_read_safe": True,
+ "parallel_write_safe": True
+ }
+
class KernelCmd(Directive):
-
u"""KernelABI (``kernel-abi``) directive"""
required_arguments = 1
@@ -99,8 +98,8 @@ class KernelCmd(Directive):
if "debug" in self.options:
code_block = "\n\n.. code-block:: rst\n :linenos:\n"
- for l in lines.split("\n"):
- code_block += "\n " + l
+ for line in lines.split("\n"):
+ code_block += "\n " + line
lines = code_block + "\n\n"
line_regex = re.compile(r"^\.\. LINENO (\S+)\#([0-9]+)$")
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 03/27] docs: admin-guide: abi: add SPDX tags to ABI files
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
2025-02-10 10:17 ` [PATCH 01/27] docs: sphinx: remove kernellog.py file Mauro Carvalho Chehab
2025-02-10 10:17 ` [PATCH 02/27] docs: sphinx/kernel_abi: adjust coding style Mauro Carvalho Chehab
@ 2025-02-10 10:17 ` Mauro Carvalho Chehab
2025-02-10 10:17 ` [PATCH 04/27] ABI: sysfs-class-rfkill: fix kernelversion tags Mauro Carvalho Chehab
` (25 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:17 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Such files are missing SPDX tags containing the licensing information.
Add them.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/admin-guide/abi-obsolete.rst | 2 ++
Documentation/admin-guide/abi-removed.rst | 2 ++
Documentation/admin-guide/abi-stable.rst | 2 ++
Documentation/admin-guide/abi-testing.rst | 2 ++
Documentation/admin-guide/abi.rst | 2 ++
5 files changed, 10 insertions(+)
diff --git a/Documentation/admin-guide/abi-obsolete.rst b/Documentation/admin-guide/abi-obsolete.rst
index 594e697aa1b2..b655615917f1 100644
--- a/Documentation/admin-guide/abi-obsolete.rst
+++ b/Documentation/admin-guide/abi-obsolete.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: GPL-2.0
+
ABI obsolete symbols
====================
diff --git a/Documentation/admin-guide/abi-removed.rst b/Documentation/admin-guide/abi-removed.rst
index f9e000c81828..ba941c1af178 100644
--- a/Documentation/admin-guide/abi-removed.rst
+++ b/Documentation/admin-guide/abi-removed.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: GPL-2.0
+
ABI removed symbols
===================
diff --git a/Documentation/admin-guide/abi-stable.rst b/Documentation/admin-guide/abi-stable.rst
index fc3361d847b1..5d738f345333 100644
--- a/Documentation/admin-guide/abi-stable.rst
+++ b/Documentation/admin-guide/abi-stable.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: GPL-2.0
+
ABI stable symbols
==================
diff --git a/Documentation/admin-guide/abi-testing.rst b/Documentation/admin-guide/abi-testing.rst
index 19767926b344..a867e6578bf7 100644
--- a/Documentation/admin-guide/abi-testing.rst
+++ b/Documentation/admin-guide/abi-testing.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: GPL-2.0
+
ABI testing symbols
===================
diff --git a/Documentation/admin-guide/abi.rst b/Documentation/admin-guide/abi.rst
index bcab3ef2597c..64e772bde943 100644
--- a/Documentation/admin-guide/abi.rst
+++ b/Documentation/admin-guide/abi.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: GPL-2.0
+
=====================
Linux ABI description
=====================
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 04/27] ABI: sysfs-class-rfkill: fix kernelversion tags
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (2 preceding siblings ...)
2025-02-10 10:17 ` [PATCH 03/27] docs: admin-guide: abi: add SPDX tags to ABI files Mauro Carvalho Chehab
@ 2025-02-10 10:17 ` Mauro Carvalho Chehab
2025-02-10 10:17 ` [PATCH 05/27] ABI: sysfs-bus-coresight-*: " Mauro Carvalho Chehab
` (24 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:17 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
Johannes Berg, linux-kernel, linux-wireless
Some kernelversion tags are missing colons. Add them to comply with
ABI description and produce right results when converted to html/pdf.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/ABI/removed/sysfs-class-rfkill | 2 +-
Documentation/ABI/stable/sysfs-class-rfkill | 12 ++++++------
2 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/Documentation/ABI/removed/sysfs-class-rfkill b/Documentation/ABI/removed/sysfs-class-rfkill
index f25174eafd55..20cb688af173 100644
--- a/Documentation/ABI/removed/sysfs-class-rfkill
+++ b/Documentation/ABI/removed/sysfs-class-rfkill
@@ -4,7 +4,7 @@ For details to this subsystem look at Documentation/driver-api/rfkill.rst.
What: /sys/class/rfkill/rfkill[0-9]+/claim
Date: 09-Jul-2007
-KernelVersion v2.6.22
+KernelVersion: v2.6.22
Contact: linux-wireless@vger.kernel.org
Description: This file was deprecated because there no longer was a way to
claim just control over a single rfkill instance.
diff --git a/Documentation/ABI/stable/sysfs-class-rfkill b/Documentation/ABI/stable/sysfs-class-rfkill
index 037979f7dc4b..67b605e3dd16 100644
--- a/Documentation/ABI/stable/sysfs-class-rfkill
+++ b/Documentation/ABI/stable/sysfs-class-rfkill
@@ -16,7 +16,7 @@ Description: The rfkill class subsystem folder.
What: /sys/class/rfkill/rfkill[0-9]+/name
Date: 09-Jul-2007
-KernelVersion v2.6.22
+KernelVersion: v2.6.22
Contact: linux-wireless@vger.kernel.org
Description: Name assigned by driver to this key (interface or driver name).
Values: arbitrary string.
@@ -24,7 +24,7 @@ Values: arbitrary string.
What: /sys/class/rfkill/rfkill[0-9]+/type
Date: 09-Jul-2007
-KernelVersion v2.6.22
+KernelVersion: v2.6.22
Contact: linux-wireless@vger.kernel.org
Description: Driver type string ("wlan", "bluetooth", etc).
Values: See include/linux/rfkill.h.
@@ -32,7 +32,7 @@ Values: See include/linux/rfkill.h.
What: /sys/class/rfkill/rfkill[0-9]+/persistent
Date: 09-Jul-2007
-KernelVersion v2.6.22
+KernelVersion: v2.6.22
Contact: linux-wireless@vger.kernel.org
Description: Whether the soft blocked state is initialised from non-volatile
storage at startup.
@@ -44,7 +44,7 @@ Values: A numeric value:
What: /sys/class/rfkill/rfkill[0-9]+/state
Date: 09-Jul-2007
-KernelVersion v2.6.22
+KernelVersion: v2.6.22
Contact: linux-wireless@vger.kernel.org
Description: Current state of the transmitter.
This file was scheduled to be removed in 2014, but due to its
@@ -67,7 +67,7 @@ Values: A numeric value.
What: /sys/class/rfkill/rfkill[0-9]+/hard
Date: 12-March-2010
-KernelVersion v2.6.34
+KernelVersion: v2.6.34
Contact: linux-wireless@vger.kernel.org
Description: Current hardblock state. This file is read only.
Values: A numeric value.
@@ -81,7 +81,7 @@ Values: A numeric value.
What: /sys/class/rfkill/rfkill[0-9]+/soft
Date: 12-March-2010
-KernelVersion v2.6.34
+KernelVersion: v2.6.34
Contact: linux-wireless@vger.kernel.org
Description: Current softblock state. This file is read and write.
Values: A numeric value.
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 05/27] ABI: sysfs-bus-coresight-*: fix kernelversion tags
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (3 preceding siblings ...)
2025-02-10 10:17 ` [PATCH 04/27] ABI: sysfs-class-rfkill: fix kernelversion tags Mauro Carvalho Chehab
@ 2025-02-10 10:17 ` Mauro Carvalho Chehab
2025-02-10 10:33 ` Suzuki K Poulose
2025-02-10 10:17 ` [PATCH 06/27] ABI: sysfs-driver-dma-idxd: fix date tags Mauro Carvalho Chehab
` (23 subsequent siblings)
28 siblings, 1 reply; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:17 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
James Clark, Mike Leach, Suzuki K Poulose, coresight,
linux-arm-kernel, linux-kernel
Some kernelversion tags are missing colons. Add them to comply with
ABI description and produce right results when converted to html/pdf.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
.../testing/sysfs-bus-coresight-devices-cti | 78 +++++++++----------
.../testing/sysfs-bus-coresight-devices-tpdm | 52 ++++++-------
2 files changed, 65 insertions(+), 65 deletions(-)
diff --git a/Documentation/ABI/testing/sysfs-bus-coresight-devices-cti b/Documentation/ABI/testing/sysfs-bus-coresight-devices-cti
index bf2869c413e7..a97b70f588da 100644
--- a/Documentation/ABI/testing/sysfs-bus-coresight-devices-cti
+++ b/Documentation/ABI/testing/sysfs-bus-coresight-devices-cti
@@ -1,241 +1,241 @@
What: /sys/bus/coresight/devices/<cti-name>/enable
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (RW) Enable/Disable the CTI hardware.
What: /sys/bus/coresight/devices/<cti-name>/powered
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Read) Indicate if the CTI hardware is powered.
What: /sys/bus/coresight/devices/<cti-name>/ctmid
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Read) Display the associated CTM ID
What: /sys/bus/coresight/devices/<cti-name>/nr_trigger_cons
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Read) Number of devices connected to triggers on this CTI
What: /sys/bus/coresight/devices/<cti-name>/triggers<N>/name
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Read) Name of connected device <N>
What: /sys/bus/coresight/devices/<cti-name>/triggers<N>/in_signals
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Read) Input trigger signals from connected device <N>
What: /sys/bus/coresight/devices/<cti-name>/triggers<N>/in_types
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Read) Functional types for the input trigger signals
from connected device <N>
What: /sys/bus/coresight/devices/<cti-name>/triggers<N>/out_signals
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Read) Output trigger signals to connected device <N>
What: /sys/bus/coresight/devices/<cti-name>/triggers<N>/out_types
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Read) Functional types for the output trigger signals
to connected device <N>
What: /sys/bus/coresight/devices/<cti-name>/regs/inout_sel
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (RW) Select the index for inen and outen registers.
What: /sys/bus/coresight/devices/<cti-name>/regs/inen
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (RW) Read or write the CTIINEN register selected by inout_sel.
What: /sys/bus/coresight/devices/<cti-name>/regs/outen
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (RW) Read or write the CTIOUTEN register selected by inout_sel.
What: /sys/bus/coresight/devices/<cti-name>/regs/gate
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (RW) Read or write CTIGATE register.
What: /sys/bus/coresight/devices/<cti-name>/regs/asicctl
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (RW) Read or write ASICCTL register.
What: /sys/bus/coresight/devices/<cti-name>/regs/intack
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Write) Write the INTACK register.
What: /sys/bus/coresight/devices/<cti-name>/regs/appset
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (RW) Set CTIAPPSET register to activate channel. Read back to
determine current value of register.
What: /sys/bus/coresight/devices/<cti-name>/regs/appclear
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Write) Write APPCLEAR register to deactivate channel.
What: /sys/bus/coresight/devices/<cti-name>/regs/apppulse
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Write) Write APPPULSE to pulse a channel active for one clock
cycle.
What: /sys/bus/coresight/devices/<cti-name>/regs/chinstatus
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Read) Read current status of channel inputs.
What: /sys/bus/coresight/devices/<cti-name>/regs/choutstatus
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Read) read current status of channel outputs.
What: /sys/bus/coresight/devices/<cti-name>/regs/triginstatus
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Read) read current status of input trigger signals
What: /sys/bus/coresight/devices/<cti-name>/regs/trigoutstatus
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Read) read current status of output trigger signals.
What: /sys/bus/coresight/devices/<cti-name>/channels/trigin_attach
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Write) Attach a CTI input trigger to a CTM channel.
What: /sys/bus/coresight/devices/<cti-name>/channels/trigin_detach
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Write) Detach a CTI input trigger from a CTM channel.
What: /sys/bus/coresight/devices/<cti-name>/channels/trigout_attach
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Write) Attach a CTI output trigger to a CTM channel.
What: /sys/bus/coresight/devices/<cti-name>/channels/trigout_detach
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Write) Detach a CTI output trigger from a CTM channel.
What: /sys/bus/coresight/devices/<cti-name>/channels/chan_gate_enable
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (RW) Enable CTIGATE for single channel (Write) or list enabled
channels through the gate (R).
What: /sys/bus/coresight/devices/<cti-name>/channels/chan_gate_disable
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Write) Disable CTIGATE for single channel.
What: /sys/bus/coresight/devices/<cti-name>/channels/chan_set
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Write) Activate a single channel.
What: /sys/bus/coresight/devices/<cti-name>/channels/chan_clear
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Write) Deactivate a single channel.
What: /sys/bus/coresight/devices/<cti-name>/channels/chan_pulse
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Write) Pulse a single channel - activate for a single clock cycle.
What: /sys/bus/coresight/devices/<cti-name>/channels/trigout_filtered
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Read) List of output triggers filtered across all connections.
What: /sys/bus/coresight/devices/<cti-name>/channels/trig_filter_enable
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (RW) Enable or disable trigger output signal filtering.
What: /sys/bus/coresight/devices/<cti-name>/channels/chan_inuse
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Read) show channels with at least one attached trigger signal.
What: /sys/bus/coresight/devices/<cti-name>/channels/chan_free
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Read) show channels with no attached trigger signals.
What: /sys/bus/coresight/devices/<cti-name>/channels/chan_xtrigs_sel
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (RW) Write channel number to select a channel to view, read to
see selected channel number.
What: /sys/bus/coresight/devices/<cti-name>/channels/chan_xtrigs_in
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Read) Read to see input triggers connected to selected view
channel.
What: /sys/bus/coresight/devices/<cti-name>/channels/chan_xtrigs_out
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Read) Read to see output triggers connected to selected view
channel.
What: /sys/bus/coresight/devices/<cti-name>/channels/chan_xtrigs_reset
Date: March 2020
-KernelVersion 5.7
+KernelVersion: 5.7
Contact: Mike Leach or Mathieu Poirier
Description: (Write) Clear all channel / trigger programming.
diff --git a/Documentation/ABI/testing/sysfs-bus-coresight-devices-tpdm b/Documentation/ABI/testing/sysfs-bus-coresight-devices-tpdm
index bf710ea6e0ef..53cb454b60d0 100644
--- a/Documentation/ABI/testing/sysfs-bus-coresight-devices-tpdm
+++ b/Documentation/ABI/testing/sysfs-bus-coresight-devices-tpdm
@@ -1,6 +1,6 @@
What: /sys/bus/coresight/devices/<tpdm-name>/integration_test
Date: January 2023
-KernelVersion 6.2
+KernelVersion: 6.2
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
(Write) Run integration test for tpdm. Integration test
@@ -14,7 +14,7 @@ Description:
What: /sys/bus/coresight/devices/<tpdm-name>/reset_dataset
Date: March 2023
-KernelVersion 6.7
+KernelVersion: 6.7
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
(Write) Reset the dataset of the tpdm.
@@ -24,7 +24,7 @@ Description:
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_trig_type
Date: March 2023
-KernelVersion 6.7
+KernelVersion: 6.7
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
(RW) Set/Get the trigger type of the DSB for tpdm.
@@ -35,7 +35,7 @@ Description:
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_trig_ts
Date: March 2023
-KernelVersion 6.7
+KernelVersion: 6.7
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
(RW) Set/Get the trigger timestamp of the DSB for tpdm.
@@ -46,7 +46,7 @@ Description:
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_mode
Date: March 2023
-KernelVersion 6.7
+KernelVersion: 6.7
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
(RW) Set/Get the programming mode of the DSB for tpdm.
@@ -60,7 +60,7 @@ Description:
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_edge/ctrl_idx
Date: March 2023
-KernelVersion 6.7
+KernelVersion: 6.7
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
(RW) Set/Get the index number of the edge detection for the DSB
@@ -69,7 +69,7 @@ Description:
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_edge/ctrl_val
Date: March 2023
-KernelVersion 6.7
+KernelVersion: 6.7
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
Write a data to control the edge detection corresponding to
@@ -85,7 +85,7 @@ Description:
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_edge/ctrl_mask
Date: March 2023
-KernelVersion 6.7
+KernelVersion: 6.7
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
Write a data to mask the edge detection corresponding to the index
@@ -97,21 +97,21 @@ Description:
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_edge/edcr[0:15]
Date: March 2023
-KernelVersion 6.7
+KernelVersion: 6.7
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
Read a set of the edge control value of the DSB in TPDM.
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_edge/edcmr[0:7]
Date: March 2023
-KernelVersion 6.7
+KernelVersion: 6.7
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
Read a set of the edge control mask of the DSB in TPDM.
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_trig_patt/xpr[0:7]
Date: March 2023
-KernelVersion 6.7
+KernelVersion: 6.7
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
(RW) Set/Get the value of the trigger pattern for the DSB
@@ -119,7 +119,7 @@ Description:
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_trig_patt/xpmr[0:7]
Date: March 2023
-KernelVersion 6.7
+KernelVersion: 6.7
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
(RW) Set/Get the mask of the trigger pattern for the DSB
@@ -127,21 +127,21 @@ Description:
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_patt/tpr[0:7]
Date: March 2023
-KernelVersion 6.7
+KernelVersion: 6.7
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
(RW) Set/Get the value of the pattern for the DSB subunit TPDM.
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_patt/tpmr[0:7]
Date: March 2023
-KernelVersion 6.7
+KernelVersion: 6.7
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
(RW) Set/Get the mask of the pattern for the DSB subunit TPDM.
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_patt/enable_ts
Date: March 2023
-KernelVersion 6.7
+KernelVersion: 6.7
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
(Write) Set the pattern timestamp of DSB tpdm. Read
@@ -153,7 +153,7 @@ Description:
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_patt/set_type
Date: March 2023
-KernelVersion 6.7
+KernelVersion: 6.7
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
(Write) Set the pattern type of DSB tpdm. Read
@@ -165,7 +165,7 @@ Description:
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_msr/msr[0:31]
Date: March 2023
-KernelVersion 6.7
+KernelVersion: 6.7
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
(RW) Set/Get the MSR(mux select register) for the DSB subunit
@@ -173,7 +173,7 @@ Description:
What: /sys/bus/coresight/devices/<tpdm-name>/cmb_mode
Date: January 2024
-KernelVersion 6.9
+KernelVersion: 6.9
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description: (Write) Set the data collection mode of CMB tpdm. Continuous
change creates CMB data set elements on every CMBCLK edge.
@@ -187,7 +187,7 @@ Description: (Write) Set the data collection mode of CMB tpdm. Continuous
What: /sys/bus/coresight/devices/<tpdm-name>/cmb_trig_patt/xpr[0:1]
Date: January 2024
-KernelVersion 6.9
+KernelVersion: 6.9
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
(RW) Set/Get the value of the trigger pattern for the CMB
@@ -195,7 +195,7 @@ Description:
What: /sys/bus/coresight/devices/<tpdm-name>/cmb_trig_patt/xpmr[0:1]
Date: January 2024
-KernelVersion 6.9
+KernelVersion: 6.9
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
(RW) Set/Get the mask of the trigger pattern for the CMB
@@ -203,21 +203,21 @@ Description:
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_patt/tpr[0:1]
Date: January 2024
-KernelVersion 6.9
+KernelVersion: 6.9
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
(RW) Set/Get the value of the pattern for the CMB subunit TPDM.
What: /sys/bus/coresight/devices/<tpdm-name>/dsb_patt/tpmr[0:1]
Date: January 2024
-KernelVersion 6.9
+KernelVersion: 6.9
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
(RW) Set/Get the mask of the pattern for the CMB subunit TPDM.
What: /sys/bus/coresight/devices/<tpdm-name>/cmb_patt/enable_ts
Date: January 2024
-KernelVersion 6.9
+KernelVersion: 6.9
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
(Write) Set the pattern timestamp of CMB tpdm. Read
@@ -229,7 +229,7 @@ Description:
What: /sys/bus/coresight/devices/<tpdm-name>/cmb_trig_ts
Date: January 2024
-KernelVersion 6.9
+KernelVersion: 6.9
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
(RW) Set/Get the trigger timestamp of the CMB for tpdm.
@@ -240,7 +240,7 @@ Description:
What: /sys/bus/coresight/devices/<tpdm-name>/cmb_ts_all
Date: January 2024
-KernelVersion 6.9
+KernelVersion: 6.9
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
(RW) Read or write the status of timestamp upon all interface.
@@ -252,7 +252,7 @@ Description:
What: /sys/bus/coresight/devices/<tpdm-name>/cmb_msr/msr[0:31]
Date: January 2024
-KernelVersion 6.9
+KernelVersion: 6.9
Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
Description:
(RW) Set/Get the MSR(mux select register) for the CMB subunit
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 06/27] ABI: sysfs-driver-dma-idxd: fix date tags
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (4 preceding siblings ...)
2025-02-10 10:17 ` [PATCH 05/27] ABI: sysfs-bus-coresight-*: " Mauro Carvalho Chehab
@ 2025-02-10 10:17 ` Mauro Carvalho Chehab
2025-02-10 10:17 ` [PATCH 07/27] ABI: sysfs-fs-f2fs: " Mauro Carvalho Chehab
` (22 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:17 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Some date tags are missing colons. Add them to comply with
ABI description and produce right results when converted to html/pdf.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/ABI/stable/sysfs-driver-dma-idxd | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/Documentation/ABI/stable/sysfs-driver-dma-idxd b/Documentation/ABI/stable/sysfs-driver-dma-idxd
index f2ec42949a54..4a355e6747ae 100644
--- a/Documentation/ABI/stable/sysfs-driver-dma-idxd
+++ b/Documentation/ABI/stable/sysfs-driver-dma-idxd
@@ -246,14 +246,14 @@ Description: Controls whether PRS disable is turned on for the workqueue.
capability.
What: /sys/bus/dsa/devices/wq<m>.<n>/occupancy
-Date May 25, 2021
+Date: May 25, 2021
KernelVersion: 5.14.0
Contact: dmaengine@vger.kernel.org
Description: Show the current number of entries in this WQ if WQ Occupancy
Support bit WQ capabilities is 1.
What: /sys/bus/dsa/devices/wq<m>.<n>/enqcmds_retries
-Date Oct 29, 2021
+Date: Oct 29, 2021
KernelVersion: 5.17.0
Contact: dmaengine@vger.kernel.org
Description: Indicate the number of retires for an enqcmds submission on a sharedwq.
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 07/27] ABI: sysfs-fs-f2fs: fix date tags
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (5 preceding siblings ...)
2025-02-10 10:17 ` [PATCH 06/27] ABI: sysfs-driver-dma-idxd: fix date tags Mauro Carvalho Chehab
@ 2025-02-10 10:17 ` Mauro Carvalho Chehab
2025-02-10 10:17 ` [PATCH 08/27] ABI: sysfs-power: fix a what tag Mauro Carvalho Chehab
` (21 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:17 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
Chao Yu, Jaegeuk Kim, linux-f2fs-devel, linux-kernel
Some date tags are missing colons. Add them to comply with
ABI description and produce right results when converted to html/pdf.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/ABI/testing/sysfs-fs-f2fs | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/Documentation/ABI/testing/sysfs-fs-f2fs b/Documentation/ABI/testing/sysfs-fs-f2fs
index 3e1630c70d8a..e44bb614964b 100644
--- a/Documentation/ABI/testing/sysfs-fs-f2fs
+++ b/Documentation/ABI/testing/sysfs-fs-f2fs
@@ -347,7 +347,7 @@ Description: Used to control configure extension list:
- [c] means add/del cold file extension
What: /sys/fs/f2fs/<disk>/unusable
-Date April 2019
+Date: April 2019
Contact: "Daniel Rosenberg" <drosen@google.com>
Description: If checkpoint=disable, it displays the number of blocks that
are unusable.
@@ -355,7 +355,7 @@ Description: If checkpoint=disable, it displays the number of blocks that
would be unusable if checkpoint=disable were to be set.
What: /sys/fs/f2fs/<disk>/encoding
-Date July 2019
+Date: July 2019
Contact: "Daniel Rosenberg" <drosen@google.com>
Description: Displays name and version of the encoding set for the filesystem.
If no encoding is set, displays (none)
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 08/27] ABI: sysfs-power: fix a what tag
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (6 preceding siblings ...)
2025-02-10 10:17 ` [PATCH 07/27] ABI: sysfs-fs-f2fs: " Mauro Carvalho Chehab
@ 2025-02-10 10:17 ` Mauro Carvalho Chehab
2025-02-10 10:17 ` [PATCH 09/27] scripts/documentation-file-ref-check: don't check perl/python scripts Mauro Carvalho Chehab
` (20 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:17 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
There is one What tag that it is using semicolon instead of colon.
Fix it to comply with ABI description and produce right results when
converted to html/pdf.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/ABI/testing/sysfs-power | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Documentation/ABI/testing/sysfs-power b/Documentation/ABI/testing/sysfs-power
index a3942b1036e2..2192478e83cf 100644
--- a/Documentation/ABI/testing/sysfs-power
+++ b/Documentation/ABI/testing/sysfs-power
@@ -131,7 +131,7 @@ Description:
CAUTION: Using it will cause your machine's real-time (CMOS)
clock to be set to a random invalid time after a resume.
-What; /sys/power/pm_trace_dev_match
+What: /sys/power/pm_trace_dev_match
Date: October 2010
Contact: James Hogan <jhogan@kernel.org>
Description:
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 09/27] scripts/documentation-file-ref-check: don't check perl/python scripts
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (7 preceding siblings ...)
2025-02-10 10:17 ` [PATCH 08/27] ABI: sysfs-power: fix a what tag Mauro Carvalho Chehab
@ 2025-02-10 10:17 ` Mauro Carvalho Chehab
2025-02-10 10:17 ` [PATCH 10/27] scripts/get_abi.py: add a Python tool to generate ReST output Mauro Carvalho Chehab
` (19 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:17 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Such scripts may have regular expressions, which would make the
parser confusing. Also, they shouldn't hardcode filenames there,
so skipping them is OK.
While here, also don't check references on extensions used for file
backup and patch rej/orig.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/documentation-file-ref-check | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/documentation-file-ref-check b/scripts/documentation-file-ref-check
index 68083f2f1122..408b1dbe7884 100755
--- a/scripts/documentation-file-ref-check
+++ b/scripts/documentation-file-ref-check
@@ -92,7 +92,7 @@ while (<IN>) {
next if ($f =~ m,^Next/,);
# Makefiles and scripts contain nasty expressions to parse docs
- next if ($f =~ m/Makefile/ || $f =~ m/\.sh$/);
+ next if ($f =~ m/Makefile/ || $f =~ m/\.(sh|py|pl|~|rej|org|orig)$/);
# It doesn't make sense to parse hidden files
next if ($f =~ m#/\.#);
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 10/27] scripts/get_abi.py: add a Python tool to generate ReST output
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (8 preceding siblings ...)
2025-02-10 10:17 ` [PATCH 09/27] scripts/documentation-file-ref-check: don't check perl/python scripts Mauro Carvalho Chehab
@ 2025-02-10 10:17 ` Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 11/27] scripts/get_abi.py: add support for symbol search Mauro Carvalho Chehab
` (18 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:17 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
The get_abi.pl script is requiring some care, but it seems that
the number of changes on it since when I originally wrote it
was not too high.
Maintaining perl scripts without using classes requires a higher
efforted than on python, due to global variables management.
Also, it sounds easier to find python developer those days than
perl ones.
As a plus, using a Python class to handle ABI allows a better
integration with Sphinx extensions, allowing, for instance,
to let automarkup to generate cross-references for ABI
symbols.
With that in mind, rewrite the core of get_abi.pl in Python,
using classes, to help producing documentation. This will
allow a better integration in the future with the Sphinx
ABI extension.
The algorithms used there are the same as the ones in Perl,
with a couple of cleanups to remove redundant variables and
to help with cross-reference generation. While doing that,
remove some code that were important in the past, where
ABI files weren't using ReST format.
Some minor improvements were added like using a fixed seed
when generating ABI keys for duplicated names, making its
results reproductible.
The end script is a little bit faster than the original one
(tested on a machine with ssd disks). That's probably because
we're now using only pre-compiled regular expressions, and it
is using string replacement methods instead of regex where
possible.
The new version is a little bit more conservative when
converting text to cross-references to avoid adding them into
literal blocks.
To ensure that the ReST output is parsing all variables
and files properly, the end result was compared using diff
with the one produced by the perl script and showed no regressions.
There are minor improvements at the results, as it now
properly groups What on some special cases. It also better
escape some XREF names.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/get_abi.py | 118 ++++++++
scripts/lib/abi/abi_parser.py | 512 ++++++++++++++++++++++++++++++++++
scripts/lib/abi/helpers.py | 28 ++
3 files changed, 658 insertions(+)
create mode 100755 scripts/get_abi.py
create mode 100644 scripts/lib/abi/abi_parser.py
create mode 100644 scripts/lib/abi/helpers.py
diff --git a/scripts/get_abi.py b/scripts/get_abi.py
new file mode 100755
index 000000000000..bb17c54feeff
--- /dev/null
+++ b/scripts/get_abi.py
@@ -0,0 +1,118 @@
+#!/usr/bin/env python3
+# pylint: disable=R0903
+# Copyright(c) 2025: Mauro Carvalho Chehab <mchehab@kernel.org>.
+# SPDX-License-Identifier: GPL-2.0
+
+"""
+Parse ABI documentation and produce results from it.
+"""
+
+import argparse
+import logging
+import os
+import sys
+
+# Import Python modules
+
+LIB_DIR = "lib/abi"
+SRC_DIR = os.path.dirname(os.path.realpath(__file__))
+
+sys.path.insert(0, os.path.join(SRC_DIR, LIB_DIR))
+
+from abi_parser import AbiParser # pylint: disable=C0413
+from helpers import ABI_DIR, DEBUG_HELP # pylint: disable=C0413
+
+# Command line classes
+
+
+REST_DESC = """
+Produce output in ReST format.
+
+The output is done on two sections:
+
+- Symbols: show all parsed symbols in alphabetic order;
+- Files: cross reference the content of each file with the symbols on it.
+"""
+
+class AbiRest:
+ """Initialize an argparse subparser for rest output"""
+
+ def __init__(self, subparsers):
+ """Initialize argparse subparsers"""
+
+ parser = subparsers.add_parser("rest",
+ formatter_class=argparse.RawTextHelpFormatter,
+ description=REST_DESC)
+
+ parser.add_argument("--enable-lineno", action="store_true",
+ help="enable lineno")
+ parser.add_argument("--raw", action="store_true",
+ help="output text as contained in the ABI files. "
+ "It not used, output will contain dynamically"
+ " generated cross references when possible.")
+ parser.add_argument("--no-file", action="store_true",
+ help="Don't the files section")
+ parser.add_argument("--show-hints", help="Show-hints")
+
+ parser.set_defaults(func=self.run)
+
+ def run(self, args):
+ """Run subparser"""
+
+ parser = AbiParser(args.dir, debug=args.debug)
+ parser.parse_abi()
+ parser.check_issues()
+ parser.print_data(args.enable_lineno, args.raw, not args.no_file)
+
+
+class AbiValidate:
+ """Initialize an argparse subparser for ABI validation"""
+
+ def __init__(self, subparsers):
+ """Initialize argparse subparsers"""
+
+ parser = subparsers.add_parser("validate",
+ formatter_class=argparse.ArgumentDefaultsHelpFormatter,
+ description="list events")
+
+ parser.set_defaults(func=self.run)
+
+ def run(self, args):
+ """Run subparser"""
+
+ parser = AbiParser(args.dir, debug=args.debug)
+ parser.parse_abi()
+ parser.check_issues()
+
+
+def main():
+ """Main program"""
+
+ parser = argparse.ArgumentParser(formatter_class=argparse.RawTextHelpFormatter)
+
+ parser.add_argument("-d", "--debug", type=int, default=0, help="debug level")
+ parser.add_argument("-D", "--dir", default=ABI_DIR, help=DEBUG_HELP)
+
+ subparsers = parser.add_subparsers()
+
+ AbiRest(subparsers)
+ AbiValidate(subparsers)
+
+ args = parser.parse_args()
+
+ if args.debug:
+ level = logging.DEBUG
+ else:
+ level = logging.INFO
+
+ logging.basicConfig(level=level, format="[%(levelname)s] %(message)s")
+
+ if "func" in args:
+ args.func(args)
+ else:
+ sys.exit(f"Please specify a valid command for {sys.argv[0]}")
+
+
+# Call main method
+if __name__ == "__main__":
+ main()
diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py
new file mode 100644
index 000000000000..b3fa70eee412
--- /dev/null
+++ b/scripts/lib/abi/abi_parser.py
@@ -0,0 +1,512 @@
+#!/usr/bin/env python3
+# pylint: disable=R0902,R0903,R0911,R0912,R0913,R0914,R0915,R0917,C0302
+# Copyright(c) 2025: Mauro Carvalho Chehab <mchehab@kernel.org>.
+# SPDX-License-Identifier: GPL-2.0
+
+"""
+Parse ABI documentation and produce results from it.
+"""
+
+from argparse import Namespace
+import logging
+import os
+import re
+
+from glob import glob
+from pprint import pformat
+from random import randrange, seed
+
+# Import Python modules
+
+from helpers import AbiDebug, ABI_DIR
+
+
+class AbiParser:
+ """Main class to parse ABI files"""
+
+ TAGS = r"(what|where|date|kernelversion|contact|description|users)"
+ XREF = r"(?:^|\s|\()(\/(?:sys|config|proc|dev|kvd)\/[^,.:;\)\s]+)(?:[,.:;\)\s]|\Z)"
+
+ def __init__(self, directory, logger=None,
+ enable_lineno=False, show_warnings=True, debug=0):
+ """Stores arguments for the class and initialize class vars"""
+
+ self.directory = directory
+ self.enable_lineno = enable_lineno
+ self.show_warnings = show_warnings
+ self.debug = debug
+
+ if not logger:
+ self.log = logging.getLogger("get_abi")
+ else:
+ self.log = logger
+
+ self.data = {}
+ self.what_symbols = {}
+ self.file_refs = {}
+ self.what_refs = {}
+
+ # Regular expressions used on parser
+ self.re_tag = re.compile(r"(\S+)(:\s*)(.*)", re.I)
+ self.re_valid = re.compile(self.TAGS)
+ self.re_start_spc = re.compile(r"(\s*)(\S.*)")
+ self.re_whitespace = re.compile(r"^\s+")
+
+ # Regular used on print
+ self.re_what = re.compile(r"(\/?(?:[\w\-]+\/?){1,2})")
+ self.re_escape = re.compile(r"([\.\x01-\x08\x0e-\x1f\x21-\x2f\x3a-\x40\x7b-\xff])")
+ self.re_unprintable = re.compile(r"([\x00-\x2f\x3a-\x40\x5b-\x60\x7b-\xff]+)")
+ self.re_title_mark = re.compile(r"\n[\-\*\=\^\~]+\n")
+ self.re_doc = re.compile(r"Documentation/(?!devicetree)(\S+)\.rst")
+ self.re_abi = re.compile(r"(Documentation/ABI/)([\w\/\-]+)")
+ self.re_xref_node = re.compile(self.XREF)
+
+ def warn(self, fdata, msg, extra=None):
+ """Displays a parse error if warning is enabled"""
+
+ if not self.show_warnings:
+ return
+
+ msg = f"{fdata.fname}:{fdata.ln}: {msg}"
+ if extra:
+ msg += "\n\t\t" + extra
+
+ self.log.warning(msg)
+
+ def add_symbol(self, what, fname, ln=None, xref=None):
+ """Create a reference table describing where each 'what' is located"""
+
+ if what not in self.what_symbols:
+ self.what_symbols[what] = {"file": {}}
+
+ if fname not in self.what_symbols[what]["file"]:
+ self.what_symbols[what]["file"][fname] = []
+
+ if ln and ln not in self.what_symbols[what]["file"][fname]:
+ self.what_symbols[what]["file"][fname].append(ln)
+
+ if xref:
+ self.what_symbols[what]["xref"] = xref
+
+ def _parse_line(self, fdata, line):
+ """Parse a single line of an ABI file"""
+
+ new_what = False
+ new_tag = False
+ content = None
+
+ match = self.re_tag.match(line)
+ if match:
+ new = match.group(1).lower()
+ sep = match.group(2)
+ content = match.group(3)
+
+ match = self.re_valid.search(new)
+ if match:
+ new_tag = match.group(1)
+ else:
+ if fdata.tag == "description":
+ # New "tag" is actually part of description.
+ # Don't consider it a tag
+ new_tag = False
+ elif fdata.tag != "":
+ self.warn(fdata, f"tag '{fdata.tag}' is invalid", line)
+
+ if new_tag:
+ # "where" is Invalid, but was a common mistake. Warn if found
+ if new_tag == "where":
+ self.warn(fdata, "tag 'Where' is invalid. Should be 'What:' instead")
+ new_tag = "what"
+
+ if new_tag == "what":
+ fdata.space = None
+
+ if content not in self.what_symbols:
+ self.add_symbol(what=content, fname=fdata.fname, ln=fdata.ln)
+
+ if fdata.tag == "what":
+ fdata.what.append(content.strip("\n"))
+ else:
+ if fdata.key:
+ if "description" not in self.data.get(fdata.key, {}):
+ self.warn(fdata, f"{fdata.key} doesn't have a description")
+
+ for w in fdata.what:
+ self.add_symbol(what=w, fname=fdata.fname,
+ ln=fdata.what_ln, xref=fdata.key)
+
+ fdata.label = content
+ new_what = True
+
+ key = "abi_" + content.lower()
+ fdata.key = self.re_unprintable.sub("_", key).strip("_")
+
+ # Avoid duplicated keys but using a defined seed, to make
+ # the namespace identical if there aren't changes at the
+ # ABI symbols
+ seed(42)
+
+ while fdata.key in self.data:
+ char = randrange(0, 51) + ord("A")
+ if char > ord("Z"):
+ char += ord("a") - ord("Z") - 1
+
+ fdata.key += chr(char)
+
+ if fdata.key and fdata.key not in self.data:
+ self.data[fdata.key] = {
+ "what": [content],
+ "file": [fdata.file_ref],
+ "line_no": fdata.ln,
+ }
+
+ fdata.what = self.data[fdata.key]["what"]
+
+ self.what_refs[content] = fdata.key
+ fdata.tag = new_tag
+ fdata.what_ln = fdata.ln
+
+ if fdata.nametag["what"]:
+ t = (content, fdata.key)
+ if t not in fdata.nametag["symbols"]:
+ fdata.nametag["symbols"].append(t)
+
+ return
+
+ if fdata.tag and new_tag:
+ fdata.tag = new_tag
+
+ if new_what:
+ fdata.label = ""
+
+ self.data[fdata.key]["type"] = fdata.ftype
+
+ if "description" in self.data[fdata.key]:
+ self.data[fdata.key]["description"] += "\n\n"
+
+ if fdata.file_ref not in self.data[fdata.key]["file"]:
+ self.data[fdata.key]["file"].append(fdata.file_ref)
+
+ if self.debug == AbiDebug.WHAT_PARSING:
+ self.log.debug("what: %s", fdata.what)
+
+ if not fdata.what:
+ self.warn(fdata, "'What:' should come first:", line)
+ return
+
+ if new_tag == "description":
+ fdata.space = None
+
+ if content:
+ sep = sep.replace(":", " ")
+
+ c = " " * len(new_tag) + sep + content
+ c = c.expandtabs()
+
+ match = self.re_start_spc.match(c)
+ if match:
+ # Preserve initial spaces for the first line
+ fdata.space = match.group(1)
+ content = match.group(2) + "\n"
+
+ self.data[fdata.key][fdata.tag] = content
+
+ return
+
+ # Store any contents before tags at the database
+ if not fdata.tag and "what" in fdata.nametag:
+ fdata.nametag["description"] += line
+ return
+
+ if fdata.tag == "description":
+ content = line.expandtabs()
+
+ if self.re_whitespace.sub("", content) == "":
+ self.data[fdata.key][fdata.tag] += "\n"
+ return
+
+ if fdata.space is None:
+ match = self.re_start_spc.match(content)
+ if match:
+ # Preserve initial spaces for the first line
+ fdata.space = match.group(1)
+
+ content = match.group(2) + "\n"
+ else:
+ if content.startswith(fdata.space):
+ content = content[len(fdata.space):]
+
+ else:
+ fdata.space = ""
+
+ if fdata.tag == "what":
+ w = content.strip("\n")
+ if w:
+ self.data[fdata.key][fdata.tag].append(w)
+ else:
+ self.data[fdata.key][fdata.tag] += content
+ return
+
+ content = line.strip()
+ if fdata.tag:
+ if fdata.tag == "what":
+ w = content.strip("\n")
+ if w:
+ self.data[fdata.key][fdata.tag].append(w)
+ else:
+ self.data[fdata.key][fdata.tag] += "\n" + content.rstrip("\n")
+ return
+
+ # Everything else is error
+ if content:
+ self.warn(fdata, "Unexpected content", line)
+
+ def parse_file(self, fname, path, basename):
+ """Parse a single file"""
+
+ ref = f"abi_file_{path}_{basename}"
+ ref = self.re_unprintable.sub("_", ref).strip("_")
+
+ # Store per-file state into a namespace variable. This will be used
+ # by the per-line parser state machine and by the warning function.
+ fdata = Namespace
+
+ fdata.fname = fname
+ fdata.name = basename
+
+ pos = fname.find(ABI_DIR)
+ if pos > 0:
+ f = fname[pos:]
+ else:
+ f = fname
+
+ fdata.file_ref = (f, ref)
+ self.file_refs[f] = ref
+
+ fdata.ln = 0
+ fdata.what_ln = 0
+ fdata.tag = ""
+ fdata.label = ""
+ fdata.what = []
+ fdata.key = None
+ fdata.xrefs = None
+ fdata.space = None
+ fdata.ftype = path.split("/")[0]
+
+ fdata.nametag = {}
+ fdata.nametag["what"] = [f"File {path}/{basename}"]
+ fdata.nametag["type"] = "File"
+ fdata.nametag["file"] = [fdata.file_ref]
+ fdata.nametag["line_no"] = 1
+ fdata.nametag["description"] = ""
+ fdata.nametag["symbols"] = []
+
+ self.data[ref] = fdata.nametag
+
+ if self.debug & AbiDebug.WHAT_OPEN:
+ self.log.debug("Opening file %s", fname)
+
+ with open(fname, "r", encoding="utf8", errors="backslashreplace") as fp:
+ for line in fp:
+ fdata.ln += 1
+
+ self._parse_line(fdata, line)
+
+ if "description" in fdata.nametag:
+ fdata.nametag["description"] = fdata.nametag["description"].lstrip("\n")
+
+ if fdata.key:
+ if "description" not in self.data.get(fdata.key, {}):
+ self.warn(fdata, f"{fdata.key} doesn't have a description")
+
+ for w in fdata.what:
+ self.add_symbol(what=w, fname=fname, xref=fdata.key)
+
+ def parse_abi(self):
+ """Parse documentation ABI"""
+
+ ignore_suffixes = ("rej", "org", "orig", "bak", "~")
+ re_abi = re.compile(r".*" + ABI_DIR)
+
+ for fname in glob(os.path.join(self.directory, "**"), recursive=True):
+ if os.path.isdir(fname):
+ continue
+
+ basename = os.path.basename(fname)
+
+ if basename == "README":
+ continue
+ if basename.startswith(".") or basename.endswith(ignore_suffixes):
+ continue
+
+ path = re_abi.sub("", os.path.dirname(fname))
+
+ self.parse_file(fname, path, basename)
+
+ if self.debug & AbiDebug.DUMP_ABI_STRUCTS:
+ self.log.debug(pformat(self.data))
+
+ def print_desc_txt(self, desc):
+ """Print description as found inside ABI files"""
+
+ desc = desc.strip(" \t\n")
+
+ print(desc + "\n")
+
+ def print_desc_rst(self, desc):
+ """Enrich ReST output by creating cross-references"""
+
+ # Remove title markups from the description
+ # Having titles inside ABI files will only work if extra
+ # care would be taken in order to strictly follow the same
+ # level order for each markup.
+ desc = self.re_title_mark.sub("\n\n", "\n" + desc)
+ desc = desc.rstrip(" \t\n").lstrip("\n")
+
+ # Python's regex performance for non-compiled expressions is a lot
+ # than Perl, as Perl automatically caches them at their
+ # first usage. Here, we'll need to do the same, as otherwise the
+ # performance penalty is be high
+
+ new_desc = ""
+ for d in desc.split("\n"):
+ if d == "":
+ new_desc += "\n"
+ continue
+
+ # Use cross-references for doc files where needed
+ d = self.re_doc.sub(r":doc:`/\1`", d)
+
+ # Use cross-references for ABI generated docs where needed
+ matches = self.re_abi.findall(d)
+ for m in matches:
+ abi = m[0] + m[1]
+
+ xref = self.file_refs.get(abi)
+ if not xref:
+ # This may happen if ABI is on a separate directory,
+ # like parsing ABI testing and symbol is at stable.
+ # The proper solution is to move this part of the code
+ # for it to be inside sphinx/kernel_abi.py
+ self.log.info("Didn't find ABI reference for '%s'", abi)
+ else:
+ new = self.re_escape.sub(r"\\\1", m[1])
+ d = re.sub(fr"\b{abi}\b", f":ref:`{new} <{xref}>`", d)
+
+ # Seek for cross reference symbols like /sys/...
+ # Need to be careful to avoid doing it on a code block
+ if d[0] not in [" ", "\t"]:
+ matches = self.re_xref_node.findall(d)
+ for m in matches:
+ # Finding ABI here is more complex due to wildcards
+ xref = self.what_refs.get(m)
+ if xref:
+ new = self.re_escape.sub(r"\\\1", m)
+ d = re.sub(fr"\b{m}\b", f":ref:`{new} <{xref}>`", d)
+
+ new_desc += d + "\n"
+
+ print(new_desc + "\n")
+
+ def print_data(self, enable_lineno, output_in_txt, show_file=False):
+ """Print ABI at stdout"""
+
+ part = None
+ for key, v in sorted(self.data.items(),
+ key=lambda x: (x[1].get("type", ""),
+ x[1].get("what"))):
+
+ wtype = v.get("type", "Var")
+ file_ref = v.get("file")
+ names = v.get("what", [""])
+
+ if not show_file and wtype == "File":
+ continue
+
+ if enable_lineno:
+ ln = v.get("line_no", 1)
+ print(f".. LINENO {file_ref[0][0]}#{ln}\n")
+
+ if wtype != "File":
+ cur_part = names[0]
+ if cur_part.find("/") >= 0:
+ match = self.re_what.match(cur_part)
+ if match:
+ symbol = match.group(1).rstrip("/")
+ cur_part = "Symbols under " + symbol
+
+ if cur_part and cur_part != part:
+ part = cur_part
+ print(f"{part}\n{"-" * len(part)}\n")
+
+ print(f".. _{key}:\n")
+
+ max_len = 0
+ for i in range(0, len(names)): # pylint: disable=C0200
+ names[i] = "**" + self.re_escape.sub(r"\\\1", names[i]) + "**"
+
+ max_len = max(max_len, len(names[i]))
+
+ print("+-" + "-" * max_len + "-+")
+ for name in names:
+ print(f"| {name}" + " " * (max_len - len(name)) + " |")
+ print("+-" + "-" * max_len + "-+")
+ print()
+
+ for ref in file_ref:
+ if wtype == "File":
+ print(f".. _{ref[1]}:\n")
+ else:
+ base = os.path.basename(ref[0])
+ print(f"Defined on file :ref:`{base} <{ref[1]}>`\n")
+
+ if wtype == "File":
+ print(f"{names[0]}\n{"-" * len(names[0])}\n")
+
+ desc = v.get("description")
+ if not desc and wtype != "File":
+ print(f"DESCRIPTION MISSING for {names[0]}\n")
+
+ if desc:
+ if output_in_txt:
+ self.print_desc_txt(desc)
+ else:
+ self.print_desc_rst(desc)
+
+ symbols = v.get("symbols")
+ if symbols:
+ print("Has the following ABI:\n")
+
+ for w, label in symbols:
+ # Escape special chars from content
+ content = self.re_escape.sub(r"\\\1", w)
+
+ print(f"- :ref:`{content} <{label}>`\n")
+
+ users = v.get("users")
+ if users and users.strip(" \t\n"):
+ print(f"Users:\n\t{users.strip("\n").replace('\n', '\n\t')}\n")
+
+ def check_issues(self):
+ """Warn about duplicated ABI entries"""
+
+ for what, v in self.what_symbols.items():
+ files = v.get("file")
+ if not files:
+ # Should never happen if the parser works properly
+ self.log.warning("%s doesn't have a file associated", what)
+ continue
+
+ if len(files) == 1:
+ continue
+
+ f = []
+ for fname, lines in sorted(files.items()):
+ if not lines:
+ f.append(f"{fname}")
+ elif len(lines) == 1:
+ f.append(f"{fname}:{lines[0]}")
+ else:
+ f.append(f"{fname} lines {", ".join(str(x) for x in lines)}")
+
+ self.log.warning("%s is defined %d times: %s", what, len(f), "; ".join(f))
diff --git a/scripts/lib/abi/helpers.py b/scripts/lib/abi/helpers.py
new file mode 100644
index 000000000000..84a253ed5058
--- /dev/null
+++ b/scripts/lib/abi/helpers.py
@@ -0,0 +1,28 @@
+#!/usr/bin/env python3
+# Copyright(c) 2025: Mauro Carvalho Chehab <mchehab@kernel.org>.
+# pylint: disable=R0903
+# SPDX-License-Identifier: GPL-2.0
+
+"""
+Helper classes for ABI parser
+"""
+
+ABI_DIR = "Documentation/ABI/"
+
+
+class AbiDebug:
+ """Debug levels"""
+
+ WHAT_PARSING = 1
+ WHAT_OPEN = 2
+ DUMP_ABI_STRUCTS = 4
+
+
+DEBUG_HELP = """
+Print debug information according with the level(s),
+which is given by the following bitmask:
+
+1 - enable debug parsing logic
+2 - enable debug messages on file open
+4 - enable debug for ABI parse data
+"""
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 11/27] scripts/get_abi.py: add support for symbol search
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (9 preceding siblings ...)
2025-02-10 10:17 ` [PATCH 10/27] scripts/get_abi.py: add a Python tool to generate ReST output Mauro Carvalho Chehab
@ 2025-02-10 10:18 ` Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 12/27] docs: use get_abi.py for ABI generation Mauro Carvalho Chehab
` (17 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:18 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Add support for searching symbols from Documentation/ABI using
regular expressions to match the symbols' names.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/get_abi.py | 24 ++++++++++++++++
scripts/lib/abi/abi_parser.py | 52 +++++++++++++++++++++++++++++++++++
2 files changed, 76 insertions(+)
diff --git a/scripts/get_abi.py b/scripts/get_abi.py
index bb17c54feeff..30439f21fdd0 100755
--- a/scripts/get_abi.py
+++ b/scripts/get_abi.py
@@ -85,6 +85,29 @@ class AbiValidate:
parser.check_issues()
+class AbiSearch:
+ """Initialize an argparse subparser for ABI search"""
+
+ def __init__(self, subparsers):
+ """Initialize argparse subparsers"""
+
+ parser = subparsers.add_parser("search",
+ formatter_class=argparse.ArgumentDefaultsHelpFormatter,
+ description="Search ABI using a regular expression")
+
+ parser.add_argument("expression",
+ help="Case-insensitive search pattern for the ABI symbol")
+
+ parser.set_defaults(func=self.run)
+
+ def run(self, args):
+ """Run subparser"""
+
+ parser = AbiParser(args.dir, debug=args.debug)
+ parser.parse_abi()
+ parser.search_symbols(args.expression)
+
+
def main():
"""Main program"""
@@ -97,6 +120,7 @@ def main():
AbiRest(subparsers)
AbiValidate(subparsers)
+ AbiSearch(subparsers)
args = parser.parse_args()
diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py
index b3fa70eee412..bea7f1a76165 100644
--- a/scripts/lib/abi/abi_parser.py
+++ b/scripts/lib/abi/abi_parser.py
@@ -510,3 +510,55 @@ class AbiParser:
f.append(f"{fname} lines {", ".join(str(x) for x in lines)}")
self.log.warning("%s is defined %d times: %s", what, len(f), "; ".join(f))
+
+ def search_symbols(self, expr):
+ """ Searches for ABI symbols """
+
+ regex = re.compile(expr, re.I)
+
+ found_keys = 0
+ for t in sorted(self.data.items(), key=lambda x: [0]):
+ v = t[1]
+
+ wtype = v.get("type", "")
+ if wtype == "File":
+ continue
+
+ for what in v.get("what", [""]):
+ if regex.search(what):
+ found_keys += 1
+
+ kernelversion = v.get("kernelversion", "").strip(" \t\n")
+ date = v.get("date", "").strip(" \t\n")
+ contact = v.get("contact", "").strip(" \t\n")
+ users = v.get("users", "").strip(" \t\n")
+ desc = v.get("description", "").strip(" \t\n")
+
+ files = []
+ for f in v.get("file", ()):
+ files.append(f[0])
+
+ what = str(found_keys) + ". " + what
+ title_tag = "-" * len(what)
+
+ print(f"\n{what}\n{title_tag}\n")
+
+ if kernelversion:
+ print(f"Kernel version:\t\t{kernelversion}")
+
+ if date:
+ print(f"Date:\t\t\t{date}")
+
+ if contact:
+ print(f"Contact:\t\t{contact}")
+
+ if users:
+ print(f"Users:\t\t\t{users}")
+
+ print(f"Defined on file{'s'[:len(files) ^ 1]}:\t{", ".join(files)}")
+
+ if desc:
+ print(f"\n{desc.strip("\n")}\n")
+
+ if not found_keys:
+ print(f"Regular expression /{expr}/ not found.")
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 12/27] docs: use get_abi.py for ABI generation
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (10 preceding siblings ...)
2025-02-10 10:18 ` [PATCH 11/27] scripts/get_abi.py: add support for symbol search Mauro Carvalho Chehab
@ 2025-02-10 10:18 ` Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 13/27] scripts/lib/abi/abi_parser.py: optimize parse_abi() function Mauro Carvalho Chehab
` (16 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:18 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Use the new script instead of the old one when generating ABI docs.
For now, execute it via exec. Future changes will better integrate it
by using the class defined there directly.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/admin-guide/abi-obsolete.rst | 1 -
Documentation/admin-guide/abi-removed.rst | 1 -
Documentation/admin-guide/abi-stable.rst | 1 -
Documentation/admin-guide/abi-testing.rst | 1 -
Documentation/sphinx/kernel_abi.py | 10 +++-------
5 files changed, 3 insertions(+), 11 deletions(-)
diff --git a/Documentation/admin-guide/abi-obsolete.rst b/Documentation/admin-guide/abi-obsolete.rst
index b655615917f1..6d4d9ab7b8c3 100644
--- a/Documentation/admin-guide/abi-obsolete.rst
+++ b/Documentation/admin-guide/abi-obsolete.rst
@@ -10,4 +10,3 @@ The description of the interface will document the reason why it is
obsolete and when it can be expected to be removed.
.. kernel-abi:: ABI/obsolete
- :rst:
diff --git a/Documentation/admin-guide/abi-removed.rst b/Documentation/admin-guide/abi-removed.rst
index ba941c1af178..9fc78af6f077 100644
--- a/Documentation/admin-guide/abi-removed.rst
+++ b/Documentation/admin-guide/abi-removed.rst
@@ -4,4 +4,3 @@ ABI removed symbols
===================
.. kernel-abi:: ABI/removed
- :rst:
diff --git a/Documentation/admin-guide/abi-stable.rst b/Documentation/admin-guide/abi-stable.rst
index 5d738f345333..c47c2a295865 100644
--- a/Documentation/admin-guide/abi-stable.rst
+++ b/Documentation/admin-guide/abi-stable.rst
@@ -13,4 +13,3 @@ Most interfaces (like syscalls) are expected to never change and always
be available.
.. kernel-abi:: ABI/stable
- :rst:
diff --git a/Documentation/admin-guide/abi-testing.rst b/Documentation/admin-guide/abi-testing.rst
index a867e6578bf7..40b31985e587 100644
--- a/Documentation/admin-guide/abi-testing.rst
+++ b/Documentation/admin-guide/abi-testing.rst
@@ -19,4 +19,3 @@ name to the description of these interfaces, so that the kernel
developers can easily notify them if any changes occur.
.. kernel-abi:: ABI/testing
- :rst:
diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
index 38653f5706c0..f314b888d3de 100644
--- a/Documentation/sphinx/kernel_abi.py
+++ b/Documentation/sphinx/kernel_abi.py
@@ -14,7 +14,7 @@ u"""
:license: GPL Version 2, June 1991 see Linux/COPYING for details.
The ``kernel-abi`` (:py:class:`KernelCmd`) directive calls the
- scripts/get_abi.pl script to parse the Kernel ABI files.
+ scripts/get_abi.py script to parse the Kernel ABI files.
Overview of directive's argument and options.
@@ -67,7 +67,6 @@ class KernelCmd(Directive):
option_spec = {
"debug" : directives.flag,
- "rst" : directives.unchanged
}
def run(self):
@@ -78,15 +77,12 @@ class KernelCmd(Directive):
srctree = os.path.abspath(os.environ["srctree"])
args = [
- os.path.join(srctree, 'scripts/get_abi.pl'),
+ os.path.join(srctree, 'scripts/get_abi.py'),
+ '-D', os.path.join(srctree, 'Documentation', self.arguments[0]),
'rest',
'--enable-lineno',
- '--dir', os.path.join(srctree, 'Documentation', self.arguments[0]),
]
- if 'rst' in self.options:
- args.append('--rst-source')
-
lines = subprocess.check_output(args, cwd=os.path.dirname(doc.current_source)).decode('utf-8')
nodeList = self.nestedParse(lines, self.arguments[0])
return nodeList
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 13/27] scripts/lib/abi/abi_parser.py: optimize parse_abi() function
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (11 preceding siblings ...)
2025-02-10 10:18 ` [PATCH 12/27] docs: use get_abi.py for ABI generation Mauro Carvalho Chehab
@ 2025-02-10 10:18 ` Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 14/27] scripts/lib/abi/abi_parser.py: use an interactor for ReST output Mauro Carvalho Chehab
` (15 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:18 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Instead of using glob, use a recursive function to parse all files.
Such change reduces the total excecution time by 15% with my SSD disks.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/lib/abi/abi_parser.py | 57 +++++++++++++++++++++++------------
1 file changed, 38 insertions(+), 19 deletions(-)
diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py
index bea7f1a76165..6052a8aec443 100644
--- a/scripts/lib/abi/abi_parser.py
+++ b/scripts/lib/abi/abi_parser.py
@@ -12,7 +12,6 @@ import logging
import os
import re
-from glob import glob
from pprint import pformat
from random import randrange, seed
@@ -46,7 +45,11 @@ class AbiParser:
self.file_refs = {}
self.what_refs = {}
+ # Ignore files that contain such suffixes
+ self.ignore_suffixes = (".rej", ".org", ".orig", ".bak", "~")
+
# Regular expressions used on parser
+ self.re_abi_dir = re.compile(r"(.*)" + ABI_DIR)
self.re_tag = re.compile(r"(\S+)(:\s*)(.*)", re.I)
self.re_valid = re.compile(self.TAGS)
self.re_start_spc = re.compile(r"(\s*)(\S.*)")
@@ -322,26 +325,42 @@ class AbiParser:
for w in fdata.what:
self.add_symbol(what=w, fname=fname, xref=fdata.key)
- def parse_abi(self):
+ def _parse_abi(self, root=None):
+ """Internal function to parse documentation ABI recursively"""
+
+ if not root:
+ root = self.directory
+
+ with os.scandir(root) as obj:
+ for entry in obj:
+ name = os.path.join(root, entry.name)
+
+ if entry.is_dir():
+ self.parse_abi(name)
+ continue
+
+ if not entry.is_file():
+ continue
+
+ basename = os.path.basename(name)
+
+ if basename == "README":
+ continue
+
+ if basename.startswith("."):
+ continue
+
+ if basename.endswith(self.ignore_suffixes):
+ continue
+
+ path = self.re_abi_dir.sub("", os.path.dirname(name))
+
+ self.parse_file(name, path, basename)
+
+ def parse_abi(self, root=None):
"""Parse documentation ABI"""
- ignore_suffixes = ("rej", "org", "orig", "bak", "~")
- re_abi = re.compile(r".*" + ABI_DIR)
-
- for fname in glob(os.path.join(self.directory, "**"), recursive=True):
- if os.path.isdir(fname):
- continue
-
- basename = os.path.basename(fname)
-
- if basename == "README":
- continue
- if basename.startswith(".") or basename.endswith(ignore_suffixes):
- continue
-
- path = re_abi.sub("", os.path.dirname(fname))
-
- self.parse_file(fname, path, basename)
+ self._parse_abi(root)
if self.debug & AbiDebug.DUMP_ABI_STRUCTS:
self.log.debug(pformat(self.data))
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 14/27] scripts/lib/abi/abi_parser.py: use an interactor for ReST output
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (12 preceding siblings ...)
2025-02-10 10:18 ` [PATCH 13/27] scripts/lib/abi/abi_parser.py: optimize parse_abi() function Mauro Carvalho Chehab
@ 2025-02-10 10:18 ` Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 15/27] docs: sphinx/kernel_abi: use AbiParser directly Mauro Carvalho Chehab
` (14 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:18 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Instead of printing all results line per line, use an interactor
to return each variable as a separate message.
This won't change much when using it via command line, but it
will help Sphinx integration by providing an interactor that
could be used there to handle ABI symbol by symbol.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/get_abi.py | 3 ++-
scripts/lib/abi/abi_parser.py | 48 +++++++++++++++++++----------------
2 files changed, 28 insertions(+), 23 deletions(-)
diff --git a/scripts/get_abi.py b/scripts/get_abi.py
index 30439f21fdd0..93b973bc07ed 100755
--- a/scripts/get_abi.py
+++ b/scripts/get_abi.py
@@ -62,8 +62,9 @@ class AbiRest:
parser = AbiParser(args.dir, debug=args.debug)
parser.parse_abi()
parser.check_issues()
- parser.print_data(args.enable_lineno, args.raw, not args.no_file)
+ for msg in parser.doc(args.enable_lineno, args.raw, not args.no_file):
+ print(msg)
class AbiValidate:
"""Initialize an argparse subparser for ABI validation"""
diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py
index 6052a8aec443..960e27161c26 100644
--- a/scripts/lib/abi/abi_parser.py
+++ b/scripts/lib/abi/abi_parser.py
@@ -336,7 +336,7 @@ class AbiParser:
name = os.path.join(root, entry.name)
if entry.is_dir():
- self.parse_abi(name)
+ self._parse_abi(name)
continue
if not entry.is_file():
@@ -365,14 +365,14 @@ class AbiParser:
if self.debug & AbiDebug.DUMP_ABI_STRUCTS:
self.log.debug(pformat(self.data))
- def print_desc_txt(self, desc):
+ def desc_txt(self, desc):
"""Print description as found inside ABI files"""
desc = desc.strip(" \t\n")
- print(desc + "\n")
+ return desc + "\n\n"
- def print_desc_rst(self, desc):
+ def desc_rst(self, desc):
"""Enrich ReST output by creating cross-references"""
# Remove title markups from the description
@@ -425,9 +425,9 @@ class AbiParser:
new_desc += d + "\n"
- print(new_desc + "\n")
+ return new_desc + "\n\n"
- def print_data(self, enable_lineno, output_in_txt, show_file=False):
+ def doc(self, enable_lineno, output_in_txt, show_file=False):
"""Print ABI at stdout"""
part = None
@@ -442,9 +442,11 @@ class AbiParser:
if not show_file and wtype == "File":
continue
+ msg = ""
+
if enable_lineno:
ln = v.get("line_no", 1)
- print(f".. LINENO {file_ref[0][0]}#{ln}\n")
+ msg += f".. LINENO {file_ref[0][0]}#{ln}\n\n"
if wtype != "File":
cur_part = names[0]
@@ -456,9 +458,9 @@ class AbiParser:
if cur_part and cur_part != part:
part = cur_part
- print(f"{part}\n{"-" * len(part)}\n")
+ msg += f"{part}\n{"-" * len(part)}\n\n"
- print(f".. _{key}:\n")
+ msg += f".. _{key}:\n\n"
max_len = 0
for i in range(0, len(names)): # pylint: disable=C0200
@@ -466,45 +468,47 @@ class AbiParser:
max_len = max(max_len, len(names[i]))
- print("+-" + "-" * max_len + "-+")
+ msg += "+-" + "-" * max_len + "-+\n"
for name in names:
- print(f"| {name}" + " " * (max_len - len(name)) + " |")
- print("+-" + "-" * max_len + "-+")
- print()
+ msg += f"| {name}" + " " * (max_len - len(name)) + " |\n"
+ msg += "+-" + "-" * max_len + "-+\n"
+ msg += "\n"
for ref in file_ref:
if wtype == "File":
- print(f".. _{ref[1]}:\n")
+ msg += f".. _{ref[1]}:\n\n"
else:
base = os.path.basename(ref[0])
- print(f"Defined on file :ref:`{base} <{ref[1]}>`\n")
+ msg += f"Defined on file :ref:`{base} <{ref[1]}>`\n\n"
if wtype == "File":
- print(f"{names[0]}\n{"-" * len(names[0])}\n")
+ msg += f"{names[0]}\n{"-" * len(names[0])}\n\n"
desc = v.get("description")
if not desc and wtype != "File":
- print(f"DESCRIPTION MISSING for {names[0]}\n")
+ msg += f"DESCRIPTION MISSING for {names[0]}\n\n"
if desc:
if output_in_txt:
- self.print_desc_txt(desc)
+ msg += self.desc_txt(desc)
else:
- self.print_desc_rst(desc)
+ msg += self.desc_rst(desc)
symbols = v.get("symbols")
if symbols:
- print("Has the following ABI:\n")
+ msg += "Has the following ABI:\n\n"
for w, label in symbols:
# Escape special chars from content
content = self.re_escape.sub(r"\\\1", w)
- print(f"- :ref:`{content} <{label}>`\n")
+ msg += f"- :ref:`{content} <{label}>`\n\n"
users = v.get("users")
if users and users.strip(" \t\n"):
- print(f"Users:\n\t{users.strip("\n").replace('\n', '\n\t')}\n")
+ msg += f"Users:\n\t{users.strip("\n").replace('\n', '\n\t')}\n\n"
+
+ yield msg
def check_issues(self):
"""Warn about duplicated ABI entries"""
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 15/27] docs: sphinx/kernel_abi: use AbiParser directly
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (13 preceding siblings ...)
2025-02-10 10:18 ` [PATCH 14/27] scripts/lib/abi/abi_parser.py: use an interactor for ReST output Mauro Carvalho Chehab
@ 2025-02-10 10:18 ` Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 16/27] docs: sphinx/kernel_abi: reduce buffer usage for ABI messages Mauro Carvalho Chehab
` (13 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:18 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Instead of running get_abi.py script, import AbiParser class and
handle messages directly there using an interactor. This shold save some
memory, as there's no need to exec python inside the Sphinx python
extension.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/sphinx/kernel_abi.py | 29 ++++++++++++++++-------------
scripts/lib/abi/abi_parser.py | 2 +-
2 files changed, 17 insertions(+), 14 deletions(-)
diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
index f314b888d3de..f7b22abebcf4 100644
--- a/Documentation/sphinx/kernel_abi.py
+++ b/Documentation/sphinx/kernel_abi.py
@@ -34,7 +34,6 @@ u"""
import os
import re
-import subprocess
import sys
from docutils import nodes
@@ -43,6 +42,11 @@ from docutils.parsers.rst import directives, Directive
from sphinx.util.docutils import switch_source_input
from sphinx.util import logging
+srctree = os.path.abspath(os.environ["srctree"])
+sys.path.insert(0, os.path.join(srctree, "scripts/lib/abi"))
+
+from abi_parser import AbiParser
+
__version__ = "1.0"
@@ -66,7 +70,7 @@ class KernelCmd(Directive):
logger = logging.getLogger('kernel_abi')
option_spec = {
- "debug" : directives.flag,
+ "debug": directives.flag,
}
def run(self):
@@ -74,20 +78,19 @@ class KernelCmd(Directive):
if not doc.settings.file_insertion_enabled:
raise self.warning("docutils: file insertion disabled")
- srctree = os.path.abspath(os.environ["srctree"])
+ path = os.path.join(srctree, "Documentation", self.arguments[0])
+ parser = AbiParser(path, logger=self.logger)
+ parser.parse_abi()
+ parser.check_issues()
- args = [
- os.path.join(srctree, 'scripts/get_abi.py'),
- '-D', os.path.join(srctree, 'Documentation', self.arguments[0]),
- 'rest',
- '--enable-lineno',
- ]
+ msg = ""
+ for m in parser.doc(enable_lineno=True, show_file=True):
+ msg += m
- lines = subprocess.check_output(args, cwd=os.path.dirname(doc.current_source)).decode('utf-8')
- nodeList = self.nestedParse(lines, self.arguments[0])
- return nodeList
+ node = self.nested_parse(msg, self.arguments[0])
+ return node
- def nestedParse(self, lines, fname):
+ def nested_parse(self, lines, fname):
env = self.state.document.settings.env
content = ViewList()
node = nodes.section()
diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py
index 960e27161c26..57c125fd40a5 100644
--- a/scripts/lib/abi/abi_parser.py
+++ b/scripts/lib/abi/abi_parser.py
@@ -427,7 +427,7 @@ class AbiParser:
return new_desc + "\n\n"
- def doc(self, enable_lineno, output_in_txt, show_file=False):
+ def doc(self, enable_lineno, output_in_txt=False, show_file=False):
"""Print ABI at stdout"""
part = None
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 16/27] docs: sphinx/kernel_abi: reduce buffer usage for ABI messages
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (14 preceding siblings ...)
2025-02-10 10:18 ` [PATCH 15/27] docs: sphinx/kernel_abi: use AbiParser directly Mauro Carvalho Chehab
@ 2025-02-10 10:18 ` Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 17/27] docs: sphinx/kernel_abi: properly split lines Mauro Carvalho Chehab
` (12 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:18 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Instead of producing a big message with all ABI contents and then
parse as a whole, simplify the code by handling each ABI symbol
in separate. As an additional benefit, there's no need to place
file/line nubers inlined at the data and use a regex to convert
them.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/sphinx/kernel_abi.py | 76 +++++++++++++++---------------
scripts/get_abi.py | 7 ++-
scripts/lib/abi/abi_parser.py | 10 ++--
3 files changed, 48 insertions(+), 45 deletions(-)
diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
index f7b22abebcf4..742ebd35454f 100644
--- a/Documentation/sphinx/kernel_abi.py
+++ b/Documentation/sphinx/kernel_abi.py
@@ -68,6 +68,7 @@ class KernelCmd(Directive):
has_content = False
final_argument_whitespace = True
logger = logging.getLogger('kernel_abi')
+ parser = None
option_spec = {
"debug": directives.flag,
@@ -79,59 +80,60 @@ class KernelCmd(Directive):
raise self.warning("docutils: file insertion disabled")
path = os.path.join(srctree, "Documentation", self.arguments[0])
- parser = AbiParser(path, logger=self.logger)
- parser.parse_abi()
- parser.check_issues()
+ self.parser = AbiParser(path, logger=self.logger)
+ self.parser.parse_abi()
+ self.parser.check_issues()
- msg = ""
- for m in parser.doc(enable_lineno=True, show_file=True):
- msg += m
-
- node = self.nested_parse(msg, self.arguments[0])
+ node = self.nested_parse(None, self.arguments[0])
return node
- def nested_parse(self, lines, fname):
+ def nested_parse(self, data, fname):
env = self.state.document.settings.env
content = ViewList()
node = nodes.section()
- if "debug" in self.options:
- code_block = "\n\n.. code-block:: rst\n :linenos:\n"
- for line in lines.split("\n"):
- code_block += "\n " + line
- lines = code_block + "\n\n"
+ if data is not None:
+ # Handles the .rst file
+ for line in data.split("\n"):
+ content.append(line, fname, 0)
- line_regex = re.compile(r"^\.\. LINENO (\S+)\#([0-9]+)$")
- ln = 0
- n = 0
- f = fname
+ self.do_parse(content, node)
- for line in lines.split("\n"):
- n = n + 1
- match = line_regex.search(line)
- if match:
- new_f = match.group(1)
+ else:
+ # Handles the ABI parser content, symbol by symbol
- # Sphinx parser is lazy: it stops parsing contents in the
- # middle, if it is too big. So, handle it per input file
- if new_f != f and content:
- self.do_parse(content, node)
- content = ViewList()
+ old_f = fname
+ n = 0
+ for msg, f, ln in self.parser.doc():
+ msg_list = msg.split("\n")
+ if "debug" in self.options:
+ lines = [
+ "", "", ".. code-block:: rst",
+ " :linenos:", ""
+ ]
+ for m in msg_list:
+ lines.append(" " + m)
+ else:
+ lines = msg_list
+ for line in lines:
+ # sphinx counts lines from 0
+ content.append(line, f, ln - 1)
+ n += 1
+
+ if f != old_f:
# Add the file to Sphinx build dependencies
env.note_dependency(os.path.abspath(f))
- f = new_f
+ old_f = f
- # sphinx counts lines from 0
- ln = int(match.group(2)) - 1
- else:
- content.append(line, f, ln)
+ # Sphinx doesn't like to parse big messages. So, let's
+ # add content symbol by symbol
+ if content:
+ self.do_parse(content, node)
+ content = ViewList()
- self.logger.info("%s: parsed %i lines" % (fname, n))
-
- if content:
- self.do_parse(content, node)
+ self.logger.info("%s: parsed %i lines" % (fname, n))
return node.children
diff --git a/scripts/get_abi.py b/scripts/get_abi.py
index 93b973bc07ed..19f78d6aa407 100755
--- a/scripts/get_abi.py
+++ b/scripts/get_abi.py
@@ -63,8 +63,11 @@ class AbiRest:
parser.parse_abi()
parser.check_issues()
- for msg in parser.doc(args.enable_lineno, args.raw, not args.no_file):
- print(msg)
+ for t in parser.doc(args.raw, not args.no_file):
+ if args.enable_lineno:
+ print (f".. LINENO {t[1]}#{t[2]}\n\n")
+
+ print(t[0])
class AbiValidate:
"""Initialize an argparse subparser for ABI validation"""
diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py
index 57c125fd40a5..1db6c54fc65a 100644
--- a/scripts/lib/abi/abi_parser.py
+++ b/scripts/lib/abi/abi_parser.py
@@ -427,7 +427,7 @@ class AbiParser:
return new_desc + "\n\n"
- def doc(self, enable_lineno, output_in_txt=False, show_file=False):
+ def doc(self, output_in_txt=False, show_file=True):
"""Print ABI at stdout"""
part = None
@@ -444,10 +444,6 @@ class AbiParser:
msg = ""
- if enable_lineno:
- ln = v.get("line_no", 1)
- msg += f".. LINENO {file_ref[0][0]}#{ln}\n\n"
-
if wtype != "File":
cur_part = names[0]
if cur_part.find("/") >= 0:
@@ -508,7 +504,9 @@ class AbiParser:
if users and users.strip(" \t\n"):
msg += f"Users:\n\t{users.strip("\n").replace('\n', '\n\t')}\n\n"
- yield msg
+ ln = v.get("line_no", 1)
+
+ yield (msg, file_ref[0][0], ln)
def check_issues(self):
"""Warn about duplicated ABI entries"""
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 17/27] docs: sphinx/kernel_abi: properly split lines
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (15 preceding siblings ...)
2025-02-10 10:18 ` [PATCH 16/27] docs: sphinx/kernel_abi: reduce buffer usage for ABI messages Mauro Carvalho Chehab
@ 2025-02-10 10:18 ` Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 18/27] scripts/get_abi.pl: Add filtering capabilities to rest output Mauro Carvalho Chehab
` (11 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:18 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Sphinx doesn't like to have lines split with str.split("\n").
Instead, it uses its own splitter, with handles line breaks the way
Spinx expects. Not using it cause issues at the output files.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/sphinx/kernel_abi.py | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
index 742ebd35454f..0a4057183208 100644
--- a/Documentation/sphinx/kernel_abi.py
+++ b/Documentation/sphinx/kernel_abi.py
@@ -36,7 +36,7 @@ import os
import re
import sys
-from docutils import nodes
+from docutils import nodes, statemachine
from docutils.statemachine import ViewList
from docutils.parsers.rst import directives, Directive
from sphinx.util.docutils import switch_source_input
@@ -105,7 +105,8 @@ class KernelCmd(Directive):
old_f = fname
n = 0
for msg, f, ln in self.parser.doc():
- msg_list = msg.split("\n")
+ msg_list = statemachine.string2lines(msg, tab_width,
+ convert_whitespace=True)
if "debug" in self.options:
lines = [
"", "", ".. code-block:: rst",
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 18/27] scripts/get_abi.pl: Add filtering capabilities to rest output
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (16 preceding siblings ...)
2025-02-10 10:18 ` [PATCH 17/27] docs: sphinx/kernel_abi: properly split lines Mauro Carvalho Chehab
@ 2025-02-10 10:18 ` Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 19/27] scripts/get_abi.pl: add support to parse ABI README file Mauro Carvalho Chehab
` (10 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:18 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
This way, Sphinx ABI extension can parse symbols only once, while
keep displaying results in separate files.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/lib/abi/abi_parser.py | 21 +++++++++++++++------
1 file changed, 15 insertions(+), 6 deletions(-)
diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py
index 1db6c54fc65a..b20d5c9d920e 100644
--- a/scripts/lib/abi/abi_parser.py
+++ b/scripts/lib/abi/abi_parser.py
@@ -160,6 +160,7 @@ class AbiParser:
self.data[fdata.key] = {
"what": [content],
"file": [fdata.file_ref],
+ "path": fdata.ftype,
"line_no": fdata.ln,
}
@@ -182,8 +183,6 @@ class AbiParser:
if new_what:
fdata.label = ""
- self.data[fdata.key]["type"] = fdata.ftype
-
if "description" in self.data[fdata.key]:
self.data[fdata.key]["description"] += "\n\n"
@@ -299,6 +298,7 @@ class AbiParser:
fdata.nametag = {}
fdata.nametag["what"] = [f"File {path}/{basename}"]
fdata.nametag["type"] = "File"
+ fdata.nametag["path"] = fdata.ftype
fdata.nametag["file"] = [fdata.file_ref]
fdata.nametag["line_no"] = 1
fdata.nametag["description"] = ""
@@ -427,7 +427,8 @@ class AbiParser:
return new_desc + "\n\n"
- def doc(self, output_in_txt=False, show_file=True):
+ def doc(self, output_in_txt=False, show_symbols=True, show_file=True,
+ filter_path=None):
"""Print ABI at stdout"""
part = None
@@ -435,12 +436,20 @@ class AbiParser:
key=lambda x: (x[1].get("type", ""),
x[1].get("what"))):
- wtype = v.get("type", "Var")
+ wtype = v.get("type", "Symbol")
file_ref = v.get("file")
names = v.get("what", [""])
- if not show_file and wtype == "File":
- continue
+ if wtype == "File":
+ if not show_file:
+ continue
+ else:
+ if not show_symbols:
+ continue
+
+ if filter_path:
+ if v.get("path") != filter_path:
+ continue
msg = ""
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 19/27] scripts/get_abi.pl: add support to parse ABI README file
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (17 preceding siblings ...)
2025-02-10 10:18 ` [PATCH 18/27] scripts/get_abi.pl: Add filtering capabilities to rest output Mauro Carvalho Chehab
@ 2025-02-10 10:18 ` Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 20/27] docs: sphinx/kernel_abi: parse ABI files only once Mauro Carvalho Chehab
` (9 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:18 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
The Documentation/ABI/README file is currently outside the
documentation tree. Yet, it may still provide some useful
information. Add it to the documentation parsing.
As a plus, this avoids a warning when detecting missing
cross-references.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/lib/abi/abi_parser.py | 25 ++++++++++++++++++++-----
1 file changed, 20 insertions(+), 5 deletions(-)
diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py
index b20d5c9d920e..6fac461d794c 100644
--- a/scripts/lib/abi/abi_parser.py
+++ b/scripts/lib/abi/abi_parser.py
@@ -263,6 +263,16 @@ class AbiParser:
if content:
self.warn(fdata, "Unexpected content", line)
+ def parse_readme(self, nametag, fname):
+ """Parse ABI README file"""
+
+ with open(fname, "r", encoding="utf8", errors="backslashreplace") as fp:
+ nametag["description"] = "```\n"
+ for line in fp:
+ nametag["description"] += " " + line
+
+ nametag["description"] += "```\n"
+
def parse_file(self, fname, path, basename):
"""Parse a single file"""
@@ -309,6 +319,10 @@ class AbiParser:
if self.debug & AbiDebug.WHAT_OPEN:
self.log.debug("Opening file %s", fname)
+ if basename == "README":
+ self.parse_readme(fdata.nametag, fname)
+ return
+
with open(fname, "r", encoding="utf8", errors="backslashreplace") as fp:
for line in fp:
fdata.ln += 1
@@ -344,9 +358,6 @@ class AbiParser:
basename = os.path.basename(name)
- if basename == "README":
- continue
-
if basename.startswith("."):
continue
@@ -448,8 +459,12 @@ class AbiParser:
continue
if filter_path:
- if v.get("path") != filter_path:
- continue
+ if filter_path == "README":
+ if not names[0].endswith("README"):
+ continue
+ else:
+ if v.get("path") != filter_path:
+ continue
msg = ""
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 20/27] docs: sphinx/kernel_abi: parse ABI files only once
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (18 preceding siblings ...)
2025-02-10 10:18 ` [PATCH 19/27] scripts/get_abi.pl: add support to parse ABI README file Mauro Carvalho Chehab
@ 2025-02-10 10:18 ` Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 21/27] docs: admin-guide/abi: split files from symbols Mauro Carvalho Chehab
` (8 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:18 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Right now, the logic parses ABI files on 4 steps, one for each
directory. While this is fine in principle, by doing that, not
all symbol cross-references will be created.
Change the logic to do the parsing only once in order to get
a global dictionary to be used when creating ABI cross-references.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/admin-guide/abi-obsolete.rst | 2 +-
Documentation/admin-guide/abi-removed.rst | 2 +-
Documentation/admin-guide/abi-stable.rst | 2 +-
Documentation/admin-guide/abi-testing.rst | 2 +-
Documentation/sphinx/kernel_abi.py | 115 ++++++++++++---------
scripts/lib/abi/abi_parser.py | 22 ++--
6 files changed, 81 insertions(+), 64 deletions(-)
diff --git a/Documentation/admin-guide/abi-obsolete.rst b/Documentation/admin-guide/abi-obsolete.rst
index 6d4d9ab7b8c3..bdef91d2cea4 100644
--- a/Documentation/admin-guide/abi-obsolete.rst
+++ b/Documentation/admin-guide/abi-obsolete.rst
@@ -9,4 +9,4 @@ marked to be removed at some later point in time.
The description of the interface will document the reason why it is
obsolete and when it can be expected to be removed.
-.. kernel-abi:: ABI/obsolete
+.. kernel-abi:: obsolete
diff --git a/Documentation/admin-guide/abi-removed.rst b/Documentation/admin-guide/abi-removed.rst
index 9fc78af6f077..bea0608b8442 100644
--- a/Documentation/admin-guide/abi-removed.rst
+++ b/Documentation/admin-guide/abi-removed.rst
@@ -3,4 +3,4 @@
ABI removed symbols
===================
-.. kernel-abi:: ABI/removed
+.. kernel-abi:: removed
diff --git a/Documentation/admin-guide/abi-stable.rst b/Documentation/admin-guide/abi-stable.rst
index c47c2a295865..33637c0d4fd5 100644
--- a/Documentation/admin-guide/abi-stable.rst
+++ b/Documentation/admin-guide/abi-stable.rst
@@ -12,4 +12,4 @@ for at least 2 years.
Most interfaces (like syscalls) are expected to never change and always
be available.
-.. kernel-abi:: ABI/stable
+.. kernel-abi:: stable
diff --git a/Documentation/admin-guide/abi-testing.rst b/Documentation/admin-guide/abi-testing.rst
index 40b31985e587..55054985a8ff 100644
--- a/Documentation/admin-guide/abi-testing.rst
+++ b/Documentation/admin-guide/abi-testing.rst
@@ -18,4 +18,4 @@ Programs that use these interfaces are strongly encouraged to add their
name to the description of these interfaces, so that the kernel
developers can easily notify them if any changes occur.
-.. kernel-abi:: ABI/testing
+.. kernel-abi:: testing
diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
index 0a4057183208..964f586de171 100644
--- a/Documentation/sphinx/kernel_abi.py
+++ b/Documentation/sphinx/kernel_abi.py
@@ -49,6 +49,13 @@ from abi_parser import AbiParser
__version__ = "1.0"
+logger = logging.getLogger('kernel_abi')
+path = os.path.join(srctree, "Documentation/ABI")
+
+# Parse ABI symbols only once
+kernel_abi = AbiParser(path, logger=logger)
+kernel_abi.parse_abi()
+kernel_abi.check_issues()
def setup(app):
@@ -64,14 +71,15 @@ class KernelCmd(Directive):
u"""KernelABI (``kernel-abi``) directive"""
required_arguments = 1
- optional_arguments = 2
+ optional_arguments = 3
has_content = False
final_argument_whitespace = True
- logger = logging.getLogger('kernel_abi')
parser = None
option_spec = {
"debug": directives.flag,
+ "no-symbols": directives.flag,
+ "no-files": directives.flag,
}
def run(self):
@@ -79,62 +87,67 @@ class KernelCmd(Directive):
if not doc.settings.file_insertion_enabled:
raise self.warning("docutils: file insertion disabled")
- path = os.path.join(srctree, "Documentation", self.arguments[0])
- self.parser = AbiParser(path, logger=self.logger)
- self.parser.parse_abi()
- self.parser.check_issues()
-
- node = self.nested_parse(None, self.arguments[0])
- return node
-
- def nested_parse(self, data, fname):
env = self.state.document.settings.env
content = ViewList()
node = nodes.section()
- if data is not None:
- # Handles the .rst file
- for line in data.split("\n"):
- content.append(line, fname, 0)
+ abi_type = self.arguments[0]
- self.do_parse(content, node)
+ if "no-symbols" in self.options:
+ show_symbols = False
+ else:
+ show_symbols = True
+ if "no-files" in self.options:
+ show_file = False
+ else:
+ show_file = True
+
+ tab_width = self.options.get('tab-width',
+ self.state.document.settings.tab_width)
+
+ old_f = None
+ n = 0
+ n_sym = 0
+ for msg, f, ln in kernel_abi.doc(show_file=show_file,
+ show_symbols=show_symbols,
+ filter_path=abi_type):
+ n_sym += 1
+ msg_list = statemachine.string2lines(msg, tab_width,
+ convert_whitespace=True)
+ if "debug" in self.options:
+ lines = [
+ "", "", ".. code-block:: rst",
+ " :linenos:", ""
+ ]
+ for m in msg_list:
+ lines.append(" " + m)
+ else:
+ lines = msg_list
+
+ for line in lines:
+ # sphinx counts lines from 0
+ content.append(line, f, ln - 1)
+ n += 1
+
+ if f != old_f:
+ # Add the file to Sphinx build dependencies
+ env.note_dependency(os.path.abspath(f))
+
+ old_f = f
+
+ # Sphinx doesn't like to parse big messages. So, let's
+ # add content symbol by symbol
+ if content:
+ self.do_parse(content, node)
+ content = ViewList()
+
+ if show_symbols and not show_file:
+ logger.verbose("%s ABI: %i symbols (%i ReST lines)" % (abi_type, n_sym, n))
+ elif not show_symbols and show_file:
+ logger.verbose("%s ABI: %i files (%i ReST lines)" % (abi_type, n_sym, n))
else:
- # Handles the ABI parser content, symbol by symbol
-
- old_f = fname
- n = 0
- for msg, f, ln in self.parser.doc():
- msg_list = statemachine.string2lines(msg, tab_width,
- convert_whitespace=True)
- if "debug" in self.options:
- lines = [
- "", "", ".. code-block:: rst",
- " :linenos:", ""
- ]
- for m in msg_list:
- lines.append(" " + m)
- else:
- lines = msg_list
-
- for line in lines:
- # sphinx counts lines from 0
- content.append(line, f, ln - 1)
- n += 1
-
- if f != old_f:
- # Add the file to Sphinx build dependencies
- env.note_dependency(os.path.abspath(f))
-
- old_f = f
-
- # Sphinx doesn't like to parse big messages. So, let's
- # add content symbol by symbol
- if content:
- self.do_parse(content, node)
- content = ViewList()
-
- self.logger.info("%s: parsed %i lines" % (fname, n))
+ logger.verbose("%s ABI: %i data (%i ReST lines)" % (abi_type, n_sym, n))
return node.children
diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py
index 6fac461d794c..87d1b9e14bb3 100644
--- a/scripts/lib/abi/abi_parser.py
+++ b/scripts/lib/abi/abi_parser.py
@@ -266,12 +266,20 @@ class AbiParser:
def parse_readme(self, nametag, fname):
"""Parse ABI README file"""
+ nametag["what"] = ["ABI file contents"]
+ nametag["path"] = "README"
with open(fname, "r", encoding="utf8", errors="backslashreplace") as fp:
- nametag["description"] = "```\n"
for line in fp:
- nametag["description"] += " " + line
+ match = self.re_tag.match(line)
+ if match:
+ new = match.group(1).lower()
- nametag["description"] += "```\n"
+ match = self.re_valid.search(new)
+ if match:
+ nametag["description"] += "\n:" + line
+ continue
+
+ nametag["description"] += line
def parse_file(self, fname, path, basename):
"""Parse a single file"""
@@ -459,12 +467,8 @@ class AbiParser:
continue
if filter_path:
- if filter_path == "README":
- if not names[0].endswith("README"):
- continue
- else:
- if v.get("path") != filter_path:
- continue
+ if v.get("path") != filter_path:
+ continue
msg = ""
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 21/27] docs: admin-guide/abi: split files from symbols
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (19 preceding siblings ...)
2025-02-10 10:18 ` [PATCH 20/27] docs: sphinx/kernel_abi: parse ABI files only once Mauro Carvalho Chehab
@ 2025-02-10 10:18 ` Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 22/27] docs: sphinx/automarkup: add cross-references for ABI Mauro Carvalho Chehab
` (7 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:18 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Now that get_abi has gained support for filtering its output,
split ABI symbols from files at the html output.
That makes pages smaller and easier to navigate.
As an additional bonus, as it will paralelize files handling,
it gives an additional performance improvement.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/admin-guide/abi-obsolete-files.rst | 7 +++++++
Documentation/admin-guide/abi-obsolete.rst | 1 +
Documentation/admin-guide/abi-readme-file.rst | 6 ++++++
Documentation/admin-guide/abi-removed-files.rst | 7 +++++++
Documentation/admin-guide/abi-removed.rst | 1 +
Documentation/admin-guide/abi-stable-files.rst | 7 +++++++
Documentation/admin-guide/abi-stable.rst | 1 +
Documentation/admin-guide/abi-testing-files.rst | 7 +++++++
Documentation/admin-guide/abi-testing.rst | 1 +
Documentation/admin-guide/abi.rst | 15 +++++++++++++++
10 files changed, 53 insertions(+)
create mode 100644 Documentation/admin-guide/abi-obsolete-files.rst
create mode 100644 Documentation/admin-guide/abi-readme-file.rst
create mode 100644 Documentation/admin-guide/abi-removed-files.rst
create mode 100644 Documentation/admin-guide/abi-stable-files.rst
create mode 100644 Documentation/admin-guide/abi-testing-files.rst
diff --git a/Documentation/admin-guide/abi-obsolete-files.rst b/Documentation/admin-guide/abi-obsolete-files.rst
new file mode 100644
index 000000000000..3061a916b4b5
--- /dev/null
+++ b/Documentation/admin-guide/abi-obsolete-files.rst
@@ -0,0 +1,7 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+Obsolete ABI Files
+==================
+
+.. kernel-abi:: obsolete
+ :no-symbols:
diff --git a/Documentation/admin-guide/abi-obsolete.rst b/Documentation/admin-guide/abi-obsolete.rst
index bdef91d2cea4..640f3903e847 100644
--- a/Documentation/admin-guide/abi-obsolete.rst
+++ b/Documentation/admin-guide/abi-obsolete.rst
@@ -10,3 +10,4 @@ The description of the interface will document the reason why it is
obsolete and when it can be expected to be removed.
.. kernel-abi:: obsolete
+ :no-files:
diff --git a/Documentation/admin-guide/abi-readme-file.rst b/Documentation/admin-guide/abi-readme-file.rst
new file mode 100644
index 000000000000..6172e4ccbda2
--- /dev/null
+++ b/Documentation/admin-guide/abi-readme-file.rst
@@ -0,0 +1,6 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+ABI README
+==========
+
+.. kernel-abi:: README
diff --git a/Documentation/admin-guide/abi-removed-files.rst b/Documentation/admin-guide/abi-removed-files.rst
new file mode 100644
index 000000000000..f1bdfadd2ec4
--- /dev/null
+++ b/Documentation/admin-guide/abi-removed-files.rst
@@ -0,0 +1,7 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+Removed ABI Files
+=================
+
+.. kernel-abi:: removed
+ :no-symbols:
diff --git a/Documentation/admin-guide/abi-removed.rst b/Documentation/admin-guide/abi-removed.rst
index bea0608b8442..88832d3eacd6 100644
--- a/Documentation/admin-guide/abi-removed.rst
+++ b/Documentation/admin-guide/abi-removed.rst
@@ -4,3 +4,4 @@ ABI removed symbols
===================
.. kernel-abi:: removed
+ :no-files:
diff --git a/Documentation/admin-guide/abi-stable-files.rst b/Documentation/admin-guide/abi-stable-files.rst
new file mode 100644
index 000000000000..f867738fc178
--- /dev/null
+++ b/Documentation/admin-guide/abi-stable-files.rst
@@ -0,0 +1,7 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+Stable ABI Files
+================
+
+.. kernel-abi:: stable
+ :no-symbols:
diff --git a/Documentation/admin-guide/abi-stable.rst b/Documentation/admin-guide/abi-stable.rst
index 33637c0d4fd5..528c68401f4b 100644
--- a/Documentation/admin-guide/abi-stable.rst
+++ b/Documentation/admin-guide/abi-stable.rst
@@ -13,3 +13,4 @@ Most interfaces (like syscalls) are expected to never change and always
be available.
.. kernel-abi:: stable
+ :no-files:
diff --git a/Documentation/admin-guide/abi-testing-files.rst b/Documentation/admin-guide/abi-testing-files.rst
new file mode 100644
index 000000000000..1da868e42fdb
--- /dev/null
+++ b/Documentation/admin-guide/abi-testing-files.rst
@@ -0,0 +1,7 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+Testing ABI Files
+=================
+
+.. kernel-abi:: testing
+ :no-symbols:
diff --git a/Documentation/admin-guide/abi-testing.rst b/Documentation/admin-guide/abi-testing.rst
index 55054985a8ff..6153ebd38e2d 100644
--- a/Documentation/admin-guide/abi-testing.rst
+++ b/Documentation/admin-guide/abi-testing.rst
@@ -19,3 +19,4 @@ name to the description of these interfaces, so that the kernel
developers can easily notify them if any changes occur.
.. kernel-abi:: testing
+ :no-files:
diff --git a/Documentation/admin-guide/abi.rst b/Documentation/admin-guide/abi.rst
index 64e772bde943..15a2dcb1388c 100644
--- a/Documentation/admin-guide/abi.rst
+++ b/Documentation/admin-guide/abi.rst
@@ -4,6 +4,9 @@
Linux ABI description
=====================
+ABI symbols
+-----------
+
.. toctree::
:maxdepth: 2
@@ -11,3 +14,15 @@ Linux ABI description
abi-testing
abi-obsolete
abi-removed
+
+ABI files
+---------
+
+.. toctree::
+ :maxdepth: 2
+
+ abi-readme-file
+ abi-stable-files
+ abi-testing-files
+ abi-obsolete-files
+ abi-removed-files
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 22/27] docs: sphinx/automarkup: add cross-references for ABI
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (20 preceding siblings ...)
2025-02-10 10:18 ` [PATCH 21/27] docs: admin-guide/abi: split files from symbols Mauro Carvalho Chehab
@ 2025-02-10 10:18 ` Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 23/27] docs: sphinx/kernel_abi: avoid warnings during Sphinx module init Mauro Carvalho Chehab
` (6 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:18 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Now that all ABI files are handled together, we can add a feature
at automarkup for it to generate cross-references for ABI symbols.
The cross-reference logic can produce references for all existing
files, except for README (as this is not parsed).
For symbols, they need to be an exact match of what it is
described at the docs, which is not always true due to wildcards.
If symbols at /sys /proc and /config are identical, a cross-reference
will be used.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/sphinx/automarkup.py | 45 ++++++++++++++++++++++++++++++
scripts/lib/abi/abi_parser.py | 11 ++++++++
2 files changed, 56 insertions(+)
diff --git a/Documentation/sphinx/automarkup.py b/Documentation/sphinx/automarkup.py
index a413f8dd5115..7d91c39b4ca6 100644
--- a/Documentation/sphinx/automarkup.py
+++ b/Documentation/sphinx/automarkup.py
@@ -11,6 +11,8 @@ from sphinx.errors import NoUri
import re
from itertools import chain
+from kernel_abi import kernel_abi
+
#
# Python 2 lacks re.ASCII...
#
@@ -48,6 +50,8 @@ RE_typedef = re.compile(r'\b(typedef)\s+([a-zA-Z_]\w+)', flags=ascii_p3)
# an optional extension
#
RE_doc = re.compile(r'(\bDocumentation/)?((\.\./)*[\w\-/]+)\.(rst|txt)')
+RE_abi_file = re.compile(r'(\bDocumentation/ABI/[\w\-/]+)')
+RE_abi_symbol = re.compile(r'(\b/(sys|config|proc)/[\w\-/]+)')
RE_namespace = re.compile(r'^\s*..\s*c:namespace::\s*(\S+)\s*$')
@@ -84,10 +88,14 @@ def markup_refs(docname, app, node):
# Associate each regex with the function that will markup its matches
#
markup_func_sphinx2 = {RE_doc: markup_doc_ref,
+ RE_abi_file: markup_abi_ref,
+ RE_abi_symbol: markup_abi_ref,
RE_function: markup_c_ref,
RE_generic_type: markup_c_ref}
markup_func_sphinx3 = {RE_doc: markup_doc_ref,
+ RE_abi_file: markup_abi_ref,
+ RE_abi_symbol: markup_abi_ref,
RE_function: markup_func_ref_sphinx3,
RE_struct: markup_c_ref,
RE_union: markup_c_ref,
@@ -270,6 +278,43 @@ def markup_doc_ref(docname, app, match):
else:
return nodes.Text(match.group(0))
+#
+# Try to replace a documentation reference of the form Documentation/ABI/...
+# with a cross reference to that page
+#
+def markup_abi_ref(docname, app, match):
+ stddom = app.env.domains['std']
+ #
+ # Go through the dance of getting an xref out of the std domain
+ #
+ fname = match.group(1)
+ target = kernel_abi.xref(fname)
+
+ # Kernel ABI doesn't describe such file or symbol
+ if not target:
+ return nodes.Text(match.group(0))
+
+ pxref = addnodes.pending_xref('', refdomain = 'std', reftype = 'ref',
+ reftarget = target, modname = None,
+ classname = None, refexplicit = False)
+
+ #
+ # XXX The Latex builder will throw NoUri exceptions here,
+ # work around that by ignoring them.
+ #
+ try:
+ xref = stddom.resolve_xref(app.env, docname, app.builder, 'ref',
+ target, pxref, None)
+ except NoUri:
+ xref = None
+ #
+ # Return the xref if we got it; otherwise just return the plain text.
+ #
+ if xref:
+ return xref
+ else:
+ return nodes.Text(match.group(0))
+
def get_c_namespace(app, docname):
source = app.env.doc2path(docname)
with open(source) as f:
diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py
index 87d1b9e14bb3..3b1ab4c0bdd7 100644
--- a/scripts/lib/abi/abi_parser.py
+++ b/scripts/lib/abi/abi_parser.py
@@ -391,6 +391,17 @@ class AbiParser:
return desc + "\n\n"
+ def xref(self, fname):
+ """
+ Converts a Documentation/ABI + basename into a ReST cross-reference
+ """
+
+ xref = self.file_refs.get(fname)
+ if not xref:
+ return None
+ else:
+ return xref
+
def desc_rst(self, desc):
"""Enrich ReST output by creating cross-references"""
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 23/27] docs: sphinx/kernel_abi: avoid warnings during Sphinx module init
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (21 preceding siblings ...)
2025-02-10 10:18 ` [PATCH 22/27] docs: sphinx/automarkup: add cross-references for ABI Mauro Carvalho Chehab
@ 2025-02-10 10:18 ` Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 24/27] scripts/lib/abi/abi_parser.py: Rename title name for ABI files Mauro Carvalho Chehab
` (5 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:18 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
Sphinx logging system doesn't like warnings during module load,
as it understands that such logs are produced at the wrong time:
WARNING: while setting up extension automarkup: /sys/devices/system/cpu/cpuX/topology/physical_package_id is defined 2 times: /new_devel/v4l/docs/Documentation/ABI/stable/sysfs-devices-system-cpu:27; /new_devel/v4l/docs/Documentation/ABI/testing/sysfs-devices-system-cpu:70
WARNING: while setting up extension automarkup: /sys/devices/system/cpu/cpuX/topology/ppin is defined 2 times: /new_devel/v4l/docs/Documentation/ABI/stable/sysfs-devices-system-cpu:89; /new_devel/v4l/docs/Documentation/ABI/testing/sysfs-devices-system-cpu:70
So, use a function to allocate/process ABI files and use it to
be called at kernel_abi.py, as automarkup also needs it to
produce the right cross-references.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/sphinx/automarkup.py | 4 +++-
Documentation/sphinx/kernel_abi.py | 23 +++++++++++++++++++----
2 files changed, 22 insertions(+), 5 deletions(-)
diff --git a/Documentation/sphinx/automarkup.py b/Documentation/sphinx/automarkup.py
index 7d91c39b4ca6..22defc18d6d2 100644
--- a/Documentation/sphinx/automarkup.py
+++ b/Documentation/sphinx/automarkup.py
@@ -11,7 +11,7 @@ from sphinx.errors import NoUri
import re
from itertools import chain
-from kernel_abi import kernel_abi
+from kernel_abi import get_kernel_abi
#
# Python 2 lacks re.ASCII...
@@ -287,6 +287,8 @@ def markup_abi_ref(docname, app, match):
#
# Go through the dance of getting an xref out of the std domain
#
+ kernel_abi = get_kernel_abi()
+
fname = match.group(1)
target = kernel_abi.xref(fname)
diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
index 964f586de171..e017b0299953 100644
--- a/Documentation/sphinx/kernel_abi.py
+++ b/Documentation/sphinx/kernel_abi.py
@@ -52,10 +52,23 @@ __version__ = "1.0"
logger = logging.getLogger('kernel_abi')
path = os.path.join(srctree, "Documentation/ABI")
-# Parse ABI symbols only once
-kernel_abi = AbiParser(path, logger=logger)
-kernel_abi.parse_abi()
-kernel_abi.check_issues()
+_kernel_abi = None
+
+def get_kernel_abi():
+ u"""
+ Initialize kernel_abi global var, if not initialized yet.
+
+ This is needed to avoid warnings during Sphinx module initialization.
+ """
+ global _kernel_abi
+
+ if not _kernel_abi:
+ # Parse ABI symbols only once
+ _kernel_abi = AbiParser(path, logger=logger)
+ _kernel_abi.parse_abi()
+ _kernel_abi.check_issues()
+
+ return _kernel_abi
def setup(app):
@@ -83,6 +96,8 @@ class KernelCmd(Directive):
}
def run(self):
+ kernel_abi = get_kernel_abi()
+
doc = self.state.document
if not doc.settings.file_insertion_enabled:
raise self.warning("docutils: file insertion disabled")
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 24/27] scripts/lib/abi/abi_parser.py: Rename title name for ABI files
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (22 preceding siblings ...)
2025-02-10 10:18 ` [PATCH 23/27] docs: sphinx/kernel_abi: avoid warnings during Sphinx module init Mauro Carvalho Chehab
@ 2025-02-10 10:18 ` Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 25/27] scripts/lib/abi/abi_parser.py: make it backward-compatible with Python 3.6 Mauro Carvalho Chehab
` (4 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:18 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel
This makes them look better when generating cross-references.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/lib/abi/abi_parser.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py
index 3b1ab4c0bdd7..0c3837e52afa 100644
--- a/scripts/lib/abi/abi_parser.py
+++ b/scripts/lib/abi/abi_parser.py
@@ -314,7 +314,7 @@ class AbiParser:
fdata.ftype = path.split("/")[0]
fdata.nametag = {}
- fdata.nametag["what"] = [f"File {path}/{basename}"]
+ fdata.nametag["what"] = [f"ABI file {path}/{basename}"]
fdata.nametag["type"] = "File"
fdata.nametag["path"] = fdata.ftype
fdata.nametag["file"] = [fdata.file_ref]
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 25/27] scripts/lib/abi/abi_parser.py: make it backward-compatible with Python 3.6
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (23 preceding siblings ...)
2025-02-10 10:18 ` [PATCH 24/27] scripts/lib/abi/abi_parser.py: Rename title name for ABI files Mauro Carvalho Chehab
@ 2025-02-10 10:18 ` Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 26/27] scripts/get_abi.py: add support for undefined ABIs Mauro Carvalho Chehab
` (3 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:18 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Jonathan Corbet, Mauro Carvalho Chehab,
linux-kernel, Akira Yokosawa
Despite being introduced on Python 3.6, the original implementation
was too limited: it doesn't accept anything but the argument.
Even on python 3.10.12, support was still limited, as more complex
operations cause SyntaxError:
Exception occurred:
File ".../linux/Documentation/sphinx/kernel_abi.py", line 48, in <module>
from get_abi import AbiParser
File ".../linux/scripts/lib/abi/abi_parser.py", line 525
msg += f"{part}\n{"-" * len(part)}\n\n"
^
SyntaxError: f-string: expecting '}'
Replace f-strings by normal string concatenation when it doesn't
work on Python 3.6.
Reported-by: Akira Yokosawa <akiyks@gmail.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/lib/abi/abi_parser.py | 16 ++++++++++------
1 file changed, 10 insertions(+), 6 deletions(-)
diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py
index 0c3837e52afa..f08de6d3bf7c 100644
--- a/scripts/lib/abi/abi_parser.py
+++ b/scripts/lib/abi/abi_parser.py
@@ -493,7 +493,7 @@ class AbiParser:
if cur_part and cur_part != part:
part = cur_part
- msg += f"{part}\n{"-" * len(part)}\n\n"
+ msg += part + "\n"+ "-" * len(part) +"\n\n"
msg += f".. _{key}:\n\n"
@@ -517,7 +517,7 @@ class AbiParser:
msg += f"Defined on file :ref:`{base} <{ref[1]}>`\n\n"
if wtype == "File":
- msg += f"{names[0]}\n{"-" * len(names[0])}\n\n"
+ msg += names[0] +"\n" + "-" * len(names[0]) +"\n\n"
desc = v.get("description")
if not desc and wtype != "File":
@@ -541,7 +541,8 @@ class AbiParser:
users = v.get("users")
if users and users.strip(" \t\n"):
- msg += f"Users:\n\t{users.strip("\n").replace('\n', '\n\t')}\n\n"
+ users = users.strip("\n").replace('\n', '\n\t')
+ msg += f"Users:\n\t{users}\n\n"
ln = v.get("line_no", 1)
@@ -567,7 +568,9 @@ class AbiParser:
elif len(lines) == 1:
f.append(f"{fname}:{lines[0]}")
else:
- f.append(f"{fname} lines {", ".join(str(x) for x in lines)}")
+ m = fname + "lines "
+ m += ", ".join(str(x) for x in lines)
+ f.append(m)
self.log.warning("%s is defined %d times: %s", what, len(f), "; ".join(f))
@@ -615,10 +618,11 @@ class AbiParser:
if users:
print(f"Users:\t\t\t{users}")
- print(f"Defined on file{'s'[:len(files) ^ 1]}:\t{", ".join(files)}")
+ print("Defined on file(s):\t" + ", ".join(files))
if desc:
- print(f"\n{desc.strip("\n")}\n")
+ desc = desc.strip("\n")
+ print(f"\n{desc}\n")
if not found_keys:
print(f"Regular expression /{expr}/ not found.")
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 26/27] scripts/get_abi.py: add support for undefined ABIs
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (24 preceding siblings ...)
2025-02-10 10:18 ` [PATCH 25/27] scripts/lib/abi/abi_parser.py: make it backward-compatible with Python 3.6 Mauro Carvalho Chehab
@ 2025-02-10 10:18 ` Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 27/27] scripts/get_abi.pl: drop now obsoleted script Mauro Carvalho Chehab
` (2 subsequent siblings)
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:18 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Guilherme G. Piccoli, Jonathan Corbet,
Mauro Carvalho Chehab, Kees Cook, Tony Luck, bpf, linux-hardening,
linux-kernel
The undefined logic is complex and has lots of magic on it.
Implement it, using the same algorithm we have at get_abi.pl. Yet,
some tweaks to optimize performance and to make the code simpler
were added here:
- at the perl version, the tree graph had loops, so we had to
use BFS to traverse it. On this version, the graph is a tree,
so, it simplifies the what group for sysfs aliases;
- the logic which splits regular expressions into subgroups
was re-written to make it faster;
- it may optionally use multiple processes to search for symbol
matches;
- it has some additional debug levels.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/get_abi.py | 68 ++++++
scripts/lib/abi/abi_regex.py | 234 ++++++++++++++++++
scripts/lib/abi/helpers.py | 16 +-
scripts/lib/abi/system_symbols.py | 378 ++++++++++++++++++++++++++++++
4 files changed, 693 insertions(+), 3 deletions(-)
create mode 100644 scripts/lib/abi/abi_regex.py
create mode 100644 scripts/lib/abi/system_symbols.py
diff --git a/scripts/get_abi.py b/scripts/get_abi.py
index 19f78d6aa407..7ce4748a46d2 100755
--- a/scripts/get_abi.py
+++ b/scripts/get_abi.py
@@ -20,7 +20,9 @@ SRC_DIR = os.path.dirname(os.path.realpath(__file__))
sys.path.insert(0, os.path.join(SRC_DIR, LIB_DIR))
from abi_parser import AbiParser # pylint: disable=C0413
+from abi_regex import AbiRegex # pylint: disable=C0413
from helpers import ABI_DIR, DEBUG_HELP # pylint: disable=C0413
+from system_symbols import SystemSymbols # pylint: disable=C0413
# Command line classes
@@ -111,6 +113,71 @@ class AbiSearch:
parser.parse_abi()
parser.search_symbols(args.expression)
+UNDEFINED_DESC="""
+Check undefined ABIs on local machine.
+
+Read sysfs devnodes and check if the devnodes there are defined inside
+ABI documentation.
+
+The search logic tries to minimize the number of regular expressions to
+search per each symbol.
+
+By default, it runs on a single CPU, as Python support for CPU threads
+is still experimental, and multi-process runs on Python is very slow.
+
+On experimental tests, if the number of ABI symbols to search per devnode
+is contained on a limit of ~150 regular expressions, using a single CPU
+is a lot faster than using multiple processes. However, if the number of
+regular expressions to check is at the order of ~30000, using multiple
+CPUs speeds up the check.
+"""
+
+class AbiUndefined:
+ """
+ Initialize an argparse subparser for logic to check undefined ABI at
+ the current machine's sysfs
+ """
+
+ def __init__(self, subparsers):
+ """Initialize argparse subparsers"""
+
+ parser = subparsers.add_parser("undefined",
+ formatter_class=argparse.RawTextHelpFormatter,
+ description=UNDEFINED_DESC)
+
+ parser.add_argument("-S", "--sysfs-dir", default="/sys",
+ help="directory where sysfs is mounted")
+ parser.add_argument("-s", "--search-string",
+ help="search string regular expression to limit symbol search")
+ parser.add_argument("-H", "--show-hints", action="store_true",
+ help="Hints about definitions for missing ABI symbols.")
+ parser.add_argument("-j", "--jobs", "--max-workers", type=int, default=1,
+ help="If bigger than one, enables multiprocessing.")
+ parser.add_argument("-c", "--max-chunk-size", type=int, default=50,
+ help="Maximum number of chunk size")
+ parser.add_argument("-f", "--found", action="store_true",
+ help="Also show found items. "
+ "Helpful to debug the parser."),
+ parser.add_argument("-d", "--dry-run", action="store_true",
+ help="Don't actually search for undefined. "
+ "Helpful to debug the parser."),
+
+ parser.set_defaults(func=self.run)
+
+ def run(self, args):
+ """Run subparser"""
+
+ abi = AbiRegex(args.dir, debug=args.debug,
+ search_string=args.search_string)
+
+ abi_symbols = SystemSymbols(abi=abi, hints=args.show_hints,
+ sysfs=args.sysfs_dir)
+
+ abi_symbols.check_undefined_symbols(dry_run=args.dry_run,
+ found=args.found,
+ max_workers=args.jobs,
+ chunk_size=args.max_chunk_size)
+
def main():
"""Main program"""
@@ -125,6 +192,7 @@ def main():
AbiRest(subparsers)
AbiValidate(subparsers)
AbiSearch(subparsers)
+ AbiUndefined(subparsers)
args = parser.parse_args()
diff --git a/scripts/lib/abi/abi_regex.py b/scripts/lib/abi/abi_regex.py
new file mode 100644
index 000000000000..8a57846cbc69
--- /dev/null
+++ b/scripts/lib/abi/abi_regex.py
@@ -0,0 +1,234 @@
+#!/usr/bin/env python3
+# xxpylint: disable=R0903
+# Copyright(c) 2025: Mauro Carvalho Chehab <mchehab@kernel.org>.
+# SPDX-License-Identifier: GPL-2.0
+
+"""
+Convert ABI what into regular expressions
+"""
+
+import re
+import sys
+
+from pprint import pformat
+
+from abi_parser import AbiParser
+from helpers import AbiDebug
+
+class AbiRegex(AbiParser):
+ """Extends AbiParser to search ABI nodes with regular expressions"""
+
+ # Escape only ASCII visible characters
+ escape_symbols = r"([\x21-\x29\x2b-\x2d\x3a-\x40\x5c\x60\x7b-\x7e])"
+ leave_others = "others"
+
+ # Tuples with regular expressions to be compiled and replacement data
+ re_whats = [
+ # Drop escape characters that might exist
+ (re.compile("\\\\"), ""),
+
+ # Temporarily escape dot characters
+ (re.compile(r"\."), "\xf6"),
+
+ # Temporarily change [0-9]+ type of patterns
+ (re.compile(r"\[0\-9\]\+"), "\xff"),
+
+ # Temporarily change [\d+-\d+] type of patterns
+ (re.compile(r"\[0\-\d+\]"), "\xff"),
+ (re.compile(r"\[0:\d+\]"), "\xff"),
+ (re.compile(r"\[(\d+)\]"), "\xf4\\\\d+\xf5"),
+
+ # Temporarily change [0-9] type of patterns
+ (re.compile(r"\[(\d)\-(\d)\]"), "\xf4\1-\2\xf5"),
+
+ # Handle multiple option patterns
+ (re.compile(r"[\{\<\[]([\w_]+)(?:[,|]+([\w_]+)){1,}[\}\>\]]"), r"(\1|\2)"),
+
+ # Handle wildcards
+ (re.compile(r"([^\/])\*"), "\\1\\\\w\xf7"),
+ (re.compile(r"/\*/"), "/.*/"),
+ (re.compile(r"/\xf6\xf6\xf6"), "/.*"),
+ (re.compile(r"\<[^\>]+\>"), "\\\\w\xf7"),
+ (re.compile(r"\{[^\}]+\}"), "\\\\w\xf7"),
+ (re.compile(r"\[[^\]]+\]"), "\\\\w\xf7"),
+
+ (re.compile(r"XX+"), "\\\\w\xf7"),
+ (re.compile(r"([^A-Z])[XYZ]([^A-Z])"), "\\1\\\\w\xf7\\2"),
+ (re.compile(r"([^A-Z])[XYZ]$"), "\\1\\\\w\xf7"),
+ (re.compile(r"_[AB]_"), "_\\\\w\xf7_"),
+
+ # Recover [0-9] type of patterns
+ (re.compile(r"\xf4"), "["),
+ (re.compile(r"\xf5"), "]"),
+
+ # Remove duplicated spaces
+ (re.compile(r"\s+"), r" "),
+
+ # Special case: drop comparison as in:
+ # What: foo = <something>
+ # (this happens on a few IIO definitions)
+ (re.compile(r"\s*\=.*$"), ""),
+
+ # Escape all other symbols
+ (re.compile(escape_symbols), r"\\\1"),
+ (re.compile(r"\\\\"), r"\\"),
+ (re.compile(r"\\([\[\]\(\)\|])"), r"\1"),
+ (re.compile(r"(\d+)\\(-\d+)"), r"\1\2"),
+
+ (re.compile(r"\xff"), r"\\d+"),
+
+ # Special case: IIO ABI which a parenthesis.
+ (re.compile(r"sqrt(.*)"), r"sqrt(.*)"),
+
+ # Simplify regexes with multiple .*
+ (re.compile(r"(?:\.\*){2,}"), ""),
+
+ # Recover dot characters
+ (re.compile(r"\xf6"), "\\."),
+ # Recover plus characters
+ (re.compile(r"\xf7"), "+"),
+ ]
+ re_has_num = re.compile(r"\\d")
+
+ # Symbol name after escape_chars that are considered a devnode basename
+ re_symbol_name = re.compile(r"(\w|\\[\.\-\:])+$")
+
+ # List of popular group names to be skipped to minimize regex group size
+ # Use AbiDebug.SUBGROUP_SIZE to detect those
+ skip_names = set(["devices", "hwmon"])
+
+ def regex_append(self, what, new):
+ """
+ Get a search group for a subset of regular expressions.
+
+ As ABI may have thousands of symbols, using a for to search all
+ regular expressions is at least O(n^2). When there are wildcards,
+ the complexity increases substantially, eventually becoming exponential.
+
+ To avoid spending too much time on them, use a logic to split
+ them into groups. The smaller the group, the better, as it would
+ mean that searches will be confined to a small number of regular
+ expressions.
+
+ The conversion to a regex subset is tricky, as we need something
+ that can be easily obtained from the sysfs symbol and from the
+ regular expression. So, we need to discard nodes that have
+ wildcards.
+
+ If it can't obtain a subgroup, place the regular expression inside
+ a special group (self.leave_others).
+ """
+
+ search_group = None
+
+ for search_group in reversed(new.split("/")):
+ if not search_group or search_group in self.skip_names:
+ continue
+ if self.re_symbol_name.match(search_group):
+ break
+
+ if not search_group:
+ search_group = self.leave_others
+
+ if self.debug & AbiDebug.SUBGROUP_MAP:
+ self.log.debug("%s: mapped as %s", what, search_group)
+
+ try:
+ if search_group not in self.regex_group:
+ self.regex_group[search_group] = []
+
+ self.regex_group[search_group].append(re.compile(new))
+ if self.search_string:
+ if what.find(self.search_string) >= 0:
+ print(f"What: {what}")
+ except re.PatternError:
+ self.log.warning("Ignoring '%s' as it produced an invalid regex:\n"
+ " '%s'", what, new)
+
+ def get_regexes(self, what):
+ """
+ Given an ABI devnode, return a list of all regular expressions that
+ may match it, based on the sub-groups created by regex_append()
+ """
+
+ re_list = []
+
+ patches = what.split("/")
+ patches.reverse()
+ patches.append(self.leave_others)
+
+ for search_group in patches:
+ if search_group in self.regex_group:
+ re_list += self.regex_group[search_group]
+
+ return re_list
+
+ def __init__(self, *args, **kwargs):
+ """
+ Override init method to get verbose argument
+ """
+
+ self.regex_group = None
+ self.search_string = None
+ self.re_string = None
+
+ if "search_string" in kwargs:
+ self.search_string = kwargs.get("search_string")
+ del kwargs["search_string"]
+
+ if self.search_string:
+
+ try:
+ self.re_string = re.compile(self.search_string)
+ except re.PatternError as e:
+ msg = f"{self.search_string} is not a valid regular expression"
+ raise ValueError(msg) from e
+
+ super().__init__(*args, **kwargs)
+
+ def parse_abi(self, *args, **kwargs):
+
+ super().parse_abi(*args, **kwargs)
+
+ self.regex_group = {}
+
+ print("Converting ABI What fields into regexes...", file=sys.stderr)
+
+ for t in sorted(self.data.items(), key=lambda x: x[0]):
+ v = t[1]
+ if v.get("type") == "File":
+ continue
+
+ v["regex"] = []
+
+ for what in v.get("what", []):
+ if not what.startswith("/sys"):
+ continue
+
+ new = what
+ for r, s in self.re_whats:
+ try:
+ new = r.sub(s, new)
+ except re.PatternError as e:
+ # Help debugging troubles with new regexes
+ raise re.PatternError(f"{e}\nwhile re.sub('{r.pattern}', {s}, str)") from e
+
+ v["regex"].append(new)
+
+ if self.debug & AbiDebug.REGEX:
+ self.log.debug("%-90s <== %s", new, what)
+
+ # Store regex into a subgroup to speedup searches
+ self.regex_append(what, new)
+
+ if self.debug & AbiDebug.SUBGROUP_DICT:
+ self.log.debug("%s", pformat(self.regex_group))
+
+ if self.debug & AbiDebug.SUBGROUP_SIZE:
+ biggestd_keys = sorted(self.regex_group.keys(),
+ key= lambda k: len(self.regex_group[k]),
+ reverse=True)
+
+ print("Top regex subgroups:", file=sys.stderr)
+ for k in biggestd_keys[:10]:
+ print(f"{k} has {len(self.regex_group[k])} elements", file=sys.stderr)
diff --git a/scripts/lib/abi/helpers.py b/scripts/lib/abi/helpers.py
index 84a253ed5058..639b23e4ca33 100644
--- a/scripts/lib/abi/helpers.py
+++ b/scripts/lib/abi/helpers.py
@@ -16,13 +16,23 @@ class AbiDebug:
WHAT_PARSING = 1
WHAT_OPEN = 2
DUMP_ABI_STRUCTS = 4
+ UNDEFINED = 8
+ REGEX = 16
+ SUBGROUP_MAP = 32
+ SUBGROUP_DICT = 64
+ SUBGROUP_SIZE = 128
+ GRAPH = 256
DEBUG_HELP = """
-Print debug information according with the level(s),
-which is given by the following bitmask:
-
1 - enable debug parsing logic
2 - enable debug messages on file open
4 - enable debug for ABI parse data
+8 - enable extra debug information to identify troubles
+ with ABI symbols found at the local machine that
+ weren't found on ABI documentation (used only for
+ undefined subcommand)
+16 - enable debug for what to regex conversion
+32 - enable debug for symbol regex subgroups
+64 - enable debug for sysfs graph tree variable
"""
diff --git a/scripts/lib/abi/system_symbols.py b/scripts/lib/abi/system_symbols.py
new file mode 100644
index 000000000000..f15c94a6e33c
--- /dev/null
+++ b/scripts/lib/abi/system_symbols.py
@@ -0,0 +1,378 @@
+#!/usr/bin/env python3
+# pylint: disable=R0902,R0912,R0914,R0915,R1702
+# Copyright(c) 2025: Mauro Carvalho Chehab <mchehab@kernel.org>.
+# SPDX-License-Identifier: GPL-2.0
+
+"""
+Parse ABI documentation and produce results from it.
+"""
+
+import os
+import re
+import sys
+
+from concurrent import futures
+from datetime import datetime
+from random import shuffle
+
+from helpers import AbiDebug
+
+class SystemSymbols:
+ """Stores arguments for the class and initialize class vars"""
+
+ def graph_add_file(self, path, link=None):
+ """
+ add a file path to the sysfs graph stored at self.root
+ """
+
+ if path in self.files:
+ return
+
+ name = ""
+ ref = self.root
+ for edge in path.split("/"):
+ name += edge + "/"
+ if edge not in ref:
+ ref[edge] = {"__name": [name.rstrip("/")]}
+
+ ref = ref[edge]
+
+ if link and link not in ref["__name"]:
+ ref["__name"].append(link.rstrip("/"))
+
+ self.files.add(path)
+
+ def print_graph(self, root_prefix="", root=None, level=0):
+ """Prints a reference tree graph using UTF-8 characters"""
+
+ if not root:
+ root = self.root
+ level = 0
+
+ # Prevent endless traverse
+ if level > 5:
+ return
+
+ if level > 0:
+ prefix = "├──"
+ last_prefix = "└──"
+ else:
+ prefix = ""
+ last_prefix = ""
+
+ items = list(root.items())
+
+ names = root.get("__name", [])
+ for k, edge in items:
+ if k == "__name":
+ continue
+
+ if not k:
+ k = "/"
+
+ if len(names) > 1:
+ k += " links: " + ",".join(names[1:])
+
+ if edge == items[-1][1]:
+ print(root_prefix + last_prefix + k)
+ p = root_prefix
+ if level > 0:
+ p += " "
+ self.print_graph(p, edge, level + 1)
+ else:
+ print(root_prefix + prefix + k)
+ p = root_prefix + "│ "
+ self.print_graph(p, edge, level + 1)
+
+ def _walk(self, root):
+ """
+ Walk through sysfs to get all devnodes that aren't ignored.
+
+ By default, uses /sys as sysfs mounting point. If another
+ directory is used, it replaces them to /sys at the patches.
+ """
+
+ with os.scandir(root) as obj:
+ for entry in obj:
+ path = os.path.join(root, entry.name)
+ if self.sysfs:
+ p = path.replace(self.sysfs, "/sys", count=1)
+ else:
+ p = path
+
+ if self.re_ignore.search(p):
+ return
+
+ # Handle link first to avoid directory recursion
+ if entry.is_symlink():
+ real = os.path.realpath(path)
+ if not self.sysfs:
+ self.aliases[path] = real
+ else:
+ real = real.replace(self.sysfs, "/sys", count=1)
+
+ # Add absfile location to graph if it doesn't exist
+ if not self.re_ignore.search(real):
+ # Add link to the graph
+ self.graph_add_file(real, p)
+
+ elif entry.is_file():
+ self.graph_add_file(p)
+
+ elif entry.is_dir():
+ self._walk(path)
+
+ def __init__(self, abi, sysfs="/sys", hints=False):
+ """
+ Initialize internal variables and get a list of all files inside
+ sysfs that can currently be parsed.
+
+ Please notice that there are several entries on sysfs that aren't
+ documented as ABI. Ignore those.
+
+ The real paths will be stored under self.files. Aliases will be
+ stored in separate, as self.aliases.
+ """
+
+ self.abi = abi
+ self.log = abi.log
+
+ if sysfs != "/sys":
+ self.sysfs = sysfs.rstrip("/")
+ else:
+ self.sysfs = None
+
+ self.hints = hints
+
+ self.root = {}
+ self.aliases = {}
+ self.files = set()
+
+ dont_walk = [
+ # Those require root access and aren't documented at ABI
+ f"^{sysfs}/kernel/debug",
+ f"^{sysfs}/kernel/tracing",
+ f"^{sysfs}/fs/pstore",
+ f"^{sysfs}/fs/bpf",
+ f"^{sysfs}/fs/fuse",
+
+ # This is not documented at ABI
+ f"^{sysfs}/module",
+
+ f"^{sysfs}/fs/cgroup", # this is big and has zero docs under ABI
+ f"^{sysfs}/firmware", # documented elsewhere: ACPI, DT bindings
+ "sections|notes", # aren't actually part of ABI
+
+ # kernel-parameters.txt - not easy to parse
+ "parameters",
+ ]
+
+ self.re_ignore = re.compile("|".join(dont_walk))
+
+ print(f"Reading {sysfs} directory contents...", file=sys.stderr)
+ self._walk(sysfs)
+
+ def check_file(self, refs, found):
+ """Check missing ABI symbols for a given sysfs file"""
+
+ res_list = []
+
+ try:
+ for names in refs:
+ fname = names[0]
+
+ res = {
+ "found": False,
+ "fname": fname,
+ "msg": "",
+ }
+ res_list.append(res)
+
+ re_what = self.abi.get_regexes(fname)
+ if not re_what:
+ self.abi.log.warning(f"missing rules for {fname}")
+ continue
+
+ for name in names:
+ for r in re_what:
+ if self.abi.debug & AbiDebug.UNDEFINED:
+ self.log.debug("check if %s matches '%s'", name, r.pattern)
+ if r.match(name):
+ res["found"] = True
+ if found:
+ res["msg"] += f" {fname}: regex:\n\t"
+ continue
+
+ if self.hints and not res["found"]:
+ res["msg"] += f" {fname} not found. Tested regexes:\n"
+ for r in re_what:
+ res["msg"] += " " + r.pattern + "\n"
+
+ except KeyboardInterrupt:
+ pass
+
+ return res_list
+
+ def _ref_interactor(self, root):
+ """Recursive function to interact over the sysfs tree"""
+
+ for k, v in root.items():
+ if isinstance(v, dict):
+ yield from self._ref_interactor(v)
+
+ if root == self.root or k == "__name":
+ continue
+
+ if self.abi.re_string:
+ fname = v["__name"][0]
+ if self.abi.re_string.search(fname):
+ yield v
+ else:
+ yield v
+
+
+ def get_fileref(self, all_refs, chunk_size):
+ """Interactor to group refs into chunks"""
+
+ n = 0
+ refs = []
+
+ for ref in all_refs:
+ refs.append(ref)
+
+ n += 1
+ if n >= chunk_size:
+ yield refs
+ n = 0
+ refs = []
+
+ yield refs
+
+ def check_undefined_symbols(self, max_workers=None, chunk_size=50,
+ found=None, dry_run=None):
+ """Seach ABI for sysfs symbols missing documentation"""
+
+ self.abi.parse_abi()
+
+ if self.abi.debug & AbiDebug.GRAPH:
+ self.print_graph()
+
+ all_refs = []
+ for ref in self._ref_interactor(self.root):
+ all_refs.append(ref["__name"])
+
+ if dry_run:
+ print("Would check", file=sys.stderr)
+ for ref in all_refs:
+ print(", ".join(ref))
+
+ return
+
+ print("Starting to search symbols (it may take several minutes):",
+ file=sys.stderr)
+ start = datetime.now()
+ old_elapsed = None
+
+ # Python doesn't support multithreading due to limitations on its
+ # global lock (GIL). While Python 3.13 finally made GIL optional,
+ # there are still issues related to it. Also, we want to have
+ # backward compatibility with older versions of Python.
+ #
+ # So, use instead multiprocess. However, Python is very slow passing
+ # data from/to multiple processes. Also, it may consume lots of memory
+ # if the data to be shared is not small. So, we need to group workload
+ # in chunks that are big enough to generate performance gains while
+ # not being so big that would cause out-of-memory.
+
+ num_refs = len(all_refs)
+ print(f"Number of references to parse: {num_refs}", file=sys.stderr)
+
+ if not max_workers:
+ max_workers = os.cpu_count()
+ elif max_workers > os.cpu_count():
+ max_workers = os.cpu_count()
+
+ max_workers = max(max_workers, 1)
+
+ max_chunk_size = int((num_refs + max_workers - 1) / max_workers)
+ chunk_size = min(chunk_size, max_chunk_size)
+ chunk_size = max(1, chunk_size)
+
+ if max_workers > 1:
+ executor = futures.ProcessPoolExecutor
+
+ # Place references in a random order. This may help improving
+ # performance, by mixing complex/simple expressions when creating
+ # chunks
+ shuffle(all_refs)
+ else:
+ # Python has a high overhead with processes. When there's just
+ # one worker, it is faster to not create a new process.
+ # Yet, User still deserves to have a progress print. So, use
+ # python's "thread", which is actually a single process, using
+ # an internal schedule to switch between tasks. No performance
+ # gains for non-IO tasks, but still it can be quickly interrupted
+ # from time to time to display progress.
+ executor = futures.ThreadPoolExecutor
+
+ not_found = []
+ f_list = []
+ with executor(max_workers=max_workers) as exe:
+ for refs in self.get_fileref(all_refs, chunk_size):
+ if refs:
+ try:
+ f_list.append(exe.submit(self.check_file, refs, found))
+
+ except KeyboardInterrupt:
+ return
+
+ total = len(f_list)
+
+ if not total:
+ if self.abi.re_string:
+ print(f"No ABI symbol matches {self.abi.search_string}")
+ else:
+ self.abi.log.warning("No ABI symbols found")
+ return
+
+ print(f"{len(f_list):6d} jobs queued on {max_workers} workers",
+ file=sys.stderr)
+
+ while f_list:
+ try:
+ t = futures.wait(f_list, timeout=1,
+ return_when=futures.FIRST_COMPLETED)
+
+ done = t[0]
+
+ for fut in done:
+ res_list = fut.result()
+
+ for res in res_list:
+ if not res["found"]:
+ not_found.append(res["fname"])
+ if res["msg"]:
+ print(res["msg"])
+
+ f_list.remove(fut)
+ except KeyboardInterrupt:
+ return
+
+ except RuntimeError as e:
+ self.abi.log.warning(f"Future: {e}")
+ break
+
+ if sys.stderr.isatty():
+ elapsed = str(datetime.now() - start).split(".", maxsplit=1)[0]
+ if len(f_list) < total:
+ elapsed += f" ({total - len(f_list)}/{total} jobs completed). "
+ if elapsed != old_elapsed:
+ print(elapsed + "\r", end="", flush=True,
+ file=sys.stderr)
+ old_elapsed = elapsed
+
+ elapsed = str(datetime.now() - start).split(".", maxsplit=1)[0]
+ print(elapsed, file=sys.stderr)
+
+ for f in sorted(not_found):
+ print(f"{f} not found.")
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH 27/27] scripts/get_abi.pl: drop now obsoleted script
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (25 preceding siblings ...)
2025-02-10 10:18 ` [PATCH 26/27] scripts/get_abi.py: add support for undefined ABIs Mauro Carvalho Chehab
@ 2025-02-10 10:18 ` Mauro Carvalho Chehab
2025-02-10 18:30 ` [PATCH 00/27] Improve ABI documentation generation Jonathan Corbet
2025-03-27 21:58 ` [f2fs-dev] " patchwork-bot+f2fs
28 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 10:18 UTC (permalink / raw)
To: Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Guilherme G. Piccoli, Jonathan Corbet,
Mauro Carvalho Chehab, Kees Cook, Tony Luck, bpf, linux-hardening,
linux-kernel
As all functionalities of it were migrated to get_abi.py,
drop the now obsoleted script.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
scripts/get_abi.pl | 1103 --------------------------------------------
1 file changed, 1103 deletions(-)
delete mode 100755 scripts/get_abi.pl
diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
deleted file mode 100755
index de1c0354b50c..000000000000
--- a/scripts/get_abi.pl
+++ /dev/null
@@ -1,1103 +0,0 @@
-#!/usr/bin/env perl
-# SPDX-License-Identifier: GPL-2.0
-
-BEGIN { $Pod::Usage::Formatter = 'Pod::Text::Termcap'; }
-
-use strict;
-use warnings;
-use utf8;
-use Pod::Usage qw(pod2usage);
-use Getopt::Long;
-use File::Find;
-use IO::Handle;
-use Fcntl ':mode';
-use Cwd 'abs_path';
-use Data::Dumper;
-
-my $help = 0;
-my $hint = 0;
-my $man = 0;
-my $debug = 0;
-my $enable_lineno = 0;
-my $show_warnings = 1;
-my $prefix="Documentation/ABI";
-my $sysfs_prefix="/sys";
-my $search_string;
-
-# Debug options
-my $dbg_what_parsing = 1;
-my $dbg_what_open = 2;
-my $dbg_dump_abi_structs = 4;
-my $dbg_undefined = 8;
-
-$Data::Dumper::Indent = 1;
-$Data::Dumper::Terse = 1;
-
-#
-# If true, assumes that the description is formatted with ReST
-#
-my $description_is_rst = 1;
-
-GetOptions(
- "debug=i" => \$debug,
- "enable-lineno" => \$enable_lineno,
- "rst-source!" => \$description_is_rst,
- "dir=s" => \$prefix,
- 'help|?' => \$help,
- "show-hints" => \$hint,
- "search-string=s" => \$search_string,
- man => \$man
-) or pod2usage(2);
-
-pod2usage(1) if $help;
-pod2usage(-exitstatus => 0, -noperldoc, -verbose => 2) if $man;
-
-pod2usage(2) if (scalar @ARGV < 1 || @ARGV > 2);
-
-my ($cmd, $arg) = @ARGV;
-
-pod2usage(2) if ($cmd ne "search" && $cmd ne "rest" && $cmd ne "validate" && $cmd ne "undefined");
-pod2usage(2) if ($cmd eq "search" && !$arg);
-
-require Data::Dumper if ($debug & $dbg_dump_abi_structs);
-
-my %data;
-my %symbols;
-
-#
-# Displays an error message, printing file name and line
-#
-sub parse_error($$$$) {
- my ($file, $ln, $msg, $data) = @_;
-
- return if (!$show_warnings);
-
- $data =~ s/\s+$/\n/;
-
- print STDERR "Warning: file $file#$ln:\n\t$msg";
-
- if ($data ne "") {
- print STDERR ". Line\n\t\t$data";
- } else {
- print STDERR "\n";
- }
-}
-
-#
-# Parse an ABI file, storing its contents at %data
-#
-sub parse_abi {
- my $file = $File::Find::name;
-
- my $mode = (stat($file))[2];
- return if ($mode & S_IFDIR);
- return if ($file =~ m,/README,);
- return if ($file =~ m,/\.,);
- return if ($file =~ m,\.(rej|org|orig|bak)$,);
-
- my $name = $file;
- $name =~ s,.*/,,;
-
- my $fn = $file;
- $fn =~ s,.*Documentation/ABI/,,;
-
- my $nametag = "File $fn";
- $data{$nametag}->{what} = "File $name";
- $data{$nametag}->{type} = "File";
- $data{$nametag}->{file} = $name;
- $data{$nametag}->{filepath} = $file;
- $data{$nametag}->{is_file} = 1;
- $data{$nametag}->{line_no} = 1;
-
- my $type = $file;
- $type =~ s,.*/(.*)/.*,$1,;
-
- my $what;
- my $new_what;
- my $tag = "";
- my $ln;
- my $xrefs;
- my $space;
- my @labels;
- my $label = "";
-
- print STDERR "Opening $file\n" if ($debug & $dbg_what_open);
- open IN, $file;
- while(<IN>) {
- $ln++;
- if (m/^(\S+)(:\s*)(.*)/i) {
- my $new_tag = lc($1);
- my $sep = $2;
- my $content = $3;
-
- if (!($new_tag =~ m/(what|where|date|kernelversion|contact|description|users)/)) {
- if ($tag eq "description") {
- # New "tag" is actually part of
- # description. Don't consider it a tag
- $new_tag = "";
- } elsif ($tag ne "") {
- parse_error($file, $ln, "tag '$tag' is invalid", $_);
- }
- }
-
- # Invalid, but it is a common mistake
- if ($new_tag eq "where") {
- parse_error($file, $ln, "tag 'Where' is invalid. Should be 'What:' instead", "");
- $new_tag = "what";
- }
-
- if ($new_tag =~ m/what/) {
- $space = "";
- $content =~ s/[,.;]$//;
-
- push @{$symbols{$content}->{file}}, " $file:" . ($ln - 1);
-
- if ($tag =~ m/what/) {
- $what .= "\xac" . $content;
- } else {
- if ($what) {
- parse_error($file, $ln, "What '$what' doesn't have a description", "") if (!$data{$what}->{description});
-
- foreach my $w(split /\xac/, $what) {
- $symbols{$w}->{xref} = $what;
- };
- }
-
- $what = $content;
- $label = $content;
- $new_what = 1;
- }
- push @labels, [($content, $label)];
- $tag = $new_tag;
-
- push @{$data{$nametag}->{symbols}}, $content if ($data{$nametag}->{what});
- next;
- }
-
- if ($tag ne "" && $new_tag) {
- $tag = $new_tag;
-
- if ($new_what) {
- @{$data{$what}->{label_list}} = @labels if ($data{$nametag}->{what});
- @labels = ();
- $label = "";
- $new_what = 0;
-
- $data{$what}->{type} = $type;
- if (!defined($data{$what}->{file})) {
- $data{$what}->{file} = $name;
- $data{$what}->{filepath} = $file;
- } else {
- $data{$what}->{description} .= "\n\n" if (defined($data{$what}->{description}));
- if ($name ne $data{$what}->{file}) {
- $data{$what}->{file} .= " " . $name;
- $data{$what}->{filepath} .= " " . $file;
- }
- }
- print STDERR "\twhat: $what\n" if ($debug & $dbg_what_parsing);
- $data{$what}->{line_no} = $ln;
- } else {
- $data{$what}->{line_no} = $ln if (!defined($data{$what}->{line_no}));
- }
-
- if (!$what) {
- parse_error($file, $ln, "'What:' should come first:", $_);
- next;
- }
- if ($new_tag eq "description") {
- $sep =~ s,:, ,;
- $content = ' ' x length($new_tag) . $sep . $content;
- while ($content =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {}
- if ($content =~ m/^(\s*)(\S.*)$/) {
- # Preserve initial spaces for the first line
- $space = $1;
- $content = "$2\n";
- $data{$what}->{$tag} .= $content;
- } else {
- undef($space);
- }
-
- } else {
- $data{$what}->{$tag} = $content;
- }
- next;
- }
- }
-
- # Store any contents before tags at the database
- if (!$tag && $data{$nametag}->{what}) {
- $data{$nametag}->{description} .= $_;
- next;
- }
-
- if ($tag eq "description") {
- my $content = $_;
- while ($content =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {}
- if (m/^\s*\n/) {
- $data{$what}->{$tag} .= "\n";
- next;
- }
-
- if (!defined($space)) {
- # Preserve initial spaces for the first line
- if ($content =~ m/^(\s*)(\S.*)$/) {
- $space = $1;
- $content = "$2\n";
- }
- } else {
- $space = "" if (!($content =~ s/^($space)//));
- }
- $data{$what}->{$tag} .= $content;
-
- next;
- }
- if (m/^\s*(.*)/) {
- $data{$what}->{$tag} .= "\n$1";
- $data{$what}->{$tag} =~ s/\n+$//;
- next;
- }
-
- # Everything else is error
- parse_error($file, $ln, "Unexpected content", $_);
- }
- $data{$nametag}->{description} =~ s/^\n+// if ($data{$nametag}->{description});
- if ($what) {
- parse_error($file, $ln, "What '$what' doesn't have a description", "") if (!$data{$what}->{description});
-
- foreach my $w(split /\xac/,$what) {
- $symbols{$w}->{xref} = $what;
- };
- }
- close IN;
-}
-
-sub create_labels {
- my %labels;
-
- foreach my $what (keys %data) {
- next if ($data{$what}->{file} eq "File");
-
- foreach my $p (@{$data{$what}->{label_list}}) {
- my ($content, $label) = @{$p};
- $label = "abi_" . $label . " ";
- $label =~ tr/A-Z/a-z/;
-
- # Convert special chars to "_"
- $label =~s/([\x00-\x2f\x3a-\x40\x5b-\x60\x7b-\xff])/_/g;
- $label =~ s,_+,_,g;
- $label =~ s,_$,,;
-
- # Avoid duplicated labels
- while (defined($labels{$label})) {
- my @chars = ("A".."Z", "a".."z");
- $label .= $chars[rand @chars];
- }
- $labels{$label} = 1;
-
- $data{$what}->{label} = $label;
-
- # only one label is enough
- last;
- }
- }
-}
-
-#
-# Outputs the book on ReST format
-#
-
-# \b doesn't work well with paths. So, we need to define something else:
-# Boundaries are punct characters, spaces and end-of-line
-my $start = qr {(^|\s|\() }x;
-my $bondary = qr { ([,.:;\)\s]|\z) }x;
-my $xref_match = qr { $start(\/(sys|config|proc|dev|kvd)\/[^,.:;\)\s]+)$bondary }x;
-my $symbols = qr { ([\x01-\x08\x0e-\x1f\x21-\x2f\x3a-\x40\x7b-\xff]) }x;
-
-sub output_rest {
- create_labels();
-
- my $part = "";
-
- foreach my $what (sort {
- ($data{$a}->{type} eq "File") cmp ($data{$b}->{type} eq "File") ||
- $a cmp $b
- } keys %data) {
- my $type = $data{$what}->{type};
-
- my @file = split / /, $data{$what}->{file};
- my @filepath = split / /, $data{$what}->{filepath};
-
- if ($enable_lineno) {
- printf ".. LINENO %s%s#%s\n\n",
- $prefix, $file[0],
- $data{$what}->{line_no};
- }
-
- my $w = $what;
-
- if ($type ne "File") {
- my $cur_part = $what;
- if ($what =~ '/') {
- if ($what =~ m#^(\/?(?:[\w\-]+\/?){1,2})#) {
- $cur_part = "Symbols under $1";
- $cur_part =~ s,/$,,;
- }
- }
-
- if ($cur_part ne "" && $part ne $cur_part) {
- $part = $cur_part;
- my $bar = $part;
- $bar =~ s/./-/g;
- print "$part\n$bar\n\n";
- }
-
- printf ".. _%s:\n\n", $data{$what}->{label};
-
- my @names = split /\xac/,$w;
- my $len = 0;
-
- foreach my $name (@names) {
- $name =~ s/$symbols/\\$1/g;
- $name = "**$name**";
- $len = length($name) if (length($name) > $len);
- }
-
- print "+-" . "-" x $len . "-+\n";
- foreach my $name (@names) {
- printf "| %s", $name . " " x ($len - length($name)) . " |\n";
- print "+-" . "-" x $len . "-+\n";
- }
-
- print "\n";
- }
-
- for (my $i = 0; $i < scalar(@filepath); $i++) {
- my $path = $filepath[$i];
- my $f = $file[$i];
-
- $path =~ s,.*/(.*/.*),$1,;;
- $path =~ s,[/\-],_,g;;
- my $fileref = "abi_file_".$path;
-
- if ($type eq "File") {
- print ".. _$fileref:\n\n";
- } else {
- print "Defined on file :ref:`$f <$fileref>`\n\n";
- }
- }
-
- if ($type eq "File") {
- my $bar = $w;
- $bar =~ s/./-/g;
- print "$w\n$bar\n\n";
- }
-
- my $desc = "";
- $desc = $data{$what}->{description} if (defined($data{$what}->{description}));
- $desc =~ s/\s+$/\n/;
-
- if (!($desc =~ /^\s*$/)) {
- if ($description_is_rst) {
- # Remove title markups from the description
- # Having titles inside ABI files will only work if extra
- # care would be taken in order to strictly follow the same
- # level order for each markup.
- $desc =~ s/\n[\-\*\=\^\~]+\n/\n\n/g;
-
- # Enrich text by creating cross-references
-
- my $new_desc = "";
- my $init_indent = -1;
- my $literal_indent = -1;
-
- open(my $fh, "+<", \$desc);
- while (my $d = <$fh>) {
- my $indent = $d =~ m/^(\s+)/;
- my $spaces = length($indent);
- $init_indent = $indent if ($init_indent < 0);
- if ($literal_indent >= 0) {
- if ($spaces > $literal_indent) {
- $new_desc .= $d;
- next;
- } else {
- $literal_indent = -1;
- }
- } else {
- if ($d =~ /()::$/ && !($d =~ /^\s*\.\./)) {
- $literal_indent = $spaces;
- }
- }
-
- $d =~ s,Documentation/(?!devicetree)(\S+)\.rst,:doc:`/$1`,g;
-
- my @matches = $d =~ m,Documentation/ABI/([\w\/\-]+),g;
- foreach my $f (@matches) {
- my $xref = $f;
- my $path = $f;
- $path =~ s,.*/(.*/.*),$1,;;
- $path =~ s,[/\-],_,g;;
- $xref .= " <abi_file_" . $path . ">";
- $d =~ s,\bDocumentation/ABI/$f\b,:ref:`$xref`,g;
- }
-
- # Seek for cross reference symbols like /sys/...
- @matches = $d =~ m/$xref_match/g;
-
- foreach my $s (@matches) {
- next if (!($s =~ m,/,));
- if (defined($data{$s}) && defined($data{$s}->{label})) {
- my $xref = $s;
-
- $xref =~ s/$symbols/\\$1/g;
- $xref = ":ref:`$xref <" . $data{$s}->{label} . ">`";
-
- $d =~ s,$start$s$bondary,$1$xref$2,g;
- }
- }
- $new_desc .= $d;
- }
- close $fh;
-
-
- print "$new_desc\n\n";
- } else {
- $desc =~ s/^\s+//;
-
- # Remove title markups from the description, as they won't work
- $desc =~ s/\n[\-\*\=\^\~]+\n/\n\n/g;
-
- if ($desc =~ m/\:\n/ || $desc =~ m/\n[\t ]+/ || $desc =~ m/[\x00-\x08\x0b-\x1f\x7b-\xff]/) {
- # put everything inside a code block
- $desc =~ s/\n/\n /g;
-
- print "::\n\n";
- print " $desc\n\n";
- } else {
- # Escape any special chars from description
- $desc =~s/([\x00-\x08\x0b-\x1f\x21-\x2a\x2d\x2f\x3c-\x40\x5c\x5e-\x60\x7b-\xff])/\\$1/g;
- print "$desc\n\n";
- }
- }
- } else {
- print "DESCRIPTION MISSING for $what\n\n" if (!$data{$what}->{is_file});
- }
-
- if ($data{$what}->{symbols}) {
- printf "Has the following ABI:\n\n";
-
- foreach my $content(@{$data{$what}->{symbols}}) {
- my $label = $data{$symbols{$content}->{xref}}->{label};
-
- # Escape special chars from content
- $content =~s/([\x00-\x1f\x21-\x2f\x3a-\x40\x7b-\xff])/\\$1/g;
-
- print "- :ref:`$content <$label>`\n\n";
- }
- }
-
- if (defined($data{$what}->{users})) {
- my $users = $data{$what}->{users};
-
- $users =~ s/\n/\n\t/g;
- printf "Users:\n\t%s\n\n", $users if ($users ne "");
- }
-
- }
-}
-
-#
-# Searches for ABI symbols
-#
-sub search_symbols {
- foreach my $what (sort keys %data) {
- next if (!($what =~ m/($arg)/));
-
- my $type = $data{$what}->{type};
- next if ($type eq "File");
-
- my $file = $data{$what}->{filepath};
-
- $what =~ s/\xac/, /g;
- my $bar = $what;
- $bar =~ s/./-/g;
-
- print "\n$what\n$bar\n\n";
-
- my $kernelversion = $data{$what}->{kernelversion} if (defined($data{$what}->{kernelversion}));
- my $contact = $data{$what}->{contact} if (defined($data{$what}->{contact}));
- my $users = $data{$what}->{users} if (defined($data{$what}->{users}));
- my $date = $data{$what}->{date} if (defined($data{$what}->{date}));
- my $desc = $data{$what}->{description} if (defined($data{$what}->{description}));
-
- $kernelversion =~ s/^\s+// if ($kernelversion);
- $contact =~ s/^\s+// if ($contact);
- if ($users) {
- $users =~ s/^\s+//;
- $users =~ s/\n//g;
- }
- $date =~ s/^\s+// if ($date);
- $desc =~ s/^\s+// if ($desc);
-
- printf "Kernel version:\t\t%s\n", $kernelversion if ($kernelversion);
- printf "Date:\t\t\t%s\n", $date if ($date);
- printf "Contact:\t\t%s\n", $contact if ($contact);
- printf "Users:\t\t\t%s\n", $users if ($users);
- print "Defined on file(s):\t$file\n\n";
- print "Description:\n\n$desc";
- }
-}
-
-# Exclude /sys/kernel/debug and /sys/kernel/tracing from the search path
-sub dont_parse_special_attributes {
- if (($File::Find::dir =~ m,^/sys/kernel,)) {
- return grep {!/(debug|tracing)/ } @_;
- }
-
- if (($File::Find::dir =~ m,^/sys/fs,)) {
- return grep {!/(pstore|bpf|fuse)/ } @_;
- }
-
- return @_
-}
-
-my %leaf;
-my %aliases;
-my @files;
-my %root;
-
-sub graph_add_file {
- my $file = shift;
- my $type = shift;
-
- my $dir = $file;
- $dir =~ s,^(.*/).*,$1,;
- $file =~ s,.*/,,;
-
- my $name;
- my $file_ref = \%root;
- foreach my $edge(split "/", $dir) {
- $name .= "$edge/";
- if (!defined ${$file_ref}{$edge}) {
- ${$file_ref}{$edge} = { };
- }
- $file_ref = \%{$$file_ref{$edge}};
- ${$file_ref}{"__name"} = [ $name ];
- }
- $name .= "$file";
- ${$file_ref}{$file} = {
- "__name" => [ $name ]
- };
-
- return \%{$$file_ref{$file}};
-}
-
-sub graph_add_link {
- my $file = shift;
- my $link = shift;
-
- # Traverse graph to find the reference
- my $file_ref = \%root;
- foreach my $edge(split "/", $file) {
- $file_ref = \%{$$file_ref{$edge}} || die "Missing node!";
- }
-
- # do a BFS
-
- my @queue;
- my %seen;
- my $st;
-
- push @queue, $file_ref;
- $seen{$start}++;
-
- while (@queue) {
- my $v = shift @queue;
- my @child = keys(%{$v});
-
- foreach my $c(@child) {
- next if $seen{$$v{$c}};
- next if ($c eq "__name");
-
- if (!defined($$v{$c}{"__name"})) {
- printf STDERR "Error: Couldn't find a non-empty name on a children of $file/.*: ";
- print STDERR Dumper(%{$v});
- exit;
- }
-
- # Add new name
- my $name = @{$$v{$c}{"__name"}}[0];
- if ($name =~ s#^$file/#$link/#) {
- push @{$$v{$c}{"__name"}}, $name;
- }
- # Add child to the queue and mark as seen
- push @queue, $$v{$c};
- $seen{$c}++;
- }
- }
-}
-
-my $escape_symbols = qr { ([\x01-\x08\x0e-\x1f\x21-\x29\x2b-\x2d\x3a-\x40\x7b-\xfe]) }x;
-sub parse_existing_sysfs {
- my $file = $File::Find::name;
-
- my $mode = (lstat($file))[2];
- my $abs_file = abs_path($file);
-
- my @tmp;
- push @tmp, $file;
- push @tmp, $abs_file if ($abs_file ne $file);
-
- foreach my $f(@tmp) {
- # Ignore cgroup, as this is big and has zero docs under ABI
- return if ($f =~ m#^/sys/fs/cgroup/#);
-
- # Ignore firmware as it is documented elsewhere
- # Either ACPI or under Documentation/devicetree/bindings/
- return if ($f =~ m#^/sys/firmware/#);
-
- # Ignore some sysfs nodes that aren't actually part of ABI
- return if ($f =~ m#/sections|notes/#);
-
- # Would need to check at
- # Documentation/admin-guide/kernel-parameters.txt, but this
- # is not easily parseable.
- return if ($f =~ m#/parameters/#);
- }
-
- if (S_ISLNK($mode)) {
- $aliases{$file} = $abs_file;
- return;
- }
-
- return if (S_ISDIR($mode));
-
- # Trivial: file is defined exactly the same way at ABI What:
- return if (defined($data{$file}));
- return if (defined($data{$abs_file}));
-
- push @files, graph_add_file($abs_file, "file");
-}
-
-sub get_leave($)
-{
- my $what = shift;
- my $leave;
-
- my $l = $what;
- my $stop = 1;
-
- $leave = $l;
- $leave =~ s,/$,,;
- $leave =~ s,.*/,,;
- $leave =~ s/[\(\)]//g;
-
- # $leave is used to improve search performance at
- # check_undefined_symbols, as the algorithm there can seek
- # for a small number of "what". It also allows giving a
- # hint about a leave with the same name somewhere else.
- # However, there are a few occurences where the leave is
- # either a wildcard or a number. Just group such cases
- # altogether.
- if ($leave =~ m/\.\*/ || $leave eq "" || $leave =~ /\\d/) {
- $leave = "others";
- }
-
- return $leave;
-}
-
-my @not_found;
-
-sub check_file($$)
-{
- my $file_ref = shift;
- my $names_ref = shift;
- my @names = @{$names_ref};
- my $file = $names[0];
-
- my $found_string;
-
- my $leave = get_leave($file);
- if (!defined($leaf{$leave})) {
- $leave = "others";
- }
- my @expr = @{$leaf{$leave}->{expr}};
- die ("\rmissing rules for $leave") if (!defined($leaf{$leave}));
-
- my $path = $file;
- $path =~ s,(.*/).*,$1,;
-
- if ($search_string) {
- return if (!($file =~ m#$search_string#));
- $found_string = 1;
- }
-
- for (my $i = 0; $i < @names; $i++) {
- if ($found_string && $hint) {
- if (!$i) {
- print STDERR "--> $names[$i]\n";
- } else {
- print STDERR " $names[$i]\n";
- }
- }
- foreach my $re (@expr) {
- print STDERR "$names[$i] =~ /^$re\$/\n" if ($debug && $dbg_undefined);
- if ($names[$i] =~ $re) {
- return;
- }
- }
- }
-
- if ($leave ne "others") {
- my @expr = @{$leaf{"others"}->{expr}};
- for (my $i = 0; $i < @names; $i++) {
- foreach my $re (@expr) {
- print STDERR "$names[$i] =~ /^$re\$/\n" if ($debug && $dbg_undefined);
- if ($names[$i] =~ $re) {
- return;
- }
- }
- }
- }
-
- push @not_found, $file if (!$search_string || $found_string);
-
- if ($hint && (!$search_string || $found_string)) {
- my $what = $leaf{$leave}->{what};
- $what =~ s/\xac/\n\t/g;
- if ($leave ne "others") {
- print STDERR "\r more likely regexes:\n\t$what\n";
- } else {
- print STDERR "\r tested regexes:\n\t$what\n";
- }
- }
-}
-
-sub check_undefined_symbols {
- my $num_files = scalar @files;
- my $next_i = 0;
- my $start_time = times;
-
- @files = sort @files;
-
- my $last_time = $start_time;
-
- # When either debug or hint is enabled, there's no sense showing
- # progress, as the progress will be overriden.
- if ($hint || ($debug && $dbg_undefined)) {
- $next_i = $num_files;
- }
-
- my $is_console;
- $is_console = 1 if (-t STDERR);
-
- for (my $i = 0; $i < $num_files; $i++) {
- my $file_ref = $files[$i];
- my @names = @{$$file_ref{"__name"}};
-
- check_file($file_ref, \@names);
-
- my $cur_time = times;
-
- if ($i == $next_i || $cur_time > $last_time + 1) {
- my $percent = $i * 100 / $num_files;
-
- my $tm = $cur_time - $start_time;
- my $time = sprintf "%d:%02d", int($tm), 60 * ($tm - int($tm));
-
- printf STDERR "\33[2K\r", if ($is_console);
- printf STDERR "%s: processing sysfs files... %i%%: $names[0]", $time, $percent;
- printf STDERR "\n", if (!$is_console);
- STDERR->flush();
-
- $next_i = int (($percent + 1) * $num_files / 100);
- $last_time = $cur_time;
- }
- }
-
- my $cur_time = times;
- my $tm = $cur_time - $start_time;
- my $time = sprintf "%d:%02d", int($tm), 60 * ($tm - int($tm));
-
- printf STDERR "\33[2K\r", if ($is_console);
- printf STDERR "%s: processing sysfs files... done\n", $time;
-
- foreach my $file (@not_found) {
- print "$file not found.\n";
- }
-}
-
-sub undefined_symbols {
- print STDERR "Reading $sysfs_prefix directory contents...";
- find({
- wanted =>\&parse_existing_sysfs,
- preprocess =>\&dont_parse_special_attributes,
- no_chdir => 1
- }, $sysfs_prefix);
- print STDERR "done.\n";
-
- $leaf{"others"}->{what} = "";
-
- print STDERR "Converting ABI What fields into regexes...";
- foreach my $w (sort keys %data) {
- foreach my $what (split /\xac/,$w) {
- next if (!($what =~ m/^$sysfs_prefix/));
-
- # Convert what into regular expressions
-
- # Escape dot characters
- $what =~ s/\./\xf6/g;
-
- # Temporarily change [0-9]+ type of patterns
- $what =~ s/\[0\-9\]\+/\xff/g;
-
- # Temporarily change [\d+-\d+] type of patterns
- $what =~ s/\[0\-\d+\]/\xff/g;
- $what =~ s/\[(\d+)\]/\xf4$1\xf5/g;
-
- # Temporarily change [0-9] type of patterns
- $what =~ s/\[(\d)\-(\d)\]/\xf4$1-$2\xf5/g;
-
- # Handle multiple option patterns
- $what =~ s/[\{\<\[]([\w_]+)(?:[,|]+([\w_]+)){1,}[\}\>\]]/($1|$2)/g;
-
- # Handle wildcards
- $what =~ s,\*,.*,g;
- $what =~ s,/\xf6..,/.*,g;
- $what =~ s/\<[^\>]+\>/.*/g;
- $what =~ s/\{[^\}]+\}/.*/g;
- $what =~ s/\[[^\]]+\]/.*/g;
-
- $what =~ s/[XYZ]/.*/g;
-
- # Recover [0-9] type of patterns
- $what =~ s/\xf4/[/g;
- $what =~ s/\xf5/]/g;
-
- # Remove duplicated spaces
- $what =~ s/\s+/ /g;
-
- # Special case: this ABI has a parenthesis on it
- $what =~ s/sqrt\(x^2\+y^2\+z^2\)/sqrt\(x^2\+y^2\+z^2\)/;
-
- # Special case: drop comparition as in:
- # What: foo = <something>
- # (this happens on a few IIO definitions)
- $what =~ s,\s*\=.*$,,;
-
- # Escape all other symbols
- $what =~ s/$escape_symbols/\\$1/g;
- $what =~ s/\\\\/\\/g;
- $what =~ s/\\([\[\]\(\)\|])/$1/g;
- $what =~ s/(\d+)\\(-\d+)/$1$2/g;
-
- $what =~ s/\xff/\\d+/g;
-
- # Special case: IIO ABI which a parenthesis.
- $what =~ s/sqrt(.*)/sqrt\(.*\)/;
-
- # Simplify regexes with multiple .*
- $what =~ s#(?:\.\*){2,}##g;
-# $what =~ s#\.\*/\.\*#.*#g;
-
- # Recover dot characters
- $what =~ s/\xf6/\./g;
-
- my $leave = get_leave($what);
-
- my $added = 0;
- foreach my $l (split /\|/, $leave) {
- if (defined($leaf{$l})) {
- next if ($leaf{$l}->{what} =~ m/\b$what\b/);
- $leaf{$l}->{what} .= "\xac" . $what;
- $added = 1;
- } else {
- $leaf{$l}->{what} = $what;
- $added = 1;
- }
- }
- if ($search_string && $added) {
- print STDERR "What: $what\n" if ($what =~ m#$search_string#);
- }
-
- }
- }
- # Compile regexes
- foreach my $l (sort keys %leaf) {
- my @expr;
- foreach my $w(sort split /\xac/, $leaf{$l}->{what}) {
- push @expr, qr /^$w$/;
- }
- $leaf{$l}->{expr} = \@expr;
- }
-
- # Take links into account
- foreach my $link (sort keys %aliases) {
- my $abs_file = $aliases{$link};
- graph_add_link($abs_file, $link);
- }
- print STDERR "done.\n";
-
- check_undefined_symbols;
-}
-
-# Ensure that the prefix will always end with a slash
-# While this is not needed for find, it makes the patch nicer
-# with --enable-lineno
-$prefix =~ s,/?$,/,;
-
-if ($cmd eq "undefined" || $cmd eq "search") {
- $show_warnings = 0;
-}
-#
-# Parses all ABI files located at $prefix dir
-#
-find({wanted =>\&parse_abi, no_chdir => 1}, $prefix);
-
-print STDERR Data::Dumper->Dump([\%data], [qw(*data)]) if ($debug & $dbg_dump_abi_structs);
-
-#
-# Handles the command
-#
-if ($cmd eq "undefined") {
- undefined_symbols;
-} elsif ($cmd eq "search") {
- search_symbols;
-} else {
- if ($cmd eq "rest") {
- output_rest;
- }
-
- # Warn about duplicated ABI entries
- foreach my $what(sort keys %symbols) {
- my @files = @{$symbols{$what}->{file}};
-
- next if (scalar(@files) == 1);
-
- printf STDERR "Warning: $what is defined %d times: @files\n",
- scalar(@files);
- }
-}
-
-__END__
-
-=head1 NAME
-
-get_abi.pl - parse the Linux ABI files and produce a ReST book.
-
-=head1 SYNOPSIS
-
-B<get_abi.pl> [--debug <level>] [--enable-lineno] [--man] [--help]
- [--(no-)rst-source] [--dir=<dir>] [--show-hints]
- [--search-string <regex>]
- <COMMAND> [<ARGUMENT>]
-
-Where B<COMMAND> can be:
-
-=over 8
-
-B<search> I<SEARCH_REGEX> - search for I<SEARCH_REGEX> inside ABI
-
-B<rest> - output the ABI in ReST markup language
-
-B<validate> - validate the ABI contents
-
-B<undefined> - existing symbols at the system that aren't
- defined at Documentation/ABI
-
-=back
-
-=head1 OPTIONS
-
-=over 8
-
-=item B<--dir>
-
-Changes the location of the ABI search. By default, it uses
-the Documentation/ABI directory.
-
-=item B<--rst-source> and B<--no-rst-source>
-
-The input file may be using ReST syntax or not. Those two options allow
-selecting between a rst-compliant source ABI (B<--rst-source>), or a
-plain text that may be violating ReST spec, so it requres some escaping
-logic (B<--no-rst-source>).
-
-=item B<--enable-lineno>
-
-Enable output of .. LINENO lines.
-
-=item B<--debug> I<debug level>
-
-Print debug information according with the level, which is given by the
-following bitmask:
-
- - 1: Debug parsing What entries from ABI files;
- - 2: Shows what files are opened from ABI files;
- - 4: Dump the structs used to store the contents of the ABI files.
-
-=item B<--show-hints>
-
-Show hints about possible definitions for the missing ABI symbols.
-Used only when B<undefined>.
-
-=item B<--search-string> I<regex string>
-
-Show only occurences that match a search string.
-Used only when B<undefined>.
-
-=item B<--help>
-
-Prints a brief help message and exits.
-
-=item B<--man>
-
-Prints the manual page and exits.
-
-=back
-
-=head1 DESCRIPTION
-
-Parse the Linux ABI files from ABI DIR (usually located at Documentation/ABI),
-allowing to search for ABI symbols or to produce a ReST book containing
-the Linux ABI documentation.
-
-=head1 EXAMPLES
-
-Search for all stable symbols with the word "usb":
-
-=over 8
-
-$ scripts/get_abi.pl search usb --dir Documentation/ABI/stable
-
-=back
-
-Search for all symbols that match the regex expression "usb.*cap":
-
-=over 8
-
-$ scripts/get_abi.pl search usb.*cap
-
-=back
-
-Output all obsoleted symbols in ReST format
-
-=over 8
-
-$ scripts/get_abi.pl rest --dir Documentation/ABI/obsolete
-
-=back
-
-=head1 BUGS
-
-Report bugs to Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
-
-=head1 COPYRIGHT
-
-Copyright (c) 2016-2021 by Mauro Carvalho Chehab <mchehab+huawei@kernel.org>.
-
-License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
-
-This is free software: you are free to change and redistribute it.
-There is NO WARRANTY, to the extent permitted by law.
-
-=cut
--
2.48.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* Re: [PATCH 05/27] ABI: sysfs-bus-coresight-*: fix kernelversion tags
2025-02-10 10:17 ` [PATCH 05/27] ABI: sysfs-bus-coresight-*: " Mauro Carvalho Chehab
@ 2025-02-10 10:33 ` Suzuki K Poulose
0 siblings, 0 replies; 32+ messages in thread
From: Suzuki K Poulose @ 2025-02-10 10:33 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Linux Doc Mailing List
Cc: Jonathan Corbet, James Clark, Mike Leach, coresight,
linux-arm-kernel, linux-kernel
On 10/02/2025 10:17, Mauro Carvalho Chehab wrote:
> Some kernelversion tags are missing colons. Add them to comply with
> ABI description and produce right results when converted to html/pdf.
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> ---
> .../testing/sysfs-bus-coresight-devices-cti | 78 +++++++++----------
> .../testing/sysfs-bus-coresight-devices-tpdm | 52 ++++++-------
> 2 files changed, 65 insertions(+), 65 deletions(-)
Acked-by: Suzuki K Poulose <suzuki.poulose@arm.com>
>
> diff --git a/Documentation/ABI/testing/sysfs-bus-coresight-devices-cti b/Documentation/ABI/testing/sysfs-bus-coresight-devices-cti
> index bf2869c413e7..a97b70f588da 100644
> --- a/Documentation/ABI/testing/sysfs-bus-coresight-devices-cti
> +++ b/Documentation/ABI/testing/sysfs-bus-coresight-devices-cti
> @@ -1,241 +1,241 @@
> What: /sys/bus/coresight/devices/<cti-name>/enable
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (RW) Enable/Disable the CTI hardware.
>
> What: /sys/bus/coresight/devices/<cti-name>/powered
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Read) Indicate if the CTI hardware is powered.
>
> What: /sys/bus/coresight/devices/<cti-name>/ctmid
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Read) Display the associated CTM ID
>
> What: /sys/bus/coresight/devices/<cti-name>/nr_trigger_cons
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Read) Number of devices connected to triggers on this CTI
>
> What: /sys/bus/coresight/devices/<cti-name>/triggers<N>/name
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Read) Name of connected device <N>
>
> What: /sys/bus/coresight/devices/<cti-name>/triggers<N>/in_signals
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Read) Input trigger signals from connected device <N>
>
> What: /sys/bus/coresight/devices/<cti-name>/triggers<N>/in_types
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Read) Functional types for the input trigger signals
> from connected device <N>
>
> What: /sys/bus/coresight/devices/<cti-name>/triggers<N>/out_signals
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Read) Output trigger signals to connected device <N>
>
> What: /sys/bus/coresight/devices/<cti-name>/triggers<N>/out_types
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Read) Functional types for the output trigger signals
> to connected device <N>
>
> What: /sys/bus/coresight/devices/<cti-name>/regs/inout_sel
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (RW) Select the index for inen and outen registers.
>
> What: /sys/bus/coresight/devices/<cti-name>/regs/inen
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (RW) Read or write the CTIINEN register selected by inout_sel.
>
> What: /sys/bus/coresight/devices/<cti-name>/regs/outen
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (RW) Read or write the CTIOUTEN register selected by inout_sel.
>
> What: /sys/bus/coresight/devices/<cti-name>/regs/gate
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (RW) Read or write CTIGATE register.
>
> What: /sys/bus/coresight/devices/<cti-name>/regs/asicctl
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (RW) Read or write ASICCTL register.
>
> What: /sys/bus/coresight/devices/<cti-name>/regs/intack
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Write) Write the INTACK register.
>
> What: /sys/bus/coresight/devices/<cti-name>/regs/appset
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (RW) Set CTIAPPSET register to activate channel. Read back to
> determine current value of register.
>
> What: /sys/bus/coresight/devices/<cti-name>/regs/appclear
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Write) Write APPCLEAR register to deactivate channel.
>
> What: /sys/bus/coresight/devices/<cti-name>/regs/apppulse
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Write) Write APPPULSE to pulse a channel active for one clock
> cycle.
>
> What: /sys/bus/coresight/devices/<cti-name>/regs/chinstatus
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Read) Read current status of channel inputs.
>
> What: /sys/bus/coresight/devices/<cti-name>/regs/choutstatus
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Read) read current status of channel outputs.
>
> What: /sys/bus/coresight/devices/<cti-name>/regs/triginstatus
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Read) read current status of input trigger signals
>
> What: /sys/bus/coresight/devices/<cti-name>/regs/trigoutstatus
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Read) read current status of output trigger signals.
>
> What: /sys/bus/coresight/devices/<cti-name>/channels/trigin_attach
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Write) Attach a CTI input trigger to a CTM channel.
>
> What: /sys/bus/coresight/devices/<cti-name>/channels/trigin_detach
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Write) Detach a CTI input trigger from a CTM channel.
>
> What: /sys/bus/coresight/devices/<cti-name>/channels/trigout_attach
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Write) Attach a CTI output trigger to a CTM channel.
>
> What: /sys/bus/coresight/devices/<cti-name>/channels/trigout_detach
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Write) Detach a CTI output trigger from a CTM channel.
>
> What: /sys/bus/coresight/devices/<cti-name>/channels/chan_gate_enable
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (RW) Enable CTIGATE for single channel (Write) or list enabled
> channels through the gate (R).
>
> What: /sys/bus/coresight/devices/<cti-name>/channels/chan_gate_disable
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Write) Disable CTIGATE for single channel.
>
> What: /sys/bus/coresight/devices/<cti-name>/channels/chan_set
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Write) Activate a single channel.
>
> What: /sys/bus/coresight/devices/<cti-name>/channels/chan_clear
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Write) Deactivate a single channel.
>
> What: /sys/bus/coresight/devices/<cti-name>/channels/chan_pulse
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Write) Pulse a single channel - activate for a single clock cycle.
>
> What: /sys/bus/coresight/devices/<cti-name>/channels/trigout_filtered
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Read) List of output triggers filtered across all connections.
>
> What: /sys/bus/coresight/devices/<cti-name>/channels/trig_filter_enable
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (RW) Enable or disable trigger output signal filtering.
>
> What: /sys/bus/coresight/devices/<cti-name>/channels/chan_inuse
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Read) show channels with at least one attached trigger signal.
>
> What: /sys/bus/coresight/devices/<cti-name>/channels/chan_free
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Read) show channels with no attached trigger signals.
>
> What: /sys/bus/coresight/devices/<cti-name>/channels/chan_xtrigs_sel
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (RW) Write channel number to select a channel to view, read to
> see selected channel number.
>
> What: /sys/bus/coresight/devices/<cti-name>/channels/chan_xtrigs_in
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Read) Read to see input triggers connected to selected view
> channel.
>
> What: /sys/bus/coresight/devices/<cti-name>/channels/chan_xtrigs_out
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Read) Read to see output triggers connected to selected view
> channel.
>
> What: /sys/bus/coresight/devices/<cti-name>/channels/chan_xtrigs_reset
> Date: March 2020
> -KernelVersion 5.7
> +KernelVersion: 5.7
> Contact: Mike Leach or Mathieu Poirier
> Description: (Write) Clear all channel / trigger programming.
> diff --git a/Documentation/ABI/testing/sysfs-bus-coresight-devices-tpdm b/Documentation/ABI/testing/sysfs-bus-coresight-devices-tpdm
> index bf710ea6e0ef..53cb454b60d0 100644
> --- a/Documentation/ABI/testing/sysfs-bus-coresight-devices-tpdm
> +++ b/Documentation/ABI/testing/sysfs-bus-coresight-devices-tpdm
> @@ -1,6 +1,6 @@
> What: /sys/bus/coresight/devices/<tpdm-name>/integration_test
> Date: January 2023
> -KernelVersion 6.2
> +KernelVersion: 6.2
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> (Write) Run integration test for tpdm. Integration test
> @@ -14,7 +14,7 @@ Description:
>
> What: /sys/bus/coresight/devices/<tpdm-name>/reset_dataset
> Date: March 2023
> -KernelVersion 6.7
> +KernelVersion: 6.7
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> (Write) Reset the dataset of the tpdm.
> @@ -24,7 +24,7 @@ Description:
>
> What: /sys/bus/coresight/devices/<tpdm-name>/dsb_trig_type
> Date: March 2023
> -KernelVersion 6.7
> +KernelVersion: 6.7
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> (RW) Set/Get the trigger type of the DSB for tpdm.
> @@ -35,7 +35,7 @@ Description:
>
> What: /sys/bus/coresight/devices/<tpdm-name>/dsb_trig_ts
> Date: March 2023
> -KernelVersion 6.7
> +KernelVersion: 6.7
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> (RW) Set/Get the trigger timestamp of the DSB for tpdm.
> @@ -46,7 +46,7 @@ Description:
>
> What: /sys/bus/coresight/devices/<tpdm-name>/dsb_mode
> Date: March 2023
> -KernelVersion 6.7
> +KernelVersion: 6.7
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> (RW) Set/Get the programming mode of the DSB for tpdm.
> @@ -60,7 +60,7 @@ Description:
>
> What: /sys/bus/coresight/devices/<tpdm-name>/dsb_edge/ctrl_idx
> Date: March 2023
> -KernelVersion 6.7
> +KernelVersion: 6.7
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> (RW) Set/Get the index number of the edge detection for the DSB
> @@ -69,7 +69,7 @@ Description:
>
> What: /sys/bus/coresight/devices/<tpdm-name>/dsb_edge/ctrl_val
> Date: March 2023
> -KernelVersion 6.7
> +KernelVersion: 6.7
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> Write a data to control the edge detection corresponding to
> @@ -85,7 +85,7 @@ Description:
>
> What: /sys/bus/coresight/devices/<tpdm-name>/dsb_edge/ctrl_mask
> Date: March 2023
> -KernelVersion 6.7
> +KernelVersion: 6.7
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> Write a data to mask the edge detection corresponding to the index
> @@ -97,21 +97,21 @@ Description:
>
> What: /sys/bus/coresight/devices/<tpdm-name>/dsb_edge/edcr[0:15]
> Date: March 2023
> -KernelVersion 6.7
> +KernelVersion: 6.7
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> Read a set of the edge control value of the DSB in TPDM.
>
> What: /sys/bus/coresight/devices/<tpdm-name>/dsb_edge/edcmr[0:7]
> Date: March 2023
> -KernelVersion 6.7
> +KernelVersion: 6.7
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> Read a set of the edge control mask of the DSB in TPDM.
>
> What: /sys/bus/coresight/devices/<tpdm-name>/dsb_trig_patt/xpr[0:7]
> Date: March 2023
> -KernelVersion 6.7
> +KernelVersion: 6.7
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> (RW) Set/Get the value of the trigger pattern for the DSB
> @@ -119,7 +119,7 @@ Description:
>
> What: /sys/bus/coresight/devices/<tpdm-name>/dsb_trig_patt/xpmr[0:7]
> Date: March 2023
> -KernelVersion 6.7
> +KernelVersion: 6.7
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> (RW) Set/Get the mask of the trigger pattern for the DSB
> @@ -127,21 +127,21 @@ Description:
>
> What: /sys/bus/coresight/devices/<tpdm-name>/dsb_patt/tpr[0:7]
> Date: March 2023
> -KernelVersion 6.7
> +KernelVersion: 6.7
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> (RW) Set/Get the value of the pattern for the DSB subunit TPDM.
>
> What: /sys/bus/coresight/devices/<tpdm-name>/dsb_patt/tpmr[0:7]
> Date: March 2023
> -KernelVersion 6.7
> +KernelVersion: 6.7
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> (RW) Set/Get the mask of the pattern for the DSB subunit TPDM.
>
> What: /sys/bus/coresight/devices/<tpdm-name>/dsb_patt/enable_ts
> Date: March 2023
> -KernelVersion 6.7
> +KernelVersion: 6.7
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> (Write) Set the pattern timestamp of DSB tpdm. Read
> @@ -153,7 +153,7 @@ Description:
>
> What: /sys/bus/coresight/devices/<tpdm-name>/dsb_patt/set_type
> Date: March 2023
> -KernelVersion 6.7
> +KernelVersion: 6.7
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> (Write) Set the pattern type of DSB tpdm. Read
> @@ -165,7 +165,7 @@ Description:
>
> What: /sys/bus/coresight/devices/<tpdm-name>/dsb_msr/msr[0:31]
> Date: March 2023
> -KernelVersion 6.7
> +KernelVersion: 6.7
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> (RW) Set/Get the MSR(mux select register) for the DSB subunit
> @@ -173,7 +173,7 @@ Description:
>
> What: /sys/bus/coresight/devices/<tpdm-name>/cmb_mode
> Date: January 2024
> -KernelVersion 6.9
> +KernelVersion: 6.9
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description: (Write) Set the data collection mode of CMB tpdm. Continuous
> change creates CMB data set elements on every CMBCLK edge.
> @@ -187,7 +187,7 @@ Description: (Write) Set the data collection mode of CMB tpdm. Continuous
>
> What: /sys/bus/coresight/devices/<tpdm-name>/cmb_trig_patt/xpr[0:1]
> Date: January 2024
> -KernelVersion 6.9
> +KernelVersion: 6.9
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> (RW) Set/Get the value of the trigger pattern for the CMB
> @@ -195,7 +195,7 @@ Description:
>
> What: /sys/bus/coresight/devices/<tpdm-name>/cmb_trig_patt/xpmr[0:1]
> Date: January 2024
> -KernelVersion 6.9
> +KernelVersion: 6.9
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> (RW) Set/Get the mask of the trigger pattern for the CMB
> @@ -203,21 +203,21 @@ Description:
>
> What: /sys/bus/coresight/devices/<tpdm-name>/dsb_patt/tpr[0:1]
> Date: January 2024
> -KernelVersion 6.9
> +KernelVersion: 6.9
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> (RW) Set/Get the value of the pattern for the CMB subunit TPDM.
>
> What: /sys/bus/coresight/devices/<tpdm-name>/dsb_patt/tpmr[0:1]
> Date: January 2024
> -KernelVersion 6.9
> +KernelVersion: 6.9
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> (RW) Set/Get the mask of the pattern for the CMB subunit TPDM.
>
> What: /sys/bus/coresight/devices/<tpdm-name>/cmb_patt/enable_ts
> Date: January 2024
> -KernelVersion 6.9
> +KernelVersion: 6.9
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> (Write) Set the pattern timestamp of CMB tpdm. Read
> @@ -229,7 +229,7 @@ Description:
>
> What: /sys/bus/coresight/devices/<tpdm-name>/cmb_trig_ts
> Date: January 2024
> -KernelVersion 6.9
> +KernelVersion: 6.9
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> (RW) Set/Get the trigger timestamp of the CMB for tpdm.
> @@ -240,7 +240,7 @@ Description:
>
> What: /sys/bus/coresight/devices/<tpdm-name>/cmb_ts_all
> Date: January 2024
> -KernelVersion 6.9
> +KernelVersion: 6.9
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> (RW) Read or write the status of timestamp upon all interface.
> @@ -252,7 +252,7 @@ Description:
>
> What: /sys/bus/coresight/devices/<tpdm-name>/cmb_msr/msr[0:31]
> Date: January 2024
> -KernelVersion 6.9
> +KernelVersion: 6.9
> Contact: Jinlong Mao (QUIC) <quic_jinlmao@quicinc.com>, Tao Zhang (QUIC) <quic_taozha@quicinc.com>
> Description:
> (RW) Set/Get the MSR(mux select register) for the CMB subunit
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH 00/27] Improve ABI documentation generation
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (26 preceding siblings ...)
2025-02-10 10:18 ` [PATCH 27/27] scripts/get_abi.pl: drop now obsoleted script Mauro Carvalho Chehab
@ 2025-02-10 18:30 ` Jonathan Corbet
2025-02-10 20:27 ` Mauro Carvalho Chehab
2025-03-27 21:58 ` [f2fs-dev] " patchwork-bot+f2fs
28 siblings, 1 reply; 32+ messages in thread
From: Jonathan Corbet @ 2025-02-10 18:30 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Linux Doc Mailing List
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, linux-kernel,
Guilherme G. Piccoli, Chao Yu, Jaegeuk Kim, James Clark,
Johannes Berg, Mike Leach, Suzuki K Poulose, Tony Luck, bpf,
coresight, linux-arm-kernel, linux-f2fs-devel, linux-hardening,
linux-wireless
Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
> Hi Jon/Greg,
>
> This series replace get_abi.pl with a Python version.
>
> I originally started it due to some issues I noticed when searching for
> ABI symbols. While I could just go ahead and fix the already existing
> script, I noticed that the script maintainance didn't have much care over
> all those years, probably because it is easier to find Python programmers
> those days.
>
> Also, the code is complex and was not using modules or classes and
> were using lots of global variables.
>
> So, I decided to rewrite it in Python. I started with a manual conversion
> for each function. Yet, to avoid future maintainership issues, I opted to
> divide the main code on three classes, each on a sepaparate file.
>
> Just like the original RFC, I opted to keep the Sphinx kernel-abi module
> on three different phases:
>
> - call get_abi.py as an exec file;
> - import AbiParser on a minimal integration scenario;
> - cleanup the code to avoid needing to parse line numbers from the text.
>
> This way, if something goes wrong, it would be easier to just revert any
> offending patches, It also provides a better rationale about what each
> logical change is doing.
>
> The initial patches on this series do some preparation work and
> cleans some ABI symbol bugs that lack ":" delimiter.
>
> I opted to place on this series the Sphinx and Python version updates.
>
> I still have ~10 patches here with additional cleanups, from the original
> series I sent as RFC but let's get the main changes merged first.
OK, I have applied this set - it seems to work for me, though it does
lead to some changes in the organization of
Documentation/admin-guide/abi.html in the output.
It would be nice if, eventually, we could put the README link up at the
top rather than under "ABI file", or even just include its contents
there directly.
Anyway, let's see how this goes :)
Thanks,
jon
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH 00/27] Improve ABI documentation generation
2025-02-10 18:30 ` [PATCH 00/27] Improve ABI documentation generation Jonathan Corbet
@ 2025-02-10 20:27 ` Mauro Carvalho Chehab
0 siblings, 0 replies; 32+ messages in thread
From: Mauro Carvalho Chehab @ 2025-02-10 20:27 UTC (permalink / raw)
To: Jonathan Corbet
Cc: Linux Doc Mailing List, linux-kernel, Guilherme G. Piccoli,
James Clark, Johannes Berg, Mike Leach, Suzuki K Poulose,
Tony Luck, bpf, coresight, linux-arm-kernel, linux-f2fs-devel,
linux-hardening, linux-wireless
Em Mon, 10 Feb 2025 11:30:46 -0700
Jonathan Corbet <corbet@lwn.net> escreveu:
> Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
>
> > Hi Jon/Greg,
> >
> > This series replace get_abi.pl with a Python version.
> >
> > I originally started it due to some issues I noticed when searching for
> > ABI symbols. While I could just go ahead and fix the already existing
> > script, I noticed that the script maintainance didn't have much care over
> > all those years, probably because it is easier to find Python programmers
> > those days.
> >
> > Also, the code is complex and was not using modules or classes and
> > were using lots of global variables.
> >
> > So, I decided to rewrite it in Python. I started with a manual conversion
> > for each function. Yet, to avoid future maintainership issues, I opted to
> > divide the main code on three classes, each on a sepaparate file.
> >
> > Just like the original RFC, I opted to keep the Sphinx kernel-abi module
> > on three different phases:
> >
> > - call get_abi.py as an exec file;
> > - import AbiParser on a minimal integration scenario;
> > - cleanup the code to avoid needing to parse line numbers from the text.
> >
> > This way, if something goes wrong, it would be easier to just revert any
> > offending patches, It also provides a better rationale about what each
> > logical change is doing.
> >
> > The initial patches on this series do some preparation work and
> > cleans some ABI symbol bugs that lack ":" delimiter.
> >
> > I opted to place on this series the Sphinx and Python version updates.
> >
> > I still have ~10 patches here with additional cleanups, from the original
> > series I sent as RFC but let's get the main changes merged first.
>
> OK, I have applied this set - it seems to work for me, though it does
> lead to some changes in the organization of
> Documentation/admin-guide/abi.html in the output.
Yes. I moved the files part to separate files, as IMHO this would
make easier for people to navigate.
> It would be nice if, eventually, we could put the README link up at the
> top rather than under "ABI file",
Moving its position is not hard: all we need to do is to change abi.rst
file. See the enclosed patch.
> or even just include its contents
> there directly.
>
> Anyway, let's see how this goes :)
>
> Thanks,
>
> jon
Thanks,
Mauro
From e1b864d1d333d94430420d1d6fc15ea7d8a99b4b Mon Sep 17 00:00:00 2001
From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Date: Mon, 10 Feb 2025 21:24:06 +0100
Subject: [PATCH] docs: ABI: move README contents to the top
The ABI documentation looks a little bit better if it starts
with the contents of the README is placed at the beginning.
Suggested-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
diff --git a/Documentation/admin-guide/abi-readme-file.rst b/Documentation/admin-guide/abi-readme-file.rst
deleted file mode 100644
index 6172e4ccbda2..000000000000
--- a/Documentation/admin-guide/abi-readme-file.rst
+++ /dev/null
@@ -1,6 +0,0 @@
-.. SPDX-License-Identifier: GPL-2.0
-
-ABI README
-==========
-
-.. kernel-abi:: README
diff --git a/Documentation/admin-guide/abi.rst b/Documentation/admin-guide/abi.rst
index 15a2dcb1388c..2067336353ae 100644
--- a/Documentation/admin-guide/abi.rst
+++ b/Documentation/admin-guide/abi.rst
@@ -4,6 +4,11 @@
Linux ABI description
=====================
+ABI README
+==========
+
+.. kernel-abi:: README
+
ABI symbols
-----------
@@ -21,7 +26,6 @@ ABI files
.. toctree::
:maxdepth: 2
- abi-readme-file
abi-stable-files
abi-testing-files
abi-obsolete-files
^ permalink raw reply related [flat|nested] 32+ messages in thread
* Re: [f2fs-dev] [PATCH 00/27] Improve ABI documentation generation
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
` (27 preceding siblings ...)
2025-02-10 18:30 ` [PATCH 00/27] Improve ABI documentation generation Jonathan Corbet
@ 2025-03-27 21:58 ` patchwork-bot+f2fs
28 siblings, 0 replies; 32+ messages in thread
From: patchwork-bot+f2fs @ 2025-03-27 21:58 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: linux-doc, tony.luck, corbet, james.clark, suzuki.poulose,
linux-wireless, linux-kernel, linux-f2fs-devel, gpiccoli,
linux-hardening, coresight, johannes, bpf, linux-arm-kernel,
mike.leach
Hello:
This patch was applied to jaegeuk/f2fs.git (dev)
by Jonathan Corbet <corbet@lwn.net>:
On Mon, 10 Feb 2025 11:17:49 +0100 you wrote:
> Hi Jon/Greg,
>
> This series replace get_abi.pl with a Python version.
>
> I originally started it due to some issues I noticed when searching for
> ABI symbols. While I could just go ahead and fix the already existing
> script, I noticed that the script maintainance didn't have much care over
> all those years, probably because it is easier to find Python programmers
> those days.
>
> [...]
Here is the summary with links:
- [f2fs-dev,07/27] ABI: sysfs-fs-f2fs: fix date tags
https://git.kernel.org/jaegeuk/f2fs/c/90800df0da78
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply [flat|nested] 32+ messages in thread
end of thread, other threads:[~2025-03-27 21:57 UTC | newest]
Thread overview: 32+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-02-10 10:17 [PATCH 00/27] Improve ABI documentation generation Mauro Carvalho Chehab
2025-02-10 10:17 ` [PATCH 01/27] docs: sphinx: remove kernellog.py file Mauro Carvalho Chehab
2025-02-10 10:17 ` [PATCH 02/27] docs: sphinx/kernel_abi: adjust coding style Mauro Carvalho Chehab
2025-02-10 10:17 ` [PATCH 03/27] docs: admin-guide: abi: add SPDX tags to ABI files Mauro Carvalho Chehab
2025-02-10 10:17 ` [PATCH 04/27] ABI: sysfs-class-rfkill: fix kernelversion tags Mauro Carvalho Chehab
2025-02-10 10:17 ` [PATCH 05/27] ABI: sysfs-bus-coresight-*: " Mauro Carvalho Chehab
2025-02-10 10:33 ` Suzuki K Poulose
2025-02-10 10:17 ` [PATCH 06/27] ABI: sysfs-driver-dma-idxd: fix date tags Mauro Carvalho Chehab
2025-02-10 10:17 ` [PATCH 07/27] ABI: sysfs-fs-f2fs: " Mauro Carvalho Chehab
2025-02-10 10:17 ` [PATCH 08/27] ABI: sysfs-power: fix a what tag Mauro Carvalho Chehab
2025-02-10 10:17 ` [PATCH 09/27] scripts/documentation-file-ref-check: don't check perl/python scripts Mauro Carvalho Chehab
2025-02-10 10:17 ` [PATCH 10/27] scripts/get_abi.py: add a Python tool to generate ReST output Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 11/27] scripts/get_abi.py: add support for symbol search Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 12/27] docs: use get_abi.py for ABI generation Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 13/27] scripts/lib/abi/abi_parser.py: optimize parse_abi() function Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 14/27] scripts/lib/abi/abi_parser.py: use an interactor for ReST output Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 15/27] docs: sphinx/kernel_abi: use AbiParser directly Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 16/27] docs: sphinx/kernel_abi: reduce buffer usage for ABI messages Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 17/27] docs: sphinx/kernel_abi: properly split lines Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 18/27] scripts/get_abi.pl: Add filtering capabilities to rest output Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 19/27] scripts/get_abi.pl: add support to parse ABI README file Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 20/27] docs: sphinx/kernel_abi: parse ABI files only once Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 21/27] docs: admin-guide/abi: split files from symbols Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 22/27] docs: sphinx/automarkup: add cross-references for ABI Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 23/27] docs: sphinx/kernel_abi: avoid warnings during Sphinx module init Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 24/27] scripts/lib/abi/abi_parser.py: Rename title name for ABI files Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 25/27] scripts/lib/abi/abi_parser.py: make it backward-compatible with Python 3.6 Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 26/27] scripts/get_abi.py: add support for undefined ABIs Mauro Carvalho Chehab
2025-02-10 10:18 ` [PATCH 27/27] scripts/get_abi.pl: drop now obsoleted script Mauro Carvalho Chehab
2025-02-10 18:30 ` [PATCH 00/27] Improve ABI documentation generation Jonathan Corbet
2025-02-10 20:27 ` Mauro Carvalho Chehab
2025-03-27 21:58 ` [f2fs-dev] " patchwork-bot+f2fs
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox