* [PATCH 0/9] A series of kernel-doc tweaks
@ 2025-06-06 16:34 Jonathan Corbet
2025-06-06 16:34 ` [PATCH 1/9] docs: kdoc: simplify the PROTO continuation logic Jonathan Corbet
` (9 more replies)
0 siblings, 10 replies; 22+ messages in thread
From: Jonathan Corbet @ 2025-06-06 16:34 UTC (permalink / raw)
To: linux-doc; +Cc: linux-kernel, Mauro Carvalho Chehab, Jonathan Corbet
I will freely confess that I merged the kernel-doc Python rewrite without
fully understanding the code; at the time, the fact that it worked as
advertised sufficed. I *do* feel the need to understand this code, though,
going forward, so I've dedicated some time to digging through it.
In the process, I've been making some adjustments to the code that, IMO,
make it a bit more approachable - for myself and, hopefully, for others.
The goal is to try to get functions to the point where people of limited
mind (like me) can soak them up, make the code slightly more Pythonic, and
removing redundant code.
Here is the first set of tweaks. The output from "make htmldocs" remains
entirely unchanged throughout the series. The docs build is slightly
faster afterward - but that's not the point.
Mauro, the more I dig into this the happier I am that you pushed this
change through - it was far overdue. Hopefully you don't mind me sweeping
up a bit around the edges...
Jonathan Corbet (9):
docs: kdoc: simplify the PROTO continuation logic
docs: kdoc: move the core dispatch into a state table
docs: kdoc: remove the section_intro variable
docs: kdoc: simplify the kerneldoc recognition code
docs: kdoc: remove the KernelEntry::is_kernel_comment member
docs: kdoc: remove the KernelEntry::descr pseudo member
docs: kdoc: remove some ineffective code
docs: kdoc: move the declaration regexes out of process_name()
docs: kdoc: some final touches for process_name()
scripts/lib/kdoc/kdoc_parser.py | 152 ++++++++++++++++----------------
1 file changed, 75 insertions(+), 77 deletions(-)
--
2.49.0
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 1/9] docs: kdoc: simplify the PROTO continuation logic
2025-06-06 16:34 [PATCH 0/9] A series of kernel-doc tweaks Jonathan Corbet
@ 2025-06-06 16:34 ` Jonathan Corbet
2025-06-07 9:59 ` Mauro Carvalho Chehab
2025-06-06 16:34 ` [PATCH 2/9] docs: kdoc: move the core dispatch into a state table Jonathan Corbet
` (8 subsequent siblings)
9 siblings, 1 reply; 22+ messages in thread
From: Jonathan Corbet @ 2025-06-06 16:34 UTC (permalink / raw)
To: linux-doc; +Cc: linux-kernel, Mauro Carvalho Chehab, Jonathan Corbet
Remove the unneeded "cont" variable and tighten up the code slightly.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
scripts/lib/kdoc/kdoc_parser.py | 7 +------
1 file changed, 1 insertion(+), 6 deletions(-)
diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index 062453eefc7a..b193cf9dc0d1 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -1680,7 +1680,6 @@ class KernelDoc:
Besides parsing kernel-doc tags, it also parses export symbols.
"""
- cont = False
prev = ""
prev_ln = None
export_table = set()
@@ -1696,18 +1695,14 @@ class KernelDoc:
if self.state == state.PROTO:
if line.endswith("\\"):
prev += line.rstrip("\\")
- cont = True
-
if not prev_ln:
prev_ln = ln
-
continue
- if cont:
+ if prev:
ln = prev_ln
line = prev + line
prev = ""
- cont = False
prev_ln = None
self.config.log.debug("%d %s%s: %s",
--
2.49.0
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 2/9] docs: kdoc: move the core dispatch into a state table
2025-06-06 16:34 [PATCH 0/9] A series of kernel-doc tweaks Jonathan Corbet
2025-06-06 16:34 ` [PATCH 1/9] docs: kdoc: simplify the PROTO continuation logic Jonathan Corbet
@ 2025-06-06 16:34 ` Jonathan Corbet
2025-06-07 10:00 ` Mauro Carvalho Chehab
2025-06-06 16:34 ` [PATCH 3/9] docs: kdoc: remove the section_intro variable Jonathan Corbet
` (7 subsequent siblings)
9 siblings, 1 reply; 22+ messages in thread
From: Jonathan Corbet @ 2025-06-06 16:34 UTC (permalink / raw)
To: linux-doc; +Cc: linux-kernel, Mauro Carvalho Chehab, Jonathan Corbet
Since all of the handlers already nicely have the same prototype, put them
into a table and call them from there and take out the extended
if-then-else series.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
scripts/lib/kdoc/kdoc_parser.py | 30 +++++++++++++++++-------------
1 file changed, 17 insertions(+), 13 deletions(-)
diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index b193cf9dc0d1..37b7e501af7c 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -1670,6 +1670,21 @@ class KernelDoc:
return export_table
+ #
+ # The state/action table telling us which function to invoke in
+ # each state.
+ #
+ state_actions = {
+ state.NORMAL: process_normal,
+ state.NAME: process_name,
+ state.BODY: process_body,
+ state.BODY_MAYBE: process_body,
+ state.BODY_WITH_BLANK_LINE: process_body,
+ state.INLINE: process_inline,
+ state.PROTO: process_proto,
+ state.DOCBLOCK: process_docblock,
+ }
+
def parse_kdoc(self):
"""
Open and process each line of a C source file.
@@ -1721,19 +1736,8 @@ class KernelDoc:
self.process_export(export_table, line)
# Hand this line to the appropriate state handler
- if self.state == state.NORMAL:
- self.process_normal(ln, line)
- elif self.state == state.NAME:
- self.process_name(ln, line)
- elif self.state in [state.BODY, state.BODY_MAYBE,
- state.BODY_WITH_BLANK_LINE]:
- self.process_body(ln, line)
- elif self.state == state.INLINE: # scanning for inline parameters
- self.process_inline(ln, line)
- elif self.state == state.PROTO:
- self.process_proto(ln, line)
- elif self.state == state.DOCBLOCK:
- self.process_docblock(ln, line)
+ self.state_actions[self.state](self, ln, line)
+
except OSError:
self.config.log.error(f"Error: Cannot open file {self.fname}")
--
2.49.0
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 3/9] docs: kdoc: remove the section_intro variable
2025-06-06 16:34 [PATCH 0/9] A series of kernel-doc tweaks Jonathan Corbet
2025-06-06 16:34 ` [PATCH 1/9] docs: kdoc: simplify the PROTO continuation logic Jonathan Corbet
2025-06-06 16:34 ` [PATCH 2/9] docs: kdoc: move the core dispatch into a state table Jonathan Corbet
@ 2025-06-06 16:34 ` Jonathan Corbet
2025-06-07 10:03 ` Mauro Carvalho Chehab
2025-06-06 16:34 ` [PATCH 4/9] docs: kdoc: simplify the kerneldoc recognition code Jonathan Corbet
` (6 subsequent siblings)
9 siblings, 1 reply; 22+ messages in thread
From: Jonathan Corbet @ 2025-06-06 16:34 UTC (permalink / raw)
To: linux-doc; +Cc: linux-kernel, Mauro Carvalho Chehab, Jonathan Corbet
It is only used in one place, so just put the constant string
"Introduction" there so people don't have to go looking for it.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
scripts/lib/kdoc/kdoc_parser.py | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index 37b7e501af7c..90b53b70cfee 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -203,7 +203,6 @@ class KernelDoc:
# Section names
- section_intro = "Introduction"
section_context = "Context"
section_return = "Return"
@@ -1207,7 +1206,7 @@ class KernelDoc:
self.entry.new_start_line = ln
if not doc_block.group(1):
- self.entry.section = self.section_intro
+ self.entry.section = "Introduction"
else:
self.entry.section = doc_block.group(1)
--
2.49.0
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 4/9] docs: kdoc: simplify the kerneldoc recognition code
2025-06-06 16:34 [PATCH 0/9] A series of kernel-doc tweaks Jonathan Corbet
` (2 preceding siblings ...)
2025-06-06 16:34 ` [PATCH 3/9] docs: kdoc: remove the section_intro variable Jonathan Corbet
@ 2025-06-06 16:34 ` Jonathan Corbet
2025-06-07 10:05 ` Mauro Carvalho Chehab
2025-06-06 16:34 ` [PATCH 5/9] docs: kdoc: remove the KernelEntry::is_kernel_comment member Jonathan Corbet
` (5 subsequent siblings)
9 siblings, 1 reply; 22+ messages in thread
From: Jonathan Corbet @ 2025-06-06 16:34 UTC (permalink / raw)
To: linux-doc; +Cc: linux-kernel, Mauro Carvalho Chehab, Jonathan Corbet
process_name() looks for the first line of a kerneldoc comment. It
contains two nearly identical regular expressions, the second of which only
catches six cases in the kernel, all of the form:
define SOME_MACRO_NAME - description
Simply put the "define" into the regex and discard it, eliminating the loop
and the code to remove it specially.
Note that this still treats these defines as if they were functions, but
that's a separate issue.
There is no change in the generated output.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
scripts/lib/kdoc/kdoc_parser.py | 24 ++++++++----------------
1 file changed, 8 insertions(+), 16 deletions(-)
diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index 90b53b70cfee..3ea260b423e2 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -1230,26 +1230,18 @@ class KernelDoc:
# Test for data declaration
r = KernRe(r"^\s*\*?\s*(struct|union|enum|typedef)\b\s*(\w*)")
+ r2 = KernRe(fr"^{decl_start}{fn_type}(?:define\s+)?(\w+)\s*{parenthesis}\s*{decl_end}?$")
if r.search(line):
self.entry.decl_type = r.group(1)
self.entry.identifier = r.group(2)
self.entry.is_kernel_comment = True
- else:
- # Look for foo() or static void foo() - description;
- # or misspelt identifier
-
- r1 = KernRe(fr"^{decl_start}{fn_type}(\w+)\s*{parenthesis}\s*{decl_end}?$")
- r2 = KernRe(fr"^{decl_start}{fn_type}(\w+[^-:]*){parenthesis}\s*{decl_end}$")
-
- for r in [r1, r2]:
- if r.search(line):
- self.entry.identifier = r.group(1)
- self.entry.decl_type = "function"
-
- r = KernRe(r"define\s+")
- self.entry.identifier = r.sub("", self.entry.identifier)
- self.entry.is_kernel_comment = True
- break
+ #
+ # Look for a function description
+ #
+ elif r2.search(line):
+ self.entry.identifier = r2.group(1)
+ self.entry.decl_type = "function"
+ self.entry.is_kernel_comment = True
self.entry.identifier = self.entry.identifier.strip(" ")
--
2.49.0
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 5/9] docs: kdoc: remove the KernelEntry::is_kernel_comment member
2025-06-06 16:34 [PATCH 0/9] A series of kernel-doc tweaks Jonathan Corbet
` (3 preceding siblings ...)
2025-06-06 16:34 ` [PATCH 4/9] docs: kdoc: simplify the kerneldoc recognition code Jonathan Corbet
@ 2025-06-06 16:34 ` Jonathan Corbet
2025-06-07 10:07 ` Mauro Carvalho Chehab
2025-06-06 16:34 ` [PATCH 6/9] docs: kdoc: remove the KernelEntry::descr pseudo member Jonathan Corbet
` (4 subsequent siblings)
9 siblings, 1 reply; 22+ messages in thread
From: Jonathan Corbet @ 2025-06-06 16:34 UTC (permalink / raw)
To: linux-doc; +Cc: linux-kernel, Mauro Carvalho Chehab, Jonathan Corbet
entry::is_kernel_comment never had anything to do with the entry itself; it
is a bit of local state in one branch of process_name(). It can, in fact,
be removed entirely; rework the code slightly so that it is no longer
needed.
No change in the rendered output.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
scripts/lib/kdoc/kdoc_parser.py | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index 3ea260b423e2..56299695abd1 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -1216,7 +1216,6 @@ class KernelDoc:
if doc_decl.search(line):
self.entry.identifier = doc_decl.group(1)
- self.entry.is_kernel_comment = False
decl_start = str(doc_com) # comment block asterisk
fn_type = r"(?:\w+\s*\*\s*)?" # type (for non-functions)
@@ -1234,14 +1233,20 @@ class KernelDoc:
if r.search(line):
self.entry.decl_type = r.group(1)
self.entry.identifier = r.group(2)
- self.entry.is_kernel_comment = True
#
# Look for a function description
#
elif r2.search(line):
self.entry.identifier = r2.group(1)
self.entry.decl_type = "function"
- self.entry.is_kernel_comment = True
+ #
+ # We struck out.
+ #
+ else:
+ self.emit_msg(ln,
+ f"This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst\n{line}")
+ self.state = state.NORMAL
+ return
self.entry.identifier = self.entry.identifier.strip(" ")
@@ -1263,11 +1268,6 @@ class KernelDoc:
else:
self.entry.declaration_purpose = ""
- if not self.entry.is_kernel_comment:
- self.emit_msg(ln,
- f"This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst\n{line}")
- self.state = state.NORMAL
-
if not self.entry.declaration_purpose and self.config.wshort_desc:
self.emit_msg(ln,
f"missing initial short description on line:\n{line}")
--
2.49.0
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 6/9] docs: kdoc: remove the KernelEntry::descr pseudo member
2025-06-06 16:34 [PATCH 0/9] A series of kernel-doc tweaks Jonathan Corbet
` (4 preceding siblings ...)
2025-06-06 16:34 ` [PATCH 5/9] docs: kdoc: remove the KernelEntry::is_kernel_comment member Jonathan Corbet
@ 2025-06-06 16:34 ` Jonathan Corbet
2025-06-07 10:14 ` Mauro Carvalho Chehab
2025-06-06 16:34 ` [PATCH 7/9] docs: kdoc: remove some ineffective code Jonathan Corbet
` (3 subsequent siblings)
9 siblings, 1 reply; 22+ messages in thread
From: Jonathan Corbet @ 2025-06-06 16:34 UTC (permalink / raw)
To: linux-doc; +Cc: linux-kernel, Mauro Carvalho Chehab, Jonathan Corbet
The entry.descr value used in process_name() is not actually a member of
the KernelEntry class; it is a bit of local state. So just manage it
locally.
A trim_whitespace() helper was added to clean up the code slightly.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
scripts/lib/kdoc/kdoc_parser.py | 14 ++++++++------
1 file changed, 8 insertions(+), 6 deletions(-)
diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index 56299695abd1..7c635000f3de 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -60,6 +60,13 @@ export_symbol_ns = KernRe(r'^\s*EXPORT_SYMBOL_NS(_GPL)?\s*\(\s*(\w+)\s*,\s*"\S+"
type_param = KernRe(r"\@(\w*((\.\w+)|(->\w+))*(\.\.\.)?)", cache=False)
+#
+# A little helper to get rid of excess white space
+#
+multi_space = KernRe(r'\s\s+')
+def trim_whitespace(s):
+ return multi_space.sub(' ', s.strip())
+
class state:
"""
State machine enums
@@ -1258,12 +1265,7 @@ class KernelDoc:
r = KernRe("[-:](.*)")
if r.search(line):
- # strip leading/trailing/multiple spaces
- self.entry.descr = r.group(1).strip(" ")
-
- r = KernRe(r"\s+")
- self.entry.descr = r.sub(" ", self.entry.descr)
- self.entry.declaration_purpose = self.entry.descr
+ self.entry.declaration_purpose = trim_whitespace(r.group(1))
self.state = state.BODY_MAYBE
else:
self.entry.declaration_purpose = ""
--
2.49.0
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 7/9] docs: kdoc: remove some ineffective code
2025-06-06 16:34 [PATCH 0/9] A series of kernel-doc tweaks Jonathan Corbet
` (5 preceding siblings ...)
2025-06-06 16:34 ` [PATCH 6/9] docs: kdoc: remove the KernelEntry::descr pseudo member Jonathan Corbet
@ 2025-06-06 16:34 ` Jonathan Corbet
2025-06-07 10:09 ` Mauro Carvalho Chehab
2025-06-06 16:34 ` [PATCH 8/9] docs: kdoc: move the declaration regexes out of process_name() Jonathan Corbet
` (2 subsequent siblings)
9 siblings, 1 reply; 22+ messages in thread
From: Jonathan Corbet @ 2025-06-06 16:34 UTC (permalink / raw)
To: linux-doc; +Cc: linux-kernel, Mauro Carvalho Chehab, Jonathan Corbet
The code testing for a pointer declaration in process_name() has no actual
effect on subsequent actions; remove it.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
scripts/lib/kdoc/kdoc_parser.py | 5 -----
1 file changed, 5 deletions(-)
diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index 7c635000f3de..4e1ab28ff7cc 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -1229,11 +1229,6 @@ class KernelDoc:
parenthesis = r"(?:\(\w*\))?" # optional parenthesis on function
decl_end = r"(?:[-:].*)" # end of the name part
- # test for pointer declaration type, foo * bar() - desc
- r = KernRe(fr"^{decl_start}([\w\s]+?){parenthesis}?\s*{decl_end}?$")
- if r.search(line):
- self.entry.identifier = r.group(1)
-
# Test for data declaration
r = KernRe(r"^\s*\*?\s*(struct|union|enum|typedef)\b\s*(\w*)")
r2 = KernRe(fr"^{decl_start}{fn_type}(?:define\s+)?(\w+)\s*{parenthesis}\s*{decl_end}?$")
--
2.49.0
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 8/9] docs: kdoc: move the declaration regexes out of process_name()
2025-06-06 16:34 [PATCH 0/9] A series of kernel-doc tweaks Jonathan Corbet
` (6 preceding siblings ...)
2025-06-06 16:34 ` [PATCH 7/9] docs: kdoc: remove some ineffective code Jonathan Corbet
@ 2025-06-06 16:34 ` Jonathan Corbet
2025-06-07 10:11 ` Mauro Carvalho Chehab
2025-06-06 16:34 ` [PATCH 9/9] docs: kdoc: some final touches for process_name() Jonathan Corbet
2025-06-07 9:58 ` [PATCH 0/9] A series of kernel-doc tweaks Mauro Carvalho Chehab
9 siblings, 1 reply; 22+ messages in thread
From: Jonathan Corbet @ 2025-06-06 16:34 UTC (permalink / raw)
To: linux-doc; +Cc: linux-kernel, Mauro Carvalho Chehab, Jonathan Corbet
Move two complex regexes up with the other patterns, decluttering this
function and allowing the compilation to be done once rather than for every
kerneldoc comment.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
scripts/lib/kdoc/kdoc_parser.py | 30 +++++++++++++++++-------------
1 file changed, 17 insertions(+), 13 deletions(-)
diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index 4e1ab28ff7cc..55f38240d4e5 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -47,7 +47,6 @@ doc_sect = doc_com + \
flags=re.I, cache=False)
doc_content = doc_com_body + KernRe(r'(.*)', cache=False)
-doc_block = doc_com + KernRe(r'DOC:\s*(.*)?', cache=False)
doc_inline_start = KernRe(r'^\s*/\*\*\s*$', cache=False)
doc_inline_sect = KernRe(r'\s*\*\s*(@\s*[\w][\w\.]*\s*):(.*)', cache=False)
doc_inline_end = KernRe(r'^\s*\*/\s*$', cache=False)
@@ -60,6 +59,18 @@ export_symbol_ns = KernRe(r'^\s*EXPORT_SYMBOL_NS(_GPL)?\s*\(\s*(\w+)\s*,\s*"\S+"
type_param = KernRe(r"\@(\w*((\.\w+)|(->\w+))*(\.\.\.)?)", cache=False)
+#
+# Tests for the beginning of a kerneldoc block in its various forms.
+#
+doc_block = doc_com + KernRe(r'DOC:\s*(.*)?', cache=False)
+doc_begin_data = KernRe(r"^\s*\*?\s*(struct|union|enum|typedef)\b\s*(\w*)", cache = False)
+doc_begin_func = KernRe(str(doc_com) + # initial " * '
+ r"(?:\w+\s*\*\s*)?" + # type (not captured)
+ r'(?:define\s+)?' + # possible "define" (not captured)
+ r'(\w+)\s*(?:\(\w*\))?\s*' + # name and optional "(...)"
+ r'(?:[-:].*)?$', # description (not captured)
+ cache = False)
+
#
# A little helper to get rid of excess white space
#
@@ -1224,22 +1235,15 @@ class KernelDoc:
if doc_decl.search(line):
self.entry.identifier = doc_decl.group(1)
- decl_start = str(doc_com) # comment block asterisk
- fn_type = r"(?:\w+\s*\*\s*)?" # type (for non-functions)
- parenthesis = r"(?:\(\w*\))?" # optional parenthesis on function
- decl_end = r"(?:[-:].*)" # end of the name part
-
# Test for data declaration
- r = KernRe(r"^\s*\*?\s*(struct|union|enum|typedef)\b\s*(\w*)")
- r2 = KernRe(fr"^{decl_start}{fn_type}(?:define\s+)?(\w+)\s*{parenthesis}\s*{decl_end}?$")
- if r.search(line):
- self.entry.decl_type = r.group(1)
- self.entry.identifier = r.group(2)
+ if doc_begin_data.search(line):
+ self.entry.decl_type = doc_begin_data.group(1)
+ self.entry.identifier = doc_begin_data.group(2)
#
# Look for a function description
#
- elif r2.search(line):
- self.entry.identifier = r2.group(1)
+ elif doc_begin_func.search(line):
+ self.entry.identifier = doc_begin_func.group(1)
self.entry.decl_type = "function"
#
# We struck out.
--
2.49.0
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 9/9] docs: kdoc: some final touches for process_name()
2025-06-06 16:34 [PATCH 0/9] A series of kernel-doc tweaks Jonathan Corbet
` (7 preceding siblings ...)
2025-06-06 16:34 ` [PATCH 8/9] docs: kdoc: move the declaration regexes out of process_name() Jonathan Corbet
@ 2025-06-06 16:34 ` Jonathan Corbet
2025-06-07 10:13 ` Mauro Carvalho Chehab
2025-06-07 9:58 ` [PATCH 0/9] A series of kernel-doc tweaks Mauro Carvalho Chehab
9 siblings, 1 reply; 22+ messages in thread
From: Jonathan Corbet @ 2025-06-06 16:34 UTC (permalink / raw)
To: linux-doc; +Cc: linux-kernel, Mauro Carvalho Chehab, Jonathan Corbet
Add some comments to process_name() to cover its broad phases of operation,
and slightly restructure the if/then/else structure to remove some early
returns.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
scripts/lib/kdoc/kdoc_parser.py | 33 ++++++++++++++++++++-------------
1 file changed, 20 insertions(+), 13 deletions(-)
diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index 55f38240d4e5..9a1ce6ed8605 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -1219,7 +1219,9 @@ class KernelDoc:
"""
STATE_NAME: Looking for the "name - description" line
"""
-
+ #
+ # Check for a DOC: block and handle them specially.
+ #
if doc_block.search(line):
self.entry.new_start_line = ln
@@ -1230,9 +1232,10 @@ class KernelDoc:
self.entry.identifier = self.entry.section
self.state = state.DOCBLOCK
- return
-
- if doc_decl.search(line):
+ #
+ # Otherwise we're looking for a normal kerneldoc declaration line.
+ #
+ elif doc_decl.search(line):
self.entry.identifier = doc_decl.group(1)
# Test for data declaration
@@ -1253,15 +1256,19 @@ class KernelDoc:
f"This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst\n{line}")
self.state = state.NORMAL
return
-
- self.entry.identifier = self.entry.identifier.strip(" ")
-
+ #
+ # OK, set up for a new kerneldoc entry.
+ #
self.state = state.BODY
-
+ self.entry.identifier = self.entry.identifier.strip(" ")
# if there's no @param blocks need to set up default section here
self.entry.section = SECTION_DEFAULT
self.entry.new_start_line = ln + 1
-
+ #
+ # Find the description portion, which *should* be there but
+ # isn't always.
+ # (We should be able to capture this from the previous parsing - someday)
+ #
r = KernRe("[-:](.*)")
if r.search(line):
self.entry.declaration_purpose = trim_whitespace(r.group(1))
@@ -1282,11 +1289,11 @@ class KernelDoc:
self.emit_msg(ln,
f"Scanning doc for {self.entry.decl_type} {self.entry.identifier}",
warning=False)
-
- return
-
+ #
# Failed to find an identifier. Emit a warning
- self.emit_msg(ln, f"Cannot find identifier on line:\n{line}")
+ #
+ else:
+ self.emit_msg(ln, f"Cannot find identifier on line:\n{line}")
def process_body(self, ln, line):
"""
--
2.49.0
^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [PATCH 0/9] A series of kernel-doc tweaks
2025-06-06 16:34 [PATCH 0/9] A series of kernel-doc tweaks Jonathan Corbet
` (8 preceding siblings ...)
2025-06-06 16:34 ` [PATCH 9/9] docs: kdoc: some final touches for process_name() Jonathan Corbet
@ 2025-06-07 9:58 ` Mauro Carvalho Chehab
9 siblings, 0 replies; 22+ messages in thread
From: Mauro Carvalho Chehab @ 2025-06-07 9:58 UTC (permalink / raw)
To: Jonathan Corbet; +Cc: linux-doc, linux-kernel
Em Fri, 6 Jun 2025 10:34:29 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:
> I will freely confess that I merged the kernel-doc Python rewrite without
> fully understanding the code; at the time, the fact that it worked as
> advertised sufficed. I *do* feel the need to understand this code, though,
> going forward, so I've dedicated some time to digging through it.
>
> In the process, I've been making some adjustments to the code that, IMO,
> make it a bit more approachable - for myself and, hopefully, for others.
> The goal is to try to get functions to the point where people of limited
> mind (like me) can soak them up, make the code slightly more Pythonic, and
> removing redundant code.
>
> Here is the first set of tweaks. The output from "make htmldocs" remains
> entirely unchanged throughout the series. The docs build is slightly
> faster afterward - but that's not the point.
>
> Mauro, the more I dig into this the happier I am that you pushed this
> change through - it was far overdue. Hopefully you don't mind me sweeping
> up a bit around the edges...
Sure, feel free to help improving it! The main goal of the conversion
was to make it as close as possible to the Perl version, being
bug-compatible. Due to that, there are several places where it can
(and should) be improved. It is now time to clean it up and improve
it ;-)
>
> Jonathan Corbet (9):
> docs: kdoc: simplify the PROTO continuation logic
> docs: kdoc: move the core dispatch into a state table
> docs: kdoc: remove the section_intro variable
> docs: kdoc: simplify the kerneldoc recognition code
> docs: kdoc: remove the KernelEntry::is_kernel_comment member
> docs: kdoc: remove the KernelEntry::descr pseudo member
> docs: kdoc: remove some ineffective code
> docs: kdoc: move the declaration regexes out of process_name()
> docs: kdoc: some final touches for process_name()
>
> scripts/lib/kdoc/kdoc_parser.py | 152 ++++++++++++++++----------------
> 1 file changed, 75 insertions(+), 77 deletions(-)
>
Thanks,
Mauro
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 1/9] docs: kdoc: simplify the PROTO continuation logic
2025-06-06 16:34 ` [PATCH 1/9] docs: kdoc: simplify the PROTO continuation logic Jonathan Corbet
@ 2025-06-07 9:59 ` Mauro Carvalho Chehab
0 siblings, 0 replies; 22+ messages in thread
From: Mauro Carvalho Chehab @ 2025-06-07 9:59 UTC (permalink / raw)
To: Jonathan Corbet; +Cc: linux-doc, linux-kernel
Em Fri, 6 Jun 2025 10:34:30 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:
> Remove the unneeded "cont" variable and tighten up the code slightly.
>
> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
LGTM.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> ---
> scripts/lib/kdoc/kdoc_parser.py | 7 +------
> 1 file changed, 1 insertion(+), 6 deletions(-)
>
> diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
> index 062453eefc7a..b193cf9dc0d1 100644
> --- a/scripts/lib/kdoc/kdoc_parser.py
> +++ b/scripts/lib/kdoc/kdoc_parser.py
> @@ -1680,7 +1680,6 @@ class KernelDoc:
> Besides parsing kernel-doc tags, it also parses export symbols.
> """
>
> - cont = False
> prev = ""
> prev_ln = None
> export_table = set()
> @@ -1696,18 +1695,14 @@ class KernelDoc:
> if self.state == state.PROTO:
> if line.endswith("\\"):
> prev += line.rstrip("\\")
> - cont = True
> -
> if not prev_ln:
> prev_ln = ln
> -
> continue
>
> - if cont:
> + if prev:
> ln = prev_ln
> line = prev + line
> prev = ""
> - cont = False
> prev_ln = None
>
> self.config.log.debug("%d %s%s: %s",
Thanks,
Mauro
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 2/9] docs: kdoc: move the core dispatch into a state table
2025-06-06 16:34 ` [PATCH 2/9] docs: kdoc: move the core dispatch into a state table Jonathan Corbet
@ 2025-06-07 10:00 ` Mauro Carvalho Chehab
0 siblings, 0 replies; 22+ messages in thread
From: Mauro Carvalho Chehab @ 2025-06-07 10:00 UTC (permalink / raw)
To: Jonathan Corbet; +Cc: linux-doc, linux-kernel
Em Fri, 6 Jun 2025 10:34:31 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:
> Since all of the handlers already nicely have the same prototype, put them
> into a table and call them from there and take out the extended
> if-then-else series.
>
> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
LGTM.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> ---
> scripts/lib/kdoc/kdoc_parser.py | 30 +++++++++++++++++-------------
> 1 file changed, 17 insertions(+), 13 deletions(-)
>
> diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
> index b193cf9dc0d1..37b7e501af7c 100644
> --- a/scripts/lib/kdoc/kdoc_parser.py
> +++ b/scripts/lib/kdoc/kdoc_parser.py
> @@ -1670,6 +1670,21 @@ class KernelDoc:
>
> return export_table
>
> + #
> + # The state/action table telling us which function to invoke in
> + # each state.
> + #
> + state_actions = {
> + state.NORMAL: process_normal,
> + state.NAME: process_name,
> + state.BODY: process_body,
> + state.BODY_MAYBE: process_body,
> + state.BODY_WITH_BLANK_LINE: process_body,
> + state.INLINE: process_inline,
> + state.PROTO: process_proto,
> + state.DOCBLOCK: process_docblock,
> + }
> +
> def parse_kdoc(self):
> """
> Open and process each line of a C source file.
> @@ -1721,19 +1736,8 @@ class KernelDoc:
> self.process_export(export_table, line)
>
> # Hand this line to the appropriate state handler
> - if self.state == state.NORMAL:
> - self.process_normal(ln, line)
> - elif self.state == state.NAME:
> - self.process_name(ln, line)
> - elif self.state in [state.BODY, state.BODY_MAYBE,
> - state.BODY_WITH_BLANK_LINE]:
> - self.process_body(ln, line)
> - elif self.state == state.INLINE: # scanning for inline parameters
> - self.process_inline(ln, line)
> - elif self.state == state.PROTO:
> - self.process_proto(ln, line)
> - elif self.state == state.DOCBLOCK:
> - self.process_docblock(ln, line)
> + self.state_actions[self.state](self, ln, line)
> +
> except OSError:
> self.config.log.error(f"Error: Cannot open file {self.fname}")
>
Thanks,
Mauro
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 3/9] docs: kdoc: remove the section_intro variable
2025-06-06 16:34 ` [PATCH 3/9] docs: kdoc: remove the section_intro variable Jonathan Corbet
@ 2025-06-07 10:03 ` Mauro Carvalho Chehab
0 siblings, 0 replies; 22+ messages in thread
From: Mauro Carvalho Chehab @ 2025-06-07 10:03 UTC (permalink / raw)
To: Jonathan Corbet; +Cc: linux-doc, linux-kernel
Em Fri, 6 Jun 2025 10:34:32 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:
> It is only used in one place, so just put the constant string
> "Introduction" there so people don't have to go looking for it.
Good catch! There were other places using it, but it sounds I
already cleanup its redundancy.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
>
> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
> ---
> scripts/lib/kdoc/kdoc_parser.py | 3 +--
> 1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
> index 37b7e501af7c..90b53b70cfee 100644
> --- a/scripts/lib/kdoc/kdoc_parser.py
> +++ b/scripts/lib/kdoc/kdoc_parser.py
> @@ -203,7 +203,6 @@ class KernelDoc:
>
> # Section names
>
> - section_intro = "Introduction"
> section_context = "Context"
> section_return = "Return"
>
> @@ -1207,7 +1206,7 @@ class KernelDoc:
> self.entry.new_start_line = ln
>
> if not doc_block.group(1):
> - self.entry.section = self.section_intro
> + self.entry.section = "Introduction"
> else:
> self.entry.section = doc_block.group(1)
>
Thanks,
Mauro
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 4/9] docs: kdoc: simplify the kerneldoc recognition code
2025-06-06 16:34 ` [PATCH 4/9] docs: kdoc: simplify the kerneldoc recognition code Jonathan Corbet
@ 2025-06-07 10:05 ` Mauro Carvalho Chehab
0 siblings, 0 replies; 22+ messages in thread
From: Mauro Carvalho Chehab @ 2025-06-07 10:05 UTC (permalink / raw)
To: Jonathan Corbet; +Cc: linux-doc, linux-kernel
Em Fri, 6 Jun 2025 10:34:33 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:
> process_name() looks for the first line of a kerneldoc comment. It
> contains two nearly identical regular expressions, the second of which only
> catches six cases in the kernel, all of the form:
>
> define SOME_MACRO_NAME - description
>
> Simply put the "define" into the regex and discard it, eliminating the loop
> and the code to remove it specially.
>
> Note that this still treats these defines as if they were functions, but
> that's a separate issue.
>
> There is no change in the generated output.
>
> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Nice cleanup!
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> ---
> scripts/lib/kdoc/kdoc_parser.py | 24 ++++++++----------------
> 1 file changed, 8 insertions(+), 16 deletions(-)
>
> diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
> index 90b53b70cfee..3ea260b423e2 100644
> --- a/scripts/lib/kdoc/kdoc_parser.py
> +++ b/scripts/lib/kdoc/kdoc_parser.py
> @@ -1230,26 +1230,18 @@ class KernelDoc:
>
> # Test for data declaration
> r = KernRe(r"^\s*\*?\s*(struct|union|enum|typedef)\b\s*(\w*)")
> + r2 = KernRe(fr"^{decl_start}{fn_type}(?:define\s+)?(\w+)\s*{parenthesis}\s*{decl_end}?$")
> if r.search(line):
> self.entry.decl_type = r.group(1)
> self.entry.identifier = r.group(2)
> self.entry.is_kernel_comment = True
> - else:
> - # Look for foo() or static void foo() - description;
> - # or misspelt identifier
> -
> - r1 = KernRe(fr"^{decl_start}{fn_type}(\w+)\s*{parenthesis}\s*{decl_end}?$")
> - r2 = KernRe(fr"^{decl_start}{fn_type}(\w+[^-:]*){parenthesis}\s*{decl_end}$")
> -
> - for r in [r1, r2]:
> - if r.search(line):
> - self.entry.identifier = r.group(1)
> - self.entry.decl_type = "function"
> -
> - r = KernRe(r"define\s+")
> - self.entry.identifier = r.sub("", self.entry.identifier)
> - self.entry.is_kernel_comment = True
> - break
> + #
> + # Look for a function description
> + #
> + elif r2.search(line):
> + self.entry.identifier = r2.group(1)
> + self.entry.decl_type = "function"
> + self.entry.is_kernel_comment = True
>
> self.entry.identifier = self.entry.identifier.strip(" ")
>
Thanks,
Mauro
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 5/9] docs: kdoc: remove the KernelEntry::is_kernel_comment member
2025-06-06 16:34 ` [PATCH 5/9] docs: kdoc: remove the KernelEntry::is_kernel_comment member Jonathan Corbet
@ 2025-06-07 10:07 ` Mauro Carvalho Chehab
2025-06-07 13:22 ` Jonathan Corbet
0 siblings, 1 reply; 22+ messages in thread
From: Mauro Carvalho Chehab @ 2025-06-07 10:07 UTC (permalink / raw)
To: Jonathan Corbet; +Cc: linux-doc, linux-kernel
Em Fri, 6 Jun 2025 10:34:34 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:
> entry::is_kernel_comment never had anything to do with the entry itself; it
> is a bit of local state in one branch of process_name(). It can, in fact,
> be removed entirely; rework the code slightly so that it is no longer
> needed.
>
> No change in the rendered output.
>
> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Not sure about this one. The idea of those warnings are to detect
non-kerneldoc markups that typically comes when someone "imports"
OOT drivers or Windows one into Linux.
I remember I catched several such cases in the past with the help
of those warnings.
> ---
> scripts/lib/kdoc/kdoc_parser.py | 16 ++++++++--------
> 1 file changed, 8 insertions(+), 8 deletions(-)
>
> diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
> index 3ea260b423e2..56299695abd1 100644
> --- a/scripts/lib/kdoc/kdoc_parser.py
> +++ b/scripts/lib/kdoc/kdoc_parser.py
> @@ -1216,7 +1216,6 @@ class KernelDoc:
>
> if doc_decl.search(line):
> self.entry.identifier = doc_decl.group(1)
> - self.entry.is_kernel_comment = False
>
> decl_start = str(doc_com) # comment block asterisk
> fn_type = r"(?:\w+\s*\*\s*)?" # type (for non-functions)
> @@ -1234,14 +1233,20 @@ class KernelDoc:
> if r.search(line):
> self.entry.decl_type = r.group(1)
> self.entry.identifier = r.group(2)
> - self.entry.is_kernel_comment = True
> #
> # Look for a function description
> #
> elif r2.search(line):
> self.entry.identifier = r2.group(1)
> self.entry.decl_type = "function"
> - self.entry.is_kernel_comment = True
> + #
> + # We struck out.
> + #
> + else:
> + self.emit_msg(ln,
> + f"This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst\n{line}")
> + self.state = state.NORMAL
> + return
>
> self.entry.identifier = self.entry.identifier.strip(" ")
>
> @@ -1263,11 +1268,6 @@ class KernelDoc:
> else:
> self.entry.declaration_purpose = ""
>
> - if not self.entry.is_kernel_comment:
> - self.emit_msg(ln,
> - f"This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst\n{line}")
> - self.state = state.NORMAL
> -
> if not self.entry.declaration_purpose and self.config.wshort_desc:
> self.emit_msg(ln,
> f"missing initial short description on line:\n{line}")
Thanks,
Mauro
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 7/9] docs: kdoc: remove some ineffective code
2025-06-06 16:34 ` [PATCH 7/9] docs: kdoc: remove some ineffective code Jonathan Corbet
@ 2025-06-07 10:09 ` Mauro Carvalho Chehab
0 siblings, 0 replies; 22+ messages in thread
From: Mauro Carvalho Chehab @ 2025-06-07 10:09 UTC (permalink / raw)
To: Jonathan Corbet; +Cc: linux-doc, linux-kernel
Em Fri, 6 Jun 2025 10:34:36 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:
> The code testing for a pointer declaration in process_name() has no actual
> effect on subsequent actions; remove it.
>
> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
> ---
> scripts/lib/kdoc/kdoc_parser.py | 5 -----
> 1 file changed, 5 deletions(-)
>
> diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
> index 7c635000f3de..4e1ab28ff7cc 100644
> --- a/scripts/lib/kdoc/kdoc_parser.py
> +++ b/scripts/lib/kdoc/kdoc_parser.py
> @@ -1229,11 +1229,6 @@ class KernelDoc:
> parenthesis = r"(?:\(\w*\))?" # optional parenthesis on function
> decl_end = r"(?:[-:].*)" # end of the name part
>
> - # test for pointer declaration type, foo * bar() - desc
> - r = KernRe(fr"^{decl_start}([\w\s]+?){parenthesis}?\s*{decl_end}?$")
> - if r.search(line):
> - self.entry.identifier = r.group(1)
> -
I'm almost sure this was added at the Perl version to catch some corner
case. If this not need anymore, better to strip ;-)
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> # Test for data declaration
> r = KernRe(r"^\s*\*?\s*(struct|union|enum|typedef)\b\s*(\w*)")
> r2 = KernRe(fr"^{decl_start}{fn_type}(?:define\s+)?(\w+)\s*{parenthesis}\s*{decl_end}?$")
Thanks,
Mauro
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 8/9] docs: kdoc: move the declaration regexes out of process_name()
2025-06-06 16:34 ` [PATCH 8/9] docs: kdoc: move the declaration regexes out of process_name() Jonathan Corbet
@ 2025-06-07 10:11 ` Mauro Carvalho Chehab
0 siblings, 0 replies; 22+ messages in thread
From: Mauro Carvalho Chehab @ 2025-06-07 10:11 UTC (permalink / raw)
To: Jonathan Corbet; +Cc: linux-doc, linux-kernel
Em Fri, 6 Jun 2025 10:34:37 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:
> Move two complex regexes up with the other patterns, decluttering this
> function and allowing the compilation to be done once rather than for every
> kerneldoc comment.
>
> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
LGTM.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> ---
> scripts/lib/kdoc/kdoc_parser.py | 30 +++++++++++++++++-------------
> 1 file changed, 17 insertions(+), 13 deletions(-)
>
> diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
> index 4e1ab28ff7cc..55f38240d4e5 100644
> --- a/scripts/lib/kdoc/kdoc_parser.py
> +++ b/scripts/lib/kdoc/kdoc_parser.py
> @@ -47,7 +47,6 @@ doc_sect = doc_com + \
> flags=re.I, cache=False)
>
> doc_content = doc_com_body + KernRe(r'(.*)', cache=False)
> -doc_block = doc_com + KernRe(r'DOC:\s*(.*)?', cache=False)
> doc_inline_start = KernRe(r'^\s*/\*\*\s*$', cache=False)
> doc_inline_sect = KernRe(r'\s*\*\s*(@\s*[\w][\w\.]*\s*):(.*)', cache=False)
> doc_inline_end = KernRe(r'^\s*\*/\s*$', cache=False)
> @@ -60,6 +59,18 @@ export_symbol_ns = KernRe(r'^\s*EXPORT_SYMBOL_NS(_GPL)?\s*\(\s*(\w+)\s*,\s*"\S+"
>
> type_param = KernRe(r"\@(\w*((\.\w+)|(->\w+))*(\.\.\.)?)", cache=False)
>
> +#
> +# Tests for the beginning of a kerneldoc block in its various forms.
> +#
> +doc_block = doc_com + KernRe(r'DOC:\s*(.*)?', cache=False)
> +doc_begin_data = KernRe(r"^\s*\*?\s*(struct|union|enum|typedef)\b\s*(\w*)", cache = False)
> +doc_begin_func = KernRe(str(doc_com) + # initial " * '
> + r"(?:\w+\s*\*\s*)?" + # type (not captured)
> + r'(?:define\s+)?' + # possible "define" (not captured)
> + r'(\w+)\s*(?:\(\w*\))?\s*' + # name and optional "(...)"
> + r'(?:[-:].*)?$', # description (not captured)
> + cache = False)
> +
> #
> # A little helper to get rid of excess white space
> #
> @@ -1224,22 +1235,15 @@ class KernelDoc:
> if doc_decl.search(line):
> self.entry.identifier = doc_decl.group(1)
>
> - decl_start = str(doc_com) # comment block asterisk
> - fn_type = r"(?:\w+\s*\*\s*)?" # type (for non-functions)
> - parenthesis = r"(?:\(\w*\))?" # optional parenthesis on function
> - decl_end = r"(?:[-:].*)" # end of the name part
> -
> # Test for data declaration
> - r = KernRe(r"^\s*\*?\s*(struct|union|enum|typedef)\b\s*(\w*)")
> - r2 = KernRe(fr"^{decl_start}{fn_type}(?:define\s+)?(\w+)\s*{parenthesis}\s*{decl_end}?$")
> - if r.search(line):
> - self.entry.decl_type = r.group(1)
> - self.entry.identifier = r.group(2)
> + if doc_begin_data.search(line):
> + self.entry.decl_type = doc_begin_data.group(1)
> + self.entry.identifier = doc_begin_data.group(2)
> #
> # Look for a function description
> #
> - elif r2.search(line):
> - self.entry.identifier = r2.group(1)
> + elif doc_begin_func.search(line):
> + self.entry.identifier = doc_begin_func.group(1)
> self.entry.decl_type = "function"
> #
> # We struck out.
Thanks,
Mauro
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 9/9] docs: kdoc: some final touches for process_name()
2025-06-06 16:34 ` [PATCH 9/9] docs: kdoc: some final touches for process_name() Jonathan Corbet
@ 2025-06-07 10:13 ` Mauro Carvalho Chehab
0 siblings, 0 replies; 22+ messages in thread
From: Mauro Carvalho Chehab @ 2025-06-07 10:13 UTC (permalink / raw)
To: Jonathan Corbet; +Cc: linux-doc, linux-kernel
Em Fri, 6 Jun 2025 10:34:38 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:
> Add some comments to process_name() to cover its broad phases of operation,
> and slightly restructure the if/then/else structure to remove some early
> returns.
>
> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
More comments describing the parse steps are always welcomed!
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> ---
> scripts/lib/kdoc/kdoc_parser.py | 33 ++++++++++++++++++++-------------
> 1 file changed, 20 insertions(+), 13 deletions(-)
>
> diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
> index 55f38240d4e5..9a1ce6ed8605 100644
> --- a/scripts/lib/kdoc/kdoc_parser.py
> +++ b/scripts/lib/kdoc/kdoc_parser.py
> @@ -1219,7 +1219,9 @@ class KernelDoc:
> """
> STATE_NAME: Looking for the "name - description" line
> """
> -
> + #
> + # Check for a DOC: block and handle them specially.
> + #
> if doc_block.search(line):
> self.entry.new_start_line = ln
>
> @@ -1230,9 +1232,10 @@ class KernelDoc:
>
> self.entry.identifier = self.entry.section
> self.state = state.DOCBLOCK
> - return
> -
> - if doc_decl.search(line):
> + #
> + # Otherwise we're looking for a normal kerneldoc declaration line.
> + #
> + elif doc_decl.search(line):
> self.entry.identifier = doc_decl.group(1)
>
> # Test for data declaration
> @@ -1253,15 +1256,19 @@ class KernelDoc:
> f"This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst\n{line}")
> self.state = state.NORMAL
> return
> -
> - self.entry.identifier = self.entry.identifier.strip(" ")
> -
> + #
> + # OK, set up for a new kerneldoc entry.
> + #
> self.state = state.BODY
> -
> + self.entry.identifier = self.entry.identifier.strip(" ")
> # if there's no @param blocks need to set up default section here
> self.entry.section = SECTION_DEFAULT
> self.entry.new_start_line = ln + 1
> -
> + #
> + # Find the description portion, which *should* be there but
> + # isn't always.
> + # (We should be able to capture this from the previous parsing - someday)
> + #
> r = KernRe("[-:](.*)")
> if r.search(line):
> self.entry.declaration_purpose = trim_whitespace(r.group(1))
> @@ -1282,11 +1289,11 @@ class KernelDoc:
> self.emit_msg(ln,
> f"Scanning doc for {self.entry.decl_type} {self.entry.identifier}",
> warning=False)
> -
> - return
> -
> + #
> # Failed to find an identifier. Emit a warning
> - self.emit_msg(ln, f"Cannot find identifier on line:\n{line}")
> + #
> + else:
> + self.emit_msg(ln, f"Cannot find identifier on line:\n{line}")
>
> def process_body(self, ln, line):
> """
Thanks,
Mauro
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 6/9] docs: kdoc: remove the KernelEntry::descr pseudo member
2025-06-06 16:34 ` [PATCH 6/9] docs: kdoc: remove the KernelEntry::descr pseudo member Jonathan Corbet
@ 2025-06-07 10:14 ` Mauro Carvalho Chehab
0 siblings, 0 replies; 22+ messages in thread
From: Mauro Carvalho Chehab @ 2025-06-07 10:14 UTC (permalink / raw)
To: Jonathan Corbet; +Cc: linux-doc, linux-kernel
Em Fri, 6 Jun 2025 10:34:35 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:
> The entry.descr value used in process_name() is not actually a member of
> the KernelEntry class; it is a bit of local state. So just manage it
> locally.
>
> A trim_whitespace() helper was added to clean up the code slightly.
>
> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
LGTM.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> ---
> scripts/lib/kdoc/kdoc_parser.py | 14 ++++++++------
> 1 file changed, 8 insertions(+), 6 deletions(-)
>
> diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
> index 56299695abd1..7c635000f3de 100644
> --- a/scripts/lib/kdoc/kdoc_parser.py
> +++ b/scripts/lib/kdoc/kdoc_parser.py
> @@ -60,6 +60,13 @@ export_symbol_ns = KernRe(r'^\s*EXPORT_SYMBOL_NS(_GPL)?\s*\(\s*(\w+)\s*,\s*"\S+"
>
> type_param = KernRe(r"\@(\w*((\.\w+)|(->\w+))*(\.\.\.)?)", cache=False)
>
> +#
> +# A little helper to get rid of excess white space
> +#
> +multi_space = KernRe(r'\s\s+')
> +def trim_whitespace(s):
> + return multi_space.sub(' ', s.strip())
> +
> class state:
> """
> State machine enums
> @@ -1258,12 +1265,7 @@ class KernelDoc:
>
> r = KernRe("[-:](.*)")
> if r.search(line):
> - # strip leading/trailing/multiple spaces
> - self.entry.descr = r.group(1).strip(" ")
> -
> - r = KernRe(r"\s+")
> - self.entry.descr = r.sub(" ", self.entry.descr)
> - self.entry.declaration_purpose = self.entry.descr
> + self.entry.declaration_purpose = trim_whitespace(r.group(1))
> self.state = state.BODY_MAYBE
> else:
> self.entry.declaration_purpose = ""
Thanks,
Mauro
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 5/9] docs: kdoc: remove the KernelEntry::is_kernel_comment member
2025-06-07 10:07 ` Mauro Carvalho Chehab
@ 2025-06-07 13:22 ` Jonathan Corbet
2025-06-08 3:18 ` Mauro Carvalho Chehab
0 siblings, 1 reply; 22+ messages in thread
From: Jonathan Corbet @ 2025-06-07 13:22 UTC (permalink / raw)
To: Mauro Carvalho Chehab; +Cc: linux-doc, linux-kernel
Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
> Em Fri, 6 Jun 2025 10:34:34 -0600
> Jonathan Corbet <corbet@lwn.net> escreveu:
>
>> entry::is_kernel_comment never had anything to do with the entry itself; it
>> is a bit of local state in one branch of process_name(). It can, in fact,
>> be removed entirely; rework the code slightly so that it is no longer
>> needed.
>>
>> No change in the rendered output.
>>
>> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
>
> Not sure about this one. The idea of those warnings are to detect
> non-kerneldoc markups that typically comes when someone "imports"
> OOT drivers or Windows one into Linux.
>
> I remember I catched several such cases in the past with the help
> of those warnings.
I haven't removed the warning, just the use of that specific variable to
trigger it.
Thanks,
jon
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 5/9] docs: kdoc: remove the KernelEntry::is_kernel_comment member
2025-06-07 13:22 ` Jonathan Corbet
@ 2025-06-08 3:18 ` Mauro Carvalho Chehab
0 siblings, 0 replies; 22+ messages in thread
From: Mauro Carvalho Chehab @ 2025-06-08 3:18 UTC (permalink / raw)
To: Jonathan Corbet; +Cc: linux-doc, linux-kernel
Em Sat, 07 Jun 2025 07:22:51 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:
> Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
>
> > Em Fri, 6 Jun 2025 10:34:34 -0600
> > Jonathan Corbet <corbet@lwn.net> escreveu:
> >
> >> entry::is_kernel_comment never had anything to do with the entry itself; it
> >> is a bit of local state in one branch of process_name(). It can, in fact,
> >> be removed entirely; rework the code slightly so that it is no longer
> >> needed.
> >>
> >> No change in the rendered output.
> >>
> >> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
> >
> > Not sure about this one. The idea of those warnings are to detect
> > non-kerneldoc markups that typically comes when someone "imports"
> > OOT drivers or Windows one into Linux.
> >
> > I remember I catched several such cases in the past with the help
> > of those warnings.
>
> I haven't removed the warning, just the use of that specific variable to
> trigger it.
After looking a the patch again, it sounds OK to me.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Regards,
Mauro
^ permalink raw reply [flat|nested] 22+ messages in thread
end of thread, other threads:[~2025-06-08 3:18 UTC | newest]
Thread overview: 22+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-06-06 16:34 [PATCH 0/9] A series of kernel-doc tweaks Jonathan Corbet
2025-06-06 16:34 ` [PATCH 1/9] docs: kdoc: simplify the PROTO continuation logic Jonathan Corbet
2025-06-07 9:59 ` Mauro Carvalho Chehab
2025-06-06 16:34 ` [PATCH 2/9] docs: kdoc: move the core dispatch into a state table Jonathan Corbet
2025-06-07 10:00 ` Mauro Carvalho Chehab
2025-06-06 16:34 ` [PATCH 3/9] docs: kdoc: remove the section_intro variable Jonathan Corbet
2025-06-07 10:03 ` Mauro Carvalho Chehab
2025-06-06 16:34 ` [PATCH 4/9] docs: kdoc: simplify the kerneldoc recognition code Jonathan Corbet
2025-06-07 10:05 ` Mauro Carvalho Chehab
2025-06-06 16:34 ` [PATCH 5/9] docs: kdoc: remove the KernelEntry::is_kernel_comment member Jonathan Corbet
2025-06-07 10:07 ` Mauro Carvalho Chehab
2025-06-07 13:22 ` Jonathan Corbet
2025-06-08 3:18 ` Mauro Carvalho Chehab
2025-06-06 16:34 ` [PATCH 6/9] docs: kdoc: remove the KernelEntry::descr pseudo member Jonathan Corbet
2025-06-07 10:14 ` Mauro Carvalho Chehab
2025-06-06 16:34 ` [PATCH 7/9] docs: kdoc: remove some ineffective code Jonathan Corbet
2025-06-07 10:09 ` Mauro Carvalho Chehab
2025-06-06 16:34 ` [PATCH 8/9] docs: kdoc: move the declaration regexes out of process_name() Jonathan Corbet
2025-06-07 10:11 ` Mauro Carvalho Chehab
2025-06-06 16:34 ` [PATCH 9/9] docs: kdoc: some final touches for process_name() Jonathan Corbet
2025-06-07 10:13 ` Mauro Carvalho Chehab
2025-06-07 9:58 ` [PATCH 0/9] A series of kernel-doc tweaks Mauro Carvalho Chehab
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).