[PATCH 00/13] docs: kdoc: improvements to dump_function() and thereabouts

[PATCH 00/13] docs: kdoc: improvements to dump_function() and thereabouts

[Jonathan Corbet]

Another series I've been slowly accumulating as I work to understand
the new kernel-doc; it consists mostly of fairly straightforward
cleanups and the addition of a number of comments.

This is likely to be the last of these from me for a little bit, life will
be hectic.

Jonathan Corbet (13):
  docs: kdoc: trim __cacheline_group_* with the other annotations
  docs: kdoc: tighten up the push_parameter() no-type case
  docs: kdoc: remove a single-use variable
  docs: kdoc: move the function transform patterns out of
    dump_function()
  doc: kdoc: unify transform handling
  docs: kdoc: remove a couple of spurious regex characters
  docs: kdoc: remove a useless empty capture group
  docs: kdoc: Simplify the dump_function() prototype regexes
  docs: kdoc: consolidate some of the macro-processing logic
  docs: kdoc: final dump_function() cleanups
  docs: kdoc: remove some dead code in dump_typedef()
  docs: kdoc: remove redundant comment stripping in dump_typedef()
  docs: kdoc: a few more dump_typedef() tweaks

 scripts/lib/kdoc/kdoc_parser.py | 269 ++++++++++++++------------------
 1 file changed, 120 insertions(+), 149 deletions(-)

-- 
2.51.0

[PATCH 01/13] docs: kdoc: trim __cacheline_group_* with the other annotations

[Jonathan Corbet]

The special case for __cacheline_group_begin/end() can be handled by just
adding another pattern to the struct_prefixes, eliminating the need for a
special case in push_parameter().

One change is that these annotations no longer appear in the rendered
output, just like all the other annotations that we clean out.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 scripts/lib/kdoc/kdoc_parser.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index a560546c1867..a90f77d6b669 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -90,6 +90,7 @@ struct_prefixes = [
     (KernRe(r'\s*CRYPTO_MINALIGN_ATTR', re.S), ' '),
     (KernRe(r'\s*____cacheline_aligned_in_smp', re.S), ' '),
     (KernRe(r'\s*____cacheline_aligned', re.S), ' '),
+    (KernRe(r'\s*__cacheline_group_(begin|end)\([^\)]+\);'), ''),
     #
     # Unwrap struct_group macros based on this definition:
     # __struct_group(TAG, NAME, ATTRS, MEMBERS...)
@@ -447,12 +448,6 @@ class KernelDoc:
             self.entry.parameterdescs[param] = "anonymous\n"
             self.entry.anon_struct_union = True
 
-        # Handle cache group enforcing variables: they do not need
-        # to be described in header files
-        elif "__cacheline_group" in param:
-            # Ignore __cacheline_group_begin and __cacheline_group_end
-            return
-
         # Warn if parameter has no description
         # (but ignore ones starting with # as these are not parameters
         # but inline preprocessor statements)
-- 
2.51.0

[PATCH 02/13] docs: kdoc: tighten up the push_parameter() no-type case

[Jonathan Corbet]

The handling of untyped parameters involved a number of redundant tests;
restructure the code to remove them and be more compact.

No output changes.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 scripts/lib/kdoc/kdoc_parser.py | 44 +++++++++++++++------------------
 1 file changed, 20 insertions(+), 24 deletions(-)

diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index a90f77d6b669..2118c20b3056 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -423,30 +423,26 @@ class KernelDoc:
 
         param = KernRe(r'[\[\)].*').sub('', param, count=1)
 
-        if dtype == "" and param.endswith("..."):
-            if KernRe(r'\w\.\.\.$').search(param):
-                # For named variable parameters of the form `x...`,
-                # remove the dots
-                param = param[:-3]
-            else:
-                # Handles unnamed variable parameters
-                param = "..."
-
-            if param not in self.entry.parameterdescs or \
-                not self.entry.parameterdescs[param]:
-
-                self.entry.parameterdescs[param] = "variable arguments"
-
-        elif dtype == "" and (not param or param == "void"):
-            param = "void"
-            self.entry.parameterdescs[param] = "no arguments"
-
-        elif dtype == "" and param in ["struct", "union"]:
-            # Handle unnamed (anonymous) union or struct
-            dtype = param
-            param = "{unnamed_" + param + "}"
-            self.entry.parameterdescs[param] = "anonymous\n"
-            self.entry.anon_struct_union = True
+        #
+        # Look at various "anonymous type" cases.
+        #
+        if dtype == '':
+            if param.endswith("..."):
+                if len(param) > 3: # there is a name provided, use that
+                    param = param[:-3]
+                if not self.entry.parameterdescs.get(param):
+                    self.entry.parameterdescs[param] = "variable arguments"
+
+            elif (not param) or param == "void":
+                param = "void"
+                self.entry.parameterdescs[param] = "no arguments"
+
+            elif param in ["struct", "union"]:
+                # Handle unnamed (anonymous) union or struct
+                dtype = param
+                param = "{unnamed_" + param + "}"
+                self.entry.parameterdescs[param] = "anonymous\n"
+                self.entry.anon_struct_union = True
 
         # Warn if parameter has no description
         # (but ignore ones starting with # as these are not parameters
-- 
2.51.0

[PATCH 03/13] docs: kdoc: remove a single-use variable

[Jonathan Corbet]

struct_attribute is only used once, so just put its value there directly
and drop the name.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 scripts/lib/kdoc/kdoc_parser.py | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index 2118c20b3056..b25c8d80b965 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -76,13 +76,11 @@ doc_begin_func = KernRe(str(doc_com) +			# initial " * '
 # Here begins a long set of transformations to turn structure member prefixes
 # and macro invocations into something we can parse and generate kdoc for.
 #
-struct_attribute = KernRe(r"__attribute__\s*\(\([a-z0-9,_\*\s\(\)]*\)\)",
-                          flags=re.I | re.S, cache=False)
 struct_args_pattern = r'([^,)]+)'
 
 struct_prefixes = [
     # Strip attributes
-    (struct_attribute, ' '),
+    (KernRe(r"__attribute__\s*\(\([a-z0-9,_\*\s\(\)]*\)\)", flags=re.I | re.S, cache=False), ' '),
     (KernRe(r'\s*__aligned\s*\([^;]*\)', re.S), ' '),
     (KernRe(r'\s*__counted_by\s*\([^;]*\)', re.S), ' '),
     (KernRe(r'\s*__counted_by_(le|be)\s*\([^;]*\)', re.S), ' '),
-- 
2.51.0

[PATCH 04/13] docs: kdoc: move the function transform patterns out of dump_function()

[Jonathan Corbet]

Move these definitions to file level, where they are executed once, and
don't clutter the function itself.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 scripts/lib/kdoc/kdoc_parser.py | 78 +++++++++++++++------------------
 1 file changed, 35 insertions(+), 43 deletions(-)

diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index b25c8d80b965..37811cddd55c 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -161,6 +161,37 @@ struct_nested_prefixes = [
     (re.compile(r'\bSTRUCT_GROUP\('), r'\1'),
 ]
 
+#
+# Transforms for function prototypes
+#
+function_xforms  = [
+    (r"^static +", "", 0),
+    (r"^extern +", "", 0),
+    (r"^asmlinkage +", "", 0),
+    (r"^inline +", "", 0),
+    (r"^__inline__ +", "", 0),
+    (r"^__inline +", "", 0),
+    (r"^__always_inline +", "", 0),
+    (r"^noinline +", "", 0),
+    (r"^__FORTIFY_INLINE +", "", 0),
+    (r"__init +", "", 0),
+    (r"__init_or_module +", "", 0),
+    (r"__deprecated +", "", 0),
+    (r"__flatten +", "", 0),
+    (r"__meminit +", "", 0),
+    (r"__must_check +", "", 0),
+    (r"__weak +", "", 0),
+    (r"__sched +", "", 0),
+    (r"_noprof", "", 0),
+    (r"__printf\s*\(\s*\d*\s*,\s*\d*\s*\) +", "", 0),
+    (r"__(?:re)?alloc_size\s*\(\s*\d+\s*(?:,\s*\d+\s*)?\) +", "", 0),
+    (r"__diagnose_as\s*\(\s*\S+\s*(?:,\s*\d+\s*)*\) +", "", 0),
+    (r"DECL_BUCKET_PARAMS\s*\(\s*(\S+)\s*,\s*(\S+)\s*\)", r"\1, \2", 0),
+    (r"__attribute_const__ +", "", 0),
+    (r"__attribute__\s*\(\((?:[\w\s]+(?:\([^)]*\))?\s*,?)+\)\)\s+", "", 0),
+]
+
+
 
 #
 # A little helper to get rid of excess white space
@@ -894,49 +925,10 @@ class KernelDoc:
         return_type = ''
         decl_type = 'function'
 
-        # Prefixes that would be removed
-        sub_prefixes = [
-            (r"^static +", "", 0),
-            (r"^extern +", "", 0),
-            (r"^asmlinkage +", "", 0),
-            (r"^inline +", "", 0),
-            (r"^__inline__ +", "", 0),
-            (r"^__inline +", "", 0),
-            (r"^__always_inline +", "", 0),
-            (r"^noinline +", "", 0),
-            (r"^__FORTIFY_INLINE +", "", 0),
-            (r"__init +", "", 0),
-            (r"__init_or_module +", "", 0),
-            (r"__deprecated +", "", 0),
-            (r"__flatten +", "", 0),
-            (r"__meminit +", "", 0),
-            (r"__must_check +", "", 0),
-            (r"__weak +", "", 0),
-            (r"__sched +", "", 0),
-            (r"_noprof", "", 0),
-            (r"__printf\s*\(\s*\d*\s*,\s*\d*\s*\) +", "", 0),
-            (r"__(?:re)?alloc_size\s*\(\s*\d+\s*(?:,\s*\d+\s*)?\) +", "", 0),
-            (r"__diagnose_as\s*\(\s*\S+\s*(?:,\s*\d+\s*)*\) +", "", 0),
-            (r"DECL_BUCKET_PARAMS\s*\(\s*(\S+)\s*,\s*(\S+)\s*\)", r"\1, \2", 0),
-            (r"__attribute_const__ +", "", 0),
-
-            # It seems that Python support for re.X is broken:
-            # At least for me (Python 3.13), this didn't work
-#            (r"""
-#              __attribute__\s*\(\(
-#                (?:
-#                    [\w\s]+          # attribute name
-#                    (?:\([^)]*\))?   # attribute arguments
-#                    \s*,?            # optional comma at the end
-#                )+
-#              \)\)\s+
-#             """, "", re.X),
-
-            # So, remove whitespaces and comments from it
-            (r"__attribute__\s*\(\((?:[\w\s]+(?:\([^)]*\))?\s*,?)+\)\)\s+", "", 0),
-        ]
-
-        for search, sub, flags in sub_prefixes:
+        #
+        # Apply the initial transformations.
+        #
+        for search, sub, flags in function_xforms:
             prototype = KernRe(search, flags).sub(sub, prototype)
 
         # Macros are a special case, as they change the prototype format
-- 
2.51.0

[PATCH 05/13] doc: kdoc: unify transform handling

[Jonathan Corbet]

Both functions and structs are passed through a set of regex-based
transforms, but the two were structured differently, despite being the same
thing.  Create a utility function to apply transformations and use it in
both cases.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 scripts/lib/kdoc/kdoc_parser.py | 65 +++++++++++++++++----------------
 1 file changed, 34 insertions(+), 31 deletions(-)

diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index 37811cddd55c..1a1558211acd 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -78,7 +78,7 @@ doc_begin_func = KernRe(str(doc_com) +			# initial " * '
 #
 struct_args_pattern = r'([^,)]+)'
 
-struct_prefixes = [
+struct_xforms = [
     # Strip attributes
     (KernRe(r"__attribute__\s*\(\([a-z0-9,_\*\s\(\)]*\)\)", flags=re.I | re.S, cache=False), ' '),
     (KernRe(r'\s*__aligned\s*\([^;]*\)', re.S), ' '),
@@ -165,33 +165,39 @@ struct_nested_prefixes = [
 # Transforms for function prototypes
 #
 function_xforms  = [
-    (r"^static +", "", 0),
-    (r"^extern +", "", 0),
-    (r"^asmlinkage +", "", 0),
-    (r"^inline +", "", 0),
-    (r"^__inline__ +", "", 0),
-    (r"^__inline +", "", 0),
-    (r"^__always_inline +", "", 0),
-    (r"^noinline +", "", 0),
-    (r"^__FORTIFY_INLINE +", "", 0),
-    (r"__init +", "", 0),
-    (r"__init_or_module +", "", 0),
-    (r"__deprecated +", "", 0),
-    (r"__flatten +", "", 0),
-    (r"__meminit +", "", 0),
-    (r"__must_check +", "", 0),
-    (r"__weak +", "", 0),
-    (r"__sched +", "", 0),
-    (r"_noprof", "", 0),
-    (r"__printf\s*\(\s*\d*\s*,\s*\d*\s*\) +", "", 0),
-    (r"__(?:re)?alloc_size\s*\(\s*\d+\s*(?:,\s*\d+\s*)?\) +", "", 0),
-    (r"__diagnose_as\s*\(\s*\S+\s*(?:,\s*\d+\s*)*\) +", "", 0),
-    (r"DECL_BUCKET_PARAMS\s*\(\s*(\S+)\s*,\s*(\S+)\s*\)", r"\1, \2", 0),
-    (r"__attribute_const__ +", "", 0),
-    (r"__attribute__\s*\(\((?:[\w\s]+(?:\([^)]*\))?\s*,?)+\)\)\s+", "", 0),
+    (KernRe(r"^static +"), ""),
+    (KernRe(r"^extern +"), ""),
+    (KernRe(r"^asmlinkage +"), ""),
+    (KernRe(r"^inline +"), ""),
+    (KernRe(r"^__inline__ +"), ""),
+    (KernRe(r"^__inline +"), ""),
+    (KernRe(r"^__always_inline +"), ""),
+    (KernRe(r"^noinline +"), ""),
+    (KernRe(r"^__FORTIFY_INLINE +"), ""),
+    (KernRe(r"__init +"), ""),
+    (KernRe(r"__init_or_module +"), ""),
+    (KernRe(r"__deprecated +"), ""),
+    (KernRe(r"__flatten +"), ""),
+    (KernRe(r"__meminit +"), ""),
+    (KernRe(r"__must_check +"), ""),
+    (KernRe(r"__weak +"), ""),
+    (KernRe(r"__sched +"), ""),
+    (KernRe(r"_noprof"), ""),
+    (KernRe(r"__printf\s*\(\s*\d*\s*,\s*\d*\s*\) +"), ""),
+    (KernRe(r"__(?:re)?alloc_size\s*\(\s*\d+\s*(?:,\s*\d+\s*)?\) +"), ""),
+    (KernRe(r"__diagnose_as\s*\(\s*\S+\s*(?:,\s*\d+\s*)*\) +"), ""),
+    (KernRe(r"DECL_BUCKET_PARAMS\s*\(\s*(\S+)\s*,\s*(\S+)\s*\)"), r"\1, \2"),
+    (KernRe(r"__attribute_const__ +"), ""),
+    (KernRe(r"__attribute__\s*\(\((?:[\w\s]+(?:\([^)]*\))?\s*,?)+\)\)\s+"), ""),
 ]
 
-
+#
+# Apply a set of transforms to a block of text.
+#
+def apply_transforms(xforms, text):
+    for search, subst in xforms:
+        text = search.sub(subst, text)
+    return text
 
 #
 # A little helper to get rid of excess white space
@@ -807,8 +813,7 @@ class KernelDoc:
         # Go through the list of members applying all of our transformations.
         #
         members = trim_private_members(members)
-        for search, sub in struct_prefixes:
-            members = search.sub(sub, members)
+        members = apply_transforms(struct_xforms, members)
 
         nested = NestedMatch()
         for search, sub in struct_nested_prefixes:
@@ -924,12 +929,10 @@ class KernelDoc:
         func_macro = False
         return_type = ''
         decl_type = 'function'
-
         #
         # Apply the initial transformations.
         #
-        for search, sub, flags in function_xforms:
-            prototype = KernRe(search, flags).sub(sub, prototype)
+        prototype = apply_transforms(function_xforms, prototype)
 
         # Macros are a special case, as they change the prototype format
         new_proto = KernRe(r"^#\s*define\s+").sub("", prototype)
-- 
2.51.0

[PATCH 06/13] docs: kdoc: remove a couple of spurious regex characters

[Jonathan Corbet]

The "name" regex in dump_function() includes both the tilde and colon
characters, but neither has any place in function prototypes.  Remove the
characters, after which the regex simplifies to "\w+"

No output changes.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 scripts/lib/kdoc/kdoc_parser.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index 1a1558211acd..decd127df82e 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -958,7 +958,7 @@ class KernelDoc:
         # - atomic_set (macro)
         # - pci_match_device, __copy_to_user (long return type)
 
-        name = r'[a-zA-Z0-9_~:]+'
+        name = r'\w+'
         prototype_end1 = r'[^\(]*'
         prototype_end2 = r'[^\{]*'
         prototype_end = fr'\(({prototype_end1}|{prototype_end2})\)'
-- 
2.51.0

[PATCH 07/13] docs: kdoc: remove a useless empty capture group

[Jonathan Corbet]

The is_define_proto case in dump_function() uses a regex with an empty
capture group - () - that has no use; just take it out.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 scripts/lib/kdoc/kdoc_parser.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index decd127df82e..f9be5414244d 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -972,11 +972,11 @@ class KernelDoc:
         found = False
 
         if is_define_proto:
-            r = KernRe(r'^()(' + name + r')\s+')
+            r = KernRe(r'^(' + name + r')\s+')
 
             if r.search(prototype):
                 return_type = ''
-                declaration_name = r.group(2)
+                declaration_name = r.group(1)
                 func_macro = True
 
                 found = True
-- 
2.51.0

[PATCH 08/13] docs: kdoc: Simplify the dump_function() prototype regexes

[Jonathan Corbet]

The regexes for the parsing of function prototypes were more complicated
than they needed to be and difficult to understand -- at least, I spent a
fair amount of time bashing my head against them.  Simplify them, and add
some documentation comments as well.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 scripts/lib/kdoc/kdoc_parser.py | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index f9be5414244d..ec2e6e83df05 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -959,15 +959,15 @@ class KernelDoc:
         # - pci_match_device, __copy_to_user (long return type)
 
         name = r'\w+'
-        prototype_end1 = r'[^\(]*'
-        prototype_end2 = r'[^\{]*'
-        prototype_end = fr'\(({prototype_end1}|{prototype_end2})\)'
-
-        # Besides compiling, Perl qr{[\w\s]+} works as a non-capturing group.
-        # So, this needs to be mapped in Python with (?:...)? or (?:...)+
-
         type1 = r'(?:[\w\s]+)?'
         type2 = r'(?:[\w\s]+\*+)+'
+        #
+        # Attempt to match first on (args) with no internal parentheses; this
+        # lets us easily filter out __acquires() and other post-args stuff.  If
+        # that fails, just grab the rest of the line to the last closing
+        # parenthesis.
+        #
+        proto_args = r'\(([^\(]*|.*)\)'
 
         found = False
 
@@ -983,9 +983,9 @@ class KernelDoc:
 
         if not found:
             patterns = [
-                rf'^()({name})\s*{prototype_end}',
-                rf'^({type1})\s+({name})\s*{prototype_end}',
-                rf'^({type2})\s*({name})\s*{prototype_end}',
+                rf'^()({name})\s*{proto_args}',
+                rf'^({type1})\s+({name})\s*{proto_args}',
+                rf'^({type2})\s*({name})\s*{proto_args}',
             ]
 
             for p in patterns:
-- 
2.51.0

[PATCH 09/13] docs: kdoc: consolidate some of the macro-processing logic

[Jonathan Corbet]

The logic to handle macros is split in dump_function(); bring it all
together into a single place and add a comment saying what's going on.
Remove the unneeded is_define_proto variable, and tighten up the code
a bit.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 scripts/lib/kdoc/kdoc_parser.py | 43 +++++++++++++++------------------
 1 file changed, 20 insertions(+), 23 deletions(-)

diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index ec2e6e83df05..27329ce9b5e9 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -926,21 +926,31 @@ class KernelDoc:
         Stores a function of function macro inside self.entries array.
         """
 
-        func_macro = False
+        found = func_macro = False
         return_type = ''
         decl_type = 'function'
         #
         # Apply the initial transformations.
         #
         prototype = apply_transforms(function_xforms, prototype)
-
-        # Macros are a special case, as they change the prototype format
+        #
+        # If we have a macro, remove the "#define" at the front.
+        #
         new_proto = KernRe(r"^#\s*define\s+").sub("", prototype)
         if new_proto != prototype:
-            is_define_proto = True
             prototype = new_proto
-        else:
-            is_define_proto = False
+            #
+            # Dispense with the simple "#define A B" case here; the key
+            # is the space after the name of the symbol being defined.
+            # NOTE that the seemingly misnamed "func_macro" indicates a
+            # macro *without* arguments.
+            #
+            r = KernRe(r'^(\w+)\s+')
+            if r.search(prototype):
+                return_type = ''
+                declaration_name = r.group(1)
+                func_macro = True
+                found = True
 
         # Yes, this truly is vile.  We are looking for:
         # 1. Return type (may be nothing if we're looking at a macro)
@@ -968,19 +978,10 @@ class KernelDoc:
         # parenthesis.
         #
         proto_args = r'\(([^\(]*|.*)\)'
-
-        found = False
-
-        if is_define_proto:
-            r = KernRe(r'^(' + name + r')\s+')
-
-            if r.search(prototype):
-                return_type = ''
-                declaration_name = r.group(1)
-                func_macro = True
-
-                found = True
-
+        #
+        # (Except for the simple macro case) attempt to split up the prototype
+        # in the various ways we understand.
+        #
         if not found:
             patterns = [
                 rf'^()({name})\s*{proto_args}',
@@ -990,16 +991,12 @@ class KernelDoc:
 
             for p in patterns:
                 r = KernRe(p)
-
                 if r.match(prototype):
-
                     return_type = r.group(1)
                     declaration_name = r.group(2)
                     args = r.group(3)
-
                     self.create_parameter_list(ln, decl_type, args, ',',
                                                declaration_name)
-
                     found = True
                     break
         if not found:
-- 
2.51.0

[PATCH 10/13] docs: kdoc: final dump_function() cleanups

[Jonathan Corbet]

Add some more comments to dump_function(), add some comments, and trim out
an unneeded duplicate output_declaration() call.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 scripts/lib/kdoc/kdoc_parser.py | 30 +++++++++++++-----------------
 1 file changed, 13 insertions(+), 17 deletions(-)

diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index 27329ce9b5e9..5e41acfef7b8 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -999,32 +999,28 @@ class KernelDoc:
                                                declaration_name)
                     found = True
                     break
+        #
+        # Parsing done; make sure that things are as we expect.
+        #
         if not found:
             self.emit_msg(ln,
                           f"cannot understand function prototype: '{prototype}'")
             return
-
         if self.entry.identifier != declaration_name:
-            self.emit_msg(ln,
-                          f"expecting prototype for {self.entry.identifier}(). Prototype was for {declaration_name}() instead")
+            self.emit_msg(ln, f"expecting prototype for {self.entry.identifier}(). "
+                          f"Prototype was for {declaration_name}() instead")
             return
-
         self.check_sections(ln, declaration_name, "function")
-
         self.check_return_section(ln, declaration_name, return_type)
+        #
+        # Store the result.
+        #
+        self.output_declaration(decl_type, declaration_name,
+                                typedef=('typedef' in return_type),
+                                functiontype=return_type,
+                                purpose=self.entry.declaration_purpose,
+                                func_macro=func_macro)
 
-        if 'typedef' in return_type:
-            self.output_declaration(decl_type, declaration_name,
-                                    typedef=True,
-                                    functiontype=return_type,
-                                    purpose=self.entry.declaration_purpose,
-                                    func_macro=func_macro)
-        else:
-            self.output_declaration(decl_type, declaration_name,
-                                    typedef=False,
-                                    functiontype=return_type,
-                                    purpose=self.entry.declaration_purpose,
-                                    func_macro=func_macro)
 
     def dump_typedef(self, ln, proto):
         """
-- 
2.51.0

[PATCH 11/13] docs: kdoc: remove some dead code in dump_typedef()

[Jonathan Corbet]

The regex in this block of code makes no sense, and a quick test shows that
it never matches anything; simply delete the code.

No output changes.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 scripts/lib/kdoc/kdoc_parser.py | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index 5e41acfef7b8..7c739b495d58 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -1060,11 +1060,6 @@ class KernelDoc:
                                     purpose=self.entry.declaration_purpose)
             return
 
-        # Handle nested parentheses or brackets
-        r = KernRe(r'(\(*.\)\s*|\[*.\]\s*);$')
-        while r.search(proto):
-            proto = r.sub('', proto)
-
         # Parse simple typedefs
         r = KernRe(r'typedef.*\s+(\w+)\s*;')
         if r.match(proto):
-- 
2.51.0

[PATCH 12/13] docs: kdoc: remove redundant comment stripping in dump_typedef()

[Jonathan Corbet]

By the time we get here, comments have long since been stripped out; there
is no need to do it again.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 scripts/lib/kdoc/kdoc_parser.py | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index 7c739b495d58..ad9df0536bbf 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -1034,9 +1034,6 @@ class KernelDoc:
         typedef1 = KernRe(r'typedef' + typedef_type + r'\(' + typedef_ident + r'\)' + typedef_args)
         typedef2 = KernRe(r'typedef' + typedef_type + typedef_ident + typedef_args)
 
-        # Strip comments
-        proto = KernRe(r'/\*.*?\*/', flags=re.S).sub('', proto)
-
         # Parse function typedef prototypes
         for r in [typedef1, typedef2]:
             if not r.match(proto):
-- 
2.51.0

[PATCH 13/13] docs: kdoc: a few more dump_typedef() tweaks

[Jonathan Corbet]

Merge "typedef" into the typedef_type pattern rather than repeating it
later, and add some comments.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 scripts/lib/kdoc/kdoc_parser.py | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)

diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index ad9df0536bbf..8215948dd548 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -1026,13 +1026,15 @@ class KernelDoc:
         """
         Stores a typedef inside self.entries array.
         """
-
-        typedef_type = r'((?:\s+[\w*]+\b){0,7}\s+(?:\w+\b|\*+))\s*'
+        #
+        # We start by looking for function typedefs.
+        #
+        typedef_type = r'typedef((?:\s+[\w*]+\b){0,7}\s+(?:\w+\b|\*+))\s*'
         typedef_ident = r'\*?\s*(\w\S+)\s*'
         typedef_args = r'\s*\((.*)\);'
 
-        typedef1 = KernRe(r'typedef' + typedef_type + r'\(' + typedef_ident + r'\)' + typedef_args)
-        typedef2 = KernRe(r'typedef' + typedef_type + typedef_ident + typedef_args)
+        typedef1 = KernRe(typedef_type + r'\(' + typedef_ident + r'\)' + typedef_args)
+        typedef2 = KernRe(typedef_type + typedef_ident + typedef_args)
 
         # Parse function typedef prototypes
         for r in [typedef1, typedef2]:
@@ -1048,16 +1050,16 @@ class KernelDoc:
                               f"expecting prototype for typedef {self.entry.identifier}. Prototype was for typedef {declaration_name} instead\n")
                 return
 
-            decl_type = 'function'
-            self.create_parameter_list(ln, decl_type, args, ',', declaration_name)
+            self.create_parameter_list(ln, 'function',_type, args, ',', declaration_name)
 
-            self.output_declaration(decl_type, declaration_name,
+            self.output_declaration('function', declaration_name,
                                     typedef=True,
                                     functiontype=return_type,
                                     purpose=self.entry.declaration_purpose)
             return
-
-        # Parse simple typedefs
+        #
+        # Not a function, try to parse a simple typedef.
+        #
         r = KernRe(r'typedef.*\s+(\w+)\s*;')
         if r.match(proto):
             declaration_name = r.group(1)
-- 
2.51.0

Re: [PATCH 13/13] docs: kdoc: a few more dump_typedef() tweaks

[Akira Yokosawa]

Hi Jon,

A quick report on minor regression. Please see below.

On Tue,  9 Sep 2025 14:43:49 -0600, Jonathan Corbet wrote:
> Merge "typedef" into the typedef_type pattern rather than repeating it
> later, and add some comments.

I'm seeing new warnings after applying 13/13:

WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno ./include/media/demux.h' processing failed with: NameError("name '_type' is not defined")
WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno ./include/media/v4l2-ioctl.h' processing failed with: NameError("name '_type' is not defined")
WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno ./include/media/v4l2-ctrls.h' processing failed with: NameError("name '_type' is not defined")
WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno ./include/media/v4l2-dv-timings.h' processing failed with: NameError("name '_type' is not defined")
WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno ./include/media/videobuf2-core.h' processing failed with: NameError("name '_type' is not defined")
WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno ./include/linux/hte.h' processing failed with: NameError("name '_type' is not defined")
WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno ./include/linux/xarray.h' processing failed with: NameError("name '_type' is not defined")
WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno -function gen_pool_add ./include/linux/genalloc.h' processing failed with: NameError("name '_type' is not defined")
WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno -function gen_pool_alloc ./include/linux/genalloc.h' processing failed with: NameError("name '_type' is not defined")
WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno -function vdso_sgx_enter_enclave_t ./arch/x86/include/uapi/asm/sgx.h' processing failed with: NameError("name '_type' is not defined")

Thanks,
Akira

> 
> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
> ---
>  scripts/lib/kdoc/kdoc_parser.py | 20 +++++++++++---------
>  1 file changed, 11 insertions(+), 9 deletions(-)
> 
> diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
> index ad9df0536bbf..8215948dd548 100644
> --- a/scripts/lib/kdoc/kdoc_parser.py
> +++ b/scripts/lib/kdoc/kdoc_parser.py
> @@ -1026,13 +1026,15 @@ class KernelDoc:
>          """
>          Stores a typedef inside self.entries array.
>          """
> -
> -        typedef_type = r'((?:\s+[\w*]+\b){0,7}\s+(?:\w+\b|\*+))\s*'
> +        #
> +        # We start by looking for function typedefs.
> +        #
> +        typedef_type = r'typedef((?:\s+[\w*]+\b){0,7}\s+(?:\w+\b|\*+))\s*'
>          typedef_ident = r'\*?\s*(\w\S+)\s*'
>          typedef_args = r'\s*\((.*)\);'
>  
> -        typedef1 = KernRe(r'typedef' + typedef_type + r'\(' + typedef_ident + r'\)' + typedef_args)
> -        typedef2 = KernRe(r'typedef' + typedef_type + typedef_ident + typedef_args)
> +        typedef1 = KernRe(typedef_type + r'\(' + typedef_ident + r'\)' + typedef_args)
> +        typedef2 = KernRe(typedef_type + typedef_ident + typedef_args)
>  
>          # Parse function typedef prototypes
>          for r in [typedef1, typedef2]:
> @@ -1048,16 +1050,16 @@ class KernelDoc:
>                                f"expecting prototype for typedef {self.entry.identifier}. Prototype was for typedef {declaration_name} instead\n")
>                  return
>  
> -            decl_type = 'function'
> -            self.create_parameter_list(ln, decl_type, args, ',', declaration_name)
> +            self.create_parameter_list(ln, 'function',_type, args, ',', declaration_name)
>  
> -            self.output_declaration(decl_type, declaration_name,
> +            self.output_declaration('function', declaration_name,
>                                      typedef=True,
>                                      functiontype=return_type,
>                                      purpose=self.entry.declaration_purpose)
>              return
> -
> -        # Parse simple typedefs
> +        #
> +        # Not a function, try to parse a simple typedef.
> +        #
>          r = KernRe(r'typedef.*\s+(\w+)\s*;')
>          if r.match(proto):
>              declaration_name = r.group(1)

Re: [PATCH 13/13] docs: kdoc: a few more dump_typedef() tweaks

[Jonathan Corbet]

Akira Yokosawa <akiyks@gmail.com> writes:

> Hi Jon,
>
> A quick report on minor regression. Please see below.
>
> On Tue,  9 Sep 2025 14:43:49 -0600, Jonathan Corbet wrote:
>> Merge "typedef" into the typedef_type pattern rather than repeating it
>> later, and add some comments.
>
> I'm seeing new warnings after applying 13/13:
>
> WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno ./include/media/demux.h' processing failed with: NameError("name '_type' is not defined")
> WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno ./include/media/v4l2-ioctl.h' processing failed with: NameError("name '_type' is not defined")
> WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno ./include/media/v4l2-ctrls.h' processing failed with: NameError("name '_type' is not defined")
> WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno ./include/media/v4l2-dv-timings.h' processing failed with: NameError("name '_type' is not defined")
> WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno ./include/media/videobuf2-core.h' processing failed with: NameError("name '_type' is not defined")
> WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno ./include/linux/hte.h' processing failed with: NameError("name '_type' is not defined")
> WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno ./include/linux/xarray.h' processing failed with: NameError("name '_type' is not defined")
> WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno -function gen_pool_add ./include/linux/genalloc.h' processing failed with: NameError("name '_type' is not defined")
> WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno -function gen_pool_alloc ./include/linux/genalloc.h' processing failed with: NameError("name '_type' is not defined")
> WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno -function vdso_sgx_enter_enclave_t ./arch/x86/include/uapi/asm/sgx.h' processing failed with: NameError("name '_type' is not defined")

OK, that is embarrassing, not sure how that got through.  My apologies.

An add-on fix is appended if you have the patience to try it; I'll
update the series before too long in any case.

Thanks for testing!

jon

diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index 8215948dd548..2376f180b1fa 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -1050,7 +1050,7 @@ class KernelDoc:
                               f"expecting prototype for typedef {self.entry.identifier}. Prototype was for typedef {declaration_name} instead\n")
                 return
 
-            self.create_parameter_list(ln, 'function',_type, args, ',', declaration_name)
+            self.create_parameter_list(ln, 'function', args, ',', declaration_name)
 
             self.output_declaration('function', declaration_name,
                                     typedef=True,

Re: [PATCH 13/13] docs: kdoc: a few more dump_typedef() tweaks

[Akira Yokosawa]

On Wed, 10 Sep 2025 07:46:16 -0600, Jonathan Corbet wrote:
> Akira Yokosawa <akiyks@gmail.com> writes:
> 
>> Hi Jon,
>>
>> A quick report on minor regression. Please see below.
>>
>> On Tue,  9 Sep 2025 14:43:49 -0600, Jonathan Corbet wrote:
>>> Merge "typedef" into the typedef_type pattern rather than repeating it
>>> later, and add some comments.
>>
>> I'm seeing new warnings after applying 13/13:
>>
>> WARNING: kernel-doc './scripts/kernel-doc.py -rst -enable-lineno ./include/media/demux.h' processing failed with: NameError("name '_type' is not defined")

[...]

> 
> OK, that is embarrassing, not sure how that got through.  My apologies.
> 
> An add-on fix is appended if you have the patience to try it; I'll
> update the series before too long in any case.

Testing the one-liner ... , and yes it silenced the warnings.

I can't do any serious reviews on python code, but can do tests on your
"improvements".

All those noisy warnings current docs-next carries ...

Regards,
Akira

> 
> Thanks for testing!
> 
> jon
> 
> diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
> index 8215948dd548..2376f180b1fa 100644
> --- a/scripts/lib/kdoc/kdoc_parser.py
> +++ b/scripts/lib/kdoc/kdoc_parser.py
> @@ -1050,7 +1050,7 @@ class KernelDoc:
>                                f"expecting prototype for typedef {self.entry.identifier}. Prototype was for typedef {declaration_name} instead\n")
>                  return
>  
> -            self.create_parameter_list(ln, 'function',_type, args, ',', declaration_name)
> +            self.create_parameter_list(ln, 'function', args, ',', declaration_name)
>  
>              self.output_declaration('function', declaration_name,
>                                      typedef=True,