linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 0/9] docs: kdoc: rework the BODY* processing states
@ 2025-06-21 20:35 Jonathan Corbet
  2025-06-21 20:35 ` [PATCH 1/9] docs: kdoc: Make body_with_blank_line parsing more flexible Jonathan Corbet
                   ` (8 more replies)
  0 siblings, 9 replies; 19+ messages in thread
From: Jonathan Corbet @ 2025-06-21 20:35 UTC (permalink / raw)
  To: linux-doc
  Cc: linux-kernel, Mauro Carvalho Chehab, Akira Yokosawa,
	Jonathan Corbet

This is a continuation of my effort to better understand the new kernel-doc
script and to clean up some of the legacy cruft brought over from the perl
version.

There were three states associated with the processing of the body of a
kerneldoc comment: BODY_MAYBE, BODY_WITH_BLANK_LINE, and BODY.
Unfortunately, these states did not actually match the state of the parser,
leading to a lot of deeply nested if-then-else code and special cases.
I've renamed the states to better reflect their meaning, and changed how
SPECIAL_SECTION, in particular, is managed.

  BODY_MAYBE becomes DECLARATION - we have seen the first line of the
  declaration that starts a kerneldoc comment, and don't know if we have
  hit the end of it or not.

  BODY_WITH_BLANK_LINE becomes SPECIAL_SECTION.  This state was used to
  recognize the sections within a comment with special rules -
  specifically, that the section ends with a blank line or the beginning of
  a new special section.  The declaration of parameters and sections like
  "Context" fit that description.  The old code recognized these sections
  at the *end*, with a lot of twisty logic; going into the proper state at
  the beginning simplifies things.  There are a few output changes, but I
  think they are all more correct.

  BODY remains BODY - a documentation section that does *not* end at a
  blank line.

To facilitate these changes, I have also begun the task of pulling more of
KernelEntry state management into the class itself; there is more to be
done on that front.

Jonathan Corbet (9):
  docs: kdoc: Make body_with_blank_line parsing more flexible
  docs: kdoc: consolidate the "begin section" logic
  docs: kdoc: separate out the handling of the declaration phase
  docs: kdoc: split out the special-section state
  docs: kdoc: coalesce the new-section handling
  docs: kdoc: rework the handling of SPECIAL_SECTION
  docs: kdoc: coalesce the end-of-comment processing
  docs: kdoc: Add some comments to process_decl()
  docs: kdoc: finish disentangling the BODY and SPECIAL_SECTION states

 scripts/lib/kdoc/kdoc_parser.py | 232 +++++++++++++++++++-------------
 1 file changed, 138 insertions(+), 94 deletions(-)

-- 
2.49.0


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [PATCH 1/9] docs: kdoc: Make body_with_blank_line parsing more flexible
  2025-06-21 20:35 [PATCH 0/9] docs: kdoc: rework the BODY* processing states Jonathan Corbet
@ 2025-06-21 20:35 ` Jonathan Corbet
  2025-06-22 11:46   ` Mauro Carvalho Chehab
  2025-06-21 20:35 ` [PATCH 2/9] docs: kdoc: consolidate the "begin section" logic Jonathan Corbet
                   ` (7 subsequent siblings)
  8 siblings, 1 reply; 19+ messages in thread
From: Jonathan Corbet @ 2025-06-21 20:35 UTC (permalink / raw)
  To: linux-doc
  Cc: linux-kernel, Mauro Carvalho Chehab, Akira Yokosawa,
	Jonathan Corbet

The regex in the BODY_WITH_BLANK_LINE case was looking for lines starting
with " * ", where exactly one space was allowed before the following text.
There are many kerneldoc comments where the authors have put multiple
spaces instead, leading to mis-formatting of the documentation.
Specifically, in this case, the description portion is associated with the
last of the parameters.

Allow multiple spaces in this context.

See, for example, synchronize_hardirq() and how its documentation is
formatted before and after the change.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 scripts/lib/kdoc/kdoc_parser.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index 42b2e0936b72..c46e1b6a7d4b 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -1309,7 +1309,7 @@ class KernelDoc:
         """
 
         if self.state == state.BODY_WITH_BLANK_LINE:
-            r = KernRe(r"\s*\*\s?\S")
+            r = KernRe(r"\s*\*\s*\S")
             if r.match(line):
                 self.dump_section()
                 self.entry.section = SECTION_DEFAULT
-- 
2.49.0


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH 2/9] docs: kdoc: consolidate the "begin section" logic
  2025-06-21 20:35 [PATCH 0/9] docs: kdoc: rework the BODY* processing states Jonathan Corbet
  2025-06-21 20:35 ` [PATCH 1/9] docs: kdoc: Make body_with_blank_line parsing more flexible Jonathan Corbet
@ 2025-06-21 20:35 ` Jonathan Corbet
  2025-06-22 11:47   ` Mauro Carvalho Chehab
  2025-06-21 20:35 ` [PATCH 3/9] docs: kdoc: separate out the handling of the declaration phase Jonathan Corbet
                   ` (6 subsequent siblings)
  8 siblings, 1 reply; 19+ messages in thread
From: Jonathan Corbet @ 2025-06-21 20:35 UTC (permalink / raw)
  To: linux-doc
  Cc: linux-kernel, Mauro Carvalho Chehab, Akira Yokosawa,
	Jonathan Corbet

Pull the repeated "begin a section" logic into a single place and hide it
within the KernelEntry class.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 scripts/lib/kdoc/kdoc_parser.py | 32 +++++++++++++++++---------------
 1 file changed, 17 insertions(+), 15 deletions(-)

diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index c46e1b6a7d4b..d29a61a06f6d 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -169,6 +169,15 @@ class KernelEntry:
         self.warnings.append(log_msg)
         return
 
+    #
+    # Begin a new section.
+    #
+    def begin_section(self, line_no, title = SECTION_DEFAULT, dump = False):
+        if dump:
+            self.dump_section(start_new = True)
+        self.section = title
+        self.new_start_line = line_no
+
     def dump_section(self, start_new=True):
         """
         Dumps section contents to arrays/hashes intended for that purpose.
@@ -1231,12 +1240,11 @@ class KernelDoc:
         # Check for a DOC: block and handle them specially.
         #
         if doc_block.search(line):
-            self.entry.new_start_line = ln
 
             if not doc_block.group(1):
-                self.entry.section = "Introduction"
+                self.entry.begin_section(ln, "Introduction")
             else:
-                self.entry.section = doc_block.group(1)
+                self.entry.begin_section(ln, doc_block.group(1))
 
             self.entry.identifier = self.entry.section
             self.state = state.DOCBLOCK
@@ -1270,8 +1278,7 @@ class KernelDoc:
             self.state = state.BODY
             self.entry.identifier = self.entry.identifier.strip(" ")
             # if there's no @param blocks need to set up default section here
-            self.entry.section = SECTION_DEFAULT
-            self.entry.new_start_line = ln + 1
+            self.entry.begin_section(ln + 1)
             #
             # Find the description portion, which *should* be there but
             # isn't always.
@@ -1312,8 +1319,7 @@ class KernelDoc:
             r = KernRe(r"\s*\*\s*\S")
             if r.match(line):
                 self.dump_section()
-                self.entry.section = SECTION_DEFAULT
-                self.entry.new_start_line = ln
+                self.entry.begin_section(ln)
                 self.entry.contents = ""
 
         if doc_sect.search(line):
@@ -1340,8 +1346,7 @@ class KernelDoc:
             if self.entry.contents.strip("\n"):
                 self.dump_section()
 
-            self.entry.new_start_line = ln
-            self.entry.section = newsection
+            self.entry.begin_section(ln, newsection)
             self.entry.leading_space = None
 
             self.entry.contents = newcontents.lstrip()
@@ -1370,9 +1375,7 @@ class KernelDoc:
 
             if cont == "":
                 if self.entry.section == self.section_context:
-                    self.dump_section()
-
-                    self.entry.new_start_line = ln
+                    self.entry.begin_section(ln, dump = True)
                     self.state = state.BODY
                 else:
                     if self.entry.section != SECTION_DEFAULT:
@@ -1427,8 +1430,7 @@ class KernelDoc:
 
         if self.inline_doc_state == state.INLINE_NAME and \
            doc_inline_sect.search(line):
-            self.entry.section = doc_inline_sect.group(1)
-            self.entry.new_start_line = ln
+            self.entry.begin_section(ln, doc_inline_sect.group(1))
 
             self.entry.contents = doc_inline_sect.group(2).lstrip()
             if self.entry.contents != "":
@@ -1627,7 +1629,7 @@ class KernelDoc:
         """STATE_PROTO: reading a function/whatever prototype."""
 
         if doc_inline_oneline.search(line):
-            self.entry.section = doc_inline_oneline.group(1)
+            self.entry.begin_section(ln, doc_inline_oneline.group(1))
             self.entry.contents = doc_inline_oneline.group(2)
 
             if self.entry.contents != "":
-- 
2.49.0


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH 3/9] docs: kdoc: separate out the handling of the declaration phase
  2025-06-21 20:35 [PATCH 0/9] docs: kdoc: rework the BODY* processing states Jonathan Corbet
  2025-06-21 20:35 ` [PATCH 1/9] docs: kdoc: Make body_with_blank_line parsing more flexible Jonathan Corbet
  2025-06-21 20:35 ` [PATCH 2/9] docs: kdoc: consolidate the "begin section" logic Jonathan Corbet
@ 2025-06-21 20:35 ` Jonathan Corbet
  2025-06-22 11:48   ` Mauro Carvalho Chehab
  2025-06-21 20:35 ` [PATCH 4/9] docs: kdoc: split out the special-section state Jonathan Corbet
                   ` (5 subsequent siblings)
  8 siblings, 1 reply; 19+ messages in thread
From: Jonathan Corbet @ 2025-06-21 20:35 UTC (permalink / raw)
  To: linux-doc
  Cc: linux-kernel, Mauro Carvalho Chehab, Akira Yokosawa,
	Jonathan Corbet

The BODY_MAYBE state really describes the "we are in a declaration" state.
Rename it accordingly, and split the handling of this state out from that
of the other BODY* states.  This change introduces a fair amount of
duplicated code that will be coalesced in a later patch.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 scripts/lib/kdoc/kdoc_parser.py | 93 +++++++++++++++++++++++++++------
 1 file changed, 78 insertions(+), 15 deletions(-)

diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index d29a61a06f6d..f1491f8c88e7 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -86,7 +86,7 @@ class state:
     # Parser states
     NORMAL        = 0        # normal code
     NAME          = 1        # looking for function name
-    BODY_MAYBE    = 2        # body - or maybe more description
+    DECLARATION   = 2        # We have seen a declaration which might not be done
     BODY          = 3        # the body of the comment
     BODY_WITH_BLANK_LINE = 4 # the body which has a blank line
     PROTO         = 5        # scanning prototype
@@ -96,7 +96,7 @@ class state:
     name = [
         "NORMAL",
         "NAME",
-        "BODY_MAYBE",
+        "DECLARATION",
         "BODY",
         "BODY_WITH_BLANK_LINE",
         "PROTO",
@@ -1287,7 +1287,7 @@ class KernelDoc:
             r = KernRe("[-:](.*)")
             if r.search(line):
                 self.entry.declaration_purpose = trim_whitespace(r.group(1))
-                self.state = state.BODY_MAYBE
+                self.state = state.DECLARATION
             else:
                 self.entry.declaration_purpose = ""
 
@@ -1310,9 +1310,82 @@ class KernelDoc:
         else:
             self.emit_msg(ln, f"Cannot find identifier on line:\n{line}")
 
+    def process_decl(self, ln, line):
+        """
+        STATE_DECLARATION: We've seen the beginning of a declaration
+        """
+        if doc_sect.search(line):
+            self.entry.in_doc_sect = True
+            newsection = doc_sect.group(1)
+
+            if newsection.lower() in ["description", "context"]:
+                newsection = newsection.title()
+
+            # Special case: @return is a section, not a param description
+            if newsection.lower() in ["@return", "@returns",
+                                      "return", "returns"]:
+                newsection = "Return"
+
+            # Perl kernel-doc has a check here for contents before sections.
+            # the logic there is always false, as in_doc_sect variable is
+            # always true. So, just don't implement Wcontents_before_sections
+
+            # .title()
+            newcontents = doc_sect.group(2)
+            if not newcontents:
+                newcontents = ""
+
+            if self.entry.contents.strip("\n"):
+                self.dump_section()
+
+            self.entry.begin_section(ln, newsection)
+            self.entry.leading_space = None
+
+            self.entry.contents = newcontents.lstrip()
+            if self.entry.contents:
+                self.entry.contents += "\n"
+
+            self.state = state.BODY
+            return
+
+        if doc_end.search(line):
+            self.dump_section()
+
+            # Look for doc_com + <text> + doc_end:
+            r = KernRe(r'\s*\*\s*[a-zA-Z_0-9:\.]+\*/')
+            if r.match(line):
+                self.emit_msg(ln, f"suspicious ending line: {line}")
+
+            self.entry.prototype = ""
+            self.entry.new_start_line = ln + 1
+
+            self.state = state.PROTO
+            return
+
+        if doc_content.search(line):
+            cont = doc_content.group(1)
+
+            if cont == "":
+                self.state = state.BODY
+                self.entry.contents += "\n"  # needed?
+
+            else:
+                # Continued declaration purpose
+                self.entry.declaration_purpose = self.entry.declaration_purpose.rstrip()
+                self.entry.declaration_purpose += " " + cont
+
+                r = KernRe(r"\s+")
+                self.entry.declaration_purpose = r.sub(' ',
+                                                       self.entry.declaration_purpose)
+            return
+
+        # Unknown line, ignore
+        self.emit_msg(ln, f"bad line: {line}")
+
+
     def process_body(self, ln, line):
         """
-        STATE_BODY and STATE_BODY_MAYBE: the bulk of a kerneldoc comment.
+        STATE_BODY: the bulk of a kerneldoc comment.
         """
 
         if self.state == state.BODY_WITH_BLANK_LINE:
@@ -1385,16 +1458,6 @@ class KernelDoc:
 
                     self.entry.contents += "\n"
 
-            elif self.state == state.BODY_MAYBE:
-
-                # Continued declaration purpose
-                self.entry.declaration_purpose = self.entry.declaration_purpose.rstrip()
-                self.entry.declaration_purpose += " " + cont
-
-                r = KernRe(r"\s+")
-                self.entry.declaration_purpose = r.sub(' ',
-                                                       self.entry.declaration_purpose)
-
             else:
                 if self.entry.section.startswith('@') or        \
                    self.entry.section == self.section_context:
@@ -1687,7 +1750,7 @@ class KernelDoc:
         state.NORMAL:			process_normal,
         state.NAME:			process_name,
         state.BODY:			process_body,
-        state.BODY_MAYBE:		process_body,
+        state.DECLARATION:		process_decl,
         state.BODY_WITH_BLANK_LINE:	process_body,
         state.INLINE:			process_inline,
         state.PROTO:			process_proto,
-- 
2.49.0


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH 4/9] docs: kdoc: split out the special-section state
  2025-06-21 20:35 [PATCH 0/9] docs: kdoc: rework the BODY* processing states Jonathan Corbet
                   ` (2 preceding siblings ...)
  2025-06-21 20:35 ` [PATCH 3/9] docs: kdoc: separate out the handling of the declaration phase Jonathan Corbet
@ 2025-06-21 20:35 ` Jonathan Corbet
  2025-06-22 11:50   ` Mauro Carvalho Chehab
  2025-06-21 20:35 ` [PATCH 5/9] docs: kdoc: coalesce the new-section handling Jonathan Corbet
                   ` (4 subsequent siblings)
  8 siblings, 1 reply; 19+ messages in thread
From: Jonathan Corbet @ 2025-06-21 20:35 UTC (permalink / raw)
  To: linux-doc
  Cc: linux-kernel, Mauro Carvalho Chehab, Akira Yokosawa,
	Jonathan Corbet

The state known as BODY_WITH_BLANK_LINE really, in a convoluted way,
indicates a "special section" that is terminated by a blank line or the
beginning of a new section.  That is either "@param: desc" sections, or the
weird "context" section that plays by the same rules.

Rename the state to SPECIAL_SECTION and split its processing into a
separate function; no real changes to the logic yet.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 scripts/lib/kdoc/kdoc_parser.py | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index f1491f8c88e7..185ffe4e1469 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -88,7 +88,7 @@ class state:
     NAME          = 1        # looking for function name
     DECLARATION   = 2        # We have seen a declaration which might not be done
     BODY          = 3        # the body of the comment
-    BODY_WITH_BLANK_LINE = 4 # the body which has a blank line
+    SPECIAL_SECTION = 4      # doc section ending with a blank line
     PROTO         = 5        # scanning prototype
     DOCBLOCK      = 6        # documentation block
     INLINE        = 7        # gathering doc outside main block
@@ -98,7 +98,7 @@ class state:
         "NAME",
         "DECLARATION",
         "BODY",
-        "BODY_WITH_BLANK_LINE",
+        "SPECIAL_SECTION",
         "PROTO",
         "DOCBLOCK",
         "INLINE",
@@ -1383,18 +1383,18 @@ class KernelDoc:
         self.emit_msg(ln, f"bad line: {line}")
 
 
+    def process_special(self, ln, line):
+        """
+        STATE_SPECIAL_SECTION: a section ending with a blank line
+        """
+        if KernRe(r"\s*\*\s*\S").match(line):
+            self.entry.begin_section(ln, dump = True)
+        self.process_body(ln, line)
+
     def process_body(self, ln, line):
         """
         STATE_BODY: the bulk of a kerneldoc comment.
         """
-
-        if self.state == state.BODY_WITH_BLANK_LINE:
-            r = KernRe(r"\s*\*\s*\S")
-            if r.match(line):
-                self.dump_section()
-                self.entry.begin_section(ln)
-                self.entry.contents = ""
-
         if doc_sect.search(line):
             self.entry.in_doc_sect = True
             newsection = doc_sect.group(1)
@@ -1452,7 +1452,7 @@ class KernelDoc:
                     self.state = state.BODY
                 else:
                     if self.entry.section != SECTION_DEFAULT:
-                        self.state = state.BODY_WITH_BLANK_LINE
+                        self.state = state.SPECIAL_SECTION
                     else:
                         self.state = state.BODY
 
@@ -1751,7 +1751,7 @@ class KernelDoc:
         state.NAME:			process_name,
         state.BODY:			process_body,
         state.DECLARATION:		process_decl,
-        state.BODY_WITH_BLANK_LINE:	process_body,
+        state.SPECIAL_SECTION:		process_special,
         state.INLINE:			process_inline,
         state.PROTO:			process_proto,
         state.DOCBLOCK:			process_docblock,
-- 
2.49.0


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH 5/9] docs: kdoc: coalesce the new-section handling
  2025-06-21 20:35 [PATCH 0/9] docs: kdoc: rework the BODY* processing states Jonathan Corbet
                   ` (3 preceding siblings ...)
  2025-06-21 20:35 ` [PATCH 4/9] docs: kdoc: split out the special-section state Jonathan Corbet
@ 2025-06-21 20:35 ` Jonathan Corbet
  2025-06-22 11:50   ` Mauro Carvalho Chehab
  2025-06-21 20:35 ` [PATCH 6/9] docs: kdoc: rework the handling of SPECIAL_SECTION Jonathan Corbet
                   ` (3 subsequent siblings)
  8 siblings, 1 reply; 19+ messages in thread
From: Jonathan Corbet @ 2025-06-21 20:35 UTC (permalink / raw)
  To: linux-doc
  Cc: linux-kernel, Mauro Carvalho Chehab, Akira Yokosawa,
	Jonathan Corbet

Merge the duplicated code back into a single implementation.  Code movement
only, no logic changes.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 scripts/lib/kdoc/kdoc_parser.py | 49 +++++++++------------------------
 1 file changed, 13 insertions(+), 36 deletions(-)

diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index 185ffe4e1469..a336d543e72b 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -1310,10 +1310,10 @@ class KernelDoc:
         else:
             self.emit_msg(ln, f"Cannot find identifier on line:\n{line}")
 
-    def process_decl(self, ln, line):
-        """
-        STATE_DECLARATION: We've seen the beginning of a declaration
-        """
+    #
+    # Helper function to determine if a new section is being started.
+    #
+    def is_new_section(self, ln, line):
         if doc_sect.search(line):
             self.entry.in_doc_sect = True
             newsection = doc_sect.group(1)
@@ -1346,6 +1346,14 @@ class KernelDoc:
                 self.entry.contents += "\n"
 
             self.state = state.BODY
+            return True
+        return False
+
+    def process_decl(self, ln, line):
+        """
+        STATE_DECLARATION: We've seen the beginning of a declaration
+        """
+        if self.is_new_section(ln, line):
             return
 
         if doc_end.search(line):
@@ -1395,38 +1403,7 @@ class KernelDoc:
         """
         STATE_BODY: the bulk of a kerneldoc comment.
         """
-        if doc_sect.search(line):
-            self.entry.in_doc_sect = True
-            newsection = doc_sect.group(1)
-
-            if newsection.lower() in ["description", "context"]:
-                newsection = newsection.title()
-
-            # Special case: @return is a section, not a param description
-            if newsection.lower() in ["@return", "@returns",
-                                      "return", "returns"]:
-                newsection = "Return"
-
-            # Perl kernel-doc has a check here for contents before sections.
-            # the logic there is always false, as in_doc_sect variable is
-            # always true. So, just don't implement Wcontents_before_sections
-
-            # .title()
-            newcontents = doc_sect.group(2)
-            if not newcontents:
-                newcontents = ""
-
-            if self.entry.contents.strip("\n"):
-                self.dump_section()
-
-            self.entry.begin_section(ln, newsection)
-            self.entry.leading_space = None
-
-            self.entry.contents = newcontents.lstrip()
-            if self.entry.contents:
-                self.entry.contents += "\n"
-
-            self.state = state.BODY
+        if self.is_new_section(ln, line):
             return
 
         if doc_end.search(line):
-- 
2.49.0


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH 6/9] docs: kdoc: rework the handling of SPECIAL_SECTION
  2025-06-21 20:35 [PATCH 0/9] docs: kdoc: rework the BODY* processing states Jonathan Corbet
                   ` (4 preceding siblings ...)
  2025-06-21 20:35 ` [PATCH 5/9] docs: kdoc: coalesce the new-section handling Jonathan Corbet
@ 2025-06-21 20:35 ` Jonathan Corbet
  2025-06-22 11:51   ` Mauro Carvalho Chehab
  2025-06-21 20:35 ` [PATCH 7/9] docs: kdoc: coalesce the end-of-comment processing Jonathan Corbet
                   ` (2 subsequent siblings)
  8 siblings, 1 reply; 19+ messages in thread
From: Jonathan Corbet @ 2025-06-21 20:35 UTC (permalink / raw)
  To: linux-doc
  Cc: linux-kernel, Mauro Carvalho Chehab, Akira Yokosawa,
	Jonathan Corbet

Move the recognition of this state to when we enter it, rather than when we
exit, eliminating some twisty logic along the way.

Some changes in output do result from this shift, generally for kerneldoc
comments that do not quite fit the format.  See, for example,
struct irqdomain.  As far as I can tell, the new behavior is more correct
in each case.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 scripts/lib/kdoc/kdoc_parser.py | 48 ++++++++++++++-------------------
 1 file changed, 20 insertions(+), 28 deletions(-)

diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index a336d543e72b..5998b02ca3a0 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -1316,21 +1316,25 @@ class KernelDoc:
     def is_new_section(self, ln, line):
         if doc_sect.search(line):
             self.entry.in_doc_sect = True
+            self.state = state.BODY
+            #
+            # Pick out the name of our new section, tweaking it if need be.
+            #
             newsection = doc_sect.group(1)
-
-            if newsection.lower() in ["description", "context"]:
-                newsection = newsection.title()
-
-            # Special case: @return is a section, not a param description
-            if newsection.lower() in ["@return", "@returns",
-                                      "return", "returns"]:
+            if newsection.lower() == 'description':
+                newsection = 'Description'
+            elif newsection.lower() == 'context':
+                newsection = 'Context'
+                self.state = state.SPECIAL_SECTION
+            elif newsection.lower() in ["@return", "@returns",
+                                        "return", "returns"]:
                 newsection = "Return"
-
-            # Perl kernel-doc has a check here for contents before sections.
-            # the logic there is always false, as in_doc_sect variable is
-            # always true. So, just don't implement Wcontents_before_sections
-
-            # .title()
+                self.state = state.SPECIAL_SECTION
+            elif newsection[0] == '@':
+                self.state = state.SPECIAL_SECTION
+            #
+            # Initialize the contents, and get the new section going.
+            #
             newcontents = doc_sect.group(2)
             if not newcontents:
                 newcontents = ""
@@ -1344,8 +1348,6 @@ class KernelDoc:
             self.entry.contents = newcontents.lstrip()
             if self.entry.contents:
                 self.entry.contents += "\n"
-
-            self.state = state.BODY
             return True
         return False
 
@@ -1395,8 +1397,9 @@ class KernelDoc:
         """
         STATE_SPECIAL_SECTION: a section ending with a blank line
         """
-        if KernRe(r"\s*\*\s*\S").match(line):
+        if KernRe(r"\s*\*\s*$").match(line):
             self.entry.begin_section(ln, dump = True)
+            self.state = state.BODY
         self.process_body(ln, line)
 
     def process_body(self, ln, line):
@@ -1424,20 +1427,9 @@ class KernelDoc:
             cont = doc_content.group(1)
 
             if cont == "":
-                if self.entry.section == self.section_context:
-                    self.entry.begin_section(ln, dump = True)
-                    self.state = state.BODY
-                else:
-                    if self.entry.section != SECTION_DEFAULT:
-                        self.state = state.SPECIAL_SECTION
-                    else:
-                        self.state = state.BODY
-
                     self.entry.contents += "\n"
-
             else:
-                if self.entry.section.startswith('@') or        \
-                   self.entry.section == self.section_context:
+                if self.state == state.SPECIAL_SECTION:
                     if self.entry.leading_space is None:
                         r = KernRe(r'^(\s+)')
                         if r.match(cont):
-- 
2.49.0


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH 7/9] docs: kdoc: coalesce the end-of-comment processing
  2025-06-21 20:35 [PATCH 0/9] docs: kdoc: rework the BODY* processing states Jonathan Corbet
                   ` (5 preceding siblings ...)
  2025-06-21 20:35 ` [PATCH 6/9] docs: kdoc: rework the handling of SPECIAL_SECTION Jonathan Corbet
@ 2025-06-21 20:35 ` Jonathan Corbet
  2025-06-22 11:52   ` Mauro Carvalho Chehab
  2025-06-21 20:35 ` [PATCH 8/9] docs: kdoc: Add some comments to process_decl() Jonathan Corbet
  2025-06-21 20:35 ` [PATCH 9/9] docs: kdoc: finish disentangling the BODY and SPECIAL_SECTION states Jonathan Corbet
  8 siblings, 1 reply; 19+ messages in thread
From: Jonathan Corbet @ 2025-06-21 20:35 UTC (permalink / raw)
  To: linux-doc
  Cc: linux-kernel, Mauro Carvalho Chehab, Akira Yokosawa,
	Jonathan Corbet

Separate out the end-of-comment logic into its own helper and remove the
duplicated code introduced earlier.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 scripts/lib/kdoc/kdoc_parser.py | 36 +++++++++++++--------------------
 1 file changed, 14 insertions(+), 22 deletions(-)

diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index 5998b02ca3a0..f7a5b85a8ed7 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -1351,13 +1351,10 @@ class KernelDoc:
             return True
         return False
 
-    def process_decl(self, ln, line):
-        """
-        STATE_DECLARATION: We've seen the beginning of a declaration
-        """
-        if self.is_new_section(ln, line):
-            return
-
+    #
+    # Helper function to detect (and effect) the end of a kerneldoc comment.
+    #
+    def is_comment_end(self, ln, line):
         if doc_end.search(line):
             self.dump_section()
 
@@ -1370,6 +1367,15 @@ class KernelDoc:
             self.entry.new_start_line = ln + 1
 
             self.state = state.PROTO
+            return True
+        return False
+
+
+    def process_decl(self, ln, line):
+        """
+        STATE_DECLARATION: We've seen the beginning of a declaration
+        """
+        if self.is_new_section(ln, line) or self.is_comment_end(ln, line):
             return
 
         if doc_content.search(line):
@@ -1406,21 +1412,7 @@ class KernelDoc:
         """
         STATE_BODY: the bulk of a kerneldoc comment.
         """
-        if self.is_new_section(ln, line):
-            return
-
-        if doc_end.search(line):
-            self.dump_section()
-
-            # Look for doc_com + <text> + doc_end:
-            r = KernRe(r'\s*\*\s*[a-zA-Z_0-9:\.]+\*/')
-            if r.match(line):
-                self.emit_msg(ln, f"suspicious ending line: {line}")
-
-            self.entry.prototype = ""
-            self.entry.new_start_line = ln + 1
-
-            self.state = state.PROTO
+        if self.is_new_section(ln, line) or self.is_comment_end(ln, line):
             return
 
         if doc_content.search(line):
-- 
2.49.0


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH 8/9] docs: kdoc: Add some comments to process_decl()
  2025-06-21 20:35 [PATCH 0/9] docs: kdoc: rework the BODY* processing states Jonathan Corbet
                   ` (6 preceding siblings ...)
  2025-06-21 20:35 ` [PATCH 7/9] docs: kdoc: coalesce the end-of-comment processing Jonathan Corbet
@ 2025-06-21 20:35 ` Jonathan Corbet
  2025-06-22 11:53   ` Mauro Carvalho Chehab
  2025-06-21 20:35 ` [PATCH 9/9] docs: kdoc: finish disentangling the BODY and SPECIAL_SECTION states Jonathan Corbet
  8 siblings, 1 reply; 19+ messages in thread
From: Jonathan Corbet @ 2025-06-21 20:35 UTC (permalink / raw)
  To: linux-doc
  Cc: linux-kernel, Mauro Carvalho Chehab, Akira Yokosawa,
	Jonathan Corbet

Now that the function can actually fit into a human brain, add a few
comments.  While I was at it, I switched to the trim_whitespace() helper
rather than open-coding it.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 scripts/lib/kdoc/kdoc_parser.py | 30 ++++++++++++++++--------------
 1 file changed, 16 insertions(+), 14 deletions(-)

diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index f7a5b85a8ed7..a6ee8bac378d 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -1377,26 +1377,28 @@ class KernelDoc:
         """
         if self.is_new_section(ln, line) or self.is_comment_end(ln, line):
             return
-
+        #
+        # Look for anything with the " * " line beginning.
+        #
         if doc_content.search(line):
             cont = doc_content.group(1)
-
+            #
+            # A blank line means that we have moved out of the declaration
+            # part of the comment (without any "special section" parameter
+            # descriptions).
+            #
             if cont == "":
                 self.state = state.BODY
                 self.entry.contents += "\n"  # needed?
-
+            #
+            # Otherwise we have more of the declaration section to soak up.
+            #
             else:
-                # Continued declaration purpose
-                self.entry.declaration_purpose = self.entry.declaration_purpose.rstrip()
-                self.entry.declaration_purpose += " " + cont
-
-                r = KernRe(r"\s+")
-                self.entry.declaration_purpose = r.sub(' ',
-                                                       self.entry.declaration_purpose)
-            return
-
-        # Unknown line, ignore
-        self.emit_msg(ln, f"bad line: {line}")
+                self.entry.declaration_purpose = \
+                    trim_whitespace(self.entry.declaration_purpose + ' ' + cont)
+        else:
+            # Unknown line, ignore
+            self.emit_msg(ln, f"bad line: {line}")
 
 
     def process_special(self, ln, line):
-- 
2.49.0


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH 9/9] docs: kdoc: finish disentangling the BODY and SPECIAL_SECTION states
  2025-06-21 20:35 [PATCH 0/9] docs: kdoc: rework the BODY* processing states Jonathan Corbet
                   ` (7 preceding siblings ...)
  2025-06-21 20:35 ` [PATCH 8/9] docs: kdoc: Add some comments to process_decl() Jonathan Corbet
@ 2025-06-21 20:35 ` Jonathan Corbet
  2025-06-22 11:54   ` Mauro Carvalho Chehab
  8 siblings, 1 reply; 19+ messages in thread
From: Jonathan Corbet @ 2025-06-21 20:35 UTC (permalink / raw)
  To: linux-doc
  Cc: linux-kernel, Mauro Carvalho Chehab, Akira Yokosawa,
	Jonathan Corbet

Move the last SPECIAL_SECTION special case into the proper handler
function, getting rid of more if/then/else logic.  The leading-space
tracking was tightened up a bit in the move.  Add some comments describing
what is going on.

No changes to the generated output.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 scripts/lib/kdoc/kdoc_parser.py | 80 ++++++++++++++++++++-------------
 1 file changed, 48 insertions(+), 32 deletions(-)

diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index a6ee8bac378d..3557c512c85a 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -1405,10 +1405,53 @@ class KernelDoc:
         """
         STATE_SPECIAL_SECTION: a section ending with a blank line
         """
+        #
+        # If we have hit a blank line (only the " * " marker), then this
+        # section is done.
+        #
         if KernRe(r"\s*\*\s*$").match(line):
             self.entry.begin_section(ln, dump = True)
+            self.entry.contents += '\n'
             self.state = state.BODY
-        self.process_body(ln, line)
+            return
+        #
+        # Not a blank line, look for the other ways to end the section.
+        #
+        if self.is_new_section(ln, line) or self.is_comment_end(ln, line):
+            return
+        #
+        # OK, we should have a continuation of the text for this section.
+        #
+        if doc_content.search(line):
+            cont = doc_content.group(1)
+            #
+            # If the lines of text after the first in a special section have
+            # leading white space, we need to trim it out or Sphinx will get
+            # confused.  For the second line (the None case), see what we
+            # find there and remember it.
+            #
+            if self.entry.leading_space is None:
+                r = KernRe(r'^(\s+)')
+                if r.match(cont):
+                    self.entry.leading_space = len(r.group(1))
+                else:
+                    self.entry.leading_space = 0
+            #
+            # Otherwise, before trimming any leading chars, be *sure*
+            # that they are white space.  We should maybe warn if this
+            # isn't the case.
+            #
+            for i in range(0, self.entry.leading_space):
+                if cont[i] != " ":
+                    self.entry.leading_space = i
+                    break
+            #
+            # Add the trimmed result to the section and we're done.
+            #
+            self.entry.contents += cont[self.entry.leading_space:] + '\n'
+        else:
+            # Unknown line, ignore
+            self.emit_msg(ln, f"bad line: {line}")
 
     def process_body(self, ln, line):
         """
@@ -1419,37 +1462,10 @@ class KernelDoc:
 
         if doc_content.search(line):
             cont = doc_content.group(1)
-
-            if cont == "":
-                    self.entry.contents += "\n"
-            else:
-                if self.state == state.SPECIAL_SECTION:
-                    if self.entry.leading_space is None:
-                        r = KernRe(r'^(\s+)')
-                        if r.match(cont):
-                            self.entry.leading_space = len(r.group(1))
-                        else:
-                            self.entry.leading_space = 0
-
-                    # Double-check if leading space are realy spaces
-                    pos = 0
-                    for i in range(0, self.entry.leading_space):
-                        if cont[i] != " ":
-                            break
-                        pos += 1
-
-                    cont = cont[pos:]
-
-                    # NEW LOGIC:
-                    # In case it is different, update it
-                    if self.entry.leading_space != pos:
-                        self.entry.leading_space = pos
-
-                self.entry.contents += cont + "\n"
-            return
-
-        # Unknown line, ignore
-        self.emit_msg(ln, f"bad line: {line}")
+            self.entry.contents += cont + "\n"
+        else:
+            # Unknown line, ignore
+            self.emit_msg(ln, f"bad line: {line}")
 
     def process_inline(self, ln, line):
         """STATE_INLINE: docbook comments within a prototype."""
-- 
2.49.0


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* Re: [PATCH 1/9] docs: kdoc: Make body_with_blank_line parsing more flexible
  2025-06-21 20:35 ` [PATCH 1/9] docs: kdoc: Make body_with_blank_line parsing more flexible Jonathan Corbet
@ 2025-06-22 11:46   ` Mauro Carvalho Chehab
  0 siblings, 0 replies; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2025-06-22 11:46 UTC (permalink / raw)
  To: Jonathan Corbet; +Cc: linux-doc, linux-kernel, Akira Yokosawa

Em Sat, 21 Jun 2025 14:35:04 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> The regex in the BODY_WITH_BLANK_LINE case was looking for lines starting
> with " * ", where exactly one space was allowed before the following text.
> There are many kerneldoc comments where the authors have put multiple
> spaces instead, leading to mis-formatting of the documentation.
> Specifically, in this case, the description portion is associated with the
> last of the parameters.
> 
> Allow multiple spaces in this context.
> 
> See, for example, synchronize_hardirq() and how its documentation is
> formatted before and after the change.
> 
> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
> ---
>  scripts/lib/kdoc/kdoc_parser.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
> index 42b2e0936b72..c46e1b6a7d4b 100644
> --- a/scripts/lib/kdoc/kdoc_parser.py
> +++ b/scripts/lib/kdoc/kdoc_parser.py
> @@ -1309,7 +1309,7 @@ class KernelDoc:
>          """
>  
>          if self.state == state.BODY_WITH_BLANK_LINE:
> -            r = KernRe(r"\s*\*\s?\S")
> +            r = KernRe(r"\s*\*\s*\S")
>              if r.match(line):
>                  self.dump_section()
>                  self.entry.section = SECTION_DEFAULT

I was a little bit afraid that this might be causing regressions,
but after checking the output, it looks ok. 

So:

Acked-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Tested-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>


Thanks,
Mauro

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH 2/9] docs: kdoc: consolidate the "begin section" logic
  2025-06-21 20:35 ` [PATCH 2/9] docs: kdoc: consolidate the "begin section" logic Jonathan Corbet
@ 2025-06-22 11:47   ` Mauro Carvalho Chehab
  0 siblings, 0 replies; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2025-06-22 11:47 UTC (permalink / raw)
  To: Jonathan Corbet; +Cc: linux-doc, linux-kernel, Akira Yokosawa

Em Sat, 21 Jun 2025 14:35:05 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> Pull the repeated "begin a section" logic into a single place and hide it
> within the KernelEntry class.
> 
> Signed-off-by: Jonathan Corbet <corbet@lwn.net>

Heh, I had a code snippet similar to this one on one of my test branches ;-)

LGTM.

Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

> ---
>  scripts/lib/kdoc/kdoc_parser.py | 32 +++++++++++++++++---------------
>  1 file changed, 17 insertions(+), 15 deletions(-)
> 
> diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
> index c46e1b6a7d4b..d29a61a06f6d 100644
> --- a/scripts/lib/kdoc/kdoc_parser.py
> +++ b/scripts/lib/kdoc/kdoc_parser.py
> @@ -169,6 +169,15 @@ class KernelEntry:
>          self.warnings.append(log_msg)
>          return
>  
> +    #
> +    # Begin a new section.
> +    #
> +    def begin_section(self, line_no, title = SECTION_DEFAULT, dump = False):
> +        if dump:
> +            self.dump_section(start_new = True)
> +        self.section = title
> +        self.new_start_line = line_no
> +
>      def dump_section(self, start_new=True):
>          """
>          Dumps section contents to arrays/hashes intended for that purpose.
> @@ -1231,12 +1240,11 @@ class KernelDoc:
>          # Check for a DOC: block and handle them specially.
>          #
>          if doc_block.search(line):
> -            self.entry.new_start_line = ln
>  
>              if not doc_block.group(1):
> -                self.entry.section = "Introduction"
> +                self.entry.begin_section(ln, "Introduction")
>              else:
> -                self.entry.section = doc_block.group(1)
> +                self.entry.begin_section(ln, doc_block.group(1))
>  
>              self.entry.identifier = self.entry.section
>              self.state = state.DOCBLOCK
> @@ -1270,8 +1278,7 @@ class KernelDoc:
>              self.state = state.BODY
>              self.entry.identifier = self.entry.identifier.strip(" ")
>              # if there's no @param blocks need to set up default section here
> -            self.entry.section = SECTION_DEFAULT
> -            self.entry.new_start_line = ln + 1
> +            self.entry.begin_section(ln + 1)
>              #
>              # Find the description portion, which *should* be there but
>              # isn't always.
> @@ -1312,8 +1319,7 @@ class KernelDoc:
>              r = KernRe(r"\s*\*\s*\S")
>              if r.match(line):
>                  self.dump_section()
> -                self.entry.section = SECTION_DEFAULT
> -                self.entry.new_start_line = ln
> +                self.entry.begin_section(ln)
>                  self.entry.contents = ""
>  
>          if doc_sect.search(line):
> @@ -1340,8 +1346,7 @@ class KernelDoc:
>              if self.entry.contents.strip("\n"):
>                  self.dump_section()
>  
> -            self.entry.new_start_line = ln
> -            self.entry.section = newsection
> +            self.entry.begin_section(ln, newsection)
>              self.entry.leading_space = None
>  
>              self.entry.contents = newcontents.lstrip()
> @@ -1370,9 +1375,7 @@ class KernelDoc:
>  
>              if cont == "":
>                  if self.entry.section == self.section_context:
> -                    self.dump_section()
> -
> -                    self.entry.new_start_line = ln
> +                    self.entry.begin_section(ln, dump = True)
>                      self.state = state.BODY
>                  else:
>                      if self.entry.section != SECTION_DEFAULT:
> @@ -1427,8 +1430,7 @@ class KernelDoc:
>  
>          if self.inline_doc_state == state.INLINE_NAME and \
>             doc_inline_sect.search(line):
> -            self.entry.section = doc_inline_sect.group(1)
> -            self.entry.new_start_line = ln
> +            self.entry.begin_section(ln, doc_inline_sect.group(1))
>  
>              self.entry.contents = doc_inline_sect.group(2).lstrip()
>              if self.entry.contents != "":
> @@ -1627,7 +1629,7 @@ class KernelDoc:
>          """STATE_PROTO: reading a function/whatever prototype."""
>  
>          if doc_inline_oneline.search(line):
> -            self.entry.section = doc_inline_oneline.group(1)
> +            self.entry.begin_section(ln, doc_inline_oneline.group(1))
>              self.entry.contents = doc_inline_oneline.group(2)
>  
>              if self.entry.contents != "":



Thanks,
Mauro

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH 3/9] docs: kdoc: separate out the handling of the declaration phase
  2025-06-21 20:35 ` [PATCH 3/9] docs: kdoc: separate out the handling of the declaration phase Jonathan Corbet
@ 2025-06-22 11:48   ` Mauro Carvalho Chehab
  0 siblings, 0 replies; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2025-06-22 11:48 UTC (permalink / raw)
  To: Jonathan Corbet; +Cc: linux-doc, linux-kernel, Akira Yokosawa

Em Sat, 21 Jun 2025 14:35:06 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> The BODY_MAYBE state really describes the "we are in a declaration" state.
> Rename it accordingly, and split the handling of this state out from that
> of the other BODY* states.  This change introduces a fair amount of
> duplicated code that will be coalesced in a later patch.
> 
> Signed-off-by: Jonathan Corbet <corbet@lwn.net>

LGTM.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

> ---
>  scripts/lib/kdoc/kdoc_parser.py | 93 +++++++++++++++++++++++++++------
>  1 file changed, 78 insertions(+), 15 deletions(-)
> 
> diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
> index d29a61a06f6d..f1491f8c88e7 100644
> --- a/scripts/lib/kdoc/kdoc_parser.py
> +++ b/scripts/lib/kdoc/kdoc_parser.py
> @@ -86,7 +86,7 @@ class state:
>      # Parser states
>      NORMAL        = 0        # normal code
>      NAME          = 1        # looking for function name
> -    BODY_MAYBE    = 2        # body - or maybe more description
> +    DECLARATION   = 2        # We have seen a declaration which might not be done

A way better now.

>      BODY          = 3        # the body of the comment
>      BODY_WITH_BLANK_LINE = 4 # the body which has a blank line
>      PROTO         = 5        # scanning prototype
> @@ -96,7 +96,7 @@ class state:
>      name = [
>          "NORMAL",
>          "NAME",
> -        "BODY_MAYBE",
> +        "DECLARATION",
>          "BODY",
>          "BODY_WITH_BLANK_LINE",
>          "PROTO",
> @@ -1287,7 +1287,7 @@ class KernelDoc:
>              r = KernRe("[-:](.*)")
>              if r.search(line):
>                  self.entry.declaration_purpose = trim_whitespace(r.group(1))
> -                self.state = state.BODY_MAYBE
> +                self.state = state.DECLARATION
>              else:
>                  self.entry.declaration_purpose = ""
>  
> @@ -1310,9 +1310,82 @@ class KernelDoc:
>          else:
>              self.emit_msg(ln, f"Cannot find identifier on line:\n{line}")
>  
> +    def process_decl(self, ln, line):
> +        """
> +        STATE_DECLARATION: We've seen the beginning of a declaration
> +        """
> +        if doc_sect.search(line):
> +            self.entry.in_doc_sect = True
> +            newsection = doc_sect.group(1)
> +
> +            if newsection.lower() in ["description", "context"]:
> +                newsection = newsection.title()
> +
> +            # Special case: @return is a section, not a param description
> +            if newsection.lower() in ["@return", "@returns",
> +                                      "return", "returns"]:
> +                newsection = "Return"
> +
> +            # Perl kernel-doc has a check here for contents before sections.
> +            # the logic there is always false, as in_doc_sect variable is
> +            # always true. So, just don't implement Wcontents_before_sections
> +
> +            # .title()
> +            newcontents = doc_sect.group(2)
> +            if not newcontents:
> +                newcontents = ""
> +
> +            if self.entry.contents.strip("\n"):
> +                self.dump_section()
> +
> +            self.entry.begin_section(ln, newsection)
> +            self.entry.leading_space = None
> +
> +            self.entry.contents = newcontents.lstrip()
> +            if self.entry.contents:
> +                self.entry.contents += "\n"
> +
> +            self.state = state.BODY
> +            return
> +
> +        if doc_end.search(line):
> +            self.dump_section()
> +
> +            # Look for doc_com + <text> + doc_end:
> +            r = KernRe(r'\s*\*\s*[a-zA-Z_0-9:\.]+\*/')
> +            if r.match(line):
> +                self.emit_msg(ln, f"suspicious ending line: {line}")
> +
> +            self.entry.prototype = ""
> +            self.entry.new_start_line = ln + 1
> +
> +            self.state = state.PROTO
> +            return
> +
> +        if doc_content.search(line):
> +            cont = doc_content.group(1)
> +
> +            if cont == "":
> +                self.state = state.BODY
> +                self.entry.contents += "\n"  # needed?
> +
> +            else:
> +                # Continued declaration purpose
> +                self.entry.declaration_purpose = self.entry.declaration_purpose.rstrip()
> +                self.entry.declaration_purpose += " " + cont
> +
> +                r = KernRe(r"\s+")
> +                self.entry.declaration_purpose = r.sub(' ',
> +                                                       self.entry.declaration_purpose)
> +            return
> +
> +        # Unknown line, ignore
> +        self.emit_msg(ln, f"bad line: {line}")
> +
> +
>      def process_body(self, ln, line):
>          """
> -        STATE_BODY and STATE_BODY_MAYBE: the bulk of a kerneldoc comment.
> +        STATE_BODY: the bulk of a kerneldoc comment.
>          """
>  
>          if self.state == state.BODY_WITH_BLANK_LINE:
> @@ -1385,16 +1458,6 @@ class KernelDoc:
>  
>                      self.entry.contents += "\n"
>  
> -            elif self.state == state.BODY_MAYBE:
> -
> -                # Continued declaration purpose
> -                self.entry.declaration_purpose = self.entry.declaration_purpose.rstrip()
> -                self.entry.declaration_purpose += " " + cont
> -
> -                r = KernRe(r"\s+")
> -                self.entry.declaration_purpose = r.sub(' ',
> -                                                       self.entry.declaration_purpose)
> -
>              else:
>                  if self.entry.section.startswith('@') or        \
>                     self.entry.section == self.section_context:
> @@ -1687,7 +1750,7 @@ class KernelDoc:
>          state.NORMAL:			process_normal,
>          state.NAME:			process_name,
>          state.BODY:			process_body,
> -        state.BODY_MAYBE:		process_body,
> +        state.DECLARATION:		process_decl,
>          state.BODY_WITH_BLANK_LINE:	process_body,
>          state.INLINE:			process_inline,
>          state.PROTO:			process_proto,



Thanks,
Mauro

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH 4/9] docs: kdoc: split out the special-section state
  2025-06-21 20:35 ` [PATCH 4/9] docs: kdoc: split out the special-section state Jonathan Corbet
@ 2025-06-22 11:50   ` Mauro Carvalho Chehab
  0 siblings, 0 replies; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2025-06-22 11:50 UTC (permalink / raw)
  To: Jonathan Corbet; +Cc: linux-doc, linux-kernel, Akira Yokosawa

Em Sat, 21 Jun 2025 14:35:07 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> The state known as BODY_WITH_BLANK_LINE really, in a convoluted way,
> indicates a "special section" that is terminated by a blank line or the
> beginning of a new section.  That is either "@param: desc" sections, or the
> weird "context" section that plays by the same rules.
> 
> Rename the state to SPECIAL_SECTION and split its processing into a
> separate function; no real changes to the logic yet.

LGTM.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

> 
> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
> ---
>  scripts/lib/kdoc/kdoc_parser.py | 24 ++++++++++++------------
>  1 file changed, 12 insertions(+), 12 deletions(-)
> 
> diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
> index f1491f8c88e7..185ffe4e1469 100644
> --- a/scripts/lib/kdoc/kdoc_parser.py
> +++ b/scripts/lib/kdoc/kdoc_parser.py
> @@ -88,7 +88,7 @@ class state:
>      NAME          = 1        # looking for function name
>      DECLARATION   = 2        # We have seen a declaration which might not be done
>      BODY          = 3        # the body of the comment
> -    BODY_WITH_BLANK_LINE = 4 # the body which has a blank line
> +    SPECIAL_SECTION = 4      # doc section ending with a blank line
>      PROTO         = 5        # scanning prototype
>      DOCBLOCK      = 6        # documentation block
>      INLINE        = 7        # gathering doc outside main block
> @@ -98,7 +98,7 @@ class state:
>          "NAME",
>          "DECLARATION",
>          "BODY",
> -        "BODY_WITH_BLANK_LINE",
> +        "SPECIAL_SECTION",
>          "PROTO",
>          "DOCBLOCK",
>          "INLINE",
> @@ -1383,18 +1383,18 @@ class KernelDoc:
>          self.emit_msg(ln, f"bad line: {line}")
>  
>  
> +    def process_special(self, ln, line):
> +        """
> +        STATE_SPECIAL_SECTION: a section ending with a blank line
> +        """
> +        if KernRe(r"\s*\*\s*\S").match(line):
> +            self.entry.begin_section(ln, dump = True)
> +        self.process_body(ln, line)
> +
>      def process_body(self, ln, line):
>          """
>          STATE_BODY: the bulk of a kerneldoc comment.
>          """
> -
> -        if self.state == state.BODY_WITH_BLANK_LINE:
> -            r = KernRe(r"\s*\*\s*\S")
> -            if r.match(line):
> -                self.dump_section()
> -                self.entry.begin_section(ln)
> -                self.entry.contents = ""
> -
>          if doc_sect.search(line):
>              self.entry.in_doc_sect = True
>              newsection = doc_sect.group(1)
> @@ -1452,7 +1452,7 @@ class KernelDoc:
>                      self.state = state.BODY
>                  else:
>                      if self.entry.section != SECTION_DEFAULT:
> -                        self.state = state.BODY_WITH_BLANK_LINE
> +                        self.state = state.SPECIAL_SECTION
>                      else:
>                          self.state = state.BODY
>  
> @@ -1751,7 +1751,7 @@ class KernelDoc:
>          state.NAME:			process_name,
>          state.BODY:			process_body,
>          state.DECLARATION:		process_decl,
> -        state.BODY_WITH_BLANK_LINE:	process_body,
> +        state.SPECIAL_SECTION:		process_special,
>          state.INLINE:			process_inline,
>          state.PROTO:			process_proto,
>          state.DOCBLOCK:			process_docblock,



Thanks,
Mauro

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH 5/9] docs: kdoc: coalesce the new-section handling
  2025-06-21 20:35 ` [PATCH 5/9] docs: kdoc: coalesce the new-section handling Jonathan Corbet
@ 2025-06-22 11:50   ` Mauro Carvalho Chehab
  0 siblings, 0 replies; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2025-06-22 11:50 UTC (permalink / raw)
  To: Jonathan Corbet; +Cc: linux-doc, linux-kernel, Akira Yokosawa

Em Sat, 21 Jun 2025 14:35:08 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> Merge the duplicated code back into a single implementation.  Code movement
> only, no logic changes.
> 
> Signed-off-by: Jonathan Corbet <corbet@lwn.net>

LGTM.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

> ---
>  scripts/lib/kdoc/kdoc_parser.py | 49 +++++++++------------------------
>  1 file changed, 13 insertions(+), 36 deletions(-)
> 
> diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
> index 185ffe4e1469..a336d543e72b 100644
> --- a/scripts/lib/kdoc/kdoc_parser.py
> +++ b/scripts/lib/kdoc/kdoc_parser.py
> @@ -1310,10 +1310,10 @@ class KernelDoc:
>          else:
>              self.emit_msg(ln, f"Cannot find identifier on line:\n{line}")
>  
> -    def process_decl(self, ln, line):
> -        """
> -        STATE_DECLARATION: We've seen the beginning of a declaration
> -        """
> +    #
> +    # Helper function to determine if a new section is being started.
> +    #
> +    def is_new_section(self, ln, line):
>          if doc_sect.search(line):
>              self.entry.in_doc_sect = True
>              newsection = doc_sect.group(1)
> @@ -1346,6 +1346,14 @@ class KernelDoc:
>                  self.entry.contents += "\n"
>  
>              self.state = state.BODY
> +            return True
> +        return False
> +
> +    def process_decl(self, ln, line):
> +        """
> +        STATE_DECLARATION: We've seen the beginning of a declaration
> +        """
> +        if self.is_new_section(ln, line):
>              return
>  
>          if doc_end.search(line):
> @@ -1395,38 +1403,7 @@ class KernelDoc:
>          """
>          STATE_BODY: the bulk of a kerneldoc comment.
>          """
> -        if doc_sect.search(line):
> -            self.entry.in_doc_sect = True
> -            newsection = doc_sect.group(1)
> -
> -            if newsection.lower() in ["description", "context"]:
> -                newsection = newsection.title()
> -
> -            # Special case: @return is a section, not a param description
> -            if newsection.lower() in ["@return", "@returns",
> -                                      "return", "returns"]:
> -                newsection = "Return"
> -
> -            # Perl kernel-doc has a check here for contents before sections.
> -            # the logic there is always false, as in_doc_sect variable is
> -            # always true. So, just don't implement Wcontents_before_sections
> -
> -            # .title()
> -            newcontents = doc_sect.group(2)
> -            if not newcontents:
> -                newcontents = ""
> -
> -            if self.entry.contents.strip("\n"):
> -                self.dump_section()
> -
> -            self.entry.begin_section(ln, newsection)
> -            self.entry.leading_space = None
> -
> -            self.entry.contents = newcontents.lstrip()
> -            if self.entry.contents:
> -                self.entry.contents += "\n"
> -
> -            self.state = state.BODY
> +        if self.is_new_section(ln, line):
>              return
>  
>          if doc_end.search(line):



Thanks,
Mauro

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH 6/9] docs: kdoc: rework the handling of SPECIAL_SECTION
  2025-06-21 20:35 ` [PATCH 6/9] docs: kdoc: rework the handling of SPECIAL_SECTION Jonathan Corbet
@ 2025-06-22 11:51   ` Mauro Carvalho Chehab
  0 siblings, 0 replies; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2025-06-22 11:51 UTC (permalink / raw)
  To: Jonathan Corbet; +Cc: linux-doc, linux-kernel, Akira Yokosawa

Em Sat, 21 Jun 2025 14:35:09 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> Move the recognition of this state to when we enter it, rather than when we
> exit, eliminating some twisty logic along the way.
> 
> Some changes in output do result from this shift, generally for kerneldoc
> comments that do not quite fit the format.  See, for example,
> struct irqdomain.  As far as I can tell, the new behavior is more correct
> in each case.
> 
> Signed-off-by: Jonathan Corbet <corbet@lwn.net>

LGTM.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

> ---
>  scripts/lib/kdoc/kdoc_parser.py | 48 ++++++++++++++-------------------
>  1 file changed, 20 insertions(+), 28 deletions(-)
> 
> diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
> index a336d543e72b..5998b02ca3a0 100644
> --- a/scripts/lib/kdoc/kdoc_parser.py
> +++ b/scripts/lib/kdoc/kdoc_parser.py
> @@ -1316,21 +1316,25 @@ class KernelDoc:
>      def is_new_section(self, ln, line):
>          if doc_sect.search(line):
>              self.entry.in_doc_sect = True
> +            self.state = state.BODY
> +            #
> +            # Pick out the name of our new section, tweaking it if need be.
> +            #
>              newsection = doc_sect.group(1)
> -
> -            if newsection.lower() in ["description", "context"]:
> -                newsection = newsection.title()
> -
> -            # Special case: @return is a section, not a param description
> -            if newsection.lower() in ["@return", "@returns",
> -                                      "return", "returns"]:
> +            if newsection.lower() == 'description':
> +                newsection = 'Description'
> +            elif newsection.lower() == 'context':
> +                newsection = 'Context'
> +                self.state = state.SPECIAL_SECTION
> +            elif newsection.lower() in ["@return", "@returns",
> +                                        "return", "returns"]:
>                  newsection = "Return"
> -
> -            # Perl kernel-doc has a check here for contents before sections.
> -            # the logic there is always false, as in_doc_sect variable is
> -            # always true. So, just don't implement Wcontents_before_sections
> -
> -            # .title()
> +                self.state = state.SPECIAL_SECTION
> +            elif newsection[0] == '@':
> +                self.state = state.SPECIAL_SECTION
> +            #
> +            # Initialize the contents, and get the new section going.
> +            #
>              newcontents = doc_sect.group(2)
>              if not newcontents:
>                  newcontents = ""
> @@ -1344,8 +1348,6 @@ class KernelDoc:
>              self.entry.contents = newcontents.lstrip()
>              if self.entry.contents:
>                  self.entry.contents += "\n"
> -
> -            self.state = state.BODY
>              return True
>          return False
>  
> @@ -1395,8 +1397,9 @@ class KernelDoc:
>          """
>          STATE_SPECIAL_SECTION: a section ending with a blank line
>          """
> -        if KernRe(r"\s*\*\s*\S").match(line):
> +        if KernRe(r"\s*\*\s*$").match(line):
>              self.entry.begin_section(ln, dump = True)
> +            self.state = state.BODY
>          self.process_body(ln, line)
>  
>      def process_body(self, ln, line):
> @@ -1424,20 +1427,9 @@ class KernelDoc:
>              cont = doc_content.group(1)
>  
>              if cont == "":
> -                if self.entry.section == self.section_context:
> -                    self.entry.begin_section(ln, dump = True)
> -                    self.state = state.BODY
> -                else:
> -                    if self.entry.section != SECTION_DEFAULT:
> -                        self.state = state.SPECIAL_SECTION
> -                    else:
> -                        self.state = state.BODY
> -
>                      self.entry.contents += "\n"
> -
>              else:
> -                if self.entry.section.startswith('@') or        \
> -                   self.entry.section == self.section_context:
> +                if self.state == state.SPECIAL_SECTION:
>                      if self.entry.leading_space is None:
>                          r = KernRe(r'^(\s+)')
>                          if r.match(cont):



Thanks,
Mauro

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH 7/9] docs: kdoc: coalesce the end-of-comment processing
  2025-06-21 20:35 ` [PATCH 7/9] docs: kdoc: coalesce the end-of-comment processing Jonathan Corbet
@ 2025-06-22 11:52   ` Mauro Carvalho Chehab
  0 siblings, 0 replies; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2025-06-22 11:52 UTC (permalink / raw)
  To: Jonathan Corbet; +Cc: linux-doc, linux-kernel, Akira Yokosawa

Em Sat, 21 Jun 2025 14:35:10 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> Separate out the end-of-comment logic into its own helper and remove the
> duplicated code introduced earlier.
> 
> Signed-off-by: Jonathan Corbet <corbet@lwn.net>

LGTM.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

> ---
>  scripts/lib/kdoc/kdoc_parser.py | 36 +++++++++++++--------------------
>  1 file changed, 14 insertions(+), 22 deletions(-)
> 
> diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
> index 5998b02ca3a0..f7a5b85a8ed7 100644
> --- a/scripts/lib/kdoc/kdoc_parser.py
> +++ b/scripts/lib/kdoc/kdoc_parser.py
> @@ -1351,13 +1351,10 @@ class KernelDoc:
>              return True
>          return False
>  
> -    def process_decl(self, ln, line):
> -        """
> -        STATE_DECLARATION: We've seen the beginning of a declaration
> -        """
> -        if self.is_new_section(ln, line):
> -            return
> -
> +    #
> +    # Helper function to detect (and effect) the end of a kerneldoc comment.
> +    #
> +    def is_comment_end(self, ln, line):
>          if doc_end.search(line):
>              self.dump_section()
>  
> @@ -1370,6 +1367,15 @@ class KernelDoc:
>              self.entry.new_start_line = ln + 1
>  
>              self.state = state.PROTO
> +            return True
> +        return False
> +
> +
> +    def process_decl(self, ln, line):
> +        """
> +        STATE_DECLARATION: We've seen the beginning of a declaration
> +        """
> +        if self.is_new_section(ln, line) or self.is_comment_end(ln, line):
>              return
>  
>          if doc_content.search(line):
> @@ -1406,21 +1412,7 @@ class KernelDoc:
>          """
>          STATE_BODY: the bulk of a kerneldoc comment.
>          """
> -        if self.is_new_section(ln, line):
> -            return
> -
> -        if doc_end.search(line):
> -            self.dump_section()
> -
> -            # Look for doc_com + <text> + doc_end:
> -            r = KernRe(r'\s*\*\s*[a-zA-Z_0-9:\.]+\*/')
> -            if r.match(line):
> -                self.emit_msg(ln, f"suspicious ending line: {line}")
> -
> -            self.entry.prototype = ""
> -            self.entry.new_start_line = ln + 1
> -
> -            self.state = state.PROTO
> +        if self.is_new_section(ln, line) or self.is_comment_end(ln, line):
>              return
>  
>          if doc_content.search(line):



Thanks,
Mauro

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH 8/9] docs: kdoc: Add some comments to process_decl()
  2025-06-21 20:35 ` [PATCH 8/9] docs: kdoc: Add some comments to process_decl() Jonathan Corbet
@ 2025-06-22 11:53   ` Mauro Carvalho Chehab
  0 siblings, 0 replies; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2025-06-22 11:53 UTC (permalink / raw)
  To: Jonathan Corbet; +Cc: linux-doc, linux-kernel, Akira Yokosawa

Em Sat, 21 Jun 2025 14:35:11 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> Now that the function can actually fit into a human brain, add a few
> comments.  While I was at it, I switched to the trim_whitespace() helper
> rather than open-coding it.
> 
> Signed-off-by: Jonathan Corbet <corbet@lwn.net>

LGTM.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

> ---
>  scripts/lib/kdoc/kdoc_parser.py | 30 ++++++++++++++++--------------
>  1 file changed, 16 insertions(+), 14 deletions(-)
> 
> diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
> index f7a5b85a8ed7..a6ee8bac378d 100644
> --- a/scripts/lib/kdoc/kdoc_parser.py
> +++ b/scripts/lib/kdoc/kdoc_parser.py
> @@ -1377,26 +1377,28 @@ class KernelDoc:
>          """
>          if self.is_new_section(ln, line) or self.is_comment_end(ln, line):
>              return
> -
> +        #
> +        # Look for anything with the " * " line beginning.
> +        #
>          if doc_content.search(line):
>              cont = doc_content.group(1)
> -
> +            #
> +            # A blank line means that we have moved out of the declaration
> +            # part of the comment (without any "special section" parameter
> +            # descriptions).
> +            #
>              if cont == "":
>                  self.state = state.BODY
>                  self.entry.contents += "\n"  # needed?
> -
> +            #
> +            # Otherwise we have more of the declaration section to soak up.
> +            #
>              else:
> -                # Continued declaration purpose
> -                self.entry.declaration_purpose = self.entry.declaration_purpose.rstrip()
> -                self.entry.declaration_purpose += " " + cont
> -
> -                r = KernRe(r"\s+")
> -                self.entry.declaration_purpose = r.sub(' ',
> -                                                       self.entry.declaration_purpose)
> -            return
> -
> -        # Unknown line, ignore
> -        self.emit_msg(ln, f"bad line: {line}")
> +                self.entry.declaration_purpose = \
> +                    trim_whitespace(self.entry.declaration_purpose + ' ' + cont)
> +        else:
> +            # Unknown line, ignore
> +            self.emit_msg(ln, f"bad line: {line}")
>  
>  
>      def process_special(self, ln, line):



Thanks,
Mauro

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH 9/9] docs: kdoc: finish disentangling the BODY and SPECIAL_SECTION states
  2025-06-21 20:35 ` [PATCH 9/9] docs: kdoc: finish disentangling the BODY and SPECIAL_SECTION states Jonathan Corbet
@ 2025-06-22 11:54   ` Mauro Carvalho Chehab
  0 siblings, 0 replies; 19+ messages in thread
From: Mauro Carvalho Chehab @ 2025-06-22 11:54 UTC (permalink / raw)
  To: Jonathan Corbet; +Cc: linux-doc, linux-kernel, Akira Yokosawa

Em Sat, 21 Jun 2025 14:35:12 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> Move the last SPECIAL_SECTION special case into the proper handler
> function, getting rid of more if/then/else logic.  The leading-space
> tracking was tightened up a bit in the move.  Add some comments describing
> what is going on.
> 
> No changes to the generated output.

LGTM.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
> ---
>  scripts/lib/kdoc/kdoc_parser.py | 80 ++++++++++++++++++++-------------
>  1 file changed, 48 insertions(+), 32 deletions(-)
> 
> diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
> index a6ee8bac378d..3557c512c85a 100644
> --- a/scripts/lib/kdoc/kdoc_parser.py
> +++ b/scripts/lib/kdoc/kdoc_parser.py
> @@ -1405,10 +1405,53 @@ class KernelDoc:
>          """
>          STATE_SPECIAL_SECTION: a section ending with a blank line
>          """
> +        #
> +        # If we have hit a blank line (only the " * " marker), then this
> +        # section is done.
> +        #
>          if KernRe(r"\s*\*\s*$").match(line):
>              self.entry.begin_section(ln, dump = True)
> +            self.entry.contents += '\n'
>              self.state = state.BODY
> -        self.process_body(ln, line)
> +            return
> +        #
> +        # Not a blank line, look for the other ways to end the section.
> +        #
> +        if self.is_new_section(ln, line) or self.is_comment_end(ln, line):
> +            return
> +        #
> +        # OK, we should have a continuation of the text for this section.
> +        #
> +        if doc_content.search(line):
> +            cont = doc_content.group(1)
> +            #
> +            # If the lines of text after the first in a special section have
> +            # leading white space, we need to trim it out or Sphinx will get
> +            # confused.  For the second line (the None case), see what we
> +            # find there and remember it.
> +            #
> +            if self.entry.leading_space is None:
> +                r = KernRe(r'^(\s+)')
> +                if r.match(cont):
> +                    self.entry.leading_space = len(r.group(1))
> +                else:
> +                    self.entry.leading_space = 0
> +            #
> +            # Otherwise, before trimming any leading chars, be *sure*
> +            # that they are white space.  We should maybe warn if this
> +            # isn't the case.
> +            #
> +            for i in range(0, self.entry.leading_space):
> +                if cont[i] != " ":
> +                    self.entry.leading_space = i
> +                    break
> +            #
> +            # Add the trimmed result to the section and we're done.
> +            #
> +            self.entry.contents += cont[self.entry.leading_space:] + '\n'
> +        else:
> +            # Unknown line, ignore
> +            self.emit_msg(ln, f"bad line: {line}")
>  
>      def process_body(self, ln, line):
>          """
> @@ -1419,37 +1462,10 @@ class KernelDoc:
>  
>          if doc_content.search(line):
>              cont = doc_content.group(1)
> -
> -            if cont == "":
> -                    self.entry.contents += "\n"
> -            else:
> -                if self.state == state.SPECIAL_SECTION:
> -                    if self.entry.leading_space is None:
> -                        r = KernRe(r'^(\s+)')
> -                        if r.match(cont):
> -                            self.entry.leading_space = len(r.group(1))
> -                        else:
> -                            self.entry.leading_space = 0
> -
> -                    # Double-check if leading space are realy spaces
> -                    pos = 0
> -                    for i in range(0, self.entry.leading_space):
> -                        if cont[i] != " ":
> -                            break
> -                        pos += 1
> -
> -                    cont = cont[pos:]
> -
> -                    # NEW LOGIC:
> -                    # In case it is different, update it
> -                    if self.entry.leading_space != pos:
> -                        self.entry.leading_space = pos
> -
> -                self.entry.contents += cont + "\n"
> -            return
> -
> -        # Unknown line, ignore
> -        self.emit_msg(ln, f"bad line: {line}")
> +            self.entry.contents += cont + "\n"
> +        else:
> +            # Unknown line, ignore
> +            self.emit_msg(ln, f"bad line: {line}")
>  
>      def process_inline(self, ln, line):
>          """STATE_INLINE: docbook comments within a prototype."""



Thanks,
Mauro

^ permalink raw reply	[flat|nested] 19+ messages in thread

end of thread, other threads:[~2025-06-22 11:54 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-06-21 20:35 [PATCH 0/9] docs: kdoc: rework the BODY* processing states Jonathan Corbet
2025-06-21 20:35 ` [PATCH 1/9] docs: kdoc: Make body_with_blank_line parsing more flexible Jonathan Corbet
2025-06-22 11:46   ` Mauro Carvalho Chehab
2025-06-21 20:35 ` [PATCH 2/9] docs: kdoc: consolidate the "begin section" logic Jonathan Corbet
2025-06-22 11:47   ` Mauro Carvalho Chehab
2025-06-21 20:35 ` [PATCH 3/9] docs: kdoc: separate out the handling of the declaration phase Jonathan Corbet
2025-06-22 11:48   ` Mauro Carvalho Chehab
2025-06-21 20:35 ` [PATCH 4/9] docs: kdoc: split out the special-section state Jonathan Corbet
2025-06-22 11:50   ` Mauro Carvalho Chehab
2025-06-21 20:35 ` [PATCH 5/9] docs: kdoc: coalesce the new-section handling Jonathan Corbet
2025-06-22 11:50   ` Mauro Carvalho Chehab
2025-06-21 20:35 ` [PATCH 6/9] docs: kdoc: rework the handling of SPECIAL_SECTION Jonathan Corbet
2025-06-22 11:51   ` Mauro Carvalho Chehab
2025-06-21 20:35 ` [PATCH 7/9] docs: kdoc: coalesce the end-of-comment processing Jonathan Corbet
2025-06-22 11:52   ` Mauro Carvalho Chehab
2025-06-21 20:35 ` [PATCH 8/9] docs: kdoc: Add some comments to process_decl() Jonathan Corbet
2025-06-22 11:53   ` Mauro Carvalho Chehab
2025-06-21 20:35 ` [PATCH 9/9] docs: kdoc: finish disentangling the BODY and SPECIAL_SECTION states Jonathan Corbet
2025-06-22 11:54   ` Mauro Carvalho Chehab

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).