Re: [PATCH v9 12/13] docs: parser_yaml.py: add support for line numbers from the parser

[Mauro Carvalho Chehab <mchehab+huawei@kernel.org>]

Em Thu, 10 Jul 2025 15:25:20 +0100
Donald Hunter <donald.hunter@gmail.com> escreveu:

> Donald Hunter <donald.hunter@gmail.com> writes:
> 
> >>              # Parse message with RSTParser
> >> -            for i, line in enumerate(msg.split('\n')):
> >> -                result.append(line, document.current_source, i)
> >> +            lineoffset = 0;
> >> +            for line in msg.split('\n'):
> >> +                match = self.re_lineno.match(line)
> >> +                if match:
> >> +                    lineoffset = int(match.group(1))
> >> +                    continue
> >> +
> >> +                result.append(line, document.current_source, lineoffset)  
> >
> > I expect this would need to be source=document.current_source, offset=lineoffset  
> 
> Ignore that. I see it's not kwargs. It's just the issue below.
> 
> >>              rst_parser = RSTParser()
> >>              rst_parser.parse('\n'.join(result), document)  
> >
> > But anyway this discards any line information by just concatenating the
> > lines together again.  
> 
> Looks to me like there's no Parser() API that works with ViewList() so
> it would be necessary to directly use the docutils RSTStateMachine() for
> this approach to work.

It sounds so.

The enclosed patch seems to address it:

	$ make cleandocs; make SPHINXDIRS="netlink/specs" htmldocs
	...
	Using alabaster theme
	source directory: netlink/specs
	Using Python kernel-doc
	/new_devel/v4l/docs/Documentation/netlink/specs/rt-neigh.yaml:13: ERROR: Unknown directive type "bogus".

	.. bogus:: [docutils]

Please notice that I added a hunk there to generate the error, just
to make easier to test - I'll drop it at the final version, and add
the proper reported-by/closes/... tags once you test it.

Regards,
Mauro

[PATCH RFC] sphinx: parser_yaml.py: preserve line numbers

Instead of converting viewlist to text, use it directly, if
docutils supports it.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

diff --git a/Documentation/netlink/specs/rt-neigh.yaml b/Documentation/netlink/specs/rt-neigh.yaml
index e9cba164e3d1..937d2563f151 100644
--- a/Documentation/netlink/specs/rt-neigh.yaml
+++ b/Documentation/netlink/specs/rt-neigh.yaml
@@ -11,6 +11,7 @@ doc:
 definitions:
   -
     name: ndmsg
+    doc: ".. bogus::"
     type: struct
     members:
       -
diff --git a/Documentation/sphinx/parser_yaml.py b/Documentation/sphinx/parser_yaml.py
index 1602b31f448e..2a2faaf759ef 100755
--- a/Documentation/sphinx/parser_yaml.py
+++ b/Documentation/sphinx/parser_yaml.py
@@ -11,7 +11,9 @@ import sys
 
 from pprint import pformat
 
+from docutils import nodes, statemachine
 from docutils.parsers.rst import Parser as RSTParser
+from docutils.parsers.rst import states
 from docutils.statemachine import ViewList
 
 from sphinx.util import logging
@@ -66,10 +68,24 @@ class YamlParser(Parser):
 
         result = ViewList()
 
+        tab_width = 8
+
+        self.state_classes = states.state_classes
+        self.initial_state = 'Body'
+
+        self.statemachine = states.RSTStateMachine(
+              state_classes=self.state_classes,
+              initial_state=self.initial_state,
+              debug=document.reporter.debug_flag)
+
         try:
             # Parse message with RSTParser
             lineoffset = 0;
-            for line in msg.split('\n'):
+
+            lines = statemachine.string2lines(msg, tab_width,
+                                            convert_whitespace=True)
+
+            for line in lines:
                 match = self.re_lineno.match(line)
                 if match:
                     lineoffset = int(match.group(1))
@@ -77,12 +93,7 @@ class YamlParser(Parser):
 
                 result.append(line, document.current_source, lineoffset)
 
-            # Fix backward compatibility with docutils < 0.17.1
-            if "tab_width" not in vars(document.settings):
-                document.settings.tab_width = 8
-
-            rst_parser = RSTParser()
-            rst_parser.parse('\n'.join(result), document)
+            self.statemachine.run(result, document, inliner=None)
 
         except Exception as e:
             document.reporter.error("YAML parsing error: %s" % pformat(e))