* [PULL v2 12/17] qapi: Rewrite parsing of doc comment section symbols and tags
2023-05-10 8:12 [PULL v2 00/17] QAPI patches patches for 2023-05-09 Markus Armbruster
@ 2023-05-10 8:12 ` Markus Armbruster
2023-05-16 12:07 ` Igor Mammedov
2023-05-10 8:12 ` [PULL v2 13/17] qapi: Relax doc string @name: description indentation rules Markus Armbruster
` (3 subsequent siblings)
4 siblings, 1 reply; 11+ messages in thread
From: Markus Armbruster @ 2023-05-10 8:12 UTC (permalink / raw)
To: qemu-devel; +Cc: richard.henderson, Juan Quintela
To recognize a line starting with a section symbol and or tag, we
first split it at the first space, then examine the part left of the
space. We can just as well examine the unsplit line, so do that.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20230428105429.1687850-13-armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
[Work around lack of walrus operator in Python 3.7 and older]
---
scripts/qapi/parser.py | 55 +++++++++++++++++++++---------------------
1 file changed, 27 insertions(+), 28 deletions(-)
diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index ddc14ceaba..a4ff9b6dbf 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -560,12 +560,12 @@ def end_comment(self) -> None:
self._switch_section(QAPIDoc.NullSection(self._parser))
@staticmethod
- def _is_section_tag(name: str) -> bool:
- return name in ('Returns:', 'Since:',
- # those are often singular or plural
- 'Note:', 'Notes:',
- 'Example:', 'Examples:',
- 'TODO:')
+ def _match_at_name_colon(string: str) -> re.Match:
+ return re.match(r'@([^:]*): *', string)
+
+ @staticmethod
+ def _match_section_tag(string: str) -> re.Match:
+ return re.match(r'(Returns|Since|Notes?|Examples?|TODO): *', string)
def _append_body_line(self, line: str) -> None:
"""
@@ -581,7 +581,6 @@ def _append_body_line(self, line: str) -> None:
Else, append the line to the current section.
"""
- name = line.split(' ', 1)[0]
# FIXME not nice: things like '# @foo:' and '# @foo: ' aren't
# recognized, and get silently treated as ordinary text
if not self.symbol and not self.body.text and line.startswith('@'):
@@ -595,12 +594,12 @@ def _append_body_line(self, line: str) -> None:
self._parser, "name required after '@'")
elif self.symbol:
# This is a definition documentation block
- if name.startswith('@') and name.endswith(':'):
+ if self._match_at_name_colon(line):
self._append_line = self._append_args_line
self._append_args_line(line)
elif line == 'Features:':
self._append_line = self._append_features_line
- elif self._is_section_tag(name):
+ elif self._match_section_tag(line):
self._append_line = self._append_various_line
self._append_various_line(line)
else:
@@ -621,16 +620,16 @@ def _append_args_line(self, line: str) -> None:
Else, append the line to the current section.
"""
- name = line.split(' ', 1)[0]
-
- if name.startswith('@') and name.endswith(':'):
+ match = self._match_at_name_colon(line)
+ if match:
# If line is "@arg: first line of description", find
# the index of 'f', which is the indent we expect for any
# following lines. We then remove the leading "@arg:"
# from line and replace it with spaces so that 'f' has the
# same index as it did in the original line and can be
# handled the same way we will handle following lines.
- indent = must_match(r'@\S*:\s*', line).end()
+ name = match.group(1)
+ indent = match.end()
line = line[indent:]
if not line:
# Line was just the "@arg:" header
@@ -638,8 +637,8 @@ def _append_args_line(self, line: str) -> None:
indent = -1
else:
line = ' ' * indent + line
- self._start_args_section(name[1:-1], indent)
- elif self._is_section_tag(name):
+ self._start_args_section(name, indent)
+ elif self._match_section_tag(line):
self._append_line = self._append_various_line
self._append_various_line(line)
return
@@ -656,16 +655,16 @@ def _append_args_line(self, line: str) -> None:
self._append_freeform(line)
def _append_features_line(self, line: str) -> None:
- name = line.split(' ', 1)[0]
-
- if name.startswith('@') and name.endswith(':'):
+ match = self._match_at_name_colon(line)
+ if match:
# If line is "@arg: first line of description", find
# the index of 'f', which is the indent we expect for any
# following lines. We then remove the leading "@arg:"
# from line and replace it with spaces so that 'f' has the
# same index as it did in the original line and can be
# handled the same way we will handle following lines.
- indent = must_match(r'@\S*:\s*', line).end()
+ name = match.group(1)
+ indent = match.end()
line = line[indent:]
if not line:
# Line was just the "@arg:" header
@@ -673,8 +672,8 @@ def _append_features_line(self, line: str) -> None:
indent = -1
else:
line = ' ' * indent + line
- self._start_features_section(name[1:-1], indent)
- elif self._is_section_tag(name):
+ self._start_features_section(name, indent)
+ elif self._match_section_tag(line):
self._append_line = self._append_various_line
self._append_various_line(line)
return
@@ -698,13 +697,13 @@ def _append_various_line(self, line: str) -> None:
Else, append the line to the current section.
"""
- name = line.split(' ', 1)[0]
-
- if name.startswith('@') and name.endswith(':'):
+ match = self._match_at_name_colon(line)
+ if match:
raise QAPIParseError(self._parser,
- "'%s' can't follow '%s' section"
- % (name, self.sections[0].name))
- if self._is_section_tag(name):
+ "'@%s:' can't follow '%s' section"
+ % (match.group(1), self.sections[0].name))
+ match = self._match_section_tag(line)
+ if match:
# If line is "Section: first line of description", find
# the index of 'f', which is the indent we expect for any
# following lines. We then remove the leading "Section:"
@@ -719,7 +718,7 @@ def _append_various_line(self, line: str) -> None:
indent = 0
else:
line = ' ' * indent + line
- self._start_section(name[:-1], indent)
+ self._start_section(match.group(1), indent)
self._append_freeform(line)
--
2.39.2
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [PULL v2 12/17] qapi: Rewrite parsing of doc comment section symbols and tags
2023-05-10 8:12 ` [PULL v2 12/17] qapi: Rewrite parsing of doc comment section symbols and tags Markus Armbruster
@ 2023-05-16 12:07 ` Igor Mammedov
2023-05-16 13:59 ` Markus Armbruster
0 siblings, 1 reply; 11+ messages in thread
From: Igor Mammedov @ 2023-05-16 12:07 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, richard.henderson, Juan Quintela
On Wed, 10 May 2023 10:12:22 +0200
Markus Armbruster <armbru@redhat.com> wrote:
> To recognize a line starting with a section symbol and or tag, we
> first split it at the first space, then examine the part left of the
> space. We can just as well examine the unsplit line, so do that.
this makes build fail on RHEL8.9 (Python 3.6.8)
with:
configure --target-list=x86_64-softmmu --disable-docs
make -j32
...
/usr/bin/python3 /builds/qemu/scripts/qapi-gen.py -o qapi -b ../../builds/qemu/qapi/qapi-schema.json
Traceback (most recent call last):
File "/builds/qemu/scripts/qapi-gen.py", line 16, in <module>
from qapi import main
File "/builds/qemu/scripts/qapi/main.py", line 14, in <module>
from .commands import gen_commands
File "/builds/qemu/scripts/qapi/commands.py", line 24, in <module>
from .gen import (
File "/builds/qemu/scripts/qapi/gen.py", line 32, in <module>
from .schema import (
File "/builds/qemu/scripts/qapi/schema.py", line 31, in <module>
from .expr import check_exprs
File "/builds/qemu/scripts/qapi/expr.py", line 46, in <module>
from .parser import QAPIExpression
File "/builds/qemu/scripts/qapi/parser.py", line 449, in <module>
class QAPIDoc:
File "/builds/qemu/scripts/qapi/parser.py", line 563, in QAPIDoc
def _match_at_name_colon(string: str) -> re.Match:
AttributeError: module 're' has no attribute 'Match'
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> Message-Id: <20230428105429.1687850-13-armbru@redhat.com>
> Reviewed-by: Juan Quintela <quintela@redhat.com>
> [Work around lack of walrus operator in Python 3.7 and older]
> ---
> scripts/qapi/parser.py | 55 +++++++++++++++++++++---------------------
> 1 file changed, 27 insertions(+), 28 deletions(-)
>
> diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
> index ddc14ceaba..a4ff9b6dbf 100644
> --- a/scripts/qapi/parser.py
> +++ b/scripts/qapi/parser.py
> @@ -560,12 +560,12 @@ def end_comment(self) -> None:
> self._switch_section(QAPIDoc.NullSection(self._parser))
>
> @staticmethod
> - def _is_section_tag(name: str) -> bool:
> - return name in ('Returns:', 'Since:',
> - # those are often singular or plural
> - 'Note:', 'Notes:',
> - 'Example:', 'Examples:',
> - 'TODO:')
> + def _match_at_name_colon(string: str) -> re.Match:
> + return re.match(r'@([^:]*): *', string)
> +
> + @staticmethod
> + def _match_section_tag(string: str) -> re.Match:
> + return re.match(r'(Returns|Since|Notes?|Examples?|TODO): *', string)
>
> def _append_body_line(self, line: str) -> None:
> """
> @@ -581,7 +581,6 @@ def _append_body_line(self, line: str) -> None:
>
> Else, append the line to the current section.
> """
> - name = line.split(' ', 1)[0]
> # FIXME not nice: things like '# @foo:' and '# @foo: ' aren't
> # recognized, and get silently treated as ordinary text
> if not self.symbol and not self.body.text and line.startswith('@'):
> @@ -595,12 +594,12 @@ def _append_body_line(self, line: str) -> None:
> self._parser, "name required after '@'")
> elif self.symbol:
> # This is a definition documentation block
> - if name.startswith('@') and name.endswith(':'):
> + if self._match_at_name_colon(line):
> self._append_line = self._append_args_line
> self._append_args_line(line)
> elif line == 'Features:':
> self._append_line = self._append_features_line
> - elif self._is_section_tag(name):
> + elif self._match_section_tag(line):
> self._append_line = self._append_various_line
> self._append_various_line(line)
> else:
> @@ -621,16 +620,16 @@ def _append_args_line(self, line: str) -> None:
> Else, append the line to the current section.
>
> """
> - name = line.split(' ', 1)[0]
> -
> - if name.startswith('@') and name.endswith(':'):
> + match = self._match_at_name_colon(line)
> + if match:
> # If line is "@arg: first line of description", find
> # the index of 'f', which is the indent we expect for any
> # following lines. We then remove the leading "@arg:"
> # from line and replace it with spaces so that 'f' has the
> # same index as it did in the original line and can be
> # handled the same way we will handle following lines.
> - indent = must_match(r'@\S*:\s*', line).end()
> + name = match.group(1)
> + indent = match.end()
> line = line[indent:]
> if not line:
> # Line was just the "@arg:" header
> @@ -638,8 +637,8 @@ def _append_args_line(self, line: str) -> None:
> indent = -1
> else:
> line = ' ' * indent + line
> - self._start_args_section(name[1:-1], indent)
> - elif self._is_section_tag(name):
> + self._start_args_section(name, indent)
> + elif self._match_section_tag(line):
> self._append_line = self._append_various_line
> self._append_various_line(line)
> return
> @@ -656,16 +655,16 @@ def _append_args_line(self, line: str) -> None:
> self._append_freeform(line)
>
> def _append_features_line(self, line: str) -> None:
> - name = line.split(' ', 1)[0]
> -
> - if name.startswith('@') and name.endswith(':'):
> + match = self._match_at_name_colon(line)
> + if match:
> # If line is "@arg: first line of description", find
> # the index of 'f', which is the indent we expect for any
> # following lines. We then remove the leading "@arg:"
> # from line and replace it with spaces so that 'f' has the
> # same index as it did in the original line and can be
> # handled the same way we will handle following lines.
> - indent = must_match(r'@\S*:\s*', line).end()
> + name = match.group(1)
> + indent = match.end()
> line = line[indent:]
> if not line:
> # Line was just the "@arg:" header
> @@ -673,8 +672,8 @@ def _append_features_line(self, line: str) -> None:
> indent = -1
> else:
> line = ' ' * indent + line
> - self._start_features_section(name[1:-1], indent)
> - elif self._is_section_tag(name):
> + self._start_features_section(name, indent)
> + elif self._match_section_tag(line):
> self._append_line = self._append_various_line
> self._append_various_line(line)
> return
> @@ -698,13 +697,13 @@ def _append_various_line(self, line: str) -> None:
>
> Else, append the line to the current section.
> """
> - name = line.split(' ', 1)[0]
> -
> - if name.startswith('@') and name.endswith(':'):
> + match = self._match_at_name_colon(line)
> + if match:
> raise QAPIParseError(self._parser,
> - "'%s' can't follow '%s' section"
> - % (name, self.sections[0].name))
> - if self._is_section_tag(name):
> + "'@%s:' can't follow '%s' section"
> + % (match.group(1), self.sections[0].name))
> + match = self._match_section_tag(line)
> + if match:
> # If line is "Section: first line of description", find
> # the index of 'f', which is the indent we expect for any
> # following lines. We then remove the leading "Section:"
> @@ -719,7 +718,7 @@ def _append_various_line(self, line: str) -> None:
> indent = 0
> else:
> line = ' ' * indent + line
> - self._start_section(name[:-1], indent)
> + self._start_section(match.group(1), indent)
>
> self._append_freeform(line)
>
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [PULL v2 12/17] qapi: Rewrite parsing of doc comment section symbols and tags
2023-05-16 12:07 ` Igor Mammedov
@ 2023-05-16 13:59 ` Markus Armbruster
0 siblings, 0 replies; 11+ messages in thread
From: Markus Armbruster @ 2023-05-16 13:59 UTC (permalink / raw)
To: Igor Mammedov; +Cc: qemu-devel, richard.henderson, Juan Quintela
Igor Mammedov <imammedo@redhat.com> writes:
> On Wed, 10 May 2023 10:12:22 +0200
> Markus Armbruster <armbru@redhat.com> wrote:
>
>> To recognize a line starting with a section symbol and or tag, we
>> first split it at the first space, then examine the part left of the
>> space. We can just as well examine the unsplit line, so do that.
>
> this makes build fail on RHEL8.9 (Python 3.6.8)
> with:
> configure --target-list=x86_64-softmmu --disable-docs
> make -j32
> ...
> /usr/bin/python3 /builds/qemu/scripts/qapi-gen.py -o qapi -b ../../builds/qemu/qapi/qapi-schema.json
[...]
> File "/builds/qemu/scripts/qapi/parser.py", line 563, in QAPIDoc
> def _match_at_name_colon(string: str) -> re.Match:
> AttributeError: module 're' has no attribute 'Match'
Working on it. First attempt "[PATCH] qapi/parser: Fix type hints"
failed.
Thanks!
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PULL v2 13/17] qapi: Relax doc string @name: description indentation rules
2023-05-10 8:12 [PULL v2 00/17] QAPI patches patches for 2023-05-09 Markus Armbruster
2023-05-10 8:12 ` [PULL v2 12/17] qapi: Rewrite parsing of doc comment section symbols and tags Markus Armbruster
@ 2023-05-10 8:12 ` Markus Armbruster
2023-05-10 8:12 ` [PULL v2 14/17] qapi: Section parameter @indent is no longer used, drop Markus Armbruster
` (2 subsequent siblings)
4 siblings, 0 replies; 11+ messages in thread
From: Markus Armbruster @ 2023-05-10 8:12 UTC (permalink / raw)
To: qemu-devel; +Cc: richard.henderson, Juan Quintela
The QAPI schema doc comment language provides special syntax for
command and event arguments, struct and union members, alternate
branches, enumeration values, and features: descriptions starting with
"@name:".
By convention, we format them like this:
# @name: Lorem ipsum dolor sit amet, consectetur adipiscing elit,
# sed do eiusmod tempor incididunt ut labore et dolore
# magna aliqua.
Okay for names as short as "name", but we have much longer ones. Their
description gets squeezed against the right margin, like this:
# @dirty-sync-missed-zero-copy: Number of times dirty RAM synchronization could
# not avoid copying dirty pages. This is between
# 0 and @dirty-sync-count * @multifd-channels.
# (since 7.1)
The description text is effectively just 50 characters wide. Easy
enough to read, but can be cumbersome to write.
The awkward squeeze against the right margin makes people go beyond it,
which produces two undesirables: arguments about style, and descriptions
that are unnecessarily hard to read, like this one:
# @postcopy-vcpu-blocktime: list of the postcopy blocktime per vCPU. This is
# only present when the postcopy-blocktime migration capability
# is enabled. (Since 3.0)
We could instead format it like
# @postcopy-vcpu-blocktime:
# list of the postcopy blocktime per vCPU. This is only present
# when the postcopy-blocktime migration capability is
# enabled. (Since 3.0)
or, since the commit before previous, like
# @postcopy-vcpu-blocktime:
# list of the postcopy blocktime per vCPU. This is only present
# when the postcopy-blocktime migration capability is
# enabled. (Since 3.0)
However, I'd rather have
# @postcopy-vcpu-blocktime: list of the postcopy blocktime per vCPU.
# This is only present when the postcopy-blocktime migration
# capability is enabled. (Since 3.0)
because this is how rST field and option lists work.
To get this, we need to let the first non-blank line after the
"@name:" line determine expected indentation.
This fills up the indentation pitfall mentioned in
docs/devel/qapi-code-gen.rst. A related pitfall still exists. Update
the text to show it.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20230428105429.1687850-14-armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
[Work around lack of walrus operator in Python 3.7 and older]
---
docs/devel/qapi-code-gen.rst | 10 ++--
scripts/qapi/parser.py | 73 +++++++--------------------
tests/qapi-schema/doc-bad-indent.err | 2 +-
tests/qapi-schema/doc-bad-indent.json | 3 +-
tests/qapi-schema/doc-good.json | 3 +-
tests/qapi-schema/doc-good.out | 3 +-
6 files changed, 32 insertions(+), 62 deletions(-)
diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
index 6386b58881..875f893be2 100644
--- a/docs/devel/qapi-code-gen.rst
+++ b/docs/devel/qapi-code-gen.rst
@@ -1074,10 +1074,14 @@ Indentation matters. Bad example::
# @none: None (no memory side cache in this proximity domain,
# or cache associativity unknown)
+ # (since 5.0)
-The description is parsed as a definition list with term "None (no
-memory side cache in this proximity domain," and definition "or cache
-associativity unknown)".
+The last line's de-indent is wrong. The second and subsequent lines
+need to line up with each other, like this::
+
+ # @none: None (no memory side cache in this proximity domain,
+ # or cache associativity unknown)
+ # (since 5.0)
Section tags are case-sensitive and end with a colon. Good example::
diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index a4ff9b6dbf..285c9ff4c9 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -474,17 +474,21 @@ def __init__(self, parser: QAPISchemaParser,
self._parser = parser
# optional section name (argument/member or section name)
self.name = name
+ # section text without section name
self.text = ''
- # the expected indent level of the text of this section
- self._indent = indent
+ # indentation to strip (None means indeterminate)
+ self._indent = None if self.name else 0
def append(self, line: str) -> None:
- # Strip leading spaces corresponding to the expected indent level
- # Blank lines are always OK.
+ line = line.rstrip()
+
if line:
indent = must_match(r'\s*', line).end()
- if self._indent < 0:
- self._indent = indent
+ if self._indent is None:
+ # indeterminate indentation
+ if self.text != '':
+ # non-blank, non-first line determines indentation
+ self._indent = indent
elif indent < self._indent:
raise QAPIParseError(
self._parser,
@@ -492,7 +496,7 @@ def append(self, line: str) -> None:
self._indent)
line = line[self._indent:]
- self.text += line.rstrip() + '\n'
+ self.text += line + '\n'
class ArgSection(Section):
def __init__(self, parser: QAPISchemaParser,
@@ -622,22 +626,8 @@ def _append_args_line(self, line: str) -> None:
"""
match = self._match_at_name_colon(line)
if match:
- # If line is "@arg: first line of description", find
- # the index of 'f', which is the indent we expect for any
- # following lines. We then remove the leading "@arg:"
- # from line and replace it with spaces so that 'f' has the
- # same index as it did in the original line and can be
- # handled the same way we will handle following lines.
- name = match.group(1)
- indent = match.end()
- line = line[indent:]
- if not line:
- # Line was just the "@arg:" header
- # The next non-blank line determines expected indent
- indent = -1
- else:
- line = ' ' * indent + line
- self._start_args_section(name, indent)
+ line = line[match.end():]
+ self._start_args_section(match.group(1), 0)
elif self._match_section_tag(line):
self._append_line = self._append_various_line
self._append_various_line(line)
@@ -657,22 +647,8 @@ def _append_args_line(self, line: str) -> None:
def _append_features_line(self, line: str) -> None:
match = self._match_at_name_colon(line)
if match:
- # If line is "@arg: first line of description", find
- # the index of 'f', which is the indent we expect for any
- # following lines. We then remove the leading "@arg:"
- # from line and replace it with spaces so that 'f' has the
- # same index as it did in the original line and can be
- # handled the same way we will handle following lines.
- name = match.group(1)
- indent = match.end()
- line = line[indent:]
- if not line:
- # Line was just the "@arg:" header
- # The next non-blank line determines expected indent
- indent = -1
- else:
- line = ' ' * indent + line
- self._start_features_section(name, indent)
+ line = line[match.end():]
+ self._start_features_section(match.group(1), 0)
elif self._match_section_tag(line):
self._append_line = self._append_various_line
self._append_various_line(line)
@@ -704,21 +680,8 @@ def _append_various_line(self, line: str) -> None:
% (match.group(1), self.sections[0].name))
match = self._match_section_tag(line)
if match:
- # If line is "Section: first line of description", find
- # the index of 'f', which is the indent we expect for any
- # following lines. We then remove the leading "Section:"
- # from line and replace it with spaces so that 'f' has the
- # same index as it did in the original line and can be
- # handled the same way we will handle following lines.
- indent = must_match(r'\S*:\s*', line).end()
- line = line[indent:]
- if not line:
- # Line was just the "Section:" header
- # The next non-blank line determines expected indent
- indent = 0
- else:
- line = ' ' * indent + line
- self._start_section(match.group(1), indent)
+ line = line[match.end():]
+ self._start_section(match.group(1), 0)
self._append_freeform(line)
@@ -754,7 +717,7 @@ def _start_section(self, name: Optional[str] = None,
self.sections.append(new_section)
def _switch_section(self, new_section: 'QAPIDoc.Section') -> None:
- text = self._section.text = self._section.text.strip()
+ text = self._section.text = self._section.text.strip('\n')
# Only the 'body' section is allowed to have an empty body.
# All other sections, including anonymous ones, must have text.
diff --git a/tests/qapi-schema/doc-bad-indent.err b/tests/qapi-schema/doc-bad-indent.err
index 67844539bd..3c9699a8e0 100644
--- a/tests/qapi-schema/doc-bad-indent.err
+++ b/tests/qapi-schema/doc-bad-indent.err
@@ -1 +1 @@
-doc-bad-indent.json:6:1: unexpected de-indent (expected at least 4 spaces)
+doc-bad-indent.json:7:1: unexpected de-indent (expected at least 2 spaces)
diff --git a/tests/qapi-schema/doc-bad-indent.json b/tests/qapi-schema/doc-bad-indent.json
index edde8f21dc..3f22a27717 100644
--- a/tests/qapi-schema/doc-bad-indent.json
+++ b/tests/qapi-schema/doc-bad-indent.json
@@ -3,6 +3,7 @@
##
# @foo:
# @a: line one
-# line two is wrongly indented
+# line two
+# line three is wrongly indented
##
{ 'command': 'foo', 'data': { 'a': 'int' } }
diff --git a/tests/qapi-schema/doc-good.json b/tests/qapi-schema/doc-good.json
index 34c3dcbe97..354dfdf461 100644
--- a/tests/qapi-schema/doc-good.json
+++ b/tests/qapi-schema/doc-good.json
@@ -144,7 +144,8 @@
# description starts on a new line,
# indented
#
-# @arg2: the second argument
+# @arg2: description starts on the same line
+# remainder indented differently
#
# Features:
# @cmd-feat1: a feature
diff --git a/tests/qapi-schema/doc-good.out b/tests/qapi-schema/doc-good.out
index 277371acc8..24d9ea954d 100644
--- a/tests/qapi-schema/doc-good.out
+++ b/tests/qapi-schema/doc-good.out
@@ -161,7 +161,8 @@ doc symbol=cmd
description starts on a new line,
indented
arg=arg2
-the second argument
+description starts on the same line
+remainder indented differently
arg=arg3
feature=cmd-feat1
--
2.39.2
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PULL v2 14/17] qapi: Section parameter @indent is no longer used, drop
2023-05-10 8:12 [PULL v2 00/17] QAPI patches patches for 2023-05-09 Markus Armbruster
2023-05-10 8:12 ` [PULL v2 12/17] qapi: Rewrite parsing of doc comment section symbols and tags Markus Armbruster
2023-05-10 8:12 ` [PULL v2 13/17] qapi: Relax doc string @name: description indentation rules Markus Armbruster
@ 2023-05-10 8:12 ` Markus Armbruster
2023-05-10 13:51 ` [PULL v2 00/17] QAPI patches patches for 2023-05-09 Richard Henderson
2023-05-10 14:59 ` Richard Henderson
4 siblings, 0 replies; 11+ messages in thread
From: Markus Armbruster @ 2023-05-10 8:12 UTC (permalink / raw)
To: qemu-devel; +Cc: richard.henderson, Juan Quintela
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20230428105429.1687850-15-armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
---
scripts/qapi/parser.py | 31 ++++++++++++++-----------------
1 file changed, 14 insertions(+), 17 deletions(-)
diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index 285c9ff4c9..4923a59d60 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -468,8 +468,7 @@ class QAPIDoc:
class Section:
# pylint: disable=too-few-public-methods
def __init__(self, parser: QAPISchemaParser,
- name: Optional[str] = None, indent: int = 0):
-
+ name: Optional[str] = None):
# parser, for error messages about indentation
self._parser = parser
# optional section name (argument/member or section name)
@@ -500,8 +499,8 @@ def append(self, line: str) -> None:
class ArgSection(Section):
def __init__(self, parser: QAPISchemaParser,
- name: str, indent: int = 0):
- super().__init__(parser, name, indent)
+ name: str):
+ super().__init__(parser, name)
self.member: Optional['QAPISchemaMember'] = None
def connect(self, member: 'QAPISchemaMember') -> None:
@@ -627,7 +626,7 @@ def _append_args_line(self, line: str) -> None:
match = self._match_at_name_colon(line)
if match:
line = line[match.end():]
- self._start_args_section(match.group(1), 0)
+ self._start_args_section(match.group(1))
elif self._match_section_tag(line):
self._append_line = self._append_various_line
self._append_various_line(line)
@@ -648,7 +647,7 @@ def _append_features_line(self, line: str) -> None:
match = self._match_at_name_colon(line)
if match:
line = line[match.end():]
- self._start_features_section(match.group(1), 0)
+ self._start_features_section(match.group(1))
elif self._match_section_tag(line):
self._append_line = self._append_various_line
self._append_various_line(line)
@@ -681,15 +680,14 @@ def _append_various_line(self, line: str) -> None:
match = self._match_section_tag(line)
if match:
line = line[match.end():]
- self._start_section(match.group(1), 0)
+ self._start_section(match.group(1))
self._append_freeform(line)
def _start_symbol_section(
self,
symbols_dict: Dict[str, 'QAPIDoc.ArgSection'],
- name: str,
- indent: int) -> None:
+ name: str) -> None:
# FIXME invalid names other than the empty string aren't flagged
if not name:
raise QAPIParseError(self._parser, "invalid parameter name")
@@ -697,22 +695,21 @@ def _start_symbol_section(
raise QAPIParseError(self._parser,
"'%s' parameter name duplicated" % name)
assert not self.sections
- new_section = QAPIDoc.ArgSection(self._parser, name, indent)
+ new_section = QAPIDoc.ArgSection(self._parser, name)
self._switch_section(new_section)
symbols_dict[name] = new_section
- def _start_args_section(self, name: str, indent: int) -> None:
- self._start_symbol_section(self.args, name, indent)
+ def _start_args_section(self, name: str) -> None:
+ self._start_symbol_section(self.args, name)
- def _start_features_section(self, name: str, indent: int) -> None:
- self._start_symbol_section(self.features, name, indent)
+ def _start_features_section(self, name: str) -> None:
+ self._start_symbol_section(self.features, name)
- def _start_section(self, name: Optional[str] = None,
- indent: int = 0) -> None:
+ def _start_section(self, name: Optional[str] = None) -> None:
if name in ('Returns', 'Since') and self.has_section(name):
raise QAPIParseError(self._parser,
"duplicated '%s' section" % name)
- new_section = QAPIDoc.Section(self._parser, name, indent)
+ new_section = QAPIDoc.Section(self._parser, name)
self._switch_section(new_section)
self.sections.append(new_section)
--
2.39.2
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [PULL v2 00/17] QAPI patches patches for 2023-05-09
2023-05-10 8:12 [PULL v2 00/17] QAPI patches patches for 2023-05-09 Markus Armbruster
` (2 preceding siblings ...)
2023-05-10 8:12 ` [PULL v2 14/17] qapi: Section parameter @indent is no longer used, drop Markus Armbruster
@ 2023-05-10 13:51 ` Richard Henderson
2023-05-10 14:59 ` Richard Henderson
4 siblings, 0 replies; 11+ messages in thread
From: Richard Henderson @ 2023-05-10 13:51 UTC (permalink / raw)
To: Markus Armbruster, qemu-devel
On 5/10/23 09:12, Markus Armbruster wrote:
> The following changes since commit 792f77f376adef944f9a03e601f6ad90c2f891b2:
>
> Merge tag 'pull-loongarch-20230506' ofhttps://gitlab.com/gaosong/qemu into staging (2023-05-06 08:11:52 +0100)
>
> are available in the Git repository at:
>
> https://repo.or.cz/qemu/armbru.git tags/pull-qapi-2023-05-09-v2
>
> for you to fetch changes up to a937b6aa739f65f2cae2ad9a7eb65a309ad2a359:
>
> qapi: Reformat doc comments to conform to current conventions (2023-05-10 10:01:01 +0200)
>
> ----------------------------------------------------------------
> QAPI patches patches for 2023-05-09
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
r~
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [PULL v2 00/17] QAPI patches patches for 2023-05-09
2023-05-10 8:12 [PULL v2 00/17] QAPI patches patches for 2023-05-09 Markus Armbruster
` (3 preceding siblings ...)
2023-05-10 13:51 ` [PULL v2 00/17] QAPI patches patches for 2023-05-09 Richard Henderson
@ 2023-05-10 14:59 ` Richard Henderson
2023-05-10 15:22 ` Markus Armbruster
4 siblings, 1 reply; 11+ messages in thread
From: Richard Henderson @ 2023-05-10 14:59 UTC (permalink / raw)
To: Markus Armbruster, qemu-devel
On 5/10/23 09:12, Markus Armbruster wrote:
> The following changes since commit 792f77f376adef944f9a03e601f6ad90c2f891b2:
>
> Merge tag 'pull-loongarch-20230506' of https://gitlab.com/gaosong/qemu into staging (2023-05-06 08:11:52 +0100)
>
> are available in the Git repository at:
>
> https://repo.or.cz/qemu/armbru.git tags/pull-qapi-2023-05-09-v2
>
> for you to fetch changes up to a937b6aa739f65f2cae2ad9a7eb65a309ad2a359:
>
> qapi: Reformat doc comments to conform to current conventions (2023-05-10 10:01:01 +0200)
>
> ----------------------------------------------------------------
> QAPI patches patches for 2023-05-09
>
> ----------------------------------------------------------------
> Markus Armbruster (17):
> docs/devel/qapi-code-gen: Clean up use of quotes a bit
> docs/devel/qapi-code-gen: Turn FIXME admonitions into comments
> qapi: Fix crash on stray double quote character
> meson: Fix to make QAPI generator output depend on main.py
> Revert "qapi: BlockExportRemoveMode: move comments to TODO"
> sphinx/qapidoc: Do not emit TODO sections into user manuals
> qapi: Tidy up a slightly awkward TODO comment
> qapi/dump: Indent bulleted lists consistently
> tests/qapi-schema/doc-good: Improve a comment
> tests/qapi-schema/doc-good: Improve argument description tests
> qapi: Fix argument description indentation stripping
> qapi: Rewrite parsing of doc comment section symbols and tags
> qapi: Relax doc string @name: description indentation rules
> qapi: Section parameter @indent is no longer used, drop
> docs/devel/qapi-code-gen: Update doc comment conventions
> qga/qapi-schema: Reformat doc comments to conform to current conventions
> qapi: Reformat doc comments to conform to current conventions
I didn't notice earlier, because centos-stream-8-x86_64 failure is optional,
but this has another error:
https://gitlab.com/qemu-project/qemu/-/jobs/4258751398#L4649
Exception occurred:
File
"/home/gitlab-runner/builds/Jpwtyaz7/0/qemu-project/qemu/docs/../scripts/qapi/parser.py",
line 566, in QAPIDoc
def _match_at_name_colon(string: str) -> re.Match:
AttributeError: module 're' has no attribute 'Match'
r~
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [PULL v2 00/17] QAPI patches patches for 2023-05-09
2023-05-10 14:59 ` Richard Henderson
@ 2023-05-10 15:22 ` Markus Armbruster
2023-05-11 11:11 ` Markus Armbruster
0 siblings, 1 reply; 11+ messages in thread
From: Markus Armbruster @ 2023-05-10 15:22 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel
Richard Henderson <richard.henderson@linaro.org> writes:
> On 5/10/23 09:12, Markus Armbruster wrote:
>> The following changes since commit 792f77f376adef944f9a03e601f6ad90c2f891b2:
>> Merge tag 'pull-loongarch-20230506' of https://gitlab.com/gaosong/qemu into staging (2023-05-06 08:11:52 +0100)
>> are available in the Git repository at:
>> https://repo.or.cz/qemu/armbru.git tags/pull-qapi-2023-05-09-v2
>> for you to fetch changes up to a937b6aa739f65f2cae2ad9a7eb65a309ad2a359:
>> qapi: Reformat doc comments to conform to current conventions (2023-05-10 10:01:01 +0200)
>> ----------------------------------------------------------------
>> QAPI patches patches for 2023-05-09
>> ----------------------------------------------------------------
>> Markus Armbruster (17):
>> docs/devel/qapi-code-gen: Clean up use of quotes a bit
>> docs/devel/qapi-code-gen: Turn FIXME admonitions into comments
>> qapi: Fix crash on stray double quote character
>> meson: Fix to make QAPI generator output depend on main.py
>> Revert "qapi: BlockExportRemoveMode: move comments to TODO"
>> sphinx/qapidoc: Do not emit TODO sections into user manuals
>> qapi: Tidy up a slightly awkward TODO comment
>> qapi/dump: Indent bulleted lists consistently
>> tests/qapi-schema/doc-good: Improve a comment
>> tests/qapi-schema/doc-good: Improve argument description tests
>> qapi: Fix argument description indentation stripping
>> qapi: Rewrite parsing of doc comment section symbols and tags
>> qapi: Relax doc string @name: description indentation rules
>> qapi: Section parameter @indent is no longer used, drop
>> docs/devel/qapi-code-gen: Update doc comment conventions
>> qga/qapi-schema: Reformat doc comments to conform to current conventions
>> qapi: Reformat doc comments to conform to current conventions
>
> I didn't notice earlier, because centos-stream-8-x86_64 failure is optional,
> but this has another error:
>
> https://gitlab.com/qemu-project/qemu/-/jobs/4258751398#L4649
>
> Exception occurred:
> File "/home/gitlab-runner/builds/Jpwtyaz7/0/qemu-project/qemu/docs/../scripts/qapi/parser.py", line 566, in QAPIDoc
> def _match_at_name_colon(string: str) -> re.Match:
> AttributeError: module 're' has no attribute 'Match'
I'll take care of it. Thanks!
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PULL v2 00/17] QAPI patches patches for 2023-05-09
2023-05-10 15:22 ` Markus Armbruster
@ 2023-05-11 11:11 ` Markus Armbruster
2023-05-11 11:17 ` Richard Henderson
0 siblings, 1 reply; 11+ messages in thread
From: Markus Armbruster @ 2023-05-11 11:11 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel
Markus Armbruster <armbru@redhat.com> writes:
> Richard Henderson <richard.henderson@linaro.org> writes:
>
>> I didn't notice earlier, because centos-stream-8-x86_64 failure is optional,
>> but this has another error:
>>
>> https://gitlab.com/qemu-project/qemu/-/jobs/4258751398#L4649
>>
>> Exception occurred:
>> File "/home/gitlab-runner/builds/Jpwtyaz7/0/qemu-project/qemu/docs/../scripts/qapi/parser.py", line 566, in QAPIDoc
>> def _match_at_name_colon(string: str) -> re.Match:
>> AttributeError: module 're' has no attribute 'Match'
>
> I'll take care of it. Thanks!
I tried to reproduce locally with make target docker-test-build@centos8,
no dice.
I'll post a patch to make mypy happy again. Perhaps it'll unbreak CI,
too.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PULL v2 00/17] QAPI patches patches for 2023-05-09
2023-05-11 11:11 ` Markus Armbruster
@ 2023-05-11 11:17 ` Richard Henderson
0 siblings, 0 replies; 11+ messages in thread
From: Richard Henderson @ 2023-05-11 11:17 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Alex Bennée
On 5/11/23 12:11, Markus Armbruster wrote:
> Markus Armbruster <armbru@redhat.com> writes:
>
>> Richard Henderson <richard.henderson@linaro.org> writes:
>>
>>> I didn't notice earlier, because centos-stream-8-x86_64 failure is optional,
>>> but this has another error:
>>>
>>> https://gitlab.com/qemu-project/qemu/-/jobs/4258751398#L4649
>>>
>>> Exception occurred:
>>> File "/home/gitlab-runner/builds/Jpwtyaz7/0/qemu-project/qemu/docs/../scripts/qapi/parser.py", line 566, in QAPIDoc
>>> def _match_at_name_colon(string: str) -> re.Match:
>>> AttributeError: module 're' has no attribute 'Match'
>>
>> I'll take care of it. Thanks!
>
> I tried to reproduce locally with make target docker-test-build@centos8,
> no dice.
Yeah, we don't seem to have a docker image that matches -- it is set up by
scripts/ci/org.centos/stream/8/x86_64/configure
for some reason. Perhaps "make vm-build-centos" was intended to be the way, but that
appears to be broken too.
Alex?
r~
^ permalink raw reply [flat|nested] 11+ messages in thread