* [PATCH 01/57] do-not-merge
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 02/57] qapi: shush pylint up John Snow
` (58 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Ad-hoc linting scripts to scrub down the new docs/sphinx files. Should
work with a reasonably modern mypy/pylint/etc, and Sphinx 8.2.0. Older
versions of Sphinx ought to still work at runtime, but may not type
check correctly.
Signed-off-by: John Snow <jsnow@redhat.com>
---
scripts/qapi-lint.sh | 55 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 55 insertions(+)
create mode 100755 scripts/qapi-lint.sh
diff --git a/scripts/qapi-lint.sh b/scripts/qapi-lint.sh
new file mode 100755
index 00000000000..0e3ff002b69
--- /dev/null
+++ b/scripts/qapi-lint.sh
@@ -0,0 +1,55 @@
+#!/usr/bin/env bash
+set -e
+
+if [[ -f qapi/.flake8 ]]; then
+ echo "flake8 --config=qapi/.flake8 qapi/"
+ flake8 --config=qapi/.flake8 qapi/
+fi
+if [[ -f qapi/pylintrc ]]; then
+ echo "pylint --rcfile=qapi/pylintrc qapi/"
+ pylint --rcfile=qapi/pylintrc qapi/
+fi
+if [[ -f qapi/mypy.ini ]]; then
+ echo "mypy --config-file=qapi/mypy.ini qapi/"
+ mypy --config-file=qapi/mypy.ini qapi/
+fi
+
+if [[ -f qapi/.isort.cfg ]]; then
+ pushd qapi
+ echo "isort -c ."
+ isort -c .
+ popd
+fi
+
+if [[ -f ../docs/sphinx/qapi_domain.py ]]; then
+ files="qapi_domain.py"
+fi
+if [[ -f ../docs/sphinx/compat.py ]]; then
+ files="${files} compat.py"
+fi
+if [[ -f ../docs/sphinx/collapse.py ]]; then
+ files="${files} collapse.py"
+fi
+
+if [[ -f ../docs/sphinx/qapi_domain.py ]]; then
+ pushd ../docs/sphinx
+
+ set -x
+ mypy --strict $files
+ flake8 --max-line-length=80 $files qapidoc.py
+ isort -c $files qapidoc.py
+ black --line-length 80 --check $files qapidoc.py
+ PYTHONPATH=../../scripts/ pylint \
+ --rc-file ../../scripts/qapi/pylintrc \
+ $files qapidoc.py
+ set +x
+
+ popd
+fi
+
+pushd ../build
+make -j13
+make check-qapi-schema
+make docs
+make sphinxdocs
+popd
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 02/57] qapi: shush pylint up
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
2025-03-05 3:45 ` [PATCH 01/57] do-not-merge John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 6:28 ` Markus Armbruster
2025-03-05 3:45 ` [PATCH 03/57] docs/sphinx: create QAPI domain extension stub John Snow
` (57 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Shhhhh!
This patch is RFC quality, I wasn't in the mood to actually solve
problems so much as I was in the mood to continue working on the Sphinx
rework. Plus, I don't think the code I am patching has hit origin/master
yet ...
Signed-off-by: John Snow <jsnow@redhat.com>
---
scripts/qapi/backend.py | 2 ++
scripts/qapi/main.py | 10 ++++------
2 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/scripts/qapi/backend.py b/scripts/qapi/backend.py
index 14e60aa67af..49ae6ecdd33 100644
--- a/scripts/qapi/backend.py
+++ b/scripts/qapi/backend.py
@@ -13,6 +13,7 @@
class QAPIBackend(ABC):
+ # pylint: disable=too-few-public-methods
@abstractmethod
def generate(self,
@@ -36,6 +37,7 @@ def generate(self,
class QAPICBackend(QAPIBackend):
+ # pylint: disable=too-few-public-methods
def generate(self,
schema: QAPISchema,
diff --git a/scripts/qapi/main.py b/scripts/qapi/main.py
index ff42b43cd68..01155373bd0 100644
--- a/scripts/qapi/main.py
+++ b/scripts/qapi/main.py
@@ -31,15 +31,14 @@ def create_backend(path: str) -> QAPIBackend:
module_path, dot, class_name = path.rpartition('.')
if not dot:
- print(f"argument of -B must be of the form MODULE.CLASS",
+ print("argument of -B must be of the form MODULE.CLASS",
file=sys.stderr)
sys.exit(1)
try:
mod = import_module(module_path)
except Exception as ex:
- print(f"unable to import '{module_path}': {ex}", file=sys.stderr)
- sys.exit(1)
+ raise QAPIError(f"unable to import '{module_path}': {ex}") from ex
try:
klass = getattr(mod, class_name)
@@ -51,9 +50,8 @@ def create_backend(path: str) -> QAPIBackend:
try:
backend = klass()
except Exception as ex:
- print(f"backend '{path}' cannot be instantiated: {ex}",
- file=sys.stderr)
- sys.exit(1)
+ raise QAPIError(
+ f"backend '{path}' cannot be instantiated: {ex}") from ex
if not isinstance(backend, QAPIBackend):
print(f"backend '{path}' must be an instance of QAPIBackend",
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 02/57] qapi: shush pylint up
2025-03-05 3:45 ` [PATCH 02/57] qapi: shush pylint up John Snow
@ 2025-03-05 6:28 ` Markus Armbruster
2025-03-05 15:31 ` John Snow
0 siblings, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-05 6:28 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> Shhhhh!
>
> This patch is RFC quality, I wasn't in the mood to actually solve
> problems so much as I was in the mood to continue working on the Sphinx
> rework. Plus, I don't think the code I am patching has hit origin/master
> yet ...
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> scripts/qapi/backend.py | 2 ++
> scripts/qapi/main.py | 10 ++++------
> 2 files changed, 6 insertions(+), 6 deletions(-)
>
> diff --git a/scripts/qapi/backend.py b/scripts/qapi/backend.py
> index 14e60aa67af..49ae6ecdd33 100644
> --- a/scripts/qapi/backend.py
> +++ b/scripts/qapi/backend.py
> @@ -13,6 +13,7 @@
>
>
> class QAPIBackend(ABC):
> + # pylint: disable=too-few-public-methods
>
> @abstractmethod
> def generate(self,
> @@ -36,6 +37,7 @@ def generate(self,
>
>
> class QAPICBackend(QAPIBackend):
> + # pylint: disable=too-few-public-methods
>
> def generate(self,
> schema: QAPISchema,
I didn't bother to ask for these in my review. Do they get us to the
point where we can enable automatic pylint?
> diff --git a/scripts/qapi/main.py b/scripts/qapi/main.py
> index ff42b43cd68..01155373bd0 100644
> --- a/scripts/qapi/main.py
> +++ b/scripts/qapi/main.py
> @@ -31,15 +31,14 @@ def create_backend(path: str) -> QAPIBackend:
>
> module_path, dot, class_name = path.rpartition('.')
> if not dot:
> - print(f"argument of -B must be of the form MODULE.CLASS",
> + print("argument of -B must be of the form MODULE.CLASS",
This one's already in "[PULL v2 0/5] QAPI patches patches for
2025-02-26". No worries.
> file=sys.stderr)
> sys.exit(1)
>
> try:
> mod = import_module(module_path)
> except Exception as ex:
> - print(f"unable to import '{module_path}': {ex}", file=sys.stderr)
> - sys.exit(1)
> + raise QAPIError(f"unable to import '{module_path}': {ex}") from ex
>
> try:
> klass = getattr(mod, class_name)
> @@ -51,9 +50,8 @@ def create_backend(path: str) -> QAPIBackend:
> try:
> backend = klass()
> except Exception as ex:
> - print(f"backend '{path}' cannot be instantiated: {ex}",
> - file=sys.stderr)
> - sys.exit(1)
> + raise QAPIError(
> + f"backend '{path}' cannot be instantiated: {ex}") from ex
>
> if not isinstance(backend, QAPIBackend):
> print(f"backend '{path}' must be an instance of QAPIBackend",
Missed in my review: the caller catches QAPIError, and returns non-zero
exit code on catch. The caller's caller passes the exit code to
sys.exit(). Leaving the sys.exit() to the callers is cleaner.
However, you convert only two out of five error paths from sys.exit() to
raise. All or nothing, please.
Maybe split the patch into a "# pylint:" and a "raise QAPIError" part?
^ permalink raw reply [flat|nested] 129+ messages in thread* Re: [PATCH 02/57] qapi: shush pylint up
2025-03-05 6:28 ` Markus Armbruster
@ 2025-03-05 15:31 ` John Snow
0 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 15:31 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 3418 bytes --]
On Wed, Mar 5, 2025, 1:28 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > Shhhhh!
> >
> > This patch is RFC quality, I wasn't in the mood to actually solve
> > problems so much as I was in the mood to continue working on the Sphinx
> > rework. Plus, I don't think the code I am patching has hit origin/master
> > yet ...
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
> > ---
> > scripts/qapi/backend.py | 2 ++
> > scripts/qapi/main.py | 10 ++++------
> > 2 files changed, 6 insertions(+), 6 deletions(-)
> >
> > diff --git a/scripts/qapi/backend.py b/scripts/qapi/backend.py
> > index 14e60aa67af..49ae6ecdd33 100644
> > --- a/scripts/qapi/backend.py
> > +++ b/scripts/qapi/backend.py
> > @@ -13,6 +13,7 @@
> >
> >
> > class QAPIBackend(ABC):
> > + # pylint: disable=too-few-public-methods
> >
> > @abstractmethod
> > def generate(self,
> > @@ -36,6 +37,7 @@ def generate(self,
> >
> >
> > class QAPICBackend(QAPIBackend):
> > + # pylint: disable=too-few-public-methods
> >
> > def generate(self,
> > schema: QAPISchema,
>
> I didn't bother to ask for these in my review. Do they get us to the
> point where we can enable automatic pylint?
>
Yes.
> > diff --git a/scripts/qapi/main.py b/scripts/qapi/main.py
> > index ff42b43cd68..01155373bd0 100644
> > --- a/scripts/qapi/main.py
> > +++ b/scripts/qapi/main.py
> > @@ -31,15 +31,14 @@ def create_backend(path: str) -> QAPIBackend:
> >
> > module_path, dot, class_name = path.rpartition('.')
> > if not dot:
> > - print(f"argument of -B must be of the form MODULE.CLASS",
> > + print("argument of -B must be of the form MODULE.CLASS",
>
> This one's already in "[PULL v2 0/5] QAPI patches patches for
> 2025-02-26". No worries.
>
Got it. I'll repull your tags.
> > file=sys.stderr)
> > sys.exit(1)
> >
> > try:
> > mod = import_module(module_path)
> > except Exception as ex:
> > - print(f"unable to import '{module_path}': {ex}",
> file=sys.stderr)
> > - sys.exit(1)
> > + raise QAPIError(f"unable to import '{module_path}': {ex}") from
> ex
> >
> > try:
> > klass = getattr(mod, class_name)
> > @@ -51,9 +50,8 @@ def create_backend(path: str) -> QAPIBackend:
> > try:
> > backend = klass()
> > except Exception as ex:
> > - print(f"backend '{path}' cannot be instantiated: {ex}",
> > - file=sys.stderr)
> > - sys.exit(1)
> > + raise QAPIError(
> > + f"backend '{path}' cannot be instantiated: {ex}") from ex
> >
> > if not isinstance(backend, QAPIBackend):
> > print(f"backend '{path}' must be an instance of QAPIBackend",
>
> Missed in my review: the caller catches QAPIError, and returns non-zero
> exit code on catch. The caller's caller passes the exit code to
> sys.exit(). Leaving the sys.exit() to the callers is cleaner.
>
> However, you convert only two out of five error paths from sys.exit() to
> raise. All or nothing, please.
>
"RFC Quality" ;)
You got it, I'll be consistent in approach here. My initial goal was only
to get the linters clean for this series.
> Maybe split the patch into a "# pylint:" and a "raise QAPIError" part?
>
Sure.
>
[-- Attachment #2: Type: text/html, Size: 5640 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 03/57] docs/sphinx: create QAPI domain extension stub
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
2025-03-05 3:45 ` [PATCH 01/57] do-not-merge John Snow
2025-03-05 3:45 ` [PATCH 02/57] qapi: shush pylint up John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 04/57] docs/sphinx: add compat.py module and nested_parse helper John Snow
` (56 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
A Sphinx domain is a collection of directive and role extensions meant
to facilitate the documentation of a specific language. For instance,
Sphinx ships with "python" and "cpp" domains. This patch introduces a
stub for the "qapi" language domain.
Please see https://www.sphinx-doc.org/en/master/usage/domains/index.html
for more information.
This stub doesn't really do anything yet, we'll get to it brick-by-brick
in the forthcoming commits to keep the series breezy and the git history
informative.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/conf.py | 9 +++++-
docs/sphinx/qapi_domain.py | 56 ++++++++++++++++++++++++++++++++++++++
2 files changed, 64 insertions(+), 1 deletion(-)
create mode 100644 docs/sphinx/qapi_domain.py
diff --git a/docs/conf.py b/docs/conf.py
index 31bb9a37893..49d9de894c0 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -60,7 +60,14 @@
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
-extensions = ['kerneldoc', 'qmp_lexer', 'hxtool', 'depfile', 'qapidoc']
+extensions = [
+ 'depfile',
+ 'hxtool',
+ 'kerneldoc',
+ 'qapi_domain',
+ 'qapidoc',
+ 'qmp_lexer',
+]
if sphinx.version_info[:3] > (4, 0, 0):
tags.add('sphinx4')
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
new file mode 100644
index 00000000000..a1983d94440
--- /dev/null
+++ b/docs/sphinx/qapi_domain.py
@@ -0,0 +1,56 @@
+"""
+QAPI domain extension.
+"""
+
+from __future__ import annotations
+
+from typing import (
+ TYPE_CHECKING,
+ AbstractSet,
+ Any,
+ Dict,
+ Tuple,
+)
+
+from sphinx.domains import Domain, ObjType
+from sphinx.util import logging
+
+
+if TYPE_CHECKING:
+ from sphinx.application import Sphinx
+
+logger = logging.getLogger(__name__)
+
+
+class QAPIDomain(Domain):
+ """QAPI language domain."""
+
+ name = "qapi"
+ label = "QAPI"
+
+ object_types: Dict[str, ObjType] = {}
+ directives = {}
+ roles = {}
+ initial_data: Dict[str, Dict[str, Tuple[Any]]] = {}
+ indices = []
+
+ def merge_domaindata(
+ self, docnames: AbstractSet[str], otherdata: Dict[str, Any]
+ ) -> None:
+ pass
+
+ def resolve_any_xref(self, *args: Any, **kwargs: Any) -> Any:
+ # pylint: disable=unused-argument
+ return []
+
+
+def setup(app: Sphinx) -> Dict[str, Any]:
+ app.setup_extension("sphinx.directives")
+ app.add_domain(QAPIDomain)
+
+ return {
+ "version": "1.0",
+ "env_version": 1,
+ "parallel_read_safe": True,
+ "parallel_write_safe": True,
+ }
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 04/57] docs/sphinx: add compat.py module and nested_parse helper
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (2 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 03/57] docs/sphinx: create QAPI domain extension stub John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-07 5:46 ` Markus Armbruster
2025-03-05 3:45 ` [PATCH 05/57] docs/qapi-domain: add qapi:module directive John Snow
` (55 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Create a compat module that handles sphinx cross-version compatibility
issues. For the inaugural function, add a nested_parse() helper that
handles differences in line number tracking for nested directive body
parsing.
Spoilers: there are more cross-version hacks to come throughout the
series.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/compat.py | 33 +++++++++++++++++++++++++++++++++
1 file changed, 33 insertions(+)
create mode 100644 docs/sphinx/compat.py
diff --git a/docs/sphinx/compat.py b/docs/sphinx/compat.py
new file mode 100644
index 00000000000..792aca10e39
--- /dev/null
+++ b/docs/sphinx/compat.py
@@ -0,0 +1,33 @@
+"""
+Sphinx cross-version compatibility goop
+"""
+
+from docutils.nodes import Element
+
+from sphinx.util.docutils import SphinxDirective, switch_source_input
+from sphinx.util.nodes import nested_parse_with_titles
+
+
+def nested_parse(directive: SphinxDirective, content_node: Element) -> None:
+ """
+ This helper preserves error parsing context across sphinx versions.
+ """
+
+ # necessary so that the child nodes get the right source/line set
+ content_node.document = directive.state.document
+
+ try:
+ # Modern sphinx (6.2.0+) supports proper offsetting for
+ # nested parse error context management
+ nested_parse_with_titles(
+ directive.state,
+ directive.content,
+ content_node,
+ content_offset=directive.content_offset,
+ )
+ except TypeError:
+ # No content_offset argument. Fall back to SSI method.
+ with switch_source_input(directive.state, directive.content):
+ nested_parse_with_titles(
+ directive.state, directive.content, content_node
+ )
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 04/57] docs/sphinx: add compat.py module and nested_parse helper
2025-03-05 3:45 ` [PATCH 04/57] docs/sphinx: add compat.py module and nested_parse helper John Snow
@ 2025-03-07 5:46 ` Markus Armbruster
2025-03-07 20:08 ` John Snow
0 siblings, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-07 5:46 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> Create a compat module that handles sphinx cross-version compatibility
> issues. For the inaugural function, add a nested_parse() helper that
> handles differences in line number tracking for nested directive body
> parsing.
>
> Spoilers: there are more cross-version hacks to come throughout the
> series.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> docs/sphinx/compat.py | 33 +++++++++++++++++++++++++++++++++
> 1 file changed, 33 insertions(+)
> create mode 100644 docs/sphinx/compat.py
>
> diff --git a/docs/sphinx/compat.py b/docs/sphinx/compat.py
> new file mode 100644
> index 00000000000..792aca10e39
> --- /dev/null
> +++ b/docs/sphinx/compat.py
> @@ -0,0 +1,33 @@
> +"""
> +Sphinx cross-version compatibility goop
> +"""
> +
> +from docutils.nodes import Element
> +
> +from sphinx.util.docutils import SphinxDirective, switch_source_input
> +from sphinx.util.nodes import nested_parse_with_titles
> +
> +
> +def nested_parse(directive: SphinxDirective, content_node: Element) -> None:
> + """
> + This helper preserves error parsing context across sphinx versions.
> + """
> +
> + # necessary so that the child nodes get the right source/line set
> + content_node.document = directive.state.document
> +
> + try:
> + # Modern sphinx (6.2.0+) supports proper offsetting for
> + # nested parse error context management
> + nested_parse_with_titles(
> + directive.state,
> + directive.content,
> + content_node,
> + content_offset=directive.content_offset,
> + )
> + except TypeError:
> + # No content_offset argument. Fall back to SSI method.
> + with switch_source_input(directive.state, directive.content):
> + nested_parse_with_titles(
> + directive.state, directive.content, content_node
> + )
The function wraps around sphinx.util.nodes.nested_parse_with_titles().
Would calling it nested_parse_with_titles() reduce readers' cognitive
load at call sites?
Please do not misinterpret my question as a demand. It's really just a
question :)
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 04/57] docs/sphinx: add compat.py module and nested_parse helper
2025-03-07 5:46 ` Markus Armbruster
@ 2025-03-07 20:08 ` John Snow
0 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-07 20:08 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 2447 bytes --]
On Fri, Mar 7, 2025 at 12:46 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > Create a compat module that handles sphinx cross-version compatibility
> > issues. For the inaugural function, add a nested_parse() helper that
> > handles differences in line number tracking for nested directive body
> > parsing.
> >
> > Spoilers: there are more cross-version hacks to come throughout the
> > series.
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
> > ---
> > docs/sphinx/compat.py | 33 +++++++++++++++++++++++++++++++++
> > 1 file changed, 33 insertions(+)
> > create mode 100644 docs/sphinx/compat.py
> >
> > diff --git a/docs/sphinx/compat.py b/docs/sphinx/compat.py
> > new file mode 100644
> > index 00000000000..792aca10e39
> > --- /dev/null
> > +++ b/docs/sphinx/compat.py
> > @@ -0,0 +1,33 @@
> > +"""
> > +Sphinx cross-version compatibility goop
> > +"""
> > +
> > +from docutils.nodes import Element
> > +
> > +from sphinx.util.docutils import SphinxDirective, switch_source_input
> > +from sphinx.util.nodes import nested_parse_with_titles
> > +
> > +
> > +def nested_parse(directive: SphinxDirective, content_node: Element) ->
> None:
> > + """
> > + This helper preserves error parsing context across sphinx versions.
> > + """
> > +
> > + # necessary so that the child nodes get the right source/line set
> > + content_node.document = directive.state.document
> > +
> > + try:
> > + # Modern sphinx (6.2.0+) supports proper offsetting for
> > + # nested parse error context management
> > + nested_parse_with_titles(
> > + directive.state,
> > + directive.content,
> > + content_node,
> > + content_offset=directive.content_offset,
> > + )
> > + except TypeError:
> > + # No content_offset argument. Fall back to SSI method.
> > + with switch_source_input(directive.state, directive.content):
> > + nested_parse_with_titles(
> > + directive.state, directive.content, content_node
> > + )
>
> The function wraps around sphinx.util.nodes.nested_parse_with_titles().
> Would calling it nested_parse_with_titles() reduce readers' cognitive
> load at call sites?
>
> Please do not misinterpret my question as a demand. It's really just a
> question :)
>
Sure, easy change.
[-- Attachment #2: Type: text/html, Size: 3348 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 05/57] docs/qapi-domain: add qapi:module directive
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (3 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 04/57] docs/sphinx: add compat.py module and nested_parse helper John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 06/57] docs/qapi-domain: add QAPI domain object registry John Snow
` (54 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
This adds a qapi:module directive, which just notes the current module
being documented and performs a nested parse of the content block, if
present.
This code is based pretty heavily on Sphinx's PyModule directive, but
with unnecessary features excised.
For example:
.. qapi:module:: block-core
Hello, and welcome to block-core!
=================================
lorem ipsum, dolor sit amet ...
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapi_domain.py | 107 ++++++++++++++++++++++++++++++++++++-
1 file changed, 106 insertions(+), 1 deletion(-)
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index a1983d94440..8ce3caf933d 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -8,20 +8,119 @@
TYPE_CHECKING,
AbstractSet,
Any,
+ ClassVar,
Dict,
+ Iterable,
+ List,
Tuple,
+ cast,
)
+from docutils import nodes
+from docutils.parsers.rst import directives
+
+from compat import nested_parse
+from sphinx import addnodes
from sphinx.domains import Domain, ObjType
from sphinx.util import logging
+from sphinx.util.docutils import SphinxDirective
+from sphinx.util.nodes import make_id
if TYPE_CHECKING:
+ from docutils.nodes import Element, Node
+
from sphinx.application import Sphinx
+ from sphinx.util.typing import OptionSpec
logger = logging.getLogger(__name__)
+class QAPIModule(SphinxDirective):
+ """
+ Directive to mark description of a new module.
+
+ This directive doesn't generate any special formatting, and is just
+ a pass-through for the content body. Named section titles are
+ allowed in the content body.
+
+ Use this directive to associate subsequent definitions with the
+ module they are defined in for purposes of search and QAPI index
+ organization.
+
+ :arg: The name of the module.
+ :opt no-index: Don't add cross-reference targets or index entries.
+ :opt no-typesetting: Don't render the content body (but preserve any
+ cross-reference target IDs in the squelched output.)
+
+ Example::
+
+ .. qapi:module:: block-core
+ :no-index:
+ :no-typesetting:
+
+ Lorem ipsum, dolor sit amet ...
+
+ """
+
+ has_content = True
+ required_arguments = 1
+ optional_arguments = 0
+ final_argument_whitespace = False
+
+ option_spec: ClassVar[OptionSpec] = {
+ # These are universal "Basic" options;
+ # https://www.sphinx-doc.org/en/master/usage/domains/index.html#basic-markup
+ "no-index": directives.flag,
+ "no-typesetting": directives.flag,
+ "no-contents-entry": directives.flag, # NB: No effect
+ # Deprecated aliases; to be removed in Sphinx 9.0
+ "noindex": directives.flag,
+ "nocontentsentry": directives.flag, # NB: No effect
+ }
+
+ def run(self) -> List[Node]:
+ modname = self.arguments[0].strip()
+ no_index = "no-index" in self.options or "noindex" in self.options
+
+ self.env.ref_context["qapi:module"] = modname
+
+ content_node: Element = nodes.section()
+ nested_parse(self, content_node)
+
+ ret: List[Node] = []
+ inode = addnodes.index(entries=[])
+
+ if not no_index:
+ node_id = make_id(self.env, self.state.document, "module", modname)
+ target = nodes.target("", "", ids=[node_id], ismod=True)
+ self.set_source_info(target)
+ self.state.document.note_explicit_target(target)
+
+ indextext = f"QAPI module; {modname}"
+ inode = addnodes.index(
+ entries=[
+ ("pair", indextext, node_id, "", None),
+ ]
+ )
+ ret.append(inode)
+ content_node.insert(0, target)
+
+ if "no-typesetting" in self.options:
+ if node_ids := [
+ node_id
+ for el in content_node.findall(nodes.Element)
+ for node_id in cast(Iterable[str], el.get("ids", ()))
+ ]:
+ target = nodes.target(ids=node_ids)
+ self.set_source_info(target)
+ ret.append(target)
+ else:
+ ret.extend(content_node.children)
+
+ return ret
+
+
class QAPIDomain(Domain):
"""QAPI language domain."""
@@ -29,7 +128,13 @@ class QAPIDomain(Domain):
label = "QAPI"
object_types: Dict[str, ObjType] = {}
- directives = {}
+
+ # Each of these provides a rST directive,
+ # e.g. .. qapi:module:: block-core
+ directives = {
+ "module": QAPIModule,
+ }
+
roles = {}
initial_data: Dict[str, Dict[str, Tuple[Any]]] = {}
indices = []
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 06/57] docs/qapi-domain: add QAPI domain object registry
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (4 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 05/57] docs/qapi-domain: add qapi:module directive John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 07/57] docs/qapi-domain: add QAPI index John Snow
` (53 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
This is the first step towards QAPI domain cross-references and a QAPI
reference index.
This patch just creates the object registry and amends the qapi:module
directive to use that registry. Update the merge_domaindata stub method
now that we have actual data we may need to merge.
This patch also defines that "module" entities can be referenced with
:qapi:mod:`foo` or :qapi:any:`foo` references, although the
implementation for those roles is handled in a forthcoming patch.
Note that how to handle merge conflict resolution is unhandled, as the
Sphinx python domain itself does not handle it either. I do not know how
to intentionally trigger it, so I've left an assertion instead if it
should ever come up ...
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapi_domain.py | 81 ++++++++++++++++++++++++++++++++++++--
1 file changed, 78 insertions(+), 3 deletions(-)
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index 8ce3caf933d..b17fcb93f24 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -12,6 +12,7 @@
Dict,
Iterable,
List,
+ NamedTuple,
Tuple,
cast,
)
@@ -22,6 +23,7 @@
from compat import nested_parse
from sphinx import addnodes
from sphinx.domains import Domain, ObjType
+from sphinx.locale import _, __
from sphinx.util import logging
from sphinx.util.docutils import SphinxDirective
from sphinx.util.nodes import make_id
@@ -36,6 +38,13 @@
logger = logging.getLogger(__name__)
+class ObjectEntry(NamedTuple):
+ docname: str
+ node_id: str
+ objtype: str
+ aliased: bool
+
+
class QAPIModule(SphinxDirective):
"""
Directive to mark description of a new module.
@@ -80,6 +89,7 @@ class QAPIModule(SphinxDirective):
}
def run(self) -> List[Node]:
+ domain = cast(QAPIDomain, self.env.get_domain("qapi"))
modname = self.arguments[0].strip()
no_index = "no-index" in self.options or "noindex" in self.options
@@ -92,11 +102,14 @@ def run(self) -> List[Node]:
inode = addnodes.index(entries=[])
if not no_index:
+ # note module to the domain
node_id = make_id(self.env, self.state.document, "module", modname)
target = nodes.target("", "", ids=[node_id], ismod=True)
self.set_source_info(target)
self.state.document.note_explicit_target(target)
+ domain.note_object(modname, "module", node_id, location=target)
+
indextext = f"QAPI module; {modname}"
inode = addnodes.index(
entries=[
@@ -127,7 +140,12 @@ class QAPIDomain(Domain):
name = "qapi"
label = "QAPI"
- object_types: Dict[str, ObjType] = {}
+ # This table associates cross-reference object types (key) with an
+ # ObjType instance, which defines the valid cross-reference roles
+ # for each object type.
+ object_types: Dict[str, ObjType] = {
+ "module": ObjType(_("module"), "mod", "any"),
+ }
# Each of these provides a rST directive,
# e.g. .. qapi:module:: block-core
@@ -136,13 +154,70 @@ class QAPIDomain(Domain):
}
roles = {}
- initial_data: Dict[str, Dict[str, Tuple[Any]]] = {}
+
+ # Moved into the data property at runtime;
+ # this is the internal index of reference-able objects.
+ initial_data: Dict[str, Dict[str, Tuple[Any]]] = {
+ "objects": {}, # fullname -> ObjectEntry
+ }
+
indices = []
+ @property
+ def objects(self) -> Dict[str, ObjectEntry]:
+ ret = self.data.setdefault("objects", {})
+ return ret # type: ignore[no-any-return]
+
+ def note_object(
+ self,
+ name: str,
+ objtype: str,
+ node_id: str,
+ aliased: bool = False,
+ location: Any = None,
+ ) -> None:
+ """Note a QAPI object for cross reference."""
+ if name in self.objects:
+ other = self.objects[name]
+ if other.aliased and aliased is False:
+ # The original definition found. Override it!
+ pass
+ elif other.aliased is False and aliased:
+ # The original definition is already registered.
+ return
+ else:
+ # duplicated
+ logger.warning(
+ __(
+ "duplicate object description of %s, "
+ "other instance in %s, use :no-index: for one of them"
+ ),
+ name,
+ other.docname,
+ location=location,
+ )
+ self.objects[name] = ObjectEntry(
+ self.env.docname, node_id, objtype, aliased
+ )
+
+ def clear_doc(self, docname: str) -> None:
+ for fullname, obj in list(self.objects.items()):
+ if obj.docname == docname:
+ del self.objects[fullname]
+
def merge_domaindata(
self, docnames: AbstractSet[str], otherdata: Dict[str, Any]
) -> None:
- pass
+ for fullname, obj in otherdata["objects"].items():
+ if obj.docname in docnames:
+ # Sphinx's own python domain doesn't appear to bother to
+ # check for collisions. Assert they don't happen and
+ # we'll fix it if/when the case arises.
+ assert fullname not in self.objects, (
+ "bug - collision on merge?"
+ f" {fullname=} {obj=} {self.objects[fullname]=}"
+ )
+ self.objects[fullname] = obj
def resolve_any_xref(self, *args: Any, **kwargs: Any) -> Any:
# pylint: disable=unused-argument
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 07/57] docs/qapi-domain: add QAPI index
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (5 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 06/57] docs/qapi-domain: add QAPI domain object registry John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 08/57] docs/qapi-domain: add resolve_any_xref() John Snow
` (52 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Use the QAPI object registry to generate a special index just for QAPI
definitions. The index can show entries both by definition type and all
together, alphabetically.
The index can be linked from anywhere in the QEMU manual by using the
reference `qapi-index`.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapi_domain.py | 76 +++++++++++++++++++++++++++++++++++---
1 file changed, 71 insertions(+), 5 deletions(-)
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index b17fcb93f24..716a38a5f00 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -13,6 +13,7 @@
Iterable,
List,
NamedTuple,
+ Optional,
Tuple,
cast,
)
@@ -22,7 +23,12 @@
from compat import nested_parse
from sphinx import addnodes
-from sphinx.domains import Domain, ObjType
+from sphinx.domains import (
+ Domain,
+ Index,
+ IndexEntry,
+ ObjType,
+)
from sphinx.locale import _, __
from sphinx.util import logging
from sphinx.util.docutils import SphinxDirective
@@ -53,9 +59,10 @@ class QAPIModule(SphinxDirective):
a pass-through for the content body. Named section titles are
allowed in the content body.
- Use this directive to associate subsequent definitions with the
- module they are defined in for purposes of search and QAPI index
- organization.
+ Use this directive to create entries for the QAPI module in the
+ global index and the qapi index; as well as to associate subsequent
+ definitions with the module they are defined in for purposes of
+ search and QAPI index organization.
:arg: The name of the module.
:opt no-index: Don't add cross-reference targets or index entries.
@@ -134,6 +141,62 @@ def run(self) -> List[Node]:
return ret
+class QAPIIndex(Index):
+ """
+ Index subclass to provide the QAPI definition index.
+ """
+
+ # pylint: disable=too-few-public-methods
+
+ name = "index"
+ localname = _("QAPI Index")
+ shortname = _("QAPI Index")
+
+ def generate(
+ self,
+ docnames: Optional[Iterable[str]] = None,
+ ) -> Tuple[List[Tuple[str, List[IndexEntry]]], bool]:
+ assert isinstance(self.domain, QAPIDomain)
+ content: Dict[str, List[IndexEntry]] = {}
+ collapse = False
+
+ # list of all object (name, ObjectEntry) pairs, sorted by name
+ # (ignoring the module)
+ objects = sorted(
+ self.domain.objects.items(),
+ key=lambda x: x[0].split(".")[-1].lower(),
+ )
+
+ for objname, obj in objects:
+ if docnames and obj.docname not in docnames:
+ continue
+
+ # Strip the module name out:
+ objname = objname.split(".")[-1]
+
+ # Add an alphabetical entry:
+ entries = content.setdefault(objname[0].upper(), [])
+ entries.append(
+ IndexEntry(
+ objname, 0, obj.docname, obj.node_id, obj.objtype, "", ""
+ )
+ )
+
+ # Add a categorical entry:
+ category = obj.objtype.title() + "s"
+ entries = content.setdefault(category, [])
+ entries.append(
+ IndexEntry(objname, 0, obj.docname, obj.node_id, "", "", "")
+ )
+
+ # alphabetically sort categories; type names first, ABC entries last.
+ sorted_content = sorted(
+ content.items(),
+ key=lambda x: (len(x[0]) == 1, x[0]),
+ )
+ return sorted_content, collapse
+
+
class QAPIDomain(Domain):
"""QAPI language domain."""
@@ -161,7 +224,10 @@ class QAPIDomain(Domain):
"objects": {}, # fullname -> ObjectEntry
}
- indices = []
+ # Index pages to generate; each entry is an Index class.
+ indices = [
+ QAPIIndex,
+ ]
@property
def objects(self) -> Dict[str, ObjectEntry]:
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 08/57] docs/qapi-domain: add resolve_any_xref()
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (6 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 07/57] docs/qapi-domain: add QAPI index John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 09/57] docs/qapi-domain: add QAPI xref roles John Snow
` (51 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Add the ability to resolve cross-references using the `any`
cross-reference syntax. Adding QAPI-specific cross-reference roles will
be added in a forthcoming commit, and will share the same find_obj()
helper.
(There's less code needed for the generic cross-reference resolver, so
it comes first in this series.)
Once again, this code is based very heavily on sphinx.domains.python.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapi_domain.py | 98 ++++++++++++++++++++++++++++++++++++--
1 file changed, 94 insertions(+), 4 deletions(-)
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index 716a38a5f00..744956045e8 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -23,6 +23,7 @@
from compat import nested_parse
from sphinx import addnodes
+from sphinx.addnodes import pending_xref
from sphinx.domains import (
Domain,
Index,
@@ -32,13 +33,15 @@
from sphinx.locale import _, __
from sphinx.util import logging
from sphinx.util.docutils import SphinxDirective
-from sphinx.util.nodes import make_id
+from sphinx.util.nodes import make_id, make_refnode
if TYPE_CHECKING:
from docutils.nodes import Element, Node
from sphinx.application import Sphinx
+ from sphinx.builders import Builder
+ from sphinx.environment import BuildEnvironment
from sphinx.util.typing import OptionSpec
logger = logging.getLogger(__name__)
@@ -285,9 +288,96 @@ def merge_domaindata(
)
self.objects[fullname] = obj
- def resolve_any_xref(self, *args: Any, **kwargs: Any) -> Any:
- # pylint: disable=unused-argument
- return []
+ def find_obj(
+ self, modname: str, name: str, typ: Optional[str]
+ ) -> list[tuple[str, ObjectEntry]]:
+ """
+ Find a QAPI object for "name", perhaps using the given module.
+
+ Returns a list of (name, object entry) tuples.
+
+ :param modname: The current module context (if any!)
+ under which we are searching.
+ :param name: The name of the x-ref to resolve;
+ may or may not include a leading module.
+ :param type: The role name of the x-ref we're resolving, if provided.
+ (This is absent for "any" lookups.)
+ """
+ if not name:
+ return []
+
+ names: list[str] = []
+ matches: list[tuple[str, ObjectEntry]] = []
+
+ fullname = name
+ if "." in fullname:
+ # We're searching for a fully qualified reference;
+ # ignore the contextual module.
+ pass
+ elif modname:
+ # We're searching for something from somewhere;
+ # try searching the current module first.
+ # e.g. :qapi:cmd:`query-block` or `query-block` is being searched.
+ fullname = f"{modname}.{name}"
+
+ if typ is None:
+ # type isn't specified, this is a generic xref.
+ # search *all* qapi-specific object types.
+ objtypes: Optional[List[str]] = list(self.object_types)
+ else:
+ # type is specified and will be a role (e.g. obj, mod, cmd)
+ # convert this to eligible object types (e.g. command, module)
+ # using the QAPIDomain.object_types table.
+ objtypes = self.objtypes_for_role(typ)
+
+ # Either we should have been given no type, or the type we were
+ # given should correspond to at least one real actual object
+ # type.
+ assert objtypes
+
+ if name in self.objects and self.objects[name].objtype in objtypes:
+ names = [name]
+ elif (
+ fullname in self.objects
+ and self.objects[fullname].objtype in objtypes
+ ):
+ names = [fullname]
+ else:
+ # exact match wasn't found; e.g. we are searching for
+ # `query-block` from a different (or no) module.
+ searchname = "." + name
+ names = [
+ oname
+ for oname in self.objects
+ if oname.endswith(searchname)
+ and self.objects[oname].objtype in objtypes
+ ]
+
+ matches = [(oname, self.objects[oname]) for oname in names]
+ if len(matches) > 1:
+ matches = [m for m in matches if not m[1].aliased]
+ return matches
+
+ def resolve_any_xref(
+ self,
+ env: BuildEnvironment,
+ fromdocname: str,
+ builder: Builder,
+ target: str,
+ node: pending_xref,
+ contnode: Element,
+ ) -> List[Tuple[str, nodes.reference]]:
+ results: List[Tuple[str, nodes.reference]] = []
+ matches = self.find_obj(node.get("qapi:module"), target, None)
+ for name, obj in matches:
+ rolename = self.role_for_objtype(obj.objtype)
+ assert rolename is not None
+ role = f"qapi:{rolename}"
+ refnode = make_refnode(
+ builder, fromdocname, obj.docname, obj.node_id, contnode, name
+ )
+ results.append((role, refnode))
+ return results
def setup(app: Sphinx) -> Dict[str, Any]:
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 09/57] docs/qapi-domain: add QAPI xref roles
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (7 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 08/57] docs/qapi-domain: add resolve_any_xref() John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-07 10:08 ` Markus Armbruster
2025-03-05 3:45 ` [PATCH 10/57] docs/qapi-domain: add compatibility node classes John Snow
` (50 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Add domain-specific cross-reference syntax. As of this commit, that
means new :qapi:mod:`block-core` and :qapi:any:`block-core` referencing
syntax.
:mod: will only find modules, but :any: will find anything registered to
the QAPI domain. (In forthcoming commits, this means commands, events,
enums, etc.)
Creating the cross-references is powered by the QAPIXRefRole class;
resolving them is handled by QAPIDomain.resolve_xref().
QAPIXrefRole is copied almost verbatim from Sphinx's own
PyXrefRole. PyXrefRole (and QAPIXrefRole) adds two features over the
base class:
(1) Creating a cross-reference with e.g. :py:class:`~class.name`
instructs sphinx to omit the fully qualified parts of the resolved name
from the actual link text. This may be useful in the future if we add
namespaces to QAPI documentation, e.g. :qapi:cmd:`~qsd.blockdev-backup`
could link to the QSD-specific documentation for blockdev-backup while
omitting that prefix from the link text.
(2) Prefixing the link target with a "." changes the search behavior to
prefer locally-scoped items first.
I think both of these are worth keeping to help manage future namespace
issues between QEMU, QSD and QGA; but it's possible it's extraneous. It
may possibly be worth keeping just to keep feature parity with Sphinx's
other domains; e.g. "principle of least surprise". Dunno.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapi_domain.py | 88 +++++++++++++++++++++++++++++++++++++-
1 file changed, 87 insertions(+), 1 deletion(-)
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index 744956045e8..104bae709f3 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -31,6 +31,7 @@
ObjType,
)
from sphinx.locale import _, __
+from sphinx.roles import XRefRole
from sphinx.util import logging
from sphinx.util.docutils import SphinxDirective
from sphinx.util.nodes import make_id, make_refnode
@@ -54,6 +55,54 @@ class ObjectEntry(NamedTuple):
aliased: bool
+class QAPIXRefRole(XRefRole):
+
+ def process_link(
+ self,
+ env: BuildEnvironment,
+ refnode: Element,
+ has_explicit_title: bool,
+ title: str,
+ target: str,
+ ) -> tuple[str, str]:
+ refnode["qapi:module"] = env.ref_context.get("qapi:module")
+
+ # Cross-references that begin with a tilde adjust the title to
+ # only show the reference without a leading module, even if one
+ # was provided. This is a Sphinx-standard syntax; give it
+ # priority over QAPI-specific type markup below.
+ hide_module = False
+ if target.startswith("~"):
+ hide_module = True
+ target = target[1:]
+
+ # Type names that end with "?" are considered optional
+ # arguments and should be documented as such, but it's not
+ # part of the xref itself.
+ if target.endswith("?"):
+ refnode["qapi:optional"] = True
+ target = target[:-1]
+
+ # Type names wrapped in brackets denote lists. strip the
+ # brackets and remember to add them back later.
+ if target.startswith("[") and target.endswith("]"):
+ refnode["qapi:array"] = True
+ target = target[1:-1]
+
+ if has_explicit_title:
+ # Don't mess with the title at all if it was explicitly set.
+ # Explicit title syntax for references is e.g.
+ # :qapi:type:`target <explicit title>`
+ # and this explicit title overrides everything else here.
+ return title, target
+
+ title = target
+ if hide_module:
+ title = target.split(".")[-1]
+
+ return title, target
+
+
class QAPIModule(SphinxDirective):
"""
Directive to mark description of a new module.
@@ -219,7 +268,13 @@ class QAPIDomain(Domain):
"module": QAPIModule,
}
- roles = {}
+ # These are all cross-reference roles; e.g.
+ # :qapi:cmd:`query-block`. The keys correlate to the names used in
+ # the object_types table values above.
+ roles = {
+ "mod": QAPIXRefRole(),
+ "any": QAPIXRefRole(), # reference *any* type of QAPI object.
+ }
# Moved into the data property at runtime;
# this is the internal index of reference-able objects.
@@ -358,6 +413,37 @@ def find_obj(
matches = [m for m in matches if not m[1].aliased]
return matches
+ def resolve_xref(
+ self,
+ env: BuildEnvironment,
+ fromdocname: str,
+ builder: Builder,
+ typ: str,
+ target: str,
+ node: pending_xref,
+ contnode: Element,
+ ) -> nodes.reference | None:
+ modname = node.get("qapi:module")
+ matches = self.find_obj(modname, target, typ)
+
+ if not matches:
+ return None
+
+ if len(matches) > 1:
+ logger.warning(
+ __("more than one target found for cross-reference %r: %s"),
+ target,
+ ", ".join(match[0] for match in matches),
+ type="ref",
+ subtype="qapi",
+ location=node,
+ )
+
+ name, obj = matches[0]
+ return make_refnode(
+ builder, fromdocname, obj.docname, obj.node_id, contnode, name
+ )
+
def resolve_any_xref(
self,
env: BuildEnvironment,
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 09/57] docs/qapi-domain: add QAPI xref roles
2025-03-05 3:45 ` [PATCH 09/57] docs/qapi-domain: add QAPI xref roles John Snow
@ 2025-03-07 10:08 ` Markus Armbruster
2025-03-07 21:23 ` John Snow
0 siblings, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-07 10:08 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> Add domain-specific cross-reference syntax. As of this commit, that
> means new :qapi:mod:`block-core` and :qapi:any:`block-core` referencing
> syntax.
>
> :mod: will only find modules, but :any: will find anything registered to
> the QAPI domain. (In forthcoming commits, this means commands, events,
> enums, etc.)
I understand :any: will find any QAPI schema definitions. Does it find
modules, too?
How could roles narrower than "definition" be useful?
I'm asking because naming rules preclude naming collisions among
definitions:
* Events are ALL_CAPS
* Commands are lower-case-with-dashes, except some older ones use
underscores (pragma command-name-exceptions).
* Types are CamelCase. Note that "C" is not considered a camel.
Fine print: these are the rules for stems, i.e. the name without RFQDN
or 'x-' prefixes, if any.
If :any: finds modules, then commands and modules could collide.
Nothing else can.
> Creating the cross-references is powered by the QAPIXRefRole class;
> resolving them is handled by QAPIDomain.resolve_xref().
>
> QAPIXrefRole is copied almost verbatim from Sphinx's own
> PyXrefRole. PyXrefRole (and QAPIXrefRole) adds two features over the
> base class:
>
> (1) Creating a cross-reference with e.g. :py:class:`~class.name`
> instructs sphinx to omit the fully qualified parts of the resolved name
> from the actual link text. This may be useful in the future if we add
> namespaces to QAPI documentation, e.g. :qapi:cmd:`~qsd.blockdev-backup`
> could link to the QSD-specific documentation for blockdev-backup while
> omitting that prefix from the link text.
>
> (2) Prefixing the link target with a "." changes the search behavior to
> prefer locally-scoped items first.
>
> I think both of these are worth keeping to help manage future namespace
> issues between QEMU, QSD and QGA; but it's possible it's extraneous. It
> may possibly be worth keeping just to keep feature parity with Sphinx's
> other domains; e.g. "principle of least surprise". Dunno.
I generally avoid features without uses. But I trust your judgement
here: you decide.
> Signed-off-by: John Snow <jsnow@redhat.com>
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 09/57] docs/qapi-domain: add QAPI xref roles
2025-03-07 10:08 ` Markus Armbruster
@ 2025-03-07 21:23 ` John Snow
0 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-07 21:23 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 3558 bytes --]
On Fri, Mar 7, 2025 at 5:09 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > Add domain-specific cross-reference syntax. As of this commit, that
> > means new :qapi:mod:`block-core` and :qapi:any:`block-core` referencing
> > syntax.
> >
> > :mod: will only find modules, but :any: will find anything registered to
> > the QAPI domain. (In forthcoming commits, this means commands, events,
> > enums, etc.)
>
> I understand :any: will find any QAPI schema definitions. Does it find
> modules, too?
>
`block-core` and :any:`block-core` will find modules, yes. So will
:qapi:mod:`block-core` and :qapi:any:`block-core`.
>
> How could roles narrower than "definition" be useful?
>
At the very least, :qapi:type:`foo` - which omits commands, events, and
modules -- is useful for the role to apply to field lists, since those
things can never possibly be anything other than a data type. The parser
may protect for this, but the QAPI domain also doesn't bother allowing any
such mishaps either.
> I'm asking because naming rules preclude naming collisions among
> definitions:
>
> * Events are ALL_CAPS
>
> * Commands are lower-case-with-dashes, except some older ones use
> underscores (pragma command-name-exceptions).
>
> * Types are CamelCase. Note that "C" is not considered a camel.
>
> Fine print: these are the rules for stems, i.e. the name without RFQDN
> or 'x-' prefixes, if any.
>
> If :any: finds modules, then commands and modules could collide.
> Nothing else can.
>
Right, the narrow roles might not be that useful in practice, but they also
cost virtually nothing codewise. I will admit to just copying the Python
domain. I almost feel as if they're interesting to leave in if only to
serve as documentation for how one could narrow the xref roles, but ...
yeah, I admit it's a bit superfluous.
>
> > Creating the cross-references is powered by the QAPIXRefRole class;
> > resolving them is handled by QAPIDomain.resolve_xref().
> >
> > QAPIXrefRole is copied almost verbatim from Sphinx's own
> > PyXrefRole. PyXrefRole (and QAPIXrefRole) adds two features over the
> > base class:
> >
> > (1) Creating a cross-reference with e.g. :py:class:`~class.name`
> > instructs sphinx to omit the fully qualified parts of the resolved name
> > from the actual link text. This may be useful in the future if we add
> > namespaces to QAPI documentation, e.g. :qapi:cmd:`~qsd.blockdev-backup`
> > could link to the QSD-specific documentation for blockdev-backup while
> > omitting that prefix from the link text.
> >
> > (2) Prefixing the link target with a "." changes the search behavior to
> > prefer locally-scoped items first.
> >
> > I think both of these are worth keeping to help manage future namespace
> > issues between QEMU, QSD and QGA; but it's possible it's extraneous. It
> > may possibly be worth keeping just to keep feature parity with Sphinx's
> > other domains; e.g. "principle of least surprise". Dunno.
>
> I generally avoid features without uses. But I trust your judgement
> here: you decide.
>
Generally, I felt it was easiest to "go with the flow". I am actually
halfway through adding namespace support (surprise...!) and I will
re-evaluate how useful these syntax features are when I'm done.
(I don't know if I will include that feature in the respin for this series
or not, it might be a just-after-freeze thing.)
>
> > Signed-off-by: John Snow <jsnow@redhat.com>
>
>
[-- Attachment #2: Type: text/html, Size: 5045 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 10/57] docs/qapi-domain: add compatibility node classes
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (8 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 09/57] docs/qapi-domain: add QAPI xref roles John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 11/57] docs/qapi-domain: add qapi:command directive John Snow
` (49 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Sphinx prior to v4.0 uses different classes for rendering elements of
documentation objects; add some compatibility classes to use the right
node classes conditionally.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/compat.py | 17 ++++++++++++++++-
1 file changed, 16 insertions(+), 1 deletion(-)
diff --git a/docs/sphinx/compat.py b/docs/sphinx/compat.py
index 792aca10e39..1f9f1f42ef7 100644
--- a/docs/sphinx/compat.py
+++ b/docs/sphinx/compat.py
@@ -2,12 +2,27 @@
Sphinx cross-version compatibility goop
"""
-from docutils.nodes import Element
+from typing import Callable
+from docutils.nodes import Element, Node, Text
+
+import sphinx
+from sphinx import addnodes
from sphinx.util.docutils import SphinxDirective, switch_source_input
from sphinx.util.nodes import nested_parse_with_titles
+SpaceNode: Callable[[str], Node]
+KeywordNode: Callable[[str, str], Node]
+
+if sphinx.version_info[:3] >= (4, 0, 0):
+ SpaceNode = addnodes.desc_sig_space
+ KeywordNode = addnodes.desc_sig_keyword
+else:
+ SpaceNode = Text
+ KeywordNode = addnodes.desc_annotation
+
+
def nested_parse(directive: SphinxDirective, content_node: Element) -> None:
"""
This helper preserves error parsing context across sphinx versions.
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 11/57] docs/qapi-domain: add qapi:command directive
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (9 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 10/57] docs/qapi-domain: add compatibility node classes John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-07 6:33 ` Markus Armbruster
2025-03-05 3:45 ` [PATCH 12/57] docs/qapi-domain: add :since: directive option John Snow
` (48 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
This commit adds a stubbed version of QAPICommand that utilizes the
QAPIObject class, the qapi:command directive, the :qapi:cmd:
cross-reference role, and the "command" object type in the QAPI object
registry.
This commit also adds the aforementioned generic QAPIObject class for
use in documenting various QAPI entities in the Sphinx ecosystem.
They don't do anything *particularly* interesting yet, but that will
come in forthcoming commits.
Note: some versions of mypy get a little confused over the difference
between class and instance variables; because sphinx's ObjectDescription
does not declare option_spec as a ClassVar (even though it's obvious
that it is), mypy may produce this error:
qapi-domain.py:125: error: Cannot override instance variable (previously
declared on base class "ObjectDescription") with class variable [misc]
I can't control that; so silence the error with a pragma.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapi_domain.py | 146 ++++++++++++++++++++++++++++++++++++-
1 file changed, 144 insertions(+), 2 deletions(-)
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index 104bae709f3..6168c23936f 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -21,9 +21,10 @@
from docutils import nodes
from docutils.parsers.rst import directives
-from compat import nested_parse
+from compat import KeywordNode, SpaceNode, nested_parse
from sphinx import addnodes
-from sphinx.addnodes import pending_xref
+from sphinx.addnodes import desc_signature, pending_xref
+from sphinx.directives import ObjectDescription
from sphinx.domains import (
Domain,
Index,
@@ -103,6 +104,144 @@ def process_link(
return title, target
+# Alias for the return of handle_signature(), which is used in several places.
+# (In the Python domain, this is Tuple[str, str] instead.)
+Signature = str
+
+
+class QAPIObject(ObjectDescription[Signature]):
+ """
+ Description of a generic QAPI object.
+
+ It's not used directly, but is instead subclassed by specific directives.
+ """
+
+ # Inherit some standard options from Sphinx's ObjectDescription
+ option_spec: OptionSpec = ( # type:ignore[misc]
+ ObjectDescription.option_spec.copy()
+ )
+ option_spec.update(
+ {
+ # Borrowed from the Python domain:
+ "module": directives.unchanged, # Override contextual module name
+ }
+ )
+
+ def get_signature_prefix(self) -> List[nodes.Node]:
+ """Returns a prefix to put before the object name in the signature."""
+ assert self.objtype
+ return [
+ KeywordNode("", self.objtype.title()),
+ SpaceNode(" "),
+ ]
+
+ def get_signature_suffix(self) -> list[nodes.Node]:
+ """Returns a suffix to put after the object name in the signature."""
+ return []
+
+ def handle_signature(self, sig: str, signode: desc_signature) -> Signature:
+ """
+ Transform a QAPI definition name into RST nodes.
+
+ This method was originally intended for handling function
+ signatures. In the QAPI domain, however, we only pass the
+ definition name as the directive argument and handle everything
+ else in the content body with field lists.
+
+ As such, the only argument here is "sig", which is just the QAPI
+ definition name.
+ """
+ modname = self.options.get(
+ "module", self.env.ref_context.get("qapi:module")
+ )
+
+ signode["fullname"] = sig
+ signode["module"] = modname
+ sig_prefix = self.get_signature_prefix()
+ if sig_prefix:
+ signode += addnodes.desc_annotation(
+ str(sig_prefix), "", *sig_prefix
+ )
+ signode += addnodes.desc_name(sig, sig)
+ signode += self.get_signature_suffix()
+
+ return sig
+
+ def _object_hierarchy_parts(
+ self, sig_node: desc_signature
+ ) -> Tuple[str, ...]:
+ if "fullname" not in sig_node:
+ return ()
+ modname = sig_node.get("module")
+ fullname = sig_node["fullname"]
+
+ if modname:
+ return (modname, *fullname.split("."))
+
+ return tuple(fullname.split("."))
+
+ def get_index_text(self, modname: str, name: Signature) -> str:
+ """Return the text for the index entry of the object."""
+ # We don't care about the module name here:
+ # pylint: disable=unused-argument
+
+ # NB: this is used for the global index, not the QAPI index.
+ return f"{name} (QMP {self.objtype})"
+
+ def add_target_and_index(
+ self, name: Signature, sig: str, signode: desc_signature
+ ) -> None:
+ # Called by ObjectDescription.run with the result of
+ # handle_signature; name is the return value of handle_signature
+ # where sig is the original argument to handle_signature. In our
+ # case, they're the same for now.
+ assert self.objtype
+
+ modname = self.options.get(
+ "module", self.env.ref_context.get("qapi:module")
+ )
+ # Here, sphinx decides to prepend the module name. OK.
+ fullname = (modname + "." if modname else "") + name
+ node_id = make_id(self.env, self.state.document, "", fullname)
+ signode["ids"].append(node_id)
+ self.state.document.note_explicit_target(signode)
+
+ domain = cast(QAPIDomain, self.env.get_domain("qapi"))
+ domain.note_object(fullname, self.objtype, node_id, location=signode)
+
+ if "no-index-entry" not in self.options:
+ indextext = self.get_index_text(modname, name)
+ assert self.indexnode is not None
+ if indextext:
+ self.indexnode["entries"].append(
+ ("single", indextext, node_id, "", None)
+ )
+
+ def _toc_entry_name(self, sig_node: desc_signature) -> str:
+ # This controls the name in the TOC and on the sidebar.
+
+ # This is the return type of _object_hierarchy_parts().
+ toc_parts = cast(Tuple[str, ...], sig_node.get("_toc_parts", ()))
+ if not toc_parts:
+ return ""
+
+ config = self.env.app.config
+ *parents, name = toc_parts
+ if config.toc_object_entries_show_parents == "domain":
+ return sig_node.get("fullname", name)
+ if config.toc_object_entries_show_parents == "hide":
+ return name
+ if config.toc_object_entries_show_parents == "all":
+ return ".".join(parents + [name])
+ return ""
+
+
+class QAPICommand(QAPIObject):
+ """Description of a QAPI Command."""
+
+ # Nothing unique for now! Changed in later commits O:-)
+
+
class QAPIModule(SphinxDirective):
"""
Directive to mark description of a new module.
@@ -260,12 +399,14 @@ class QAPIDomain(Domain):
# for each object type.
object_types: Dict[str, ObjType] = {
"module": ObjType(_("module"), "mod", "any"),
+ "command": ObjType(_("command"), "cmd", "any"),
}
# Each of these provides a rST directive,
# e.g. .. qapi:module:: block-core
directives = {
"module": QAPIModule,
+ "command": QAPICommand,
}
# These are all cross-reference roles; e.g.
@@ -273,6 +414,7 @@ class QAPIDomain(Domain):
# the object_types table values above.
roles = {
"mod": QAPIXRefRole(),
+ "cmd": QAPIXRefRole(),
"any": QAPIXRefRole(), # reference *any* type of QAPI object.
}
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 11/57] docs/qapi-domain: add qapi:command directive
2025-03-05 3:45 ` [PATCH 11/57] docs/qapi-domain: add qapi:command directive John Snow
@ 2025-03-07 6:33 ` Markus Armbruster
2025-03-07 22:39 ` John Snow
0 siblings, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-07 6:33 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> This commit adds a stubbed version of QAPICommand that utilizes the
> QAPIObject class, the qapi:command directive, the :qapi:cmd:
> cross-reference role, and the "command" object type in the QAPI object
> registry.
>
> This commit also adds the aforementioned generic QAPIObject class for
> use in documenting various QAPI entities in the Sphinx ecosystem.
>
> They don't do anything *particularly* interesting yet, but that will
> come in forthcoming commits.
>
> Note: some versions of mypy get a little confused over the difference
> between class and instance variables; because sphinx's ObjectDescription
> does not declare option_spec as a ClassVar (even though it's obvious
> that it is), mypy may produce this error:
>
> qapi-domain.py:125: error: Cannot override instance variable (previously
> declared on base class "ObjectDescription") with class variable [misc]
>
> I can't control that; so silence the error with a pragma.
Is this still accurate? qapi-domain.py line 125 is a comment. I can't
see the pragma.
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> docs/sphinx/qapi_domain.py | 146 ++++++++++++++++++++++++++++++++++++-
> 1 file changed, 144 insertions(+), 2 deletions(-)
>
> diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
> index 104bae709f3..6168c23936f 100644
> --- a/docs/sphinx/qapi_domain.py
> +++ b/docs/sphinx/qapi_domain.py
> @@ -21,9 +21,10 @@
> from docutils import nodes
> from docutils.parsers.rst import directives
>
> -from compat import nested_parse
> +from compat import KeywordNode, SpaceNode, nested_parse
> from sphinx import addnodes
> -from sphinx.addnodes import pending_xref
> +from sphinx.addnodes import desc_signature, pending_xref
> +from sphinx.directives import ObjectDescription
> from sphinx.domains import (
> Domain,
> Index,
> @@ -103,6 +104,144 @@ def process_link(
> return title, target
>
>
> +# Alias for the return of handle_signature(), which is used in several places.
> +# (In the Python domain, this is Tuple[str, str] instead.)
> +Signature = str
> +
> +
> +class QAPIObject(ObjectDescription[Signature]):
> + """
> + Description of a generic QAPI object.
> +
> + It's not used directly, but is instead subclassed by specific directives.
> + """
> +
> + # Inherit some standard options from Sphinx's ObjectDescription
> + option_spec: OptionSpec = ( # type:ignore[misc]
> + ObjectDescription.option_spec.copy()
> + )
> + option_spec.update(
> + {
> + # Borrowed from the Python domain:
This is line 125 mentioned above.
> + "module": directives.unchanged, # Override contextual module name
> + }
> + )
[...]
^ permalink raw reply [flat|nested] 129+ messages in thread* Re: [PATCH 11/57] docs/qapi-domain: add qapi:command directive
2025-03-07 6:33 ` Markus Armbruster
@ 2025-03-07 22:39 ` John Snow
0 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-07 22:39 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 3142 bytes --]
On Fri, Mar 7, 2025 at 1:34 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > This commit adds a stubbed version of QAPICommand that utilizes the
> > QAPIObject class, the qapi:command directive, the :qapi:cmd:
> > cross-reference role, and the "command" object type in the QAPI object
> > registry.
> >
> > This commit also adds the aforementioned generic QAPIObject class for
> > use in documenting various QAPI entities in the Sphinx ecosystem.
> >
> > They don't do anything *particularly* interesting yet, but that will
> > come in forthcoming commits.
> >
> > Note: some versions of mypy get a little confused over the difference
> > between class and instance variables; because sphinx's ObjectDescription
> > does not declare option_spec as a ClassVar (even though it's obvious
> > that it is), mypy may produce this error:
> >
> > qapi-domain.py:125: error: Cannot override instance variable (previously
> > declared on base class "ObjectDescription") with class variable [misc]
> >
> > I can't control that; so silence the error with a pragma.
>
> Is this still accurate? qapi-domain.py line 125 is a comment. I can't
> see the pragma.
>
> > Signed-off-by: John Snow <jsnow@redhat.com>
> > ---
> > docs/sphinx/qapi_domain.py | 146 ++++++++++++++++++++++++++++++++++++-
> > 1 file changed, 144 insertions(+), 2 deletions(-)
> >
> > diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
> > index 104bae709f3..6168c23936f 100644
> > --- a/docs/sphinx/qapi_domain.py
> > +++ b/docs/sphinx/qapi_domain.py
> > @@ -21,9 +21,10 @@
> > from docutils import nodes
> > from docutils.parsers.rst import directives
> >
> > -from compat import nested_parse
> > +from compat import KeywordNode, SpaceNode, nested_parse
> > from sphinx import addnodes
> > -from sphinx.addnodes import pending_xref
> > +from sphinx.addnodes import desc_signature, pending_xref
> > +from sphinx.directives import ObjectDescription
> > from sphinx.domains import (
> > Domain,
> > Index,
> > @@ -103,6 +104,144 @@ def process_link(
> > return title, target
> >
> >
> > +# Alias for the return of handle_signature(), which is used in several
> places.
> > +# (In the Python domain, this is Tuple[str, str] instead.)
> > +Signature = str
> > +
> > +
> > +class QAPIObject(ObjectDescription[Signature]):
> > + """
> > + Description of a generic QAPI object.
> > +
> > + It's not used directly, but is instead subclassed by specific
> directives.
> > + """
> > +
> > + # Inherit some standard options from Sphinx's ObjectDescription
> > + option_spec: OptionSpec = ( # type:ignore[misc]
>
Originally, that pointed here.
> > + ObjectDescription.option_spec.copy()
> > + )
> > + option_spec.update(
> > + {
> > + # Borrowed from the Python domain:
>
> This is line 125 mentioned above.
>
Slightly stale.
>
> > + "module": directives.unchanged, # Override contextual
> module name
> > + }
> > + )
>
> [...]
>
>
[-- Attachment #2: Type: text/html, Size: 4433 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 12/57] docs/qapi-domain: add :since: directive option
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (10 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 11/57] docs/qapi-domain: add qapi:command directive John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-07 6:59 ` Markus Armbruster
2025-03-05 3:45 ` [PATCH 13/57] docs/qapi-domain: add "Arguments:" field lists John Snow
` (47 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Add a little special markup for registering "Since:" information. Adding
it as an option instead of generic content lets us hoist the information
into the Signature bar, optionally put it in the index, etc.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapi_domain.py | 29 +++++++++++++++++++++++++++--
1 file changed, 27 insertions(+), 2 deletions(-)
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index 6168c23936f..9919dacd4e6 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -4,6 +4,7 @@
from __future__ import annotations
+import re
from typing import (
TYPE_CHECKING,
AbstractSet,
@@ -104,6 +105,18 @@ def process_link(
return title, target
+def since_validator(param: str) -> str:
+ """
+ Validate the `:since: X.Y` option field.
+ """
+ match = re.match(r"[0-9]+\.[0-9]+", param)
+ if not match:
+ raise ValueError(
+ f":since: requires a version number in X.Y format; not {param!r}"
+ )
+ return param
+
+
# Alias for the return of handle_signature(), which is used in several places.
# (In the Python domain, this is Tuple[str, str] instead.)
Signature = str
@@ -124,6 +137,8 @@ class QAPIObject(ObjectDescription[Signature]):
{
# Borrowed from the Python domain:
"module": directives.unchanged, # Override contextual module name
+ # These are QAPI originals:
+ "since": since_validator,
}
)
@@ -135,9 +150,19 @@ def get_signature_prefix(self) -> List[nodes.Node]:
SpaceNode(" "),
]
- def get_signature_suffix(self) -> list[nodes.Node]:
+ def get_signature_suffix(self) -> List[nodes.Node]:
"""Returns a suffix to put after the object name in the signature."""
- return []
+ ret: List[nodes.Node] = []
+
+ if "since" in self.options:
+ ret += [
+ SpaceNode(" "),
+ addnodes.desc_sig_element(
+ "", f"(Since: {self.options['since']})"
+ ),
+ ]
+
+ return ret
def handle_signature(self, sig: str, signode: desc_signature) -> Signature:
"""
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 12/57] docs/qapi-domain: add :since: directive option
2025-03-05 3:45 ` [PATCH 12/57] docs/qapi-domain: add :since: directive option John Snow
@ 2025-03-07 6:59 ` Markus Armbruster
2025-03-07 22:43 ` John Snow
0 siblings, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-07 6:59 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> Add a little special markup for registering "Since:" information. Adding
> it as an option instead of generic content lets us hoist the information
> into the Signature bar, optionally put it in the index, etc.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> docs/sphinx/qapi_domain.py | 29 +++++++++++++++++++++++++++--
> 1 file changed, 27 insertions(+), 2 deletions(-)
>
> diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
> index 6168c23936f..9919dacd4e6 100644
> --- a/docs/sphinx/qapi_domain.py
> +++ b/docs/sphinx/qapi_domain.py
> @@ -4,6 +4,7 @@
>
> from __future__ import annotations
>
> +import re
> from typing import (
> TYPE_CHECKING,
> AbstractSet,
> @@ -104,6 +105,18 @@ def process_link(
> return title, target
>
>
> +def since_validator(param: str) -> str:
> + """
> + Validate the `:since: X.Y` option field.
> + """
> + match = re.match(r"[0-9]+\.[0-9]+", param)
This accepts arbitrary crap after the version. Example:
"9.2.50v9.2.0-2253-ge8a0110293" is fine. Intentional?
> + if not match:
> + raise ValueError(
> + f":since: requires a version number in X.Y format; not {param!r}"
> + )
> + return param
Schema validation is the frontend's job. Ideally, a backend doesn't
report any errors. The backends generating C don't. A backend
generating docs has to: all the reST processing happens there, and
therefore reST errors can only be diagnosed there. Since "no errors"
purity is impossible for this backend, we can be pragmatic about sinning
a bit more.
Still, I think this one should rather go into the doc comment parser.
This is not a demand. We can always clean it up later.
> +
> +
> # Alias for the return of handle_signature(), which is used in several places.
> # (In the Python domain, this is Tuple[str, str] instead.)
> Signature = str
> @@ -124,6 +137,8 @@ class QAPIObject(ObjectDescription[Signature]):
> {
> # Borrowed from the Python domain:
> "module": directives.unchanged, # Override contextual module name
> + # These are QAPI originals:
> + "since": since_validator,
> }
> )
>
> @@ -135,9 +150,19 @@ def get_signature_prefix(self) -> List[nodes.Node]:
> SpaceNode(" "),
> ]
>
> - def get_signature_suffix(self) -> list[nodes.Node]:
> + def get_signature_suffix(self) -> List[nodes.Node]:
> """Returns a suffix to put after the object name in the signature."""
> - return []
> + ret: List[nodes.Node] = []
> +
> + if "since" in self.options:
> + ret += [
> + SpaceNode(" "),
> + addnodes.desc_sig_element(
> + "", f"(Since: {self.options['since']})"
> + ),
> + ]
> +
> + return ret
>
> def handle_signature(self, sig: str, signode: desc_signature) -> Signature:
> """
^ permalink raw reply [flat|nested] 129+ messages in thread* Re: [PATCH 12/57] docs/qapi-domain: add :since: directive option
2025-03-07 6:59 ` Markus Armbruster
@ 2025-03-07 22:43 ` John Snow
0 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-07 22:43 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 3673 bytes --]
On Fri, Mar 7, 2025 at 1:59 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > Add a little special markup for registering "Since:" information. Adding
> > it as an option instead of generic content lets us hoist the information
> > into the Signature bar, optionally put it in the index, etc.
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
> > ---
> > docs/sphinx/qapi_domain.py | 29 +++++++++++++++++++++++++++--
> > 1 file changed, 27 insertions(+), 2 deletions(-)
> >
> > diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
> > index 6168c23936f..9919dacd4e6 100644
> > --- a/docs/sphinx/qapi_domain.py
> > +++ b/docs/sphinx/qapi_domain.py
> > @@ -4,6 +4,7 @@
> >
> > from __future__ import annotations
> >
> > +import re
> > from typing import (
> > TYPE_CHECKING,
> > AbstractSet,
> > @@ -104,6 +105,18 @@ def process_link(
> > return title, target
> >
> >
> > +def since_validator(param: str) -> str:
> > + """
> > + Validate the `:since: X.Y` option field.
> > + """
> > + match = re.match(r"[0-9]+\.[0-9]+", param)
>
> This accepts arbitrary crap after the version. Example:
> "9.2.50v9.2.0-2253-ge8a0110293" is fine. Intentional?
>
Nope! O:-) I forgot that match doesn't imply ^...$
>
> > + if not match:
> > + raise ValueError(
> > + f":since: requires a version number in X.Y format; not
> {param!r}"
> > + )
> > + return param
>
> Schema validation is the frontend's job. Ideally, a backend doesn't
> report any errors. The backends generating C don't. A backend
> generating docs has to: all the reST processing happens there, and
> therefore reST errors can only be diagnosed there. Since "no errors"
> purity is impossible for this backend, we can be pragmatic about sinning
> a bit more.
>
> Still, I think this one should rather go into the doc comment parser.
>
> This is not a demand. We can always clean it up later.
>
You *can* technically use this without touching the QAPI parser at all,
nothing stops you. I.e., you *could* write a QMP reference manual by hand
into an .rst if you wanted.
That said, I know we probably won't. I can remove the validator.
>
> > +
> > +
> > # Alias for the return of handle_signature(), which is used in several
> places.
> > # (In the Python domain, this is Tuple[str, str] instead.)
> > Signature = str
> > @@ -124,6 +137,8 @@ class QAPIObject(ObjectDescription[Signature]):
> > {
> > # Borrowed from the Python domain:
> > "module": directives.unchanged, # Override contextual
> module name
> > + # These are QAPI originals:
> > + "since": since_validator,
> > }
> > )
> >
> > @@ -135,9 +150,19 @@ def get_signature_prefix(self) -> List[nodes.Node]:
> > SpaceNode(" "),
> > ]
> >
> > - def get_signature_suffix(self) -> list[nodes.Node]:
> > + def get_signature_suffix(self) -> List[nodes.Node]:
> > """Returns a suffix to put after the object name in the
> signature."""
> > - return []
> > + ret: List[nodes.Node] = []
> > +
> > + if "since" in self.options:
> > + ret += [
> > + SpaceNode(" "),
> > + addnodes.desc_sig_element(
> > + "", f"(Since: {self.options['since']})"
> > + ),
> > + ]
> > +
> > + return ret
> >
> > def handle_signature(self, sig: str, signode: desc_signature) ->
> Signature:
> > """
>
>
[-- Attachment #2: Type: text/html, Size: 5285 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 13/57] docs/qapi-domain: add "Arguments:" field lists
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (11 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 12/57] docs/qapi-domain: add :since: directive option John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-07 7:46 ` Markus Armbruster
2025-03-05 3:45 ` [PATCH 14/57] docs/qapi-domain: add "Features:" " John Snow
` (46 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
This adds special rendering for Sphinx's typed field lists.
This patch does not add any QAPI-aware markup, rendering, or
cross-referencing for the type names, yet. That feature requires a
subclass to TypedField which will happen in its own commit quite a bit
later in this series; after all the basic fields and objects have been
established first.
The syntax for this field is:
:arg type name: description
description cont'd
You can omit the type or the description, but you cannot omit the name
-- if you do so, it degenerates into a "normal field list" entry, and
probably isn't what you want.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapi_domain.py | 14 +++++++++++++-
1 file changed, 13 insertions(+), 1 deletion(-)
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index 9919dacd4e6..64b540ff940 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -35,6 +35,7 @@
from sphinx.locale import _, __
from sphinx.roles import XRefRole
from sphinx.util import logging
+from sphinx.util.docfields import TypedField
from sphinx.util.docutils import SphinxDirective
from sphinx.util.nodes import make_id, make_refnode
@@ -264,7 +265,18 @@ def _toc_entry_name(self, sig_node: desc_signature) -> str:
class QAPICommand(QAPIObject):
"""Description of a QAPI Command."""
- # Nothing unique for now! Changed in later commits O:-)
+ doc_field_types = QAPIObject.doc_field_types.copy()
+ doc_field_types.extend(
+ [
+ # :arg TypeName ArgName: descr
+ TypedField(
+ "argument",
+ label=_("Arguments"),
+ names=("arg",),
+ can_collapse=False,
+ ),
+ ]
+ )
class QAPIModule(SphinxDirective):
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 13/57] docs/qapi-domain: add "Arguments:" field lists
2025-03-05 3:45 ` [PATCH 13/57] docs/qapi-domain: add "Arguments:" field lists John Snow
@ 2025-03-07 7:46 ` Markus Armbruster
2025-03-07 22:48 ` John Snow
0 siblings, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-07 7:46 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> This adds special rendering for Sphinx's typed field lists.
>
> This patch does not add any QAPI-aware markup, rendering, or
> cross-referencing for the type names, yet. That feature requires a
> subclass to TypedField which will happen in its own commit quite a bit
> later in this series; after all the basic fields and objects have been
> established first.
>
> The syntax for this field is:
>
> :arg type name: description
> description cont'd
>
> You can omit the type or the description, but you cannot omit the name
> -- if you do so, it degenerates into a "normal field list" entry, and
> probably isn't what you want.
Exuse my nitpicking... "cannot omit" suggests omission is a hard error.
The text after "--" suggests it isn't, it gives you something that
"probably isn't what you want". Which way does it actually behave?
>
> Signed-off-by: John Snow <jsnow@redhat.com>
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 13/57] docs/qapi-domain: add "Arguments:" field lists
2025-03-07 7:46 ` Markus Armbruster
@ 2025-03-07 22:48 ` John Snow
0 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-07 22:48 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 1492 bytes --]
On Fri, Mar 7, 2025 at 2:46 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > This adds special rendering for Sphinx's typed field lists.
> >
> > This patch does not add any QAPI-aware markup, rendering, or
> > cross-referencing for the type names, yet. That feature requires a
> > subclass to TypedField which will happen in its own commit quite a bit
> > later in this series; after all the basic fields and objects have been
> > established first.
> >
> > The syntax for this field is:
> >
> > :arg type name: description
> > description cont'd
> >
> > You can omit the type or the description, but you cannot omit the name
> > -- if you do so, it degenerates into a "normal field list" entry, and
> > probably isn't what you want.
>
> Exuse my nitpicking... "cannot omit" suggests omission is a hard error.
> The text after "--" suggests it isn't, it gives you something that
> "probably isn't what you want". Which way does it actually behave?
>
Haha.
If you omit the name, the info field list entry degrades into a
rst-standard field list entry. Later in the series, I explicitly prohibit
that because it is definitely not what you want at all.
info field list = special processing is applied to group entries, labels
are added, add xrefs, etc
field list = regular definition list exactly as typed. I.e. the label will
be "arg:"
>
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
>
>
[-- Attachment #2: Type: text/html, Size: 2328 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 14/57] docs/qapi-domain: add "Features:" field lists
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (12 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 13/57] docs/qapi-domain: add "Arguments:" field lists John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 15/57] docs/qapi-domain: add "Errors:" " John Snow
` (45 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Add support for Features field lists. There is no QAPI-specific
functionality here, but this could be changed if desired (if we wanted
the feature names to link somewhere, for instance.)
This feature list doesn't have any restrictions, so it can be used to
document object-wide features or per-member features as deemed
appropriate. It's essentially free-form text.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapi_domain.py | 12 +++++++++++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index 64b540ff940..e1b766d9481 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -35,7 +35,7 @@
from sphinx.locale import _, __
from sphinx.roles import XRefRole
from sphinx.util import logging
-from sphinx.util.docfields import TypedField
+from sphinx.util.docfields import GroupedField, TypedField
from sphinx.util.docutils import SphinxDirective
from sphinx.util.nodes import make_id, make_refnode
@@ -143,6 +143,16 @@ class QAPIObject(ObjectDescription[Signature]):
}
)
+ doc_field_types = [
+ # :feat name: descr
+ GroupedField(
+ "feature",
+ label=_("Features"),
+ names=("feat",),
+ can_collapse=False,
+ ),
+ ]
+
def get_signature_prefix(self) -> List[nodes.Node]:
"""Returns a prefix to put before the object name in the signature."""
assert self.objtype
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 15/57] docs/qapi-domain: add "Errors:" field lists
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (13 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 14/57] docs/qapi-domain: add "Features:" " John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-07 7:48 ` Markus Armbruster
2025-03-05 3:45 ` [PATCH 16/57] docs/qapi-domain: add "Returns:" " John Snow
` (44 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
``:error: descr`` can now be used to document error conditions. The
format of the description is not defined here; so the ability to name
specific types is left to the document writer.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapi_domain.py | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index e1b766d9481..d60cccb8e95 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -35,7 +35,7 @@
from sphinx.locale import _, __
from sphinx.roles import XRefRole
from sphinx.util import logging
-from sphinx.util.docfields import GroupedField, TypedField
+from sphinx.util.docfields import Field, GroupedField, TypedField
from sphinx.util.docutils import SphinxDirective
from sphinx.util.nodes import make_id, make_refnode
@@ -285,6 +285,13 @@ class QAPICommand(QAPIObject):
names=("arg",),
can_collapse=False,
),
+ # :error: descr
+ Field(
+ "error",
+ label=_("Errors"),
+ names=("error", "errors"),
+ has_arg=False,
+ ),
]
)
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 15/57] docs/qapi-domain: add "Errors:" field lists
2025-03-05 3:45 ` [PATCH 15/57] docs/qapi-domain: add "Errors:" " John Snow
@ 2025-03-07 7:48 ` Markus Armbruster
2025-03-07 22:50 ` John Snow
0 siblings, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-07 7:48 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> ``:error: descr`` can now be used to document error conditions. The
> format of the description is not defined here; so the ability to name
> specific types is left to the document writer.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
By convention, the description is a bullet list, but we don't enforce
that in software. Observation, not a suggestion to start enforcing it
now.
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 15/57] docs/qapi-domain: add "Errors:" field lists
2025-03-07 7:48 ` Markus Armbruster
@ 2025-03-07 22:50 ` John Snow
2025-03-08 6:49 ` Markus Armbruster
0 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-07 22:50 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 824 bytes --]
On Fri, Mar 7, 2025 at 2:48 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > ``:error: descr`` can now be used to document error conditions. The
> > format of the description is not defined here; so the ability to name
> > specific types is left to the document writer.
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
>
> By convention, the description is a bullet list, but we don't enforce
> that in software. Observation, not a suggestion to start enforcing it
> now.
>
Right. This commit message is accurate, though: this field list type will
accept any old thing. I have an idea to fix this to make the aesthetics
better (fixes the misalignment) and to enforce the bulleted list in one
shot. Not for this series, sorry. Please accept a raincheck.
[-- Attachment #2: Type: text/html, Size: 1339 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 15/57] docs/qapi-domain: add "Errors:" field lists
2025-03-07 22:50 ` John Snow
@ 2025-03-08 6:49 ` Markus Armbruster
0 siblings, 0 replies; 129+ messages in thread
From: Markus Armbruster @ 2025-03-08 6:49 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> On Fri, Mar 7, 2025 at 2:48 AM Markus Armbruster <armbru@redhat.com> wrote:
>
>> John Snow <jsnow@redhat.com> writes:
>>
>> > ``:error: descr`` can now be used to document error conditions. The
>> > format of the description is not defined here; so the ability to name
>> > specific types is left to the document writer.
>> >
>> > Signed-off-by: John Snow <jsnow@redhat.com>
>>
>> By convention, the description is a bullet list, but we don't enforce
>> that in software. Observation, not a suggestion to start enforcing it
>> now.
>>
>
> Right. This commit message is accurate, though: this field list type will
> accept any old thing. I have an idea to fix this to make the aesthetics
> better (fixes the misalignment) and to enforce the bulleted list in one
> shot. Not for this series, sorry. Please accept a raincheck.
You're quite right to resist mission creep.
The commit message could mention the convention, say "The format of the
description is not defined here (we make it a bulleted list, but that's
just convention); ..." Up to you.
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 16/57] docs/qapi-domain: add "Returns:" field lists
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (14 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 15/57] docs/qapi-domain: add "Errors:" " John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-07 7:58 ` Markus Armbruster
2025-03-05 3:45 ` [PATCH 17/57] docs/qapi-domain: add qapi:enum directive John Snow
` (43 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Add "Returns:" field list syntax to QAPI Commands.
Like "Arguments:" and "Errors:", the type name isn't currently processed
for cross-referencing, but this will be addressed in a forthcoming
commit.
This patch adds "errors" as a GroupedField, which means that multiple
return values can be annotated - this is only done because Sphinx does
not seemingly (Maybe I missed it?) support mandatory type arguments to
Ungrouped fields. Because we want to cross-reference this type
information later, we want to make the type argument mandatory. As a
result, you can technically add multiple :return: fields, though I'm not
aware of any circumstance in which you'd need or want
to. Recommendation: "Don't do that, then."
Since this field describes an action/event instead of describing a list
of nouns (arguments, features, errors), I added both the imperative and
indicative forms (:return: and :returns:) to allow doc writers to use
whichever mood "feels right" in the source document. The rendered output
will always use the "Returns:" label, however.
I'm sure you'll let me know how you feel about that. O:-)
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapi_domain.py | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index d60cccb8e95..7531bdfbba7 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -292,6 +292,13 @@ class QAPICommand(QAPIObject):
names=("error", "errors"),
has_arg=False,
),
+ # :returns TypeName: descr
+ GroupedField(
+ "returnvalue",
+ label=_("Returns"),
+ names=("return", "returns"),
+ can_collapse=True,
+ ),
]
)
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 16/57] docs/qapi-domain: add "Returns:" field lists
2025-03-05 3:45 ` [PATCH 16/57] docs/qapi-domain: add "Returns:" " John Snow
@ 2025-03-07 7:58 ` Markus Armbruster
2025-03-07 22:58 ` John Snow
0 siblings, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-07 7:58 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> Add "Returns:" field list syntax to QAPI Commands.
>
> Like "Arguments:" and "Errors:", the type name isn't currently processed
> for cross-referencing, but this will be addressed in a forthcoming
> commit.
>
> This patch adds "errors" as a GroupedField, which means that multiple
"errors"?
> return values can be annotated - this is only done because Sphinx does
> not seemingly (Maybe I missed it?) support mandatory type arguments to
> Ungrouped fields. Because we want to cross-reference this type
> information later, we want to make the type argument mandatory. As a
> result, you can technically add multiple :return: fields, though I'm not
> aware of any circumstance in which you'd need or want
> to. Recommendation: "Don't do that, then."
scripts/qapi/parser.py rejects duplicate 'Returns:' tags. So, to do the
thing you shouldn't do, you'd have to use the QAPI domain directly.
I doubt such shenanigans would survive review :)
> Since this field describes an action/event instead of describing a list
> of nouns (arguments, features, errors), I added both the imperative and
> indicative forms (:return: and :returns:) to allow doc writers to use
> whichever mood "feels right" in the source document. The rendered output
> will always use the "Returns:" label, however.
>
> I'm sure you'll let me know how you feel about that. O:-)
My personal taste is imperative mood, always.
Sadly, the QAPI schema language uses 'Returns:'.
The Sphinx Python Domain appears to use :return:.
I recommend to go for consistency with the Python Domain, and ditch
:returns:.
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> docs/sphinx/qapi_domain.py | 7 +++++++
> 1 file changed, 7 insertions(+)
>
> diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
> index d60cccb8e95..7531bdfbba7 100644
> --- a/docs/sphinx/qapi_domain.py
> +++ b/docs/sphinx/qapi_domain.py
> @@ -292,6 +292,13 @@ class QAPICommand(QAPIObject):
> names=("error", "errors"),
> has_arg=False,
> ),
> + # :returns TypeName: descr
> + GroupedField(
> + "returnvalue",
> + label=_("Returns"),
> + names=("return", "returns"),
> + can_collapse=True,
> + ),
> ]
> )
^ permalink raw reply [flat|nested] 129+ messages in thread* Re: [PATCH 16/57] docs/qapi-domain: add "Returns:" field lists
2025-03-07 7:58 ` Markus Armbruster
@ 2025-03-07 22:58 ` John Snow
2025-03-08 6:57 ` Markus Armbruster
0 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-07 22:58 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 3124 bytes --]
On Fri, Mar 7, 2025 at 2:58 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > Add "Returns:" field list syntax to QAPI Commands.
> >
> > Like "Arguments:" and "Errors:", the type name isn't currently processed
> > for cross-referencing, but this will be addressed in a forthcoming
> > commit.
> >
> > This patch adds "errors" as a GroupedField, which means that multiple
>
> "errors"?
>
Copy-pasto :)
>
> > return values can be annotated - this is only done because Sphinx does
> > not seemingly (Maybe I missed it?) support mandatory type arguments to
> > Ungrouped fields. Because we want to cross-reference this type
> > information later, we want to make the type argument mandatory. As a
> > result, you can technically add multiple :return: fields, though I'm not
> > aware of any circumstance in which you'd need or want
> > to. Recommendation: "Don't do that, then."
>
> scripts/qapi/parser.py rejects duplicate 'Returns:' tags. So, to do the
> thing you shouldn't do, you'd have to use the QAPI domain directly.
> I doubt such shenanigans would survive review :)
>
Sure, but it's a little weird to be in the headspace of writing a domain
extension that was based on one which *can* be used directly. I know we
won't, but I suppose I am still documenting it and treating it as if you
could.
More the case, it serves as reference if anyone wants to adjust the
behavior of the transmogrifier.
So, consider this documentation for me in the future, or whoever touches
qapidoc if I am felled by an errant spacerock.
>
> > Since this field describes an action/event instead of describing a list
> > of nouns (arguments, features, errors), I added both the imperative and
> > indicative forms (:return: and :returns:) to allow doc writers to use
> > whichever mood "feels right" in the source document. The rendered output
> > will always use the "Returns:" label, however.
> >
> > I'm sure you'll let me know how you feel about that. O:-)
>
> My personal taste is imperative mood, always.
>
> Sadly, the QAPI schema language uses 'Returns:'.
>
> The Sphinx Python Domain appears to use :return:.
>
> I recommend to go for consistency with the Python Domain, and ditch
> :returns:.
>
Done.
>
> > Signed-off-by: John Snow <jsnow@redhat.com>
> > ---
> > docs/sphinx/qapi_domain.py | 7 +++++++
> > 1 file changed, 7 insertions(+)
> >
> > diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
> > index d60cccb8e95..7531bdfbba7 100644
> > --- a/docs/sphinx/qapi_domain.py
> > +++ b/docs/sphinx/qapi_domain.py
> > @@ -292,6 +292,13 @@ class QAPICommand(QAPIObject):
> > names=("error", "errors"),
> > has_arg=False,
> > ),
> > + # :returns TypeName: descr
> > + GroupedField(
> > + "returnvalue",
> > + label=_("Returns"),
> > + names=("return", "returns"),
> > + can_collapse=True,
> > + ),
> > ]
> > )
>
>
[-- Attachment #2: Type: text/html, Size: 4640 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread* Re: [PATCH 16/57] docs/qapi-domain: add "Returns:" field lists
2025-03-07 22:58 ` John Snow
@ 2025-03-08 6:57 ` Markus Armbruster
0 siblings, 0 replies; 129+ messages in thread
From: Markus Armbruster @ 2025-03-08 6:57 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> On Fri, Mar 7, 2025 at 2:58 AM Markus Armbruster <armbru@redhat.com> wrote:
>
>> John Snow <jsnow@redhat.com> writes:
>>
>> > Add "Returns:" field list syntax to QAPI Commands.
>> >
>> > Like "Arguments:" and "Errors:", the type name isn't currently processed
>> > for cross-referencing, but this will be addressed in a forthcoming
>> > commit.
>> >
>> > This patch adds "errors" as a GroupedField, which means that multiple
>>
>> "errors"?
>>
>
> Copy-pasto :)
>
>
>>
>> > return values can be annotated - this is only done because Sphinx does
>> > not seemingly (Maybe I missed it?) support mandatory type arguments to
>> > Ungrouped fields. Because we want to cross-reference this type
>> > information later, we want to make the type argument mandatory. As a
>> > result, you can technically add multiple :return: fields, though I'm not
>> > aware of any circumstance in which you'd need or want
>> > to. Recommendation: "Don't do that, then."
>>
>> scripts/qapi/parser.py rejects duplicate 'Returns:' tags. So, to do the
>> thing you shouldn't do, you'd have to use the QAPI domain directly.
>> I doubt such shenanigans would survive review :)
>>
>
> Sure, but it's a little weird to be in the headspace of writing a domain
> extension that was based on one which *can* be used directly. I know we
> won't, but I suppose I am still documenting it and treating it as if you
> could.
Valid argument.
Our test suite only covers use via transmogrifier, not direct use.
Fixable. I'm not asking you to fix it now.
Commit message could mention the emerging new QAPI doc tool chain obeys
"Don't do that, then". But I figure you have bigger fish to fry.
> More the case, it serves as reference if anyone wants to adjust the
> behavior of the transmogrifier.
>
> So, consider this documentation for me in the future, or whoever touches
> qapidoc if I am felled by an errant spacerock.
>
>
>>
>> > Since this field describes an action/event instead of describing a list
>> > of nouns (arguments, features, errors), I added both the imperative and
>> > indicative forms (:return: and :returns:) to allow doc writers to use
>> > whichever mood "feels right" in the source document. The rendered output
>> > will always use the "Returns:" label, however.
>> >
>> > I'm sure you'll let me know how you feel about that. O:-)
>>
>> My personal taste is imperative mood, always.
>>
>> Sadly, the QAPI schema language uses 'Returns:'.
>>
>> The Sphinx Python Domain appears to use :return:.
>>
>> I recommend to go for consistency with the Python Domain, and ditch
>> :returns:.
>>
>
> Done.
Thanks! A few instrances of "returns" in later commit messages need
adjustment.
[...]
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 17/57] docs/qapi-domain: add qapi:enum directive
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (15 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 16/57] docs/qapi-domain: add "Returns:" " John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 18/57] docs/qapi-domain: add qapi:alternate directive John Snow
` (42 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Add the .. qapi:enum:: directive, object, and :qapi:enum:`name`
cross-reference role.
Add the :value name: field list for documenting Enum values.
Of note, also introduce a new "type" role that is intended to be used by
other QAPI object directives to cross-reference arbitrary QAPI type
names, but will exclude commands, events, and modules from
consideration.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapi_domain.py | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index 7531bdfbba7..66fdeec1d37 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -303,6 +303,23 @@ class QAPICommand(QAPIObject):
)
+class QAPIEnum(QAPIObject):
+ """Description of a QAPI Enum."""
+
+ doc_field_types = QAPIObject.doc_field_types.copy()
+ doc_field_types.extend(
+ [
+ # :value name: descr
+ GroupedField(
+ "value",
+ label=_("Values"),
+ names=("value",),
+ can_collapse=False,
+ )
+ ]
+ )
+
+
class QAPIModule(SphinxDirective):
"""
Directive to mark description of a new module.
@@ -458,9 +475,14 @@ class QAPIDomain(Domain):
# This table associates cross-reference object types (key) with an
# ObjType instance, which defines the valid cross-reference roles
# for each object type.
+ #
+ # e.g., the :qapi:type: cross-reference role can refer to enum,
+ # struct, union, or alternate objects; but :qapi:obj: can refer to
+ # anything. Each object also gets its own targeted cross-reference role.
object_types: Dict[str, ObjType] = {
"module": ObjType(_("module"), "mod", "any"),
"command": ObjType(_("command"), "cmd", "any"),
+ "enum": ObjType(_("enum"), "enum", "type", "any"),
}
# Each of these provides a rST directive,
@@ -468,6 +490,7 @@ class QAPIDomain(Domain):
directives = {
"module": QAPIModule,
"command": QAPICommand,
+ "enum": QAPIEnum,
}
# These are all cross-reference roles; e.g.
@@ -476,6 +499,9 @@ class QAPIDomain(Domain):
roles = {
"mod": QAPIXRefRole(),
"cmd": QAPIXRefRole(),
+ "enum": QAPIXRefRole(),
+ # reference any data type (excludes modules, commands, events)
+ "type": QAPIXRefRole(),
"any": QAPIXRefRole(), # reference *any* type of QAPI object.
}
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 18/57] docs/qapi-domain: add qapi:alternate directive
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (16 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 17/57] docs/qapi-domain: add qapi:enum directive John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-07 10:18 ` Markus Armbruster
2025-03-05 3:45 ` [PATCH 19/57] docs/qapi-domain: add qapi:event directive John Snow
` (41 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Add the .. qapi:alternate:: directive, object, and qapi:alt:`name`
cross-reference role.
Add the "Choices:" field list for describing alternate choices. Like
other field lists that reference QAPI types, a forthcoming commit will
add cross-referencing support to this field.
RFC: In the future, it would be nice to directly inline Alternates as
part of the type information in the containing object (i.e. directly in
arguments/members) - but that's a task for another series. For now, the
branch "names" are documented just like qapidoc.py does, even though
this information is superfluous for user documentation. Room for future
improvement, but not now.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapi_domain.py | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index 66fdeec1d37..d4120fa5ac6 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -320,6 +320,23 @@ class QAPIEnum(QAPIObject):
)
+class QAPIAlternate(QAPIObject):
+ """Description of a QAPI Alternate."""
+
+ doc_field_types = QAPIObject.doc_field_types.copy()
+ doc_field_types.extend(
+ [
+ # :choice type name: descr
+ TypedField(
+ "choice",
+ label=_("Choices"),
+ names=("choice",),
+ can_collapse=False,
+ ),
+ ]
+ )
+
+
class QAPIModule(SphinxDirective):
"""
Directive to mark description of a new module.
@@ -483,6 +500,7 @@ class QAPIDomain(Domain):
"module": ObjType(_("module"), "mod", "any"),
"command": ObjType(_("command"), "cmd", "any"),
"enum": ObjType(_("enum"), "enum", "type", "any"),
+ "alternate": ObjType(_("alternate"), "alt", "type", "any"),
}
# Each of these provides a rST directive,
@@ -491,6 +509,7 @@ class QAPIDomain(Domain):
"module": QAPIModule,
"command": QAPICommand,
"enum": QAPIEnum,
+ "alternate": QAPIAlternate,
}
# These are all cross-reference roles; e.g.
@@ -500,6 +519,7 @@ class QAPIDomain(Domain):
"mod": QAPIXRefRole(),
"cmd": QAPIXRefRole(),
"enum": QAPIXRefRole(),
+ "alt": QAPIXRefRole(),
# reference any data type (excludes modules, commands, events)
"type": QAPIXRefRole(),
"any": QAPIXRefRole(), # reference *any* type of QAPI object.
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 18/57] docs/qapi-domain: add qapi:alternate directive
2025-03-05 3:45 ` [PATCH 18/57] docs/qapi-domain: add qapi:alternate directive John Snow
@ 2025-03-07 10:18 ` Markus Armbruster
2025-03-07 23:02 ` John Snow
0 siblings, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-07 10:18 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> Add the .. qapi:alternate:: directive, object, and qapi:alt:`name`
> cross-reference role.
>
> Add the "Choices:" field list for describing alternate choices. Like
> other field lists that reference QAPI types, a forthcoming commit will
> add cross-referencing support to this field.
Nothing wrong with the term "choices" per se, but
docs/devel/qapi-code-gen.rst and the Python code call these things
"alternatives". I'd prefer consistency. Could be done as a follow-up
if that's more convenient for you.
> RFC: In the future, it would be nice to directly inline Alternates as
> part of the type information in the containing object (i.e. directly in
> arguments/members) - but that's a task for another series.
Does it make sense to talk about future inlining projects when we
haven't even gotten to the present one, yet?
> For now, the
> branch "names" are documented just like qapidoc.py does, even though
> this information is superfluous for user documentation. Room for future
> improvement, but not now.
Same as before. Good enough!
> Signed-off-by: John Snow <jsnow@redhat.com>
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 18/57] docs/qapi-domain: add qapi:alternate directive
2025-03-07 10:18 ` Markus Armbruster
@ 2025-03-07 23:02 ` John Snow
2025-03-08 6:58 ` Markus Armbruster
0 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-07 23:02 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 1443 bytes --]
On Fri, Mar 7, 2025 at 5:18 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > Add the .. qapi:alternate:: directive, object, and qapi:alt:`name`
> > cross-reference role.
> >
> > Add the "Choices:" field list for describing alternate choices. Like
> > other field lists that reference QAPI types, a forthcoming commit will
> > add cross-referencing support to this field.
>
> Nothing wrong with the term "choices" per se, but
> docs/devel/qapi-code-gen.rst and the Python code call these things
> "alternatives". I'd prefer consistency. Could be done as a follow-up
> if that's more convenient for you.
>
Done. (Wish you'd said so sooner, though.)
>
> > RFC: In the future, it would be nice to directly inline Alternates as
> > part of the type information in the containing object (i.e. directly in
> > arguments/members) - but that's a task for another series.
>
> Does it make sense to talk about future inlining projects when we
> haven't even gotten to the present one, yet?
>
Vestigial, byebye.
>
> > For now, the
> > branch "names" are documented just like qapidoc.py does, even though
> > this information is superfluous for user documentation. Room for future
> > improvement, but not now.
>
> Same as before. Good enough!
>
> > Signed-off-by: John Snow <jsnow@redhat.com>
>
>
[-- Attachment #2: Type: text/html, Size: 2411 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 18/57] docs/qapi-domain: add qapi:alternate directive
2025-03-07 23:02 ` John Snow
@ 2025-03-08 6:58 ` Markus Armbruster
0 siblings, 0 replies; 129+ messages in thread
From: Markus Armbruster @ 2025-03-08 6:58 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> On Fri, Mar 7, 2025 at 5:18 AM Markus Armbruster <armbru@redhat.com> wrote:
>
>> John Snow <jsnow@redhat.com> writes:
>>
>> > Add the .. qapi:alternate:: directive, object, and qapi:alt:`name`
>> > cross-reference role.
>> >
>> > Add the "Choices:" field list for describing alternate choices. Like
>> > other field lists that reference QAPI types, a forthcoming commit will
>> > add cross-referencing support to this field.
>>
>> Nothing wrong with the term "choices" per se, but
>> docs/devel/qapi-code-gen.rst and the Python code call these things
>> "alternatives". I'd prefer consistency. Could be done as a follow-up
>> if that's more convenient for you.
>>
>
> Done. (Wish you'd said so sooner, though.)
Fair!
[...]
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 19/57] docs/qapi-domain: add qapi:event directive
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (17 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 18/57] docs/qapi-domain: add qapi:alternate directive John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-07 10:26 ` Markus Armbruster
2025-03-05 3:45 ` [PATCH 20/57] docs/qapi-domain: add qapi:object directive John Snow
` (40 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Adds the .. qapi:event:: directive, object, and :qapi:event:`name`
cross-referencing role.
Adds the :memb type name: field list syntax for documenting event data
members. As this syntax and phrasing will be shared with Structs and
Unions as well, add the field list definition to a shared abstract
class.
As per usual, QAPI cross-referencing for types in the member field list
will be added in a forthcoming commit.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapi_domain.py | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index d4120fa5ac6..d9c69af229d 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -337,6 +337,27 @@ class QAPIAlternate(QAPIObject):
)
+class QAPIObjectWithMembers(QAPIObject):
+ """Base class for Events/Structs/Unions"""
+
+ doc_field_types = QAPIObject.doc_field_types.copy()
+ doc_field_types.extend(
+ [
+ # :member type name: descr
+ TypedField(
+ "member",
+ label=_("Members"),
+ names=("memb",),
+ can_collapse=False,
+ ),
+ ]
+ )
+
+
+class QAPIEvent(QAPIObjectWithMembers):
+ """Description of a QAPI Event."""
+
+
class QAPIModule(SphinxDirective):
"""
Directive to mark description of a new module.
@@ -499,6 +520,7 @@ class QAPIDomain(Domain):
object_types: Dict[str, ObjType] = {
"module": ObjType(_("module"), "mod", "any"),
"command": ObjType(_("command"), "cmd", "any"),
+ "event": ObjType(_("event"), "event", "any"),
"enum": ObjType(_("enum"), "enum", "type", "any"),
"alternate": ObjType(_("alternate"), "alt", "type", "any"),
}
@@ -508,6 +530,7 @@ class QAPIDomain(Domain):
directives = {
"module": QAPIModule,
"command": QAPICommand,
+ "event": QAPIEvent,
"enum": QAPIEnum,
"alternate": QAPIAlternate,
}
@@ -518,6 +541,7 @@ class QAPIDomain(Domain):
roles = {
"mod": QAPIXRefRole(),
"cmd": QAPIXRefRole(),
+ "event": QAPIXRefRole(),
"enum": QAPIXRefRole(),
"alt": QAPIXRefRole(),
# reference any data type (excludes modules, commands, events)
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 19/57] docs/qapi-domain: add qapi:event directive
2025-03-05 3:45 ` [PATCH 19/57] docs/qapi-domain: add qapi:event directive John Snow
@ 2025-03-07 10:26 ` Markus Armbruster
2025-03-07 23:06 ` John Snow
0 siblings, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-07 10:26 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> Adds the .. qapi:event:: directive, object, and :qapi:event:`name`
> cross-referencing role.
>
> Adds the :memb type name: field list syntax for documenting event data
> members. As this syntax and phrasing will be shared with Structs and
> Unions as well, add the field list definition to a shared abstract
> class.
docs/devel/qapi-code-gen.rst calls it "event-specific data". This is
quite a mouthful, so the code usually calls it "arguments". Not least
because events are kind of like commands going in the other direction
(client to server), and the code dealing with them is often similar.
Both names make more sense to me than "member". Hmm. A rename could be
done as a follow-up if that's more convenient for you.
> As per usual, QAPI cross-referencing for types in the member field list
> will be added in a forthcoming commit.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 19/57] docs/qapi-domain: add qapi:event directive
2025-03-07 10:26 ` Markus Armbruster
@ 2025-03-07 23:06 ` John Snow
2025-03-08 7:00 ` Markus Armbruster
0 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-07 23:06 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 1383 bytes --]
On Fri, Mar 7, 2025 at 5:26 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > Adds the .. qapi:event:: directive, object, and :qapi:event:`name`
> > cross-referencing role.
> >
> > Adds the :memb type name: field list syntax for documenting event data
> > members. As this syntax and phrasing will be shared with Structs and
> > Unions as well, add the field list definition to a shared abstract
> > class.
>
> docs/devel/qapi-code-gen.rst calls it "event-specific data". This is
> quite a mouthful, so the code usually calls it "arguments". Not least
> because events are kind of like commands going in the other direction
> (client to server), and the code dealing with them is often similar.
>
> Both names make more sense to me than "member". Hmm. A rename could be
> done as a follow-up if that's more convenient for you.
>
Also wish you'd have said sooner :)
I might punt this one. At least, the next respin probably won't include
this.
I suppose I think about this in terms of "members of the event object."
Let's talk and fix later, changing the name outside of a gigantic series is
not difficult at all.
>
> > As per usual, QAPI cross-referencing for types in the member field list
> > will be added in a forthcoming commit.
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
>
>
[-- Attachment #2: Type: text/html, Size: 2171 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 19/57] docs/qapi-domain: add qapi:event directive
2025-03-07 23:06 ` John Snow
@ 2025-03-08 7:00 ` Markus Armbruster
0 siblings, 0 replies; 129+ messages in thread
From: Markus Armbruster @ 2025-03-08 7:00 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> On Fri, Mar 7, 2025 at 5:26 AM Markus Armbruster <armbru@redhat.com> wrote:
>
>> John Snow <jsnow@redhat.com> writes:
>>
>> > Adds the .. qapi:event:: directive, object, and :qapi:event:`name`
>> > cross-referencing role.
>> >
>> > Adds the :memb type name: field list syntax for documenting event data
>> > members. As this syntax and phrasing will be shared with Structs and
>> > Unions as well, add the field list definition to a shared abstract
>> > class.
>>
>> docs/devel/qapi-code-gen.rst calls it "event-specific data". This is
>> quite a mouthful, so the code usually calls it "arguments". Not least
>> because events are kind of like commands going in the other direction
>> (client to server), and the code dealing with them is often similar.
>>
>> Both names make more sense to me than "member". Hmm. A rename could be
>> done as a follow-up if that's more convenient for you.
>>
>
> Also wish you'd have said sooner :)
Fair again!
> I might punt this one. At least, the next respin probably won't include
> this.
>
> I suppose I think about this in terms of "members of the event object."
> Let's talk and fix later, changing the name outside of a gigantic series is
> not difficult at all.
Agree.
[...]
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 20/57] docs/qapi-domain: add qapi:object directive
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (18 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 19/57] docs/qapi-domain: add qapi:event directive John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 21/57] docs/qapi-domain: add :deprecated: directive option John Snow
` (39 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Adds the .. qapi:object:: directive, object, and :qapi:obj:`name`
cross-referencing role. This directive is meant to document both structs
and unions.
As per usual, QAPI cross-referencing for types in the member field list
will be added in a forthcoming commit.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapi_domain.py | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index d9c69af229d..fff5cca24cc 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -358,6 +358,10 @@ class QAPIEvent(QAPIObjectWithMembers):
"""Description of a QAPI Event."""
+class QAPIJSONObject(QAPIObjectWithMembers):
+ """Description of a QAPI Object: structs and unions."""
+
+
class QAPIModule(SphinxDirective):
"""
Directive to mark description of a new module.
@@ -522,6 +526,7 @@ class QAPIDomain(Domain):
"command": ObjType(_("command"), "cmd", "any"),
"event": ObjType(_("event"), "event", "any"),
"enum": ObjType(_("enum"), "enum", "type", "any"),
+ "object": ObjType(_("object"), "obj", "type", "any"),
"alternate": ObjType(_("alternate"), "alt", "type", "any"),
}
@@ -532,6 +537,7 @@ class QAPIDomain(Domain):
"command": QAPICommand,
"event": QAPIEvent,
"enum": QAPIEnum,
+ "object": QAPIJSONObject,
"alternate": QAPIAlternate,
}
@@ -543,6 +549,7 @@ class QAPIDomain(Domain):
"cmd": QAPIXRefRole(),
"event": QAPIXRefRole(),
"enum": QAPIXRefRole(),
+ "obj": QAPIXRefRole(), # specifically structs and unions.
"alt": QAPIXRefRole(),
# reference any data type (excludes modules, commands, events)
"type": QAPIXRefRole(),
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 21/57] docs/qapi-domain: add :deprecated: directive option
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (19 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 20/57] docs/qapi-domain: add qapi:object directive John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 9:13 ` Markus Armbruster
2025-03-07 10:47 ` Markus Armbruster
2025-03-05 3:45 ` [PATCH 22/57] docs/qapi-domain: add :unstable: " John Snow
` (38 subsequent siblings)
59 siblings, 2 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow, Harmonie Snow
Although "deprecated" is a feature (and *will* appear in the features
list), add a special :deprecated: option to generate an eye-catch that
makes this information very hard to miss.
(The intent is to modify qapidoc.py to add this option whenever it
detects that the features list attached to a definition contains the
"deprecated" entry.)
-
RFC: Technically, this object-level option is un-needed and could be
replaced with a standard content-level directive that e.g. qapidoc.py
could insert at the beginning of the content block. I've done it here as
an option to demonstrate how it would be possible to do.
It's a matter of taste for "where" we feel like implementing it.
One benefit of doing it this way is that we can create a single
containing box to set CSS style options controlling the flow of multiple
infoboxes. The other way to achieve that would be to create a directive
that allows us to set multiple options instead, e.g.:
.. qapi:infoboxes:: deprecated unstable
or possibly:
.. qapi:infoboxes::
:deprecated:
:unstable:
For now, I've left these as top-level QAPI object options. "Hey, it works."
P.S., I outsourced the CSS ;)
Signed-off-by: Harmonie Snow <harmonie@gmail.com>
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx-static/theme_overrides.css | 25 +++++++++++++++++++++++++
docs/sphinx/qapi_domain.py | 26 ++++++++++++++++++++++++++
2 files changed, 51 insertions(+)
diff --git a/docs/sphinx-static/theme_overrides.css b/docs/sphinx-static/theme_overrides.css
index 965ecac54fd..3765cab1b20 100644
--- a/docs/sphinx-static/theme_overrides.css
+++ b/docs/sphinx-static/theme_overrides.css
@@ -208,3 +208,28 @@ div[class^="highlight"] pre {
color: inherit;
}
}
+
+/* QAPI domain theming */
+
+.qapi-infopips {
+ margin-bottom: 1em;
+}
+
+.qapi-infopip {
+ display: inline-block;
+ padding: 0em 0.5em 0em 0.5em;
+ margin: 0.25em;
+}
+
+.qapi-deprecated {
+ background-color: #fffef5;
+ border: solid #fff176 6px;
+ font-weight: bold;
+ padding: 8px;
+ border-radius: 15px;
+ margin: 5px;
+}
+
+.qapi-deprecated::before {
+ content: '⚠️ ';
+}
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index fff5cca24cc..1be59e36bdf 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -140,6 +140,7 @@ class QAPIObject(ObjectDescription[Signature]):
"module": directives.unchanged, # Override contextual module name
# These are QAPI originals:
"since": since_validator,
+ "deprecated": directives.flag,
}
)
@@ -253,6 +254,31 @@ def add_target_and_index(
("single", indextext, node_id, "", None)
)
+ def _add_infopips(self, contentnode: addnodes.desc_content) -> None:
+ # Add various eye-catches and things that go below the signature
+ # bar, but precede the user-defined content.
+ infopips = nodes.container()
+ infopips.attributes["classes"].append("qapi-infopips")
+
+ def _add_pip(source: str, content: str, classname: str) -> None:
+ node = nodes.container(source)
+ node.append(nodes.Text(content))
+ node.attributes["classes"].extend(["qapi-infopip", classname])
+ infopips.append(node)
+
+ if "deprecated" in self.options:
+ _add_pip(
+ ":deprecated:",
+ f"This {self.objtype} is deprecated.",
+ "qapi-deprecated",
+ )
+
+ if infopips.children:
+ contentnode.insert(0, infopips)
+
+ def transform_content(self, content_node: addnodes.desc_content) -> None:
+ self._add_infopips(content_node)
+
def _toc_entry_name(self, sig_node: desc_signature) -> str:
# This controls the name in the TOC and on the sidebar.
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 21/57] docs/qapi-domain: add :deprecated: directive option
2025-03-05 3:45 ` [PATCH 21/57] docs/qapi-domain: add :deprecated: directive option John Snow
@ 2025-03-05 9:13 ` Markus Armbruster
2025-03-05 15:34 ` John Snow
2025-03-07 10:47 ` Markus Armbruster
1 sibling, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-05 9:13 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé, Harmonie Snow
John Snow <jsnow@redhat.com> writes:
> Although "deprecated" is a feature (and *will* appear in the features
> list), add a special :deprecated: option to generate an eye-catch that
> makes this information very hard to miss.
>
> (The intent is to modify qapidoc.py to add this option whenever it
> detects that the features list attached to a definition contains the
> "deprecated" entry.)
>
> -
>
> RFC: Technically, this object-level option is un-needed and could be
> replaced with a standard content-level directive that e.g. qapidoc.py
> could insert at the beginning of the content block. I've done it here as
> an option to demonstrate how it would be possible to do.
>
> It's a matter of taste for "where" we feel like implementing it.
>
> One benefit of doing it this way is that we can create a single
> containing box to set CSS style options controlling the flow of multiple
> infoboxes. The other way to achieve that would be to create a directive
> that allows us to set multiple options instead, e.g.:
>
> .. qapi:infoboxes:: deprecated unstable
>
> or possibly:
>
> .. qapi:infoboxes::
> :deprecated:
> :unstable:
>
> For now, I've left these as top-level QAPI object options. "Hey, it works."
>
> P.S., I outsourced the CSS ;)
>
> Signed-off-by: Harmonie Snow <harmonie@gmail.com>
> Signed-off-by: John Snow <jsnow@redhat.com>
[...]
> diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
> index fff5cca24cc..1be59e36bdf 100644
> --- a/docs/sphinx/qapi_domain.py
> +++ b/docs/sphinx/qapi_domain.py
> @@ -140,6 +140,7 @@ class QAPIObject(ObjectDescription[Signature]):
> "module": directives.unchanged, # Override contextual module name
> # These are QAPI originals:
> "since": since_validator,
> + "deprecated": directives.flag,
> }
> )
>
> @@ -253,6 +254,31 @@ def add_target_and_index(
> ("single", indextext, node_id, "", None)
> )
>
> + def _add_infopips(self, contentnode: addnodes.desc_content) -> None:
> + # Add various eye-catches and things that go below the signature
> + # bar, but precede the user-defined content.
> + infopips = nodes.container()
> + infopips.attributes["classes"].append("qapi-infopips")
> +
> + def _add_pip(source: str, content: str, classname: str) -> None:
> + node = nodes.container(source)
> + node.append(nodes.Text(content))
> + node.attributes["classes"].extend(["qapi-infopip", classname])
> + infopips.append(node)
> +
> + if "deprecated" in self.options:
> + _add_pip(
> + ":deprecated:",
> + f"This {self.objtype} is deprecated.",
> + "qapi-deprecated",
> + )
> +
> + if infopips.children:
> + contentnode.insert(0, infopips)
> +
> + def transform_content(self, content_node: addnodes.desc_content) -> None:
pylint warns:
docs/sphinx/qapi_domain.py:279:4: W0237: Parameter 'contentnode' has been renamed to 'content_node' in overriding 'QAPIObject.transform_content' method (arguments-renamed)
For what it's worth, @content_node is easier on on my eyes than
@contentnode.
> + self._add_infopips(content_node)
> +
> def _toc_entry_name(self, sig_node: desc_signature) -> str:
> # This controls the name in the TOC and on the sidebar.
^ permalink raw reply [flat|nested] 129+ messages in thread* Re: [PATCH 21/57] docs/qapi-domain: add :deprecated: directive option
2025-03-05 9:13 ` Markus Armbruster
@ 2025-03-05 15:34 ` John Snow
2025-03-06 7:22 ` Markus Armbruster
0 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 15:34 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé, Harmonie Snow
[-- Attachment #1: Type: text/plain, Size: 3989 bytes --]
On Wed, Mar 5, 2025, 4:13 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > Although "deprecated" is a feature (and *will* appear in the features
> > list), add a special :deprecated: option to generate an eye-catch that
> > makes this information very hard to miss.
> >
> > (The intent is to modify qapidoc.py to add this option whenever it
> > detects that the features list attached to a definition contains the
> > "deprecated" entry.)
> >
> > -
> >
> > RFC: Technically, this object-level option is un-needed and could be
> > replaced with a standard content-level directive that e.g. qapidoc.py
> > could insert at the beginning of the content block. I've done it here as
> > an option to demonstrate how it would be possible to do.
> >
> > It's a matter of taste for "where" we feel like implementing it.
> >
> > One benefit of doing it this way is that we can create a single
> > containing box to set CSS style options controlling the flow of multiple
> > infoboxes. The other way to achieve that would be to create a directive
> > that allows us to set multiple options instead, e.g.:
> >
> > .. qapi:infoboxes:: deprecated unstable
> >
> > or possibly:
> >
> > .. qapi:infoboxes::
> > :deprecated:
> > :unstable:
> >
> > For now, I've left these as top-level QAPI object options. "Hey, it
> works."
> >
> > P.S., I outsourced the CSS ;)
> >
> > Signed-off-by: Harmonie Snow <harmonie@gmail.com>
> > Signed-off-by: John Snow <jsnow@redhat.com>
>
> [...]
>
> > diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
> > index fff5cca24cc..1be59e36bdf 100644
> > --- a/docs/sphinx/qapi_domain.py
> > +++ b/docs/sphinx/qapi_domain.py
> > @@ -140,6 +140,7 @@ class QAPIObject(ObjectDescription[Signature]):
> > "module": directives.unchanged, # Override contextual
> module name
> > # These are QAPI originals:
> > "since": since_validator,
> > + "deprecated": directives.flag,
> > }
> > )
> >
> > @@ -253,6 +254,31 @@ def add_target_and_index(
> > ("single", indextext, node_id, "", None)
> > )
> >
> > + def _add_infopips(self, contentnode: addnodes.desc_content) -> None:
> > + # Add various eye-catches and things that go below the signature
> > + # bar, but precede the user-defined content.
> > + infopips = nodes.container()
> > + infopips.attributes["classes"].append("qapi-infopips")
> > +
> > + def _add_pip(source: str, content: str, classname: str) -> None:
> > + node = nodes.container(source)
> > + node.append(nodes.Text(content))
> > + node.attributes["classes"].extend(["qapi-infopip",
> classname])
> > + infopips.append(node)
> > +
> > + if "deprecated" in self.options:
> > + _add_pip(
> > + ":deprecated:",
> > + f"This {self.objtype} is deprecated.",
> > + "qapi-deprecated",
> > + )
> > +
> > + if infopips.children:
> > + contentnode.insert(0, infopips)
> > +
> > + def transform_content(self, content_node: addnodes.desc_content) ->
> None:
>
> pylint warns:
>
> docs/sphinx/qapi_domain.py:279:4: W0237: Parameter 'contentnode' has
> been renamed to 'content_node' in overriding 'QAPIObject.transform_content'
> method (arguments-renamed)
>
> For what it's worth, @content_node is easier on on my eyes than
> @contentnode.
>
Almost certifiably a Sphinx version difference that I simply won't be able
to accommodate. It comes back clean against 8.x, and does not impact the
runtime functionality at all.
> > + self._add_infopips(content_node)
> > +
> > def _toc_entry_name(self, sig_node: desc_signature) -> str:
> > # This controls the name in the TOC and on the sidebar.
>
>
[-- Attachment #2: Type: text/html, Size: 5586 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread* Re: [PATCH 21/57] docs/qapi-domain: add :deprecated: directive option
2025-03-05 15:34 ` John Snow
@ 2025-03-06 7:22 ` Markus Armbruster
0 siblings, 0 replies; 129+ messages in thread
From: Markus Armbruster @ 2025-03-06 7:22 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé, Harmonie Snow
John Snow <jsnow@redhat.com> writes:
> On Wed, Mar 5, 2025, 4:13 AM Markus Armbruster <armbru@redhat.com> wrote:
[...]
>> pylint warns:
>>
>> docs/sphinx/qapi_domain.py:279:4: W0237: Parameter 'contentnode' has
>> been renamed to 'content_node' in overriding 'QAPIObject.transform_content'
>> method (arguments-renamed)
>>
>> For what it's worth, @content_node is easier on on my eyes than
>> @contentnode.
>>
>
> Almost certifiably a Sphinx version difference that I simply won't be able
> to accommodate. It comes back clean against 8.x, and does not impact the
> runtime functionality at all.
If that's the case, don't worry about it.
[...]
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 21/57] docs/qapi-domain: add :deprecated: directive option
2025-03-05 3:45 ` [PATCH 21/57] docs/qapi-domain: add :deprecated: directive option John Snow
2025-03-05 9:13 ` Markus Armbruster
@ 2025-03-07 10:47 ` Markus Armbruster
2025-03-08 18:24 ` John Snow
1 sibling, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-07 10:47 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé, Markus Armbruster, Harmonie Snow
John Snow <jsnow@redhat.com> writes:
> Although "deprecated" is a feature (and *will* appear in the features
> list), add a special :deprecated: option to generate an eye-catch that
> makes this information very hard to miss.
>
> (The intent is to modify qapidoc.py to add this option whenever it
> detects that the features list attached to a definition contains the
> "deprecated" entry.)
Let me explain it in my own words to make sure I understand.
1. The transmogrifier emits a :feat: for the feature like for any other.
Therefore, the feature is rendered like any other.,
2. The transmogrifier additionally emits the owning directive with a
:deprecated: option. This gets the eye-catch rendered.
Example:
Command drive-backup (Since: 1.6)
--> This command is deprecated.
Start a point-in-time copy of a block device to a new destination.
The status of ongoing drive-backup operations can be checked with
query-block-jobs where the BlockJobInfo.type field has the value
'backup'. The operation can be stopped before it has completed
using the block-job-cancel command.
Arguments:
* The members of "DriveBackup".
Features:
--> * **deprecated** -- This command is deprecated. Use "blockdev-
backup" instead.
The first arrow marks the eye-catch. The second marks the normally
rendered feature.
The eye-catch is redundant with the feature rendering. Readers may
nevertheless find an eye-catch useful.
For what it's worth, the Python documentation has deprecation
information at the end of a definition documentation, together with
"changed in" information. Both are colored to catch the eye. More
restrained than your eye-catch. Also uses less space.
Hmm.
Not a blocker.
> -
>
> RFC: Technically, this object-level option is un-needed and could be
> replaced with a standard content-level directive that e.g. qapidoc.py
> could insert at the beginning of the content block. I've done it here as
> an option to demonstrate how it would be possible to do.
>
> It's a matter of taste for "where" we feel like implementing it.
>
> One benefit of doing it this way is that we can create a single
> containing box to set CSS style options controlling the flow of multiple
> infoboxes. The other way to achieve that would be to create a directive
> that allows us to set multiple options instead, e.g.:
>
> .. qapi:infoboxes:: deprecated unstable
>
> or possibly:
>
> .. qapi:infoboxes::
> :deprecated:
> :unstable:
>
> For now, I've left these as top-level QAPI object options. "Hey, it works."
Do you intend to drop this part in the final version?
Having the commit message explain paths not taken can be useful. But
this is phrased as an RFC, which suggests to me you plan to drop it.
> P.S., I outsourced the CSS ;)
Hi, Harmonie!
> Signed-off-by: Harmonie Snow <harmonie@gmail.com>
> Signed-off-by: John Snow <jsnow@redhat.com>
^ permalink raw reply [flat|nested] 129+ messages in thread* Re: [PATCH 21/57] docs/qapi-domain: add :deprecated: directive option
2025-03-07 10:47 ` Markus Armbruster
@ 2025-03-08 18:24 ` John Snow
0 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-08 18:24 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé, Harmonie Snow
[-- Attachment #1: Type: text/plain, Size: 3573 bytes --]
On Fri, Mar 7, 2025 at 5:48 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > Although "deprecated" is a feature (and *will* appear in the features
> > list), add a special :deprecated: option to generate an eye-catch that
> > makes this information very hard to miss.
> >
> > (The intent is to modify qapidoc.py to add this option whenever it
> > detects that the features list attached to a definition contains the
> > "deprecated" entry.)
>
> Let me explain it in my own words to make sure I understand.
>
> 1. The transmogrifier emits a :feat: for the feature like for any other.
> Therefore, the feature is rendered like any other.,
>
> 2. The transmogrifier additionally emits the owning directive with a
> :deprecated: option. This gets the eye-catch rendered.
>
> Example:
>
> Command drive-backup (Since: 1.6)
> --> This command is deprecated.
>
> Start a point-in-time copy of a block device to a new destination.
> The status of ongoing drive-backup operations can be checked with
> query-block-jobs where the BlockJobInfo.type field has the value
> 'backup'. The operation can be stopped before it has completed
> using the block-job-cancel command.
>
> Arguments:
> * The members of "DriveBackup".
>
> Features:
> --> * **deprecated** -- This command is deprecated. Use "blockdev-
> backup" instead.
>
> The first arrow marks the eye-catch. The second marks the normally
> rendered feature.
>
> The eye-catch is redundant with the feature rendering. Readers may
> nevertheless find an eye-catch useful.
>
Yep, you've got it.
>
> For what it's worth, the Python documentation has deprecation
> information at the end of a definition documentation, together with
> "changed in" information. Both are colored to catch the eye. More
> restrained than your eye-catch. Also uses less space.
>
> Hmm.
>
> Not a blocker.
>
I am extremely happy to take patches to change the layout and formatting
after we merge. I'm providing the canvas, others can paint :)
>
> > -
> >
> > RFC: Technically, this object-level option is un-needed and could be
> > replaced with a standard content-level directive that e.g. qapidoc.py
> > could insert at the beginning of the content block. I've done it here as
> > an option to demonstrate how it would be possible to do.
> >
> > It's a matter of taste for "where" we feel like implementing it.
> >
> > One benefit of doing it this way is that we can create a single
> > containing box to set CSS style options controlling the flow of multiple
> > infoboxes. The other way to achieve that would be to create a directive
> > that allows us to set multiple options instead, e.g.:
> >
> > .. qapi:infoboxes:: deprecated unstable
> >
> > or possibly:
> >
> > .. qapi:infoboxes::
> > :deprecated:
> > :unstable:
> >
> > For now, I've left these as top-level QAPI object options. "Hey, it
> works."
>
> Do you intend to drop this part in the final version?
>
> Having the commit message explain paths not taken can be useful. But
> this is phrased as an RFC, which suggests to me you plan to drop it.
>
Yep. You just hadn't reviewed these yet so I left the commentary in :) Now
that you've seen it, it can go.
>
> > P.S., I outsourced the CSS ;)
>
> Hi, Harmonie!
>
> > Signed-off-by: Harmonie Snow <harmonie@gmail.com>
> > Signed-off-by: John Snow <jsnow@redhat.com>
>
>
[-- Attachment #2: Type: text/html, Size: 5032 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 22/57] docs/qapi-domain: add :unstable: directive option
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (20 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 21/57] docs/qapi-domain: add :deprecated: directive option John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 23/57] docs/qapi-domain: add :ifcond: " John Snow
` (37 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow, Harmonie Snow
Although "unstable" is a feature (and *will* appear in the features
list), add a special :unstable: option to generate an eye-catch that
makes this information very hard to miss.
(The intent is to modify qapidoc.py to add this option whenever it
detects that the features list attached to a definition contains the
"unstable" entry.)
RFC: Same comments as last patch.
Signed-off-by: Harmonie Snow <harmonie@gmail.com>
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx-static/theme_overrides.css | 6 +++++-
docs/sphinx/qapi_domain.py | 8 ++++++++
2 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/docs/sphinx-static/theme_overrides.css b/docs/sphinx-static/theme_overrides.css
index 3765cab1b20..5f58f1d5246 100644
--- a/docs/sphinx-static/theme_overrides.css
+++ b/docs/sphinx-static/theme_overrides.css
@@ -221,7 +221,7 @@ div[class^="highlight"] pre {
margin: 0.25em;
}
-.qapi-deprecated {
+.qapi-deprecated,.qapi-unstable {
background-color: #fffef5;
border: solid #fff176 6px;
font-weight: bold;
@@ -230,6 +230,10 @@ div[class^="highlight"] pre {
margin: 5px;
}
+.qapi-unstable::before {
+ content: '🚧 ';
+}
+
.qapi-deprecated::before {
content: '⚠️ ';
}
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index 1be59e36bdf..ef4fa978a3a 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -141,6 +141,7 @@ class QAPIObject(ObjectDescription[Signature]):
# These are QAPI originals:
"since": since_validator,
"deprecated": directives.flag,
+ "unstable": directives.flag,
}
)
@@ -273,6 +274,13 @@ def _add_pip(source: str, content: str, classname: str) -> None:
"qapi-deprecated",
)
+ if "unstable" in self.options:
+ _add_pip(
+ ":unstable:",
+ f"This {self.objtype} is unstable/experimental.",
+ "qapi-unstable",
+ )
+
if infopips.children:
contentnode.insert(0, infopips)
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 23/57] docs/qapi-domain: add :ifcond: directive option
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (21 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 22/57] docs/qapi-domain: add :unstable: " John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-07 10:57 ` Markus Armbruster
2025-03-05 3:45 ` [PATCH 24/57] docs/qapi-domain: add warnings for malformed field lists John Snow
` (36 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow, Harmonie Snow
Add a special :ifcond: option that allows us to annotate the
definition-level conditionals.
RFC: This patch renders IFCOND information in two places, because I'm
undecided about how to style this information. One option is in the
signature bar, and another option is in an eye-catch, like :deprecated:
or :unstable:.
A benefit to having this be a directive option is that we can put it in
the signature bar, the QAPI index, etc. However, if we merely want it in
the content section, a directive would work just as well,
e.g. ".. qapi:ifcond:: CONFIG_LINUX".
(Though, having it be in the same containing box as the unstable/ifcond
boxes might require some extra fiddling/post-processing to
achieve. Generally, the less docutils tree muddling I have to do, the
happier I am.)
The syntax of the argument is currently undefined, but it is possible to
parse it back down into constituent parts to avoid applying literal
formatting to "AND" or "&&" or whichever syntax we formalize. (Or, in
the future, applying cross-reference links to the config values for
additional reading on some of those build options. Not for this series.)
"Vote now on your phones!"
Signed-off-by: Harmonie Snow <harmonie@gmail.com>
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx-static/theme_overrides.css | 13 +++++++++
docs/sphinx/qapi_domain.py | 39 ++++++++++++++++++++++++--
2 files changed, 50 insertions(+), 2 deletions(-)
diff --git a/docs/sphinx-static/theme_overrides.css b/docs/sphinx-static/theme_overrides.css
index 5f58f1d5246..3fd326613d9 100644
--- a/docs/sphinx-static/theme_overrides.css
+++ b/docs/sphinx-static/theme_overrides.css
@@ -237,3 +237,16 @@ div[class^="highlight"] pre {
.qapi-deprecated::before {
content: '⚠️ ';
}
+
+.qapi-ifcond::before {
+ /* gaze ye into the crystal ball to determine feature availability */
+ content: '🔮 ';
+}
+
+.qapi-ifcond {
+ background-color: #f9f5ff;
+ border: solid #dac2ff 6px;
+ padding: 8px;
+ border-radius: 15px;
+ margin: 5px;
+}
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index ef4fa978a3a..c75b0e6e22d 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -16,6 +16,7 @@
NamedTuple,
Optional,
Tuple,
+ Union,
cast,
)
@@ -140,6 +141,7 @@ class QAPIObject(ObjectDescription[Signature]):
"module": directives.unchanged, # Override contextual module name
# These are QAPI originals:
"since": since_validator,
+ "ifcond": directives.unchanged,
"deprecated": directives.flag,
"unstable": directives.flag,
}
@@ -167,6 +169,22 @@ def get_signature_suffix(self) -> List[nodes.Node]:
"""Returns a suffix to put after the object name in the signature."""
ret: List[nodes.Node] = []
+ if "ifcond" in self.options:
+ ret += [
+ SpaceNode(" "),
+ nodes.inline(
+ self.options["ifcond"],
+ "",
+ nodes.Text("["),
+ nodes.literal("", "#if"),
+ SpaceNode(" "),
+ nodes.literal(
+ self.options["ifcond"], self.options["ifcond"]
+ ),
+ nodes.Text("]"),
+ ),
+ ]
+
if "since" in self.options:
ret += [
SpaceNode(" "),
@@ -261,9 +279,14 @@ def _add_infopips(self, contentnode: addnodes.desc_content) -> None:
infopips = nodes.container()
infopips.attributes["classes"].append("qapi-infopips")
- def _add_pip(source: str, content: str, classname: str) -> None:
+ def _add_pip(
+ source: str, content: Union[str, List[nodes.Node]], classname: str
+ ) -> None:
node = nodes.container(source)
- node.append(nodes.Text(content))
+ if isinstance(content, str):
+ node.append(nodes.Text(content))
+ else:
+ node.extend(content)
node.attributes["classes"].extend(["qapi-infopip", classname])
infopips.append(node)
@@ -281,6 +304,18 @@ def _add_pip(source: str, content: str, classname: str) -> None:
"qapi-unstable",
)
+ if self.options.get("ifcond", ""):
+ ifcond = self.options["ifcond"]
+ _add_pip(
+ f":ifcond: {ifcond}",
+ [
+ nodes.emphasis("", "Availability"),
+ nodes.Text(": "),
+ nodes.literal(ifcond, ifcond),
+ ],
+ "qapi-ifcond",
+ )
+
if infopips.children:
contentnode.insert(0, infopips)
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 23/57] docs/qapi-domain: add :ifcond: directive option
2025-03-05 3:45 ` [PATCH 23/57] docs/qapi-domain: add :ifcond: " John Snow
@ 2025-03-07 10:57 ` Markus Armbruster
0 siblings, 0 replies; 129+ messages in thread
From: Markus Armbruster @ 2025-03-07 10:57 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé, Harmonie Snow
John Snow <jsnow@redhat.com> writes:
> Add a special :ifcond: option that allows us to annotate the
> definition-level conditionals.
>
> RFC: This patch renders IFCOND information in two places, because I'm
> undecided about how to style this information. One option is in the
> signature bar, and another option is in an eye-catch, like :deprecated:
> or :unstable:.
>
> A benefit to having this be a directive option is that we can put it in
> the signature bar, the QAPI index, etc. However, if we merely want it in
> the content section, a directive would work just as well,
> e.g. ".. qapi:ifcond:: CONFIG_LINUX".
You haven't implemented conditionals that aren't at definition-level.
As I said elsewhere, that's okay for now. All I want to say here is
that implementing it might influence your preference on how to do the
definition-level conditionals. That's fine, we can revisit this.
> (Though, having it be in the same containing box as the unstable/ifcond
> boxes might require some extra fiddling/post-processing to
> achieve. Generally, the less docutils tree muddling I have to do, the
> happier I am.)
>
> The syntax of the argument is currently undefined, but it is possible to
> parse it back down into constituent parts to avoid applying literal
> formatting to "AND" or "&&" or whichever syntax we formalize. (Or, in
> the future, applying cross-reference links to the config values for
> additional reading on some of those build options. Not for this series.)
>
> "Vote now on your phones!"
Find my vote here:
Message-ID: <87zfhya0is.fsf@pond.sub.org>
https://lore.kernel.org/qemu-devel/87zfhya0is.fsf@pond.sub.org/
> Signed-off-by: Harmonie Snow <harmonie@gmail.com>
> Signed-off-by: John Snow <jsnow@redhat.com>
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 24/57] docs/qapi-domain: add warnings for malformed field lists
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (22 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 23/57] docs/qapi-domain: add :ifcond: " John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 25/57] docs/qapi-domain: add type cross-refs to " John Snow
` (35 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Normally, Sphinx will silently fall back to its standard field list
processing if it doesn't match one of your defined fields. A lot of the
time, that's not what we want - we want to be warned if we goof
something up.
For instance, the canonical argument field list form is:
:arg type name: descr
This form is captured by Sphinx and transformed so that the field label
will become "Arguments:". It's possible to omit the type name and descr
and still have it be processed correctly. However, if you omit the type
name, Sphinx no longer recognizes it:
:arg: this is not recognized.
This will turn into an arbitrary field list entry whose label is "Arg:",
and it otherwise silently fails. You may also see failures for doing
things like using :values: instead of :value:, or :errors: instead of
:error:, and so on. It's also case sensitive, and easy to trip up.
Add a validator that guarantees all field list entries that are the
direct child of an ObjectDescription use only recognized forms of field
lists, and emit a warning (treated as error by default in most build
configurations) whenever we detect one that is goofed up.
However, there's still benefit to allowing arbitrary fields -- they are
after all not a Sphinx invention, but perfectly normal docutils
syntax. Create an allow list for known spellings we don't mind letting
through, but warn against anything else.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/conf.py | 9 +++++
docs/sphinx/qapi_domain.py | 77 ++++++++++++++++++++++++++++++++++++++
2 files changed, 86 insertions(+)
diff --git a/docs/conf.py b/docs/conf.py
index 49d9de894c0..a3f9fa63d94 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -153,6 +153,15 @@
with open(os.path.join(qemu_docdir, 'defs.rst.inc')) as f:
rst_epilog += f.read()
+
+# Normally, the QAPI domain is picky about what field lists you use to
+# describe a QAPI entity. If you'd like to use arbitrary additional
+# fields in source documentation, add them here.
+qapi_allowed_fields = {
+ "see also",
+}
+
+
# -- Options for HTML output ----------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index c75b0e6e22d..6eaf8ce92b4 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -52,6 +52,19 @@
logger = logging.getLogger(__name__)
+def _unpack_field(
+ field: nodes.Node,
+) -> Tuple[nodes.field_name, nodes.field_body]:
+ """
+ docutils helper: unpack a field node in a type-safe manner.
+ """
+ assert isinstance(field, nodes.field)
+ assert len(field.children) == 2
+ assert isinstance(field.children[0], nodes.field_name)
+ assert isinstance(field.children[1], nodes.field_body)
+ return (field.children[0], field.children[1])
+
+
class ObjectEntry(NamedTuple):
docname: str
node_id: str
@@ -319,9 +332,67 @@ def _add_pip(
if infopips.children:
contentnode.insert(0, infopips)
+ def _validate_field(self, field: nodes.field) -> None:
+ """Validate field lists in this QAPI Object Description."""
+ name, _ = _unpack_field(field)
+ allowed_fields = set(self.env.app.config.qapi_allowed_fields)
+
+ field_label = name.astext()
+ if (
+ re.match(r"\[\S+ = \S+\]", field_label)
+ or field_label in allowed_fields
+ ):
+ # okie-dokey. branch entry or known good allowed name.
+ return
+
+ try:
+ # split into field type and argument (if provided)
+ # e.g. `:arg type name: descr` is
+ # field_type = "arg", field_arg = "type name".
+ field_type, field_arg = field_label.split(None, 1)
+ except ValueError:
+ # No arguments provided
+ field_type = field_label
+ field_arg = ""
+
+ typemap = self.get_field_type_map()
+ if field_type in typemap:
+ # This is a special docfield, yet-to-be-processed. Catch
+ # correct names, but incorrect arguments. This mismatch WILL
+ # cause Sphinx to render this field incorrectly (without a
+ # warning), which is never what we want.
+ typedesc = typemap[field_type][0]
+ if typedesc.has_arg != bool(field_arg):
+ msg = f"docfield field list type {field_type!r} "
+ if typedesc.has_arg:
+ msg += "requires an argument."
+ else:
+ msg += "takes no arguments."
+ logger.warning(msg, location=field)
+ else:
+ # This is unrecognized entirely. It's valid rST to use
+ # arbitrary fields, but let's ensure the documentation
+ # writer has done this intentionally.
+ valid = ", ".join(sorted(set(typemap) | allowed_fields))
+ msg = (
+ f"Unrecognized field list name {field_label!r}.\n"
+ f"Valid fields for qapi:{self.objtype} are: {valid}\n"
+ "\n"
+ "If this usage is intentional, please add it to "
+ "'qapi_allowed_fields' in docs/conf.py."
+ )
+ logger.warning(msg, location=field)
+
def transform_content(self, content_node: addnodes.desc_content) -> None:
self._add_infopips(content_node)
+ # Validate field lists.
+ for child in content_node:
+ if isinstance(child, nodes.field_list):
+ for field in child.children:
+ assert isinstance(field, nodes.field)
+ self._validate_field(field)
+
def _toc_entry_name(self, sig_node: desc_signature) -> str:
# This controls the name in the TOC and on the sidebar.
@@ -817,6 +888,12 @@ def resolve_any_xref(
def setup(app: Sphinx) -> Dict[str, Any]:
app.setup_extension("sphinx.directives")
+ app.add_config_value(
+ "qapi_allowed_fields",
+ set(),
+ "env", # Setting impacts parsing phase
+ types=set,
+ )
app.add_domain(QAPIDomain)
return {
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 25/57] docs/qapi-domain: add type cross-refs to field lists
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (23 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 24/57] docs/qapi-domain: add warnings for malformed field lists John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 26/57] docs/qapi-domain: add CSS styling John Snow
` (34 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
This commit, finally, adds cross-referencing support to various field
lists; modeled tightly after Sphinx's own Python domain code.
Cross-referencing support is added to type names provided to :arg:,
:memb:, :returns: and :choice:.
:feat:, :error: and :value:, which do not take type names, do not
support this syntax.
The general syntax is simple:
:arg TypeName ArgName: Lorem Ipsum ...
The domain will transform TypeName into :qapi:type:`TypeName` in this
basic case, and also apply the ``literal`` decoration to indicate that
this is a type cross-reference.
For optional arguments, the special "?" suffix is used. Because "*" has
special meaning in rST that would cause parsing errors, we elect to use
"?" instead. The special syntax processing strips this character from
the end of any type name argument and will append ", optional" to the
rendered output, applying the cross-reference only to the actual type
name.
The intent here is that the actual syntax in doc-blocks need not change;
but e.g. qapidoc.py will need to process and transform "@arg foo lorem
ipsum" into ":arg type? foo: lorem ipsum" based on the schema
information. Therefore, nobody should ever actually witness this
intermediate syntax unless they are writing manual documentation or the
doc transmogrifier breaks.
For array arguments, type names can similarly be surrounded by "[]",
which are stripped off and then re-appended outside of the
cross-reference.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapi_domain.py | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index 6eaf8ce92b4..753e07024f5 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -2,6 +2,9 @@
QAPI domain extension.
"""
+# The best laid plans of mice and men, ...
+# pylint: disable=too-many-lines
+
from __future__ import annotations
import re
@@ -119,6 +122,28 @@ def process_link(
return title, target
+ def result_nodes(
+ self,
+ document: nodes.document,
+ env: BuildEnvironment,
+ node: Element,
+ is_ref: bool,
+ ) -> Tuple[List[nodes.Node], List[nodes.system_message]]:
+
+ # node here is the pending_xref node (or whatever nodeclass was
+ # configured at XRefRole class instantiation time).
+ results: List[nodes.Node] = [node]
+
+ if node.get("qapi:array"):
+ results.insert(0, nodes.literal("[", "["))
+ results.append(nodes.literal("]", "]"))
+
+ if node.get("qapi:optional"):
+ results.append(nodes.Text(", "))
+ results.append(nodes.emphasis("?", "optional"))
+
+ return results, []
+
def since_validator(param: str) -> str:
"""
@@ -423,6 +448,7 @@ class QAPICommand(QAPIObject):
"argument",
label=_("Arguments"),
names=("arg",),
+ typerolename="type",
can_collapse=False,
),
# :error: descr
@@ -436,6 +462,7 @@ class QAPICommand(QAPIObject):
GroupedField(
"returnvalue",
label=_("Returns"),
+ rolename="type",
names=("return", "returns"),
can_collapse=True,
),
@@ -471,6 +498,7 @@ class QAPIAlternate(QAPIObject):
"choice",
label=_("Choices"),
names=("choice",),
+ typerolename="type",
can_collapse=False,
),
]
@@ -488,6 +516,7 @@ class QAPIObjectWithMembers(QAPIObject):
"member",
label=_("Members"),
names=("memb",),
+ typerolename="type",
can_collapse=False,
),
]
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 26/57] docs/qapi-domain: add CSS styling
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (24 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 25/57] docs/qapi-domain: add type cross-refs to " John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 27/57] docs/qapi-domain: add XREF compatibility goop for Sphinx < 4.1 John Snow
` (33 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow, Harmonie Snow
Improve the general look and feel of generated QAPI docs.
Attempt to limit line lengths to offer a more comfortable measure on
maximized windows, and improve some margin and spacing for field lists.
Signed-off-by: Harmonie Snow <harmonie@gmail.com>
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx-static/theme_overrides.css | 56 +++++++++++++++++++++++++-
1 file changed, 54 insertions(+), 2 deletions(-)
diff --git a/docs/sphinx-static/theme_overrides.css b/docs/sphinx-static/theme_overrides.css
index 3fd326613d9..92f395054a8 100644
--- a/docs/sphinx-static/theme_overrides.css
+++ b/docs/sphinx-static/theme_overrides.css
@@ -18,8 +18,8 @@ h1, h2, .rst-content .toctree-wrapper p.caption, h3, h4, h5, h6, legend {
.rst-content dl:not(.docutils) dt {
border-top: none;
- border-left: solid 3px #ccc;
- background-color: #f0f0f0;
+ border-left: solid 5px #bcc6d2;
+ background-color: #eaedf1;
color: black;
}
@@ -211,6 +211,18 @@ div[class^="highlight"] pre {
/* QAPI domain theming */
+/* most content in a qapi object definition should not eclipse about
+ 80ch, but nested field lists are explicitly exempt due to their
+ two-column nature */
+.qapi dd *:not(dl) {
+ max-width: 80ch;
+}
+
+/* but the content column itself should still be less than ~80ch. */
+.qapi .field-list dd {
+ max-width: 80ch;
+}
+
.qapi-infopips {
margin-bottom: 1em;
}
@@ -250,3 +262,43 @@ div[class^="highlight"] pre {
border-radius: 15px;
margin: 5px;
}
+
+/* code blocks */
+.qapi div[class^="highlight"] {
+ width: fit-content;
+ background-color: #fffafd;
+ border: 2px solid #ffe1f3;
+}
+
+/* note, warning, etc. */
+.qapi .admonition {
+ width: fit-content;
+}
+
+/* pad the top of the field-list so the text doesn't start directly at
+ the top border; primarily for the field list labels, but adjust the
+ field bodies as well for parity. */
+dl.field-list > dt:first-of-type, dl.field-list > dd:first-of-type {
+ padding-top: 0.3em;
+}
+
+dl.field-list > dt:last-of-type, dl.field-list > dd:last-of-type {
+ padding-bottom: 0.3em;
+}
+
+/* pad the field list labels so they don't crash into the border */
+dl.field-list > dt {
+ padding-left: 0.5em;
+ padding-right: 0.5em;
+}
+
+/* Add a little padding between field list sections */
+dl.field-list > dd:not(:last-child) {
+ padding-bottom: 1em;
+}
+
+/* Sphinx 3.x: unresolved xrefs */
+.rst-content *:not(a) > code.xref {
+ font-weight: 400;
+ color: #333333;
+}
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 27/57] docs/qapi-domain: add XREF compatibility goop for Sphinx < 4.1
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (25 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 26/57] docs/qapi-domain: add CSS styling John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 28/57] docs/qapi-domain: warn when QAPI domain xrefs fail to resolve John Snow
` (32 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Sphinx < 4.1 handles cross-references ... differently. Factor out and
isolate the compatibility goop we need to make cross references work
properly in old versions of Sphinx.
Yes, it's ugly. Yes, it works. No, I don't want to talk about
it.
Understand that this patch exists because of the overflowing love in my
heart.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/compat.py | 129 ++++++++++++++++++++++++++++++++++++-
docs/sphinx/qapi_domain.py | 24 ++++---
2 files changed, 142 insertions(+), 11 deletions(-)
diff --git a/docs/sphinx/compat.py b/docs/sphinx/compat.py
index 1f9f1f42ef7..7e8d72c9b1f 100644
--- a/docs/sphinx/compat.py
+++ b/docs/sphinx/compat.py
@@ -2,14 +2,32 @@
Sphinx cross-version compatibility goop
"""
-from typing import Callable
+import re
+from typing import (
+ Any,
+ Callable,
+ Optional,
+ Type,
+)
+from docutils import nodes
from docutils.nodes import Element, Node, Text
import sphinx
from sphinx import addnodes
-from sphinx.util.docutils import SphinxDirective, switch_source_input
+from sphinx.environment import BuildEnvironment
+from sphinx.roles import XRefRole
+from sphinx.util import docfields
+from sphinx.util.docutils import (
+ ReferenceRole,
+ SphinxDirective,
+ switch_source_input,
+)
from sphinx.util.nodes import nested_parse_with_titles
+from sphinx.util.typing import TextlikeNode
+
+
+MAKE_XREF_WORKAROUND = sphinx.version_info[:3] < (4, 1, 0)
SpaceNode: Callable[[str], Node]
@@ -46,3 +64,110 @@ def nested_parse(directive: SphinxDirective, content_node: Element) -> None:
nested_parse_with_titles(
directive.state, directive.content, content_node
)
+
+
+# ###########################################
+# xref compatibility hacks for Sphinx < 4.1 #
+# ###########################################
+
+# When we require >= Sphinx 4.1, the following function and the
+# subsequent 3 compatibility classes can be removed. Anywhere in
+# qapi_domain that uses one of these Compat* types can be switched to
+# using the garden-variety lib-provided classes with no trickery.
+
+
+def _compat_make_xref( # pylint: disable=unused-argument
+ self: sphinx.util.docfields.Field,
+ rolename: str,
+ domain: str,
+ target: str,
+ innernode: Type[TextlikeNode] = addnodes.literal_emphasis,
+ contnode: Optional[Node] = None,
+ env: Optional[BuildEnvironment] = None,
+ inliner: Any = None,
+ location: Any = None,
+) -> Node:
+ """
+ Compatibility workaround for Sphinx versions prior to 4.1.0.
+
+ Older sphinx versions do not use the domain's XRefRole for parsing
+ and formatting cross-references, so we need to perform this magick
+ ourselves to avoid needing to write the parser/formatter in two
+ separate places.
+
+ This workaround isn't brick-for-brick compatible with modern Sphinx
+ versions, because we do not have access to the parent directive's
+ state during this parsing like we do in more modern versions.
+
+ It's no worse than what pre-Sphinx 4.1.0 does, so... oh well!
+ """
+
+ # Yes, this function is gross. Pre-4.1 support is a miracle.
+ # pylint: disable=too-many-locals
+
+ assert env
+ # Note: Sphinx's own code ignores the type warning here, too.
+ if not rolename:
+ return contnode or innernode(target, target) # type: ignore[call-arg]
+
+ # Get the role instance, but don't *execute it* - we lack the
+ # correct state to do so. Instead, we'll just use its public
+ # methods to do our reference formatting, and emulate the rest.
+ role = env.get_domain(domain).roles[rolename]
+ assert isinstance(role, XRefRole)
+
+ # XRefRole features not supported by this compatibility shim;
+ # these were not supported in Sphinx 3.x either, so nothing of
+ # value is really lost.
+ assert not target.startswith("!")
+ assert not re.match(ReferenceRole.explicit_title_re, target)
+ assert not role.lowercase
+ assert not role.fix_parens
+
+ # Code below based mostly on sphinx.roles.XRefRole; run() and
+ # create_xref_node()
+ options = {
+ "refdoc": env.docname,
+ "refdomain": domain,
+ "reftype": rolename,
+ "refexplicit": False,
+ "refwarn": role.warn_dangling,
+ }
+ refnode = role.nodeclass(target, **options)
+ title, target = role.process_link(env, refnode, False, target, target)
+ refnode["reftarget"] = target
+ classes = ["xref", domain, f"{domain}-{rolename}"]
+ refnode += role.innernodeclass(target, title, classes=classes)
+
+ # This is the very gross part of the hack. Normally,
+ # result_nodes takes a document object to which we would pass
+ # self.inliner.document. Prior to Sphinx 4.1, we don't *have* an
+ # inliner to pass, so we have nothing to pass here. However, the
+ # actual implementation of role.result_nodes in this case
+ # doesn't actually use that argument, so this winds up being
+ # ... fine. Rest easy at night knowing this code only runs under
+ # old versions of Sphinx, so at least it won't change in the
+ # future on us and lead to surprising new failures.
+ # Gross, I know.
+ result_nodes, _messages = role.result_nodes(
+ None, # type: ignore
+ env,
+ refnode,
+ is_ref=True,
+ )
+ return nodes.inline(target, "", *result_nodes)
+
+
+class CompatField(docfields.Field):
+ if MAKE_XREF_WORKAROUND:
+ make_xref = _compat_make_xref
+
+
+class CompatGroupedField(docfields.GroupedField):
+ if MAKE_XREF_WORKAROUND:
+ make_xref = _compat_make_xref
+
+
+class CompatTypedField(docfields.TypedField):
+ if MAKE_XREF_WORKAROUND:
+ make_xref = _compat_make_xref
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index 753e07024f5..d3487c5dfeb 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -26,7 +26,14 @@
from docutils import nodes
from docutils.parsers.rst import directives
-from compat import KeywordNode, SpaceNode, nested_parse
+from compat import (
+ CompatField,
+ CompatGroupedField,
+ CompatTypedField,
+ KeywordNode,
+ SpaceNode,
+ nested_parse,
+)
from sphinx import addnodes
from sphinx.addnodes import desc_signature, pending_xref
from sphinx.directives import ObjectDescription
@@ -39,7 +46,6 @@
from sphinx.locale import _, __
from sphinx.roles import XRefRole
from sphinx.util import logging
-from sphinx.util.docfields import Field, GroupedField, TypedField
from sphinx.util.docutils import SphinxDirective
from sphinx.util.nodes import make_id, make_refnode
@@ -187,7 +193,7 @@ class QAPIObject(ObjectDescription[Signature]):
doc_field_types = [
# :feat name: descr
- GroupedField(
+ CompatGroupedField(
"feature",
label=_("Features"),
names=("feat",),
@@ -444,7 +450,7 @@ class QAPICommand(QAPIObject):
doc_field_types.extend(
[
# :arg TypeName ArgName: descr
- TypedField(
+ CompatTypedField(
"argument",
label=_("Arguments"),
names=("arg",),
@@ -452,14 +458,14 @@ class QAPICommand(QAPIObject):
can_collapse=False,
),
# :error: descr
- Field(
+ CompatField(
"error",
label=_("Errors"),
names=("error", "errors"),
has_arg=False,
),
# :returns TypeName: descr
- GroupedField(
+ CompatGroupedField(
"returnvalue",
label=_("Returns"),
rolename="type",
@@ -477,7 +483,7 @@ class QAPIEnum(QAPIObject):
doc_field_types.extend(
[
# :value name: descr
- GroupedField(
+ CompatGroupedField(
"value",
label=_("Values"),
names=("value",),
@@ -494,7 +500,7 @@ class QAPIAlternate(QAPIObject):
doc_field_types.extend(
[
# :choice type name: descr
- TypedField(
+ CompatTypedField(
"choice",
label=_("Choices"),
names=("choice",),
@@ -512,7 +518,7 @@ class QAPIObjectWithMembers(QAPIObject):
doc_field_types.extend(
[
# :member type name: descr
- TypedField(
+ CompatTypedField(
"member",
label=_("Members"),
names=("memb",),
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 28/57] docs/qapi-domain: warn when QAPI domain xrefs fail to resolve
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (26 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 27/57] docs/qapi-domain: add XREF compatibility goop for Sphinx < 4.1 John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 29/57] docs/qapi-domain: Fix error context reporting in Sphinx 5.x and 6.x John Snow
` (31 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
This patch adds a warning (which is a build failure under our current
build settings) whenever a QAPI cross-reference fails to resolve.
This applies to any cross-references of the form :qapi:{role}:`foo`,
which covers all of the automatically generated references by the qapi
domain, and any such references that are manually written into the
documentation rst files.
Cross-references of the form `foo` do not use this system, but are
already configured to issue a warning (Again, a build failure) if the
cross-reference isn't found anywhere.
Adds warnings that look like the following:
docs/qapi/index.rst:48: WARNING: qapi:type reference target not found: 'footype' [ref.qapi]
docs/qapi/index.rst:50: WARNING: qapi:mod reference target not found: 'foomod' [ref.qapi]
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapi_domain.py | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index d3487c5dfeb..9a11a2dcbe0 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -882,6 +882,29 @@ def resolve_xref(
matches = self.find_obj(modname, target, typ)
if not matches:
+ # Normally, we could pass warn_dangling=True to QAPIXRefRole(),
+ # but that will trigger on references to these built-in types,
+ # which we'd like to ignore instead.
+
+ # Take care of that warning here instead, so long as the
+ # reference isn't to one of our built-in core types.
+ if target not in (
+ "string",
+ "number",
+ "int",
+ "boolean",
+ "null",
+ "value",
+ "q_empty",
+ ):
+ logger.warning(
+ __("qapi:%s reference target not found: %r"),
+ typ,
+ target,
+ type="ref",
+ subtype="qapi",
+ location=node,
+ )
return None
if len(matches) > 1:
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 29/57] docs/qapi-domain: Fix error context reporting in Sphinx 5.x and 6.x
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (27 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 28/57] docs/qapi-domain: warn when QAPI domain xrefs fail to resolve John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 30/57] qapi/parser: adjust info location for doc body section John Snow
` (30 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Sphinx 5.3.0 to Sphinx 6.2.0 has a bug where nested content in an
ObjectDescription content block has its error position reported
incorrectly due to an oversight when they added nested section support
to this directive.
(This bug is present in Sphinx's own Python and C domains; test it
yourself by creating a py:func directive and creating a syntax error in
the directive's content block. The reporting will be incorrect.)
To avoid overriding and re-implementing the entirety of the run()
method, a workaround is employed where we parse the content block
ourselves in before_content(), then null the content block to make
Sphinx's own parsing a no-op. Then, in transform_content (which occurs
after Sphinx's nested parse), we simply swap our own parsed content tree
back in for Sphinx's.
It appears a little tricky, but it's the nicest solution I can find.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/compat.py | 56 ++++++++++++++++++++++++++++++++++++++
docs/sphinx/qapi_domain.py | 13 +++++----
2 files changed, 63 insertions(+), 6 deletions(-)
diff --git a/docs/sphinx/compat.py b/docs/sphinx/compat.py
index 7e8d72c9b1f..e13bee5c209 100644
--- a/docs/sphinx/compat.py
+++ b/docs/sphinx/compat.py
@@ -4,6 +4,7 @@
import re
from typing import (
+ TYPE_CHECKING,
Any,
Callable,
Optional,
@@ -12,9 +13,11 @@
from docutils import nodes
from docutils.nodes import Element, Node, Text
+from docutils.statemachine import StringList
import sphinx
from sphinx import addnodes
+from sphinx.directives import ObjectDescription
from sphinx.environment import BuildEnvironment
from sphinx.roles import XRefRole
from sphinx.util import docfields
@@ -171,3 +174,56 @@ class CompatGroupedField(docfields.GroupedField):
class CompatTypedField(docfields.TypedField):
if MAKE_XREF_WORKAROUND:
make_xref = _compat_make_xref
+
+
+# ################################################################
+# Nested parsing error location fix for Sphinx 5.3.0 < x < 6.2.0 #
+# ################################################################
+
+# When we require Sphinx 4.x, the TYPE_CHECKING hack where we avoid
+# subscripting ObjectDescription at runtime can be removed in favor of
+# just always subscripting the class.
+
+# When we require Sphinx > 6.2.0, the rest of this compatibility hack
+# can be dropped and QAPIObject can just inherit directly from
+# ObjectDescription[Signature].
+
+SOURCE_LOCATION_FIX = (5, 3, 0) <= sphinx.version_info[:3] < (6, 2, 0)
+
+Signature = str
+
+
+if TYPE_CHECKING:
+ _BaseClass = ObjectDescription[Signature]
+else:
+ _BaseClass = ObjectDescription
+
+
+class ParserFix(_BaseClass):
+
+ _temp_content: StringList
+ _temp_offset: int
+ _temp_node: Optional[addnodes.desc_content]
+
+ def before_content(self) -> None:
+ # Work around a sphinx bug and parse the content ourselves.
+ self._temp_content = self.content
+ self._temp_offset = self.content_offset
+ self._temp_node = None
+
+ if SOURCE_LOCATION_FIX:
+ self._temp_node = addnodes.desc_content()
+ self.state.nested_parse(
+ self.content, self.content_offset, self._temp_node
+ )
+ # Sphinx will try to parse the content block itself,
+ # Give it nothingness to parse instead.
+ self.content = StringList()
+ self.content_offset = 0
+
+ def transform_content(self, content_node: addnodes.desc_content) -> None:
+ # Sphinx workaround: Inject our parsed content and restore state.
+ if self._temp_node:
+ content_node += self._temp_node.children
+ self.content = self._temp_content
+ self.content_offset = self._temp_offset
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index 9a11a2dcbe0..077ab7ae47a 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -31,6 +31,8 @@
CompatGroupedField,
CompatTypedField,
KeywordNode,
+ ParserFix,
+ Signature,
SpaceNode,
nested_parse,
)
@@ -163,12 +165,7 @@ def since_validator(param: str) -> str:
return param
-# Alias for the return of handle_signature(), which is used in several places.
-# (In the Python domain, this is Tuple[str, str] instead.)
-Signature = str
-
-
-class QAPIObject(ObjectDescription[Signature]):
+class QAPIObject(ParserFix):
"""
Description of a generic QAPI object.
@@ -415,6 +412,10 @@ def _validate_field(self, field: nodes.field) -> None:
logger.warning(msg, location=field)
def transform_content(self, content_node: addnodes.desc_content) -> None:
+ # This hook runs after before_content and the nested parse, but
+ # before the DocFieldTransformer is executed.
+ super().transform_content(content_node)
+
self._add_infopips(content_node)
# Validate field lists.
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 30/57] qapi/parser: adjust info location for doc body section
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (28 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 29/57] docs/qapi-domain: Fix error context reporting in Sphinx 5.x and 6.x John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 10:10 ` Markus Armbruster
2025-03-05 3:45 ` [PATCH 31/57] qapi: expand tags to all doc sections John Snow
` (29 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Instead of using the info object for the doc block as a whole (which
always points to the very first line of the block), update the info
pointer for each call to ensure_untagged_section when the existing
section is otherwise empty. This way, Sphinx error information will
match precisely to where the text actually starts.
For example, this patch will move the info pointer for the "Hello!"
untagged section ...
> ## <-- from here ...
> # Hello! <-- ... to here.
> ##
Signed-off-by: John Snow <jsnow@redhat.com>
---
scripts/qapi/parser.py | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index adc85b5b394..36cb64a677a 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -687,7 +687,11 @@ def end(self) -> None:
def ensure_untagged_section(self, info: QAPISourceInfo) -> None:
if self.all_sections and not self.all_sections[-1].tag:
# extend current section
- self.all_sections[-1].text += '\n'
+ section = self.all_sections[-1]
+ if not section.text:
+ # Section is empty so far; update info to start *here*.
+ section.info = info
+ section.text += '\n'
return
# start new section
section = self.Section(info)
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 30/57] qapi/parser: adjust info location for doc body section
2025-03-05 3:45 ` [PATCH 30/57] qapi/parser: adjust info location for doc body section John Snow
@ 2025-03-05 10:10 ` Markus Armbruster
2025-03-06 3:42 ` John Snow
0 siblings, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-05 10:10 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> Instead of using the info object for the doc block as a whole (which
> always points to the very first line of the block), update the info
> pointer for each call to ensure_untagged_section when the existing
> section is otherwise empty. This way, Sphinx error information will
> match precisely to where the text actually starts.
>
> For example, this patch will move the info pointer for the "Hello!"
> untagged section ...
>
>> ## <-- from here ...
>> # Hello! <-- ... to here.
>> ##
>
> Signed-off-by: John Snow <jsnow@redhat.com>
This patch would be easier to accept with a test case where it improves
the error location. I tried to construct one quickly, but failed. Can
you help?
Possible substitute: point to a patch later in this series where things
become worse without this patch.
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 30/57] qapi/parser: adjust info location for doc body section
2025-03-05 10:10 ` Markus Armbruster
@ 2025-03-06 3:42 ` John Snow
2025-03-06 6:53 ` Markus Armbruster
0 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-06 3:42 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 3108 bytes --]
On Wed, Mar 5, 2025 at 5:10 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > Instead of using the info object for the doc block as a whole (which
> > always points to the very first line of the block), update the info
> > pointer for each call to ensure_untagged_section when the existing
> > section is otherwise empty. This way, Sphinx error information will
> > match precisely to where the text actually starts.
> >
> > For example, this patch will move the info pointer for the "Hello!"
> > untagged section ...
> >
> >> ## <-- from here ...
> >> # Hello! <-- ... to here.
> >> ##
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
>
> This patch would be easier to accept with a test case where it improves
> the error location. I tried to construct one quickly, but failed. Can
> you help?
>
> Possible substitute: point to a patch later in this series where things
> become worse without this patch.
Maybe we can use the "if build_docs" section of the qapi schema testing to
run things through Sphinx and harvest the error messages for negative
cases...? I gotta sit down and figure out how.
In the meantime, if I unapply my series, then edit block-core to look like
this:
##
# @SnapshotInfo:
#
# rST syntax error: *ahh!
#
(Lines 13-17, error is on line 16)
Building, I get this error:
/home/jsnow/src/qemu/docs/../qapi/block-core.json:14: WARNING: Inline
emphasis start-string without end-string. [docutils]
/home/jsnow/src/qemu/docs/../storage-daemon/qapi/../../qapi/block-core.json:14:
WARNING: Inline emphasis start-string without end-string. [docutils]
Mmm, nope. Not quite.
If I re-push my series and try again with the same edit ...
/home/jsnow/src/qemu/docs/../qapi/block-core.json:14: WARNING: Inline
emphasis start-string without end-string. [docutils]
/home/jsnow/src/qemu/docs/../storage-daemon/qapi/../../qapi/block-core.json:14:
WARNING: Inline emphasis start-string without end-string. [docutils]
/home/jsnow/src/qemu/docs/../qapi/block-core.json:16: WARNING: Inline
emphasis start-string without end-string. [docutils]
The two inclusions from the old qapidoc are still wrong, but the inclusion
through the new transmogrifier is correct.
(I'm going to be honest with you, I don't know why the error location
didn't change at all for the old qapidoc. I think one of the many error
location bugs I fixed when writing the new transmogrifier that just never
got applied to the old system...)
If I undo this fix but keep the rest of my series, I get these errors:
/home/jsnow/src/qemu/docs/../qapi/block-core.json:14: WARNING: Inline
emphasis start-string without end-string. [docutils]
/home/jsnow/src/qemu/docs/../storage-daemon/qapi/../../qapi/block-core.json:14:
WARNING: Inline emphasis start-string without end-string. [docutils]
/home/jsnow/src/qemu/docs/../qapi/block-core.json:13: WARNING: Inline
emphasis start-string without end-string. [docutils]
Two are from the old qapidoc, one is from the new one. They're all wrong.
--js
[-- Attachment #2: Type: text/html, Size: 4464 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 30/57] qapi/parser: adjust info location for doc body section
2025-03-06 3:42 ` John Snow
@ 2025-03-06 6:53 ` Markus Armbruster
0 siblings, 0 replies; 129+ messages in thread
From: Markus Armbruster @ 2025-03-06 6:53 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> On Wed, Mar 5, 2025 at 5:10 AM Markus Armbruster <armbru@redhat.com> wrote:
>
>> John Snow <jsnow@redhat.com> writes:
>>
>> > Instead of using the info object for the doc block as a whole (which
>> > always points to the very first line of the block), update the info
>> > pointer for each call to ensure_untagged_section when the existing
>> > section is otherwise empty. This way, Sphinx error information will
>> > match precisely to where the text actually starts.
>> >
>> > For example, this patch will move the info pointer for the "Hello!"
>> > untagged section ...
>> >
>> >> ## <-- from here ...
>> >> # Hello! <-- ... to here.
>> >> ##
>> >
>> > Signed-off-by: John Snow <jsnow@redhat.com>
>>
>> This patch would be easier to accept with a test case where it improves
>> the error location. I tried to construct one quickly, but failed. Can
>> you help?
>>
>> Possible substitute: point to a patch later in this series where things
>> become worse without this patch.
>
>
> Maybe we can use the "if build_docs" section of the qapi schema testing to
> run things through Sphinx and harvest the error messages for negative
> cases...? I gotta sit down and figure out how.
Rule of thumb for compilers: every error needs a negative test. So,
test coverage for transmogrifier errors is certainly wanted, but it's
not a blocker for an initial merge. We need to make progress, and to
make progress, we need to limit the size of the steps.
> In the meantime, if I unapply my series, then edit block-core to look like
> this:
>
> ##
> # @SnapshotInfo:
> #
> # rST syntax error: *ahh!
> #
>
> (Lines 13-17, error is on line 16)
>
> Building, I get this error:
>
> /home/jsnow/src/qemu/docs/../qapi/block-core.json:14: WARNING: Inline
> emphasis start-string without end-string. [docutils]
> /home/jsnow/src/qemu/docs/../storage-daemon/qapi/../../qapi/block-core.json:14:
> WARNING: Inline emphasis start-string without end-string. [docutils]
>
> Mmm, nope. Not quite.
>
> If I re-push my series and try again with the same edit ...
>
> /home/jsnow/src/qemu/docs/../qapi/block-core.json:14: WARNING: Inline
> emphasis start-string without end-string. [docutils]
> /home/jsnow/src/qemu/docs/../storage-daemon/qapi/../../qapi/block-core.json:14:
> WARNING: Inline emphasis start-string without end-string. [docutils]
> /home/jsnow/src/qemu/docs/../qapi/block-core.json:16: WARNING: Inline
> emphasis start-string without end-string. [docutils]
>
> The two inclusions from the old qapidoc are still wrong, but the inclusion
> through the new transmogrifier is correct.
Reproduced. To get non-first warnings, you have to configure
--disable-werror. Observation, not a complaint.
> (I'm going to be honest with you, I don't know why the error location
> didn't change at all for the old qapidoc. I think one of the many error
> location bugs I fixed when writing the new transmogrifier that just never
> got applied to the old system...)
I'm definitely not asking you to find out more :)
> If I undo this fix but keep the rest of my series, I get these errors:
>
> /home/jsnow/src/qemu/docs/../qapi/block-core.json:14: WARNING: Inline
> emphasis start-string without end-string. [docutils]
> /home/jsnow/src/qemu/docs/../storage-daemon/qapi/../../qapi/block-core.json:14:
> WARNING: Inline emphasis start-string without end-string. [docutils]
> /home/jsnow/src/qemu/docs/../qapi/block-core.json:13: WARNING: Inline
> emphasis start-string without end-string. [docutils]
>
> Two are from the old qapidoc, one is from the new one. They're all wrong.
Also reproduced. Thanks!
Perhaps the patch could be moved closer to where it's needed, or even be
squashed into one the first one that depends it. Bah, good enough as
is.
Here's my attempt to improve the commit message:
qapi/parser: adjust info location for doc body section
Instead of using the info object for the doc block as a whole (which
always points to the very first line of the block), update the info
pointer for each call to ensure_untagged_section when the existing
section is otherwise empty. This way, Sphinx error information will
match precisely to where the text actually starts.
For example, this patch will move the info pointer for the "Hello!"
untagged section ...
## <-- from here ...
# Hello! <-- ... to here.
##
This doesn't seem to improve error reporting now. It will with the
QAPI doc transmogrifier I'm about to add.
If I stick bad rST into qapi/block-core.json like this
##
# @SnapshotInfo:
#
+# rST syntax error: *ahh!
+#
# @id: unique snapshot id
#
# @name: user chosen name
the existing code's error message will point to the beginning of the
doc comment, which is less than helpful. The transmogrifier's
message will point to the erroneous line, but to accomplish this, it
needs this patch.
What do you think?
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 31/57] qapi: expand tags to all doc sections
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (29 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 30/57] qapi/parser: adjust info location for doc body section John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 10:16 ` Markus Armbruster
2025-03-05 3:45 ` [PATCH 32/57] qapi/schema: add __repr__ to QAPIDoc.Section John Snow
` (28 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
This patch adds an explicit section "kind" to all QAPIDoc
sections. Members/Features are now explicitly marked as such, with the
name now being stored in a dedicated "name" field (which qapidoc.py was
not actually using anyway.)
The qapi-schema tests are updated to account for the new section names;
mostly "TODO" becomes "Todo" and `None` becomes "Plain".
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapidoc.py | 7 ++-
scripts/qapi/parser.py | 109 ++++++++++++++++++++++++---------
tests/qapi-schema/doc-good.out | 10 +--
tests/qapi-schema/test-qapi.py | 2 +-
4 files changed, 90 insertions(+), 38 deletions(-)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index 61997fd21af..d622398f1da 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -35,6 +35,7 @@
from docutils.statemachine import ViewList
from qapi.error import QAPIError, QAPISemError
from qapi.gen import QAPISchemaVisitor
+from qapi.parser import QAPIDoc
from qapi.schema import QAPISchema
from sphinx import addnodes
@@ -258,11 +259,11 @@ def _nodes_for_sections(self, doc):
"""Return list of doctree nodes for additional sections"""
nodelist = []
for section in doc.sections:
- if section.tag and section.tag == 'TODO':
+ if section.kind == QAPIDoc.Kind.TODO:
# Hide TODO: sections
continue
- if not section.tag:
+ if section.kind == QAPIDoc.Kind.PLAIN:
# Sphinx cannot handle sectionless titles;
# Instead, just append the results to the prior section.
container = nodes.container()
@@ -270,7 +271,7 @@ def _nodes_for_sections(self, doc):
nodelist += container.children
continue
- snode = self._make_section(section.tag)
+ snode = self._make_section(section.kind.name.title())
self._parse_text_into_node(dedent(section.text), snode)
nodelist.append(snode)
return nodelist
diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index 36cb64a677a..c3004aa70c6 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -15,6 +15,7 @@
# See the COPYING file in the top-level directory.
from collections import OrderedDict
+import enum
import os
import re
from typing import (
@@ -575,7 +576,10 @@ def get_doc(self) -> 'QAPIDoc':
)
raise QAPIParseError(self, emsg)
- doc.new_tagged_section(self.info, match.group(1))
+ doc.new_tagged_section(
+ self.info,
+ QAPIDoc.Kind.from_string(match.group(1))
+ )
text = line[match.end():]
if text:
doc.append_line(text)
@@ -586,7 +590,7 @@ def get_doc(self) -> 'QAPIDoc':
self,
"unexpected '=' markup in definition documentation")
else:
- # tag-less paragraph
+ # plain paragraph(s)
doc.ensure_untagged_section(self.info)
doc.append_line(line)
line = self.get_doc_paragraph(doc)
@@ -635,14 +639,37 @@ class QAPIDoc:
Free-form documentation blocks consist only of a body section.
"""
+ class Kind(enum.Enum):
+ PLAIN = 0
+ MEMBER = 1
+ FEATURE = 2
+ RETURNS = 3
+ ERRORS = 4
+ SINCE = 5
+ TODO = 6
+
+ @staticmethod
+ def from_string(kind: str) -> 'QAPIDoc.Kind':
+ return QAPIDoc.Kind[kind.upper()]
+
+ def text_required(self) -> bool:
+ # Only "plain" sections can be empty
+ return self.value not in (0,)
+
+ def __str__(self) -> str:
+ return self.name.title()
+
class Section:
# pylint: disable=too-few-public-methods
- def __init__(self, info: QAPISourceInfo,
- tag: Optional[str] = None):
+ def __init__(
+ self,
+ info: QAPISourceInfo,
+ kind: 'QAPIDoc.Kind',
+ ):
# section source info, i.e. where it begins
self.info = info
- # section tag, if any ('Returns', '@name', ...)
- self.tag = tag
+ # section kind
+ self.kind = kind
# section text without tag
self.text = ''
@@ -650,8 +677,14 @@ def append_line(self, line: str) -> None:
self.text += line + '\n'
class ArgSection(Section):
- def __init__(self, info: QAPISourceInfo, tag: str):
- super().__init__(info, tag)
+ def __init__(
+ self,
+ info: QAPISourceInfo,
+ kind: 'QAPIDoc.Kind',
+ name: str
+ ):
+ super().__init__(info, kind)
+ self.name = name
self.member: Optional['QAPISchemaMember'] = None
def connect(self, member: 'QAPISchemaMember') -> None:
@@ -663,7 +696,9 @@ def __init__(self, info: QAPISourceInfo, symbol: Optional[str] = None):
# definition doc's symbol, None for free-form doc
self.symbol: Optional[str] = symbol
# the sections in textual order
- self.all_sections: List[QAPIDoc.Section] = [QAPIDoc.Section(info)]
+ self.all_sections: List[QAPIDoc.Section] = [
+ QAPIDoc.Section(info, QAPIDoc.Kind.PLAIN)
+ ]
# the body section
self.body: Optional[QAPIDoc.Section] = self.all_sections[0]
# dicts mapping parameter/feature names to their description
@@ -680,12 +715,17 @@ def __init__(self, info: QAPISourceInfo, symbol: Optional[str] = None):
def end(self) -> None:
for section in self.all_sections:
section.text = section.text.strip('\n')
- if section.tag is not None and section.text == '':
+ if section.kind.text_required() and section.text == '':
raise QAPISemError(
- section.info, "text required after '%s:'" % section.tag)
+ section.info, "text required after '%s:'" % section.kind)
- def ensure_untagged_section(self, info: QAPISourceInfo) -> None:
- if self.all_sections and not self.all_sections[-1].tag:
+ def ensure_untagged_section(
+ self,
+ info: QAPISourceInfo,
+ ) -> None:
+ kind = QAPIDoc.Kind.PLAIN
+
+ if self.all_sections and self.all_sections[-1].kind == kind:
# extend current section
section = self.all_sections[-1]
if not section.text:
@@ -693,46 +733,56 @@ def ensure_untagged_section(self, info: QAPISourceInfo) -> None:
section.info = info
section.text += '\n'
return
+
# start new section
- section = self.Section(info)
+ section = self.Section(info, kind)
self.sections.append(section)
self.all_sections.append(section)
- def new_tagged_section(self, info: QAPISourceInfo, tag: str) -> None:
- section = self.Section(info, tag)
- if tag == 'Returns':
+ def new_tagged_section(
+ self,
+ info: QAPISourceInfo,
+ kind: 'QAPIDoc.Kind',
+ ) -> None:
+ section = self.Section(info, kind)
+ if kind == QAPIDoc.Kind.RETURNS:
if self.returns:
raise QAPISemError(
- info, "duplicated '%s' section" % tag)
+ info, "duplicated '%s' section" % kind)
self.returns = section
- elif tag == 'Errors':
+ elif kind == QAPIDoc.Kind.ERRORS:
if self.errors:
raise QAPISemError(
- info, "duplicated '%s' section" % tag)
+ info, "duplicated '%s' section" % kind)
self.errors = section
- elif tag == 'Since':
+ elif kind == QAPIDoc.Kind.SINCE:
if self.since:
raise QAPISemError(
- info, "duplicated '%s' section" % tag)
+ info, "duplicated '%s' section" % kind)
self.since = section
self.sections.append(section)
self.all_sections.append(section)
- def _new_description(self, info: QAPISourceInfo, name: str,
- desc: Dict[str, ArgSection]) -> None:
+ def _new_description(
+ self,
+ info: QAPISourceInfo,
+ name: str,
+ kind: 'QAPIDoc.Kind',
+ desc: Dict[str, ArgSection]
+ ) -> None:
if not name:
raise QAPISemError(info, "invalid parameter name")
if name in desc:
raise QAPISemError(info, "'%s' parameter name duplicated" % name)
- section = self.ArgSection(info, '@' + name)
+ section = self.ArgSection(info, kind, name)
self.all_sections.append(section)
desc[name] = section
def new_argument(self, info: QAPISourceInfo, name: str) -> None:
- self._new_description(info, name, self.args)
+ self._new_description(info, name, QAPIDoc.Kind.MEMBER, self.args)
def new_feature(self, info: QAPISourceInfo, name: str) -> None:
- self._new_description(info, name, self.features)
+ self._new_description(info, name, QAPIDoc.Kind.FEATURE, self.features)
def append_line(self, line: str) -> None:
self.all_sections[-1].append_line(line)
@@ -744,8 +794,9 @@ def connect_member(self, member: 'QAPISchemaMember') -> None:
raise QAPISemError(member.info,
"%s '%s' lacks documentation"
% (member.role, member.name))
- self.args[member.name] = QAPIDoc.ArgSection(
- self.info, '@' + member.name)
+ section = QAPIDoc.ArgSection(
+ self.info, QAPIDoc.Kind.MEMBER, member.name)
+ self.args[member.name] = section
self.args[member.name].connect(member)
def connect_feature(self, feature: 'QAPISchemaFeature') -> None:
diff --git a/tests/qapi-schema/doc-good.out b/tests/qapi-schema/doc-good.out
index 0a9da3efdeb..5773f1dd6d6 100644
--- a/tests/qapi-schema/doc-good.out
+++ b/tests/qapi-schema/doc-good.out
@@ -113,7 +113,7 @@ The _one_ {and only}, description on the same line
Also _one_ {and only}
feature=enum-member-feat
a member feature
- section=None
+ section=Plain
@two is undocumented
doc symbol=Base
body=
@@ -171,15 +171,15 @@ description starts on the same line
a feature
feature=cmd-feat2
another feature
- section=None
+ section=Plain
.. note:: @arg3 is undocumented
section=Returns
@Object
section=Errors
some
- section=TODO
+ section=Todo
frobnicate
- section=None
+ section=Plain
.. admonition:: Notes
- Lorem ipsum dolor sit amet
@@ -212,7 +212,7 @@ If you're bored enough to read this, go see a video of boxed cats
a feature
feature=cmd-feat2
another feature
- section=None
+ section=Plain
.. qmp-example::
-> "this example"
diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py
index 7e3f9f4aa1f..bca924309be 100755
--- a/tests/qapi-schema/test-qapi.py
+++ b/tests/qapi-schema/test-qapi.py
@@ -131,7 +131,7 @@ def test_frontend(fname):
for feat, section in doc.features.items():
print(' feature=%s\n%s' % (feat, section.text))
for section in doc.sections:
- print(' section=%s\n%s' % (section.tag, section.text))
+ print(' section=%s\n%s' % (section.kind, section.text))
def open_test_result(dir_name, file_name, update):
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 31/57] qapi: expand tags to all doc sections
2025-03-05 3:45 ` [PATCH 31/57] qapi: expand tags to all doc sections John Snow
@ 2025-03-05 10:16 ` Markus Armbruster
2025-03-06 3:58 ` John Snow
0 siblings, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-05 10:16 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
Replaying review of a previous posting for your convenience...
John Snow <jsnow@redhat.com> writes:
> This patch adds an explicit section "kind" to all QAPIDoc
> sections. Members/Features are now explicitly marked as such, with the
> name now being stored in a dedicated "name" field (which qapidoc.py was
> not actually using anyway.)
I'm not sure what the parenthesis is trying to convey.
Before the patch, we have:
type tag
untagged Section None
@foo: ArgSection 'foo'
Returns: Section 'Returns'
Errors: Section 'Errors'
Since: Section 'Since'
TODO: Section 'TODO'
Afterwards, I believe:
type kind name
untagged Section PLAIN
@foo: ArgSection MEMBER 'foo' if member or argument
ArgSection FEATURE 'foo' if feature
Returns: Section RETURNS
Errors: Section ERRORS
Since: Section SINCE
TODO: Section TODO
So, .tag is replaced by .kind and .name, member vs. feature vs. other
tags is now obvious from .kind alone, i.e. there's no need to account
for context or type.
Fine print: why do we need to account for type before the patch?
Consider @Since: ...
> The qapi-schema tests are updated to account for the new section names;
> mostly "TODO" becomes "Todo" and `None` becomes "Plain".
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> docs/sphinx/qapidoc.py | 7 ++-
> scripts/qapi/parser.py | 109 ++++++++++++++++++++++++---------
> tests/qapi-schema/doc-good.out | 10 +--
> tests/qapi-schema/test-qapi.py | 2 +-
> 4 files changed, 90 insertions(+), 38 deletions(-)
>
> diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> index 61997fd21af..d622398f1da 100644
> --- a/docs/sphinx/qapidoc.py
> +++ b/docs/sphinx/qapidoc.py
> @@ -35,6 +35,7 @@
> from docutils.statemachine import ViewList
> from qapi.error import QAPIError, QAPISemError
> from qapi.gen import QAPISchemaVisitor
> +from qapi.parser import QAPIDoc
> from qapi.schema import QAPISchema
>
> from sphinx import addnodes
> @@ -258,11 +259,11 @@ def _nodes_for_sections(self, doc):
> """Return list of doctree nodes for additional sections"""
> nodelist = []
> for section in doc.sections:
> - if section.tag and section.tag == 'TODO':
> + if section.kind == QAPIDoc.Kind.TODO:
> # Hide TODO: sections
> continue
>
> - if not section.tag:
> + if section.kind == QAPIDoc.Kind.PLAIN:
> # Sphinx cannot handle sectionless titles;
> # Instead, just append the results to the prior section.
> container = nodes.container()
> @@ -270,7 +271,7 @@ def _nodes_for_sections(self, doc):
> nodelist += container.children
> continue
>
> - snode = self._make_section(section.tag)
> + snode = self._make_section(section.kind.name.title())
> self._parse_text_into_node(dedent(section.text), snode)
> nodelist.append(snode)
> return nodelist
> diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
> index 36cb64a677a..c3004aa70c6 100644
> --- a/scripts/qapi/parser.py
> +++ b/scripts/qapi/parser.py
> @@ -15,6 +15,7 @@
> # See the COPYING file in the top-level directory.
>
> from collections import OrderedDict
> +import enum
> import os
> import re
> from typing import (
> @@ -575,7 +576,10 @@ def get_doc(self) -> 'QAPIDoc':
> )
> raise QAPIParseError(self, emsg)
>
> - doc.new_tagged_section(self.info, match.group(1))
> + doc.new_tagged_section(
> + self.info,
> + QAPIDoc.Kind.from_string(match.group(1))
> + )
> text = line[match.end():]
> if text:
> doc.append_line(text)
> @@ -586,7 +590,7 @@ def get_doc(self) -> 'QAPIDoc':
> self,
> "unexpected '=' markup in definition documentation")
> else:
> - # tag-less paragraph
> + # plain paragraph(s)
We're parsing a single pargraph here. The plain section we add it to
may have any number of paragraphs. But for me, the comment is about
what's being parsed. Mind to drop (s)?
> doc.ensure_untagged_section(self.info)
> doc.append_line(line)
> line = self.get_doc_paragraph(doc)
> @@ -635,14 +639,37 @@ class QAPIDoc:
> Free-form documentation blocks consist only of a body section.
> """
>
> + class Kind(enum.Enum):
> + PLAIN = 0
> + MEMBER = 1
> + FEATURE = 2
> + RETURNS = 3
> + ERRORS = 4
> + SINCE = 5
> + TODO = 6
> +
> + @staticmethod
> + def from_string(kind: str) -> 'QAPIDoc.Kind':
Remind me, why do we need to quote the type here?
> + return QAPIDoc.Kind[kind.upper()]
> +
> + def text_required(self) -> bool:
> + # Only "plain" sections can be empty
> + return self.value not in (0,)
Rather roundabout way to check for PLAIN, isn't it?
There's just one caller (see below). I doubt the method is worth its
keep.
> +
> + def __str__(self) -> str:
> + return self.name.title()
> +
I wonder whether a simple StrEnum without methods would do. Oh, StrEnum
is new in 3.11. Nevermind.
Hmm.
>>> Kind = Enum('Kind', [('PLAIN', 'Plain'), ('TODO, 'TODO)])
>>> kind=Kind('Plain')
>>> kind.value
'Plain'
What do you think?
> class Section:
> # pylint: disable=too-few-public-methods
> - def __init__(self, info: QAPISourceInfo,
> - tag: Optional[str] = None):
> + def __init__(
> + self,
> + info: QAPISourceInfo,
> + kind: 'QAPIDoc.Kind',
> + ):
> # section source info, i.e. where it begins
> self.info = info
> - # section tag, if any ('Returns', '@name', ...)
> - self.tag = tag
> + # section kind
> + self.kind = kind
> # section text without tag
> self.text = ''
>
> @@ -650,8 +677,14 @@ def append_line(self, line: str) -> None:
> self.text += line + '\n'
>
> class ArgSection(Section):
> - def __init__(self, info: QAPISourceInfo, tag: str):
> - super().__init__(info, tag)
> + def __init__(
> + self,
> + info: QAPISourceInfo,
> + kind: 'QAPIDoc.Kind',
> + name: str
> + ):
> + super().__init__(info, kind)
> + self.name = name
> self.member: Optional['QAPISchemaMember'] = None
Before the patch, use of a separate type for members, arguments and
features was necessary to distinguish between '@TAG:' and 'TAG:' for the
various TAGs. This is no longer the case. Fold ArgSection into
Section? Not sure. If yes, separate patch to keep this one as
mechanical as possible.
>
> def connect(self, member: 'QAPISchemaMember') -> None:
> @@ -663,7 +696,9 @@ def __init__(self, info: QAPISourceInfo, symbol: Optional[str] = None):
> # definition doc's symbol, None for free-form doc
> self.symbol: Optional[str] = symbol
> # the sections in textual order
> - self.all_sections: List[QAPIDoc.Section] = [QAPIDoc.Section(info)]
> + self.all_sections: List[QAPIDoc.Section] = [
> + QAPIDoc.Section(info, QAPIDoc.Kind.PLAIN)
> + ]
> # the body section
> self.body: Optional[QAPIDoc.Section] = self.all_sections[0]
> # dicts mapping parameter/feature names to their description
> @@ -680,12 +715,17 @@ def __init__(self, info: QAPISourceInfo, symbol: Optional[str] = None):
> def end(self) -> None:
> for section in self.all_sections:
> section.text = section.text.strip('\n')
> - if section.tag is not None and section.text == '':
> + if section.kind.text_required() and section.text == '':
This is the only use of .text_required(). I believe checking for PLAIN
would be clearer.
> raise QAPISemError(
> - section.info, "text required after '%s:'" % section.tag)
> + section.info, "text required after '%s:'" % section.kind)
>
> - def ensure_untagged_section(self, info: QAPISourceInfo) -> None:
> - if self.all_sections and not self.all_sections[-1].tag:
> + def ensure_untagged_section(
> + self,
> + info: QAPISourceInfo,
> + ) -> None:
Accidental line breaking?
> + kind = QAPIDoc.Kind.PLAIN
> +
> + if self.all_sections and self.all_sections[-1].kind == kind:
I'd prefer not to hide PLAIN behind a variable, but I'd also prefer
the condition to fit on a line. Hmm.
> # extend current section
> section = self.all_sections[-1]
> if not section.text:
Maybe
section = self.all_sections[-1] if self.all_sections else None
if second and section.kind = QAPIDoc.Kind.Plain:
# extend current section
if not section.text:
> @@ -693,46 +733,56 @@ def ensure_untagged_section(self, info: QAPISourceInfo) -> None:
> section.info = info
> section.text += '\n'
> return
> +
> # start new section
> - section = self.Section(info)
> + section = self.Section(info, kind)
> self.sections.append(section)
> self.all_sections.append(section)
>
> - def new_tagged_section(self, info: QAPISourceInfo, tag: str) -> None:
> - section = self.Section(info, tag)
> - if tag == 'Returns':
> + def new_tagged_section(
> + self,
> + info: QAPISourceInfo,
> + kind: 'QAPIDoc.Kind',
> + ) -> None:
> + section = self.Section(info, kind)
> + if kind == QAPIDoc.Kind.RETURNS:
> if self.returns:
> raise QAPISemError(
> - info, "duplicated '%s' section" % tag)
> + info, "duplicated '%s' section" % kind)
> self.returns = section
> - elif tag == 'Errors':
> + elif kind == QAPIDoc.Kind.ERRORS:
> if self.errors:
> raise QAPISemError(
> - info, "duplicated '%s' section" % tag)
> + info, "duplicated '%s' section" % kind)
> self.errors = section
> - elif tag == 'Since':
> + elif kind == QAPIDoc.Kind.SINCE:
> if self.since:
> raise QAPISemError(
> - info, "duplicated '%s' section" % tag)
> + info, "duplicated '%s' section" % kind)
> self.since = section
> self.sections.append(section)
> self.all_sections.append(section)
>
> - def _new_description(self, info: QAPISourceInfo, name: str,
> - desc: Dict[str, ArgSection]) -> None:
> + def _new_description(
> + self,
> + info: QAPISourceInfo,
> + name: str,
> + kind: 'QAPIDoc.Kind',
> + desc: Dict[str, ArgSection]
> + ) -> None:
> if not name:
> raise QAPISemError(info, "invalid parameter name")
> if name in desc:
> raise QAPISemError(info, "'%s' parameter name duplicated" % name)
> - section = self.ArgSection(info, '@' + name)
> + section = self.ArgSection(info, kind, name)
> self.all_sections.append(section)
> desc[name] = section
>
> def new_argument(self, info: QAPISourceInfo, name: str) -> None:
> - self._new_description(info, name, self.args)
> + self._new_description(info, name, QAPIDoc.Kind.MEMBER, self.args)
>
> def new_feature(self, info: QAPISourceInfo, name: str) -> None:
> - self._new_description(info, name, self.features)
> + self._new_description(info, name, QAPIDoc.Kind.FEATURE, self.features)
QAPIDoc.Kind.FOO is a mouthful, and it tends to result in long lines,
like here. Can't see an easy and clean way to reduce the verbosity.
>
> def append_line(self, line: str) -> None:
> self.all_sections[-1].append_line(line)
> @@ -744,8 +794,9 @@ def connect_member(self, member: 'QAPISchemaMember') -> None:
> raise QAPISemError(member.info,
> "%s '%s' lacks documentation"
> % (member.role, member.name))
> - self.args[member.name] = QAPIDoc.ArgSection(
> - self.info, '@' + member.name)
> + section = QAPIDoc.ArgSection(
> + self.info, QAPIDoc.Kind.MEMBER, member.name)
> + self.args[member.name] = section
Why the extra variable?
> self.args[member.name].connect(member)
>
> def connect_feature(self, feature: 'QAPISchemaFeature') -> None:
> diff --git a/tests/qapi-schema/doc-good.out b/tests/qapi-schema/doc-good.out
> index 0a9da3efdeb..5773f1dd6d6 100644
> --- a/tests/qapi-schema/doc-good.out
> +++ b/tests/qapi-schema/doc-good.out
> @@ -113,7 +113,7 @@ The _one_ {and only}, description on the same line
> Also _one_ {and only}
> feature=enum-member-feat
> a member feature
> - section=None
> + section=Plain
> @two is undocumented
> doc symbol=Base
> body=
> @@ -171,15 +171,15 @@ description starts on the same line
> a feature
> feature=cmd-feat2
> another feature
> - section=None
> + section=Plain
> .. note:: @arg3 is undocumented
> section=Returns
> @Object
> section=Errors
> some
> - section=TODO
> + section=Todo
With the method-less Enum I suggested, this hunk would go away. Not
that it matters :)
> frobnicate
> - section=None
> + section=Plain
> .. admonition:: Notes
>
> - Lorem ipsum dolor sit amet
> @@ -212,7 +212,7 @@ If you're bored enough to read this, go see a video of boxed cats
> a feature
> feature=cmd-feat2
> another feature
> - section=None
> + section=Plain
> .. qmp-example::
>
> -> "this example"
> diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py
> index 7e3f9f4aa1f..bca924309be 100755
> --- a/tests/qapi-schema/test-qapi.py
> +++ b/tests/qapi-schema/test-qapi.py
> @@ -131,7 +131,7 @@ def test_frontend(fname):
> for feat, section in doc.features.items():
> print(' feature=%s\n%s' % (feat, section.text))
> for section in doc.sections:
> - print(' section=%s\n%s' % (section.tag, section.text))
> + print(' section=%s\n%s' % (section.kind, section.text))
>
>
> def open_test_result(dir_name, file_name, update):
^ permalink raw reply [flat|nested] 129+ messages in thread* Re: [PATCH 31/57] qapi: expand tags to all doc sections
2025-03-05 10:16 ` Markus Armbruster
@ 2025-03-06 3:58 ` John Snow
2025-03-06 7:42 ` Markus Armbruster
0 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-06 3:58 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 17710 bytes --]
On Wed, Mar 5, 2025 at 5:16 AM Markus Armbruster <armbru@redhat.com> wrote:
> Replaying review of a previous posting for your convenience...
>
> John Snow <jsnow@redhat.com> writes:
>
> > This patch adds an explicit section "kind" to all QAPIDoc
> > sections. Members/Features are now explicitly marked as such, with the
> > name now being stored in a dedicated "name" field (which qapidoc.py was
> > not actually using anyway.)
>
> I'm not sure what the parenthesis is trying to convey.
>
The old qapidoc.py doesn't actually use the name field, so there's nothing
to adjust for old callers.
>
> Before the patch, we have:
>
> type tag
> untagged Section None
> @foo: ArgSection 'foo'
> Returns: Section 'Returns'
> Errors: Section 'Errors'
> Since: Section 'Since'
> TODO: Section 'TODO'
>
> Afterwards, I believe:
>
> type kind name
> untagged Section PLAIN
> @foo: ArgSection MEMBER 'foo' if member or argument
> ArgSection FEATURE 'foo' if feature
> Returns: Section RETURNS
> Errors: Section ERRORS
> Since: Section SINCE
> TODO: Section TODO
>
> So, .tag is replaced by .kind and .name, member vs. feature vs. other
> tags is now obvious from .kind alone, i.e. there's no need to account
> for context or type.
>
> Fine print: why do we need to account for type before the patch?
> Consider @Since: ...
>
I'm not sure I follow...
>
> > The qapi-schema tests are updated to account for the new section names;
> > mostly "TODO" becomes "Todo" and `None` becomes "Plain".
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
> > ---
> > docs/sphinx/qapidoc.py | 7 ++-
> > scripts/qapi/parser.py | 109 ++++++++++++++++++++++++---------
> > tests/qapi-schema/doc-good.out | 10 +--
> > tests/qapi-schema/test-qapi.py | 2 +-
> > 4 files changed, 90 insertions(+), 38 deletions(-)
> >
> > diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> > index 61997fd21af..d622398f1da 100644
> > --- a/docs/sphinx/qapidoc.py
> > +++ b/docs/sphinx/qapidoc.py
> > @@ -35,6 +35,7 @@
> > from docutils.statemachine import ViewList
> > from qapi.error import QAPIError, QAPISemError
> > from qapi.gen import QAPISchemaVisitor
> > +from qapi.parser import QAPIDoc
> > from qapi.schema import QAPISchema
> >
> > from sphinx import addnodes
> > @@ -258,11 +259,11 @@ def _nodes_for_sections(self, doc):
> > """Return list of doctree nodes for additional sections"""
> > nodelist = []
> > for section in doc.sections:
> > - if section.tag and section.tag == 'TODO':
> > + if section.kind == QAPIDoc.Kind.TODO:
> > # Hide TODO: sections
> > continue
> >
> > - if not section.tag:
> > + if section.kind == QAPIDoc.Kind.PLAIN:
> > # Sphinx cannot handle sectionless titles;
> > # Instead, just append the results to the prior section.
> > container = nodes.container()
> > @@ -270,7 +271,7 @@ def _nodes_for_sections(self, doc):
> > nodelist += container.children
> > continue
> >
> > - snode = self._make_section(section.tag)
> > + snode = self._make_section(section.kind.name.title())
> > self._parse_text_into_node(dedent(section.text), snode)
> > nodelist.append(snode)
> > return nodelist
> > diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
> > index 36cb64a677a..c3004aa70c6 100644
> > --- a/scripts/qapi/parser.py
> > +++ b/scripts/qapi/parser.py
> > @@ -15,6 +15,7 @@
> > # See the COPYING file in the top-level directory.
> >
> > from collections import OrderedDict
> > +import enum
> > import os
> > import re
> > from typing import (
> > @@ -575,7 +576,10 @@ def get_doc(self) -> 'QAPIDoc':
> > )
> > raise QAPIParseError(self, emsg)
> >
> > - doc.new_tagged_section(self.info, match.group(1))
> > + doc.new_tagged_section(
> > + self.info,
> > + QAPIDoc.Kind.from_string(match.group(1))
> > + )
> > text = line[match.end():]
> > if text:
> > doc.append_line(text)
> > @@ -586,7 +590,7 @@ def get_doc(self) -> 'QAPIDoc':
> > self,
> > "unexpected '=' markup in definition
> documentation")
> > else:
> > - # tag-less paragraph
> > + # plain paragraph(s)
>
> We're parsing a single pargraph here. The plain section we add it to
> may have any number of paragraphs. But for me, the comment is about
> what's being parsed. Mind to drop (s)?
>
Anguish. I can't keep this straight in my head. OK. It wasn't obvious at a
glance where the break is if we get an empty newline ...
>
> > doc.ensure_untagged_section(self.info)
> > doc.append_line(line)
> > line = self.get_doc_paragraph(doc)
> > @@ -635,14 +639,37 @@ class QAPIDoc:
> > Free-form documentation blocks consist only of a body section.
> > """
> >
> > + class Kind(enum.Enum):
> > + PLAIN = 0
> > + MEMBER = 1
> > + FEATURE = 2
> > + RETURNS = 3
> > + ERRORS = 4
> > + SINCE = 5
> > + TODO = 6
> > +
> > + @staticmethod
> > + def from_string(kind: str) -> 'QAPIDoc.Kind':
>
> Remind me, why do we need to quote the type here?
>
It doesn't exist yet; it's a forward reference, basically. While we are in
the context of defining the class, we don't have access to variables scoped
within the class :)
>
> > + return QAPIDoc.Kind[kind.upper()]
> > +
> > + def text_required(self) -> bool:
> > + # Only "plain" sections can be empty
> > + return self.value not in (0,)
>
> Rather roundabout way to check for PLAIN, isn't it?
>
Vestigial from intro/details split. It can be removed here for now.
>
> There's just one caller (see below). I doubt the method is worth its
> keep.
>
Vestigial again. Whether it's worth it once there are multiple such
sections, who knows. I thought it made the code read nicer in context. We
don't need it right now anyway...
>
> > +
> > + def __str__(self) -> str:
> > + return self.name.title()
> > +
>
> I wonder whether a simple StrEnum without methods would do. Oh, StrEnum
> is new in 3.11. Nevermind.
>
> Hmm.
>
> >>> Kind = Enum('Kind', [('PLAIN', 'Plain'), ('TODO, 'TODO)])
> >>> kind=Kind('Plain')
> >>> kind.value
> 'Plain'
>
> What do you think?
>
Maybe, lemme play with it and see if it makes something else worse, I don't
know right away.
>
> > class Section:
> > # pylint: disable=too-few-public-methods
> > - def __init__(self, info: QAPISourceInfo,
> > - tag: Optional[str] = None):
> > + def __init__(
> > + self,
> > + info: QAPISourceInfo,
> > + kind: 'QAPIDoc.Kind',
> > + ):
> > # section source info, i.e. where it begins
> > self.info = info
> > - # section tag, if any ('Returns', '@name', ...)
> > - self.tag = tag
> > + # section kind
> > + self.kind = kind
> > # section text without tag
> > self.text = ''
> >
> > @@ -650,8 +677,14 @@ def append_line(self, line: str) -> None:
> > self.text += line + '\n'
> >
> > class ArgSection(Section):
> > - def __init__(self, info: QAPISourceInfo, tag: str):
> > - super().__init__(info, tag)
> > + def __init__(
> > + self,
> > + info: QAPISourceInfo,
> > + kind: 'QAPIDoc.Kind',
> > + name: str
> > + ):
> > + super().__init__(info, kind)
> > + self.name = name
> > self.member: Optional['QAPISchemaMember'] = None
>
> Before the patch, use of a separate type for members, arguments and
> features was necessary to distinguish between '@TAG:' and 'TAG:' for the
> various TAGs. This is no longer the case. Fold ArgSection into
> Section? Not sure. If yes, separate patch to keep this one as
> mechanical as possible.
>
Possibly the case. I'll play with it. Do you want it in this series? (It's
pretty long as is...!)
>
> >
> > def connect(self, member: 'QAPISchemaMember') -> None:
> > @@ -663,7 +696,9 @@ def __init__(self, info: QAPISourceInfo, symbol:
> Optional[str] = None):
> > # definition doc's symbol, None for free-form doc
> > self.symbol: Optional[str] = symbol
> > # the sections in textual order
> > - self.all_sections: List[QAPIDoc.Section] =
> [QAPIDoc.Section(info)]
> > + self.all_sections: List[QAPIDoc.Section] = [
> > + QAPIDoc.Section(info, QAPIDoc.Kind.PLAIN)
> > + ]
> > # the body section
> > self.body: Optional[QAPIDoc.Section] = self.all_sections[0]
> > # dicts mapping parameter/feature names to their description
> > @@ -680,12 +715,17 @@ def __init__(self, info: QAPISourceInfo, symbol:
> Optional[str] = None):
> > def end(self) -> None:
> > for section in self.all_sections:
> > section.text = section.text.strip('\n')
> > - if section.tag is not None and section.text == '':
> > + if section.kind.text_required() and section.text == '':
>
> This is the only use of .text_required(). I believe checking for PLAIN
> would be clearer.
>
> > raise QAPISemError(
> > - section.info, "text required after '%s:'" %
> section.tag)
> > + section.info, "text required after '%s:'" %
> section.kind)
> >
> > - def ensure_untagged_section(self, info: QAPISourceInfo) -> None:
> > - if self.all_sections and not self.all_sections[-1].tag:
> > + def ensure_untagged_section(
> > + self,
> > + info: QAPISourceInfo,
> > + ) -> None:
>
> Accidental line breaking?
>
Something something something black autoformatter. I wonder why it did
this, though... I'll see if I can undo it.
>
> > + kind = QAPIDoc.Kind.PLAIN
> > +
> > + if self.all_sections and self.all_sections[-1].kind == kind:
>
> I'd prefer not to hide PLAIN behind a variable, but I'd also prefer
> the condition to fit on a line. Hmm.
>
Also somewhat vestigial from when I had intro/details. When that split
comes, kind = becomes a conditional ternary.
>
> > # extend current section
> > section = self.all_sections[-1]
> > if not section.text:
>
> Maybe
>
> section = self.all_sections[-1] if self.all_sections else None
>
> if second and section.kind = QAPIDoc.Kind.Plain:
> # extend current section
> if not section.text:
>
Could, but it's going to get rewritten when the inliner comes anyway, ...
>
> > @@ -693,46 +733,56 @@ def ensure_untagged_section(self, info:
> QAPISourceInfo) -> None:
> > section.info = info
> > section.text += '\n'
> > return
> > +
> > # start new section
> > - section = self.Section(info)
> > + section = self.Section(info, kind)
> > self.sections.append(section)
> > self.all_sections.append(section)
> >
> > - def new_tagged_section(self, info: QAPISourceInfo, tag: str) ->
> None:
> > - section = self.Section(info, tag)
> > - if tag == 'Returns':
> > + def new_tagged_section(
> > + self,
> > + info: QAPISourceInfo,
> > + kind: 'QAPIDoc.Kind',
> > + ) -> None:
> > + section = self.Section(info, kind)
> > + if kind == QAPIDoc.Kind.RETURNS:
> > if self.returns:
> > raise QAPISemError(
> > - info, "duplicated '%s' section" % tag)
> > + info, "duplicated '%s' section" % kind)
> > self.returns = section
> > - elif tag == 'Errors':
> > + elif kind == QAPIDoc.Kind.ERRORS:
> > if self.errors:
> > raise QAPISemError(
> > - info, "duplicated '%s' section" % tag)
> > + info, "duplicated '%s' section" % kind)
> > self.errors = section
> > - elif tag == 'Since':
> > + elif kind == QAPIDoc.Kind.SINCE:
> > if self.since:
> > raise QAPISemError(
> > - info, "duplicated '%s' section" % tag)
> > + info, "duplicated '%s' section" % kind)
> > self.since = section
> > self.sections.append(section)
> > self.all_sections.append(section)
> >
> > - def _new_description(self, info: QAPISourceInfo, name: str,
> > - desc: Dict[str, ArgSection]) -> None:
> > + def _new_description(
> > + self,
> > + info: QAPISourceInfo,
> > + name: str,
> > + kind: 'QAPIDoc.Kind',
> > + desc: Dict[str, ArgSection]
> > + ) -> None:
> > if not name:
> > raise QAPISemError(info, "invalid parameter name")
> > if name in desc:
> > raise QAPISemError(info, "'%s' parameter name duplicated" %
> name)
> > - section = self.ArgSection(info, '@' + name)
> > + section = self.ArgSection(info, kind, name)
> > self.all_sections.append(section)
> > desc[name] = section
> >
> > def new_argument(self, info: QAPISourceInfo, name: str) -> None:
> > - self._new_description(info, name, self.args)
> > + self._new_description(info, name, QAPIDoc.Kind.MEMBER,
> self.args)
> >
> > def new_feature(self, info: QAPISourceInfo, name: str) -> None:
> > - self._new_description(info, name, self.features)
> > + self._new_description(info, name, QAPIDoc.Kind.FEATURE,
> self.features)
>
> QAPIDoc.Kind.FOO is a mouthful, and it tends to result in long lines,
> like here. Can't see an easy and clean way to reduce the verbosity.
>
Yeah...
>
> >
> > def append_line(self, line: str) -> None:
> > self.all_sections[-1].append_line(line)
> > @@ -744,8 +794,9 @@ def connect_member(self, member: 'QAPISchemaMember')
> -> None:
> > raise QAPISemError(member.info,
> > "%s '%s' lacks documentation"
> > % (member.role, member.name))
> > - self.args[member.name] = QAPIDoc.ArgSection(
> > - self.info, '@' + member.name)
> > + section = QAPIDoc.ArgSection(
> > + self.info, QAPIDoc.Kind.MEMBER, member.name)
> > + self.args[member.name] = section
>
> Why the extra variable?
>
Wish I remembered. I can undo it and see if anything barks.
>
> > self.args[member.name].connect(member)
> >
> > def connect_feature(self, feature: 'QAPISchemaFeature') -> None:
> > diff --git a/tests/qapi-schema/doc-good.out
> b/tests/qapi-schema/doc-good.out
> > index 0a9da3efdeb..5773f1dd6d6 100644
> > --- a/tests/qapi-schema/doc-good.out
> > +++ b/tests/qapi-schema/doc-good.out
> > @@ -113,7 +113,7 @@ The _one_ {and only}, description on the same line
> > Also _one_ {and only}
> > feature=enum-member-feat
> > a member feature
> > - section=None
> > + section=Plain
> > @two is undocumented
> > doc symbol=Base
> > body=
> > @@ -171,15 +171,15 @@ description starts on the same line
> > a feature
> > feature=cmd-feat2
> > another feature
> > - section=None
> > + section=Plain
> > .. note:: @arg3 is undocumented
> > section=Returns
> > @Object
> > section=Errors
> > some
> > - section=TODO
> > + section=Todo
>
> With the method-less Enum I suggested, this hunk would go away. Not
> that it matters :)
>
> > frobnicate
> > - section=None
> > + section=Plain
> > .. admonition:: Notes
> >
> > - Lorem ipsum dolor sit amet
> > @@ -212,7 +212,7 @@ If you're bored enough to read this, go see a video
> of boxed cats
> > a feature
> > feature=cmd-feat2
> > another feature
> > - section=None
> > + section=Plain
> > .. qmp-example::
> >
> > -> "this example"
> > diff --git a/tests/qapi-schema/test-qapi.py
> b/tests/qapi-schema/test-qapi.py
> > index 7e3f9f4aa1f..bca924309be 100755
> > --- a/tests/qapi-schema/test-qapi.py
> > +++ b/tests/qapi-schema/test-qapi.py
> > @@ -131,7 +131,7 @@ def test_frontend(fname):
> > for feat, section in doc.features.items():
> > print(' feature=%s\n%s' % (feat, section.text))
> > for section in doc.sections:
> > - print(' section=%s\n%s' % (section.tag, section.text))
> > + print(' section=%s\n%s' % (section.kind, section.text))
> >
> >
> > def open_test_result(dir_name, file_name, update):
>
>
[-- Attachment #2: Type: text/html, Size: 25512 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread* Re: [PATCH 31/57] qapi: expand tags to all doc sections
2025-03-06 3:58 ` John Snow
@ 2025-03-06 7:42 ` Markus Armbruster
0 siblings, 0 replies; 129+ messages in thread
From: Markus Armbruster @ 2025-03-06 7:42 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> On Wed, Mar 5, 2025 at 5:16 AM Markus Armbruster <armbru@redhat.com> wrote:
>
>> Replaying review of a previous posting for your convenience...
>>
>> John Snow <jsnow@redhat.com> writes:
>>
>> > This patch adds an explicit section "kind" to all QAPIDoc
>> > sections. Members/Features are now explicitly marked as such, with the
>> > name now being stored in a dedicated "name" field (which qapidoc.py was
>> > not actually using anyway.)
>>
>> I'm not sure what the parenthesis is trying to convey.
>>
>
> The old qapidoc.py doesn't actually use the name field, so there's nothing
> to adjust for old callers.
>
>
>>
>> Before the patch, we have:
>>
>> type tag
>> untagged Section None
>> @foo: ArgSection 'foo'
>> Returns: Section 'Returns'
>> Errors: Section 'Errors'
>> Since: Section 'Since'
>> TODO: Section 'TODO'
>>
>> Afterwards, I believe:
>>
>> type kind name
>> untagged Section PLAIN
>> @foo: ArgSection MEMBER 'foo' if member or argument
>> ArgSection FEATURE 'foo' if feature
>> Returns: Section RETURNS
>> Errors: Section ERRORS
>> Since: Section SINCE
>> TODO: Section TODO
>>
>> So, .tag is replaced by .kind and .name, member vs. feature vs. other
>> tags is now obvious from .kind alone, i.e. there's no need to account
>> for context or type.
>>
>> Fine print: why do we need to account for type before the patch?
>> Consider @Since: ...
>>
>
> I'm not sure I follow...
Consider
##
# @some-command
#
# @Since: an anti-socially named argument
#
# Since: 10.0
##
Before the patch, both the argument description and the Since section
have tag 'Since'. The former has type ArgSection, which is a subtype of
Section. The latter has type Section. This is how we tell them apart.
There's potential for confusion and misuse of .tag.
After the patch, they have different kinds.
Do you follow now?
>> > The qapi-schema tests are updated to account for the new section names;
>> > mostly "TODO" becomes "Todo" and `None` becomes "Plain".
>> >
>> > Signed-off-by: John Snow <jsnow@redhat.com>
>> > ---
>> > docs/sphinx/qapidoc.py | 7 ++-
>> > scripts/qapi/parser.py | 109 ++++++++++++++++++++++++---------
>> > tests/qapi-schema/doc-good.out | 10 +--
>> > tests/qapi-schema/test-qapi.py | 2 +-
>> > 4 files changed, 90 insertions(+), 38 deletions(-)
>> >
>> > diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
>> > index 61997fd21af..d622398f1da 100644
>> > --- a/docs/sphinx/qapidoc.py
>> > +++ b/docs/sphinx/qapidoc.py
>> > @@ -35,6 +35,7 @@
>> > from docutils.statemachine import ViewList
>> > from qapi.error import QAPIError, QAPISemError
>> > from qapi.gen import QAPISchemaVisitor
>> > +from qapi.parser import QAPIDoc
>> > from qapi.schema import QAPISchema
>> >
>> > from sphinx import addnodes
>> > @@ -258,11 +259,11 @@ def _nodes_for_sections(self, doc):
>> > """Return list of doctree nodes for additional sections"""
>> > nodelist = []
>> > for section in doc.sections:
>> > - if section.tag and section.tag == 'TODO':
>> > + if section.kind == QAPIDoc.Kind.TODO:
>> > # Hide TODO: sections
>> > continue
>> >
>> > - if not section.tag:
>> > + if section.kind == QAPIDoc.Kind.PLAIN:
>> > # Sphinx cannot handle sectionless titles;
>> > # Instead, just append the results to the prior section.
>> > container = nodes.container()
>> > @@ -270,7 +271,7 @@ def _nodes_for_sections(self, doc):
>> > nodelist += container.children
>> > continue
>> >
>> > - snode = self._make_section(section.tag)
>> > + snode = self._make_section(section.kind.name.title())
>> > self._parse_text_into_node(dedent(section.text), snode)
>> > nodelist.append(snode)
>> > return nodelist
>> > diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
>> > index 36cb64a677a..c3004aa70c6 100644
>> > --- a/scripts/qapi/parser.py
>> > +++ b/scripts/qapi/parser.py
>> > @@ -15,6 +15,7 @@
>> > # See the COPYING file in the top-level directory.
>> >
>> > from collections import OrderedDict
>> > +import enum
>> > import os
>> > import re
>> > from typing import (
>> > @@ -575,7 +576,10 @@ def get_doc(self) -> 'QAPIDoc':
>> > )
>> > raise QAPIParseError(self, emsg)
>> >
>> > - doc.new_tagged_section(self.info, match.group(1))
>> > + doc.new_tagged_section(
>> > + self.info,
>> > + QAPIDoc.Kind.from_string(match.group(1))
>> > + )
>> > text = line[match.end():]
>> > if text:
>> > doc.append_line(text)
>> > @@ -586,7 +590,7 @@ def get_doc(self) -> 'QAPIDoc':
>> > self,
>> > "unexpected '=' markup in definition documentation")
>> > else:
>> > - # tag-less paragraph
>> > + # plain paragraph(s)
>>
>> We're parsing a single pargraph here. The plain section we add it to
>> may have any number of paragraphs. But for me, the comment is about
>> what's being parsed. Mind to drop (s)?
>>
>
> Anguish. I can't keep this straight in my head. OK. It wasn't obvious at a
> glance where the break is if we get an empty newline ...
The doc parsing code is more subtle than I'd like it to be.
>> > doc.ensure_untagged_section(self.info)
>> > doc.append_line(line)
>> > line = self.get_doc_paragraph(doc)
>> > @@ -635,14 +639,37 @@ class QAPIDoc:
>> > Free-form documentation blocks consist only of a body section.
>> > """
>> >
>> > + class Kind(enum.Enum):
>> > + PLAIN = 0
>> > + MEMBER = 1
>> > + FEATURE = 2
>> > + RETURNS = 3
>> > + ERRORS = 4
>> > + SINCE = 5
>> > + TODO = 6
>> > +
>> > + @staticmethod
>> > + def from_string(kind: str) -> 'QAPIDoc.Kind':
>>
>> Remind me, why do we need to quote the type here?
>>
>
> It doesn't exist yet; it's a forward reference, basically. While we are in
> the context of defining the class, we don't have access to variables scoped
> within the class :)
Thanks! Sure this wasn't designed in the 70s?
>> > + return QAPIDoc.Kind[kind.upper()]
>> > +
>> > + def text_required(self) -> bool:
>> > + # Only "plain" sections can be empty
>> > + return self.value not in (0,)
>>
>> Rather roundabout way to check for PLAIN, isn't it?
>>
>
> Vestigial from intro/details split. It can be removed here for now.
>
>
>>
>> There's just one caller (see below). I doubt the method is worth its
>> keep.
>>
>
> Vestigial again. Whether it's worth it once there are multiple such
> sections, who knows. I thought it made the code read nicer in context. We
> don't need it right now anyway...
Let's drop it then.
>> > +
>> > + def __str__(self) -> str:
>> > + return self.name.title()
>> > +
>>
>> I wonder whether a simple StrEnum without methods would do. Oh, StrEnum
>> is new in 3.11. Nevermind.
>>
>> Hmm.
>>
>> >>> Kind = Enum('Kind', [('PLAIN', 'Plain'), ('TODO, 'TODO)])
>> >>> kind=Kind('Plain')
>> >>> kind.value
>> 'Plain'
>>
>> What do you think?
>>
>
> Maybe, lemme play with it and see if it makes something else worse, I don't
> know right away.
>
>
>>
>> > class Section:
>> > # pylint: disable=too-few-public-methods
>> > - def __init__(self, info: QAPISourceInfo,
>> > - tag: Optional[str] = None):
>> > + def __init__(
>> > + self,
>> > + info: QAPISourceInfo,
>> > + kind: 'QAPIDoc.Kind',
>> > + ):
>> > # section source info, i.e. where it begins
>> > self.info = info
>> > - # section tag, if any ('Returns', '@name', ...)
>> > - self.tag = tag
>> > + # section kind
>> > + self.kind = kind
>> > # section text without tag
>> > self.text = ''
>> >
>> > @@ -650,8 +677,14 @@ def append_line(self, line: str) -> None:
>> > self.text += line + '\n'
>> >
>> > class ArgSection(Section):
>> > - def __init__(self, info: QAPISourceInfo, tag: str):
>> > - super().__init__(info, tag)
>> > + def __init__(
>> > + self,
>> > + info: QAPISourceInfo,
>> > + kind: 'QAPIDoc.Kind',
>> > + name: str
>> > + ):
>> > + super().__init__(info, kind)
>> > + self.name = name
>> > self.member: Optional['QAPISchemaMember'] = None
>>
>> Before the patch, use of a separate type for members, arguments and
>> features was necessary to distinguish between '@TAG:' and 'TAG:' for the
>> various TAGs. This is no longer the case. Fold ArgSection into
>> Section? Not sure. If yes, separate patch to keep this one as
>> mechanical as possible.
>>
>
> Possibly the case. I'll play with it. Do you want it in this series? (It's
> pretty long as is...!)
Perfectly fine as a follow-up. We should resist mission-creep.
>> >
>> > def connect(self, member: 'QAPISchemaMember') -> None:
>> > @@ -663,7 +696,9 @@ def __init__(self, info: QAPISourceInfo, symbol: Optional[str] = None):
>> > # definition doc's symbol, None for free-form doc
>> > self.symbol: Optional[str] = symbol
>> > # the sections in textual order
>> > - self.all_sections: List[QAPIDoc.Section] = [QAPIDoc.Section(info)]
>> > + self.all_sections: List[QAPIDoc.Section] = [
>> > + QAPIDoc.Section(info, QAPIDoc.Kind.PLAIN)
>> > + ]
>> > # the body section
>> > self.body: Optional[QAPIDoc.Section] = self.all_sections[0]
>> > # dicts mapping parameter/feature names to their description
>> > @@ -680,12 +715,17 @@ def __init__(self, info: QAPISourceInfo, symbol: Optional[str] = None):
>> > def end(self) -> None:
>> > for section in self.all_sections:
>> > section.text = section.text.strip('\n')
>> > - if section.tag is not None and section.text == '':
>> > + if section.kind.text_required() and section.text == '':
>>
>> This is the only use of .text_required(). I believe checking for PLAIN
>> would be clearer.
>>
>> > raise QAPISemError(
>> > - section.info, "text required after '%s:'" % section.tag)
>> > + section.info, "text required after '%s:'" % section.kind)
>> >
>> > - def ensure_untagged_section(self, info: QAPISourceInfo) -> None:
>> > - if self.all_sections and not self.all_sections[-1].tag:
>> > + def ensure_untagged_section(
>> > + self,
>> > + info: QAPISourceInfo,
>> > + ) -> None:
>>
>> Accidental line breaking?
>>
>
> Something something something black autoformatter. I wonder why it did
> this, though... I'll see if I can undo it.
Yes, please.
>> > + kind = QAPIDoc.Kind.PLAIN
>> > +
>> > + if self.all_sections and self.all_sections[-1].kind == kind:
>>
>> I'd prefer not to hide PLAIN behind a variable, but I'd also prefer
>> the condition to fit on a line. Hmm.
>>
>
> Also somewhat vestigial from when I had intro/details. When that split
> comes, kind = becomes a conditional ternary.
>
>
>>
>> > # extend current section
>> > section = self.all_sections[-1]
>> > if not section.text:
>>
>> Maybe
>>
>> section = self.all_sections[-1] if self.all_sections else None
>>
>> if second and section.kind = QAPIDoc.Kind.Plain:
>> # extend current section
>> if not section.text:
>>
>
> Could, but it's going to get rewritten when the inliner comes anyway, ...
You decide.
[...]
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 32/57] qapi/schema: add __repr__ to QAPIDoc.Section
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (30 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 31/57] qapi: expand tags to all doc sections John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 33/57] docs/qapidoc: add transmogrifier stub John Snow
` (27 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Makes debugging far more pleasant when you can just print(section) and
get something reasonable to display.
Signed-off-by: John Snow <jsnow@redhat.com>
---
scripts/qapi/parser.py | 3 +++
1 file changed, 3 insertions(+)
diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index c3004aa70c6..af5d7bf892c 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -673,6 +673,9 @@ def __init__(
# section text without tag
self.text = ''
+ def __repr__(self) -> str:
+ return f"<QAPIDoc.Section kind={self.kind!r} text={self.text!r}>"
+
def append_line(self, line: str) -> None:
self.text += line + '\n'
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 33/57] docs/qapidoc: add transmogrifier stub
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (31 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 32/57] qapi/schema: add __repr__ to QAPIDoc.Section John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 34/57] docs/qapidoc: split old implementation into qapidoc_legacy.py John Snow
` (26 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
This commit adds a stubbed option to the qapi-doc directive that opts-in
to the new rST generator; the implementation of which will follow in
subsequent commits.
Once all QAPI documents have been converted, this option and the old
qapidoc implementation can be dropped.
Note that moving code outside of the try...except block has no impact
because the code moved outside of that block does not ever raise a
QAPIError.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapidoc.py | 41 ++++++++++++++++++++++++++++-------------
1 file changed, 28 insertions(+), 13 deletions(-)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index d622398f1da..dc72f3fd3f3 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -452,9 +452,9 @@ def _parse_text_into_node(self, doctext, node):
rstlist.append("", self._cur_doc.info.fname, self._cur_doc.info.line)
self._sphinx_directive.do_parse(rstlist, node)
- def get_document_nodes(self):
- """Return the list of docutils nodes which make up the document"""
- return self._top_node.children
+ def get_document_node(self):
+ """Return the root docutils node which makes up the document"""
+ return self._top_node
# Turn the black formatter on for the rest of the file.
@@ -503,7 +503,10 @@ class QAPIDocDirective(NestedDirective):
required_argument = 1
optional_arguments = 1
- option_spec = {"qapifile": directives.unchanged_required}
+ option_spec = {
+ "qapifile": directives.unchanged_required,
+ "transmogrify": directives.flag,
+ }
has_content = False
def new_serialno(self):
@@ -511,10 +514,24 @@ def new_serialno(self):
env = self.state.document.settings.env
return "qapidoc-%d" % env.new_serialno("qapidoc")
+ def transmogrify(self, schema) -> nodes.Element:
+ raise NotImplementedError
+
+ def legacy(self, schema) -> nodes.Element:
+ vis = QAPISchemaGenRSTVisitor(self)
+ vis.visit_begin(schema)
+ for doc in schema.docs:
+ if doc.symbol:
+ vis.symbol(doc, schema.lookup_entity(doc.symbol))
+ else:
+ vis.freeform(doc)
+ return vis.get_document_node()
+
def run(self):
env = self.state.document.settings.env
qapifile = env.config.qapidoc_srctree + "/" + self.arguments[0]
qapidir = os.path.dirname(qapifile)
+ transmogrify = "transmogrify" in self.options
try:
schema = QAPISchema(qapifile)
@@ -522,20 +539,18 @@ def run(self):
# First tell Sphinx about all the schema files that the
# output documentation depends on (including 'qapifile' itself)
schema.visit(QAPISchemaGenDepVisitor(env, qapidir))
-
- vis = QAPISchemaGenRSTVisitor(self)
- vis.visit_begin(schema)
- for doc in schema.docs:
- if doc.symbol:
- vis.symbol(doc, schema.lookup_entity(doc.symbol))
- else:
- vis.freeform(doc)
- return vis.get_document_nodes()
except QAPIError as err:
# Launder QAPI parse errors into Sphinx extension errors
# so they are displayed nicely to the user
raise ExtensionError(str(err)) from err
+ if transmogrify:
+ contentnode = self.transmogrify(schema)
+ else:
+ contentnode = self.legacy(schema)
+
+ return contentnode.children
+
class QMPExample(CodeBlock, NestedDirective):
"""
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 34/57] docs/qapidoc: split old implementation into qapidoc_legacy.py
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (32 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 33/57] docs/qapidoc: add transmogrifier stub John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 35/57] docs/qapidoc: Fix static typing on qapidoc.py John Snow
` (25 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
This is being done primarily to be able to type check and delint the new
implementation without needing to worry about fixing up the old
implementation.
I'm adding the new implementation into the existing file instead of into
a new file so that when the dust settles, qapidoc.py will contain the
full history of development on this generative module.
This patch *should* be pure motion, give or take the import statements.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapidoc.py | 420 +-------------------------------
docs/sphinx/qapidoc_legacy.py | 439 ++++++++++++++++++++++++++++++++++
2 files changed, 441 insertions(+), 418 deletions(-)
create mode 100644 docs/sphinx/qapidoc_legacy.py
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index dc72f3fd3f3..f4abf42e7bf 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -25,19 +25,16 @@
"""
import os
-import re
import sys
-import textwrap
from typing import List
from docutils import nodes
from docutils.parsers.rst import Directive, directives
-from docutils.statemachine import ViewList
-from qapi.error import QAPIError, QAPISemError
+from qapi.error import QAPIError
from qapi.gen import QAPISchemaVisitor
-from qapi.parser import QAPIDoc
from qapi.schema import QAPISchema
+from qapidoc_legacy import QAPISchemaGenRSTVisitor
from sphinx import addnodes
from sphinx.directives.code import CodeBlock
from sphinx.errors import ExtensionError
@@ -48,419 +45,6 @@
__version__ = "1.0"
-def dedent(text: str) -> str:
- # Adjust indentation to make description text parse as paragraph.
-
- lines = text.splitlines(True)
- if re.match(r"\s+", lines[0]):
- # First line is indented; description started on the line after
- # the name. dedent the whole block.
- return textwrap.dedent(text)
-
- # Descr started on same line. Dedent line 2+.
- return lines[0] + textwrap.dedent("".join(lines[1:]))
-
-
-# Disable black auto-formatter until re-enabled:
-# fmt: off
-
-
-class QAPISchemaGenRSTVisitor(QAPISchemaVisitor):
- """A QAPI schema visitor which generates docutils/Sphinx nodes
-
- This class builds up a tree of docutils/Sphinx nodes corresponding
- to documentation for the various QAPI objects. To use it, first
- create a QAPISchemaGenRSTVisitor object, and call its
- visit_begin() method. Then you can call one of the two methods
- 'freeform' (to add documentation for a freeform documentation
- chunk) or 'symbol' (to add documentation for a QAPI symbol). These
- will cause the visitor to build up the tree of document
- nodes. Once you've added all the documentation via 'freeform' and
- 'symbol' method calls, you can call 'get_document_nodes' to get
- the final list of document nodes (in a form suitable for returning
- from a Sphinx directive's 'run' method).
- """
- def __init__(self, sphinx_directive):
- self._cur_doc = None
- self._sphinx_directive = sphinx_directive
- self._top_node = nodes.section()
- self._active_headings = [self._top_node]
-
- def _make_dlitem(self, term, defn):
- """Return a dlitem node with the specified term and definition.
-
- term should be a list of Text and literal nodes.
- defn should be one of:
- - a string, which will be handed to _parse_text_into_node
- - a list of Text and literal nodes, which will be put into
- a paragraph node
- """
- dlitem = nodes.definition_list_item()
- dlterm = nodes.term('', '', *term)
- dlitem += dlterm
- if defn:
- dldef = nodes.definition()
- if isinstance(defn, list):
- dldef += nodes.paragraph('', '', *defn)
- else:
- self._parse_text_into_node(defn, dldef)
- dlitem += dldef
- return dlitem
-
- def _make_section(self, title):
- """Return a section node with optional title"""
- section = nodes.section(ids=[self._sphinx_directive.new_serialno()])
- if title:
- section += nodes.title(title, title)
- return section
-
- def _nodes_for_ifcond(self, ifcond, with_if=True):
- """Return list of Text, literal nodes for the ifcond
-
- Return a list which gives text like ' (If: condition)'.
- If with_if is False, we don't return the "(If: " and ")".
- """
-
- doc = ifcond.docgen()
- if not doc:
- return []
- doc = nodes.literal('', doc)
- if not with_if:
- return [doc]
-
- nodelist = [nodes.Text(' ('), nodes.strong('', 'If: ')]
- nodelist.append(doc)
- nodelist.append(nodes.Text(')'))
- return nodelist
-
- def _nodes_for_one_member(self, member):
- """Return list of Text, literal nodes for this member
-
- Return a list of doctree nodes which give text like
- 'name: type (optional) (If: ...)' suitable for use as the
- 'term' part of a definition list item.
- """
- term = [nodes.literal('', member.name)]
- if member.type.doc_type():
- term.append(nodes.Text(': '))
- term.append(nodes.literal('', member.type.doc_type()))
- if member.optional:
- term.append(nodes.Text(' (optional)'))
- if member.ifcond.is_present():
- term.extend(self._nodes_for_ifcond(member.ifcond))
- return term
-
- def _nodes_for_variant_when(self, branches, variant):
- """Return list of Text, literal nodes for variant 'when' clause
-
- Return a list of doctree nodes which give text like
- 'when tagname is variant (If: ...)' suitable for use in
- the 'branches' part of a definition list.
- """
- term = [nodes.Text(' when '),
- nodes.literal('', branches.tag_member.name),
- nodes.Text(' is '),
- nodes.literal('', '"%s"' % variant.name)]
- if variant.ifcond.is_present():
- term.extend(self._nodes_for_ifcond(variant.ifcond))
- return term
-
- def _nodes_for_members(self, doc, what, base=None, branches=None):
- """Return list of doctree nodes for the table of members"""
- dlnode = nodes.definition_list()
- for section in doc.args.values():
- term = self._nodes_for_one_member(section.member)
- # TODO drop fallbacks when undocumented members are outlawed
- if section.text:
- defn = dedent(section.text)
- else:
- defn = [nodes.Text('Not documented')]
-
- dlnode += self._make_dlitem(term, defn)
-
- if base:
- dlnode += self._make_dlitem([nodes.Text('The members of '),
- nodes.literal('', base.doc_type())],
- None)
-
- if branches:
- for v in branches.variants:
- if v.type.name == 'q_empty':
- continue
- assert not v.type.is_implicit()
- term = [nodes.Text('The members of '),
- nodes.literal('', v.type.doc_type())]
- term.extend(self._nodes_for_variant_when(branches, v))
- dlnode += self._make_dlitem(term, None)
-
- if not dlnode.children:
- return []
-
- section = self._make_section(what)
- section += dlnode
- return [section]
-
- def _nodes_for_enum_values(self, doc):
- """Return list of doctree nodes for the table of enum values"""
- seen_item = False
- dlnode = nodes.definition_list()
- for section in doc.args.values():
- termtext = [nodes.literal('', section.member.name)]
- if section.member.ifcond.is_present():
- termtext.extend(self._nodes_for_ifcond(section.member.ifcond))
- # TODO drop fallbacks when undocumented members are outlawed
- if section.text:
- defn = dedent(section.text)
- else:
- defn = [nodes.Text('Not documented')]
-
- dlnode += self._make_dlitem(termtext, defn)
- seen_item = True
-
- if not seen_item:
- return []
-
- section = self._make_section('Values')
- section += dlnode
- return [section]
-
- def _nodes_for_arguments(self, doc, arg_type):
- """Return list of doctree nodes for the arguments section"""
- if arg_type and not arg_type.is_implicit():
- assert not doc.args
- section = self._make_section('Arguments')
- dlnode = nodes.definition_list()
- dlnode += self._make_dlitem(
- [nodes.Text('The members of '),
- nodes.literal('', arg_type.name)],
- None)
- section += dlnode
- return [section]
-
- return self._nodes_for_members(doc, 'Arguments')
-
- def _nodes_for_features(self, doc):
- """Return list of doctree nodes for the table of features"""
- seen_item = False
- dlnode = nodes.definition_list()
- for section in doc.features.values():
- dlnode += self._make_dlitem(
- [nodes.literal('', section.member.name)], dedent(section.text))
- seen_item = True
-
- if not seen_item:
- return []
-
- section = self._make_section('Features')
- section += dlnode
- return [section]
-
- def _nodes_for_sections(self, doc):
- """Return list of doctree nodes for additional sections"""
- nodelist = []
- for section in doc.sections:
- if section.kind == QAPIDoc.Kind.TODO:
- # Hide TODO: sections
- continue
-
- if section.kind == QAPIDoc.Kind.PLAIN:
- # Sphinx cannot handle sectionless titles;
- # Instead, just append the results to the prior section.
- container = nodes.container()
- self._parse_text_into_node(section.text, container)
- nodelist += container.children
- continue
-
- snode = self._make_section(section.kind.name.title())
- self._parse_text_into_node(dedent(section.text), snode)
- nodelist.append(snode)
- return nodelist
-
- def _nodes_for_if_section(self, ifcond):
- """Return list of doctree nodes for the "If" section"""
- nodelist = []
- if ifcond.is_present():
- snode = self._make_section('If')
- snode += nodes.paragraph(
- '', '', *self._nodes_for_ifcond(ifcond, with_if=False)
- )
- nodelist.append(snode)
- return nodelist
-
- def _add_doc(self, typ, sections):
- """Add documentation for a command/object/enum...
-
- We assume we're documenting the thing defined in self._cur_doc.
- typ is the type of thing being added ("Command", "Object", etc)
-
- sections is a list of nodes for sections to add to the definition.
- """
-
- doc = self._cur_doc
- snode = nodes.section(ids=[self._sphinx_directive.new_serialno()])
- snode += nodes.title('', '', *[nodes.literal(doc.symbol, doc.symbol),
- nodes.Text(' (' + typ + ')')])
- self._parse_text_into_node(doc.body.text, snode)
- for s in sections:
- if s is not None:
- snode += s
- self._add_node_to_current_heading(snode)
-
- def visit_enum_type(self, name, info, ifcond, features, members, prefix):
- doc = self._cur_doc
- self._add_doc('Enum',
- self._nodes_for_enum_values(doc)
- + self._nodes_for_features(doc)
- + self._nodes_for_sections(doc)
- + self._nodes_for_if_section(ifcond))
-
- def visit_object_type(self, name, info, ifcond, features,
- base, members, branches):
- doc = self._cur_doc
- if base and base.is_implicit():
- base = None
- self._add_doc('Object',
- self._nodes_for_members(doc, 'Members', base, branches)
- + self._nodes_for_features(doc)
- + self._nodes_for_sections(doc)
- + self._nodes_for_if_section(ifcond))
-
- def visit_alternate_type(self, name, info, ifcond, features,
- alternatives):
- doc = self._cur_doc
- self._add_doc('Alternate',
- self._nodes_for_members(doc, 'Members')
- + self._nodes_for_features(doc)
- + self._nodes_for_sections(doc)
- + self._nodes_for_if_section(ifcond))
-
- def visit_command(self, name, info, ifcond, features, arg_type,
- ret_type, gen, success_response, boxed, allow_oob,
- allow_preconfig, coroutine):
- doc = self._cur_doc
- self._add_doc('Command',
- self._nodes_for_arguments(doc, arg_type)
- + self._nodes_for_features(doc)
- + self._nodes_for_sections(doc)
- + self._nodes_for_if_section(ifcond))
-
- def visit_event(self, name, info, ifcond, features, arg_type, boxed):
- doc = self._cur_doc
- self._add_doc('Event',
- self._nodes_for_arguments(doc, arg_type)
- + self._nodes_for_features(doc)
- + self._nodes_for_sections(doc)
- + self._nodes_for_if_section(ifcond))
-
- def symbol(self, doc, entity):
- """Add documentation for one symbol to the document tree
-
- This is the main entry point which causes us to add documentation
- nodes for a symbol (which could be a 'command', 'object', 'event',
- etc). We do this by calling 'visit' on the schema entity, which
- will then call back into one of our visit_* methods, depending
- on what kind of thing this symbol is.
- """
- self._cur_doc = doc
- entity.visit(self)
- self._cur_doc = None
-
- def _start_new_heading(self, heading, level):
- """Start a new heading at the specified heading level
-
- Create a new section whose title is 'heading' and which is placed
- in the docutils node tree as a child of the most recent level-1
- heading. Subsequent document sections (commands, freeform doc chunks,
- etc) will be placed as children of this new heading section.
- """
- if len(self._active_headings) < level:
- raise QAPISemError(self._cur_doc.info,
- 'Level %d subheading found outside a '
- 'level %d heading'
- % (level, level - 1))
- snode = self._make_section(heading)
- self._active_headings[level - 1] += snode
- self._active_headings = self._active_headings[:level]
- self._active_headings.append(snode)
- return snode
-
- def _add_node_to_current_heading(self, node):
- """Add the node to whatever the current active heading is"""
- self._active_headings[-1] += node
-
- def freeform(self, doc):
- """Add a piece of 'freeform' documentation to the document tree
-
- A 'freeform' document chunk doesn't relate to any particular
- symbol (for instance, it could be an introduction).
-
- If the freeform document starts with a line of the form
- '= Heading text', this is a section or subsection heading, with
- the heading level indicated by the number of '=' signs.
- """
-
- # QAPIDoc documentation says free-form documentation blocks
- # must have only a body section, nothing else.
- assert not doc.sections
- assert not doc.args
- assert not doc.features
- self._cur_doc = doc
-
- text = doc.body.text
- if re.match(r'=+ ', text):
- # Section/subsection heading (if present, will always be
- # the first line of the block)
- (heading, _, text) = text.partition('\n')
- (leader, _, heading) = heading.partition(' ')
- node = self._start_new_heading(heading, len(leader))
- if text == '':
- return
- else:
- node = nodes.container()
-
- self._parse_text_into_node(text, node)
- self._cur_doc = None
-
- def _parse_text_into_node(self, doctext, node):
- """Parse a chunk of QAPI-doc-format text into the node
-
- The doc comment can contain most inline rST markup, including
- bulleted and enumerated lists.
- As an extra permitted piece of markup, @var will be turned
- into ``var``.
- """
-
- # Handle the "@var means ``var`` case
- doctext = re.sub(r'@([\w-]+)', r'``\1``', doctext)
-
- rstlist = ViewList()
- for line in doctext.splitlines():
- # The reported line number will always be that of the start line
- # of the doc comment, rather than the actual location of the error.
- # Being more precise would require overhaul of the QAPIDoc class
- # to track lines more exactly within all the sub-parts of the doc
- # comment, as well as counting lines here.
- rstlist.append(line, self._cur_doc.info.fname,
- self._cur_doc.info.line)
- # Append a blank line -- in some cases rST syntax errors get
- # attributed to the line after one with actual text, and if there
- # isn't anything in the ViewList corresponding to that then Sphinx
- # 1.6's AutodocReporter will then misidentify the source/line location
- # in the error message (usually attributing it to the top-level
- # .rst file rather than the offending .json file). The extra blank
- # line won't affect the rendered output.
- rstlist.append("", self._cur_doc.info.fname, self._cur_doc.info.line)
- self._sphinx_directive.do_parse(rstlist, node)
-
- def get_document_node(self):
- """Return the root docutils node which makes up the document"""
- return self._top_node
-
-
-# Turn the black formatter on for the rest of the file.
-# fmt: on
-
-
class QAPISchemaGenDepVisitor(QAPISchemaVisitor):
"""A QAPI schema visitor which adds Sphinx dependencies each module
diff --git a/docs/sphinx/qapidoc_legacy.py b/docs/sphinx/qapidoc_legacy.py
new file mode 100644
index 00000000000..679f38356b1
--- /dev/null
+++ b/docs/sphinx/qapidoc_legacy.py
@@ -0,0 +1,439 @@
+# coding=utf-8
+#
+# QEMU qapidoc QAPI file parsing extension
+#
+# Copyright (c) 2020 Linaro
+#
+# This work is licensed under the terms of the GNU GPLv2 or later.
+# See the COPYING file in the top-level directory.
+
+"""
+qapidoc is a Sphinx extension that implements the qapi-doc directive
+
+The purpose of this extension is to read the documentation comments
+in QAPI schema files, and insert them all into the current document.
+
+It implements one new rST directive, "qapi-doc::".
+Each qapi-doc:: directive takes one argument, which is the
+pathname of the schema file to process, relative to the source tree.
+
+The docs/conf.py file must set the qapidoc_srctree config value to
+the root of the QEMU source tree.
+
+The Sphinx documentation on writing extensions is at:
+https://www.sphinx-doc.org/en/master/development/index.html
+"""
+
+import re
+import textwrap
+
+from docutils import nodes
+from docutils.statemachine import ViewList
+from qapi.error import QAPISemError
+from qapi.gen import QAPISchemaVisitor
+from qapi.parser import QAPIDoc
+
+
+def dedent(text: str) -> str:
+ # Adjust indentation to make description text parse as paragraph.
+
+ lines = text.splitlines(True)
+ if re.match(r"\s+", lines[0]):
+ # First line is indented; description started on the line after
+ # the name. dedent the whole block.
+ return textwrap.dedent(text)
+
+ # Descr started on same line. Dedent line 2+.
+ return lines[0] + textwrap.dedent("".join(lines[1:]))
+
+
+class QAPISchemaGenRSTVisitor(QAPISchemaVisitor):
+ """A QAPI schema visitor which generates docutils/Sphinx nodes
+
+ This class builds up a tree of docutils/Sphinx nodes corresponding
+ to documentation for the various QAPI objects. To use it, first
+ create a QAPISchemaGenRSTVisitor object, and call its
+ visit_begin() method. Then you can call one of the two methods
+ 'freeform' (to add documentation for a freeform documentation
+ chunk) or 'symbol' (to add documentation for a QAPI symbol). These
+ will cause the visitor to build up the tree of document
+ nodes. Once you've added all the documentation via 'freeform' and
+ 'symbol' method calls, you can call 'get_document_nodes' to get
+ the final list of document nodes (in a form suitable for returning
+ from a Sphinx directive's 'run' method).
+ """
+ def __init__(self, sphinx_directive):
+ self._cur_doc = None
+ self._sphinx_directive = sphinx_directive
+ self._top_node = nodes.section()
+ self._active_headings = [self._top_node]
+
+ def _make_dlitem(self, term, defn):
+ """Return a dlitem node with the specified term and definition.
+
+ term should be a list of Text and literal nodes.
+ defn should be one of:
+ - a string, which will be handed to _parse_text_into_node
+ - a list of Text and literal nodes, which will be put into
+ a paragraph node
+ """
+ dlitem = nodes.definition_list_item()
+ dlterm = nodes.term('', '', *term)
+ dlitem += dlterm
+ if defn:
+ dldef = nodes.definition()
+ if isinstance(defn, list):
+ dldef += nodes.paragraph('', '', *defn)
+ else:
+ self._parse_text_into_node(defn, dldef)
+ dlitem += dldef
+ return dlitem
+
+ def _make_section(self, title):
+ """Return a section node with optional title"""
+ section = nodes.section(ids=[self._sphinx_directive.new_serialno()])
+ if title:
+ section += nodes.title(title, title)
+ return section
+
+ def _nodes_for_ifcond(self, ifcond, with_if=True):
+ """Return list of Text, literal nodes for the ifcond
+
+ Return a list which gives text like ' (If: condition)'.
+ If with_if is False, we don't return the "(If: " and ")".
+ """
+
+ doc = ifcond.docgen()
+ if not doc:
+ return []
+ doc = nodes.literal('', doc)
+ if not with_if:
+ return [doc]
+
+ nodelist = [nodes.Text(' ('), nodes.strong('', 'If: ')]
+ nodelist.append(doc)
+ nodelist.append(nodes.Text(')'))
+ return nodelist
+
+ def _nodes_for_one_member(self, member):
+ """Return list of Text, literal nodes for this member
+
+ Return a list of doctree nodes which give text like
+ 'name: type (optional) (If: ...)' suitable for use as the
+ 'term' part of a definition list item.
+ """
+ term = [nodes.literal('', member.name)]
+ if member.type.doc_type():
+ term.append(nodes.Text(': '))
+ term.append(nodes.literal('', member.type.doc_type()))
+ if member.optional:
+ term.append(nodes.Text(' (optional)'))
+ if member.ifcond.is_present():
+ term.extend(self._nodes_for_ifcond(member.ifcond))
+ return term
+
+ def _nodes_for_variant_when(self, branches, variant):
+ """Return list of Text, literal nodes for variant 'when' clause
+
+ Return a list of doctree nodes which give text like
+ 'when tagname is variant (If: ...)' suitable for use in
+ the 'branches' part of a definition list.
+ """
+ term = [nodes.Text(' when '),
+ nodes.literal('', branches.tag_member.name),
+ nodes.Text(' is '),
+ nodes.literal('', '"%s"' % variant.name)]
+ if variant.ifcond.is_present():
+ term.extend(self._nodes_for_ifcond(variant.ifcond))
+ return term
+
+ def _nodes_for_members(self, doc, what, base=None, branches=None):
+ """Return list of doctree nodes for the table of members"""
+ dlnode = nodes.definition_list()
+ for section in doc.args.values():
+ term = self._nodes_for_one_member(section.member)
+ # TODO drop fallbacks when undocumented members are outlawed
+ if section.text:
+ defn = dedent(section.text)
+ else:
+ defn = [nodes.Text('Not documented')]
+
+ dlnode += self._make_dlitem(term, defn)
+
+ if base:
+ dlnode += self._make_dlitem([nodes.Text('The members of '),
+ nodes.literal('', base.doc_type())],
+ None)
+
+ if branches:
+ for v in branches.variants:
+ if v.type.name == 'q_empty':
+ continue
+ assert not v.type.is_implicit()
+ term = [nodes.Text('The members of '),
+ nodes.literal('', v.type.doc_type())]
+ term.extend(self._nodes_for_variant_when(branches, v))
+ dlnode += self._make_dlitem(term, None)
+
+ if not dlnode.children:
+ return []
+
+ section = self._make_section(what)
+ section += dlnode
+ return [section]
+
+ def _nodes_for_enum_values(self, doc):
+ """Return list of doctree nodes for the table of enum values"""
+ seen_item = False
+ dlnode = nodes.definition_list()
+ for section in doc.args.values():
+ termtext = [nodes.literal('', section.member.name)]
+ if section.member.ifcond.is_present():
+ termtext.extend(self._nodes_for_ifcond(section.member.ifcond))
+ # TODO drop fallbacks when undocumented members are outlawed
+ if section.text:
+ defn = dedent(section.text)
+ else:
+ defn = [nodes.Text('Not documented')]
+
+ dlnode += self._make_dlitem(termtext, defn)
+ seen_item = True
+
+ if not seen_item:
+ return []
+
+ section = self._make_section('Values')
+ section += dlnode
+ return [section]
+
+ def _nodes_for_arguments(self, doc, arg_type):
+ """Return list of doctree nodes for the arguments section"""
+ if arg_type and not arg_type.is_implicit():
+ assert not doc.args
+ section = self._make_section('Arguments')
+ dlnode = nodes.definition_list()
+ dlnode += self._make_dlitem(
+ [nodes.Text('The members of '),
+ nodes.literal('', arg_type.name)],
+ None)
+ section += dlnode
+ return [section]
+
+ return self._nodes_for_members(doc, 'Arguments')
+
+ def _nodes_for_features(self, doc):
+ """Return list of doctree nodes for the table of features"""
+ seen_item = False
+ dlnode = nodes.definition_list()
+ for section in doc.features.values():
+ dlnode += self._make_dlitem(
+ [nodes.literal('', section.member.name)], dedent(section.text))
+ seen_item = True
+
+ if not seen_item:
+ return []
+
+ section = self._make_section('Features')
+ section += dlnode
+ return [section]
+
+ def _nodes_for_sections(self, doc):
+ """Return list of doctree nodes for additional sections"""
+ nodelist = []
+ for section in doc.sections:
+ if section.kind == QAPIDoc.Kind.TODO:
+ # Hide TODO: sections
+ continue
+
+ if section.kind == QAPIDoc.Kind.PLAIN:
+ # Sphinx cannot handle sectionless titles;
+ # Instead, just append the results to the prior section.
+ container = nodes.container()
+ self._parse_text_into_node(section.text, container)
+ nodelist += container.children
+ continue
+
+ snode = self._make_section(section.kind.name.title())
+ self._parse_text_into_node(dedent(section.text), snode)
+ nodelist.append(snode)
+ return nodelist
+
+ def _nodes_for_if_section(self, ifcond):
+ """Return list of doctree nodes for the "If" section"""
+ nodelist = []
+ if ifcond.is_present():
+ snode = self._make_section('If')
+ snode += nodes.paragraph(
+ '', '', *self._nodes_for_ifcond(ifcond, with_if=False)
+ )
+ nodelist.append(snode)
+ return nodelist
+
+ def _add_doc(self, typ, sections):
+ """Add documentation for a command/object/enum...
+
+ We assume we're documenting the thing defined in self._cur_doc.
+ typ is the type of thing being added ("Command", "Object", etc)
+
+ sections is a list of nodes for sections to add to the definition.
+ """
+
+ doc = self._cur_doc
+ snode = nodes.section(ids=[self._sphinx_directive.new_serialno()])
+ snode += nodes.title('', '', *[nodes.literal(doc.symbol, doc.symbol),
+ nodes.Text(' (' + typ + ')')])
+ self._parse_text_into_node(doc.body.text, snode)
+ for s in sections:
+ if s is not None:
+ snode += s
+ self._add_node_to_current_heading(snode)
+
+ def visit_enum_type(self, name, info, ifcond, features, members, prefix):
+ doc = self._cur_doc
+ self._add_doc('Enum',
+ self._nodes_for_enum_values(doc)
+ + self._nodes_for_features(doc)
+ + self._nodes_for_sections(doc)
+ + self._nodes_for_if_section(ifcond))
+
+ def visit_object_type(self, name, info, ifcond, features,
+ base, members, branches):
+ doc = self._cur_doc
+ if base and base.is_implicit():
+ base = None
+ self._add_doc('Object',
+ self._nodes_for_members(doc, 'Members', base, branches)
+ + self._nodes_for_features(doc)
+ + self._nodes_for_sections(doc)
+ + self._nodes_for_if_section(ifcond))
+
+ def visit_alternate_type(self, name, info, ifcond, features,
+ alternatives):
+ doc = self._cur_doc
+ self._add_doc('Alternate',
+ self._nodes_for_members(doc, 'Members')
+ + self._nodes_for_features(doc)
+ + self._nodes_for_sections(doc)
+ + self._nodes_for_if_section(ifcond))
+
+ def visit_command(self, name, info, ifcond, features, arg_type,
+ ret_type, gen, success_response, boxed, allow_oob,
+ allow_preconfig, coroutine):
+ doc = self._cur_doc
+ self._add_doc('Command',
+ self._nodes_for_arguments(doc, arg_type)
+ + self._nodes_for_features(doc)
+ + self._nodes_for_sections(doc)
+ + self._nodes_for_if_section(ifcond))
+
+ def visit_event(self, name, info, ifcond, features, arg_type, boxed):
+ doc = self._cur_doc
+ self._add_doc('Event',
+ self._nodes_for_arguments(doc, arg_type)
+ + self._nodes_for_features(doc)
+ + self._nodes_for_sections(doc)
+ + self._nodes_for_if_section(ifcond))
+
+ def symbol(self, doc, entity):
+ """Add documentation for one symbol to the document tree
+
+ This is the main entry point which causes us to add documentation
+ nodes for a symbol (which could be a 'command', 'object', 'event',
+ etc). We do this by calling 'visit' on the schema entity, which
+ will then call back into one of our visit_* methods, depending
+ on what kind of thing this symbol is.
+ """
+ self._cur_doc = doc
+ entity.visit(self)
+ self._cur_doc = None
+
+ def _start_new_heading(self, heading, level):
+ """Start a new heading at the specified heading level
+
+ Create a new section whose title is 'heading' and which is placed
+ in the docutils node tree as a child of the most recent level-1
+ heading. Subsequent document sections (commands, freeform doc chunks,
+ etc) will be placed as children of this new heading section.
+ """
+ if len(self._active_headings) < level:
+ raise QAPISemError(self._cur_doc.info,
+ 'Level %d subheading found outside a '
+ 'level %d heading'
+ % (level, level - 1))
+ snode = self._make_section(heading)
+ self._active_headings[level - 1] += snode
+ self._active_headings = self._active_headings[:level]
+ self._active_headings.append(snode)
+ return snode
+
+ def _add_node_to_current_heading(self, node):
+ """Add the node to whatever the current active heading is"""
+ self._active_headings[-1] += node
+
+ def freeform(self, doc):
+ """Add a piece of 'freeform' documentation to the document tree
+
+ A 'freeform' document chunk doesn't relate to any particular
+ symbol (for instance, it could be an introduction).
+
+ If the freeform document starts with a line of the form
+ '= Heading text', this is a section or subsection heading, with
+ the heading level indicated by the number of '=' signs.
+ """
+
+ # QAPIDoc documentation says free-form documentation blocks
+ # must have only a body section, nothing else.
+ assert not doc.sections
+ assert not doc.args
+ assert not doc.features
+ self._cur_doc = doc
+
+ text = doc.body.text
+ if re.match(r'=+ ', text):
+ # Section/subsection heading (if present, will always be
+ # the first line of the block)
+ (heading, _, text) = text.partition('\n')
+ (leader, _, heading) = heading.partition(' ')
+ node = self._start_new_heading(heading, len(leader))
+ if text == '':
+ return
+ else:
+ node = nodes.container()
+
+ self._parse_text_into_node(text, node)
+ self._cur_doc = None
+
+ def _parse_text_into_node(self, doctext, node):
+ """Parse a chunk of QAPI-doc-format text into the node
+
+ The doc comment can contain most inline rST markup, including
+ bulleted and enumerated lists.
+ As an extra permitted piece of markup, @var will be turned
+ into ``var``.
+ """
+
+ # Handle the "@var means ``var`` case
+ doctext = re.sub(r'@([\w-]+)', r'``\1``', doctext)
+
+ rstlist = ViewList()
+ for line in doctext.splitlines():
+ # The reported line number will always be that of the start line
+ # of the doc comment, rather than the actual location of the error.
+ # Being more precise would require overhaul of the QAPIDoc class
+ # to track lines more exactly within all the sub-parts of the doc
+ # comment, as well as counting lines here.
+ rstlist.append(line, self._cur_doc.info.fname,
+ self._cur_doc.info.line)
+ # Append a blank line -- in some cases rST syntax errors get
+ # attributed to the line after one with actual text, and if there
+ # isn't anything in the ViewList corresponding to that then Sphinx
+ # 1.6's AutodocReporter will then misidentify the source/line location
+ # in the error message (usually attributing it to the top-level
+ # .rst file rather than the offending .json file). The extra blank
+ # line won't affect the rendered output.
+ rstlist.append("", self._cur_doc.info.fname, self._cur_doc.info.line)
+ self._sphinx_directive.do_parse(rstlist, node)
+
+ def get_document_node(self):
+ """Return the root docutils node which makes up the document"""
+ return self._top_node
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 35/57] docs/qapidoc: Fix static typing on qapidoc.py
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (33 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 34/57] docs/qapidoc: split old implementation into qapidoc_legacy.py John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-07 11:59 ` Markus Armbruster
2025-03-05 3:45 ` [PATCH 36/57] do-not-merge John Snow
` (24 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Now that the legacy code is factored out, fix up the typing on the
remaining code in qapidoc.py. Add a type ignore to qapi_legacy.py to
prevent the errors there from bleeding out into qapidoc.py.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapidoc.py | 40 ++++++++++++++++++++++-------------
docs/sphinx/qapidoc_legacy.py | 1 +
2 files changed, 26 insertions(+), 15 deletions(-)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index f4abf42e7bf..5246832b68c 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -24,17 +24,18 @@
https://www.sphinx-doc.org/en/master/development/index.html
"""
+from __future__ import annotations
+
import os
import sys
-from typing import List
+from typing import TYPE_CHECKING
from docutils import nodes
from docutils.parsers.rst import Directive, directives
from qapi.error import QAPIError
-from qapi.gen import QAPISchemaVisitor
-from qapi.schema import QAPISchema
+from qapi.schema import QAPISchema, QAPISchemaVisitor
-from qapidoc_legacy import QAPISchemaGenRSTVisitor
+from qapidoc_legacy import QAPISchemaGenRSTVisitor # type: ignore
from sphinx import addnodes
from sphinx.directives.code import CodeBlock
from sphinx.errors import ExtensionError
@@ -42,6 +43,15 @@
from sphinx.util.nodes import nested_parse_with_titles
+if TYPE_CHECKING:
+ from typing import Any, List, Sequence
+
+ from docutils.statemachine import StringList
+
+ from sphinx.application import Sphinx
+ from sphinx.util.typing import ExtensionMetadata
+
+
__version__ = "1.0"
@@ -53,11 +63,11 @@ class QAPISchemaGenDepVisitor(QAPISchemaVisitor):
schema file associated with each module in the QAPI input.
"""
- def __init__(self, env, qapidir):
+ def __init__(self, env: Any, qapidir: str) -> None:
self._env = env
self._qapidir = qapidir
- def visit_module(self, name):
+ def visit_module(self, name: str) -> None:
if name != "./builtin":
qapifile = self._qapidir + "/" + name
self._env.note_dependency(os.path.abspath(qapifile))
@@ -65,10 +75,10 @@ def visit_module(self, name):
class NestedDirective(Directive):
- def run(self):
+ def run(self) -> Sequence[nodes.Node]:
raise NotImplementedError
- def do_parse(self, rstlist, node):
+ def do_parse(self, rstlist: StringList, node: nodes.Node) -> None:
"""
Parse rST source lines and add them to the specified node
@@ -93,15 +103,15 @@ class QAPIDocDirective(NestedDirective):
}
has_content = False
- def new_serialno(self):
+ def new_serialno(self) -> str:
"""Return a unique new ID string suitable for use as a node's ID"""
env = self.state.document.settings.env
return "qapidoc-%d" % env.new_serialno("qapidoc")
- def transmogrify(self, schema) -> nodes.Element:
+ def transmogrify(self, schema: QAPISchema) -> nodes.Element:
raise NotImplementedError
- def legacy(self, schema) -> nodes.Element:
+ def legacy(self, schema: QAPISchema) -> nodes.Element:
vis = QAPISchemaGenRSTVisitor(self)
vis.visit_begin(schema)
for doc in schema.docs:
@@ -109,9 +119,9 @@ def legacy(self, schema) -> nodes.Element:
vis.symbol(doc, schema.lookup_entity(doc.symbol))
else:
vis.freeform(doc)
- return vis.get_document_node()
+ return vis.get_document_node() # type: ignore
- def run(self):
+ def run(self) -> Sequence[nodes.Node]:
env = self.state.document.settings.env
qapifile = env.config.qapidoc_srctree + "/" + self.arguments[0]
qapidir = os.path.dirname(qapifile)
@@ -185,7 +195,7 @@ def _highlightlang(self) -> addnodes.highlightlang:
)
return node
- def admonition_wrap(self, *content) -> List[nodes.Node]:
+ def admonition_wrap(self, *content: nodes.Node) -> List[nodes.Node]:
title = "Example:"
if "title" in self.options:
title = f"{title} {self.options['title']}"
@@ -231,7 +241,7 @@ def run(self) -> List[nodes.Node]:
return self.admonition_wrap(*content_nodes)
-def setup(app):
+def setup(app: Sphinx) -> ExtensionMetadata:
"""Register qapi-doc directive with Sphinx"""
app.add_config_value("qapidoc_srctree", None, "env")
app.add_directive("qapi-doc", QAPIDocDirective)
diff --git a/docs/sphinx/qapidoc_legacy.py b/docs/sphinx/qapidoc_legacy.py
index 679f38356b1..13520f4c26b 100644
--- a/docs/sphinx/qapidoc_legacy.py
+++ b/docs/sphinx/qapidoc_legacy.py
@@ -1,4 +1,5 @@
# coding=utf-8
+# type: ignore
#
# QEMU qapidoc QAPI file parsing extension
#
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 35/57] docs/qapidoc: Fix static typing on qapidoc.py
2025-03-05 3:45 ` [PATCH 35/57] docs/qapidoc: Fix static typing on qapidoc.py John Snow
@ 2025-03-07 11:59 ` Markus Armbruster
2025-03-08 18:32 ` John Snow
0 siblings, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-07 11:59 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> Now that the legacy code is factored out, fix up the typing on the
> remaining code in qapidoc.py. Add a type ignore to qapi_legacy.py to
> prevent the errors there from bleeding out into qapidoc.py.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> docs/sphinx/qapidoc.py | 40 ++++++++++++++++++++++-------------
> docs/sphinx/qapidoc_legacy.py | 1 +
> 2 files changed, 26 insertions(+), 15 deletions(-)
>
> diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> index f4abf42e7bf..5246832b68c 100644
> --- a/docs/sphinx/qapidoc.py
> +++ b/docs/sphinx/qapidoc.py
> @@ -24,17 +24,18 @@
> https://www.sphinx-doc.org/en/master/development/index.html
> """
>
> +from __future__ import annotations
> +
> import os
> import sys
> -from typing import List
> +from typing import TYPE_CHECKING
>
> from docutils import nodes
> from docutils.parsers.rst import Directive, directives
> from qapi.error import QAPIError
> -from qapi.gen import QAPISchemaVisitor
> -from qapi.schema import QAPISchema
> +from qapi.schema import QAPISchema, QAPISchemaVisitor
>
> -from qapidoc_legacy import QAPISchemaGenRSTVisitor
> +from qapidoc_legacy import QAPISchemaGenRSTVisitor # type: ignore
> from sphinx import addnodes
> from sphinx.directives.code import CodeBlock
> from sphinx.errors import ExtensionError
> @@ -42,6 +43,15 @@
> from sphinx.util.nodes import nested_parse_with_titles
>
>
> +if TYPE_CHECKING:
> + from typing import Any, List, Sequence
> +
> + from docutils.statemachine import StringList
> +
> + from sphinx.application import Sphinx
> + from sphinx.util.typing import ExtensionMetadata
Can you briefly explain why this needs to be conditional?
> +
> +
> __version__ = "1.0"
>
>
> @@ -53,11 +63,11 @@ class QAPISchemaGenDepVisitor(QAPISchemaVisitor):
> schema file associated with each module in the QAPI input.
> """
>
> - def __init__(self, env, qapidir):
> + def __init__(self, env: Any, qapidir: str) -> None:
@env is QAPIDocDirective.state.document.settings.env, i.e. something
deep in Sphinx. I assume Any is the best Sphinx lets you do here.
> self._env = env
> self._qapidir = qapidir
>
> - def visit_module(self, name):
> + def visit_module(self, name: str) -> None:
> if name != "./builtin":
> qapifile = self._qapidir + "/" + name
> self._env.note_dependency(os.path.abspath(qapifile))
> @@ -65,10 +75,10 @@ def visit_module(self, name):
>
>
> class NestedDirective(Directive):
> - def run(self):
> + def run(self) -> Sequence[nodes.Node]:
> raise NotImplementedError
>
> - def do_parse(self, rstlist, node):
> + def do_parse(self, rstlist: StringList, node: nodes.Node) -> None:
> """
> Parse rST source lines and add them to the specified node
>
> @@ -93,15 +103,15 @@ class QAPIDocDirective(NestedDirective):
> }
> has_content = False
>
> - def new_serialno(self):
> + def new_serialno(self) -> str:
> """Return a unique new ID string suitable for use as a node's ID"""
> env = self.state.document.settings.env
> return "qapidoc-%d" % env.new_serialno("qapidoc")
>
> - def transmogrify(self, schema) -> nodes.Element:
> + def transmogrify(self, schema: QAPISchema) -> nodes.Element:
> raise NotImplementedError
>
> - def legacy(self, schema) -> nodes.Element:
> + def legacy(self, schema: QAPISchema) -> nodes.Element:
> vis = QAPISchemaGenRSTVisitor(self)
> vis.visit_begin(schema)
> for doc in schema.docs:
> @@ -109,9 +119,9 @@ def legacy(self, schema) -> nodes.Element:
> vis.symbol(doc, schema.lookup_entity(doc.symbol))
> else:
> vis.freeform(doc)
> - return vis.get_document_node()
> + return vis.get_document_node() # type: ignore
This is where you call qapidoc_legacy.py, which remains untyped. Okay.
>
> - def run(self):
> + def run(self) -> Sequence[nodes.Node]:
> env = self.state.document.settings.env
> qapifile = env.config.qapidoc_srctree + "/" + self.arguments[0]
> qapidir = os.path.dirname(qapifile)
> @@ -185,7 +195,7 @@ def _highlightlang(self) -> addnodes.highlightlang:
> )
> return node
>
> - def admonition_wrap(self, *content) -> List[nodes.Node]:
> + def admonition_wrap(self, *content: nodes.Node) -> List[nodes.Node]:
> title = "Example:"
> if "title" in self.options:
> title = f"{title} {self.options['title']}"
> @@ -231,7 +241,7 @@ def run(self) -> List[nodes.Node]:
> return self.admonition_wrap(*content_nodes)
>
>
> -def setup(app):
> +def setup(app: Sphinx) -> ExtensionMetadata:
> """Register qapi-doc directive with Sphinx"""
> app.add_config_value("qapidoc_srctree", None, "env")
> app.add_directive("qapi-doc", QAPIDocDirective)
> diff --git a/docs/sphinx/qapidoc_legacy.py b/docs/sphinx/qapidoc_legacy.py
> index 679f38356b1..13520f4c26b 100644
> --- a/docs/sphinx/qapidoc_legacy.py
> +++ b/docs/sphinx/qapidoc_legacy.py
> @@ -1,4 +1,5 @@
> # coding=utf-8
> +# type: ignore
> #
> # QEMU qapidoc QAPI file parsing extension
> #
Types look good to me.
^ permalink raw reply [flat|nested] 129+ messages in thread* Re: [PATCH 35/57] docs/qapidoc: Fix static typing on qapidoc.py
2025-03-07 11:59 ` Markus Armbruster
@ 2025-03-08 18:32 ` John Snow
2025-03-09 5:37 ` Markus Armbruster
0 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-08 18:32 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 6155 bytes --]
On Fri, Mar 7, 2025 at 7:00 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > Now that the legacy code is factored out, fix up the typing on the
> > remaining code in qapidoc.py. Add a type ignore to qapi_legacy.py to
> > prevent the errors there from bleeding out into qapidoc.py.
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
> > ---
> > docs/sphinx/qapidoc.py | 40 ++++++++++++++++++++++-------------
> > docs/sphinx/qapidoc_legacy.py | 1 +
> > 2 files changed, 26 insertions(+), 15 deletions(-)
> >
> > diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> > index f4abf42e7bf..5246832b68c 100644
> > --- a/docs/sphinx/qapidoc.py
> > +++ b/docs/sphinx/qapidoc.py
> > @@ -24,17 +24,18 @@
> > https://www.sphinx-doc.org/en/master/development/index.html
> > """
> >
> > +from __future__ import annotations
> > +
> > import os
> > import sys
> > -from typing import List
> > +from typing import TYPE_CHECKING
> >
> > from docutils import nodes
> > from docutils.parsers.rst import Directive, directives
> > from qapi.error import QAPIError
> > -from qapi.gen import QAPISchemaVisitor
> > -from qapi.schema import QAPISchema
> > +from qapi.schema import QAPISchema, QAPISchemaVisitor
> >
> > -from qapidoc_legacy import QAPISchemaGenRSTVisitor
> > +from qapidoc_legacy import QAPISchemaGenRSTVisitor # type: ignore
> > from sphinx import addnodes
> > from sphinx.directives.code import CodeBlock
> > from sphinx.errors import ExtensionError
> > @@ -42,6 +43,15 @@
> > from sphinx.util.nodes import nested_parse_with_titles
> >
> >
> > +if TYPE_CHECKING:
> > + from typing import Any, List, Sequence
> > +
> > + from docutils.statemachine import StringList
> > +
> > + from sphinx.application import Sphinx
> > + from sphinx.util.typing import ExtensionMetadata
>
> Can you briefly explain why this needs to be conditional?
>
No requisite, but if they aren't used outside of type hints, they don't
actually need to be imported at runtime (when we use from __future__ import
annotations). Improves startup speed slightly and potentially makes the
plugin less porcelain at runtime.
>
> > +
> > +
> > __version__ = "1.0"
> >
> >
> > @@ -53,11 +63,11 @@ class QAPISchemaGenDepVisitor(QAPISchemaVisitor):
> > schema file associated with each module in the QAPI input.
> > """
> >
> > - def __init__(self, env, qapidir):
> > + def __init__(self, env: Any, qapidir: str) -> None:
>
> @env is QAPIDocDirective.state.document.settings.env, i.e. something
> deep in Sphinx. I assume Any is the best Sphinx lets you do here.
>
Will double-check, I did this a while ago.
>
> > self._env = env
> > self._qapidir = qapidir
> >
> > - def visit_module(self, name):
> > + def visit_module(self, name: str) -> None:
> > if name != "./builtin":
> > qapifile = self._qapidir + "/" + name
> > self._env.note_dependency(os.path.abspath(qapifile))
> > @@ -65,10 +75,10 @@ def visit_module(self, name):
> >
> >
> > class NestedDirective(Directive):
> > - def run(self):
> > + def run(self) -> Sequence[nodes.Node]:
> > raise NotImplementedError
> >
> > - def do_parse(self, rstlist, node):
> > + def do_parse(self, rstlist: StringList, node: nodes.Node) -> None:
> > """
> > Parse rST source lines and add them to the specified node
> >
> > @@ -93,15 +103,15 @@ class QAPIDocDirective(NestedDirective):
> > }
> > has_content = False
> >
> > - def new_serialno(self):
> > + def new_serialno(self) -> str:
> > """Return a unique new ID string suitable for use as a node's
> ID"""
> > env = self.state.document.settings.env
> > return "qapidoc-%d" % env.new_serialno("qapidoc")
> >
> > - def transmogrify(self, schema) -> nodes.Element:
> > + def transmogrify(self, schema: QAPISchema) -> nodes.Element:
> > raise NotImplementedError
> >
> > - def legacy(self, schema) -> nodes.Element:
> > + def legacy(self, schema: QAPISchema) -> nodes.Element:
> > vis = QAPISchemaGenRSTVisitor(self)
> > vis.visit_begin(schema)
> > for doc in schema.docs:
> > @@ -109,9 +119,9 @@ def legacy(self, schema) -> nodes.Element:
> > vis.symbol(doc, schema.lookup_entity(doc.symbol))
> > else:
> > vis.freeform(doc)
> > - return vis.get_document_node()
> > + return vis.get_document_node() # type: ignore
>
> This is where you call qapidoc_legacy.py, which remains untyped. Okay.
>
> >
> > - def run(self):
> > + def run(self) -> Sequence[nodes.Node]:
> > env = self.state.document.settings.env
> > qapifile = env.config.qapidoc_srctree + "/" + self.arguments[0]
> > qapidir = os.path.dirname(qapifile)
> > @@ -185,7 +195,7 @@ def _highlightlang(self) -> addnodes.highlightlang:
> > )
> > return node
> >
> > - def admonition_wrap(self, *content) -> List[nodes.Node]:
> > + def admonition_wrap(self, *content: nodes.Node) -> List[nodes.Node]:
> > title = "Example:"
> > if "title" in self.options:
> > title = f"{title} {self.options['title']}"
> > @@ -231,7 +241,7 @@ def run(self) -> List[nodes.Node]:
> > return self.admonition_wrap(*content_nodes)
> >
> >
> > -def setup(app):
> > +def setup(app: Sphinx) -> ExtensionMetadata:
> > """Register qapi-doc directive with Sphinx"""
> > app.add_config_value("qapidoc_srctree", None, "env")
> > app.add_directive("qapi-doc", QAPIDocDirective)
> > diff --git a/docs/sphinx/qapidoc_legacy.py
> b/docs/sphinx/qapidoc_legacy.py
> > index 679f38356b1..13520f4c26b 100644
> > --- a/docs/sphinx/qapidoc_legacy.py
> > +++ b/docs/sphinx/qapidoc_legacy.py
> > @@ -1,4 +1,5 @@
> > # coding=utf-8
> > +# type: ignore
> > #
> > # QEMU qapidoc QAPI file parsing extension
> > #
>
> Types look good to me.
>
>
[-- Attachment #2: Type: text/html, Size: 8358 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread* Re: [PATCH 35/57] docs/qapidoc: Fix static typing on qapidoc.py
2025-03-08 18:32 ` John Snow
@ 2025-03-09 5:37 ` Markus Armbruster
2025-03-09 6:46 ` John Snow
0 siblings, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-09 5:37 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> On Fri, Mar 7, 2025 at 7:00 AM Markus Armbruster <armbru@redhat.com> wrote:
>
>> John Snow <jsnow@redhat.com> writes:
>>
>> > Now that the legacy code is factored out, fix up the typing on the
>> > remaining code in qapidoc.py. Add a type ignore to qapi_legacy.py to
>> > prevent the errors there from bleeding out into qapidoc.py.
>> >
>> > Signed-off-by: John Snow <jsnow@redhat.com>
>> > ---
>> > docs/sphinx/qapidoc.py | 40 ++++++++++++++++++++++-------------
>> > docs/sphinx/qapidoc_legacy.py | 1 +
>> > 2 files changed, 26 insertions(+), 15 deletions(-)
>> >
>> > diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
>> > index f4abf42e7bf..5246832b68c 100644
>> > --- a/docs/sphinx/qapidoc.py
>> > +++ b/docs/sphinx/qapidoc.py
>> > @@ -24,17 +24,18 @@
>> > https://www.sphinx-doc.org/en/master/development/index.html
>> > """
>> >
>> > +from __future__ import annotations
>> > +
>> > import os
>> > import sys
>> > -from typing import List
>> > +from typing import TYPE_CHECKING
>> >
>> > from docutils import nodes
>> > from docutils.parsers.rst import Directive, directives
>> > from qapi.error import QAPIError
>> > -from qapi.gen import QAPISchemaVisitor
>> > -from qapi.schema import QAPISchema
>> > +from qapi.schema import QAPISchema, QAPISchemaVisitor
>> >
>> > -from qapidoc_legacy import QAPISchemaGenRSTVisitor
>> > +from qapidoc_legacy import QAPISchemaGenRSTVisitor # type: ignore
>> > from sphinx import addnodes
>> > from sphinx.directives.code import CodeBlock
>> > from sphinx.errors import ExtensionError
>> > @@ -42,6 +43,15 @@
>> > from sphinx.util.nodes import nested_parse_with_titles
>> >
>> >
>> > +if TYPE_CHECKING:
>> > + from typing import Any, List, Sequence
>> > +
>> > + from docutils.statemachine import StringList
>> > +
>> > + from sphinx.application import Sphinx
>> > + from sphinx.util.typing import ExtensionMetadata
>>
>> Can you briefly explain why this needs to be conditional?
>>
>
> No requisite, but if they aren't used outside of type hints, they don't
> actually need to be imported at runtime (when we use from __future__ import
> annotations). Improves startup speed slightly and potentially makes the
> plugin less porcelain at runtime.
Should we do that for all typing-only imports everywhere?
[...]
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 35/57] docs/qapidoc: Fix static typing on qapidoc.py
2025-03-09 5:37 ` Markus Armbruster
@ 2025-03-09 6:46 ` John Snow
0 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-09 6:46 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 2777 bytes --]
On Sun, Mar 9, 2025 at 12:38 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > On Fri, Mar 7, 2025 at 7:00 AM Markus Armbruster <armbru@redhat.com>
> wrote:
> >
> >> John Snow <jsnow@redhat.com> writes:
> >>
> >> > Now that the legacy code is factored out, fix up the typing on the
> >> > remaining code in qapidoc.py. Add a type ignore to qapi_legacy.py to
> >> > prevent the errors there from bleeding out into qapidoc.py.
> >> >
> >> > Signed-off-by: John Snow <jsnow@redhat.com>
> >> > ---
> >> > docs/sphinx/qapidoc.py | 40
> ++++++++++++++++++++++-------------
> >> > docs/sphinx/qapidoc_legacy.py | 1 +
> >> > 2 files changed, 26 insertions(+), 15 deletions(-)
> >> >
> >> > diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> >> > index f4abf42e7bf..5246832b68c 100644
> >> > --- a/docs/sphinx/qapidoc.py
> >> > +++ b/docs/sphinx/qapidoc.py
> >> > @@ -24,17 +24,18 @@
> >> > https://www.sphinx-doc.org/en/master/development/index.html
> >> > """
> >> >
> >> > +from __future__ import annotations
> >> > +
> >> > import os
> >> > import sys
> >> > -from typing import List
> >> > +from typing import TYPE_CHECKING
> >> >
> >> > from docutils import nodes
> >> > from docutils.parsers.rst import Directive, directives
> >> > from qapi.error import QAPIError
> >> > -from qapi.gen import QAPISchemaVisitor
> >> > -from qapi.schema import QAPISchema
> >> > +from qapi.schema import QAPISchema, QAPISchemaVisitor
> >> >
> >> > -from qapidoc_legacy import QAPISchemaGenRSTVisitor
> >> > +from qapidoc_legacy import QAPISchemaGenRSTVisitor # type: ignore
> >> > from sphinx import addnodes
> >> > from sphinx.directives.code import CodeBlock
> >> > from sphinx.errors import ExtensionError
> >> > @@ -42,6 +43,15 @@
> >> > from sphinx.util.nodes import nested_parse_with_titles
> >> >
> >> >
> >> > +if TYPE_CHECKING:
> >> > + from typing import Any, List, Sequence
> >> > +
> >> > + from docutils.statemachine import StringList
> >> > +
> >> > + from sphinx.application import Sphinx
> >> > + from sphinx.util.typing import ExtensionMetadata
> >>
> >> Can you briefly explain why this needs to be conditional?
> >>
> >
> > No requisite, but if they aren't used outside of type hints, they don't
> > actually need to be imported at runtime (when we use from __future__
> import
> > annotations). Improves startup speed slightly and potentially makes the
> > plugin less porcelain at runtime.
>
> Should we do that for all typing-only imports everywhere?
>
Maybe! It's probably not too important, I just noticed Sphinx internals
doing it a lot and started following along because it seemed like a good
idea.
[-- Attachment #2: Type: text/html, Size: 4123 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 36/57] do-not-merge
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (34 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 35/57] docs/qapidoc: Fix static typing on qapidoc.py John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 37/57] docs/qapidoc: add transmogrifier class stub John Snow
` (23 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Add strict typing to qapidoc.py for the remainder of this series.
Signed-off-by: John Snow <jsnow@redhat.com>
---
scripts/qapi-lint.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/qapi-lint.sh b/scripts/qapi-lint.sh
index 0e3ff002b69..dd3e6eb84c5 100755
--- a/scripts/qapi-lint.sh
+++ b/scripts/qapi-lint.sh
@@ -35,7 +35,7 @@ if [[ -f ../docs/sphinx/qapi_domain.py ]]; then
pushd ../docs/sphinx
set -x
- mypy --strict $files
+ PYTHONPATH=../../scripts/ mypy --follow-untyped-imports --strict $files qapidoc.py
flake8 --max-line-length=80 $files qapidoc.py
isort -c $files qapidoc.py
black --line-length 80 --check $files qapidoc.py
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 37/57] docs/qapidoc: add transmogrifier class stub
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (35 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 36/57] do-not-merge John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 38/57] docs/qapidoc: add visit_module() method John Snow
` (22 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Add the beginnings of the Transmogrifier class by adding the rST
conversion helpers that will be used to build the virtual rST document.
This version of the class does not actually "do anything" yet; each
individual feature is added one-at-a-time in the forthcoming commits.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapidoc.py | 73 ++++++++++++++++++++++++++++++++++++++++--
1 file changed, 70 insertions(+), 3 deletions(-)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index 5246832b68c..c243bb6faaa 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -26,14 +26,17 @@
from __future__ import annotations
+from contextlib import contextmanager
import os
import sys
from typing import TYPE_CHECKING
from docutils import nodes
from docutils.parsers.rst import Directive, directives
+from docutils.statemachine import StringList
from qapi.error import QAPIError
from qapi.schema import QAPISchema, QAPISchemaVisitor
+from qapi.source import QAPISourceInfo
from qapidoc_legacy import QAPISchemaGenRSTVisitor # type: ignore
from sphinx import addnodes
@@ -44,9 +47,12 @@
if TYPE_CHECKING:
- from typing import Any, List, Sequence
-
- from docutils.statemachine import StringList
+ from typing import (
+ Any,
+ Generator,
+ List,
+ Sequence,
+ )
from sphinx.application import Sphinx
from sphinx.util.typing import ExtensionMetadata
@@ -55,6 +61,67 @@
__version__ = "1.0"
+class Transmogrifier:
+ def __init__(self) -> None:
+ self._result = StringList()
+ self.indent = 0
+
+ # General-purpose rST generation functions
+
+ def get_indent(self) -> str:
+ return " " * self.indent
+
+ @contextmanager
+ def indented(self) -> Generator[None]:
+ self.indent += 1
+ try:
+ yield
+ finally:
+ self.indent -= 1
+
+ def add_line_raw(self, line: str, source: str, *lineno: int) -> None:
+ """Append one line of generated reST to the output."""
+
+ # NB: Sphinx uses zero-indexed lines; subtract one.
+ lineno = tuple((n - 1 for n in lineno))
+
+ if line.strip():
+ # not a blank line
+ self._result.append(
+ self.get_indent() + line.rstrip("\n"), source, *lineno
+ )
+ else:
+ self._result.append("", source, *lineno)
+
+ def add_line(self, content: str, info: QAPISourceInfo) -> None:
+ # NB: We *require* an info object; this works out OK because we
+ # don't document built-in objects that don't have
+ # one. Everything else should.
+ self.add_line_raw(content, info.fname, info.line)
+
+ def add_lines(
+ self,
+ content: str,
+ info: QAPISourceInfo,
+ ) -> None:
+ lines = content.splitlines(True)
+ for i, line in enumerate(lines):
+ self.add_line_raw(line, info.fname, info.line + i)
+
+ def ensure_blank_line(self) -> None:
+ # Empty document -- no blank line required.
+ if not self._result:
+ return
+
+ # Last line isn't blank, add one.
+ if self._result[-1].strip(): # pylint: disable=no-member
+ fname, line = self._result.info(-1)
+ assert isinstance(line, int)
+ # New blank line is credited to one-after the current last line.
+ # +2: correct for zero/one index, then increment by one.
+ self.add_line_raw("", fname, line + 2)
+
+
class QAPISchemaGenDepVisitor(QAPISchemaVisitor):
"""A QAPI schema visitor which adds Sphinx dependencies each module
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 38/57] docs/qapidoc: add visit_module() method
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (36 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 37/57] docs/qapidoc: add transmogrifier class stub John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 39/57] qapi/source: allow multi-line QAPISourceInfo advancing John Snow
` (21 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
This method annotates the start of a new module, crediting the source
location to the first line of the module file.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapidoc.py | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index c243bb6faaa..6de8c900543 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -28,6 +28,7 @@
from contextlib import contextmanager
import os
+from pathlib import Path
import sys
from typing import TYPE_CHECKING
@@ -121,6 +122,14 @@ def ensure_blank_line(self) -> None:
# +2: correct for zero/one index, then increment by one.
self.add_line_raw("", fname, line + 2)
+ # Transmogrification core methods
+
+ def visit_module(self, path: str) -> None:
+ name = Path(path).stem
+ # module directives are credited to the first line of a module file.
+ self.add_line_raw(f".. qapi:module:: {name}", path, 1)
+ self.ensure_blank_line()
+
class QAPISchemaGenDepVisitor(QAPISchemaVisitor):
"""A QAPI schema visitor which adds Sphinx dependencies each module
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 39/57] qapi/source: allow multi-line QAPISourceInfo advancing
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (37 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 38/57] docs/qapidoc: add visit_module() method John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 10:35 ` Markus Armbruster
2025-03-05 3:45 ` [PATCH 40/57] docs/qapidoc: add visit_freeform() method John Snow
` (20 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
This is for the sake of the new rST generator (the "transmogrifier") so
we can advance multiple lines on occasion while keeping the
generated<-->source mappings accurate.
next_line now simply takes an optional n parameter which chooses the
number of lines to advance.
RFC: Here's the exorbitant detail on why I want this:
This is used mainly when converting section syntax in free-form
documentation to more traditional rST section header syntax, which
does not always line up 1:1 for line counts.
For example:
```
##
# = Section <-- Info is pointing here, "L1"
#
# Lorem Ipsum
##
```
would be transformed to rST as:
```
======= <-- L1
Section <-- L1
======= <-- L1
<-- L2
Lorem Ipsum <-- L3
```
After consuming the single "Section" line from the source, we want to
advance the source pointer to the next non-empty line which requires
jumping by more than one line.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
---
scripts/qapi/source.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/scripts/qapi/source.py b/scripts/qapi/source.py
index 7b379fdc925..ffdc3f482ac 100644
--- a/scripts/qapi/source.py
+++ b/scripts/qapi/source.py
@@ -47,9 +47,9 @@ def set_defn(self, meta: str, name: str) -> None:
self.defn_meta = meta
self.defn_name = name
- def next_line(self: T) -> T:
+ def next_line(self: T, n: int = 1) -> T:
info = copy.copy(self)
- info.line += 1
+ info.line += n
return info
def loc(self) -> str:
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 39/57] qapi/source: allow multi-line QAPISourceInfo advancing
2025-03-05 3:45 ` [PATCH 39/57] qapi/source: allow multi-line QAPISourceInfo advancing John Snow
@ 2025-03-05 10:35 ` Markus Armbruster
2025-03-06 3:13 ` John Snow
0 siblings, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-05 10:35 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> This is for the sake of the new rST generator (the "transmogrifier") so
> we can advance multiple lines on occasion while keeping the
> generated<-->source mappings accurate.
>
> next_line now simply takes an optional n parameter which chooses the
> number of lines to advance.
>
> RFC: Here's the exorbitant detail on why I want this:
>
> This is used mainly when converting section syntax in free-form
> documentation to more traditional rST section header syntax, which
> does not always line up 1:1 for line counts.
Obvious way to resolve the RFC:
The next patch will use this when converting ...
>
> For example:
>
> ```
> ##
> # = Section <-- Info is pointing here, "L1"
> #
> # Lorem Ipsum
> ##
> ```
>
> would be transformed to rST as:
>
> ```
> ======= <-- L1
> Section <-- L1
> ======= <-- L1
> <-- L2
> Lorem Ipsum <-- L3
> ```
Not a demand, just wondering: could we drop our headings syntax and just
use rST?
>
> After consuming the single "Section" line from the source, we want to
> advance the source pointer to the next non-empty line which requires
> jumping by more than one line.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> Reviewed-by: Markus Armbruster <armbru@redhat.com>
> ---
> scripts/qapi/source.py | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/scripts/qapi/source.py b/scripts/qapi/source.py
> index 7b379fdc925..ffdc3f482ac 100644
> --- a/scripts/qapi/source.py
> +++ b/scripts/qapi/source.py
> @@ -47,9 +47,9 @@ def set_defn(self, meta: str, name: str) -> None:
> self.defn_meta = meta
> self.defn_name = name
>
> - def next_line(self: T) -> T:
> + def next_line(self: T, n: int = 1) -> T:
> info = copy.copy(self)
> - info.line += 1
> + info.line += n
> return info
>
> def loc(self) -> str:
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 39/57] qapi/source: allow multi-line QAPISourceInfo advancing
2025-03-05 10:35 ` Markus Armbruster
@ 2025-03-06 3:13 ` John Snow
2025-03-06 7:01 ` Markus Armbruster
0 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-06 3:13 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 1623 bytes --]
On Wed, Mar 5, 2025 at 5:35 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > This is for the sake of the new rST generator (the "transmogrifier") so
> > we can advance multiple lines on occasion while keeping the
> > generated<-->source mappings accurate.
> >
> > next_line now simply takes an optional n parameter which chooses the
> > number of lines to advance.
> >
> > RFC: Here's the exorbitant detail on why I want this:
> >
> > This is used mainly when converting section syntax in free-form
> > documentation to more traditional rST section header syntax, which
> > does not always line up 1:1 for line counts.
>
> Obvious way to resolve the RFC:
>
> The next patch will use this when converting ...
>
> >
> > For example:
> >
> > ```
> > ##
> > # = Section <-- Info is pointing here, "L1"
> > #
> > # Lorem Ipsum
> > ##
> > ```
> >
> > would be transformed to rST as:
> >
> > ```
> > ======= <-- L1
> > Section <-- L1
> > ======= <-- L1
> > <-- L2
> > Lorem Ipsum <-- L3
> > ```
>
> Not a demand, just wondering: could we drop our headings syntax and just
> use rST?
>
Yes, once we drop the old qapidoc fully, which I am not sure I can do
before freeze ... So we have some goofy stuff in the meantime.
You suggested before I can rewrite the freeform generator to avoid needing
this; I wrote the freeform generator to be as close to the old one as I
could, but we could kerjiggle it if needed.
... On the other hand, this is a patch for a += n, so... eh.
--js
[-- Attachment #2: Type: text/html, Size: 2338 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 39/57] qapi/source: allow multi-line QAPISourceInfo advancing
2025-03-06 3:13 ` John Snow
@ 2025-03-06 7:01 ` Markus Armbruster
0 siblings, 0 replies; 129+ messages in thread
From: Markus Armbruster @ 2025-03-06 7:01 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> On Wed, Mar 5, 2025 at 5:35 AM Markus Armbruster <armbru@redhat.com> wrote:
>
>> John Snow <jsnow@redhat.com> writes:
>>
>> > This is for the sake of the new rST generator (the "transmogrifier") so
>> > we can advance multiple lines on occasion while keeping the
>> > generated<-->source mappings accurate.
>> >
>> > next_line now simply takes an optional n parameter which chooses the
>> > number of lines to advance.
>> >
>> > RFC: Here's the exorbitant detail on why I want this:
>> >
>> > This is used mainly when converting section syntax in free-form
>> > documentation to more traditional rST section header syntax, which
>> > does not always line up 1:1 for line counts.
>>
>> Obvious way to resolve the RFC:
>>
>> The next patch will use this when converting ...
>>
>> >
>> > For example:
>> >
>> > ```
>> > ##
>> > # = Section <-- Info is pointing here, "L1"
>> > #
>> > # Lorem Ipsum
>> > ##
>> > ```
>> >
>> > would be transformed to rST as:
>> >
>> > ```
>> > ======= <-- L1
>> > Section <-- L1
>> > ======= <-- L1
>> > <-- L2
>> > Lorem Ipsum <-- L3
>> > ```
>>
>> Not a demand, just wondering: could we drop our headings syntax and just
>> use rST?
>>
>
> Yes, once we drop the old qapidoc fully, which I am not sure I can do
> before freeze ... So we have some goofy stuff in the meantime.
No need to do it before the freeze. Trying to do it before might be
self-sabotage ;)
> You suggested before I can rewrite the freeform generator to avoid needing
> this; I wrote the freeform generator to be as close to the old one as I
> could, but we could kerjiggle it if needed.
>
> ... On the other hand, this is a patch for a += n, so... eh.
>
> --js
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 40/57] docs/qapidoc: add visit_freeform() method
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (38 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 39/57] qapi/source: allow multi-line QAPISourceInfo advancing John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-07 12:10 ` Markus Armbruster
2025-03-05 3:45 ` [PATCH 41/57] docs/qapidoc: add preamble() method John Snow
` (19 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapidoc.py | 50 ++++++++++++++++++++++++++++++++++++++++++
1 file changed, 50 insertions(+)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index 6de8c900543..cf5dbb0133d 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -29,6 +29,7 @@
from contextlib import contextmanager
import os
from pathlib import Path
+import re
import sys
from typing import TYPE_CHECKING
@@ -55,6 +56,8 @@
Sequence,
)
+ from qapi.parser import QAPIDoc
+
from sphinx.application import Sphinx
from sphinx.util.typing import ExtensionMetadata
@@ -130,6 +133,53 @@ def visit_module(self, path: str) -> None:
self.add_line_raw(f".. qapi:module:: {name}", path, 1)
self.ensure_blank_line()
+ def visit_freeform(self, doc: QAPIDoc) -> None:
+ # TODO: Once the old qapidoc transformer is deprecated, freeform
+ # sections can be updated to pure rST, and this transformed removed.
+ #
+ # For now, translate our micro-format into rST. Code adapted
+ # from Peter Maydell's freeform().
+
+ assert len(doc.all_sections) == 1, doc.all_sections
+ body = doc.all_sections[0]
+ text = body.text
+ info = doc.info
+
+ if re.match(r"=+ ", text):
+ # Section/subsection heading (if present, will always be the
+ # first line of the block)
+ (heading, _, text) = text.partition("\n")
+ (leader, _, heading) = heading.partition(" ")
+ level = len(leader) + 1 # Implicit +1 for heading in .rST stub
+
+ # https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#sections
+ markers = {
+ 1: "#",
+ 2: "*",
+ 3: "=",
+ 4: "-",
+ 5: "^",
+ 6: '"',
+ }
+ overline = level <= 2
+ marker = markers[level]
+
+ self.ensure_blank_line()
+ # This credits all 2 or 3 lines to the single source line.
+ if overline:
+ self.add_line(marker * len(heading), info)
+ self.add_line(heading, info)
+ self.add_line(marker * len(heading), info)
+ self.ensure_blank_line()
+
+ # Eat blank line(s) and advance info
+ trimmed = text.lstrip("\n")
+ text = trimmed
+ info = info.next_line(len(text) - len(trimmed) + 1)
+
+ self.add_lines(text, info)
+ self.ensure_blank_line()
+
class QAPISchemaGenDepVisitor(QAPISchemaVisitor):
"""A QAPI schema visitor which adds Sphinx dependencies each module
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 40/57] docs/qapidoc: add visit_freeform() method
2025-03-05 3:45 ` [PATCH 40/57] docs/qapidoc: add visit_freeform() method John Snow
@ 2025-03-07 12:10 ` Markus Armbruster
2025-03-08 8:02 ` John Snow
0 siblings, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-07 12:10 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé, Markus Armbruster
John Snow <jsnow@redhat.com> writes:
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> docs/sphinx/qapidoc.py | 50 ++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 50 insertions(+)
>
> diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> index 6de8c900543..cf5dbb0133d 100644
> --- a/docs/sphinx/qapidoc.py
> +++ b/docs/sphinx/qapidoc.py
> @@ -29,6 +29,7 @@
> from contextlib import contextmanager
> import os
> from pathlib import Path
> +import re
> import sys
> from typing import TYPE_CHECKING
>
> @@ -55,6 +56,8 @@
> Sequence,
> )
>
> + from qapi.parser import QAPIDoc
> +
> from sphinx.application import Sphinx
> from sphinx.util.typing import ExtensionMetadata
>
> @@ -130,6 +133,53 @@ def visit_module(self, path: str) -> None:
> self.add_line_raw(f".. qapi:module:: {name}", path, 1)
> self.ensure_blank_line()
>
> + def visit_freeform(self, doc: QAPIDoc) -> None:
> + # TODO: Once the old qapidoc transformer is deprecated, freeform
> + # sections can be updated to pure rST, and this transformed removed.
> + #
> + # For now, translate our micro-format into rST. Code adapted
> + # from Peter Maydell's freeform().
> +
> + assert len(doc.all_sections) == 1, doc.all_sections
> + body = doc.all_sections[0]
> + text = body.text
> + info = doc.info
> +
> + if re.match(r"=+ ", text):
> + # Section/subsection heading (if present, will always be the
> + # first line of the block)
> + (heading, _, text) = text.partition("\n")
> + (leader, _, heading) = heading.partition(" ")
> + level = len(leader) + 1 # Implicit +1 for heading in .rST stub
What is the ".rST stub"?
> +
> + # https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#sections
> + markers = {
> + 1: "#",
> + 2: "*",
> + 3: "=",
> + 4: "-",
> + 5: "^",
> + 6: '"',
> + }
I'd be tempted to use markers = '#*=-^". Matter of taste, yours takes
precedence here.
> + overline = level <= 2
> + marker = markers[level]
> +
> + self.ensure_blank_line()
> + # This credits all 2 or 3 lines to the single source line.
> + if overline:
> + self.add_line(marker * len(heading), info)
> + self.add_line(heading, info)
> + self.add_line(marker * len(heading), info)
> + self.ensure_blank_line()
> +
> + # Eat blank line(s) and advance info
> + trimmed = text.lstrip("\n")
> + text = trimmed
> + info = info.next_line(len(text) - len(trimmed) + 1)
> +
> + self.add_lines(text, info)
> + self.ensure_blank_line()
> +
>
> class QAPISchemaGenDepVisitor(QAPISchemaVisitor):
> """A QAPI schema visitor which adds Sphinx dependencies each module
^ permalink raw reply [flat|nested] 129+ messages in thread* Re: [PATCH 40/57] docs/qapidoc: add visit_freeform() method
2025-03-07 12:10 ` Markus Armbruster
@ 2025-03-08 8:02 ` John Snow
0 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-08 8:02 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 3672 bytes --]
On Fri, Mar 7, 2025 at 7:10 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > Signed-off-by: John Snow <jsnow@redhat.com>
> > ---
> > docs/sphinx/qapidoc.py | 50 ++++++++++++++++++++++++++++++++++++++++++
> > 1 file changed, 50 insertions(+)
> >
> > diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> > index 6de8c900543..cf5dbb0133d 100644
> > --- a/docs/sphinx/qapidoc.py
> > +++ b/docs/sphinx/qapidoc.py
> > @@ -29,6 +29,7 @@
> > from contextlib import contextmanager
> > import os
> > from pathlib import Path
> > +import re
> > import sys
> > from typing import TYPE_CHECKING
> >
> > @@ -55,6 +56,8 @@
> > Sequence,
> > )
> >
> > + from qapi.parser import QAPIDoc
> > +
> > from sphinx.application import Sphinx
> > from sphinx.util.typing import ExtensionMetadata
> >
> > @@ -130,6 +133,53 @@ def visit_module(self, path: str) -> None:
> > self.add_line_raw(f".. qapi:module:: {name}", path, 1)
> > self.ensure_blank_line()
> >
> > + def visit_freeform(self, doc: QAPIDoc) -> None:
> > + # TODO: Once the old qapidoc transformer is deprecated, freeform
> > + # sections can be updated to pure rST, and this transformed
> removed.
> > + #
> > + # For now, translate our micro-format into rST. Code adapted
> > + # from Peter Maydell's freeform().
> > +
> > + assert len(doc.all_sections) == 1, doc.all_sections
> > + body = doc.all_sections[0]
> > + text = body.text
> > + info = doc.info
> > +
> > + if re.match(r"=+ ", text):
> > + # Section/subsection heading (if present, will always be the
> > + # first line of the block)
> > + (heading, _, text) = text.partition("\n")
> > + (leader, _, heading) = heading.partition(" ")
>
>
>
> > + level = len(leader) + 1 # Implicit +1 for heading in .rST
> stub
>
> What is the ".rST stub"?
>
sorry, I meant the .rst document in-tree that invokes the qapidoc
directive. that document inherently has a title, so I treat everything in
the generated doc as a sub-heading of some kind.
(i.e., only one <h1>)
>
> > +
> > + #
> https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#sections
> > + markers = {
> > + 1: "#",
> > + 2: "*",
> > + 3: "=",
> > + 4: "-",
> > + 5: "^",
> > + 6: '"',
> > + }
>
> I'd be tempted to use markers = '#*=-^". Matter of taste, yours takes
> precedence here.
>
Oh, yeah. I think I need a vacation from Python.
>
> > + overline = level <= 2
> > + marker = markers[level]
> > +
> > + self.ensure_blank_line()
> > + # This credits all 2 or 3 lines to the single source line.
> > + if overline:
> > + self.add_line(marker * len(heading), info)
> > + self.add_line(heading, info)
> > + self.add_line(marker * len(heading), info)
> > + self.ensure_blank_line()
> > +
> > + # Eat blank line(s) and advance info
> > + trimmed = text.lstrip("\n")
> > + text = trimmed
> > + info = info.next_line(len(text) - len(trimmed) + 1)
> > +
> > + self.add_lines(text, info)
> > + self.ensure_blank_line()
> > +
> >
> > class QAPISchemaGenDepVisitor(QAPISchemaVisitor):
> > """A QAPI schema visitor which adds Sphinx dependencies each module
>
>
[-- Attachment #2: Type: text/html, Size: 5524 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 41/57] docs/qapidoc: add preamble() method
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (39 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 40/57] docs/qapidoc: add visit_freeform() method John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-07 12:13 ` Markus Armbruster
2025-03-05 3:45 ` [PATCH 42/57] docs/qapidoc: add visit_paragraph() method John Snow
` (18 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
This method adds the options/preamble to each definition block. Notably,
:since: and :ifcond: are added, as are any "special features" such as
:deprecated: and :unstable:.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapidoc.py | 41 ++++++++++++++++++++++++++++++++++++++---
1 file changed, 38 insertions(+), 3 deletions(-)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index cf5dbb0133d..d8bf0073dfa 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -37,7 +37,12 @@
from docutils.parsers.rst import Directive, directives
from docutils.statemachine import StringList
from qapi.error import QAPIError
-from qapi.schema import QAPISchema, QAPISchemaVisitor
+from qapi.parser import QAPIDoc
+from qapi.schema import (
+ QAPISchema,
+ QAPISchemaDefinition,
+ QAPISchemaVisitor,
+)
from qapi.source import QAPISourceInfo
from qapidoc_legacy import QAPISchemaGenRSTVisitor # type: ignore
@@ -56,8 +61,6 @@
Sequence,
)
- from qapi.parser import QAPIDoc
-
from sphinx.application import Sphinx
from sphinx.util.typing import ExtensionMetadata
@@ -125,6 +128,38 @@ def ensure_blank_line(self) -> None:
# +2: correct for zero/one index, then increment by one.
self.add_line_raw("", fname, line + 2)
+ # Transmogrification helpers
+
+ def preamble(self, ent: QAPISchemaDefinition) -> None:
+ """
+ Generate option lines for qapi entity directives.
+ """
+ if ent.doc and ent.doc.since:
+ assert ent.doc.since.kind == QAPIDoc.Kind.SINCE
+ # Generated from the entity's docblock; info location is exact.
+ self.add_line(f":since: {ent.doc.since.text}", ent.doc.since.info)
+
+ if ent.ifcond.is_present():
+ doc = ent.ifcond.docgen()
+ assert ent.info
+ # Generated from entity definition; info location is approximate.
+ self.add_line(f":ifcond: {doc}", ent.info)
+
+ # Hoist special features such as :deprecated: and :unstable:
+ # into the options block for the entity. If, in the future, new
+ # special features are added, qapi-domain will chirp about
+ # unrecognized options and fail until they are handled in
+ # qapi-domain.
+ for feat in ent.features:
+ if feat.is_special():
+ # FIXME: handle ifcond if present. How to display that
+ # information is TBD.
+ # Generated from entity def; info location is approximate.
+ assert feat.info
+ self.add_line(f":{feat.name}:", feat.info)
+
+ self.ensure_blank_line()
+
# Transmogrification core methods
def visit_module(self, path: str) -> None:
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 41/57] docs/qapidoc: add preamble() method
2025-03-05 3:45 ` [PATCH 41/57] docs/qapidoc: add preamble() method John Snow
@ 2025-03-07 12:13 ` Markus Armbruster
2025-03-08 8:03 ` John Snow
0 siblings, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-07 12:13 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> This method adds the options/preamble to each definition block. Notably,
> :since: and :ifcond: are added, as are any "special features" such as
> :deprecated: and :unstable:.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> docs/sphinx/qapidoc.py | 41 ++++++++++++++++++++++++++++++++++++++---
> 1 file changed, 38 insertions(+), 3 deletions(-)
>
> diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> index cf5dbb0133d..d8bf0073dfa 100644
> --- a/docs/sphinx/qapidoc.py
> +++ b/docs/sphinx/qapidoc.py
> @@ -37,7 +37,12 @@
> from docutils.parsers.rst import Directive, directives
> from docutils.statemachine import StringList
> from qapi.error import QAPIError
> -from qapi.schema import QAPISchema, QAPISchemaVisitor
> +from qapi.parser import QAPIDoc
> +from qapi.schema import (
> + QAPISchema,
> + QAPISchemaDefinition,
> + QAPISchemaVisitor,
> +)
> from qapi.source import QAPISourceInfo
>
> from qapidoc_legacy import QAPISchemaGenRSTVisitor # type: ignore
> @@ -56,8 +61,6 @@
> Sequence,
> )
>
> - from qapi.parser import QAPIDoc
> -
Some kind of accident?
> from sphinx.application import Sphinx
> from sphinx.util.typing import ExtensionMetadata
>
> @@ -125,6 +128,38 @@ def ensure_blank_line(self) -> None:
> # +2: correct for zero/one index, then increment by one.
> self.add_line_raw("", fname, line + 2)
>
> + # Transmogrification helpers
> +
> + def preamble(self, ent: QAPISchemaDefinition) -> None:
> + """
> + Generate option lines for qapi entity directives.
> + """
> + if ent.doc and ent.doc.since:
> + assert ent.doc.since.kind == QAPIDoc.Kind.SINCE
> + # Generated from the entity's docblock; info location is exact.
> + self.add_line(f":since: {ent.doc.since.text}", ent.doc.since.info)
Break the line afte the comma?
> +
> + if ent.ifcond.is_present():
> + doc = ent.ifcond.docgen()
> + assert ent.info
> + # Generated from entity definition; info location is approximate.
> + self.add_line(f":ifcond: {doc}", ent.info)
> +
> + # Hoist special features such as :deprecated: and :unstable:
> + # into the options block for the entity. If, in the future, new
> + # special features are added, qapi-domain will chirp about
> + # unrecognized options and fail until they are handled in
> + # qapi-domain.
> + for feat in ent.features:
> + if feat.is_special():
> + # FIXME: handle ifcond if present. How to display that
> + # information is TBD.
Is the FIXME worth mentioning in the commit message?
> + # Generated from entity def; info location is approximate.
> + assert feat.info
> + self.add_line(f":{feat.name}:", feat.info)
> +
> + self.ensure_blank_line()
> +
> # Transmogrification core methods
>
> def visit_module(self, path: str) -> None:
^ permalink raw reply [flat|nested] 129+ messages in thread* Re: [PATCH 41/57] docs/qapidoc: add preamble() method
2025-03-07 12:13 ` Markus Armbruster
@ 2025-03-08 8:03 ` John Snow
0 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-08 8:03 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 3528 bytes --]
On Fri, Mar 7, 2025 at 7:13 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > This method adds the options/preamble to each definition block. Notably,
> > :since: and :ifcond: are added, as are any "special features" such as
> > :deprecated: and :unstable:.
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
> > ---
> > docs/sphinx/qapidoc.py | 41 ++++++++++++++++++++++++++++++++++++++---
> > 1 file changed, 38 insertions(+), 3 deletions(-)
> >
> > diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> > index cf5dbb0133d..d8bf0073dfa 100644
> > --- a/docs/sphinx/qapidoc.py
> > +++ b/docs/sphinx/qapidoc.py
> > @@ -37,7 +37,12 @@
> > from docutils.parsers.rst import Directive, directives
> > from docutils.statemachine import StringList
> > from qapi.error import QAPIError
> > -from qapi.schema import QAPISchema, QAPISchemaVisitor
> > +from qapi.parser import QAPIDoc
> > +from qapi.schema import (
> > + QAPISchema,
> > + QAPISchemaDefinition,
> > + QAPISchemaVisitor,
> > +)
> > from qapi.source import QAPISourceInfo
> >
> > from qapidoc_legacy import QAPISchemaGenRSTVisitor # type: ignore
> > @@ -56,8 +61,6 @@
> > Sequence,
> > )
> >
> > - from qapi.parser import QAPIDoc
> > -
>
> Some kind of accident?
>
> > from sphinx.application import Sphinx
> > from sphinx.util.typing import ExtensionMetadata
> >
> > @@ -125,6 +128,38 @@ def ensure_blank_line(self) -> None:
> > # +2: correct for zero/one index, then increment by one.
> > self.add_line_raw("", fname, line + 2)
> >
> > + # Transmogrification helpers
> > +
> > + def preamble(self, ent: QAPISchemaDefinition) -> None:
> > + """
> > + Generate option lines for qapi entity directives.
> > + """
> > + if ent.doc and ent.doc.since:
> > + assert ent.doc.since.kind == QAPIDoc.Kind.SINCE
> > + # Generated from the entity's docblock; info location is
> exact.
> > + self.add_line(f":since: {ent.doc.since.text}",
> ent.doc.since.info)
>
> Break the line afte the comma?
>
Is this an 80col vs 79col thing? I'll tighten it up.
>
> > +
> > + if ent.ifcond.is_present():
> > + doc = ent.ifcond.docgen()
> > + assert ent.info
> > + # Generated from entity definition; info location is
> approximate.
> > + self.add_line(f":ifcond: {doc}", ent.info)
> > +
> > + # Hoist special features such as :deprecated: and :unstable:
> > + # into the options block for the entity. If, in the future, new
> > + # special features are added, qapi-domain will chirp about
> > + # unrecognized options and fail until they are handled in
> > + # qapi-domain.
> > + for feat in ent.features:
> > + if feat.is_special():
> > + # FIXME: handle ifcond if present. How to display that
> > + # information is TBD.
>
> Is the FIXME worth mentioning in the commit message?
>
Yes, and yes to all subsequent asks.
>
> > + # Generated from entity def; info location is
> approximate.
> > + assert feat.info
> > + self.add_line(f":{feat.name}:", feat.info)
> > +
> > + self.ensure_blank_line()
> > +
> > # Transmogrification core methods
> >
> > def visit_module(self, path: str) -> None:
>
>
[-- Attachment #2: Type: text/html, Size: 5381 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 42/57] docs/qapidoc: add visit_paragraph() method
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (40 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 41/57] docs/qapidoc: add preamble() method John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 43/57] docs/qapidoc: add visit_errors() method John Snow
` (17 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
This transforms "formerly known as untagged sections" into our pure
intermediate rST format. These sections are already pure rST, so this
method doesn't do a whole lot except ensure appropriate newlines.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapidoc.py | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index d8bf0073dfa..b96445f0802 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -130,6 +130,15 @@ def ensure_blank_line(self) -> None:
# Transmogrification helpers
+ def visit_paragraph(self, section: QAPIDoc.Section) -> None:
+ # Squelch empty paragraphs.
+ if not section.text:
+ return
+
+ self.ensure_blank_line()
+ self.add_lines(section.text, section.info)
+ self.ensure_blank_line()
+
def preamble(self, ent: QAPISchemaDefinition) -> None:
"""
Generate option lines for qapi entity directives.
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 43/57] docs/qapidoc: add visit_errors() method
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (41 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 42/57] docs/qapidoc: add visit_paragraph() method John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-07 12:14 ` Markus Armbruster
2025-03-05 3:45 ` [PATCH 44/57] docs/qapidoc: add format_type() method John Snow
` (16 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Notably, this method does not currently address the formatting issues
present with the "errors" section in QAPIDoc and just vomits the text
verbatim into the rST doc, with somewhat inconsistent results.
To be addressed in a future revision.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapidoc.py | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index b96445f0802..14feafe866e 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -139,6 +139,12 @@ def visit_paragraph(self, section: QAPIDoc.Section) -> None:
self.add_lines(section.text, section.info)
self.ensure_blank_line()
+ def visit_errors(self, section: QAPIDoc.Section) -> None:
+ # FIXME: the formatting for errors may be inconsistent and may
+ # or may not require different newline placement to ensure
+ # proper rendering as a nested list.
+ self.add_lines(f":error:\n{section.text}", section.info)
+
def preamble(self, ent: QAPISchemaDefinition) -> None:
"""
Generate option lines for qapi entity directives.
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 43/57] docs/qapidoc: add visit_errors() method
2025-03-05 3:45 ` [PATCH 43/57] docs/qapidoc: add visit_errors() method John Snow
@ 2025-03-07 12:14 ` Markus Armbruster
0 siblings, 0 replies; 129+ messages in thread
From: Markus Armbruster @ 2025-03-07 12:14 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> Notably, this method does not currently address the formatting issues
> present with the "errors" section in QAPIDoc and just vomits the text
> verbatim into the rST doc, with somewhat inconsistent results.
>
> To be addressed in a future revision.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> docs/sphinx/qapidoc.py | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> index b96445f0802..14feafe866e 100644
> --- a/docs/sphinx/qapidoc.py
> +++ b/docs/sphinx/qapidoc.py
> @@ -139,6 +139,12 @@ def visit_paragraph(self, section: QAPIDoc.Section) -> None:
> self.add_lines(section.text, section.info)
> self.ensure_blank_line()
>
> + def visit_errors(self, section: QAPIDoc.Section) -> None:
> + # FIXME: the formatting for errors may be inconsistent and may
> + # or may not require different newline placement to ensure
> + # proper rendering as a nested list.
Is the FIXME worth mentioning in the commit message?
> + self.add_lines(f":error:\n{section.text}", section.info)
> +
> def preamble(self, ent: QAPISchemaDefinition) -> None:
> """
> Generate option lines for qapi entity directives.
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 44/57] docs/qapidoc: add format_type() method
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (42 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 43/57] docs/qapidoc: add visit_errors() method John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 45/57] docs/qapidoc: add add_field() and generate_field() helper methods John Snow
` (15 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
This method is responsible for generating a type name for a given member
with the correct annotations for the QAPI domain. Features and enums do
not *have* types, so they return None. Everything else returns the type
name with a "?" suffix if that type is optional, and ensconced in
[brackets] if it's an array type.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapidoc.py | 32 ++++++++++++++++++++++++++++++++
1 file changed, 32 insertions(+)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index 14feafe866e..0f895a3624a 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -40,7 +40,13 @@
from qapi.parser import QAPIDoc
from qapi.schema import (
QAPISchema,
+ QAPISchemaArrayType,
QAPISchemaDefinition,
+ QAPISchemaEnumMember,
+ QAPISchemaFeature,
+ QAPISchemaMember,
+ QAPISchemaObjectTypeMember,
+ QAPISchemaType,
QAPISchemaVisitor,
)
from qapi.source import QAPISourceInfo
@@ -58,7 +64,9 @@
Any,
Generator,
List,
+ Optional,
Sequence,
+ Union,
)
from sphinx.application import Sphinx
@@ -128,6 +136,30 @@ def ensure_blank_line(self) -> None:
# +2: correct for zero/one index, then increment by one.
self.add_line_raw("", fname, line + 2)
+ def format_type(
+ self, ent: Union[QAPISchemaDefinition | QAPISchemaMember]
+ ) -> Optional[str]:
+ if isinstance(ent, (QAPISchemaEnumMember, QAPISchemaFeature)):
+ return None
+
+ qapi_type = ent
+ optional = False
+ if isinstance(ent, QAPISchemaObjectTypeMember):
+ qapi_type = ent.type
+ optional = ent.optional
+
+ if isinstance(qapi_type, QAPISchemaArrayType):
+ ret = f"[{qapi_type.element_type.doc_type()}]"
+ else:
+ assert isinstance(qapi_type, QAPISchemaType)
+ tmp = qapi_type.doc_type()
+ assert tmp
+ ret = tmp
+ if optional:
+ ret += "?"
+
+ return ret
+
# Transmogrification helpers
def visit_paragraph(self, section: QAPIDoc.Section) -> None:
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 45/57] docs/qapidoc: add add_field() and generate_field() helper methods
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (43 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 44/57] docs/qapidoc: add format_type() method John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 46/57] docs/qapidoc: add visit_feature() method John Snow
` (14 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
These are simple rST generation methods that assist in getting the types
and formatting correct for a field list entry. add_field() is a more
raw, direct call while generate_field() is intended to be used for
generating the correct field from a member object.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapidoc.py | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index 0f895a3624a..b87ce288837 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -136,6 +136,20 @@ def ensure_blank_line(self) -> None:
# +2: correct for zero/one index, then increment by one.
self.add_line_raw("", fname, line + 2)
+ def add_field(
+ self,
+ kind: str,
+ name: str,
+ body: str,
+ info: QAPISourceInfo,
+ typ: Optional[str] = None,
+ ) -> None:
+ if typ:
+ text = f":{kind} {typ} {name}: {body}"
+ else:
+ text = f":{kind} {name}: {body}"
+ self.add_lines(text, info)
+
def format_type(
self, ent: Union[QAPISchemaDefinition | QAPISchemaMember]
) -> Optional[str]:
@@ -160,6 +174,16 @@ def format_type(
return ret
+ def generate_field(
+ self,
+ kind: str,
+ member: QAPISchemaMember,
+ body: str,
+ info: QAPISourceInfo,
+ ) -> None:
+ typ = self.format_type(member)
+ self.add_field(kind, member.name, body, info, typ)
+
# Transmogrification helpers
def visit_paragraph(self, section: QAPIDoc.Section) -> None:
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 46/57] docs/qapidoc: add visit_feature() method
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (44 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 45/57] docs/qapidoc: add add_field() and generate_field() helper methods John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-07 12:16 ` Markus Armbruster
2025-03-05 3:45 ` [PATCH 47/57] docs/qapidoc: prepare to record entity being transmogrified John Snow
` (13 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
This adds a simple ":feat name: lorem ipsum ..." line to the generated
rST document, so at the moment it's only for "top level" features.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapidoc.py | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index b87ce288837..eaea19af7ac 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -195,6 +195,15 @@ def visit_paragraph(self, section: QAPIDoc.Section) -> None:
self.add_lines(section.text, section.info)
self.ensure_blank_line()
+ def visit_feature(self, section: QAPIDoc.ArgSection) -> None:
+ # FIXME - ifcond for features is not handled at all yet!
+ # Proposal: decorate the right-hand column with some graphical
+ # element to indicate conditional availability?
+ assert section.text # Guaranteed by parser.py
+ assert section.member
+
+ self.generate_field("feat", section.member, section.text, section.info)
+
def visit_errors(self, section: QAPIDoc.Section) -> None:
# FIXME: the formatting for errors may be inconsistent and may
# or may not require different newline placement to ensure
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 46/57] docs/qapidoc: add visit_feature() method
2025-03-05 3:45 ` [PATCH 46/57] docs/qapidoc: add visit_feature() method John Snow
@ 2025-03-07 12:16 ` Markus Armbruster
0 siblings, 0 replies; 129+ messages in thread
From: Markus Armbruster @ 2025-03-07 12:16 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> This adds a simple ":feat name: lorem ipsum ..." line to the generated
> rST document, so at the moment it's only for "top level" features.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> docs/sphinx/qapidoc.py | 9 +++++++++
> 1 file changed, 9 insertions(+)
>
> diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> index b87ce288837..eaea19af7ac 100644
> --- a/docs/sphinx/qapidoc.py
> +++ b/docs/sphinx/qapidoc.py
> @@ -195,6 +195,15 @@ def visit_paragraph(self, section: QAPIDoc.Section) -> None:
> self.add_lines(section.text, section.info)
> self.ensure_blank_line()
>
> + def visit_feature(self, section: QAPIDoc.ArgSection) -> None:
> + # FIXME - ifcond for features is not handled at all yet!
> + # Proposal: decorate the right-hand column with some graphical
> + # element to indicate conditional availability?
Is the FIXME worth mentioning in the commit message?
> + assert section.text # Guaranteed by parser.py
> + assert section.member
> +
> + self.generate_field("feat", section.member, section.text, section.info)
> +
> def visit_errors(self, section: QAPIDoc.Section) -> None:
> # FIXME: the formatting for errors may be inconsistent and may
> # or may not require different newline placement to ensure
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 47/57] docs/qapidoc: prepare to record entity being transmogrified
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (45 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 46/57] docs/qapidoc: add visit_feature() method John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-05 3:45 ` [PATCH 48/57] docs/qapidoc: add visit_returns() method John Snow
` (12 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Prepare to keep a record of which entity we're working on documenting
for the purposes of being able to change certain generative features
conditionally and create stronger assertions.
If you find yourself asking: "Wait, but where does the current entity
actually get recorded?!", you're right! That part comes with the
visit_entity() implementation, which gets added later.
This patch is front-loaded for the sake of type checking in the
forthcoming commits before visit_entity() is ready to be added.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapidoc.py | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index eaea19af7ac..834f12ba6e9 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -78,9 +78,15 @@
class Transmogrifier:
def __init__(self) -> None:
+ self._curr_ent: Optional[QAPISchemaDefinition] = None
self._result = StringList()
self.indent = 0
+ @property
+ def entity(self) -> QAPISchemaDefinition:
+ assert self._curr_ent is not None
+ return self._curr_ent
+
# General-purpose rST generation functions
def get_indent(self) -> str:
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 48/57] docs/qapidoc: add visit_returns() method
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (46 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 47/57] docs/qapidoc: prepare to record entity being transmogrified John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-07 12:18 ` Markus Armbruster
2025-03-05 3:45 ` [PATCH 49/57] docs/qapidoc: add visit_member() method John Snow
` (11 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Generates :returns: fields for explicit returns statements. Note that
this does not presently handle undocumented returns, which is handled in
a later commit.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapidoc.py | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index 834f12ba6e9..6458790fe55 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -41,6 +41,7 @@
from qapi.schema import (
QAPISchema,
QAPISchemaArrayType,
+ QAPISchemaCommand,
QAPISchemaDefinition,
QAPISchemaEnumMember,
QAPISchemaFeature,
@@ -210,6 +211,20 @@ def visit_feature(self, section: QAPIDoc.ArgSection) -> None:
self.generate_field("feat", section.member, section.text, section.info)
+ def visit_returns(self, section: QAPIDoc.Section) -> None:
+ assert isinstance(self.entity, QAPISchemaCommand)
+ rtype = self.entity.ret_type
+ # q_empty can produce None, but we won't be documenting anything
+ # without an explicit return statement in the doc block, and we
+ # should not have any such explicit statements when there is no
+ # return value.
+ assert rtype
+
+ typ = self.format_type(rtype)
+ assert typ
+ assert section.text # We don't expect empty returns sections.
+ self.add_field("returns", typ, section.text, section.info)
+
def visit_errors(self, section: QAPIDoc.Section) -> None:
# FIXME: the formatting for errors may be inconsistent and may
# or may not require different newline placement to ensure
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 48/57] docs/qapidoc: add visit_returns() method
2025-03-05 3:45 ` [PATCH 48/57] docs/qapidoc: add visit_returns() method John Snow
@ 2025-03-07 12:18 ` Markus Armbruster
2025-03-08 8:06 ` John Snow
0 siblings, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-07 12:18 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> Generates :returns: fields for explicit returns statements. Note that
> this does not presently handle undocumented returns, which is handled in
> a later commit.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> docs/sphinx/qapidoc.py | 15 +++++++++++++++
> 1 file changed, 15 insertions(+)
>
> diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> index 834f12ba6e9..6458790fe55 100644
> --- a/docs/sphinx/qapidoc.py
> +++ b/docs/sphinx/qapidoc.py
> @@ -41,6 +41,7 @@
> from qapi.schema import (
> QAPISchema,
> QAPISchemaArrayType,
> + QAPISchemaCommand,
> QAPISchemaDefinition,
> QAPISchemaEnumMember,
> QAPISchemaFeature,
> @@ -210,6 +211,20 @@ def visit_feature(self, section: QAPIDoc.ArgSection) -> None:
>
> self.generate_field("feat", section.member, section.text, section.info)
>
> + def visit_returns(self, section: QAPIDoc.Section) -> None:
> + assert isinstance(self.entity, QAPISchemaCommand)
> + rtype = self.entity.ret_type
> + # q_empty can produce None, but we won't be documenting anything
> + # without an explicit return statement in the doc block, and we
> + # should not have any such explicit statements when there is no
> + # return value.
> + assert rtype
> +
> + typ = self.format_type(rtype)
> + assert typ
> + assert section.text # We don't expect empty returns sections.
Not sure the comment is worth its keep. Up to you.
> + self.add_field("returns", typ, section.text, section.info)
> +
> def visit_errors(self, section: QAPIDoc.Section) -> None:
> # FIXME: the formatting for errors may be inconsistent and may
> # or may not require different newline placement to ensure
^ permalink raw reply [flat|nested] 129+ messages in thread* Re: [PATCH 48/57] docs/qapidoc: add visit_returns() method
2025-03-07 12:18 ` Markus Armbruster
@ 2025-03-08 8:06 ` John Snow
0 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-08 8:06 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 2323 bytes --]
On Fri, Mar 7, 2025 at 7:18 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > Generates :returns: fields for explicit returns statements. Note that
> > this does not presently handle undocumented returns, which is handled in
> > a later commit.
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
> > ---
> > docs/sphinx/qapidoc.py | 15 +++++++++++++++
> > 1 file changed, 15 insertions(+)
> >
> > diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> > index 834f12ba6e9..6458790fe55 100644
> > --- a/docs/sphinx/qapidoc.py
> > +++ b/docs/sphinx/qapidoc.py
> > @@ -41,6 +41,7 @@
> > from qapi.schema import (
> > QAPISchema,
> > QAPISchemaArrayType,
> > + QAPISchemaCommand,
> > QAPISchemaDefinition,
> > QAPISchemaEnumMember,
> > QAPISchemaFeature,
> > @@ -210,6 +211,20 @@ def visit_feature(self, section:
> QAPIDoc.ArgSection) -> None:
> >
> > self.generate_field("feat", section.member, section.text,
> section.info)
> >
> > + def visit_returns(self, section: QAPIDoc.Section) -> None:
> > + assert isinstance(self.entity, QAPISchemaCommand)
> > + rtype = self.entity.ret_type
> > + # q_empty can produce None, but we won't be documenting anything
> > + # without an explicit return statement in the doc block, and we
> > + # should not have any such explicit statements when there is no
> > + # return value.
> > + assert rtype
> > +
> > + typ = self.format_type(rtype)
> > + assert typ
>
I wonder if I can write "assert typ := self.format_type(rtype)" here.
> > + assert section.text # We don't expect empty returns sections.
>
> Not sure the comment is worth its keep. Up to you.
>
Will remove. Not the first time I've talked to myself with assert messages.
Something from the very, very early days of this project and I had to
remind myself of some truths here and there (:
>
> > + self.add_field("returns", typ, section.text, section.info)
> > +
> > def visit_errors(self, section: QAPIDoc.Section) -> None:
> > # FIXME: the formatting for errors may be inconsistent and may
> > # or may not require different newline placement to ensure
>
>
[-- Attachment #2: Type: text/html, Size: 3583 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 49/57] docs/qapidoc: add visit_member() method
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (47 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 48/57] docs/qapidoc: add visit_returns() method John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-07 12:24 ` Markus Armbruster
2025-03-05 3:45 ` [PATCH 50/57] docs/qapidoc: add visit_sections() method John Snow
` (10 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
This method is used for generating the "members" of a wide variety of
things, including structs, unions, enums, alternates, etc. The field
name it uses to do so is dependent on the type of entity the "member"
belongs to.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapidoc.py | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index 6458790fe55..ed0269af27d 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -78,6 +78,16 @@
class Transmogrifier:
+ # Field names used for different entity types:
+ field_types = {
+ "enum": "value",
+ "struct": "memb",
+ "union": "memb",
+ "event": "memb",
+ "command": "arg",
+ "alternate": "choice",
+ }
+
def __init__(self) -> None:
self._curr_ent: Optional[QAPISchemaDefinition] = None
self._result = StringList()
@@ -88,6 +98,10 @@ def entity(self) -> QAPISchemaDefinition:
assert self._curr_ent is not None
return self._curr_ent
+ @property
+ def member_field_type(self) -> str:
+ return self.field_types[self.entity.meta]
+
# General-purpose rST generation functions
def get_indent(self) -> str:
@@ -202,6 +216,19 @@ def visit_paragraph(self, section: QAPIDoc.Section) -> None:
self.add_lines(section.text, section.info)
self.ensure_blank_line()
+ def visit_member(self, section: QAPIDoc.ArgSection) -> None:
+ # TODO: ifcond for members
+ # TODO?: features for members (documented at entity-level,
+ # but sometimes defined per-member. Should we add such
+ # information to member descriptions when we can?)
+ assert section.text and section.member
+ self.generate_field(
+ self.member_field_type,
+ section.member,
+ section.text,
+ section.info,
+ )
+
def visit_feature(self, section: QAPIDoc.ArgSection) -> None:
# FIXME - ifcond for features is not handled at all yet!
# Proposal: decorate the right-hand column with some graphical
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 49/57] docs/qapidoc: add visit_member() method
2025-03-05 3:45 ` [PATCH 49/57] docs/qapidoc: add visit_member() method John Snow
@ 2025-03-07 12:24 ` Markus Armbruster
2025-03-08 8:07 ` John Snow
0 siblings, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-07 12:24 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> This method is used for generating the "members" of a wide variety of
> things, including structs, unions, enums, alternates, etc. The field
> name it uses to do so is dependent on the type of entity the "member"
> belongs to.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> docs/sphinx/qapidoc.py | 27 +++++++++++++++++++++++++++
> 1 file changed, 27 insertions(+)
>
> diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> index 6458790fe55..ed0269af27d 100644
> --- a/docs/sphinx/qapidoc.py
> +++ b/docs/sphinx/qapidoc.py
> @@ -78,6 +78,16 @@
>
>
> class Transmogrifier:
> + # Field names used for different entity types:
> + field_types = {
> + "enum": "value",
> + "struct": "memb",
> + "union": "memb",
> + "event": "memb",
> + "command": "arg",
> + "alternate": "choice",
> + }
> +
> def __init__(self) -> None:
> self._curr_ent: Optional[QAPISchemaDefinition] = None
> self._result = StringList()
> @@ -88,6 +98,10 @@ def entity(self) -> QAPISchemaDefinition:
> assert self._curr_ent is not None
> return self._curr_ent
>
> + @property
> + def member_field_type(self) -> str:
> + return self.field_types[self.entity.meta]
> +
> # General-purpose rST generation functions
>
> def get_indent(self) -> str:
> @@ -202,6 +216,19 @@ def visit_paragraph(self, section: QAPIDoc.Section) -> None:
> self.add_lines(section.text, section.info)
> self.ensure_blank_line()
>
> + def visit_member(self, section: QAPIDoc.ArgSection) -> None:
> + # TODO: ifcond for members
Similar issues elsewhere are marked FIXME.
Worth mentioning in the commit message?
> + # TODO?: features for members (documented at entity-level,
> + # but sometimes defined per-member. Should we add such
> + # information to member descriptions when we can?)
I guess the '?' in 'TODO?' is there because you're not sure there's
anything to be done about member features. But you phrased the TODO as
a question. That makes the uncertainty obvious enough, doesn't it?
Suggest to delete the '?' so a grep for 'TODO:' isn't deceived. Best to
grep just for 'TODO', of course.
> + assert section.text and section.member
> + self.generate_field(
> + self.member_field_type,
> + section.member,
> + section.text,
> + section.info,
> + )
> +
> def visit_feature(self, section: QAPIDoc.ArgSection) -> None:
> # FIXME - ifcond for features is not handled at all yet!
> # Proposal: decorate the right-hand column with some graphical
^ permalink raw reply [flat|nested] 129+ messages in thread* Re: [PATCH 49/57] docs/qapidoc: add visit_member() method
2025-03-07 12:24 ` Markus Armbruster
@ 2025-03-08 8:07 ` John Snow
0 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-08 8:07 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 3076 bytes --]
On Fri, Mar 7, 2025 at 7:25 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > This method is used for generating the "members" of a wide variety of
> > things, including structs, unions, enums, alternates, etc. The field
> > name it uses to do so is dependent on the type of entity the "member"
> > belongs to.
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
> > ---
> > docs/sphinx/qapidoc.py | 27 +++++++++++++++++++++++++++
> > 1 file changed, 27 insertions(+)
> >
> > diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> > index 6458790fe55..ed0269af27d 100644
> > --- a/docs/sphinx/qapidoc.py
> > +++ b/docs/sphinx/qapidoc.py
> > @@ -78,6 +78,16 @@
> >
> >
> > class Transmogrifier:
> > + # Field names used for different entity types:
> > + field_types = {
> > + "enum": "value",
> > + "struct": "memb",
> > + "union": "memb",
> > + "event": "memb",
> > + "command": "arg",
> > + "alternate": "choice",
> > + }
> > +
> > def __init__(self) -> None:
> > self._curr_ent: Optional[QAPISchemaDefinition] = None
> > self._result = StringList()
> > @@ -88,6 +98,10 @@ def entity(self) -> QAPISchemaDefinition:
> > assert self._curr_ent is not None
> > return self._curr_ent
> >
> > + @property
> > + def member_field_type(self) -> str:
> > + return self.field_types[self.entity.meta]
> > +
> > # General-purpose rST generation functions
> >
> > def get_indent(self) -> str:
> > @@ -202,6 +216,19 @@ def visit_paragraph(self, section: QAPIDoc.Section)
> -> None:
> > self.add_lines(section.text, section.info)
> > self.ensure_blank_line()
> >
> > + def visit_member(self, section: QAPIDoc.ArgSection) -> None:
> > + # TODO: ifcond for members
>
> Similar issues elsewhere are marked FIXME.
>
> Worth mentioning in the commit message?
>
Will change, and yes.
>
>
> > + # TODO?: features for members (documented at entity-level,
> > + # but sometimes defined per-member. Should we add such
> > + # information to member descriptions when we can?)
>
> I guess the '?' in 'TODO?' is there because you're not sure there's
> anything to be done about member features. But you phrased the TODO as
> a question. That makes the uncertainty obvious enough, doesn't it?
> Suggest to delete the '?' so a grep for 'TODO:' isn't deceived. Best to
> grep just for 'TODO', of course.
>
Obie-kaybie.
>
> > + assert section.text and section.member
> > + self.generate_field(
> > + self.member_field_type,
> > + section.member,
> > + section.text,
> > + section.info,
> > + )
> > +
> > def visit_feature(self, section: QAPIDoc.ArgSection) -> None:
> > # FIXME - ifcond for features is not handled at all yet!
> > # Proposal: decorate the right-hand column with some graphical
>
>
[-- Attachment #2: Type: text/html, Size: 4704 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 50/57] docs/qapidoc: add visit_sections() method
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (48 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 49/57] docs/qapidoc: add visit_member() method John Snow
@ 2025-03-05 3:45 ` John Snow
2025-03-07 12:25 ` Markus Armbruster
2025-03-05 3:46 ` [PATCH 51/57] docs/qapidoc: add visit_entity() John Snow
` (9 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:45 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Implement the actual main dispatch method that processes and handles the
list of doc sections for a given QAPI entity.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapidoc.py | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index ed0269af27d..7308fa0a767 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -288,6 +288,31 @@ def preamble(self, ent: QAPISchemaDefinition) -> None:
self.ensure_blank_line()
+ def visit_sections(self, ent: QAPISchemaDefinition) -> None:
+ sections = ent.doc.all_sections if ent.doc else []
+
+ # Add sections *in the order they are documented*:
+ for section in sections:
+ if section.kind == QAPIDoc.Kind.PLAIN:
+ self.visit_paragraph(section)
+ elif section.kind == QAPIDoc.Kind.MEMBER:
+ assert isinstance(section, QAPIDoc.ArgSection)
+ self.visit_member(section)
+ elif section.kind == QAPIDoc.Kind.FEATURE:
+ assert isinstance(section, QAPIDoc.ArgSection)
+ self.visit_feature(section)
+ elif section.kind in (QAPIDoc.Kind.SINCE, QAPIDoc.Kind.TODO):
+ # Since is handled in preamble, TODO is skipped intentionally.
+ pass
+ elif section.kind == QAPIDoc.Kind.RETURNS:
+ self.visit_returns(section)
+ elif section.kind == QAPIDoc.Kind.ERRORS:
+ self.visit_errors(section)
+ else:
+ assert False
+
+ self.ensure_blank_line()
+
# Transmogrification core methods
def visit_module(self, path: str) -> None:
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 50/57] docs/qapidoc: add visit_sections() method
2025-03-05 3:45 ` [PATCH 50/57] docs/qapidoc: add visit_sections() method John Snow
@ 2025-03-07 12:25 ` Markus Armbruster
2025-03-08 8:10 ` John Snow
0 siblings, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-07 12:25 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> Implement the actual main dispatch method that processes and handles the
> list of doc sections for a given QAPI entity.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> docs/sphinx/qapidoc.py | 25 +++++++++++++++++++++++++
> 1 file changed, 25 insertions(+)
>
> diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> index ed0269af27d..7308fa0a767 100644
> --- a/docs/sphinx/qapidoc.py
> +++ b/docs/sphinx/qapidoc.py
> @@ -288,6 +288,31 @@ def preamble(self, ent: QAPISchemaDefinition) -> None:
>
> self.ensure_blank_line()
>
> + def visit_sections(self, ent: QAPISchemaDefinition) -> None:
> + sections = ent.doc.all_sections if ent.doc else []
> +
> + # Add sections *in the order they are documented*:
Is the order important, or just a matter of style?
> + for section in sections:
> + if section.kind == QAPIDoc.Kind.PLAIN:
> + self.visit_paragraph(section)
> + elif section.kind == QAPIDoc.Kind.MEMBER:
> + assert isinstance(section, QAPIDoc.ArgSection)
> + self.visit_member(section)
> + elif section.kind == QAPIDoc.Kind.FEATURE:
> + assert isinstance(section, QAPIDoc.ArgSection)
> + self.visit_feature(section)
> + elif section.kind in (QAPIDoc.Kind.SINCE, QAPIDoc.Kind.TODO):
> + # Since is handled in preamble, TODO is skipped intentionally.
> + pass
> + elif section.kind == QAPIDoc.Kind.RETURNS:
> + self.visit_returns(section)
> + elif section.kind == QAPIDoc.Kind.ERRORS:
> + self.visit_errors(section)
> + else:
> + assert False
> +
> + self.ensure_blank_line()
> +
> # Transmogrification core methods
>
> def visit_module(self, path: str) -> None:
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 50/57] docs/qapidoc: add visit_sections() method
2025-03-07 12:25 ` Markus Armbruster
@ 2025-03-08 8:10 ` John Snow
2025-03-08 9:20 ` Markus Armbruster
0 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-08 8:10 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 2604 bytes --]
On Fri, Mar 7, 2025 at 7:26 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > Implement the actual main dispatch method that processes and handles the
> > list of doc sections for a given QAPI entity.
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
> > ---
> > docs/sphinx/qapidoc.py | 25 +++++++++++++++++++++++++
> > 1 file changed, 25 insertions(+)
> >
> > diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> > index ed0269af27d..7308fa0a767 100644
> > --- a/docs/sphinx/qapidoc.py
> > +++ b/docs/sphinx/qapidoc.py
> > @@ -288,6 +288,31 @@ def preamble(self, ent: QAPISchemaDefinition) ->
> None:
> >
> > self.ensure_blank_line()
> >
> > + def visit_sections(self, ent: QAPISchemaDefinition) -> None:
> > + sections = ent.doc.all_sections if ent.doc else []
> > +
> > + # Add sections *in the order they are documented*:
>
> Is the order important, or just a matter of style?
>
I meant to emphasize the fact that the transmogrifier works "dumbly" on a
sequence of sections and nothing else; so the output is strictly in source
order.
The order does wind up mattering a *little*; if you randomized the section
order you'd not get good field list grouping, and/or the grouping
mechanisms would re-order the documentation so that it wasn't source order
anymore.
Not that this would happen with our parser, but, you asked.
>
> > + for section in sections:
> > + if section.kind == QAPIDoc.Kind.PLAIN:
> > + self.visit_paragraph(section)
> > + elif section.kind == QAPIDoc.Kind.MEMBER:
> > + assert isinstance(section, QAPIDoc.ArgSection)
> > + self.visit_member(section)
> > + elif section.kind == QAPIDoc.Kind.FEATURE:
> > + assert isinstance(section, QAPIDoc.ArgSection)
> > + self.visit_feature(section)
> > + elif section.kind in (QAPIDoc.Kind.SINCE,
> QAPIDoc.Kind.TODO):
> > + # Since is handled in preamble, TODO is skipped
> intentionally.
> > + pass
> > + elif section.kind == QAPIDoc.Kind.RETURNS:
> > + self.visit_returns(section)
> > + elif section.kind == QAPIDoc.Kind.ERRORS:
> > + self.visit_errors(section)
> > + else:
> > + assert False
> > +
> > + self.ensure_blank_line()
> > +
> > # Transmogrification core methods
> >
> > def visit_module(self, path: str) -> None:
>
>
[-- Attachment #2: Type: text/html, Size: 3662 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 50/57] docs/qapidoc: add visit_sections() method
2025-03-08 8:10 ` John Snow
@ 2025-03-08 9:20 ` Markus Armbruster
0 siblings, 0 replies; 129+ messages in thread
From: Markus Armbruster @ 2025-03-08 9:20 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> On Fri, Mar 7, 2025 at 7:26 AM Markus Armbruster <armbru@redhat.com> wrote:
>
>> John Snow <jsnow@redhat.com> writes:
>>
>> > Implement the actual main dispatch method that processes and handles the
>> > list of doc sections for a given QAPI entity.
>> >
>> > Signed-off-by: John Snow <jsnow@redhat.com>
>> > ---
>> > docs/sphinx/qapidoc.py | 25 +++++++++++++++++++++++++
>> > 1 file changed, 25 insertions(+)
>> >
>> > diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
>> > index ed0269af27d..7308fa0a767 100644
>> > --- a/docs/sphinx/qapidoc.py
>> > +++ b/docs/sphinx/qapidoc.py
>> > @@ -288,6 +288,31 @@ def preamble(self, ent: QAPISchemaDefinition) ->
>> None:
>> >
>> > self.ensure_blank_line()
>> >
>> > + def visit_sections(self, ent: QAPISchemaDefinition) -> None:
>> > + sections = ent.doc.all_sections if ent.doc else []
>> > +
>> > + # Add sections *in the order they are documented*:
>>
>> Is the order important, or just a matter of style?
>>
>
> I meant to emphasize the fact that the transmogrifier works "dumbly" on a
> sequence of sections and nothing else; so the output is strictly in source
> order.
Would
# Add sections in source order
be clearer?
> The order does wind up mattering a *little*; if you randomized the section
> order you'd not get good field list grouping, and/or the grouping
> mechanisms would re-order the documentation so that it wasn't source order
> anymore.
>
> Not that this would happen with our parser, but, you asked.
[...]
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 51/57] docs/qapidoc: add visit_entity()
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (49 preceding siblings ...)
2025-03-05 3:45 ` [PATCH 50/57] docs/qapidoc: add visit_sections() method John Snow
@ 2025-03-05 3:46 ` John Snow
2025-03-07 12:26 ` Markus Armbruster
2025-03-05 3:46 ` [PATCH 52/57] docs/qapidoc: implement transmogrify() method John Snow
` (8 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:46 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Finally, the core entry method for a qapi entity.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapidoc.py | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index 7308fa0a767..fb2ad7492ae 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -78,6 +78,8 @@
class Transmogrifier:
+ # pylint: disable=too-many-public-methods
+
# Field names used for different entity types:
field_types = {
"enum": "value",
@@ -368,6 +370,25 @@ def visit_freeform(self, doc: QAPIDoc) -> None:
self.add_lines(text, info)
self.ensure_blank_line()
+ def visit_entity(self, ent: QAPISchemaDefinition) -> None:
+ assert ent.info
+
+ try:
+ self._curr_ent = ent
+
+ # Squish structs and unions together into an "object" directive.
+ meta = ent.meta
+ if meta in ("struct", "union"):
+ meta = "object"
+
+ # This line gets credited to the start of the /definition/.
+ self.add_line(f".. qapi:{meta}:: {ent.name}", ent.info)
+ with self.indented():
+ self.preamble(ent)
+ self.visit_sections(ent)
+ finally:
+ self._curr_ent = None
+
class QAPISchemaGenDepVisitor(QAPISchemaVisitor):
"""A QAPI schema visitor which adds Sphinx dependencies each module
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 51/57] docs/qapidoc: add visit_entity()
2025-03-05 3:46 ` [PATCH 51/57] docs/qapidoc: add visit_entity() John Snow
@ 2025-03-07 12:26 ` Markus Armbruster
2025-03-08 8:12 ` John Snow
0 siblings, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-07 12:26 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé, Markus Armbruster
John Snow <jsnow@redhat.com> writes:
> Finally, the core entry method for a qapi entity.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> docs/sphinx/qapidoc.py | 21 +++++++++++++++++++++
> 1 file changed, 21 insertions(+)
>
> diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> index 7308fa0a767..fb2ad7492ae 100644
> --- a/docs/sphinx/qapidoc.py
> +++ b/docs/sphinx/qapidoc.py
> @@ -78,6 +78,8 @@
>
>
> class Transmogrifier:
> + # pylint: disable=too-many-public-methods
> +
Tsk, tsk, tsk ... ;-P
> # Field names used for different entity types:
> field_types = {
> "enum": "value",
> @@ -368,6 +370,25 @@ def visit_freeform(self, doc: QAPIDoc) -> None:
> self.add_lines(text, info)
> self.ensure_blank_line()
>
> + def visit_entity(self, ent: QAPISchemaDefinition) -> None:
> + assert ent.info
> +
> + try:
> + self._curr_ent = ent
> +
> + # Squish structs and unions together into an "object" directive.
> + meta = ent.meta
> + if meta in ("struct", "union"):
> + meta = "object"
> +
> + # This line gets credited to the start of the /definition/.
> + self.add_line(f".. qapi:{meta}:: {ent.name}", ent.info)
> + with self.indented():
> + self.preamble(ent)
> + self.visit_sections(ent)
> + finally:
> + self._curr_ent = None
> +
>
> class QAPISchemaGenDepVisitor(QAPISchemaVisitor):
> """A QAPI schema visitor which adds Sphinx dependencies each module
^ permalink raw reply [flat|nested] 129+ messages in thread* Re: [PATCH 51/57] docs/qapidoc: add visit_entity()
2025-03-07 12:26 ` Markus Armbruster
@ 2025-03-08 8:12 ` John Snow
0 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-08 8:12 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 2085 bytes --]
On Fri, Mar 7, 2025 at 7:27 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > Finally, the core entry method for a qapi entity.
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
> > ---
> > docs/sphinx/qapidoc.py | 21 +++++++++++++++++++++
> > 1 file changed, 21 insertions(+)
> >
> > diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> > index 7308fa0a767..fb2ad7492ae 100644
> > --- a/docs/sphinx/qapidoc.py
> > +++ b/docs/sphinx/qapidoc.py
> > @@ -78,6 +78,8 @@
> >
> >
> > class Transmogrifier:
> > + # pylint: disable=too-many-public-methods
> > +
>
> Tsk, tsk, tsk ... ;-P
>
Strongly tempted to make all the "visit_foo" methods "_handle_foo" and
delete this line.
... But I don't want to fight with the rebase conflicts on all of the chunk
context not lining up anymore. So I'll live with this little dishonor for
now.
>
> > # Field names used for different entity types:
> > field_types = {
> > "enum": "value",
> > @@ -368,6 +370,25 @@ def visit_freeform(self, doc: QAPIDoc) -> None:
> > self.add_lines(text, info)
> > self.ensure_blank_line()
> >
> > + def visit_entity(self, ent: QAPISchemaDefinition) -> None:
> > + assert ent.info
> > +
> > + try:
> > + self._curr_ent = ent
> > +
> > + # Squish structs and unions together into an "object"
> directive.
> > + meta = ent.meta
> > + if meta in ("struct", "union"):
> > + meta = "object"
> > +
> > + # This line gets credited to the start of the /definition/.
> > + self.add_line(f".. qapi:{meta}:: {ent.name}", ent.info)
> > + with self.indented():
> > + self.preamble(ent)
> > + self.visit_sections(ent)
> > + finally:
> > + self._curr_ent = None
> > +
> >
> > class QAPISchemaGenDepVisitor(QAPISchemaVisitor):
> > """A QAPI schema visitor which adds Sphinx dependencies each module
>
>
[-- Attachment #2: Type: text/html, Size: 3471 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 52/57] docs/qapidoc: implement transmogrify() method
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (50 preceding siblings ...)
2025-03-05 3:46 ` [PATCH 51/57] docs/qapidoc: add visit_entity() John Snow
@ 2025-03-05 3:46 ` John Snow
2025-03-05 3:46 ` [PATCH 53/57] docs: disambiguate cross-references John Snow
` (7 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:46 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
This is the true top-level processor for the new transmogrifier;
responsible both for generating the intermediate rST and then running
the nested parse on that generated document to produce the final
docutils tree that is then - very finally - postprocessed by sphinx for
final rendering to HTML &c.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapidoc.py | 49 +++++++++++++++++++++++++++++++++++++++++-
1 file changed, 48 insertions(+), 1 deletion(-)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index fb2ad7492ae..a4a0523d8ef 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -2,6 +2,7 @@
#
# QEMU qapidoc QAPI file parsing extension
#
+# Copyright (c) 2024 Red Hat
# Copyright (c) 2020 Linaro
#
# This work is licensed under the terms of the GNU GPLv2 or later.
@@ -24,6 +25,8 @@
https://www.sphinx-doc.org/en/master/development/index.html
"""
+# pylint: disable=too-many-lines
+
from __future__ import annotations
from contextlib import contextmanager
@@ -56,6 +59,7 @@
from sphinx import addnodes
from sphinx.directives.code import CodeBlock
from sphinx.errors import ExtensionError
+from sphinx.util import logging
from sphinx.util.docutils import switch_source_input
from sphinx.util.nodes import nested_parse_with_titles
@@ -76,6 +80,8 @@
__version__ = "1.0"
+logger = logging.getLogger(__name__)
+
class Transmogrifier:
# pylint: disable=too-many-public-methods
@@ -95,6 +101,10 @@ def __init__(self) -> None:
self._result = StringList()
self.indent = 0
+ @property
+ def result(self) -> StringList:
+ return self._result
+
@property
def entity(self) -> QAPISchemaDefinition:
assert self._curr_ent is not None
@@ -444,7 +454,43 @@ def new_serialno(self) -> str:
return "qapidoc-%d" % env.new_serialno("qapidoc")
def transmogrify(self, schema: QAPISchema) -> nodes.Element:
- raise NotImplementedError
+ logger.info("Transmogrifying QAPI to rST ...")
+ vis = Transmogrifier()
+ modules = set()
+
+ for doc in schema.docs:
+ module_source = doc.info.fname
+ if module_source not in modules:
+ vis.visit_module(module_source)
+ modules.add(module_source)
+
+ if doc.symbol:
+ ent = schema.lookup_entity(doc.symbol)
+ assert isinstance(ent, QAPISchemaDefinition)
+ vis.visit_entity(ent)
+ else:
+ vis.visit_freeform(doc)
+
+ logger.info("Transmogrification complete.")
+
+ contentnode = nodes.section()
+ content = vis.result
+ titles_allowed = True
+
+ logger.info("Transmogrifier running nested parse ...")
+ with switch_source_input(self.state, content):
+ if titles_allowed:
+ node: nodes.Element = nodes.section()
+ node.document = self.state.document
+ nested_parse_with_titles(self.state, content, contentnode)
+ else:
+ node = nodes.paragraph()
+ node.document = self.state.document
+ self.state.nested_parse(content, 0, contentnode)
+ logger.info("Transmogrifier's nested parse completed.")
+ sys.stdout.flush()
+
+ return contentnode
def legacy(self, schema: QAPISchema) -> nodes.Element:
vis = QAPISchemaGenRSTVisitor(self)
@@ -578,6 +624,7 @@ def run(self) -> List[nodes.Node]:
def setup(app: Sphinx) -> ExtensionMetadata:
"""Register qapi-doc directive with Sphinx"""
+ app.setup_extension("qapi_domain")
app.add_config_value("qapidoc_srctree", None, "env")
app.add_directive("qapi-doc", QAPIDocDirective)
app.add_directive("qmp-example", QMPExample)
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 53/57] docs: disambiguate cross-references
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (51 preceding siblings ...)
2025-03-05 3:46 ` [PATCH 52/57] docs/qapidoc: implement transmogrify() method John Snow
@ 2025-03-05 3:46 ` John Snow
2025-03-05 3:46 ` [PATCH 54/57] docs/qapidoc: add transmogrifier test document John Snow
` (6 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:46 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
The next patch will engage the qapidoc transmogrifier, which creates a
lot of cross-reference targets. Some of the existing targets
("migration", "qom", "replay") will become ambiguous as a result. Nail
them down more explicitly to prevent ambiguous cross-reference warnings.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/devel/codebase.rst | 6 +++---
docs/glossary.rst | 10 +++++-----
2 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/docs/devel/codebase.rst b/docs/devel/codebase.rst
index 4039875ee04..1b09953197b 100644
--- a/docs/devel/codebase.rst
+++ b/docs/devel/codebase.rst
@@ -23,7 +23,7 @@ Some of the main QEMU subsystems are:
- `Devices<device-emulation>` & Board models
- `Documentation <documentation-root>`
- `GDB support<GDB usage>`
-- `Migration<migration>`
+- :ref:`Migration<migration>`
- `Monitor<QEMU monitor>`
- :ref:`QOM (QEMU Object Model)<qom>`
- `System mode<System emulation>`
@@ -112,7 +112,7 @@ yet, so sometimes the source code is all you have.
* `libdecnumber <https://gitlab.com/qemu-project/qemu/-/tree/master/libdecnumber>`_:
Import of gcc library, used to implement decimal number arithmetic.
* `migration <https://gitlab.com/qemu-project/qemu/-/tree/master/migration>`__:
- `Migration framework <migration>`.
+ :ref:`Migration framework <migration>`.
* `monitor <https://gitlab.com/qemu-project/qemu/-/tree/master/monitor>`_:
`Monitor <QEMU monitor>` implementation (HMP & QMP).
* `nbd <https://gitlab.com/qemu-project/qemu/-/tree/master/nbd>`_:
@@ -193,7 +193,7 @@ yet, so sometimes the source code is all you have.
- `lcitool <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/lcitool>`_:
Generate dockerfiles for CI containers.
- `migration <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/migration>`_:
- Test scripts and data for `Migration framework <migration>`.
+ Test scripts and data for :ref:`Migration framework <migration>`.
- `multiboot <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/multiboot>`_:
Test multiboot functionality for x86_64/i386.
- `qapi-schema <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/qapi-schema>`_:
diff --git a/docs/glossary.rst b/docs/glossary.rst
index 693d9855dd1..4fa044bfb6e 100644
--- a/docs/glossary.rst
+++ b/docs/glossary.rst
@@ -120,7 +120,7 @@ Migration
---------
QEMU can save and restore the execution of a virtual machine between different
-host systems. This is provided by the `Migration framework<migration>`.
+host systems. This is provided by the :ref:`Migration framework<migration>`.
NBD
---
@@ -212,14 +212,14 @@ machine emulator and virtualizer.
QOM
---
-`QEMU Object Model <qom>` is an object oriented API used to define various
-devices and hardware in the QEMU codebase.
+:ref:`QEMU Object Model <qom>` is an object oriented API used to define
+various devices and hardware in the QEMU codebase.
Record/replay
-------------
-`Record/replay <replay>` is a feature of QEMU allowing to have a deterministic
-and reproducible execution of a virtual machine.
+:ref:`Record/replay <replay>` is a feature of QEMU allowing to have a
+deterministic and reproducible execution of a virtual machine.
Rust
----
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 54/57] docs/qapidoc: add transmogrifier test document
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (52 preceding siblings ...)
2025-03-05 3:46 ` [PATCH 53/57] docs: disambiguate cross-references John Snow
@ 2025-03-05 3:46 ` John Snow
2025-03-09 7:19 ` Markus Armbruster
2025-03-05 3:46 ` [PATCH 55/57] docs/qapidoc: process @foo into ``foo`` John Snow
` (5 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:46 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
This is just a test document that demonstrates the new qapi-domain doc
generator. Note that this test document uses a nesting depth of 2 for
the TOC unlike the existing QMP's reference nesting depth of 3. It's
arbitrary and can be changed to suit taste, it has nothing to do with
the new domain itself.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/index.rst | 1 +
docs/qapi/index.rst | 53 +++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 54 insertions(+)
create mode 100644 docs/qapi/index.rst
diff --git a/docs/index.rst b/docs/index.rst
index 5665de85cab..4364f9f1618 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -21,3 +21,4 @@ Welcome to QEMU's documentation!
specs/index
devel/index
glossary
+ qapi/index
diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
new file mode 100644
index 00000000000..e40dce09119
--- /dev/null
+++ b/docs/qapi/index.rst
@@ -0,0 +1,53 @@
+########################
+QAPI Transmogrifier Test
+########################
+
+This is a test render of the QEMU QMP reference manual using the new
+"transmogrifier" generator in qapidoc.py in conjunction with the
+qapi-domain.py sphinx extension.
+
+Some notable features:
+
+ * Every QAPI definition visible below is available to be
+ cross-referenced from anywhere else in the Sphinx docs; for example
+ ```blockdev-add``` will render to `blockdev-add`.
+
+ * There are type-specific cross-referencing roles available for
+ alternates, commands, events, enums, structs, unions and modules. for
+ example, ``:qapi:cmd:`block-dirty-bitmap-add``` resolves to
+ :qapi:cmd:`block-dirty-bitmap-add`, and only works for commands. The
+ roles available are ``cmd``, ``alt``, ``event``, ``enum``,
+ ``struct``, ``union``, and ``mod``; with two meta-roles available:
+ ``obj`` for absolutely any QAPI definition, and ``type`` for
+ everything except commands, events, and modules.
+
+ * There is a new `qapi-index` page which can be linked to with
+ ```qapi-index```. There, you can browse a list of all QAPI
+ definitions by type or alphabetically.
+
+ * QAPI definitions are also added to the existing `genindex` page.
+
+ * All member/argument/return types are now cross-references to that
+ type's definition. `chardev-add` is a good example.
+
+ * This work-in-progress version does not perform any inlining.
+
+ * This work-in-progress version actually also ignores branches entirely
+ right now!
+
+ * This version currently does not "prune" unnecessary docs.
+
+ * This version does not add undocumented members or return values.
+
+ * This version does not handle ifcond for anything other than top-level
+ entity definitions.
+
+ * This version renders sections in precisely the order they appear in
+ source, even if that winds up looking silly.
+
+
+.. contents::
+ :depth: 2
+
+.. qapi-doc:: qapi/qapi-schema.json
+ :transmogrify:
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 54/57] docs/qapidoc: add transmogrifier test document
2025-03-05 3:46 ` [PATCH 54/57] docs/qapidoc: add transmogrifier test document John Snow
@ 2025-03-09 7:19 ` Markus Armbruster
0 siblings, 0 replies; 129+ messages in thread
From: Markus Armbruster @ 2025-03-09 7:19 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> This is just a test document that demonstrates the new qapi-domain doc
> generator. Note that this test document uses a nesting depth of 2 for
> the TOC unlike the existing QMP's reference nesting depth of 3. It's
> arbitrary and can be changed to suit taste, it has nothing to do with
> the new domain itself.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> docs/index.rst | 1 +
> docs/qapi/index.rst | 53 +++++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 54 insertions(+)
> create mode 100644 docs/qapi/index.rst
>
> diff --git a/docs/index.rst b/docs/index.rst
> index 5665de85cab..4364f9f1618 100644
> --- a/docs/index.rst
> +++ b/docs/index.rst
> @@ -21,3 +21,4 @@ Welcome to QEMU's documentation!
> specs/index
> devel/index
> glossary
> + qapi/index
> diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
> new file mode 100644
> index 00000000000..e40dce09119
> --- /dev/null
> +++ b/docs/qapi/index.rst
> @@ -0,0 +1,53 @@
> +########################
> +QAPI Transmogrifier Test
> +########################
> +
> +This is a test render of the QEMU QMP reference manual using the new
> +"transmogrifier" generator in qapidoc.py in conjunction with the
> +qapi-domain.py sphinx extension.
> +
> +Some notable features:
> +
> + * Every QAPI definition visible below is available to be
> + cross-referenced from anywhere else in the Sphinx docs; for example
> + ```blockdev-add``` will render to `blockdev-add`.
> +
> + * There are type-specific cross-referencing roles available for
> + alternates, commands, events, enums, structs, unions and modules. for
> + example, ``:qapi:cmd:`block-dirty-bitmap-add``` resolves to
> + :qapi:cmd:`block-dirty-bitmap-add`, and only works for commands. The
> + roles available are ``cmd``, ``alt``, ``event``, ``enum``,
> + ``struct``, ``union``, and ``mod``; with two meta-roles available:
> + ``obj`` for absolutely any QAPI definition, and ``type`` for
> + everything except commands, events, and modules.
> +
> + * There is a new `qapi-index` page which can be linked to with
> + ```qapi-index```. There, you can browse a list of all QAPI
> + definitions by type or alphabetically.
```qapi-index``` becomes a link in HTML. The link takes me to an index
page. Two observations:
* The index page appears not to be linked from the navigation thingie on
the left. Searching for "QAPI Index" there doesn't find it, either.
* The index is structured into sections titled Alternates, Commands |
Enums, Events, Modules, Objects, A, ... Z. As I scrolled down
quickly, the transition from Objects to A confused me briefly: I
didn't immediately understand that A, ... Z contains the union of
everything above sorted into letter buckets.
> +
> + * QAPI definitions are also added to the existing `genindex` page.
> +
> + * All member/argument/return types are now cross-references to that
> + type's definition. `chardev-add` is a good example.
> +
> + * This work-in-progress version does not perform any inlining.
> +
> + * This work-in-progress version actually also ignores branches entirely
> + right now!
> +
> + * This version currently does not "prune" unnecessary docs.
> +
> + * This version does not add undocumented members or return values.
> +
> + * This version does not handle ifcond for anything other than top-level
> + entity definitions.
> +
> + * This version renders sections in precisely the order they appear in
> + source, even if that winds up looking silly.
> +
> +
> +.. contents::
> + :depth: 2
> +
> +.. qapi-doc:: qapi/qapi-schema.json
> + :transmogrify:
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 55/57] docs/qapidoc: process @foo into ``foo``
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (53 preceding siblings ...)
2025-03-05 3:46 ` [PATCH 54/57] docs/qapidoc: add transmogrifier test document John Snow
@ 2025-03-05 3:46 ` John Snow
2025-03-05 3:46 ` [PATCH 56/57] docs/qapidoc: add intermediate output debugger John Snow
` (4 subsequent siblings)
59 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-05 3:46 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Add support for the special QAPI doc syntax to process @references as
``preformatted text``. At the moment, there are no actual
cross-references for individual members, so there is nothing to link
against. For now, process it identically to how we did in the old
qapidoc system.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapidoc.py | 3 +++
1 file changed, 3 insertions(+)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index a4a0523d8ef..c84fff95697 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -305,6 +305,9 @@ def visit_sections(self, ent: QAPISchemaDefinition) -> None:
# Add sections *in the order they are documented*:
for section in sections:
+ # @var is translated to ``var``:
+ section.text = re.sub(r"@([\w-]+)", r"``\1``", section.text)
+
if section.kind == QAPIDoc.Kind.PLAIN:
self.visit_paragraph(section)
elif section.kind == QAPIDoc.Kind.MEMBER:
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* [PATCH 56/57] docs/qapidoc: add intermediate output debugger
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (54 preceding siblings ...)
2025-03-05 3:46 ` [PATCH 55/57] docs/qapidoc: process @foo into ``foo`` John Snow
@ 2025-03-05 3:46 ` John Snow
2025-03-07 12:34 ` Markus Armbruster
2025-03-05 3:46 ` [PATCH 57/57] docs/qapidoc: Add "the members of" pointers John Snow
` (3 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:46 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Add debugging output for the qapidoc transmogrifier - setting DEBUG=1
will produce .ir files (one for each qapidoc directive) that write the
generated rst file to disk to allow for easy debugging and verification
of the generated document.
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapidoc.py | 41 +++++++++++++++++++++++++++++++++++++----
1 file changed, 37 insertions(+), 4 deletions(-)
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index c84fff95697..511bab1592c 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -37,7 +37,7 @@
from typing import TYPE_CHECKING
from docutils import nodes
-from docutils.parsers.rst import Directive, directives
+from docutils.parsers.rst import directives
from docutils.statemachine import StringList
from qapi.error import QAPIError
from qapi.parser import QAPIDoc
@@ -60,7 +60,7 @@
from sphinx.directives.code import CodeBlock
from sphinx.errors import ExtensionError
from sphinx.util import logging
-from sphinx.util.docutils import switch_source_input
+from sphinx.util.docutils import SphinxDirective, switch_source_input
from sphinx.util.nodes import nested_parse_with_titles
@@ -422,7 +422,7 @@ def visit_module(self, name: str) -> None:
super().visit_module(name)
-class NestedDirective(Directive):
+class NestedDirective(SphinxDirective):
def run(self) -> Sequence[nodes.Node]:
raise NotImplementedError
@@ -491,10 +491,43 @@ def transmogrify(self, schema: QAPISchema) -> nodes.Element:
node.document = self.state.document
self.state.nested_parse(content, 0, contentnode)
logger.info("Transmogrifier's nested parse completed.")
+
+ if self.env.app.verbosity >= 2 or os.environ.get("DEBUG"):
+ argname = "_".join(Path(self.arguments[0]).parts)
+ name = Path(argname).stem + ".ir"
+ self.write_intermediate(content, name)
+
sys.stdout.flush()
-
return contentnode
+ def write_intermediate(self, content: StringList, filename: str) -> None:
+ logger.info(
+ "writing intermediate rST for '%s' to '%s'",
+ self.arguments[0],
+ filename,
+ )
+
+ srctree = Path(self.env.app.config.qapidoc_srctree).resolve()
+ outlines = []
+ lcol_width = 0
+
+ for i, line in enumerate(content):
+ src, lineno = content.info(i)
+ srcpath = Path(src).resolve()
+ srcpath = srcpath.relative_to(srctree)
+
+ lcol = f"{srcpath}:{lineno:04d}"
+ lcol_width = max(lcol_width, len(lcol))
+ outlines.append((lcol, line))
+
+ with open(filename, "w", encoding="UTF-8") as outfile:
+ for lcol, rcol in outlines:
+ outfile.write(lcol.rjust(lcol_width))
+ outfile.write(" |")
+ if rcol:
+ outfile.write(f" {rcol}")
+ outfile.write("\n")
+
def legacy(self, schema: QAPISchema) -> nodes.Element:
vis = QAPISchemaGenRSTVisitor(self)
vis.visit_begin(schema)
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 56/57] docs/qapidoc: add intermediate output debugger
2025-03-05 3:46 ` [PATCH 56/57] docs/qapidoc: add intermediate output debugger John Snow
@ 2025-03-07 12:34 ` Markus Armbruster
2025-03-08 8:13 ` John Snow
0 siblings, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-07 12:34 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> Add debugging output for the qapidoc transmogrifier - setting DEBUG=1
> will produce .ir files (one for each qapidoc directive) that write the
> generated rst file to disk to allow for easy debugging and verification
> of the generated document.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
I understand we generally need to examine these .ir files only when
things go wrong, or maybe to help understanding the transmogrifier. I
guess few people will care, and only rarely. But when we care, we
likely care a *lot*. Sure we want to dig the information on how to get
.ir files out of a commit message then?
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 56/57] docs/qapidoc: add intermediate output debugger
2025-03-07 12:34 ` Markus Armbruster
@ 2025-03-08 8:13 ` John Snow
2025-03-08 9:21 ` Markus Armbruster
0 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-08 8:13 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 894 bytes --]
On Fri, Mar 7, 2025 at 7:34 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > Add debugging output for the qapidoc transmogrifier - setting DEBUG=1
> > will produce .ir files (one for each qapidoc directive) that write the
> > generated rst file to disk to allow for easy debugging and verification
> > of the generated document.
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
>
> I understand we generally need to examine these .ir files only when
> things go wrong, or maybe to help understanding the transmogrifier. I
> guess few people will care, and only rarely. But when we care, we
> likely care a *lot*. Sure we want to dig the information on how to get
> .ir files out of a commit message then?
>
Intend to advertise it in the transmogrifier doc and/or extend the
doc-writing section of the qapi code gen doc.
[-- Attachment #2: Type: text/html, Size: 1418 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 56/57] docs/qapidoc: add intermediate output debugger
2025-03-08 8:13 ` John Snow
@ 2025-03-08 9:21 ` Markus Armbruster
0 siblings, 0 replies; 129+ messages in thread
From: Markus Armbruster @ 2025-03-08 9:21 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> On Fri, Mar 7, 2025 at 7:34 AM Markus Armbruster <armbru@redhat.com> wrote:
>
>> John Snow <jsnow@redhat.com> writes:
>>
>> > Add debugging output for the qapidoc transmogrifier - setting DEBUG=1
>> > will produce .ir files (one for each qapidoc directive) that write the
>> > generated rst file to disk to allow for easy debugging and verification
>> > of the generated document.
>> >
>> > Signed-off-by: John Snow <jsnow@redhat.com>
>>
>> I understand we generally need to examine these .ir files only when
>> things go wrong, or maybe to help understanding the transmogrifier. I
>> guess few people will care, and only rarely. But when we care, we
>> likely care a *lot*. Sure we want to dig the information on how to get
>> .ir files out of a commit message then?
>>
>
> Intend to advertise it in the transmogrifier doc and/or extend the
> doc-writing section of the qapi code gen doc.
Okay!
^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH 57/57] docs/qapidoc: Add "the members of" pointers
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (55 preceding siblings ...)
2025-03-05 3:46 ` [PATCH 56/57] docs/qapidoc: add intermediate output debugger John Snow
@ 2025-03-05 3:46 ` John Snow
2025-03-07 12:37 ` Markus Armbruster
2025-03-05 11:31 ` [PATCH 00/57] docs: Add new QAPI transmogrifier Markus Armbruster
` (2 subsequent siblings)
59 siblings, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 3:46 UTC (permalink / raw)
To: qemu-devel
Cc: Michael Roth, Alex Bennée, Philippe Mathieu-Daudé,
Peter Maydell, Thomas Huth, Daniel P. Berrangé,
Markus Armbruster, John Snow
Add "the members of ..." pointers to Members and Arguments lists where
appropriate, with clickable cross-references - so it's a slight
improvement over the old system :)
This patch is meant to be a temporary solution until we can review and
merge the inliner.
The implementation of this patch is a little bit of a hack: Sphinx is
not designed to allow you to mix fields of different "type"; i.e. mixing
member descriptions and free-form text under the same heading. To
accomplish this with a minimum of hackery, we technically document a
"dummy field" and then just strip off the documentation for that dummy
field in a post-processing step. We use the "q_dummy" variable for this
purpose, then strip it back out before final processing. If this
processing step should fail, you'll see warnings for a bad
cross-reference. (So if you don't see any, it must be working!)
Signed-off-by: John Snow <jsnow@redhat.com>
---
docs/sphinx/qapi_domain.py | 22 +++++++++++++--
docs/sphinx/qapidoc.py | 58 +++++++++++++++++++++++++++++++++++++-
2 files changed, 77 insertions(+), 3 deletions(-)
diff --git a/docs/sphinx/qapi_domain.py b/docs/sphinx/qapi_domain.py
index 077ab7ae47a..a3c89a85f44 100644
--- a/docs/sphinx/qapi_domain.py
+++ b/docs/sphinx/qapi_domain.py
@@ -165,6 +165,24 @@ def since_validator(param: str) -> str:
return param
+class SpecialTypedField(CompatTypedField):
+ def make_field(self, *args: Any, **kwargs: Any) -> nodes.field:
+ ret = super().make_field(*args, **kwargs)
+
+ # Look for the characteristic " -- " text node that Sphinx
+ # inserts for each TypedField entry ...
+ for node in ret.traverse(lambda n: str(n) == " -- "):
+ par = node.parent
+ if par.children[0].astext() != "q_dummy":
+ continue
+
+ # If the first node's text is q_dummy, this is a dummy
+ # field we want to strip down to just its contents.
+ del par.children[:-1]
+
+ return ret
+
+
class QAPIObject(ParserFix):
"""
Description of a generic QAPI object.
@@ -451,7 +469,7 @@ class QAPICommand(QAPIObject):
doc_field_types.extend(
[
# :arg TypeName ArgName: descr
- CompatTypedField(
+ SpecialTypedField(
"argument",
label=_("Arguments"),
names=("arg",),
@@ -519,7 +537,7 @@ class QAPIObjectWithMembers(QAPIObject):
doc_field_types.extend(
[
# :member type name: descr
- CompatTypedField(
+ SpecialTypedField(
"member",
label=_("Members"),
names=("memb",),
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index 511bab1592c..daddb709628 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -47,8 +47,10 @@
QAPISchemaCommand,
QAPISchemaDefinition,
QAPISchemaEnumMember,
+ QAPISchemaEvent,
QAPISchemaFeature,
QAPISchemaMember,
+ QAPISchemaObjectType,
QAPISchemaObjectTypeMember,
QAPISchemaType,
QAPISchemaVisitor,
@@ -300,11 +302,61 @@ def preamble(self, ent: QAPISchemaDefinition) -> None:
self.ensure_blank_line()
+ def _insert_member_pointer(self, ent: QAPISchemaDefinition) -> None:
+
+ def _get_target(
+ ent: QAPISchemaDefinition,
+ ) -> Optional[QAPISchemaDefinition]:
+ if isinstance(ent, (QAPISchemaCommand, QAPISchemaEvent)):
+ return ent.arg_type
+ if isinstance(ent, QAPISchemaObjectType):
+ return ent.base
+ return None
+
+ target = _get_target(ent)
+ if target is not None and not target.is_implicit():
+ assert ent.info
+ self.add_field(
+ self.member_field_type,
+ "q_dummy",
+ f"The members of :qapi:type:`{target.name}`.",
+ ent.info,
+ "q_dummy",
+ )
+
+ if isinstance(ent, QAPISchemaObjectType) and ent.branches is not None:
+ for variant in ent.branches.variants:
+ if variant.type.name == "q_empty":
+ continue
+ assert ent.info
+ self.add_field(
+ self.member_field_type,
+ "q_dummy",
+ f" When ``{ent.branches.tag_member.name}`` is "
+ f"``{variant.name}``; "
+ f"The members of :qapi:type:`{variant.type.name}`.",
+ ent.info,
+ "q_dummy",
+ )
+
def visit_sections(self, ent: QAPISchemaDefinition) -> None:
sections = ent.doc.all_sections if ent.doc else []
+ # Determine the index location at which we should generate
+ # documentation for "The members of ..." pointers. This should
+ # go at the end of the members section(s) if any. Note that
+ # index 0 is assumed to be a plain intro section, even if it is
+ # empty; and that a members section if present will always
+ # immediately follow the opening PLAIN section.
+ gen_index = 1
+ if len(sections) > 1:
+ while sections[gen_index].kind == QAPIDoc.Kind.MEMBER:
+ gen_index += 1
+ if gen_index >= len(sections):
+ break
+
# Add sections *in the order they are documented*:
- for section in sections:
+ for i, section in enumerate(sections):
# @var is translated to ``var``:
section.text = re.sub(r"@([\w-]+)", r"``\1``", section.text)
@@ -326,6 +378,10 @@ def visit_sections(self, ent: QAPISchemaDefinition) -> None:
else:
assert False
+ # Generate "The members of ..." entries if necessary:
+ if i == gen_index - 1:
+ self._insert_member_pointer(ent)
+
self.ensure_blank_line()
# Transmogrification core methods
--
2.48.1
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH 57/57] docs/qapidoc: Add "the members of" pointers
2025-03-05 3:46 ` [PATCH 57/57] docs/qapidoc: Add "the members of" pointers John Snow
@ 2025-03-07 12:37 ` Markus Armbruster
2025-03-08 8:18 ` John Snow
0 siblings, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-07 12:37 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> Add "the members of ..." pointers to Members and Arguments lists where
> appropriate, with clickable cross-references - so it's a slight
> improvement over the old system :)
>
> This patch is meant to be a temporary solution until we can review and
> merge the inliner.
>
> The implementation of this patch is a little bit of a hack: Sphinx is
> not designed to allow you to mix fields of different "type"; i.e. mixing
> member descriptions and free-form text under the same heading. To
> accomplish this with a minimum of hackery, we technically document a
> "dummy field" and then just strip off the documentation for that dummy
> field in a post-processing step. We use the "q_dummy" variable for this
> purpose, then strip it back out before final processing. If this
> processing step should fail, you'll see warnings for a bad
> cross-reference. (So if you don't see any, it must be working!)
>
> Signed-off-by: John Snow <jsnow@redhat.com>
Yes, it's a hack, and possibly fragile, but it'll all go away when the
inliner lands. I understand the inliner already exists, but you're
holding it back to not balloon the series even more.
I'm on board.
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 57/57] docs/qapidoc: Add "the members of" pointers
2025-03-07 12:37 ` Markus Armbruster
@ 2025-03-08 8:18 ` John Snow
0 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-08 8:18 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 2142 bytes --]
On Fri, Mar 7, 2025 at 7:37 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > Add "the members of ..." pointers to Members and Arguments lists where
> > appropriate, with clickable cross-references - so it's a slight
> > improvement over the old system :)
> >
> > This patch is meant to be a temporary solution until we can review and
> > merge the inliner.
> >
> > The implementation of this patch is a little bit of a hack: Sphinx is
> > not designed to allow you to mix fields of different "type"; i.e. mixing
> > member descriptions and free-form text under the same heading. To
> > accomplish this with a minimum of hackery, we technically document a
> > "dummy field" and then just strip off the documentation for that dummy
> > field in a post-processing step. We use the "q_dummy" variable for this
> > purpose, then strip it back out before final processing. If this
> > processing step should fail, you'll see warnings for a bad
> > cross-reference. (So if you don't see any, it must be working!)
> >
> > Signed-off-by: John Snow <jsnow@redhat.com>
>
> Yes, it's a hack, and possibly fragile, but it'll all go away when the
> inliner lands. I understand the inliner already exists, but you're
> holding it back to not balloon the series even more.
>
Just couldn't find any other way to do it with less SLOC. It's a weird
limitation in the bowels of Sphinx, and the Field classes aren't factored
aggressively enough to just write my own new Field subclass without also
having to do a lot of re-implementation that touches a lot of APIs I'd
rather not muck around with: I literally think that approach, while
"cleaner" in the traditional sense, is likely *more* porcelain than what
I've done here.
Thought it was better to just let Sphinx do whatever it does in make_xref,
and then make some tactical edits on the tail.
Could write a John Steinbeck novella's analysis on all the other approaches
I tried and why I ruled them out, but. Eh. Just read a John Steinbeck
novella instead if you want to.
> I'm on board.
>
>
Thank goodness.
[-- Attachment #2: Type: text/html, Size: 3169 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 00/57] docs: Add new QAPI transmogrifier
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (56 preceding siblings ...)
2025-03-05 3:46 ` [PATCH 57/57] docs/qapidoc: Add "the members of" pointers John Snow
@ 2025-03-05 11:31 ` Markus Armbruster
2025-03-05 15:40 ` John Snow
2025-03-06 10:58 ` Markus Armbruster
2025-03-06 12:35 ` Markus Armbruster
2025-03-07 18:19 ` Markus Armbruster
59 siblings, 2 replies; 129+ messages in thread
From: Markus Armbruster @ 2025-03-05 11:31 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> Hi! This series is based on armbru/pull-qapi-2025-02-26.
>
> This series is a "minimum viable" version of the new QAPI documentation
> system. It does the bare minimum under the new framework, saving nice
> features for later.
Not saved for later: a massive improvement of the generated
documentation's looks and usability. Moreover, I hope the new generator
will be easier to maintain than the old one, because its inner workings
are closer to how Sphinx expects such things to work. Fair?
>
> Patches 3-29 implement the qapi_domain extension.
> Patches 30-57 implement the qapidoc "Transmogrifier".
>
> This series is still "RFC" quality, though it's quite nearly actually
> ready for inclusion. The "add transmogrifier test document" patch is not
> intended for actual merge, it's just there to demonstrate the new
> document generator by producing output in docs/manual/qapi/index.html.
>
> Known shortcomings in this series:
>
> - Still no new QAPI unit tests. I'll add those for next go-around.
Not a blocker as far as I'm concerned, because I feel you're unlikely to
run away from this :)
> - No new documentation. Also for next revision. I'll document the QAPI
> domain syntax and give a brief overview of how the transmogrifier
> functions, and a quick rundown of any new rST syntax that may be
> pertinent to QAPI documentation writers.
Likewise.
> - IFCOND information is still rendered in two places, we'll need to
> decide where and how we want to render it.
I'll have a look, and then we'll talk.
> - No QAPI namespace support ... yet. So we can't enable it for QMP, QGA
> and QSD simultaneously just yet. I don't think it will be difficult.
>
> Unknown shortcomings in this series:
>
> - ???
I'll try to find some, but I'm not overly optimistic ;)
> New stuff overall from the last iteration of this series:
>
> - @foo is processed into ``foo`
> - "The members of ..." messages have been temporarily re-added until we
> can smooth over the inliner.
> - This series runs under Sphinx 3.4.3 to Sphinx 8.2.0 inclusive. It
> truly is a Christmas miracle. (please clap)
*clap* clap* clap*
This is waaaay harder than it has any right to be.
> - This series now fully type checks and lint checks under Sphinx 8.2.0,
> but may not type check under earlier Sphinx versions. Achieving this
> alone, nevermind in conjunction with the above, was a literal
> herculean labor.
scripts/qapi/ remains clean for me. docs/sphinx/ improves from no type
checking to type checking with a version newer than the one I have on my
development box right now, which I count as an improvement.
> I really must stress again how frustratingly difficult it was to achieve
> the prior two bullet points. I *do* in fact want a cookie and/or an
> award ribbon.
We owe you an entire layer cake, with a marzipan figurine of your
conquering self on top. Seriously, I could not have done this.
^ permalink raw reply [flat|nested] 129+ messages in thread* Re: [PATCH 00/57] docs: Add new QAPI transmogrifier
2025-03-05 11:31 ` [PATCH 00/57] docs: Add new QAPI transmogrifier Markus Armbruster
@ 2025-03-05 15:40 ` John Snow
2025-03-06 7:19 ` Markus Armbruster
2025-03-06 10:58 ` Markus Armbruster
1 sibling, 1 reply; 129+ messages in thread
From: John Snow @ 2025-03-05 15:40 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 3862 bytes --]
On Wed, Mar 5, 2025, 6:31 AM Markus Armbruster <armbru@redhat.com> wrote:
> John Snow <jsnow@redhat.com> writes:
>
> > Hi! This series is based on armbru/pull-qapi-2025-02-26.
> >
> > This series is a "minimum viable" version of the new QAPI documentation
> > system. It does the bare minimum under the new framework, saving nice
> > features for later.
>
> Not saved for later: a massive improvement of the generated
> documentation's looks and usability. Moreover, I hope the new generator
> will be easier to maintain than the old one, because its inner workings
> are closer to how Sphinx expects such things to work. Fair?
>
I hope so too, though some of the sphinx version compatibility hacks are
possibly fairly porcelain. The most egregious of them are for versions
prior to Sphinx 4.1.
Almost all of the compatibility hacks are factored into compat.py and they
are all documented with what versions they're there for. Most of the
ugliest is pre-4.1.
> >
> > Patches 3-29 implement the qapi_domain extension.
> > Patches 30-57 implement the qapidoc "Transmogrifier".
> >
> > This series is still "RFC" quality, though it's quite nearly actually
> > ready for inclusion. The "add transmogrifier test document" patch is not
> > intended for actual merge, it's just there to demonstrate the new
> > document generator by producing output in docs/manual/qapi/index.html.
> >
> > Known shortcomings in this series:
> >
> > - Still no new QAPI unit tests. I'll add those for next go-around.
>
> Not a blocker as far as I'm concerned, because I feel you're unlikely to
> run away from this :)
>
😅
> > - No new documentation. Also for next revision. I'll document the QAPI
> > domain syntax and give a brief overview of how the transmogrifier
> > functions, and a quick rundown of any new rST syntax that may be
> > pertinent to QAPI documentation writers.
>
> Likewise.
>
> > - IFCOND information is still rendered in two places, we'll need to
> > decide where and how we want to render it.
>
> I'll have a look, and then we'll talk.
>
> > - No QAPI namespace support ... yet. So we can't enable it for QMP, QGA
> > and QSD simultaneously just yet. I don't think it will be difficult.
> >
> > Unknown shortcomings in this series:
> >
> > - ???
>
> I'll try to find some, but I'm not overly optimistic ;)
>
Missing documentation for undocumented members. You found one! :)
> > New stuff overall from the last iteration of this series:
> >
> > - @foo is processed into ``foo`
> > - "The members of ..." messages have been temporarily re-added until we
> > can smooth over the inliner.
> > - This series runs under Sphinx 3.4.3 to Sphinx 8.2.0 inclusive. It
> > truly is a Christmas miracle. (please clap)
>
> *clap* clap* clap*
>
> This is waaaay harder than it has any right to be.
>
> > - This series now fully type checks and lint checks under Sphinx 8.2.0,
> > but may not type check under earlier Sphinx versions. Achieving this
> > alone, nevermind in conjunction with the above, was a literal
> > herculean labor.
>
> scripts/qapi/ remains clean for me. docs/sphinx/ improves from no type
> checking to type checking with a version newer than the one I have on my
> development box right now, which I count as an improvement.
>
> > I really must stress again how frustratingly difficult it was to achieve
> > the prior two bullet points. I *do* in fact want a cookie and/or an
> > award ribbon.
>
> We owe you an entire layer cake, with a marzipan figurine of your
> conquering self on top. Seriously, I could not have done this.
>
It's the stuff in compat.py that was the absolute hardest. You need that
stuff for this to work on the version ranges it works for, but typing it
was an extra special torture.
>
[-- Attachment #2: Type: text/html, Size: 5705 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 00/57] docs: Add new QAPI transmogrifier
2025-03-05 15:40 ` John Snow
@ 2025-03-06 7:19 ` Markus Armbruster
0 siblings, 0 replies; 129+ messages in thread
From: Markus Armbruster @ 2025-03-06 7:19 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
John Snow <jsnow@redhat.com> writes:
> On Wed, Mar 5, 2025, 6:31 AM Markus Armbruster <armbru@redhat.com> wrote:
>
>> John Snow <jsnow@redhat.com> writes:
>>
>> > Hi! This series is based on armbru/pull-qapi-2025-02-26.
>> >
>> > This series is a "minimum viable" version of the new QAPI documentation
>> > system. It does the bare minimum under the new framework, saving nice
>> > features for later.
>>
>> Not saved for later: a massive improvement of the generated
>> documentation's looks and usability. Moreover, I hope the new generator
>> will be easier to maintain than the old one, because its inner workings
>> are closer to how Sphinx expects such things to work. Fair?
>>
>
> I hope so too, though some of the sphinx version compatibility hacks are
> possibly fairly porcelain. The most egregious of them are for versions
> prior to Sphinx 4.1.
>
> Almost all of the compatibility hacks are factored into compat.py and they
> are all documented with what versions they're there for. Most of the
> ugliest is pre-4.1.
[...]
> It's the stuff in compat.py that was the absolute hardest. You need that
> stuff for this to work on the version ranges it works for, but typing it
> was an extra special torture.
Supporting such a wide range of versions is *expensive*.
Writing compat.py is sunk cost.
Maintaining it is not. It's fragile hacks, hard to understand even with
deep Sphinx innards expertise, impossible to understand without. I
figure if we somehow lost your expertise, we'd likely have to drop
support for old Sphinx the moment compat.py breaks.
Just because we can solve certain hard problems doesn't mean we should.
I question the wisdom of supporting such a wide range of Sphinx
versions.
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 00/57] docs: Add new QAPI transmogrifier
2025-03-05 11:31 ` [PATCH 00/57] docs: Add new QAPI transmogrifier Markus Armbruster
2025-03-05 15:40 ` John Snow
@ 2025-03-06 10:58 ` Markus Armbruster
1 sibling, 0 replies; 129+ messages in thread
From: Markus Armbruster @ 2025-03-06 10:58 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
Markus Armbruster <armbru@redhat.com> writes:
> John Snow <jsnow@redhat.com> writes:
>
>> Hi! This series is based on armbru/pull-qapi-2025-02-26.
>>
>> This series is a "minimum viable" version of the new QAPI documentation
>> system. It does the bare minimum under the new framework, saving nice
>> features for later.
>
> Not saved for later: a massive improvement of the generated
> documentation's looks and usability. Moreover, I hope the new generator
> will be easier to maintain than the old one, because its inner workings
> are closer to how Sphinx expects such things to work. Fair?
>
>>
>> Patches 3-29 implement the qapi_domain extension.
>> Patches 30-57 implement the qapidoc "Transmogrifier".
>>
>> This series is still "RFC" quality, though it's quite nearly actually
>> ready for inclusion. The "add transmogrifier test document" patch is not
>> intended for actual merge, it's just there to demonstrate the new
>> document generator by producing output in docs/manual/qapi/index.html.
>>
>> Known shortcomings in this series:
>>
>> - Still no new QAPI unit tests. I'll add those for next go-around.
>
> Not a blocker as far as I'm concerned, because I feel you're unlikely to
> run away from this :)
>
>> - No new documentation. Also for next revision. I'll document the QAPI
>> domain syntax and give a brief overview of how the transmogrifier
>> functions, and a quick rundown of any new rST syntax that may be
>> pertinent to QAPI documentation writers.
>
> Likewise.
>
>> - IFCOND information is still rendered in two places, we'll need to
>> decide where and how we want to render it.
>
> I'll have a look, and then we'll talk.
Two shortcomings, actually:
- IFCOND in definitions (enum, struct, union, alternate, command,
event) are rendered in two places.
Example: query-tpm has 'if': 'CONFIG_TPM'. The rendered
documentation looks like this:
Command query-tpm ["#if" "CONFIG_TPM"] (Since: 1.5)
*Availability*: "CONFIG_TPM"
Return information about the TPM device
Example::
[...]
With the old generator, it looks like
"query-tpm" (Command)
---------------------
Return information about the TPM device
Since
~~~~~
1.5
Example::
[...]
If
~~
"CONFIG_TPM"
So, three ways to present the information, none of them immediately
obvious to a casual reader. The easiest to guess right is perhaps
the "Availability" box.
The two new ways are in more conspicious spots than the old one is.
Not sure I like that; I believe ifconds are fairly uninteresting to
users of QMP most of the time. More on that below.
The "Availability" box is even more conspicious than the [#if ...]
bracket. It also uses more space.
For complex conditions, the [#if ...] bracket can make the first
line less readable. Example:
Command query-cpu-definitions ["#if" "TARGET_PPC or TARGET_ARM or TARGET_I386 or TARGET_S390X or TARGET_MIPS or TARGET_LOONGARCH64 or TARGET_RISCV"] (Since: 1.2)
*Availability*: "TARGET_PPC or TARGET_ARM or TARGET_I386 or
TARGET_S390X or TARGET_MIPS or TARGET_LOONGARCH64 or TARGET_RISCV"
Return a list of supported virtual CPU definitions
The first line matters. This tips the balance for me. Let's go
with the "Availability" box at least for now.
There's a symbol before "Availability" in the generated HTML. Its
meaning is less than obvious. I remember you explained it to me,
but I forgot. We can polish this later.
- IFCOND elsewhere is not rendered at all. This is a regression. The
old generator shows it like this:
"mptcp": "boolean" (optional) (**If: **"HAVE_IPPROTO_MPTCP")
enable multi-path TCP. (Since 6.1)
I believe the regression is tolerable for now, because the
information lost is only modestly useful for users of QMP, and the
massive improvements the new generator provides outweigh this small
loss.
Why only modestly useful? It tells the reader the thing's
availability depends on how QEMU was built. It also provides a clue
that should let a developer or sufficiently clever non-developer
rebuild QEMU to provide the thing. This would involve some code
spelunking.
>> - No QAPI namespace support ... yet. So we can't enable it for QMP, QGA
>> and QSD simultaneously just yet. I don't think it will be difficult.
[...]
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 00/57] docs: Add new QAPI transmogrifier
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (57 preceding siblings ...)
2025-03-05 11:31 ` [PATCH 00/57] docs: Add new QAPI transmogrifier Markus Armbruster
@ 2025-03-06 12:35 ` Markus Armbruster
2025-03-06 13:27 ` John Snow
2025-03-07 18:19 ` Markus Armbruster
59 siblings, 1 reply; 129+ messages in thread
From: Markus Armbruster @ 2025-03-06 12:35 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
When I eyeballed the previous iteration, I made a few observations[1].
Let's see what has changed.
} New table of contents shows one level, old two. No objection; the
} navigation thingie on the left is more useful anyway.
No change. Okay.
} The new generator elides unreferenced types. Generally good, but two
} observations:
}
} * QapiErrorClass is unreferenced, but its members are mentioned in
} Errors sections. QapiErrorClass serves as better than nothing error
} code documentation, but it's gone in the new doc. So this is a minor
} regression. We can figure out what to do about it later.
}
} * Section "QMP errors" is empty in the new doc, because its entire
} contents is elided. I guess we should elide the section as well, but
} it's fine to leave that for later.
Unreferenced types are back. Okay; we can elide them later.
} Old doc shows a definition's since information like any other section.
} New doc has it in the heading box. Looks prettier and uses much less
} space. Not sure the heading box is the best place, but it'll do for
} now, we can always move it around later.
No change.
} The new doc's headings use "Struct" or "Union" where the old one uses
} just "Object". Let's keep "Object", please.
Fixed.
} In the new doc, some member references are no longer rendered as such,
} e.g. @on-source-error and @on-target-error in BackupCommon's note.
} Another small regression.
Fixed.
} Union branches are busted in the new generator's output. I know they
} used to work, so I'm not worried about it.
Fixed: "When TAG-MEMBER is VALUE; The members of BRANCH-TYPE."
The semicolon feels odd. I'd use a colon there. Or put the when at the
end like the old generator does: "The members of BRANCH-TYPE when
TAG-MEMBER is VALUE".
Side effects, I think:
* Members of explicit base types are no longer inlined. Instead: "The
members of BASE-TYPE."
* Members of explicit command / event argument types are no longer
inlined. Instead: "The members of ARG-TYPE."
Both fine for the initial version.
} The new doc shows the return type, the old doc doesn't. Showing it is
} definitely an improvement, but we need to adjust the doc text to avoid
} silliness like "Returns: SnapshotInfo – SnapshotInfo".
No change. Can polish on top.
} The new doc shows Arguments / Members, Returns, and Errors in two-column
} format. Looks nice. But for some reason, the two columns don't align
} horizontally for Errors like they do for the others. Certainly not a
} blocker of anything, but we should try to fix it at some point.
No change. Fine to leave for later.
} The new doc doesn't show non-definition conditionals, as mentioned in
} the cover letter. It shows definition conditionals twice. Once should
} suffice.
No change. You asked for advice, and I gave some[2].
} There's probably more, but this is it for now.
Again, this is it for now.
[1] Message-ID: <87wmds4tpk.fsf@pond.sub.org>
https://lore.kernel.org/qemu-devel/87wmds4tpk.fsf@pond.sub.org/
[2] Message-ID: <87zfhya0is.fsf@pond.sub.org>
https://lore.kernel.org/qemu-devel/87zfhya0is.fsf@pond.sub.org/
^ permalink raw reply [flat|nested] 129+ messages in thread* Re: [PATCH 00/57] docs: Add new QAPI transmogrifier
2025-03-06 12:35 ` Markus Armbruster
@ 2025-03-06 13:27 ` John Snow
0 siblings, 0 replies; 129+ messages in thread
From: John Snow @ 2025-03-06 13:27 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
[-- Attachment #1: Type: text/plain, Size: 4205 bytes --]
On Thu, Mar 6, 2025, 7:35 AM Markus Armbruster <armbru@redhat.com> wrote:
> When I eyeballed the previous iteration, I made a few observations[1].
> Let's see what has changed.
>
> } New table of contents shows one level, old two. No objection; the
> } navigation thingie on the left is more useful anyway.
>
> No change. Okay.
>
This is configured in the .rst file, not in code. I just happened to use a
different level in my test document.
For initial merge, I'll probably enable the new generator on QMP only while
I work on namespace support.
> } The new generator elides unreferenced types. Generally good, but two
> } observations:
> }
> } * QapiErrorClass is unreferenced, but its members are mentioned in
> } Errors sections. QapiErrorClass serves as better than nothing error
> } code documentation, but it's gone in the new doc. So this is a minor
> } regression. We can figure out what to do about it later.
> }
> } * Section "QMP errors" is empty in the new doc, because its entire
> } contents is elided. I guess we should elide the section as well, but
> } it's fine to leave that for later.
>
> Unreferenced types are back. Okay; we can elide them later.
>
Yep, it will come along with the inliner later.
> } Old doc shows a definition's since information like any other section.
> } New doc has it in the heading box. Looks prettier and uses much less
> } space. Not sure the heading box is the best place, but it'll do for
> } now, we can always move it around later.
>
> No change.
>
Always happy to take HTML/CSS patches from someone with a strong opinion.
For now, though, ...
> } The new doc's headings use "Struct" or "Union" where the old one uses
> } just "Object". Let's keep "Object", please.
>
> Fixed.
>
> } In the new doc, some member references are no longer rendered as such,
> } e.g. @on-source-error and @on-target-error in BackupCommon's note.
> } Another small regression.
>
> Fixed.
>
> } Union branches are busted in the new generator's output. I know they
> } used to work, so I'm not worried about it.
>
> Fixed: "When TAG-MEMBER is VALUE; The members of BRANCH-TYPE."
>
> The semicolon feels odd. I'd use a colon there. Or put the when at the
> end like the old generator does: "The members of BRANCH-TYPE when
> TAG-MEMBER is VALUE".
>
Easy to change!
> Side effects, I think:
>
> * Members of explicit base types are no longer inlined. Instead: "The
> members of BASE-TYPE."
>
> * Members of explicit command / event argument types are no longer
> inlined. Instead: "The members of ARG-TYPE."
>
> Both fine for the initial version.
>
Yeah, inliner is all or nothing.
> } The new doc shows the return type, the old doc doesn't. Showing it is
> } definitely an improvement, but we need to adjust the doc text to avoid
> } silliness like "Returns: SnapshotInfo – SnapshotInfo".
>
> No change. Can polish on top.
>
I have a patch for this in my kvm forum branch, along with "convert
everything into actual cross-references". Will send afterwards.
> } The new doc shows Arguments / Members, Returns, and Errors in two-column
> } format. Looks nice. But for some reason, the two columns don't align
> } horizontally for Errors like they do for the others. Certainly not a
> } blocker of anything, but we should try to fix it at some point.
>
> No change. Fine to leave for later.
>
Yes, but I did figure out why and I do have a solution planned now. It's
something I had to work around for "The members of..." pointers.
> } The new doc doesn't show non-definition conditionals, as mentioned in
> } the cover letter. It shows definition conditionals twice. Once should
> } suffice.
>
> No change. You asked for advice, and I gave some[2].
>
D'oh!
> } There's probably more, but this is it for now.
>
> Again, this is it for now.
>
>
>
> [1] Message-ID: <87wmds4tpk.fsf@pond.sub.org>
> https://lore.kernel.org/qemu-devel/87wmds4tpk.fsf@pond.sub.org/
>
> [2] Message-ID: <87zfhya0is.fsf@pond.sub.org>
> https://lore.kernel.org/qemu-devel/87zfhya0is.fsf@pond.sub.org/
>
>
[-- Attachment #2: Type: text/html, Size: 7446 bytes --]
^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH 00/57] docs: Add new QAPI transmogrifier
2025-03-05 3:45 [PATCH 00/57] docs: Add new QAPI transmogrifier John Snow
` (58 preceding siblings ...)
2025-03-06 12:35 ` Markus Armbruster
@ 2025-03-07 18:19 ` Markus Armbruster
59 siblings, 0 replies; 129+ messages in thread
From: Markus Armbruster @ 2025-03-07 18:19 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, Michael Roth, Alex Bennée,
Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth,
Daniel P. Berrangé
This looks really, really close. I like how you split the work into
patches, and your commit messages have been quite useful. Thanks!
^ permalink raw reply [flat|nested] 129+ messages in thread