[RFC 0/2] python: integrate linter tests natively with meson

[RFC 0/2] python: integrate linter tests natively with meson

[Daniel P. Berrangé]

It seems everytime I post some new patches which touch python code I
end up forgetting to run the extra non-default 'make check' target from
the python/ subdir.

IMHO, if we're going to pick up on linter failures in review feedback
(which is a reasonable thing to want to do), then we should be running
these linters by default in 'make check'.

This RFC is demonstrating a possible strategy to achieve that, by making
the python tests use meson in the same way as everything else, instead
of being stuck in the old 'make' world. In keeping with our work to
remove use of avocado for functional testing, this also removes avocado
for python linters since IMHO it adds no notable value over what meson
provides already.

See last patch for the interesting stuff, and the various caveats about
this initial PoC.

Being a PoC this is minimally tested beyond 'make check' on my local
machine.

Daniel P. Berrangé (2):
  python: be more selective in hiding mypy subclassing warning
  python: integrate linter tests natively with meson

 meson.build                   |  1 +
 python/meson.build            | 43 +++++++++++++++++++++++++++++++++++
 python/qemu/qmp/qmp_tui.py    | 12 +++++-----
 python/qemu/utils/qom_fuse.py |  2 +-
 python/scripts/__init__.py    |  0
 python/setup.cfg              |  8 -------
 scripts/meson.build           |  9 ++++++++
 tests/meson.build             |  1 +
 tests/python/meson.build      | 17 ++++++++++++++
 9 files changed, 78 insertions(+), 15 deletions(-)
 create mode 100644 python/meson.build
 create mode 100644 python/scripts/__init__.py
 create mode 100644 tests/python/meson.build

-- 
2.47.1

[RFC 1/2] python: be more selective in hiding mypy subclassing warning

[Daniel P. Berrangé]

Use an annotation inline to the file instead of in setup.cfg

This has the added advantage that the mypy checks now pass when
using 'mypy /path/to/python/qemu' as well as 'mypy -p qemu'.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 python/qemu/qmp/qmp_tui.py    | 12 ++++++------
 python/qemu/utils/qom_fuse.py |  2 +-
 python/setup.cfg              |  8 --------
 3 files changed, 7 insertions(+), 15 deletions(-)

diff --git a/python/qemu/qmp/qmp_tui.py b/python/qemu/qmp/qmp_tui.py
index 2d9ebbd20b..8720d70e60 100644
--- a/python/qemu/qmp/qmp_tui.py
+++ b/python/qemu/qmp/qmp_tui.py
@@ -401,7 +401,7 @@ def run(self, debug: bool = False) -> None:
             raise err
 
 
-class StatusBar(urwid.Text):
+class StatusBar(urwid.Text):  # type: ignore
     """
     A simple statusbar modelled using the Text widget. The status can be
     set using the set_text function. All text set is aligned to right.
@@ -412,7 +412,7 @@ def __init__(self, text: str = ''):
         super().__init__(text, align='right')
 
 
-class Editor(urwid_readline.ReadlineEdit):
+class Editor(urwid_readline.ReadlineEdit):  # type: ignore
     """
     A simple editor modelled using the urwid_readline.ReadlineEdit widget.
     Mimcs GNU readline shortcuts and provides history support.
@@ -485,7 +485,7 @@ def keypress(self, size: Tuple[int, int], key: str) -> Optional[str]:
         return None
 
 
-class EditorWidget(urwid.Filler):
+class EditorWidget(urwid.Filler):  # type: ignore
     """
     Wrapper around the editor widget.
 
@@ -498,7 +498,7 @@ def __init__(self, parent: App) -> None:
         super().__init__(Editor(parent), valign='top')
 
 
-class HistoryBox(urwid.ListBox):
+class HistoryBox(urwid.ListBox):  # type: ignore
     """
     This widget is modelled using the ListBox widget, contains the list of
     all messages both QMP messages and log messages to be shown in the TUI.
@@ -535,7 +535,7 @@ def mouse_event(self, size: Tuple[int, int], _event: str, button: float,
             super().keypress(size, 'down')
 
 
-class HistoryWindow(urwid.Frame):
+class HistoryWindow(urwid.Frame):  # type: ignore
     """
     This window composes the HistoryBox and EditorWidget in a horizontal split.
     By default the first focus is given to the history box.
@@ -572,7 +572,7 @@ def cb_add_to_history(self, msg: str, level: Optional[str] = None) -> None:
         self.history.add_to_history(formatted)
 
 
-class Window(urwid.Frame):
+class Window(urwid.Frame):  # type: ignore
     """
     This window is the top most widget of the TUI and will contain other
     windows. Each child of this widget is responsible for displaying a specific
diff --git a/python/qemu/utils/qom_fuse.py b/python/qemu/utils/qom_fuse.py
index cf7e344bd5..0a1e02af87 100644
--- a/python/qemu/utils/qom_fuse.py
+++ b/python/qemu/utils/qom_fuse.py
@@ -56,7 +56,7 @@
 fuse.fuse_python_api = (0, 2)
 
 
-class QOMFuse(QOMCommand, Operations):
+class QOMFuse(QOMCommand, Operations):  # type: ignore
     """
     QOMFuse implements both fuse.Operations and QOMCommand.
 
diff --git a/python/setup.cfg b/python/setup.cfg
index cf5af7e664..8dcd4c946e 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -83,14 +83,6 @@ warn_unused_configs = True
 namespace_packages = True
 warn_unused_ignores = False
 
-[mypy-qemu.utils.qom_fuse]
-# fusepy has no type stubs:
-allow_subclassing_any = True
-
-[mypy-qemu.qmp.qmp_tui]
-# urwid and urwid_readline have no type stubs:
-allow_subclassing_any = True
-
 # The following missing import directives are because these libraries do not
 # provide type stubs. Allow them on an as-needed basis for mypy.
 [mypy-fuse]
-- 
2.47.1

[RFC 2/2] python: integrate linter tests natively with meson

[Daniel P. Berrangé]

This creates meson test targets for the python linters which has a
number of benefits:

 * They can be invoked in the same manner as all other tests
   using meson or ninja, or the top level Makefile wrappers
 * They avoid the extra indirection of running via avocado
 * They get run by default when doing 'make check'

There is one general 'python' suite defined, and then a 'pythonVER'
suite for each tox environment listed in python/setup.cfg.

All python linters across all version get triggered by 'make check'.

To run just the python linters without other tests

  $ make check-python

To run just the python linters for Python 3.12

  $ make check-python312

Static lints can be enabled per source directory in meson.build by
adding

 pythontests += {
    'dir': meson.current_source_dir(),
    'pylint': true,
    'mypy': true,
    'flake8': true,
    'isort': true,
 }

If any given lint is omitted in the config, it defaults to disabled.

When the '--disable-downloads' option is given to 'configure', running
of the python tests is disabled since tox would otherwise attempt to
download files to populate the per-version envs.

TBD: if you have meson < 1.7, then 'build_by_default: false' does
not work correctly, so the tox envs get created by 'make', rather
than delayed until 'make check'. We could bump min meson to fix this
or accept that older versions have sub-optimal behaviour

TBD: setup.cfg defines 6 python venvs 3.8 -> 3.13 currently, which
gives alot of duplication in what we run with 'make check' by default.
This adds about 1 minute of wallclock time to a non-parallel build on
my (high end) laptop, or 15 seconds to a parallel build. This is a
combination of populating the tox envs, and pylint being generally
slow. IIRC there's a way to split tests in 'quick' vs 'thorough' so
'make check' runs 'quick' only by default, but I've not investigated
this yet.

TBD: purge python/Makefile and python/tests/*.sh wrappers. It is
desirable to have only one way of running tests that everyone agrees
on, so that we have consistency of results. Assuming the native meson
integration is good enough, this should make the Makefile obsolete,
as well as the .sh wrappers.

TBD: I added isort & mypy for the 'scripts/qapi' code as a demo, but
flake8 & pylint don't pass so are disabled.

TBD: I've not figured out a nice way to integrate qemu-iotests without
the linters.py indirection which is painful to consume. Perhaps solvable
by just giving all the iotests real file extensions (.sh & .py) so the
linters can auto-detect python files. Would then need to have an exclude
list in qemu-iotests/meson.build to skip non-compliant files initially.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 meson.build                |  1 +
 python/meson.build         | 43 ++++++++++++++++++++++++++++++++++++++
 python/scripts/__init__.py |  0
 scripts/meson.build        |  9 ++++++++
 tests/meson.build          |  1 +
 tests/python/meson.build   | 17 +++++++++++++++
 6 files changed, 71 insertions(+)
 create mode 100644 python/meson.build
 create mode 100644 python/scripts/__init__.py
 create mode 100644 tests/python/meson.build

diff --git a/meson.build b/meson.build
index 0ee79c664d..f1cbfda149 100644
--- a/meson.build
+++ b/meson.build
@@ -4440,6 +4440,7 @@ if stap.found()
   endforeach
 endif
 
+subdir('python')
 subdir('scripts')
 subdir('tools')
 subdir('pc-bios')
diff --git a/python/meson.build b/python/meson.build
new file mode 100644
index 0000000000..73360b6d43
--- /dev/null
+++ b/python/meson.build
@@ -0,0 +1,43 @@
+
+tox = find_program('tox', required: false)
+toxcfg = ['-c', meson.current_source_dir(),
+          '--workdir', meson.current_source_dir(),
+          '--root', meson.current_source_dir()]
+pythonlints = ['pylint', 'mypy', 'flake8', 'isort']
+
+pythontests = [
+    {
+        'dir': meson.current_source_dir() / 'qemu',
+        'pylint': true,
+        'mypy': true,
+        'flake8': true,
+        'isort': true,
+    },
+    {
+        'dir': meson.current_source_dir() / 'scripts',
+        'pylint': true,
+        'mypy': true,
+        'flake8': true,
+        'isort': true,
+    },
+]
+
+pyenvs = []
+if tox.found() and get_option('wrap_mode') != 'nodownload'
+    rc = run_command([tox, 'list'] + toxcfg,
+                     capture: true,
+                     check: true)
+
+    foreach envline: rc.stdout().strip().split('\n')
+        if envline.startswith('py')
+            pyenv = envline.split(' ')[0]
+            message('Adding python environment: ' + pyenv)
+
+            mkenv = custom_target('tox-env-' + pyenv,
+                                  command: [tox, 'run', '-e', pyenv, '--notest'] + toxcfg,
+                                  output: pyenv,
+                                  build_by_default: false)
+            pyenvs += { 'name': pyenv, 'version': pyenv.substring(2), 'target': mkenv }
+        endif
+    endforeach
+endif
diff --git a/python/scripts/__init__.py b/python/scripts/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/scripts/meson.build b/scripts/meson.build
index 532277f5a2..6a1fd9e5a9 100644
--- a/scripts/meson.build
+++ b/scripts/meson.build
@@ -3,3 +3,12 @@ if stap.found()
 endif
 
 test('xml-preprocess', files('xml-preprocess-test.py'), suite: ['unit'])
+
+
+pythontests += {
+    'dir': meson.current_source_dir() / 'qapi',
+    'pylint': false,
+    'mypy': true,
+    'flake8': false,
+    'isort': true,
+}
diff --git a/tests/meson.build b/tests/meson.build
index c59619220f..89eed7d259 100644
--- a/tests/meson.build
+++ b/tests/meson.build
@@ -88,3 +88,4 @@ subdir('qapi-schema')
 subdir('qtest')
 subdir('migration-stress')
 subdir('functional')
+subdir('python')
diff --git a/tests/python/meson.build b/tests/python/meson.build
new file mode 100644
index 0000000000..28be424d85
--- /dev/null
+++ b/tests/python/meson.build
@@ -0,0 +1,17 @@
+
+foreach pyenv: pyenvs
+    foreach scenario: pythontests
+        suites = ['python', 'python' + pyenv['version']]
+
+        foreach lint: pythonlints
+            if scenario.get(lint, false)
+                name = lint + scenario['dir'].replace(meson.project_source_root(), '').replace('/', '-')
+                toxargs = toxcfg + ['exec', '-e', pyenv['name'],
+                                    '--',
+                                    'python3', '-m', lint,
+                                    scenario['dir']]
+                test(name, tox, args: toxargs, suite: suites, depends: pyenv['target'])
+            endif
+        endforeach
+    endforeach
+endforeach
-- 
2.47.1

Re: [RFC 0/2] python: integrate linter tests natively with meson

[Markus Armbruster]

John, how does this mix with the linting parts of your "[PATCH 00/10]
qapi: misc testing and doc patches"?

Re: [RFC 0/2] python: integrate linter tests natively with meson

[Daniel P. Berrangé]

On Tue, Feb 25, 2025 at 07:04:11AM +0100, Markus Armbruster wrote:
> John, how does this mix with the linting parts of your "[PATCH 00/10]
> qapi: misc testing and doc patches"?

Since this is only an RFC, I would expect that John's patch series can
simply ignore this & merge when ready. If we decide to turn this into
a non-RFC series, than we can figure out the resolution at that point.
At a quick glance, I don't see anything John's done in his series that
would complicate life for this proposal in any notable way.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [RFC 0/2] python: integrate linter tests natively with meson

[Markus Armbruster]

Daniel P. Berrangé <berrange@redhat.com> writes:

> On Tue, Feb 25, 2025 at 07:04:11AM +0100, Markus Armbruster wrote:
>> John, how does this mix with the linting parts of your "[PATCH 00/10]
>> qapi: misc testing and doc patches"?
>
> Since this is only an RFC, I would expect that John's patch series can
> simply ignore this & merge when ready. If we decide to turn this into
> a non-RFC series, than we can figure out the resolution at that point.
> At a quick glance, I don't see anything John's done in his series that
> would complicate life for this proposal in any notable way.

Okay.

Different question for John: do you think this RFC moves us in a forward
direction?

Re: [RFC 0/2] python: integrate linter tests natively with meson

[Daniel P. Berrangé]

Ping: would like to hear John's thoughts on this series.

We're likely to eliminate usage of avocado for functional testing
in this dev cycle. That leaves python testing as the main outstanding
bit of QEMU that is not natively using meson's test integration and
also still pulling in avocado as a dependency.

On Mon, Feb 24, 2025 at 07:11:50PM +0000, Daniel P. Berrangé wrote:
> It seems everytime I post some new patches which touch python code I
> end up forgetting to run the extra non-default 'make check' target from
> the python/ subdir.
> 
> IMHO, if we're going to pick up on linter failures in review feedback
> (which is a reasonable thing to want to do), then we should be running
> these linters by default in 'make check'.
> 
> This RFC is demonstrating a possible strategy to achieve that, by making
> the python tests use meson in the same way as everything else, instead
> of being stuck in the old 'make' world. In keeping with our work to
> remove use of avocado for functional testing, this also removes avocado
> for python linters since IMHO it adds no notable value over what meson
> provides already.
> 
> See last patch for the interesting stuff, and the various caveats about
> this initial PoC.
> 
> Being a PoC this is minimally tested beyond 'make check' on my local
> machine.
> 
> Daniel P. Berrangé (2):
>   python: be more selective in hiding mypy subclassing warning
>   python: integrate linter tests natively with meson
> 
>  meson.build                   |  1 +
>  python/meson.build            | 43 +++++++++++++++++++++++++++++++++++
>  python/qemu/qmp/qmp_tui.py    | 12 +++++-----
>  python/qemu/utils/qom_fuse.py |  2 +-
>  python/scripts/__init__.py    |  0
>  python/setup.cfg              |  8 -------
>  scripts/meson.build           |  9 ++++++++
>  tests/meson.build             |  1 +
>  tests/python/meson.build      | 17 ++++++++++++++
>  9 files changed, 78 insertions(+), 15 deletions(-)
>  create mode 100644 python/meson.build
>  create mode 100644 python/scripts/__init__.py
>  create mode 100644 tests/python/meson.build
> 
> -- 
> 2.47.1
> 

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [RFC 1/2] python: be more selective in hiding mypy subclassing warning

[Thomas Huth]

On 24/02/2025 20.11, Daniel P. Berrangé wrote:
> Use an annotation inline to the file instead of in setup.cfg
> 
> This has the added advantage that the mypy checks now pass when
> using 'mypy /path/to/python/qemu' as well as 'mypy -p qemu'.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>   python/qemu/qmp/qmp_tui.py    | 12 ++++++------
>   python/qemu/utils/qom_fuse.py |  2 +-
>   python/setup.cfg              |  8 --------
>   3 files changed, 7 insertions(+), 15 deletions(-)

Reviewed-by: Thomas Huth <thuth@redhat.com>