[PATCH 0/3] python: fix for mypy 0.930 and pylint dependencies

[PATCH 0/3] python: fix for mypy 0.930 and pylint dependencies

[John Snow]

GitLab: https://gitlab.com/jsnow/qemu/-/commits/python-ci-fix-2022-jan
CI: https://gitlab.com/jsnow/qemu/-/pipelines/445011098

Dependencies in the Python ecosystem were updated and caused CI failures
immediately after leaving for PTO. This series fixes those breakages,
thank you for your patience.

This patchset fixes the 'check-python-tox' job that was previously
configured as 'allowed to fail'. I'm not re-enabling it to always run at
present, maybe leaving it as an optional job is fine for now. The
purpose of the job is primarily to find new breakages on cutting edge
deployments of Python, which means that it's fairly likely to chirp as
regressions are discovered.

Generally, I like seeing those regressions so I can keep on top of
fixing them -- however, in practice, it seems as if what this really
means is that every last time I go on PTO, they break for several weeks
in an amazing coincidence.

Oh well.

When submitting Python changes in the future, please just check that
this CI test ran to ensure that it works well on all supported Python
interpreter versions.

John Snow (3):
  python/aqmp: use absolute import statement
  Python/aqmp: fix type definitions for mypy 0.920
  python: update type hints for mypy 0.930

 python/qemu/aqmp/aqmp_tui.py  | 3 ++-
 python/qemu/aqmp/protocol.py  | 5 +++--
 python/qemu/qmp/qom_common.py | 6 +-----
 3 files changed, 6 insertions(+), 8 deletions(-)

-- 
2.31.1

[PATCH 1/3] python/aqmp: use absolute import statement

[John Snow]

pylint's dependency astroid appears to have bugs in 2.9.1 and 2.9.2 (Dec
31 and Jan 3) that appear to erroneously expect the qemu namespace to
have an __init__.py file. astroid 2.9.3 (Jan 9) avoids that problem, but
appears to not understand a relative import within a namespace package.

Update the relative import - it was worth changing anyway, because these
packages will eventually be packaged and distributed separately.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 python/qemu/aqmp/aqmp_tui.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/python/qemu/aqmp/aqmp_tui.py b/python/qemu/aqmp/aqmp_tui.py
index a2929f771c..f1e926dd75 100644
--- a/python/qemu/aqmp/aqmp_tui.py
+++ b/python/qemu/aqmp/aqmp_tui.py
@@ -35,7 +35,8 @@
 import urwid
 import urwid_readline
 
-from ..qmp import QEMUMonitorProtocol, QMPBadPortError
+from qemu.qmp import QEMUMonitorProtocol, QMPBadPortError
+
 from .error import ProtocolError
 from .message import DeserializationError, Message, UnexpectedTypeError
 from .protocol import ConnectError, Runstate
-- 
2.31.1

Re: [PATCH 1/3] python/aqmp: use absolute import statement

[Beraldo Leal]

On Mon, Jan 10, 2022 at 02:13:47PM -0500, John Snow wrote:
> pylint's dependency astroid appears to have bugs in 2.9.1 and 2.9.2 (Dec
> 31 and Jan 3) that appear to erroneously expect the qemu namespace to
> have an __init__.py file. astroid 2.9.3 (Jan 9) avoids that problem, but
> appears to not understand a relative import within a namespace package.
> 
> Update the relative import - it was worth changing anyway, because these
> packages will eventually be packaged and distributed separately.
> 
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
>  python/qemu/aqmp/aqmp_tui.py | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/python/qemu/aqmp/aqmp_tui.py b/python/qemu/aqmp/aqmp_tui.py
> index a2929f771c..f1e926dd75 100644
> --- a/python/qemu/aqmp/aqmp_tui.py
> +++ b/python/qemu/aqmp/aqmp_tui.py
> @@ -35,7 +35,8 @@
>  import urwid
>  import urwid_readline
>  
> -from ..qmp import QEMUMonitorProtocol, QMPBadPortError
> +from qemu.qmp import QEMUMonitorProtocol, QMPBadPortError
> +
>  from .error import ProtocolError
>  from .message import DeserializationError, Message, UnexpectedTypeError
>  from .protocol import ConnectError, Runstate
> -- 
> 2.31.1
> 
>

Reviewed-by: Beraldo Leal <bleal@redhat.com>

--
Beraldo

[PATCH 2/3] Python/aqmp: fix type definitions for mypy 0.920

[John Snow]

0.920 (Released 2021-12-15) is not entirely happy with the
way that I was defining _FutureT:

qemu/aqmp/protocol.py:601: error: Item "object" of the upper bound
"Optional[Future[Any]]" of type variable "_FutureT" has no attribute
"done"

Update it with something a little mechanically simpler that works better
across a wider array of mypy versions.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 python/qemu/aqmp/protocol.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py
index 5190b33b13..c4fbe35a0e 100644
--- a/python/qemu/aqmp/protocol.py
+++ b/python/qemu/aqmp/protocol.py
@@ -43,8 +43,8 @@
 
 
 T = TypeVar('T')
+_U = TypeVar('_U')
 _TaskFN = Callable[[], Awaitable[None]]  # aka ``async def func() -> None``
-_FutureT = TypeVar('_FutureT', bound=Optional['asyncio.Future[Any]'])
 
 
 class Runstate(Enum):
@@ -591,7 +591,8 @@ def _cleanup(self) -> None:
         """
         Fully reset this object to a clean state and return to `IDLE`.
         """
-        def _paranoid_task_erase(task: _FutureT) -> Optional[_FutureT]:
+        def _paranoid_task_erase(task: Optional['asyncio.Future[_U]']
+                                 ) -> Optional['asyncio.Future[_U]']:
             # Help to erase a task, ENSURING it is fully quiesced first.
             assert (task is None) or task.done()
             return None if (task and task.done()) else task
-- 
2.31.1

[PATCH 3/3] python: update type hints for mypy 0.930

[John Snow]

Mypy 0.930, released Dec 22, changes the way argparse objects are
considered. Crafting a definition that works under Python 3.6 and an
older mypy alongside newer versions simultaneously is ... difficult,
so... eh. Stub it out with an 'Any' definition to get the CI moving
again.

Oh well.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 python/qemu/qmp/qom_common.py | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/python/qemu/qmp/qom_common.py b/python/qemu/qmp/qom_common.py
index a59ae1a2a1..2e4c741f77 100644
--- a/python/qemu/qmp/qom_common.py
+++ b/python/qemu/qmp/qom_common.py
@@ -30,10 +30,6 @@
 from . import QEMUMonitorProtocol, QMPError
 
 
-# The following is needed only for a type alias.
-Subparsers = argparse._SubParsersAction  # pylint: disable=protected-access
-
-
 class ObjectPropertyInfo:
     """
     Represents the return type from e.g. qom-list.
@@ -89,7 +85,7 @@ def __init__(self, args: argparse.Namespace):
         self.qmp.connect()
 
     @classmethod
-    def register(cls, subparsers: Subparsers) -> None:
+    def register(cls, subparsers: Any) -> None:
         """
         Register this command with the argument parser.
 
-- 
2.31.1

Re: [PATCH 3/3] python: update type hints for mypy 0.930

[Beraldo Leal]

On Mon, Jan 10, 2022 at 02:13:49PM -0500, John Snow wrote:
> Mypy 0.930, released Dec 22, changes the way argparse objects are
> considered. Crafting a definition that works under Python 3.6 and an
> older mypy alongside newer versions simultaneously is ... difficult,
> so... eh. Stub it out with an 'Any' definition to get the CI moving
> again.
> 
> Oh well.
> 
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
>  python/qemu/qmp/qom_common.py | 6 +-----
>  1 file changed, 1 insertion(+), 5 deletions(-)
> 
> diff --git a/python/qemu/qmp/qom_common.py b/python/qemu/qmp/qom_common.py
> index a59ae1a2a1..2e4c741f77 100644
> --- a/python/qemu/qmp/qom_common.py
> +++ b/python/qemu/qmp/qom_common.py
> @@ -30,10 +30,6 @@
>  from . import QEMUMonitorProtocol, QMPError
>  
>  
> -# The following is needed only for a type alias.
> -Subparsers = argparse._SubParsersAction  # pylint: disable=protected-access
> -
> -
>  class ObjectPropertyInfo:
>      """
>      Represents the return type from e.g. qom-list.
> @@ -89,7 +85,7 @@ def __init__(self, args: argparse.Namespace):
>          self.qmp.connect()
>  
>      @classmethod
> -    def register(cls, subparsers: Subparsers) -> None:
> +    def register(cls, subparsers: Any) -> None:
>          """
>          Register this command with the argument parser.
>  
> -- 
> 2.31.1
> 
>

Reviewed-by: Beraldo Leal <bleal@redhat.com>

--
Beraldo