[PATCH v2 0/2] python: a few improvements to qmp-shell

[PATCH v2 0/2] python: a few improvements to qmp-shell

[Daniel P. Berrangé]

This makes the qmp-shell program a little more pleasant to use when you
are just trying to spawn a throw-away QEMU process to query some info
from.

First it introduces a 'qmp-shell-wrap' command that takes a QEMU command
line instead of QMP socket, and spawns QEMU automatically, so its life
is tied to that of the shell.

Second it adds ability to log QMP commands/responses to a file that can
be queried with 'jq' to extract information. This is good for commands
which return huge JSON docs.

In v2:

 - Unlink unix socket path on exit
 - Fix default command name
 - Deal with flake8/pylint warnings

Daniel P. Berrangé (2):
  python: introduce qmp-shell-wrap convenience tool
  python: support recording QMP session to a file

 python/qemu/qmp/qmp_shell.py | 88 +++++++++++++++++++++++++++++++++---
 python/setup.cfg             |  3 ++
 scripts/qmp/qmp-shell-wrap   | 11 +++++
 3 files changed, 95 insertions(+), 7 deletions(-)
 create mode 100755 scripts/qmp/qmp-shell-wrap

-- 
2.33.1

[PATCH v2 1/2] python: introduce qmp-shell-wrap convenience tool

[Daniel P. Berrangé]

With the current 'qmp-shell' tool developers must first spawn QEMU with
a suitable -qmp arg and then spawn qmp-shell in a separate terminal
pointing to the right socket.

With 'qmp-shell-wrap' developers can ignore QMP sockets entirely and
just pass the QEMU command and arguments they want. The program will
listen on a UNIX socket and tell QEMU to connect QMP to that.

For example, this:

 # qmp-shell-wrap -- qemu-system-x86_64 -display none

Is roughly equivalent of running:

 # qemu-system-x86_64 -display none -qmp qmp-shell-1234 &
 # qmp-shell qmp-shell-1234

Except that 'qmp-shell-wrap' switches the socket peers around so that
it is the UNIX socket server and QEMU is the socket client. This makes
QEMU reliably go away when qmp-shell-wrap exits, closing the server
socket.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 python/qemu/qmp/qmp_shell.py | 67 +++++++++++++++++++++++++++++++++---
 scripts/qmp/qmp-shell-wrap   | 11 ++++++
 2 files changed, 74 insertions(+), 4 deletions(-)
 create mode 100755 scripts/qmp/qmp-shell-wrap

diff --git a/python/qemu/qmp/qmp_shell.py b/python/qemu/qmp/qmp_shell.py
index e7d7eb18f1..fb9e7701af 100644
--- a/python/qemu/qmp/qmp_shell.py
+++ b/python/qemu/qmp/qmp_shell.py
@@ -86,6 +86,7 @@
 import os
 import re
 import readline
+from subprocess import Popen
 import sys
 from typing import (
     Iterator,
@@ -162,8 +163,10 @@ class QMPShell(qmp.QEMUMonitorProtocol):
     :param verbose: Echo outgoing QMP messages to console.
     """
     def __init__(self, address: qmp.SocketAddrT,
-                 pretty: bool = False, verbose: bool = False):
-        super().__init__(address)
+                 pretty: bool = False,
+                 verbose: bool = False,
+                 server: bool = False):
+        super().__init__(address, server=server)
         self._greeting: Optional[QMPMessage] = None
         self._completer = QMPCompleter()
         self._transmode = False
@@ -404,8 +407,10 @@ class HMPShell(QMPShell):
     :param verbose: Echo outgoing QMP messages to console.
     """
     def __init__(self, address: qmp.SocketAddrT,
-                 pretty: bool = False, verbose: bool = False):
-        super().__init__(address, pretty, verbose)
+                 pretty: bool = False,
+                 verbose: bool = False,
+                 server: bool = False):
+        super().__init__(address, pretty, verbose, server)
         self._cpu_index = 0
 
     def _cmd_completion(self) -> None:
@@ -530,5 +535,59 @@ def main() -> None:
             pass
 
 
+def main_wrap() -> None:
+    """
+    qmp-shell-wrap entry point: parse command line arguments and
+    start the REPL.
+    """
+    parser = argparse.ArgumentParser()
+    parser.add_argument('-H', '--hmp', action='store_true',
+                        help='Use HMP interface')
+    parser.add_argument('-v', '--verbose', action='store_true',
+                        help='Verbose (echo commands sent and received)')
+    parser.add_argument('-p', '--pretty', action='store_true',
+                        help='Pretty-print JSON')
+
+    parser.add_argument('command', nargs=argparse.REMAINDER,
+                        help='QEMU command line to invoke')
+
+    args = parser.parse_args()
+
+    cmd = args.command
+    if len(cmd) != 0 and cmd[0] == '--':
+        cmd = cmd[1:]
+    if len(cmd) == 0:
+        cmd = ["qemu-system-x86_64"]
+
+    sockpath = "qmp-shell-wrap-%d" % os.getpid()
+    cmd += ["-qmp", "unix:%s" % sockpath]
+
+    shell_class = HMPShell if args.hmp else QMPShell
+
+    try:
+        address = shell_class.parse_address(sockpath)
+    except qmp.QMPBadPortError:
+        parser.error(f"Bad port number: {sockpath}")
+        return  # pycharm doesn't know error() is noreturn
+
+    try:
+        with shell_class(address, args.pretty, args.verbose, True) as qemu:
+            with Popen(cmd):
+
+                try:
+                    qemu.accept()
+                except qmp.QMPConnectError:
+                    die("Didn't get QMP greeting message")
+                except qmp.QMPCapabilitiesError:
+                    die("Couldn't negotiate capabilities")
+                except OSError as err:
+                    die(f"Couldn't connect to {sockpath}: {err!s}")
+
+                for _ in qemu.repl():
+                    pass
+    finally:
+        os.unlink(sockpath)
+
+
 if __name__ == '__main__':
     main()
diff --git a/scripts/qmp/qmp-shell-wrap b/scripts/qmp/qmp-shell-wrap
new file mode 100755
index 0000000000..9e94da114f
--- /dev/null
+++ b/scripts/qmp/qmp-shell-wrap
@@ -0,0 +1,11 @@
+#!/usr/bin/env python3
+
+import os
+import sys
+
+sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
+from qemu.qmp import qmp_shell
+
+
+if __name__ == '__main__':
+    qmp_shell.main_wrap()
-- 
2.33.1

Re: [PATCH v2 1/2] python: introduce qmp-shell-wrap convenience tool

[John Snow]

On Tue, Jan 18, 2022 at 5:01 AM Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> With the current 'qmp-shell' tool developers must first spawn QEMU with
> a suitable -qmp arg and then spawn qmp-shell in a separate terminal
> pointing to the right socket.
>
> With 'qmp-shell-wrap' developers can ignore QMP sockets entirely and
> just pass the QEMU command and arguments they want. The program will
> listen on a UNIX socket and tell QEMU to connect QMP to that.
>
> For example, this:
>
>  # qmp-shell-wrap -- qemu-system-x86_64 -display none
>
> Is roughly equivalent of running:
>
>  # qemu-system-x86_64 -display none -qmp qmp-shell-1234 &
>  # qmp-shell qmp-shell-1234
>
> Except that 'qmp-shell-wrap' switches the socket peers around so that
> it is the UNIX socket server and QEMU is the socket client. This makes
> QEMU reliably go away when qmp-shell-wrap exits, closing the server
> socket.
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

Thanks, I think this is pretty useful.

Can you look at setup.cfg and see about adding a qmp-shell-wrap entry
point there? I had intended to wean people off of using /scripts for
things that rely on the QMP packages, because I'm gonna fork them out
and then these little forwards won't work without installing something
anyway.

Also, as an FYI: Stuff that sticks around in /python/qemu/qmp/ is
going to get forked out and uploaded to PyPI; stuff that gets added to
/python/qemu/utils is going to stay local to our tree and has more
freedom to be changed liberally. If you don't think this script
belongs on PyPI, we could always stick it in util.

--js

Re: [PATCH v2 1/2] python: introduce qmp-shell-wrap convenience tool

[Daniel P. Berrangé]

On Tue, Jan 18, 2022 at 08:07:32PM -0500, John Snow wrote:
> On Tue, Jan 18, 2022 at 5:01 AM Daniel P. Berrangé <berrange@redhat.com> wrote:
> >
> > With the current 'qmp-shell' tool developers must first spawn QEMU with
> > a suitable -qmp arg and then spawn qmp-shell in a separate terminal
> > pointing to the right socket.
> >
> > With 'qmp-shell-wrap' developers can ignore QMP sockets entirely and
> > just pass the QEMU command and arguments they want. The program will
> > listen on a UNIX socket and tell QEMU to connect QMP to that.
> >
> > For example, this:
> >
> >  # qmp-shell-wrap -- qemu-system-x86_64 -display none
> >
> > Is roughly equivalent of running:
> >
> >  # qemu-system-x86_64 -display none -qmp qmp-shell-1234 &
> >  # qmp-shell qmp-shell-1234
> >
> > Except that 'qmp-shell-wrap' switches the socket peers around so that
> > it is the UNIX socket server and QEMU is the socket client. This makes
> > QEMU reliably go away when qmp-shell-wrap exits, closing the server
> > socket.
> >
> > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> 
> Thanks, I think this is pretty useful.
> 
> Can you look at setup.cfg and see about adding a qmp-shell-wrap entry
> point there? I had intended to wean people off of using /scripts for
> things that rely on the QMP packages, because I'm gonna fork them out
> and then these little forwards won't work without installing something
> anyway.
> 
> Also, as an FYI: Stuff that sticks around in /python/qemu/qmp/ is
> going to get forked out and uploaded to PyPI; stuff that gets added to
> /python/qemu/utils is going to stay local to our tree and has more
> freedom to be changed liberally. If you don't think this script
> belongs on PyPI, we could always stick it in util.

IMHO it belongs anywhere that the existing qmp-shell lives


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v2 1/2] python: introduce qmp-shell-wrap convenience tool

[Daniel P. Berrangé]

On Tue, Jan 18, 2022 at 08:07:32PM -0500, John Snow wrote:
> On Tue, Jan 18, 2022 at 5:01 AM Daniel P. Berrangé <berrange@redhat.com> wrote:
> >
> > With the current 'qmp-shell' tool developers must first spawn QEMU with
> > a suitable -qmp arg and then spawn qmp-shell in a separate terminal
> > pointing to the right socket.
> >
> > With 'qmp-shell-wrap' developers can ignore QMP sockets entirely and
> > just pass the QEMU command and arguments they want. The program will
> > listen on a UNIX socket and tell QEMU to connect QMP to that.
> >
> > For example, this:
> >
> >  # qmp-shell-wrap -- qemu-system-x86_64 -display none
> >
> > Is roughly equivalent of running:
> >
> >  # qemu-system-x86_64 -display none -qmp qmp-shell-1234 &
> >  # qmp-shell qmp-shell-1234
> >
> > Except that 'qmp-shell-wrap' switches the socket peers around so that
> > it is the UNIX socket server and QEMU is the socket client. This makes
> > QEMU reliably go away when qmp-shell-wrap exits, closing the server
> > socket.
> >
> > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> 
> Thanks, I think this is pretty useful.
> 
> Can you look at setup.cfg and see about adding a qmp-shell-wrap entry
> point there? I had intended to wean people off of using /scripts for
> things that rely on the QMP packages, because I'm gonna fork them out
> and then these little forwards won't work without installing something
> anyway.

This looks simple enough but when I test I can't actuall get any of
the existing programs to work this way.

I did:

 $ python setup.py  install --user
 ...snip...
 Processing qemu-0.6.1.0a1-py3.10.egg
 Copying qemu-0.6.1.0a1-py3.10.egg to /home/berrange/.local/lib/python3.10/site-packages
 Adding qemu 0.6.1.0a1 to easy-install.pth file
 Installing aqmp-tui script to /home/berrange/.local/bin
 Installing qemu-ga-client script to /home/berrange/.local/bin
 Installing qmp-shell script to /home/berrange/.local/bin
 Installing qmp-shell-wrap script to /home/berrange/.local/bin
 Installing qom script to /home/berrange/.local/bin
 Installing qom-fuse script to /home/berrange/.local/bin
 Installing qom-get script to /home/berrange/.local/bin
 Installing qom-list script to /home/berrange/.local/bin
 Installing qom-set script to /home/berrange/.local/bin
 Installing qom-tree script to /home/berrange/.local/bin

 Installed /home/berrange/.local/lib/python3.10/site-packages/qemu-0.6.1.0a1-py3.10.egg
 Processing dependencies for qemu==0.6.1.0a1
 Finished processing dependencies for qemu==0.6.1.0a1


$ export PYTHONPATH=/home/berrange/.local/lib/python3.10/site-packages
$ qmp-shell
$ qmp-shell
Traceback (most recent call last):
  File "/home/berrange/.local/bin/qmp-shell", line 33, in <module>
    sys.exit(load_entry_point('qemu==0.6.1.0a1', 'console_scripts', 'qmp-shell')())
  File "/home/berrange/.local/bin/qmp-shell", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/usr/lib64/python3.10/importlib/metadata/__init__.py", line 162, in load
    module = import_module(match.group('module'))
  File "/usr/lib64/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'qemu'

I can't see why this is failing to find 'qemu' when it exists fine:

$ python
>>> import qemu.aqmp.qmp_shell
>>> qemu.aqmp.qmp_shell.main()
usage: [-h] [-H] [-N] [-v] [-p] [-l LOGFILE] qmp_server
: error: the following arguments are required: qmp_server


Why is 'load_entry_point' unhappy ?

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v2 1/2] python: introduce qmp-shell-wrap convenience tool

[John Snow]

[-- Attachment #1: Type: text/plain, Size: 5405 bytes --]

On Fri, Jan 28, 2022, 11:08 AM Daniel P. Berrangé <berrange@redhat.com>
wrote:

> On Tue, Jan 18, 2022 at 08:07:32PM -0500, John Snow wrote:
> > On Tue, Jan 18, 2022 at 5:01 AM Daniel P. Berrangé <berrange@redhat.com>
> wrote:
> > >
> > > With the current 'qmp-shell' tool developers must first spawn QEMU with
> > > a suitable -qmp arg and then spawn qmp-shell in a separate terminal
> > > pointing to the right socket.
> > >
> > > With 'qmp-shell-wrap' developers can ignore QMP sockets entirely and
> > > just pass the QEMU command and arguments they want. The program will
> > > listen on a UNIX socket and tell QEMU to connect QMP to that.
> > >
> > > For example, this:
> > >
> > >  # qmp-shell-wrap -- qemu-system-x86_64 -display none
> > >
> > > Is roughly equivalent of running:
> > >
> > >  # qemu-system-x86_64 -display none -qmp qmp-shell-1234 &
> > >  # qmp-shell qmp-shell-1234
> > >
> > > Except that 'qmp-shell-wrap' switches the socket peers around so that
> > > it is the UNIX socket server and QEMU is the socket client. This makes
> > > QEMU reliably go away when qmp-shell-wrap exits, closing the server
> > > socket.
> > >
> > > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> >
> > Thanks, I think this is pretty useful.
> >
> > Can you look at setup.cfg and see about adding a qmp-shell-wrap entry
> > point there? I had intended to wean people off of using /scripts for
> > things that rely on the QMP packages, because I'm gonna fork them out
> > and then these little forwards won't work without installing something
> > anyway.
>
> This looks simple enough but when I test I can't actuall get any of
> the existing programs to work this way.
>
> I did:
>
>  $ python setup.py  install --user
>  ...snip...
>  Processing qemu-0.6.1.0a1-py3.10.egg
>  Copying qemu-0.6.1.0a1-py3.10.egg to
> /home/berrange/.local/lib/python3.10/site-packages
>  Adding qemu 0.6.1.0a1 to easy-install.pth file
>  Installing aqmp-tui script to /home/berrange/.local/bin
>  Installing qemu-ga-client script to /home/berrange/.local/bin
>  Installing qmp-shell script to /home/berrange/.local/bin
>  Installing qmp-shell-wrap script to /home/berrange/.local/bin
>  Installing qom script to /home/berrange/.local/bin
>  Installing qom-fuse script to /home/berrange/.local/bin
>  Installing qom-get script to /home/berrange/.local/bin
>  Installing qom-list script to /home/berrange/.local/bin
>  Installing qom-set script to /home/berrange/.local/bin
>  Installing qom-tree script to /home/berrange/.local/bin
>
>  Installed
> /home/berrange/.local/lib/python3.10/site-packages/qemu-0.6.1.0a1-py3.10.egg
>  Processing dependencies for qemu==0.6.1.0a1
>  Finished processing dependencies for qemu==0.6.1.0a1
>
>
> $ export PYTHONPATH=/home/berrange/.local/lib/python3.10/site-packages
> $ qmp-shell
> $ qmp-shell
> Traceback (most recent call last):
>   File "/home/berrange/.local/bin/qmp-shell", line 33, in <module>
>     sys.exit(load_entry_point('qemu==0.6.1.0a1', 'console_scripts',
> 'qmp-shell')())
>   File "/home/berrange/.local/bin/qmp-shell", line 25, in
> importlib_load_entry_point
>     return next(matches).load()
>   File "/usr/lib64/python3.10/importlib/metadata/__init__.py", line 162,
> in load
>     module = import_module(match.group('module'))
>   File "/usr/lib64/python3.10/importlib/__init__.py", line 126, in
> import_module
>     return _bootstrap._gcd_import(name[level:], package, level)
>   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
>   File "<frozen importlib._bootstrap>", line 992, in
> _find_and_load_unlocked
>   File "<frozen importlib._bootstrap>", line 241, in
> _call_with_frames_removed
>   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
>   File "<frozen importlib._bootstrap>", line 992, in
> _find_and_load_unlocked
>   File "<frozen importlib._bootstrap>", line 241, in
> _call_with_frames_removed
>   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
>   File "<frozen importlib._bootstrap>", line 1004, in
> _find_and_load_unlocked
> ModuleNotFoundError: No module named 'qemu'
>
> I can't see why this is failing to find 'qemu' when it exists fine:
>
> $ python
> >>> import qemu.aqmp.qmp_shell
> >>> qemu.aqmp.qmp_shell.main()
> usage: [-h] [-H] [-N] [-v] [-p] [-l LOGFILE] qmp_server
> : error: the following arguments are required: qmp_server
>
>
> Why is 'load_entry_point' unhappy ?
>

I'm sorry, I'm not sure.

It may be because you installed via setup.py instead of pip ... I always
use pip. "Eventually" setup.py is going away, but there's some reasons I
haven't dropped it for QEMU yet. (Our long support tail for RHEL.)

(I don't know if that's the reason, I'll test. If it is the reason, I'll
see if there's some way to guard against this in the future.)

I see you've posted a new version, I'll test with that version and if you
don't mind, if there's any small problem I'll just modify it for the PR
myself, and you can review the changes and I'll get you on your way
quicker.

Thanks (and sorry for so many long emails lately),
--js

[-- Attachment #2: Type: text/html, Size: 6865 bytes --]

[PATCH v2 2/2] python: support recording QMP session to a file

[Daniel P. Berrangé]

When running QMP commands with very large response payloads, it is often
not easy to spot the info you want. If we can save the response to a
file then tools like 'grep' or 'jq' can be used to extract information.

For convenience of processing, we merge the QMP command and response
dictionaries together:

  {
      "arguments": {},
      "execute": "query-kvm",
      "return": {
          "enabled": false,
          "present": true
      }
  }

Example usage

  $ ./scripts/qmp/qmp-shell-wrap -l q.log -p -- ./build/qemu-system-x86_64 -display none
  Welcome to the QMP low-level shell!
  Connected
  (QEMU) query-kvm
  {
      "return": {
          "enabled": false,
          "present": true
      }
  }
  (QEMU) query-mice
  {
      "return": [
          {
              "absolute": false,
              "current": true,
              "index": 2,
              "name": "QEMU PS/2 Mouse"
          }
      ]
  }

 $ jq --slurp '. | to_entries[] | select(.value.execute == "query-kvm") |
               .value.return.enabled' < q.log
   false

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 python/qemu/qmp/qmp_shell.py | 29 ++++++++++++++++++++++-------
 python/setup.cfg             |  3 +++
 2 files changed, 25 insertions(+), 7 deletions(-)

diff --git a/python/qemu/qmp/qmp_shell.py b/python/qemu/qmp/qmp_shell.py
index fb9e7701af..2b54114bd2 100644
--- a/python/qemu/qmp/qmp_shell.py
+++ b/python/qemu/qmp/qmp_shell.py
@@ -89,6 +89,7 @@
 from subprocess import Popen
 import sys
 from typing import (
+    IO,
     Iterator,
     List,
     NoReturn,
@@ -165,7 +166,8 @@ class QMPShell(qmp.QEMUMonitorProtocol):
     def __init__(self, address: qmp.SocketAddrT,
                  pretty: bool = False,
                  verbose: bool = False,
-                 server: bool = False):
+                 server: bool = False,
+                 logfile: Optional[str] = None):
         super().__init__(address, server=server)
         self._greeting: Optional[QMPMessage] = None
         self._completer = QMPCompleter()
@@ -175,6 +177,10 @@ def __init__(self, address: qmp.SocketAddrT,
                                       '.qmp-shell_history')
         self.pretty = pretty
         self.verbose = verbose
+        self.logfile = None
+
+        if logfile is not None:
+            self.logfile = open(logfile, "w", encoding='utf-8')
 
     def close(self) -> None:
         # Hook into context manager of parent to save shell history.
@@ -315,11 +321,11 @@ def _build_cmd(self, cmdline: str) -> Optional[QMPMessage]:
         self._cli_expr(cmdargs[1:], qmpcmd['arguments'])
         return qmpcmd
 
-    def _print(self, qmp_message: object) -> None:
+    def _print(self, qmp_message: object, fh: IO[str] = sys.stdout) -> None:
         jsobj = json.dumps(qmp_message,
                            indent=4 if self.pretty else None,
                            sort_keys=self.pretty)
-        print(str(jsobj))
+        print(str(jsobj), file=fh)
 
     def _execute_cmd(self, cmdline: str) -> bool:
         try:
@@ -342,6 +348,9 @@ def _execute_cmd(self, cmdline: str) -> bool:
             print('Disconnected')
             return False
         self._print(resp)
+        if self.logfile is not None:
+            cmd = {**qmpcmd, **resp}
+            self._print(cmd, fh=self.logfile)
         return True
 
     def connect(self, negotiate: bool = True) -> None:
@@ -409,8 +418,9 @@ class HMPShell(QMPShell):
     def __init__(self, address: qmp.SocketAddrT,
                  pretty: bool = False,
                  verbose: bool = False,
-                 server: bool = False):
-        super().__init__(address, pretty, verbose, server)
+                 server: bool = False,
+                 logfile: Optional[str] = None):
+        super().__init__(address, pretty, verbose, server, logfile)
         self._cpu_index = 0
 
     def _cmd_completion(self) -> None:
@@ -503,6 +513,8 @@ def main() -> None:
                         help='Verbose (echo commands sent and received)')
     parser.add_argument('-p', '--pretty', action='store_true',
                         help='Pretty-print JSON')
+    parser.add_argument('-l', '--logfile',
+                        help='Save log of all QMP messages to PATH')
 
     default_server = os.environ.get('QMP_SOCKET')
     parser.add_argument('qmp_server', action='store',
@@ -521,7 +533,7 @@ def main() -> None:
         parser.error(f"Bad port number: {args.qmp_server}")
         return  # pycharm doesn't know error() is noreturn
 
-    with shell_class(address, args.pretty, args.verbose) as qemu:
+    with shell_class(address, args.pretty, args.verbose, args.logfile) as qemu:
         try:
             qemu.connect(negotiate=not args.skip_negotiation)
         except qmp.QMPConnectError:
@@ -547,6 +559,8 @@ def main_wrap() -> None:
                         help='Verbose (echo commands sent and received)')
     parser.add_argument('-p', '--pretty', action='store_true',
                         help='Pretty-print JSON')
+    parser.add_argument('-l', '--logfile',
+                        help='Save log of all QMP messages to PATH')
 
     parser.add_argument('command', nargs=argparse.REMAINDER,
                         help='QEMU command line to invoke')
@@ -571,7 +585,8 @@ def main_wrap() -> None:
         return  # pycharm doesn't know error() is noreturn
 
     try:
-        with shell_class(address, args.pretty, args.verbose, True) as qemu:
+        with shell_class(address, args.pretty, args.verbose,
+                         True, args.logfile) as qemu:
             with Popen(cmd):
 
                 try:
diff --git a/python/setup.cfg b/python/setup.cfg
index 417e937839..a909321d00 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -113,7 +113,10 @@ ignore_missing_imports = True
 # no Warning level messages displayed, use "--disable=all --enable=classes
 # --disable=W".
 disable=consider-using-f-string,
+        consider-using-with,
+        too-many-arguments,
         too-many-function-args,  # mypy handles this with less false positives.
+        too-many-instance-attributes,
         no-member,  # mypy also handles this better.
 
 [pylint.basic]
-- 
2.33.1

Re: [PATCH v2 2/2] python: support recording QMP session to a file

[Philippe Mathieu-Daudé via]

On 1/18/22 11:01, Daniel P. Berrangé wrote:
> When running QMP commands with very large response payloads, it is often
> not easy to spot the info you want. If we can save the response to a
> file then tools like 'grep' or 'jq' can be used to extract information.

> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>  python/qemu/qmp/qmp_shell.py | 29 ++++++++++++++++++++++-------
>  python/setup.cfg             |  3 +++
>  2 files changed, 25 insertions(+), 7 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>