[Qemu-devel] [PATCH 0/3]: QMP: Improve demo scripts

[Qemu-devel] [PATCH 0/3]: QMP: Improve demo scripts

[Luiz Capitulino]

Please, check individual patches for details.

Thanks.

[Qemu-devel] [PATCH 1/3] QMP: Revamp the Python class example

[Luiz Capitulino]

This commit simplifies and fixes a number of problems in the Python
QEMUMonitorProtocol example class.

It's almost a rewrite and it DOES BREAK the qmp-shell script (which
is going to be fixed in the next commit).

However, I'm not going to split this in different commits because it
could get up to 10 commits, it's really not worth it for a simple
demo class.

Highlights:

 o TCP sockets support
 o QMP events support
 o Add documentation
 o Fix a number of unhandled errors
 o Simplify methods that send commands to the Monitor

Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
---
 QMP/qmp.py |  157 ++++++++++++++++++++++++++++++++++++++++-------------------
 1 files changed, 106 insertions(+), 51 deletions(-)

diff --git a/QMP/qmp.py b/QMP/qmp.py
index 4062f84..14ce8b0 100644
--- a/QMP/qmp.py
+++ b/QMP/qmp.py
@@ -1,6 +1,6 @@
 # QEMU Monitor Protocol Python class
 # 
-# Copyright (C) 2009 Red Hat Inc.
+# Copyright (C) 2009, 2010 Red Hat Inc.
 #
 # Authors:
 #  Luiz Capitulino <lcapitulino@redhat.com>
@@ -8,7 +8,9 @@
 # This work is licensed under the terms of the GNU GPL, version 2.  See
 # the COPYING file in the top-level directory.
 
-import socket, json
+import json
+import errno
+import socket
 
 class QMPError(Exception):
     pass
@@ -16,61 +18,114 @@ class QMPError(Exception):
 class QMPConnectError(QMPError):
     pass
 
+class QMPCapabilitiesError(QMPError):
+    pass
+
 class QEMUMonitorProtocol:
+    def __init__(self, address):
+        """
+        Create a QEMUMonitorProtocol class.
+
+        @param address: QEMU address, can be either a unix socket path (string)
+                        or a tuple in the form ( address, port ) for a TCP
+                        connection
+        @note No connection is established, this is done by the connect() method
+        """
+        self.__events = []
+        self.__address = address
+        self.__sock = self.__get_sock()
+        self.__sockfile = self.__sock.makefile()
+
+    def __get_sock(self):
+        if isinstance(self.__address, tuple):
+            family = socket.AF_INET
+        else:
+            family = socket.AF_UNIX
+        return socket.socket(family, socket.SOCK_STREAM)
+
+    def __json_read(self):
+        while True:
+            data = self.__sockfile.readline()
+            if not data:
+                return
+            resp = json.loads(data)
+            if 'event' in resp:
+                self.__events.append(resp)
+                continue
+            return resp
+
+    error = socket.error
+
     def connect(self):
-        self.sock.connect(self.filename)
-        data = self.__json_read()
-        if data == None:
-            raise QMPConnectError
-        if not data.has_key('QMP'):
+        """
+        Connect to the QMP Monitor and perform capabilities negotiation.
+
+        @return QMP greeting dict
+        @raise socket.error on socket connection errors
+        @raise QMPConnectError if the greeting is not received
+        @raise QMPCapabilitiesError if fails to negotiate capabilities
+        """
+        self.__sock.connect(self.__address)
+        greeting = self.__json_read()
+        if greeting is None or not greeting.has_key('QMP'):
             raise QMPConnectError
-        return data['QMP']['capabilities']
+        # Greeting seems ok, negotiate capabilities
+        resp = self.cmd('qmp_capabilities')
+        if "return" in resp:
+            return greeting
+        raise QMPCapabilitiesError
 
-    def close(self):
-        self.sock.close()
+    def cmd_obj(self, qmp_cmd):
+        """
+        Send a QMP command to the QMP Monitor.
 
-    def send_raw(self, line):
-        self.sock.send(str(line))
+        @param qmp_cmd: QMP command to be sent as a Python dict
+        @return QMP response as a Python dict or None if the connection has
+                been closed
+        """
+        try:
+            self.__sock.sendall(json.dumps(qmp_cmd))
+        except socket.error, err:
+            if err[0] == errno.EPIPE:
+                return
+            raise socket.error(err)
         return self.__json_read()
 
-    def send(self, cmdline):
-        cmd = self.__build_cmd(cmdline)
-        self.__json_send(cmd)
-        resp = self.__json_read()
-        if resp == None:
-            return
-        elif resp.has_key('error'):
-            return resp['error']
-        else:
-            return resp['return']
-
-    def __build_cmd(self, cmdline):
-        cmdargs = cmdline.split()
-        qmpcmd = { 'execute': cmdargs[0], 'arguments': {} }
-        for arg in cmdargs[1:]:
-            opt = arg.split('=')
-            try:
-                value = int(opt[1])
-            except ValueError:
-                value = opt[1]
-            qmpcmd['arguments'][opt[0]] = value
-        return qmpcmd
-
-    def __json_send(self, cmd):
-        # XXX: We have to send any additional char, otherwise
-        # the Server won't read our input
-        self.sock.send(json.dumps(cmd) + ' ')
+    def cmd(self, name, args=None, id=None):
+        """
+        Build a QMP command and send it to the QMP Monitor.
 
-    def __json_read(self):
+        @param name: command name (string)
+        @param args: command arguments (dict)
+        @param id: command id (dict, list, string or int)
+        """
+        qmp_cmd = { 'execute': name }
+        if args:
+            qmp_cmd['arguments'] = args
+        if id:
+            qmp_cmd['id'] = id
+        return self.cmd_obj(qmp_cmd)
+
+    def get_events(self):
+        """
+        Get a list of available QMP events.
+        """
+        self.__sock.setblocking(0)
         try:
-            while True:
-                line = json.loads(self.sockfile.readline())
-                if not 'event' in line:
-                    return line
-        except ValueError:
-            return
-
-    def __init__(self, filename):
-        self.filename = filename
-        self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
-        self.sockfile = self.sock.makefile()
+            self.__json_read()
+        except socket.error, err:
+            if err[0] == errno.EAGAIN:
+                # No data available
+                pass
+        self.__sock.setblocking(1)
+        return self.__events
+
+    def clear_events(self):
+        """
+        Clear current list of pending events.
+        """
+        self.__events = []
+
+    def close(self):
+        self.__sock.close()
+        self.__sockfile.close()
-- 
1.7.3.2.145.g7ebee

[Qemu-devel] [PATCH 2/3] QMP: Revamp the qmp-shell script

[Luiz Capitulino]

This commit updates the qmp-shell script to use the new interface
introduced by the last commit.

Additionally, the following fixes/features are also introduced:

 o TCP sockets support
 o Update/add documentation
 o Simple command-line completion
 o Fix a number of unhandled errors

Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
---
 QMP/qmp-shell |  179 +++++++++++++++++++++++++++++++++++++++++++++-----------
 1 files changed, 144 insertions(+), 35 deletions(-)

diff --git a/QMP/qmp-shell b/QMP/qmp-shell
index a5b72d1..1fb7e76 100755
--- a/QMP/qmp-shell
+++ b/QMP/qmp-shell
@@ -1,8 +1,8 @@
 #!/usr/bin/python
 #
-# Simple QEMU shell on top of QMP
+# Low-level QEMU shell on top of QMP.
 #
-# Copyright (C) 2009 Red Hat Inc.
+# Copyright (C) 2009, 2010 Red Hat Inc.
 #
 # Authors:
 #  Luiz Capitulino <lcapitulino@redhat.com>
@@ -14,60 +14,169 @@
 #
 # Start QEMU with:
 #
-# $ qemu [...] -monitor control,unix:./qmp,server
+# # qemu [...] -qmp unix:./qmp-sock,server
 #
 # Run the shell:
 #
-# $ qmp-shell ./qmp
+# $ qmp-shell ./qmp-sock
 #
 # Commands have the following format:
 #
-# < command-name > [ arg-name1=arg1 ] ... [ arg-nameN=argN ]
+#    < command-name > [ arg-name1=arg1 ] ... [ arg-nameN=argN ]
 #
 # For example:
 #
-# (QEMU) info item=network
+# (QEMU) device_add driver=e1000 id=net1
+# {u'return': {}}
+# (QEMU)
 
 import qmp
 import readline
-from sys import argv,exit
+import sys
 
-def shell_help():
-    print 'bye  exit from the shell'
+class QMPCompleter(list):
+    def complete(self, text, state):
+        for cmd in self:
+            if cmd.startswith(text):
+                if not state:
+                    return cmd
+                else:
+                    state -= 1
 
-def main():
-    if len(argv) != 2:
-        print 'qemu-shell <unix-socket>'
-        exit(1)
+class QMPShellError(Exception):
+    pass
+
+class QMPShellBadPort(QMPShellError):
+    pass
+
+# TODO: QMPShell's interface is a bit ugly (eg. _fill_completion() and
+#       _execute_cmd()). Let's design a better one.
+class QMPShell(qmp.QEMUMonitorProtocol):
+    def __init__(self, address):
+        qmp.QEMUMonitorProtocol.__init__(self, self.__get_address(address))
+        self._greeting = None
+        self._completer = None
+
+    def __get_address(self, arg):
+        """
+        Figure out if the argument is in the port:host form, if it's not it's
+        probably a file path.
+        """
+        addr = arg.split(':')
+        if len(addr) == 2:
+            try:
+                port = int(addr[1])
+            except ValueError:
+                raise QMPShellBadPort
+            return ( addr[0], port )
+        # socket path
+        return arg
+
+    def _fill_completion(self):
+        for cmd in self.cmd('query-commands')['return']:
+            self._completer.append(cmd['name'])
+
+    def __completer_setup(self):
+        self._completer = QMPCompleter()
+        self._fill_completion()
+        readline.set_completer(self._completer.complete)
+        readline.parse_and_bind("tab: complete")
+        # XXX: default delimiters conflict with some command names (eg. query-),
+        # clearing everything as it doesn't seem to matter
+        readline.set_completer_delims('')
+
+    def __build_cmd(self, cmdline):
+        """
+        Build a QMP input object from a user provided command-line in the
+        following format:
+    
+            < command-name > [ arg-name1=arg1 ] ... [ arg-nameN=argN ]
+        """
+        cmdargs = cmdline.split()
+        qmpcmd = { 'execute': cmdargs[0], 'arguments': {} }
+        for arg in cmdargs[1:]:
+            opt = arg.split('=')
+            try:
+                value = int(opt[1])
+            except ValueError:
+                value = opt[1]
+            qmpcmd['arguments'][opt[0]] = value
+        return qmpcmd
+
+    def _execute_cmd(self, cmdline):
+        try:
+            qmpcmd = self.__build_cmd(cmdline)
+        except:
+            print 'command format: <command-name> ',
+            print '[arg-name1=arg1] ... [arg-nameN=argN]'
+            return True
+        resp = self.cmd_obj(qmpcmd)
+        if resp is None:
+            print 'Disconnected'
+            return False
+        print resp
+        return True
+
+    def connect(self):
+        self._greeting = qmp.QEMUMonitorProtocol.connect(self)
+        self.__completer_setup()
 
-    qemu = qmp.QEMUMonitorProtocol(argv[1])
-    qemu.connect()
-    qemu.send("qmp_capabilities")
+    def show_banner(self, msg='Welcome to the QMP low-level shell!'):
+        print msg
+        version = self._greeting['QMP']['version']['qemu']
+        print 'Connected to QEMU %d.%d.%d\n' % (version['major'],version['minor'],version['micro'])
 
-    print 'Connected!'
+    def read_exec_command(self, prompt):
+        """
+        Read and execute a command.
 
-    while True:
+        @return True if execution was ok, return False if disconnected.
+        """
         try:
-            cmd = raw_input('(QEMU) ')
+            cmdline = raw_input(prompt)
         except EOFError:
             print
-            break
-        if cmd == '':
-            continue
-        elif cmd == 'bye':
-            break
-        elif cmd == 'help':
-            shell_help()
+            return False
+        if cmdline == '':
+            for ev in self.get_events():
+                print ev
+            self.clear_events()
+            return True
         else:
-            try:
-                resp = qemu.send(cmd)
-                if resp == None:
-                    print 'Disconnected'
-                    break
-                print resp
-            except IndexError:
-                print '-> command format: <command-name> ',
-                print '[arg-name1=arg1] ... [arg-nameN=argN]'
+            return self._execute_cmd(cmdline)
+
+def die(msg):
+    sys.stderr.write('ERROR: %s\n' % msg)
+    sys.exit(1)
+
+def fail_cmdline(option=None):
+    if option:
+        sys.stderr.write('ERROR: bad command-line option \'%s\'\n' % option)
+    sys.stderr.write('qemu-shell [ -H ] < UNIX socket path> | < TCP address:port >\n')
+    sys.exit(1)
+
+def main():
+    try:
+        if len(sys.argv) == 2:
+            qemu = QMPShell(sys.argv[1])
+        else:
+                fail_cmdline()
+    except QMPShellBadPort:
+        die('bad port number in command-line')
+
+    try:
+        qemu.connect()
+    except qmp.QMPConnectError:
+        die('Didn\'t get QMP greeting message')
+    except qmp.QMPCapabilitiesError:
+        die('Could not negotiate capabilities')
+    except qemu.error:
+        die('Could not connect to %s' % sys.argv[1])
+
+    qemu.show_banner()
+    while qemu.read_exec_command('(QEMU) '):
+        pass
+    qemu.close()
 
 if __name__ == '__main__':
     main()
-- 
1.7.3.2.145.g7ebee

[Qemu-devel] [PATCH 3/3] QMP: Drop vm-info example script

[Luiz Capitulino]

It's broken and not really useful, let's just drop it.

Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
---
 QMP/README  |    5 +----
 QMP/vm-info |   33 ---------------------------------
 2 files changed, 1 insertions(+), 37 deletions(-)
 delete mode 100755 QMP/vm-info

diff --git a/QMP/README b/QMP/README
index 80503f2..c95a08c 100644
--- a/QMP/README
+++ b/QMP/README
@@ -19,10 +19,7 @@ o qmp-spec.txt      QEMU Monitor Protocol current specification
 o qmp-commands.txt  QMP supported commands (auto-generated at build-time)
 o qmp-events.txt    List of available asynchronous events
 
-There are also two simple Python scripts available:
-
-o qmp-shell  A shell
-o vm-info    Show some information about the Virtual Machine
+There is also a simple Python script called 'qmp-shell' available.
 
 IMPORTANT: It's strongly recommended to read the 'Stability Considerations'
 section in the qmp-commands.txt file before making any serious use of QMP.
diff --git a/QMP/vm-info b/QMP/vm-info
deleted file mode 100755
index be5b038..0000000
--- a/QMP/vm-info
+++ /dev/null
@@ -1,33 +0,0 @@
-#!/usr/bin/python
-#
-# Print Virtual Machine information
-#
-# Usage:
-#
-# Start QEMU with:
-#
-# $ qemu [...] -monitor control,unix:./qmp,server
-#
-# Run vm-info:
-#
-# $ vm-info ./qmp
-#
-# Luiz Capitulino <lcapitulino@redhat.com>
-
-import qmp
-from sys import argv,exit
-
-def main():
-    if len(argv) != 2:
-        print 'vm-info <unix-socket>'
-        exit(1)
-
-    qemu = qmp.QEMUMonitorProtocol(argv[1])
-    qemu.connect()
-    qemu.send("qmp_capabilities")
-
-    for cmd in [ 'version', 'kvm', 'status', 'uuid', 'balloon' ]:
-        print cmd + ': ' + str(qemu.send('query-' + cmd))
-
-if __name__ == '__main__':
-    main()
-- 
1.7.3.2.145.g7ebee

Re: [Qemu-devel] [PATCH 0/3]: QMP: Improve demo scripts

[Markus Armbruster]

Luiz Capitulino <lcapitulino@redhat.com> writes:

> Please, check individual patches for details.

I skimmed them, couldn't spot any problems.

PATCH 1 breaks qmp-shell, PATCH 2 unbreaks it.  Bisectability no-no in
theory, but unlikely to bite in practice.  Not worth a respin.