[Qemu-devel] [PATCH 0/2] Acceptance tests: arm/aarch64 host and target enablers

[Qemu-devel] [PATCH 0/2] Acceptance tests: arm/aarch64 host and target enablers

[Cleber Rosa]

The acceptance tests have been running primarily on x86_64 hosts, and
some tests will fail when run under non-x86 hosts or targets.

This series addresses issues that enables running the acceptance tests
under aarch64 hosts, and on other hosts using arm and aarch64 targets.

Cleber Rosa (2):
  Python libs: close console sockets before shutting down the VMs
  Python libs: enable machine type auto selection

 python/qemu/machine.py                    | 92 ++++++++++++++++++++++-
 tests/acceptance/avocado_qemu/__init__.py |  1 +
 2 files changed, 89 insertions(+), 4 deletions(-)

-- 
2.21.0

[Qemu-devel] [PATCH 1/2] Python libs: close console sockets before shutting down the VMs

[Cleber Rosa]

Currently, the console socket on QEMUMachine is closed after the QMP
command to gracefully exit QEMU is executed.  Because of a possible
deadlock (QEMU waiting for the socket to become writable) let's close
the console socket earlier.

Reference: <20190607034214.GB22416@habkost.net>
Reference: https://bugs.launchpad.net/qemu/+bug/1829779
From: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 python/qemu/machine.py | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/python/qemu/machine.py b/python/qemu/machine.py
index 128a3d1dc2..4f533b7881 100644
--- a/python/qemu/machine.py
+++ b/python/qemu/machine.py
@@ -271,10 +271,6 @@ class QEMUMachine(object):
 
         self._qemu_log_path = None
 
-        if self._console_socket is not None:
-            self._console_socket.close()
-            self._console_socket = None
-
         if self._temp_dir is not None:
             shutil.rmtree(self._temp_dir)
             self._temp_dir = None
@@ -333,6 +329,13 @@ class QEMUMachine(object):
         """
         Terminate the VM and clean up
         """
+        # If we keep the console socket open, we may deadlock waiting
+        # for QEMU to exit, while QEMU is waiting for the socket to
+        # become writeable.
+        if self._console_socket is not None:
+            self._console_socket.close()
+            self._console_socket = None
+
         if self.is_running():
             try:
                 if not has_quit:
-- 
2.21.0

[Qemu-devel] [PATCH 2/2] Python libs: enable machine type auto selection

[Cleber Rosa]

The arm and aarch64 targets are noteworthy because they don't have a
default machine type.  This introduces and opt-in option to users of
the qemu.machine.QEMUMachine Python class, in which a default machine
type will be set when a number of conditions are met.

The motivation for this feature is that a number of (acceptance) tests
are target agnostic, and should run with all tarets (in theory while
it's pending continuous test execution).  These tests can be listed
with:

  avocado list --filter-by-tags='-arch' --filter-by-tags-include-empty \
               --filter-by-tags-include-empty-key tests/acceptance/

Combined with the "qemu_bin" parameter that acceptance tests take, one
can run those tests with arm and aarch64 targets without adding
specific logic in each test.  To execute those with a aarch64 QEMU
binary one can run:

  avocado run --filter-by-tags='-arch' --filter-by-tags-include-empty \
              --filter-by-tags-include-empty-key \
              -p qemu_bin=aarch64-softmmu/qemu-system-aarch64 \
              tests/acceptance/

When this feature is activated, the default automatic selection of
a machine type will be logged (as it will also be done in the QEMU
command line that is already logged).

Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 python/qemu/machine.py                    | 81 +++++++++++++++++++++++
 tests/acceptance/avocado_qemu/__init__.py |  1 +
 2 files changed, 82 insertions(+)

diff --git a/python/qemu/machine.py b/python/qemu/machine.py
index 4f533b7881..350052a30b 100644
--- a/python/qemu/machine.py
+++ b/python/qemu/machine.py
@@ -69,6 +69,15 @@ class QEMUMachine(object):
         # vm is guaranteed to be shut down here
     """
 
+
+    #: The default machine model, used by :meth:`set_machine_auto_selection` and
+    #: :meth:`get_machine_auto_selection`
+    DEFAULT_MACHINE_MODELS = {
+        'arm': 'virt',
+        'aarch64': 'virt',
+        }
+
+
     def __init__(self, binary, args=None, wrapper=None, name=None,
                  test_dir="/var/tmp", monitor_address=None,
                  socket_scm_helper=None):
@@ -108,6 +117,7 @@ class QEMUMachine(object):
         self._temp_dir = None
         self._launched = False
         self._machine = None
+        self._machine_auto_selection = False
         self._console_set = False
         self._console_device_type = None
         self._console_address = None
@@ -286,6 +296,7 @@ class QEMUMachine(object):
 
         self._iolog = None
         self._qemu_full_args = None
+        self._perform_machine_auto_selection()
         try:
             self._launch()
             self._launched = True
@@ -533,3 +544,73 @@ class QEMUMachine(object):
                                                  socket.SOCK_STREAM)
             self._console_socket.connect(self._console_address)
         return self._console_socket
+
+    def set_machine_auto_selection(self, value):
+        """
+        Defines whether QEMUMachine will set a machine model when needed
+
+        The machine model will only be set automatically if:
+
+        1) No machine model was explicity set with :meth:`set_machine`
+        2) Target binary is known to not work properly when a machine
+           is not specified (usually producing "No machine specified,
+           and there's no default" on the command line)
+        3) A default machine model exists in :data:`DEFAULT_MACHINE_MODELS`
+        4) :func:`get_target_from_qemu_binary` is capable of returning
+           the target for the binary set on this class
+
+        If machine auto selection is enabled, it will be done so by
+        calling :meth:`set_machine` during the machine launc, that is
+        on :meth:`launch` so that a manually set machine model is not
+        discarded.
+
+        @param value: wether to enable or disable the machine auto selection
+                      feature
+        @type value: bool
+        """
+        self._machine_auto_selection = value
+
+    def get_machine_auto_selection(self):
+        """
+        Returns whether QEMUMachine will set a machine model when needed
+
+        Please refer to :meth:`set_machine_auto_selection` for a more
+        complete explanation.
+        """
+        return self._machine_auto_selection
+
+    def _perform_machine_auto_selection(self):
+        if not (self.get_machine_auto_selection() and self._machine is None):
+            return
+
+        target = self._get_target_from_qemu_binary()
+        if target is None:
+            LOG.warn('Machine auto selection was requested, but it was not '
+                     'applied as the binary "%s" could not be probed for its '
+                     'target architecture', self._binary)
+            return
+
+        machine_model = self.DEFAULT_MACHINE_MODELS.get(target, None)
+        if machine_model is None:
+            LOG.warn('Machine auto selection was requested, but it was not '
+                     'applied as there is no default machine model defined '
+                     'for target architecture "%s"', target)
+            return
+
+        LOG.debug('Setting machine model to: %s', machine_model)
+        self.set_machine(machine_model)
+
+    def _get_target_from_qemu_binary(self):
+        """
+        Returns the target for this machine's QEMU binary
+        """
+        qemu_machine = QEMUMachine(self._binary)
+        qemu_machine.add_args('-S')
+        qemu_machine.set_machine('none')
+        try:
+            qemu_machine.launch()
+            target_resp = qemu_machine.command('query-target')
+            qemu_machine.shutdown()
+            return target_resp['arch']
+        except Exception:
+            return None
diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py
index bd41e0443c..025bb6a725 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -70,6 +70,7 @@ class Test(avocado.Test):
 
     def _new_vm(self, *args):
         vm = QEMUMachine(self.qemu_bin)
+        vm.set_machine_auto_selection(True)
         if args:
             vm.add_args(*args)
         return vm
-- 
2.21.0