[PATCH v3 0/2] Convert the intel_iommu avocado test

[PATCH v3 0/2] Convert the intel_iommu avocado test

[Thomas Huth]

The first patch introduces a helper function for retrieving the
hostfwd ports from QEMU.
We then use this helper function to run a HTTP server in the guest
in the second patch to exercise the network of the guest.

v3:
- Use the new hostfwd helper function instead of trying to probe
  for an unused port on the host
- Use a constant for the guest port 8080

Thomas Huth (2):
  tests/functional: Add a helper function for retrieving the hostfwd
    port
  tests/functional: Convert the intel_iommu avocado test

 MAINTAINERS                                   |   1 +
 tests/functional/meson.build                  |   2 +
 tests/functional/qemu_test/utils.py           |   7 +
 tests/functional/test_info_usernet.py         |   8 +-
 .../test_intel_iommu.py}                      | 191 +++++++++++-------
 5 files changed, 135 insertions(+), 74 deletions(-)
 rename tests/{avocado/intel_iommu.py => functional/test_intel_iommu.py} (26%)
 mode change 100644 => 100755

-- 
2.47.1

[PATCH v3 1/2] tests/functional: Add a helper function for retrieving the hostfwd port

[Thomas Huth]

It's just a wrapper around get_info_usernet_hostfwd_port from the
qemu module that is also calling the right monitor command for
retrieving the information from QEMU.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/functional/qemu_test/utils.py   | 7 +++++++
 tests/functional/test_info_usernet.py | 8 +++-----
 2 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/tests/functional/qemu_test/utils.py b/tests/functional/qemu_test/utils.py
index 43853b4366..e7c8de8165 100644
--- a/tests/functional/qemu_test/utils.py
+++ b/tests/functional/qemu_test/utils.py
@@ -10,6 +10,13 @@
 
 import os
 
+from qemu.utils import get_info_usernet_hostfwd_port
+
+
+def get_usernet_hostfwd_port(vm):
+    res = vm.cmd('human-monitor-command', command_line='info usernet')
+    return get_info_usernet_hostfwd_port(res)
+
 """
 Round up to next power of 2
 """
diff --git a/tests/functional/test_info_usernet.py b/tests/functional/test_info_usernet.py
index cd37524d94..e8cbc37eed 100755
--- a/tests/functional/test_info_usernet.py
+++ b/tests/functional/test_info_usernet.py
@@ -11,8 +11,7 @@
 # later.  See the COPYING file in the top-level directory.
 
 from qemu_test import QemuSystemTest
-
-from qemu.utils import get_info_usernet_hostfwd_port
+from qemu_test.utils import get_usernet_hostfwd_port
 
 
 class InfoUsernet(QemuSystemTest):
@@ -22,9 +21,8 @@ def test_hostfwd(self):
         self.set_machine('none')
         self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22')
         self.vm.launch()
-        res = self.vm.cmd('human-monitor-command',
-                          command_line='info usernet')
-        port = get_info_usernet_hostfwd_port(res)
+
+        port = get_usernet_hostfwd_port(self.vm)
         self.assertIsNotNone(port,
                              ('"info usernet" output content does not seem to '
                               'contain the redirected port'))
-- 
2.47.1

[PATCH v3 2/2] tests/functional: Convert the intel_iommu avocado test

[Thomas Huth]

Convert the intel_iommu test to the new functional framework.
This test needs some changes since we neither support the old 'LinuxTest'
class in the functional framework yet, nor a way to use SSH for running
commands in the guest. So we now directly download a Fedora kernel and
initrd and set up the serial console for executing the commands and for
looking for the results. Instead of configuring the cloud image via
cloud-init, we now simply mount the file system manually from an initrd
rescue shell.

While the old test was exercising the network with a "dnf install"
command (which is not the best option for the CI since this depends
on third party servers), the new code is now setting up a little
HTTP server in the guest and transfers a file from the guest to the
host instead.

The test should now run much faster and more reliable (since we
don't depend on the third party servers for "dnf install" anymore),
so we can also drop the @skipUnless decorator now.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 MAINTAINERS                                   |   1 +
 tests/functional/meson.build                  |   2 +
 .../test_intel_iommu.py}                      | 191 +++++++++++-------
 3 files changed, 125 insertions(+), 69 deletions(-)
 rename tests/{avocado/intel_iommu.py => functional/test_intel_iommu.py} (26%)
 mode change 100644 => 100755

diff --git a/MAINTAINERS b/MAINTAINERS
index a800b766d3..79df59d05c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3682,6 +3682,7 @@ S: Supported
 F: hw/i386/intel_iommu.c
 F: hw/i386/intel_iommu_internal.h
 F: include/hw/i386/intel_iommu.h
+F: tests/functional/test_intel_iommu.py
 
 AMD-Vi Emulation
 S: Orphan
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index fb13d67e7f..054c5f85fb 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -31,6 +31,7 @@ test_timeouts = {
   'arm_raspi2' : 120,
   'arm_tuxrun' : 240,
   'arm_sx1' : 360,
+  'intel_iommu': 300,
   'mips_malta' : 120,
   'netdev_ethtool' : 180,
   'ppc_40p' : 240,
@@ -248,6 +249,7 @@ tests_x86_64_system_quick = [
 
 tests_x86_64_system_thorough = [
   'acpi_bits',
+  'intel_iommu',
   'linux_initrd',
   'multiprocess',
   'netdev_ethtool',
diff --git a/tests/avocado/intel_iommu.py b/tests/functional/test_intel_iommu.py
old mode 100644
new mode 100755
similarity index 26%
rename from tests/avocado/intel_iommu.py
rename to tests/functional/test_intel_iommu.py
index 992583fa7d..a9e8f82ab5
--- a/tests/avocado/intel_iommu.py
+++ b/tests/functional/test_intel_iommu.py
@@ -1,3 +1,5 @@
+#!/usr/bin/env python3
+#
 # INTEL_IOMMU Functional tests
 #
 # Copyright (c) 2021 Red Hat, Inc.
@@ -7,116 +9,167 @@
 #
 # This work is licensed under the terms of the GNU GPL, version 2 or
 # later.  See the COPYING file in the top-level directory.
-import os
-
-from avocado import skipUnless
-from avocado_qemu.linuxtest import LinuxTest
-
-@skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab')
-class IntelIOMMU(LinuxTest):
-    """
-    :avocado: tags=arch:x86_64
-    :avocado: tags=distro:fedora
-    :avocado: tags=distro_version:31
-    :avocado: tags=machine:q35
-    :avocado: tags=accel:kvm
-    :avocado: tags=intel_iommu
-    :avocado: tags=flaky
-    """
 
+import hashlib
+import urllib.request
+
+from qemu_test import LinuxKernelTest, Asset, exec_command_and_wait_for_pattern
+from qemu_test.utils import get_usernet_hostfwd_port
+
+
+class IntelIOMMU(LinuxKernelTest):
+
+    ASSET_KERNEL = Asset(
+        ('https://archives.fedoraproject.org/pub/archive/fedora/linux/releases'
+         '/31/Server/x86_64/os/images/pxeboot/vmlinuz'),
+        'd4738d03dbbe083ca610d0821d0a8f1488bebbdccef54ce33e3adb35fda00129')
+
+    ASSET_INITRD = Asset(
+        ('https://archives.fedoraproject.org/pub/archive/fedora/linux/releases'
+         '/31/Server/x86_64/os/images/pxeboot/initrd.img'),
+        '277cd6c7adf77c7e63d73bbb2cded8ef9e2d3a2f100000e92ff1f8396513cd8b')
+
+    ASSET_DISKIMAGE = Asset(
+        ('https://archives.fedoraproject.org/pub/archive/fedora/linux/releases'
+         '/31/Cloud/x86_64/images/Fedora-Cloud-Base-31-1.9.x86_64.qcow2'),
+        'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0')
+
+    DEFAULT_KERNEL_PARAMS = ('root=/dev/vda1 console=ttyS0 net.ifnames=0 '
+                             'quiet rd.rescue ')
+    GUEST_PORT = 8080
     IOMMU_ADDON = ',iommu_platform=on,disable-modern=off,disable-legacy=on'
     kernel_path = None
     initrd_path = None
     kernel_params = None
 
-    def set_up_boot(self):
-        path = self.download_boot()
+    def add_common_args(self, path):
+        self.vm.add_args('-drive', f'file={path},if=none,id=drv0,snapshot=on')
         self.vm.add_args('-device', 'virtio-blk-pci,bus=pcie.0,' +
                          'drive=drv0,id=virtio-disk0,bootindex=1,'
                          'werror=stop,rerror=stop' + self.IOMMU_ADDON)
         self.vm.add_args('-device', 'virtio-gpu-pci' + self.IOMMU_ADDON)
-        self.vm.add_args('-drive',
-                         'file=%s,if=none,cache=writethrough,id=drv0' % path)
 
-    def setUp(self):
-        super(IntelIOMMU, self).setUp(None, 'virtio-net-pci' + self.IOMMU_ADDON)
+        self.vm.add_args('-netdev',
+                         'user,id=n1,hostfwd=tcp:127.0.0.1:0-:%d' %
+                         self.GUEST_PORT)
+        self.vm.add_args('-device',
+                         'virtio-net-pci,netdev=n1' + self.IOMMU_ADDON)
 
-    def add_common_args(self):
         self.vm.add_args('-device', 'virtio-rng-pci,rng=rng0')
         self.vm.add_args('-object',
                          'rng-random,id=rng0,filename=/dev/urandom')
-
-    def common_vm_setup(self, custom_kernel=None):
-        self.require_accelerator("kvm")
-        self.add_common_args()
+        self.vm.add_args("-m", "1G")
         self.vm.add_args("-accel", "kvm")
 
-        if custom_kernel is None:
-            return
+    def common_vm_setup(self):
+        self.set_machine('q35')
+        self.require_accelerator("kvm")
+        self.require_netdev('user')
 
-        kernel_url = self.distro.pxeboot_url + 'vmlinuz'
-        kernel_hash = '5b6f6876e1b5bda314f93893271da0d5777b1f3c'
-        initrd_url = self.distro.pxeboot_url + 'initrd.img'
-        initrd_hash = 'dd0340a1b39bd28f88532babd4581c67649ec5b1'
-        self.kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
-        self.initrd_path = self.fetch_asset(initrd_url, asset_hash=initrd_hash)
+        self.kernel_path = self.ASSET_KERNEL.fetch()
+        self.initrd_path = self.ASSET_INITRD.fetch()
+        image_path = self.ASSET_DISKIMAGE.fetch()
+        self.add_common_args(image_path)
+        self.kernel_params = self.DEFAULT_KERNEL_PARAMS
 
     def run_and_check(self):
         if self.kernel_path:
             self.vm.add_args('-kernel', self.kernel_path,
                              '-append', self.kernel_params,
                              '-initrd', self.initrd_path)
-        self.launch_and_wait()
-        self.ssh_command('cat /proc/cmdline')
-        self.ssh_command('dmesg | grep -e DMAR -e IOMMU')
-        self.ssh_command('find /sys/kernel/iommu_groups/ -type l')
-        self.ssh_command('dnf -y install numactl-devel')
+        self.vm.set_console()
+        self.vm.launch()
+        self.wait_for_console_pattern('Entering emergency mode.')
+        prompt = '# '
+        self.wait_for_console_pattern(prompt)
+
+        # Copy a file (checked later), umount afterwards to drop disk cache:
+        exec_command_and_wait_for_pattern(self, 'mount /dev/vda1 /sysroot',
+                                          prompt)
+        filename = '/boot/initramfs-5.3.7-301.fc31.x86_64.img'
+        exec_command_and_wait_for_pattern(self, (f'cp /sysroot{filename}'
+                                                 ' /sysroot/root/data'),
+                                          prompt)
+        exec_command_and_wait_for_pattern(self, 'umount /sysroot', prompt)
+
+        # Switch from initrd to the cloud image filesystem:
+        exec_command_and_wait_for_pattern(self, 'mount /dev/vda1 /sysroot',
+                                          prompt)
+        exec_command_and_wait_for_pattern(self,
+                ('for d in dev proc sys run ; do '
+                 'mount -o bind /$d /sysroot/$d ; done'), prompt)
+        exec_command_and_wait_for_pattern(self, 'chroot /sysroot', prompt)
+
+        # Checking for IOMMU enablement:
+        self.log.info("Checking whether IOMMU has been enabled...")
+        exec_command_and_wait_for_pattern(self, 'cat /proc/cmdline',
+                                          'intel_iommu=on')
+        self.wait_for_console_pattern(prompt)
+        exec_command_and_wait_for_pattern(self, 'dmesg | grep DMAR:',
+                                          'IOMMU enabled')
+        self.wait_for_console_pattern(prompt)
+        exec_command_and_wait_for_pattern(self,
+                                    'find /sys/kernel/iommu_groups/ -type l',
+                                    'devices/0000:00:')
+        self.wait_for_console_pattern(prompt)
+
+        # Check hard disk device via sha256sum:
+        self.log.info("Checking hard disk...")
+        hashsum = '0dc7472f879be70b2f3daae279e3ae47175ffe249691e7d97f47222b65b8a720'
+        exec_command_and_wait_for_pattern(self, 'sha256sum ' + filename,
+                                          hashsum)
+        self.wait_for_console_pattern(prompt)
+        exec_command_and_wait_for_pattern(self, 'sha256sum /root/data',
+                                          hashsum)
+        self.wait_for_console_pattern(prompt)
+
+        # Check virtio-net via HTTP:
+        exec_command_and_wait_for_pattern(self, 'dhclient eth0', prompt)
+        exec_command_and_wait_for_pattern(self,
+                        f'python3 -m http.server {self.GUEST_PORT} & sleep 1',
+                        f'Serving HTTP on 0.0.0.0 port {self.GUEST_PORT}')
+        hl = hashlib.sha256()
+        hostport = get_usernet_hostfwd_port(self.vm)
+        url = f'http://localhost:{hostport}{filename}'
+        self.log.info(f'Downloading {url} ...')
+        with urllib.request.urlopen(url) as response:
+            while True:
+                chunk = response.read(1 << 20)
+                if not chunk:
+                    break
+                hl.update(chunk)
+
+        digest = hl.hexdigest()
+        self.log.info(f'sha256sum of download is {digest}.')
+        self.assertEqual(digest, hashsum)
 
     def test_intel_iommu(self):
-        """
-        :avocado: tags=intel_iommu_intremap
-        """
-
-        self.common_vm_setup(True)
+        self.common_vm_setup()
         self.vm.add_args('-device', 'intel-iommu,intremap=on')
         self.vm.add_args('-machine', 'kernel_irqchip=split')
-
-        self.kernel_params = (self.distro.default_kernel_params +
-                              ' quiet intel_iommu=on')
+        self.kernel_params += 'intel_iommu=on'
         self.run_and_check()
 
     def test_intel_iommu_strict(self):
-        """
-        :avocado: tags=intel_iommu_strict
-        """
-
-        self.common_vm_setup(True)
+        self.common_vm_setup()
         self.vm.add_args('-device', 'intel-iommu,intremap=on')
         self.vm.add_args('-machine', 'kernel_irqchip=split')
-        self.kernel_params = (self.distro.default_kernel_params +
-                              ' quiet intel_iommu=on,strict')
+        self.kernel_params += 'intel_iommu=on,strict'
         self.run_and_check()
 
     def test_intel_iommu_strict_cm(self):
-        """
-        :avocado: tags=intel_iommu_strict_cm
-        """
-
-        self.common_vm_setup(True)
+        self.common_vm_setup()
         self.vm.add_args('-device', 'intel-iommu,intremap=on,caching-mode=on')
         self.vm.add_args('-machine', 'kernel_irqchip=split')
-        self.kernel_params = (self.distro.default_kernel_params +
-                              ' quiet intel_iommu=on,strict')
+        self.kernel_params += 'intel_iommu=on,strict'
         self.run_and_check()
 
     def test_intel_iommu_pt(self):
-        """
-        :avocado: tags=intel_iommu_pt
-        """
-
-        self.common_vm_setup(True)
+        self.common_vm_setup()
         self.vm.add_args('-device', 'intel-iommu,intremap=on')
         self.vm.add_args('-machine', 'kernel_irqchip=split')
-        self.kernel_params = (self.distro.default_kernel_params +
-                              ' quiet intel_iommu=on iommu=pt')
+        self.kernel_params += 'intel_iommu=on iommu=pt'
         self.run_and_check()
+
+if __name__ == '__main__':
+    LinuxKernelTest.main()
-- 
2.47.1

Re: [PATCH v3 0/2] Convert the intel_iommu avocado test

[Michael S. Tsirkin]

On Tue, Dec 17, 2024 at 01:15:48PM +0100, Thomas Huth wrote:
> The first patch introduces a helper function for retrieving the
> hostfwd ports from QEMU.
> We then use this helper function to run a HTTP server in the guest
> in the second patch to exercise the network of the guest.

Good stuff

Acked-by: Michael S. Tsirkin <mst@redhat.com>


> v3:
> - Use the new hostfwd helper function instead of trying to probe
>   for an unused port on the host
> - Use a constant for the guest port 8080
> 
> Thomas Huth (2):
>   tests/functional: Add a helper function for retrieving the hostfwd
>     port
>   tests/functional: Convert the intel_iommu avocado test
> 
>  MAINTAINERS                                   |   1 +
>  tests/functional/meson.build                  |   2 +
>  tests/functional/qemu_test/utils.py           |   7 +
>  tests/functional/test_info_usernet.py         |   8 +-
>  .../test_intel_iommu.py}                      | 191 +++++++++++-------
>  5 files changed, 135 insertions(+), 74 deletions(-)
>  rename tests/{avocado/intel_iommu.py => functional/test_intel_iommu.py} (26%)
>  mode change 100644 => 100755
> 
> -- 
> 2.47.1