[Qemu-devel] [PATCH 0/4] tests/vm: misc fixes and optimizations

[Qemu-devel] [PATCH 0/4] tests/vm: misc fixes and optimizations

[Cleber Rosa]

A collection of simple fixes and misc optimizations I found
myself doing while investigating a console issue with the
vm tests.

Cleber Rosa (4):
  tests/vm: avoid extra compressed image copy
  tests/vm: avoid image presence check and removal
  tests/vm: pin ubuntu.i386 image
  tests/vm: add source repos on ubuntu.i386

 tests/vm/centos      | 6 ++----
 tests/vm/freebsd     | 6 ++----
 tests/vm/netbsd      | 6 ++----
 tests/vm/openbsd     | 6 ++----
 tests/vm/ubuntu.i386 | 7 ++++---
 5 files changed, 12 insertions(+), 19 deletions(-)

-- 
2.21.0

[Qemu-devel] [PATCH 1/4] tests/vm: avoid extra compressed image copy

[Cleber Rosa]

The image copy is only really needed because xz doesn't know to
properly decompress a file not named properly.  Instead of
decompressing to stdout, and having to rely on a shell, let's just
create a link instead of copying the file.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 tests/vm/centos  | 4 ++--
 tests/vm/freebsd | 4 ++--
 tests/vm/netbsd  | 4 ++--
 tests/vm/openbsd | 4 ++--
 4 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/tests/vm/centos b/tests/vm/centos
index ba133ea429..466fa59fec 100755
--- a/tests/vm/centos
+++ b/tests/vm/centos
@@ -66,8 +66,8 @@ class CentosVM(basevm.BaseVM):
         cimg = self._download_with_cache("https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-1802.qcow2.xz")
         img_tmp = img + ".tmp"
         sys.stderr.write("Extracting the image...\n")
-        subprocess.check_call(["cp", "-f", cimg, img_tmp + ".xz"])
-        subprocess.check_call(["xz", "-dvf", img_tmp + ".xz"])
+        subprocess.check_call(["ln", "-f", cimg, img_tmp + ".xz"])
+        subprocess.check_call(["xz", "--keep", "-dvf", img_tmp + ".xz"])
         subprocess.check_call(["qemu-img", "resize", img_tmp, "50G"])
         self.boot(img_tmp, extra_args = ["-cdrom", self._gen_cloud_init_iso()])
         self.wait_ssh()
diff --git a/tests/vm/freebsd b/tests/vm/freebsd
index b0066017a6..5575c23a6f 100755
--- a/tests/vm/freebsd
+++ b/tests/vm/freebsd
@@ -34,8 +34,8 @@ class FreeBSDVM(basevm.BaseVM):
         img_tmp_xz = img + ".tmp.xz"
         img_tmp = img + ".tmp"
         sys.stderr.write("Extracting the image...\n")
-        subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
-        subprocess.check_call(["xz", "-dvf", img_tmp_xz])
+        subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
+        subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
         if os.path.exists(img):
             os.remove(img)
         os.rename(img_tmp, img)
diff --git a/tests/vm/netbsd b/tests/vm/netbsd
index 4c6624ea5e..d0508f4465 100755
--- a/tests/vm/netbsd
+++ b/tests/vm/netbsd
@@ -34,8 +34,8 @@ class NetBSDVM(basevm.BaseVM):
         img_tmp_xz = img + ".tmp.xz"
         img_tmp = img + ".tmp"
         sys.stderr.write("Extracting the image...\n")
-        subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
-        subprocess.check_call(["xz", "-dvf", img_tmp_xz])
+        subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
+        subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
         if os.path.exists(img):
             os.remove(img)
         os.rename(img_tmp, img)
diff --git a/tests/vm/openbsd b/tests/vm/openbsd
index 2105c01a26..87ec982489 100755
--- a/tests/vm/openbsd
+++ b/tests/vm/openbsd
@@ -36,8 +36,8 @@ class OpenBSDVM(basevm.BaseVM):
         img_tmp_xz = img + ".tmp.xz"
         img_tmp = img + ".tmp"
         sys.stderr.write("Extracting the image...\n")
-        subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
-        subprocess.check_call(["xz", "-dvf", img_tmp_xz])
+        subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
+        subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
         if os.path.exists(img):
             os.remove(img)
         os.rename(img_tmp, img)
-- 
2.21.0

[Qemu-devel] [PATCH 2/4] tests/vm: avoid image presence check and removal

[Cleber Rosa]

Python's os.rename() will silently replace an existing file,
so there's no need for the extra check and removal.

Reference: https://docs.python.org/3/library/os.html#os.rename
Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 tests/vm/centos      | 2 --
 tests/vm/freebsd     | 2 --
 tests/vm/netbsd      | 2 --
 tests/vm/openbsd     | 2 --
 tests/vm/ubuntu.i386 | 2 --
 5 files changed, 10 deletions(-)

diff --git a/tests/vm/centos b/tests/vm/centos
index 466fa59fec..3ee7ca780e 100755
--- a/tests/vm/centos
+++ b/tests/vm/centos
@@ -77,8 +77,6 @@ class CentosVM(basevm.BaseVM):
         self.ssh_root_check("systemctl enable docker")
         self.ssh_root("poweroff")
         self.wait()
-        if os.path.exists(img):
-            os.remove(img)
         os.rename(img_tmp, img)
         return 0
 
diff --git a/tests/vm/freebsd b/tests/vm/freebsd
index 5575c23a6f..091be1a065 100755
--- a/tests/vm/freebsd
+++ b/tests/vm/freebsd
@@ -36,8 +36,6 @@ class FreeBSDVM(basevm.BaseVM):
         sys.stderr.write("Extracting the image...\n")
         subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
         subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
-        if os.path.exists(img):
-            os.remove(img)
         os.rename(img_tmp, img)
 
 if __name__ == "__main__":
diff --git a/tests/vm/netbsd b/tests/vm/netbsd
index d0508f4465..ee9eaeab50 100755
--- a/tests/vm/netbsd
+++ b/tests/vm/netbsd
@@ -36,8 +36,6 @@ class NetBSDVM(basevm.BaseVM):
         sys.stderr.write("Extracting the image...\n")
         subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
         subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
-        if os.path.exists(img):
-            os.remove(img)
         os.rename(img_tmp, img)
 
 if __name__ == "__main__":
diff --git a/tests/vm/openbsd b/tests/vm/openbsd
index 87ec982489..28c7d25e29 100755
--- a/tests/vm/openbsd
+++ b/tests/vm/openbsd
@@ -38,8 +38,6 @@ class OpenBSDVM(basevm.BaseVM):
         sys.stderr.write("Extracting the image...\n")
         subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
         subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
-        if os.path.exists(img):
-            os.remove(img)
         os.rename(img_tmp, img)
 
 if __name__ == "__main__":
diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
index a22d137e76..12867b193f 100755
--- a/tests/vm/ubuntu.i386
+++ b/tests/vm/ubuntu.i386
@@ -80,8 +80,6 @@ class UbuntuX86VM(basevm.BaseVM):
         self.ssh_root_check("apt-get install -y libfdt-dev flex bison")
         self.ssh_root("poweroff")
         self.wait()
-        if os.path.exists(img):
-            os.remove(img)
         os.rename(img_tmp, img)
         return 0
 
-- 
2.21.0

[Qemu-devel] [PATCH 3/4] tests/vm: pin ubuntu.i386 image

[Cleber Rosa]

It's a good practice to always have the same components used in tests.
According to:

   https://cloud-images.ubuntu.com/releases/16.04/

New images are released from time to time, and the "release/"
directory points to the latest release.  Let's pin to the latest
available version, and while at it, set a hash for verification.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 tests/vm/ubuntu.i386 | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
index 12867b193f..7017e6d388 100755
--- a/tests/vm/ubuntu.i386
+++ b/tests/vm/ubuntu.i386
@@ -61,7 +61,9 @@ class UbuntuX86VM(basevm.BaseVM):
         return os.path.join(cidir, "cloud-init.iso")
 
     def build_image(self, img):
-        cimg = self._download_with_cache("https://cloud-images.ubuntu.com/releases/16.04/release/ubuntu-16.04-server-cloudimg-i386-disk1.img")
+        cimg = self._download_with_cache(
+            "https://cloud-images.ubuntu.com/releases/16.04/release-20190605/ubuntu-16.04-server-cloudimg-i386-disk1.img",
+            sha256sum="e30091144c73483822b7c27193e9d47346dd1064229da577c3fedcf943f7cfcc")
         img_tmp = img + ".tmp"
         subprocess.check_call(["cp", "-f", cimg, img_tmp])
         subprocess.check_call(["qemu-img", "resize", img_tmp, "50G"])
-- 
2.21.0

[Qemu-devel] [PATCH 4/4] tests/vm: add source repos on ubuntu.i386

[Cleber Rosa]

Possibly because of different behavior on the newly update
cloud-image, trying to run 'apt-get build-dep' results in:

   E: You must put some 'source' URIs in your sources.list

This enables all source repos (even though some are not
needed) for simplicity sake.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 tests/vm/ubuntu.i386 | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
index 7017e6d388..3ea459ee20 100755
--- a/tests/vm/ubuntu.i386
+++ b/tests/vm/ubuntu.i386
@@ -77,6 +77,7 @@ class UbuntuX86VM(basevm.BaseVM):
         time.sleep(5)
         self.wait_ssh()
         # The previous update sometimes doesn't survive a reboot, so do it again
+        self.ssh_root_check("sed -ie s/^#\ deb-src/deb-src/g /etc/apt/sources.list")
         self.ssh_root_check("apt-get update")
         self.ssh_root_check("apt-get build-dep -y qemu")
         self.ssh_root_check("apt-get install -y libfdt-dev flex bison")
-- 
2.21.0

Re: [Qemu-devel] [PATCH 0/4] tests/vm: misc fixes and optimizations

[Alex Bennée]

Cleber Rosa <crosa@redhat.com> writes:

> A collection of simple fixes and misc optimizations I found
> myself doing while investigating a console issue with the
> vm tests.

Queued to testing/next, thanks.

>
> Cleber Rosa (4):
>   tests/vm: avoid extra compressed image copy
>   tests/vm: avoid image presence check and removal
>   tests/vm: pin ubuntu.i386 image
>   tests/vm: add source repos on ubuntu.i386
>
>  tests/vm/centos      | 6 ++----
>  tests/vm/freebsd     | 6 ++----
>  tests/vm/netbsd      | 6 ++----
>  tests/vm/openbsd     | 6 ++----
>  tests/vm/ubuntu.i386 | 7 ++++---
>  5 files changed, 12 insertions(+), 19 deletions(-)


--
Alex Bennée