[Qemu-devel] [PATCH 0/5] tests/vm: various minor improvements

[Qemu-devel] [PATCH 0/5] tests/vm: various minor improvements

[Peter Maydell]

The setup I had for my BSD VMs broke today, so I thought I'd
have a look at using the tests/vm/ support for building QEMU
inside BSD VMs rather than re-rolling my own. This patchset
fixes some rough edges I ran into along the way:
 * fixes support for non-KVM accelerators
 * propagates J=n setting into 'make check' in the VM
 * propagates V=1 setting into 'make' and 'make check' in the VM
 * uses --output-sync so that J=n settings don't mangle output
 * bumps the RAM settings so J=n doesn't cause the guest to
   run out of memory and kill the compiler

Ideally I'd like to be able to separately invoke commands
for "build in VM" and "run make check in VM for that build"; for
instance I only want V=1 on the make-check, not the make, and in
some cases might want to do a build but not check, and so on.
But that seemed too complicated for the moment.

Is it possible for the VMs to be persistent? Currently it
looks like they throw away their contents and start afresh
for every invocation. Doing build tests as incremental is
faster than doing every one as a from-clean...

Also, I notice that the OpenBSD VM setup is using SDL 1.2,
which means that configure complains and suggests switching
to SDL 2.0. How do we update the base image?

thanks
-- PMM

Peter Maydell (5):
  tests/vm: Use -cpu max rather than -cpu host
  tests/vm: Pass the jobs parallelism setting to 'make check'
  tests/vm: Propagate V=1 down into the make inside the VM
  tests/vm: Bump guest RAM up from 2G to 4G
  tests/vm: Use make's --output-sync option

 tests/vm/Makefile.include | 1 +
 docs/devel/testing.rst    | 1 +
 tests/vm/basevm.py        | 9 ++++++---
 tests/vm/freebsd          | 4 ++--
 tests/vm/netbsd           | 4 ++--
 tests/vm/openbsd          | 4 ++--
 tests/vm/ubuntu.i386      | 4 ++--
 7 files changed, 16 insertions(+), 11 deletions(-)

-- 
2.17.1

[Qemu-devel] [PATCH 1/5] tests/vm: Use -cpu max rather than -cpu host

[Peter Maydell]

Currently attempting to run the vm-build-* tests on a system
where the user does not have access to KVM will fail, because
although they avoid using -enable-kvm, they use "-cpu host",
which only works with KVM. Switch to "-cpu max" instead, which
works with any accelerator.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 tests/vm/basevm.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 36431178161..ae9ab128c7b 100755
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -65,7 +65,7 @@ class BaseVM(object):
             self._stdout = self._devnull
         self._args = [ \
             "-nodefaults", "-m", "2G",
-            "-cpu", "host",
+            "-cpu", "max",
             "-netdev", "user,id=vnet,hostfwd=:127.0.0.1:0-:22",
             "-device", "virtio-net-pci,netdev=vnet",
             "-vnc", "127.0.0.1:0,to=20",
-- 
2.17.1

[Qemu-devel] [PATCH 2/5] tests/vm: Pass the jobs parallelism setting to 'make check'

[Peter Maydell]

Our test suite works for parallel execution too, and this can
noticeably speed up a test run; pass the 'jobs' setting to
it as well as to the build proper.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 tests/vm/freebsd     | 2 +-
 tests/vm/netbsd      | 2 +-
 tests/vm/openbsd     | 2 +-
 tests/vm/ubuntu.i386 | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/tests/vm/freebsd b/tests/vm/freebsd
index 039dad8f693..2187a173274 100755
--- a/tests/vm/freebsd
+++ b/tests/vm/freebsd
@@ -24,7 +24,7 @@ class FreeBSDVM(basevm.BaseVM):
         tar -xf /dev/vtbd1;
         ./configure {configure_opts};
         gmake -j{jobs};
-        gmake check;
+        gmake -j{jobs} check;
     """
 
     def build_image(self, img):
diff --git a/tests/vm/netbsd b/tests/vm/netbsd
index 3972d8b45cd..2cc4798f0c4 100755
--- a/tests/vm/netbsd
+++ b/tests/vm/netbsd
@@ -24,7 +24,7 @@ class NetBSDVM(basevm.BaseVM):
         tar -xf /dev/rld1a;
         ./configure --python=python2.7 {configure_opts};
         gmake -j{jobs};
-        gmake check;
+        gmake -j{jobs} check;
     """
 
     def build_image(self, img):
diff --git a/tests/vm/openbsd b/tests/vm/openbsd
index 6ae16d97fdf..df6b79fe7dc 100755
--- a/tests/vm/openbsd
+++ b/tests/vm/openbsd
@@ -25,7 +25,7 @@ class OpenBSDVM(basevm.BaseVM):
         ./configure --cc=x86_64-unknown-openbsd6.1-gcc-4.9.4 --python=python2.7 {configure_opts};
         gmake -j{jobs};
         # XXX: "gmake check" seems to always hang or fail
-        #gmake check;
+        #gmake -j{jobs} check;
     """
 
     def build_image(self, img):
diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
index fc319e0e6eb..059d295865c 100755
--- a/tests/vm/ubuntu.i386
+++ b/tests/vm/ubuntu.i386
@@ -26,7 +26,7 @@ class UbuntuX86VM(basevm.BaseVM):
         tar -xf /dev/vdb;
         ./configure {configure_opts};
         make -j{jobs};
-        make check;
+        make -j{jobs} check;
     """
 
     def _gen_cloud_init_iso(self):
-- 
2.17.1

[Qemu-devel] [PATCH 3/5] tests/vm: Propagate V=1 down into the make inside the VM

[Peter Maydell]

Invoking 'make vm-build-freebsd' and friends with V=1 should
propagate that verbosity setting down into the build run
inside the VM. Make sure we do that. This brings it into
line with how the container tests handle V=1.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 tests/vm/Makefile.include | 1 +
 docs/devel/testing.rst    | 1 +
 tests/vm/basevm.py        | 5 ++++-
 tests/vm/freebsd          | 4 ++--
 tests/vm/netbsd           | 4 ++--
 tests/vm/openbsd          | 4 ++--
 tests/vm/ubuntu.i386      | 2 +-
 7 files changed, 13 insertions(+), 8 deletions(-)

diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
index 5daa2a3b732..2efc27961df 100644
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -36,6 +36,7 @@ vm-build-%: tests/vm/%.img
 		$(if $(V)$(DEBUG), --debug) \
 		$(if $(DEBUG), --interactive) \
 		$(if $(J),--jobs $(J)) \
+		$(if $(V),--verbose) \
 		--image "$<" \
 		--build-qemu $(SRC_PATH), \
 		"  VM-BUILD $*")
diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 5e19cd50da8..56eed22a78b 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -423,6 +423,7 @@ Debugging
 
 Add ``DEBUG=1`` and/or ``V=1`` to the make command to allow interactive
 debugging and verbose output. If this is not enough, see the next section.
+``V=1`` will be propagated down into the make jobs in the guest.
 
 Manual invocation
 -----------------
diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index ae9ab128c7b..bf8b1bcfc5e 100755
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -210,6 +210,8 @@ def parse_args(vm_name):
                       help="force build image even if image exists")
     parser.add_option("--jobs", type=int, default=multiprocessing.cpu_count() / 2,
                       help="number of virtual CPUs")
+    parser.add_option("--verbose", "-V", action="store_true",
+                      help="Pass V=1 to builds within the guest")
     parser.add_option("--build-image", "-b", action="store_true",
                       help="build image")
     parser.add_option("--build-qemu",
@@ -238,7 +240,8 @@ def main(vmcls):
             vm.add_source_dir(args.build_qemu)
             cmd = [vm.BUILD_SCRIPT.format(
                    configure_opts = " ".join(argv),
-                   jobs=args.jobs)]
+                   jobs=args.jobs,
+                   verbose = "V=1" if args.verbose else "")]
         else:
             cmd = argv
         vm.boot(args.image + ",snapshot=on")
diff --git a/tests/vm/freebsd b/tests/vm/freebsd
index 2187a173274..795f739c7b3 100755
--- a/tests/vm/freebsd
+++ b/tests/vm/freebsd
@@ -23,8 +23,8 @@ class FreeBSDVM(basevm.BaseVM):
         cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
         tar -xf /dev/vtbd1;
         ./configure {configure_opts};
-        gmake -j{jobs};
-        gmake -j{jobs} check;
+        gmake -j{jobs} {verbose};
+        gmake -j{jobs} check {verbose};
     """
 
     def build_image(self, img):
diff --git a/tests/vm/netbsd b/tests/vm/netbsd
index 2cc4798f0c4..c211672bcbd 100755
--- a/tests/vm/netbsd
+++ b/tests/vm/netbsd
@@ -23,8 +23,8 @@ class NetBSDVM(basevm.BaseVM):
         cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
         tar -xf /dev/rld1a;
         ./configure --python=python2.7 {configure_opts};
-        gmake -j{jobs};
-        gmake -j{jobs} check;
+        gmake -j{jobs} {verbose};
+        gmake -j{jobs} check {verbose};
     """
 
     def build_image(self, img):
diff --git a/tests/vm/openbsd b/tests/vm/openbsd
index df6b79fe7dc..1e0c2500ad9 100755
--- a/tests/vm/openbsd
+++ b/tests/vm/openbsd
@@ -23,9 +23,9 @@ class OpenBSDVM(basevm.BaseVM):
         cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
         tar -xf /dev/rsd1c;
         ./configure --cc=x86_64-unknown-openbsd6.1-gcc-4.9.4 --python=python2.7 {configure_opts};
-        gmake -j{jobs};
+        gmake -j{jobs} {verbose};
         # XXX: "gmake check" seems to always hang or fail
-        #gmake -j{jobs} check;
+        #gmake -j{jobs} check {verbose};
     """
 
     def build_image(self, img):
diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
index 059d295865c..7ea25327071 100755
--- a/tests/vm/ubuntu.i386
+++ b/tests/vm/ubuntu.i386
@@ -26,7 +26,7 @@ class UbuntuX86VM(basevm.BaseVM):
         tar -xf /dev/vdb;
         ./configure {configure_opts};
         make -j{jobs};
-        make -j{jobs} check;
+        make check -j{jobs} {verbose};
     """
 
     def _gen_cloud_init_iso(self):
-- 
2.17.1

[Qemu-devel] [PATCH 4/5] tests/vm: Bump guest RAM up from 2G to 4G

[Peter Maydell]

Currently we run the guests in a VM which is given only 2G of RAM.
Since the guests are configured without any swap space, builds
can fail because the system runs out of memory and kills the
compiler, especially if the job count is set for a lot of
parallelism. Bump the setting up from 2G to 4G to give us some
more headroom.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
Specifically I found the NetBSD build was falling over
with a J=16 level of parallelism.
---
 tests/vm/basevm.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index bf8b1bcfc5e..879229379d8 100755
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -64,7 +64,7 @@ class BaseVM(object):
         else:
             self._stdout = self._devnull
         self._args = [ \
-            "-nodefaults", "-m", "2G",
+            "-nodefaults", "-m", "4G",
             "-cpu", "max",
             "-netdev", "user,id=vnet,hostfwd=:127.0.0.1:0-:22",
             "-device", "virtio-net-pci,netdev=vnet",
-- 
2.17.1

[Qemu-devel] [PATCH 5/5] tests/vm: Use make's --output-sync option

[Peter Maydell]

Use make's --output-sync option when running tests inside VMs,
so that if we're building with parallelization the output doesn't
get scrambled.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 tests/vm/freebsd     | 4 ++--
 tests/vm/netbsd      | 4 ++--
 tests/vm/openbsd     | 4 ++--
 tests/vm/ubuntu.i386 | 4 ++--
 4 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/tests/vm/freebsd b/tests/vm/freebsd
index 795f739c7b3..0a6ec4614a7 100755
--- a/tests/vm/freebsd
+++ b/tests/vm/freebsd
@@ -23,8 +23,8 @@ class FreeBSDVM(basevm.BaseVM):
         cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
         tar -xf /dev/vtbd1;
         ./configure {configure_opts};
-        gmake -j{jobs} {verbose};
-        gmake -j{jobs} check {verbose};
+        gmake --output-sync -j{jobs} {verbose};
+        gmake --output-sync -j{jobs} check {verbose};
     """
 
     def build_image(self, img):
diff --git a/tests/vm/netbsd b/tests/vm/netbsd
index c211672bcbd..45c9260dc0d 100755
--- a/tests/vm/netbsd
+++ b/tests/vm/netbsd
@@ -23,8 +23,8 @@ class NetBSDVM(basevm.BaseVM):
         cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
         tar -xf /dev/rld1a;
         ./configure --python=python2.7 {configure_opts};
-        gmake -j{jobs} {verbose};
-        gmake -j{jobs} check {verbose};
+        gmake --output-sync -j{jobs} {verbose};
+        gmake --output-sync -j{jobs} check {verbose};
     """
 
     def build_image(self, img):
diff --git a/tests/vm/openbsd b/tests/vm/openbsd
index 1e0c2500ad9..98edfbca4ba 100755
--- a/tests/vm/openbsd
+++ b/tests/vm/openbsd
@@ -23,9 +23,9 @@ class OpenBSDVM(basevm.BaseVM):
         cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
         tar -xf /dev/rsd1c;
         ./configure --cc=x86_64-unknown-openbsd6.1-gcc-4.9.4 --python=python2.7 {configure_opts};
-        gmake -j{jobs} {verbose};
+        gmake --output-sync -j{jobs} {verbose};
         # XXX: "gmake check" seems to always hang or fail
-        #gmake -j{jobs} check {verbose};
+        #gmake --output-sync -j{jobs} check {verbose};
     """
 
     def build_image(self, img):
diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
index 7ea25327071..54c25d5727a 100755
--- a/tests/vm/ubuntu.i386
+++ b/tests/vm/ubuntu.i386
@@ -25,8 +25,8 @@ class UbuntuX86VM(basevm.BaseVM):
         sudo chmod a+r /dev/vdb;
         tar -xf /dev/vdb;
         ./configure {configure_opts};
-        make -j{jobs};
-        make check -j{jobs} {verbose};
+        make --output-sync -j{jobs};
+        make --output-sync check -j{jobs} {verbose};
     """
 
     def _gen_cloud_init_iso(self):
-- 
2.17.1

Re: [Qemu-devel] [PATCH 0/5] tests/vm: various minor improvements

[Fam Zheng]

On Fri, 08/03 09:52, Peter Maydell wrote:
> The setup I had for my BSD VMs broke today, so I thought I'd
> have a look at using the tests/vm/ support for building QEMU
> inside BSD VMs rather than re-rolling my own. This patchset
> fixes some rough edges I ran into along the way:
>  * fixes support for non-KVM accelerators
>  * propagates J=n setting into 'make check' in the VM
>  * propagates V=1 setting into 'make' and 'make check' in the VM
>  * uses --output-sync so that J=n settings don't mangle output
>  * bumps the RAM settings so J=n doesn't cause the guest to
>    run out of memory and kill the compiler
> 
> Ideally I'd like to be able to separately invoke commands
> for "build in VM" and "run make check in VM for that build"; for
> instance I only want V=1 on the make-check, not the make, and in
> some cases might want to do a build but not check, and so on.
> But that seemed too complicated for the moment.

If we can limit the abstraction to "build" and "check" (or "test"), it shouldn't
be hard to add a --build-only option to the Python main() and VM_BUILD_ONLY=1 to
Makefile.

> 
> Is it possible for the VMs to be persistent? Currently it
> looks like they throw away their contents and start afresh
> for every invocation. Doing build tests as incremental is
> faster than doing every one as a from-clean...

Yes, we'll do that in 3.1.

> 
> Also, I notice that the OpenBSD VM setup is using SDL 1.2,
> which means that configure complains and suggests switching
> to SDL 2.0. How do we update the base image?

I prepared the image by hand and uploaded it because I couldn't find a better
way. So the easist way to update is probably booting the image by hand, do the
modifications, then upload it again.

> 
> thanks
> -- PMM
> 
> Peter Maydell (5):
>   tests/vm: Use -cpu max rather than -cpu host
>   tests/vm: Pass the jobs parallelism setting to 'make check'
>   tests/vm: Propagate V=1 down into the make inside the VM
>   tests/vm: Bump guest RAM up from 2G to 4G
>   tests/vm: Use make's --output-sync option
> 
>  tests/vm/Makefile.include | 1 +
>  docs/devel/testing.rst    | 1 +
>  tests/vm/basevm.py        | 9 ++++++---
>  tests/vm/freebsd          | 4 ++--
>  tests/vm/netbsd           | 4 ++--
>  tests/vm/openbsd          | 4 ++--
>  tests/vm/ubuntu.i386      | 4 ++--
>  7 files changed, 16 insertions(+), 11 deletions(-)
> 
> -- 
> 2.17.1
> 
> 

Re: [Qemu-devel] [PATCH 0/5] tests/vm: various minor improvements

[Fam Zheng]

On Fri, 08/03 09:52, Peter Maydell wrote:
> The setup I had for my BSD VMs broke today, so I thought I'd
> have a look at using the tests/vm/ support for building QEMU
> inside BSD VMs rather than re-rolling my own. This patchset
> fixes some rough edges I ran into along the way:
>  * fixes support for non-KVM accelerators
>  * propagates J=n setting into 'make check' in the VM
>  * propagates V=1 setting into 'make' and 'make check' in the VM
>  * uses --output-sync so that J=n settings don't mangle output
>  * bumps the RAM settings so J=n doesn't cause the guest to
>    run out of memory and kill the compiler

Queued for 3.1. Thanks!

Fam