[PULL v4 0/4] more testing fixes (all green now)

[PULL v4 0/4] more testing fixes (all green now)

[Alex Bennée]

The following changes since commit aadac5b3d9fdce28030495f80fc76a4336e97328:

  Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging (2021-01-06 15:55:29 +0000)

are available in the Git repository at:

  https://github.com/stsquad/qemu.git tags/pull-testing-060121-4

for you to fetch changes up to af229fc367021e361cebaf84acceb01f28922cc4:

  cirrus: don't run full qtest on macOS (2021-01-06 17:30:02 +0000)

----------------------------------------------------------------
Testing updates (back to green)

  - include ccache in Debian 10 docker image
  - iotests: drop 312 from auto group
  - bound reading of s390x framebuffer file
  - cirrus: drop non-x86 tests so we complete

----------------------------------------------------------------
Alex Bennée (2):
      tests/iotests: drop test 312 from auto group
      tests/acceptance: bound the size of readline in s390_ccw_virtio

Daniel P. Berrangé (1):
      cirrus: don't run full qtest on macOS

Philippe Mathieu-Daudé (1):
      tests/docker: Include 'ccache' in Debian base image

 .cirrus.yml                                 | 14 ++++++++++----
 tests/acceptance/machine_s390_ccw_virtio.py |  2 +-
 tests/docker/dockerfiles/debian10.docker    |  1 +
 tests/qemu-iotests/group                    |  2 +-
 4 files changed, 13 insertions(+), 6 deletions(-)

-- 
2.20.1

[PULL v4 1/4] tests/docker: Include 'ccache' in Debian base image

[Alex Bennée]

From: Philippe Mathieu-Daudé <f4bug@amsat.org>

Include the 'ccache' package to speed up compilation.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201213211601.253530-1-f4bug@amsat.org>
Fixes: d6db2a1cdf ("docker: add debian-buster-arm64-cross")
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/docker/dockerfiles/debian10.docker b/tests/docker/dockerfiles/debian10.docker
index 73a3caac9c..9d42b5a4b8 100644
--- a/tests/docker/dockerfiles/debian10.docker
+++ b/tests/docker/dockerfiles/debian10.docker
@@ -20,6 +20,7 @@ RUN apt update && \
         bc \
         build-essential \
         ca-certificates \
+        ccache \
         clang \
         dbus \
         gdb-multiarch \
-- 
2.20.1

[PULL v4 2/4] tests/iotests: drop test 312 from auto group

[Alex Bennée]

The "auto" documentation states:

  That means they should run with every QEMU binary (also non-x86)

which is not the case as the check-system-fedora build which only
includes a rag tag group of rare and deprecated targets doesn't
support the virtio device required.

Fixes: ef9bba1484b ("quorum: Implement bdrv_co_block_status()")
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210105100402.12350-1-alex.bennee@linaro.org>

diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index e4fb6327ae..bc5bc324fe 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -318,4 +318,4 @@
 307 rw quick export
 308 rw
 309 rw auto quick
-312 rw auto quick
+312 rw quick
-- 
2.20.1

[PULL v4 3/4] tests/acceptance: bound the size of readline in s390_ccw_virtio

[Alex Bennée]

The read binary data as text via a PPM export of the frame buffer
seems a bit sketchy and it did blow up in the real world when the
assertion failed:

  https://gitlab.com/qemu-project/qemu/-/jobs/943183183

However short of cleaning up the test to be more binary focused at
least limit the attempt to dump the whole file as hexified zeros in
the logs.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Acked-by: Halil Pasic <pasic@linux.ibm.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210105124405.15424-1-alex.bennee@linaro.org>

diff --git a/tests/acceptance/machine_s390_ccw_virtio.py b/tests/acceptance/machine_s390_ccw_virtio.py
index 0f81af9950..eccf26b262 100644
--- a/tests/acceptance/machine_s390_ccw_virtio.py
+++ b/tests/acceptance/machine_s390_ccw_virtio.py
@@ -241,7 +241,7 @@ class S390CCWVirtioMachine(Test):
             self.assertEqual(line, b"1024 768\n")
             line = ppmfile.readline()
             self.assertEqual(line, b"255\n")
-            line = ppmfile.readline()
+            line = ppmfile.readline(256)
             self.assertEqual(line, b"The quick fox jumps over a lazy dog\n")
 
         # Hot-plug a virtio-crypto device and see whether it gets accepted
-- 
2.20.1

[PULL v4 4/4] cirrus: don't run full qtest on macOS

[Alex Bennée]

From: Daniel P. Berrangé <berrange@redhat.com>

The Cirrus CI macOS build hosts have exhibited a serious performance
degradation in recent months. For example the "qom-test" qtest takes
over an hour for only the qemu-system-aarch64 binary. This is as much
20-40 times slower than other environments. The other qtests all show
similar performance degradation, as do many of the unit tests.

This does not appear related to QEMU code changes, since older git
commits which were known to fully complete in less than 1 hour on
Cirrus CI now also show similar bad performance. Either Cirrus CI
performance has degraded, or an change in its environment has exposed
a latent bug widely affecting all of QEMU. Debugging the qom-test
showed no easily identified large bottleneck - every step of the
test execution was simply slower.

macOS builds/tests run outside Cirrus CI show normal performance.

With an inability to identify any obvious problem, the only viable
way to get a reliably completing Cirrus CI macOS job is to cut out
almost all of the qtests. We choose to run the x86_64 target only,
since that has very few machine types and thus is least badly
impacted in the qom-test execution.

With this change, the macOS jobs complete in approx 35 minutes.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210106114159.981538-1-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/.cirrus.yml b/.cirrus.yml
index 62a9b57530..3907e036da 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -18,7 +18,6 @@ freebsd_12_task:
     - gmake -j$(sysctl -n hw.ncpu) check V=1
 
 macos_task:
-  timeout_in: 90m
   osx_instance:
     image: catalina-base
   install_script:
@@ -30,10 +29,13 @@ macos_task:
                    --extra-cflags='-Wno-error=deprecated-declarations'
                    || { cat config.log meson-logs/meson-log.txt; exit 1; }
     - gmake -j$(sysctl -n hw.ncpu)
-    - gmake check V=1
+    - gmake check-unit V=1
+    - gmake check-block V=1
+    - gmake check-qapi-schema V=1
+    - gmake check-softfloat V=1
+    - gmake check-qtest-x86_64 V=1
 
 macos_xcode_task:
-  timeout_in: 90m
   osx_instance:
     # this is an alias for the latest Xcode
     image: catalina-xcode
@@ -45,7 +47,11 @@ macos_xcode_task:
     - ../configure --extra-cflags='-Wno-error=deprecated-declarations' --enable-modules
                    --enable-werror --cc=clang || { cat config.log meson-logs/meson-log.txt; exit 1; }
     - gmake -j$(sysctl -n hw.ncpu)
-    - gmake check V=1
+    - gmake check-unit V=1
+    - gmake check-block V=1
+    - gmake check-qapi-schema V=1
+    - gmake check-softfloat V=1
+    - gmake check-qtest-x86_64 V=1
 
 windows_msys2_task:
   timeout_in: 90m
-- 
2.20.1

Re: [PULL v4 0/4] more testing fixes (all green now)

[Peter Maydell]

On Wed, 6 Jan 2021 at 20:26, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> The following changes since commit aadac5b3d9fdce28030495f80fc76a4336e97328:
>
>   Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging (2021-01-06 15:55:29 +0000)
>
> are available in the Git repository at:
>
>   https://github.com/stsquad/qemu.git tags/pull-testing-060121-4
>
> for you to fetch changes up to af229fc367021e361cebaf84acceb01f28922cc4:
>
>   cirrus: don't run full qtest on macOS (2021-01-06 17:30:02 +0000)
>
> ----------------------------------------------------------------
> Testing updates (back to green)
>
>   - include ccache in Debian 10 docker image
>   - iotests: drop 312 from auto group
>   - bound reading of s390x framebuffer file
>   - cirrus: drop non-x86 tests so we complete


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/6.0
for any user-visible changes.

-- PMM