[Qemu-devel] [PULL 00/21] misc testing fixes for Travis and docker

[Qemu-devel] [PULL 00/21] misc testing fixes for Travis and docker

[Alex Bennée]

The following changes since commit 7260438b7056469610ee166f7abe9ff8a26b8b16:

  Merge remote-tracking branch 'remotes/palmer/tags/riscv-for-master-3.2-part2' into staging (2019-01-14 11:41:43 +0000)

are available in the Git repository at:

  https://github.com/stsquad/qemu.git tags/pull-testing-next-140119-1

for you to fetch changes up to a36270a4d1589b1ed309065fc8b3fe0ac8d6869d:

  Revert "tests: Disable qht-bench parallel test when using gprof" (2019-01-14 14:55:32 +0000)

----------------------------------------------------------------
A bunch of fixes for testing:

  - Various Travis updates
  - "stable" SID snapshot for docker
  - avoid :latest docker tags
  - g_usleep fix for some tests

----------------------------------------------------------------
Alex Bennée (7):
      tests: use in-place sed magic for enabling deb-src in travis image
      travis: use homebrew addon for MacOSX
      travis: bump to Xenial baseline
      tests/docker: update our Travis image
      tests/docker: remove SID_AGE test hack
      tests: use g_usleep instead of rem = sleep(time)
      Revert "tests: Disable qht-bench parallel test when using gprof"

Daniel P. Berrangé (12):
      tests: run ldconfig after installing extra software
      tests: remove obsolete 'debian' dockerfile
      tests: update Fedora dockerfile to use Fedora 29
      tests: update Fedora i386 cross image to Fedora 29
      travis: add whitespace between each major section & matrix entry
      travis: define all the build matrix entries in one place
      travis: standardize the syntax used for env variables
      travis: don't clone git submodules upfront
      travis: stop redefining the script commands
      travis: stop using container based envs
      travis: run tests in verbose mode
      travis: remove matrix settings that duplicate global settings

Philippe Mathieu-Daudé (2):
      docker: Use stable git tag for virglrenderer
      docker: Use a stable snapshot for Debian Sid

 .travis.yml                                       | 172 +++++++++++++++-------
 configure                                         |   1 -
 tests/Makefile.include                            |   3 +-
 tests/atomic64-bench.c                            |   6 +-
 tests/atomic_add-bench.c                          |   6 +-
 tests/docker/Makefile.include                     |  13 --
 tests/docker/dockerfiles/debian-amd64.docker      |   5 +-
 tests/docker/dockerfiles/debian-sid.docker        |   7 +-
 tests/docker/dockerfiles/debian.docker            |  13 --
 tests/docker/dockerfiles/fedora-i386-cross.docker |   2 +-
 tests/docker/dockerfiles/fedora.docker            |   4 +-
 tests/docker/dockerfiles/travis.docker            |   4 +-
 tests/qht-bench.c                                 |   6 +-
 13 files changed, 138 insertions(+), 104 deletions(-)
 delete mode 100644 tests/docker/dockerfiles/debian.docker

--
2.17.1

[Qemu-devel] [PULL 01/21] docker: Use stable git tag for virglrenderer

[Alex Bennée]

From: Philippe Mathieu-Daudé <philmd@redhat.com>

Use a stable tag instead of some random commit from mainstream
development, to avoid unexpected build failures.

This fixes:

    CC       virglrenderer.lo
  virglrenderer.c: In function 'virgl_has_gl_colorspace':
  virglrenderer.c:208:11: error: implicit declaration of function 'virgl_has_egl_khr_gl_colorspace' [-Werror=implicit-function-declaration]
             virgl_has_egl_khr_gl_colorspace(egl_info));
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  virglrenderer.c:208:43: error: 'egl_info' undeclared (first use in this function)
             virgl_has_egl_khr_gl_colorspace(egl_info));
                                             ^~~~~~~~
  virglrenderer.c:208:43: note: each undeclared identifier is reported only once for each function it appears in
  cc1: some warnings being treated as errors

As of this commit 'git virglrenderer-0.7.0' is the last stable tag.
(virglrenderer commit breaking: fb4f7577f7ef)

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/docker/dockerfiles/debian-amd64.docker b/tests/docker/dockerfiles/debian-amd64.docker
index 24b113b76f..47a30adbdb 100644
--- a/tests/docker/dockerfiles/debian-amd64.docker
+++ b/tests/docker/dockerfiles/debian-amd64.docker
@@ -24,7 +24,8 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
         libegl1-mesa-dev \
         libepoxy-dev \
         libgbm-dev
-RUN git clone https://anongit.freedesktop.org/git/virglrenderer.git /usr/src/virglrenderer
+RUN git clone https://anongit.freedesktop.org/git/virglrenderer.git /usr/src/virglrenderer && \
+    cd /usr/src/virglrenderer && git checkout virglrenderer-0.7.0
 RUN cd /usr/src/virglrenderer && ./autogen.sh && ./configure --with-glx --disable-tests && make install
 
 # netmap
-- 
2.17.1

[Qemu-devel] [PULL 02/21] tests: run ldconfig after installing extra software

[Alex Bennée]

From: Daniel P. Berrangé <berrange@redhat.com>

The docker file builds and installs software into /usr/local but does
not run ldconfig. As a result QEMU links to libvirglrenderer.so, but
then crashes in "make check" unable to find the library.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

diff --git a/tests/docker/dockerfiles/debian-amd64.docker b/tests/docker/dockerfiles/debian-amd64.docker
index 47a30adbdb..954fcf9606 100644
--- a/tests/docker/dockerfiles/debian-amd64.docker
+++ b/tests/docker/dockerfiles/debian-amd64.docker
@@ -36,5 +36,7 @@ RUN git clone https://github.com/luigirizzo/netmap.git /usr/src/netmap
 RUN cd /usr/src/netmap/LINUX && ./configure --no-drivers --no-apps --kernel-dir=$(ls -d /usr/src/linux-headers-*-amd64) && make install
 ENV QEMU_CONFIGURE_OPTS --enable-netmap
 
+RUN ldconfig
+
 # gcrypt
 ENV QEMU_CONFIGURE_OPTS $QEMU_CONFIGURE_OPTS --enable-gcrypt
-- 
2.17.1

[Qemu-devel] [PULL 03/21] tests: remove obsolete 'debian' dockerfile

[Alex Bennée]

From: Daniel P. Berrangé <berrange@redhat.com>

The 'debian' dockerfile was deprecated in favour of versioned
dockerfiles in July 2017. That is enough time for developers to
be warned about the rename.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

diff --git a/tests/docker/dockerfiles/debian.docker b/tests/docker/dockerfiles/debian.docker
deleted file mode 100644
index fd32e71b79..0000000000
--- a/tests/docker/dockerfiles/debian.docker
+++ /dev/null
@@ -1,13 +0,0 @@
-# This template is deprecated and was previously based on Jessie on QEMU 2.9.
-# Now than Stretch is out, please use qemu:debian8 as base for Jessie,
-# and qemu:debian9 for Stretch.
-#
-FROM qemu:debian9
-
-MAINTAINER Philippe Mathieu-Daudé <f4bug@amsat.org>
-
-RUN for n in $(seq 8); do echo; done && \
-    echo "\n\t\tThis image is deprecated." && echo && \
-    echo "\tUse 'FROM qemu:debian9' to use the stable Debian Stretch image" && \
-    echo "\tor 'FROM qemu:debian8' to use old Debian Jessie." && \
-    for n in $(seq 8); do echo; done
-- 
2.17.1

[Qemu-devel] [PULL 04/21] tests: update Fedora dockerfile to use Fedora 29

[Alex Bennée]

From: Daniel P. Berrangé <berrange@redhat.com>

Fedora 29 is the current newest release, so switch to using that
from the current Fedora 28.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker
index 1d0e3dc4ec..69d4a7f5d7 100644
--- a/tests/docker/dockerfiles/fedora.docker
+++ b/tests/docker/dockerfiles/fedora.docker
@@ -1,4 +1,4 @@
-FROM fedora:28
+FROM fedora:29
 ENV PACKAGES \
     bc \
     bison \
@@ -82,7 +82,7 @@ ENV PACKAGES \
     tar \
     usbredir-devel \
     virglrenderer-devel \
-    vte3-devel \
+    vte291-devel \
     which \
     xen-devel \
     zlib-devel
-- 
2.17.1

[Qemu-devel] [PULL 05/21] tests: update Fedora i386 cross image to Fedora 29

[Alex Bennée]

From: Daniel P. Berrangé <berrange@redhat.com>

Using the "latest" tag is not a good idea because this changes what
release it points to every 6 months. Together with caching of docker
builds this can cause confusion where CI has cached & built with Fedora
N, while a developer tries to reproduce a CI problem with Fedora N + 1,
or vica-verca.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

diff --git a/tests/docker/dockerfiles/fedora-i386-cross.docker b/tests/docker/dockerfiles/fedora-i386-cross.docker
index a4fd895b07..eb8108d118 100644
--- a/tests/docker/dockerfiles/fedora-i386-cross.docker
+++ b/tests/docker/dockerfiles/fedora-i386-cross.docker
@@ -1,4 +1,4 @@
-FROM fedora:latest
+FROM fedora:29
 ENV PACKAGES \
     gcc \
     glib2-devel.i686 \
-- 
2.17.1

[Qemu-devel] [PULL 06/21] tests: use in-place sed magic for enabling deb-src in travis image

[Alex Bennée]

This avoids potential problems with duplicates.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

diff --git a/tests/docker/dockerfiles/travis.docker b/tests/docker/dockerfiles/travis.docker
index 03ebfb0ef2..46cf367488 100644
--- a/tests/docker/dockerfiles/travis.docker
+++ b/tests/docker/dockerfiles/travis.docker
@@ -2,7 +2,7 @@ FROM travisci/ci-garnet:packer-1512502276-986baf0
 ENV DEBIAN_FRONTEND noninteractive
 ENV LANG en_US.UTF-8
 ENV LC_ALL en_US.UTF-8
-RUN cat /etc/apt/sources.list | sed "s/# deb-src/deb-src/" >> /etc/apt/sources.list
+RUN sed -i "s/# deb-src/deb-src/" /etc/apt/sources.list
 RUN apt-get update
 RUN apt-get -y build-dep qemu
 RUN apt-get -y install device-tree-compiler python2.7 python-yaml dh-autoreconf gdb strace lsof net-tools gcovr
-- 
2.17.1

[Qemu-devel] [PULL 07/21] travis: add whitespace between each major section & matrix entry

[Alex Bennée]

From: Daniel P. Berrangé <berrange@redhat.com>

Improve the readability of the travis config by adding two blank lines
between each major section and matrix entry.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/.travis.yml b/.travis.yml
index d472fd650b..bb43792627 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,6 +1,7 @@
 # The current Travis default is a container based 14.04 Trust on EC2
 # Additional builds with specific requirements for a full VM need to
 # be added as additional matrix: entries later on
+
 sudo: false
 dist: trusty
 language: c
@@ -9,6 +10,8 @@ python:
 compiler:
   - gcc
 cache: ccache
+
+
 addons:
   apt:
     packages:
@@ -40,6 +43,7 @@ addons:
       - uuid-dev
       - gcovr
 
+
 # The channel name "irc.oftc.net#qemu" is encrypted against qemu/qemu
 # to prevent IRC notifications from forks. This was created using:
 # $ travis encrypt -r "qemu/qemu" "irc.oftc.net#qemu"
@@ -49,6 +53,8 @@ notifications:
       - secure: "F7GDRgjuOo5IUyRLqSkmDL7kvdU4UcH3Lm/W2db2JnDHTGCqgEdaYEYKciyCLZ57vOTsTsOgesN8iUT7hNHBd1KWKjZe9KDTZWppWRYVwAwQMzVeSOsbbU4tRoJ6Pp+3qhH1Z0eGYR9ZgKYAoTumDFgSAYRp4IscKS8jkoedOqM="
     on_success: change
     on_failure: always
+
+
 env:
   global:
     - SRC_DIR="."
@@ -63,9 +69,13 @@ env:
     - CONFIG="--enable-modules --disable-linux-user"
     - CONFIG="--with-coroutine=ucontext --disable-linux-user"
     - CONFIG="--with-coroutine=sigaltstack --disable-linux-user"
+
+
 git:
   # we want to do this ourselves
   submodules: false
+
+
 before_install:
   - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update ; fi
   - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
@@ -75,54 +85,82 @@ before_script:
   - ${SRC_DIR}/configure ${CONFIG} || { cat config.log && exit 1; }
 script:
   - make ${MAKEFLAGS} && ${TEST_CMD}
+
+
 matrix:
   include:
     # Test out-of-tree builds
     - env: CONFIG="--enable-debug --enable-debug-tcg"
            BUILD_DIR="out-of-tree/build/dir" SRC_DIR="../../.."
+
+
     # Test with Clang for compile portability (Travis uses clang-5.0)
     - env: CONFIG="--disable-system"
       compiler: clang
+
+
     - env: CONFIG="--disable-user"
       compiler: clang
+
+
     # gprof/gcov are GCC features
     - env: CONFIG="--enable-gprof --enable-gcov --disable-pie --target-list=aarch64-softmmu,arm-softmmu,i386-softmmu,mips-softmmu,mips64-softmmu,ppc64-softmmu,riscv64-softmmu,s390x-softmmu,x86_64-softmmu"
       after_success:
         - ${SRC_DIR}/scripts/travis/coverage-summary.sh
       compiler: gcc
+
+
     # We manually include builds which we disable "make check" for
     - env: CONFIG="--enable-debug --enable-tcg-interpreter"
            TEST_CMD=""
       compiler: gcc
+
+
     # We don't need to exercise every backend with every front-end
     - env: CONFIG="--enable-trace-backends=log,simple,syslog --disable-system"
            TEST_CMD=""
       compiler: gcc
+
+
     - env: CONFIG="--enable-trace-backends=ftrace --target-list=x86_64-softmmu"
            TEST_CMD=""
       compiler: gcc
+
+
     - env: CONFIG="--enable-trace-backends=ust --target-list=x86_64-softmmu"
            TEST_CMD=""
       compiler: gcc
+
+
     - env: CONFIG="--disable-tcg"
            TEST_CMD=""
       compiler: gcc
+
+
     # MacOSX builds
     - env: CONFIG="--target-list=aarch64-softmmu,arm-softmmu,i386-softmmu,mips-softmmu,mips64-softmmu,ppc64-softmmu,riscv64-softmmu,s390x-softmmu,x86_64-softmmu"
       os: osx
       osx_image: xcode9.4
       compiler: clang
+
+
     - env: CONFIG="--target-list=i386-softmmu,ppc-softmmu,ppc64-softmmu,m68k-softmmu,x86_64-softmmu"
       os: osx
       osx_image: xcode10
       compiler: clang
+
+
     # Python builds
     - env: CONFIG="--target-list=x86_64-softmmu"
       python:
         - "3.0"
+
+
     - env: CONFIG="--target-list=x86_64-softmmu"
       python:
         - "3.6"
+
+
     # Acceptance (Functional) tests
     - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
            TEST_CMD="make AVOCADO_SHOW=app check-acceptance"
@@ -131,6 +169,8 @@ matrix:
           packages:
             - python3-pip
             - python3.4-venv
+
+
     # Using newer GCC with sanitizers
     - addons:
         apt:
@@ -175,6 +215,8 @@ matrix:
         - TEST_CMD=""
       before_script:
         - ./configure ${CONFIG} --extra-cflags="-g3 -O0 -fsanitize=thread -fuse-ld=gold" || { cat config.log && exit 1; }
+
+
     - env:
         - CONFIG="--disable-system --disable-docs"
         - TEST_CMD="make check-tcg"
-- 
2.17.1

[Qemu-devel] [PULL 08/21] travis: define all the build matrix entries in one place

[Alex Bennée]

From: Daniel P. Berrangé <berrange@redhat.com>

The current build matrix is constructed from entries listed under the
environment variable config section, as well as the general purpose
build matrix section. Move everything under the general purpose section
so it is clear at a glance what is in the matrix.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/.travis.yml b/.travis.yml
index bb43792627..96917df9c5 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -61,14 +61,6 @@ env:
     - BUILD_DIR="."
     - TEST_CMD="make check"
     - MAKEFLAGS="-j3"
-  matrix:
-    - CONFIG="--disable-system"
-    - CONFIG="--disable-user"
-    - CONFIG="--enable-debug --enable-debug-tcg"
-    - CONFIG="--disable-linux-aio --disable-cap-ng --disable-attr --disable-brlapi --disable-uuid --disable-libusb --disable-user"
-    - CONFIG="--enable-modules --disable-linux-user"
-    - CONFIG="--with-coroutine=ucontext --disable-linux-user"
-    - CONFIG="--with-coroutine=sigaltstack --disable-linux-user"
 
 
 git:
@@ -89,6 +81,27 @@ script:
 
 matrix:
   include:
+    - env: CONFIG="--disable-system"
+
+
+    - env: CONFIG="--disable-user"
+
+
+    - env: CONFIG="--enable-debug --enable-debug-tcg"
+
+
+    - env: CONFIG="--disable-linux-aio --disable-cap-ng --disable-attr --disable-brlapi --disable-uuid --disable-libusb --disable-user"
+
+
+    - env: CONFIG="--enable-modules --disable-linux-user"
+
+
+    - env: CONFIG="--with-coroutine=ucontext --disable-linux-user"
+
+
+    - env: CONFIG="--with-coroutine=sigaltstack --disable-linux-user"
+
+
     # Test out-of-tree builds
     - env: CONFIG="--enable-debug --enable-debug-tcg"
            BUILD_DIR="out-of-tree/build/dir" SRC_DIR="../../.."
-- 
2.17.1

[Qemu-devel] [PULL 09/21] travis: standardize the syntax used for env variables

[Alex Bennée]

From: Daniel P. Berrangé <berrange@redhat.com>

Matrix entries are defining env variables using two different syntax
styles:

   - env: FOO=bar
          WIZZ=bang

and

   - env:
       - FOO=bar
       - WIZZ=bang

Switch everything to use the latter style as the more normal indentation
approach.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/.travis.yml b/.travis.yml
index 96917df9c5..f539f8b556 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -81,102 +81,123 @@ script:
 
 matrix:
   include:
-    - env: CONFIG="--disable-system"
+    - env:
+        - CONFIG="--disable-system"
 
 
-    - env: CONFIG="--disable-user"
+    - env:
+        - CONFIG="--disable-user"
 
 
-    - env: CONFIG="--enable-debug --enable-debug-tcg"
+    - env:
+        - CONFIG="--enable-debug --enable-debug-tcg"
 
 
-    - env: CONFIG="--disable-linux-aio --disable-cap-ng --disable-attr --disable-brlapi --disable-uuid --disable-libusb --disable-user"
+    - env:
+        - CONFIG="--disable-linux-aio --disable-cap-ng --disable-attr --disable-brlapi --disable-uuid --disable-libusb --disable-user"
 
 
-    - env: CONFIG="--enable-modules --disable-linux-user"
+    - env:
+        - CONFIG="--enable-modules --disable-linux-user"
 
 
-    - env: CONFIG="--with-coroutine=ucontext --disable-linux-user"
+    - env:
+        - CONFIG="--with-coroutine=ucontext --disable-linux-user"
 
 
-    - env: CONFIG="--with-coroutine=sigaltstack --disable-linux-user"
+    - env:
+        - CONFIG="--with-coroutine=sigaltstack --disable-linux-user"
 
 
     # Test out-of-tree builds
-    - env: CONFIG="--enable-debug --enable-debug-tcg"
-           BUILD_DIR="out-of-tree/build/dir" SRC_DIR="../../.."
+    - env:
+        - CONFIG="--enable-debug --enable-debug-tcg"
+        - BUILD_DIR="out-of-tree/build/dir" SRC_DIR="../../.."
 
 
     # Test with Clang for compile portability (Travis uses clang-5.0)
-    - env: CONFIG="--disable-system"
+    - env:
+        - CONFIG="--disable-system"
       compiler: clang
 
 
-    - env: CONFIG="--disable-user"
+    - env:
+        - CONFIG="--disable-user"
       compiler: clang
 
 
     # gprof/gcov are GCC features
-    - env: CONFIG="--enable-gprof --enable-gcov --disable-pie --target-list=aarch64-softmmu,arm-softmmu,i386-softmmu,mips-softmmu,mips64-softmmu,ppc64-softmmu,riscv64-softmmu,s390x-softmmu,x86_64-softmmu"
+    - env:
+        - CONFIG="--enable-gprof --enable-gcov --disable-pie --target-list=aarch64-softmmu,arm-softmmu,i386-softmmu,mips-softmmu,mips64-softmmu,ppc64-softmmu,riscv64-softmmu,s390x-softmmu,x86_64-softmmu"
       after_success:
         - ${SRC_DIR}/scripts/travis/coverage-summary.sh
       compiler: gcc
 
 
     # We manually include builds which we disable "make check" for
-    - env: CONFIG="--enable-debug --enable-tcg-interpreter"
-           TEST_CMD=""
+    - env:
+        - CONFIG="--enable-debug --enable-tcg-interpreter"
+        - TEST_CMD=""
       compiler: gcc
 
 
     # We don't need to exercise every backend with every front-end
-    - env: CONFIG="--enable-trace-backends=log,simple,syslog --disable-system"
-           TEST_CMD=""
+    - env:
+        - CONFIG="--enable-trace-backends=log,simple,syslog --disable-system"
+        - TEST_CMD=""
       compiler: gcc
 
 
-    - env: CONFIG="--enable-trace-backends=ftrace --target-list=x86_64-softmmu"
-           TEST_CMD=""
+    - env:
+        - CONFIG="--enable-trace-backends=ftrace --target-list=x86_64-softmmu"
+        - TEST_CMD=""
       compiler: gcc
 
 
-    - env: CONFIG="--enable-trace-backends=ust --target-list=x86_64-softmmu"
-           TEST_CMD=""
+    - env:
+        - CONFIG="--enable-trace-backends=ust --target-list=x86_64-softmmu"
+        - TEST_CMD=""
       compiler: gcc
 
 
-    - env: CONFIG="--disable-tcg"
-           TEST_CMD=""
+    - env:
+        - CONFIG="--disable-tcg"
+        - TEST_CMD=""
       compiler: gcc
 
 
     # MacOSX builds
-    - env: CONFIG="--target-list=aarch64-softmmu,arm-softmmu,i386-softmmu,mips-softmmu,mips64-softmmu,ppc64-softmmu,riscv64-softmmu,s390x-softmmu,x86_64-softmmu"
+    - env:
+        - CONFIG="--target-list=aarch64-softmmu,arm-softmmu,i386-softmmu,mips-softmmu,mips64-softmmu,ppc64-softmmu,riscv64-softmmu,s390x-softmmu,x86_64-softmmu"
       os: osx
       osx_image: xcode9.4
       compiler: clang
 
 
-    - env: CONFIG="--target-list=i386-softmmu,ppc-softmmu,ppc64-softmmu,m68k-softmmu,x86_64-softmmu"
+    - env:
+        - CONFIG="--target-list=i386-softmmu,ppc-softmmu,ppc64-softmmu,m68k-softmmu,x86_64-softmmu"
       os: osx
       osx_image: xcode10
       compiler: clang
 
 
     # Python builds
-    - env: CONFIG="--target-list=x86_64-softmmu"
+    - env:
+        - CONFIG="--target-list=x86_64-softmmu"
       python:
         - "3.0"
 
 
-    - env: CONFIG="--target-list=x86_64-softmmu"
+    - env:
+        - CONFIG="--target-list=x86_64-softmmu"
       python:
         - "3.6"
 
 
     # Acceptance (Functional) tests
-    - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
-           TEST_CMD="make AVOCADO_SHOW=app check-acceptance"
+    - env:
+        - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
+        - TEST_CMD="make AVOCADO_SHOW=app check-acceptance"
       addons:
         apt:
           packages:
-- 
2.17.1

[Qemu-devel] [PULL 10/21] travis: don't clone git submodules upfront

[Alex Bennée]

From: Daniel P. Berrangé <berrange@redhat.com>

The configure script & Makefile are already capable of figuring out
which git submodules are required for a given build platform, and
cloning them at the right time.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/.travis.yml b/.travis.yml
index f539f8b556..fa70429459 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -71,7 +71,6 @@ git:
 before_install:
   - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update ; fi
   - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
-  - git submodule update --init --recursive capstone dtc ui/keycodemapdb
 before_script:
   - mkdir -p ${BUILD_DIR} && cd ${BUILD_DIR}
   - ${SRC_DIR}/configure ${CONFIG} || { cat config.log && exit 1; }
-- 
2.17.1

[Qemu-devel] [PULL 11/21] travis: use homebrew addon for MacOSX

[Alex Bennée]

Rather than poking homebrew manually we can specify the packages
needed via the homebrew addon. These are only installed on MacOS based
builds.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>

diff --git a/.travis.yml b/.travis.yml
index fa70429459..2ef7997d25 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -42,6 +42,12 @@ addons:
       - sparse
       - uuid-dev
       - gcovr
+  homebrew:
+    packages:
+      - libffi
+      - gettext
+      - glib
+      - pixman
 
 
 # The channel name "irc.oftc.net#qemu" is encrypted against qemu/qemu
@@ -68,9 +74,6 @@ git:
   submodules: false
 
 
-before_install:
-  - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update ; fi
-  - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
 before_script:
   - mkdir -p ${BUILD_DIR} && cd ${BUILD_DIR}
   - ${SRC_DIR}/configure ${CONFIG} || { cat config.log && exit 1; }
-- 
2.17.1

[Qemu-devel] [PULL 12/21] travis: stop redefining the script commands

[Alex Bennée]

From: Daniel P. Berrangé <berrange@redhat.com>

One of the matrix entries redefines the script command in order to add
the ${MAKEFLAGS} variable. Ideally ${MAKEFLAGS} would be referenced by
the definition of the ${TEST_CMD} env variable, but this isn't possible
in travis. ${MAKEFLAGS} exists to eliminate duplication of flags in
every "make" command, but this cure causes a worse problem, namely the
reduplication of the "script" command. It is simpler to just insert "-j3"
directly into any "make" command.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/.travis.yml b/.travis.yml
index 2ef7997d25..7b18d33e9d 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -65,8 +65,7 @@ env:
   global:
     - SRC_DIR="."
     - BUILD_DIR="."
-    - TEST_CMD="make check"
-    - MAKEFLAGS="-j3"
+    - TEST_CMD="make check -j3"
 
 
 git:
@@ -78,7 +77,7 @@ before_script:
   - mkdir -p ${BUILD_DIR} && cd ${BUILD_DIR}
   - ${SRC_DIR}/configure ${CONFIG} || { cat config.log && exit 1; }
 script:
-  - make ${MAKEFLAGS} && ${TEST_CMD}
+  - make -j3 && ${TEST_CMD}
 
 
 matrix:
@@ -255,9 +254,7 @@ matrix:
 
     - env:
         - CONFIG="--disable-system --disable-docs"
-        - TEST_CMD="make check-tcg"
-      script:
-        - make ${MAKEFLAGS} && ${TEST_CMD} ${MAKEFLAGS}
+        - TEST_CMD="make -j3 check-tcg"
       sudo: required
       dist: trusty
       compiler: gcc
-- 
2.17.1

[Qemu-devel] [PULL 13/21] travis: stop using container based envs

[Alex Bennée]

From: Daniel P. Berrangé <berrange@redhat.com>

The Travis container based envs are deprecated:

  https://docs.travis-ci.com/user/reference/trusty/

  "Container-based infrastructure is currently being deprecated.
   Please remove any sudo: false keys in your .travis.yml file
   to use the default fully-virtualized Linux infrastructure
   instead."

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/.travis.yml b/.travis.yml
index 7b18d33e9d..744dc050d6 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,8 +1,4 @@
-# The current Travis default is a container based 14.04 Trust on EC2
-# Additional builds with specific requirements for a full VM need to
-# be added as additional matrix: entries later on
 
-sudo: false
 dist: trusty
 language: c
 python:
@@ -255,6 +251,5 @@ matrix:
     - env:
         - CONFIG="--disable-system --disable-docs"
         - TEST_CMD="make -j3 check-tcg"
-      sudo: required
       dist: trusty
       compiler: gcc
-- 
2.17.1

[Qemu-devel] [PULL 14/21] travis: run tests in verbose mode

[Alex Bennée]

From: Daniel P. Berrangé <berrange@redhat.com>

Travis sometimes fails a build because it produces no console output for
over 10 minutes. If this is due to a genuine hang, it would be useful to
have used verbose test output to see where it failed. If this is just
due to tests being very slow, having verbose output might allow the
build to succeed.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/.travis.yml b/.travis.yml
index 744dc050d6..39dbda7a0b 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -61,7 +61,7 @@ env:
   global:
     - SRC_DIR="."
     - BUILD_DIR="."
-    - TEST_CMD="make check -j3"
+    - TEST_CMD="make check -j3 V=1"
 
 
 git:
@@ -250,6 +250,6 @@ matrix:
 
     - env:
         - CONFIG="--disable-system --disable-docs"
-        - TEST_CMD="make -j3 check-tcg"
+        - TEST_CMD="make -j3 check-tcg V=1"
       dist: trusty
       compiler: gcc
-- 
2.17.1

[Qemu-devel] [PULL 15/21] travis: remove matrix settings that duplicate global settings

[Alex Bennée]

From: Daniel P. Berrangé <berrange@redhat.com>

The global defaults request "trusty" and "gcc", so matrix entries do not
need to repeat this.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

diff --git a/.travis.yml b/.travis.yml
index 39dbda7a0b..79da3a495a 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -128,39 +128,33 @@ matrix:
         - CONFIG="--enable-gprof --enable-gcov --disable-pie --target-list=aarch64-softmmu,arm-softmmu,i386-softmmu,mips-softmmu,mips64-softmmu,ppc64-softmmu,riscv64-softmmu,s390x-softmmu,x86_64-softmmu"
       after_success:
         - ${SRC_DIR}/scripts/travis/coverage-summary.sh
-      compiler: gcc
 
 
     # We manually include builds which we disable "make check" for
     - env:
         - CONFIG="--enable-debug --enable-tcg-interpreter"
         - TEST_CMD=""
-      compiler: gcc
 
 
     # We don't need to exercise every backend with every front-end
     - env:
         - CONFIG="--enable-trace-backends=log,simple,syslog --disable-system"
         - TEST_CMD=""
-      compiler: gcc
 
 
     - env:
         - CONFIG="--enable-trace-backends=ftrace --target-list=x86_64-softmmu"
         - TEST_CMD=""
-      compiler: gcc
 
 
     - env:
         - CONFIG="--enable-trace-backends=ust --target-list=x86_64-softmmu"
         - TEST_CMD=""
-      compiler: gcc
 
 
     - env:
         - CONFIG="--disable-tcg"
         - TEST_CMD=""
-      compiler: gcc
 
 
     # MacOSX builds
@@ -251,5 +245,3 @@ matrix:
     - env:
         - CONFIG="--disable-system --disable-docs"
         - TEST_CMD="make -j3 check-tcg V=1"
-      dist: trusty
-      compiler: gcc
-- 
2.17.1

[Qemu-devel] [PULL 16/21] docker: Use a stable snapshot for Debian Sid

[Alex Bennée]

From: Philippe Mathieu-Daudé <philmd@redhat.com>

The Debian Sid repository is not garanteed to be stable, as his
'unstable' name suggest :)
To allow quick testing, packages are pushed various time a day,
which my be annoying when trying to use it for stable development
(which is not recommended, but Sid provides edge packages we use
for testing).

Debian provides repositories snapshots which are suitable for our
use. Pick a recent date that works. When required, update to newer
releases will be easy.

This fixes current issues with this image:

  $ make docker-image-debian-sid
  [...]
  The following packages have unmet dependencies:
   build-essential : Depends: dpkg-dev (>= 1.17.11) but it is not going to be installed
   git : Depends: perl but it is not going to be installed
         Depends: liberror-perl but it is not going to be installed
   pkg-config : Depends: libdpkg-perl but it is not going to be installed
   texinfo : Depends: perl (>= 5.26.2-6) but it is not going to be installed
             Depends: libtext-unidecode-perl but it is not going to be installed
             Depends: libxml-libxml-perl but it is not going to be installed
  E: Unable to correct problems, you have held broken packages.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
[AJB: also tweak FROM to a earlier snapshot]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>

diff --git a/tests/docker/dockerfiles/debian-sid.docker b/tests/docker/dockerfiles/debian-sid.docker
index 4e4cda0ba5..676941cb32 100644
--- a/tests/docker/dockerfiles/debian-sid.docker
+++ b/tests/docker/dockerfiles/debian-sid.docker
@@ -11,7 +11,12 @@
 # updated and trigger a re-build.
 #
 
-FROM debian:sid-slim
+# This must be earlier than the snapshot date we are aiming for
+FROM debian:sid-20181011-slim
+
+# Use a snapshot known to work (see http://snapshot.debian.org/#Usage)
+ENV DEBIAN_SNAPSHOT_DATE "20181030"
+RUN sed -i "s%^deb \(https\?://\)deb.debian.org/debian/\? \(.*\)%deb [check-valid-until=no] \1snapshot.debian.org/archive/debian/${DEBIAN_SNAPSHOT_DATE} \2%" /etc/apt/sources.list
 
 # Use a snapshot known to work (see http://snapshot.debian.org/#Usage)
 ENV DEBIAN_SNAPSHOT_DATE "20181030"
-- 
2.17.1

[Qemu-devel] [PULL 17/21] travis: bump to Xenial baseline

[Alex Bennée]

Travis is slowly catching up. Move to Xenial based images for our
current builds. These are now all proper VMs.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/.travis.yml b/.travis.yml
index 79da3a495a..93fd0164a0 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,8 +1,8 @@
-
-dist: trusty
+# The current Travis default is a VM based 16.04 Xenial on GCE
+# Additional builds with specific requirements for a full VM need to
+# be added as additional matrix: entries later on
+dist: xenial
 language: c
-python:
-  - "2.6"
 compiler:
   - gcc
 cache: ccache
@@ -34,7 +34,7 @@ addons:
       - libssh2-1-dev
       - liburcu-dev
       - libusb-1.0-0-dev
-      - libvte-2.90-dev
+      - libvte-2.91-dev
       - sparse
       - uuid-dev
       - gcovr
@@ -193,9 +193,7 @@ matrix:
         apt:
           packages:
             - python3-pip
-            - python3.4-venv
-
-
+            - python3.5-venv
     # Using newer GCC with sanitizers
     - addons:
         apt:
@@ -229,7 +227,7 @@ matrix:
             - libssh2-1-dev
             - liburcu-dev
             - libusb-1.0-0-dev
-            - libvte-2.90-dev
+            - libvte-2.91-dev
             - sparse
             - uuid-dev
       language: generic
-- 
2.17.1

[Qemu-devel] [PULL 18/21] tests/docker: update our Travis image

[Alex Bennée]

We are now using Xenial based images on Travis so we should make the
same one available as our qemu:travis docker image.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/docker/dockerfiles/travis.docker b/tests/docker/dockerfiles/travis.docker
index 46cf367488..e72dc85ca7 100644
--- a/tests/docker/dockerfiles/travis.docker
+++ b/tests/docker/dockerfiles/travis.docker
@@ -1,4 +1,4 @@
-FROM travisci/ci-garnet:packer-1512502276-986baf0
+FROM travisci/ci-sardonyx:packer-1546978056-2c98a19
 ENV DEBIAN_FRONTEND noninteractive
 ENV LANG en_US.UTF-8
 ENV LC_ALL en_US.UTF-8
-- 
2.17.1

[Qemu-devel] [PULL 19/21] tests/docker: remove SID_AGE test hack

[Alex Bennée]

Now we are using "named" snapshots of debian-sid we can rely on the
existing checksum mechanism for detecting changes.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 9467e9d088..7032c68895 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -98,19 +98,6 @@ docker-image-debian-s390x-cross: docker-image-debian9
 docker-image-debian-win32-cross: docker-image-debian8-mxe
 docker-image-debian-win64-cross: docker-image-debian8-mxe
 
-# Debian SID images - we are tracking a rolling distro so we want to
-# force a re-build of the base image if we ever need to build one of
-# its children.
-ifndef SKIP_DOCKER_BUILD
-ifeq ($(HAVE_USER_DOCKER),y)
-SID_AGE=$(shell $(DOCKER_SCRIPT) check --checktype=age --olderthan=180 --quiet qemu:debian-sid)
-ifeq ($(SID_AGE),)
-else
-docker-image-debian-sid: NOCACHE=1
-endif
-endif
-endif
-
 docker-image-debian-alpha-cross: docker-image-debian-sid
 docker-image-debian-hppa-cross: docker-image-debian-sid
 docker-image-debian-m68k-cross: docker-image-debian-sid
-- 
2.17.1

[Qemu-devel] [PULL 20/21] tests: use g_usleep instead of rem = sleep(time)

[Alex Bennée]

Relying on sleep to always return having slept isn't safe as a signal
may have occurred. If signals are constantly incoming the program will
never reach its termination condition. This is believed to be the
mechanism causing time outs for qht-test in Travis.

The glib g_usleep() deals with all of this for us so lets use it instead.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>

diff --git a/tests/atomic64-bench.c b/tests/atomic64-bench.c
index 71692560ed..121a8c14f4 100644
--- a/tests/atomic64-bench.c
+++ b/tests/atomic64-bench.c
@@ -74,16 +74,14 @@ static void *thread_func(void *arg)
 
 static void run_test(void)
 {
-    unsigned int remaining;
     unsigned int i;
 
     while (atomic_read(&n_ready_threads) != n_threads) {
         cpu_relax();
     }
+
     atomic_set(&test_start, true);
-    do {
-        remaining = sleep(duration);
-    } while (remaining);
+    g_usleep(duration * G_USEC_PER_SEC);
     atomic_set(&test_stop, true);
 
     for (i = 0; i < n_threads; i++) {
diff --git a/tests/atomic_add-bench.c b/tests/atomic_add-bench.c
index 2f6c72f63a..5666f6bbff 100644
--- a/tests/atomic_add-bench.c
+++ b/tests/atomic_add-bench.c
@@ -76,16 +76,14 @@ static void *thread_func(void *arg)
 
 static void run_test(void)
 {
-    unsigned int remaining;
     unsigned int i;
 
     while (atomic_read(&n_ready_threads) != n_threads) {
         cpu_relax();
     }
+
     atomic_set(&test_start, true);
-    do {
-        remaining = sleep(duration);
-    } while (remaining);
+    g_usleep(duration * G_USEC_PER_SEC);
     atomic_set(&test_stop, true);
 
     for (i = 0; i < n_threads; i++) {
diff --git a/tests/qht-bench.c b/tests/qht-bench.c
index ab4e708180..e3b512f26f 100644
--- a/tests/qht-bench.c
+++ b/tests/qht-bench.c
@@ -398,16 +398,14 @@ static void pr_stats(void)
 
 static void run_test(void)
 {
-    unsigned int remaining;
     int i;
 
     while (atomic_read(&n_ready_threads) != n_rw_threads + n_rz_threads) {
         cpu_relax();
     }
+
     atomic_set(&test_start, true);
-    do {
-        remaining = sleep(duration);
-    } while (remaining);
+    g_usleep(duration * G_USEC_PER_SEC);
     atomic_set(&test_stop, true);
 
     for (i = 0; i < n_rw_threads; i++) {
-- 
2.17.1

[Qemu-devel] [PULL 21/21] Revert "tests: Disable qht-bench parallel test when using gprof"

[Alex Bennée]

This reverts commit ce2eefd7c21697fee87a0686353de881081d22c6. The
underlying cause was fixed with eb4f8e100f.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/configure b/configure
index 4ea3f14883..f59d484f7a 100755
--- a/configure
+++ b/configure
@@ -7487,7 +7487,6 @@ alpha)
 esac
 
 if test "$gprof" = "yes" ; then
-  echo "CONFIG_GPROF=y" >> $config_host_mak
   echo "TARGET_GPROF=y" >> $config_target_mak
   if test "$target_linux_user" = "yes" ; then
     cflags="-p $cflags"
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 601ef4f64c..f403a6571d 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -88,8 +88,7 @@ check-unit-y += tests/test-rcu-simpleq$(EXESUF)
 check-unit-y += tests/test-rcu-tailq$(EXESUF)
 check-unit-y += tests/test-qdist$(EXESUF)
 check-unit-y += tests/test-qht$(EXESUF)
-# FIXME: {test-qht-par + gprof} often break on Travis CI
-check-unit-$(call lnot,$(CONFIG_GPROF)) += tests/test-qht-par$(EXESUF)
+check-unit-y += tests/test-qht-par$(EXESUF)
 check-unit-y += tests/test-bitops$(EXESUF)
 check-unit-y += tests/test-bitcnt$(EXESUF)
 check-unit-y += tests/test-qdev-global-props$(EXESUF)
-- 
2.17.1

Re: [Qemu-devel] [PULL 21/21] Revert "tests: Disable qht-bench parallel test when using gprof"

[Alex Bennée]

Alex Bennée <alex.bennee@linaro.org> writes:

> This reverts commit ce2eefd7c21697fee87a0686353de881081d22c6. The
> underlying cause was fixed with eb4f8e100f.

A quick flurry of comments came just after I posted the PR. Should I
re-spin v2 without the last minute clean-up? I just worry this sticking
plaster is:

 a) a bit hacky, introducing a new build glag
 b) might end up masking stuff we want to catch

>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>
> diff --git a/configure b/configure
> index 4ea3f14883..f59d484f7a 100755
> --- a/configure
> +++ b/configure
> @@ -7487,7 +7487,6 @@ alpha)
>  esac
>
>  if test "$gprof" = "yes" ; then
> -  echo "CONFIG_GPROF=y" >> $config_host_mak
>    echo "TARGET_GPROF=y" >> $config_target_mak
>    if test "$target_linux_user" = "yes" ; then
>      cflags="-p $cflags"
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 601ef4f64c..f403a6571d 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -88,8 +88,7 @@ check-unit-y += tests/test-rcu-simpleq$(EXESUF)
>  check-unit-y += tests/test-rcu-tailq$(EXESUF)
>  check-unit-y += tests/test-qdist$(EXESUF)
>  check-unit-y += tests/test-qht$(EXESUF)
> -# FIXME: {test-qht-par + gprof} often break on Travis CI
> -check-unit-$(call lnot,$(CONFIG_GPROF)) += tests/test-qht-par$(EXESUF)
> +check-unit-y += tests/test-qht-par$(EXESUF)
>  check-unit-y += tests/test-bitops$(EXESUF)
>  check-unit-y += tests/test-bitcnt$(EXESUF)
>  check-unit-y += tests/test-qdev-global-props$(EXESUF)


--
Alex Bennée

Re: [Qemu-devel] [PULL 00/21] misc testing fixes for Travis and docker

[Peter Maydell]

On Mon, 14 Jan 2019 at 15:01, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> The following changes since commit 7260438b7056469610ee166f7abe9ff8a26b8b16:
>
>   Merge remote-tracking branch 'remotes/palmer/tags/riscv-for-master-3.2-part2' into staging (2019-01-14 11:41:43 +0000)
>
> are available in the Git repository at:
>
>   https://github.com/stsquad/qemu.git tags/pull-testing-next-140119-1
>
> for you to fetch changes up to a36270a4d1589b1ed309065fc8b3fe0ac8d6869d:
>
>   Revert "tests: Disable qht-bench parallel test when using gprof" (2019-01-14 14:55:32 +0000)
>
> ----------------------------------------------------------------
> A bunch of fixes for testing:
>
>   - Various Travis updates
>   - "stable" SID snapshot for docker
>   - avoid :latest docker tags
>   - g_usleep fix for some tests
>

Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/4.0
for any user-visible changes.

-- PMM