[Qemu-devel] [PULL 0/9] various Travis and Docker fixes, add gitlab CI

[Qemu-devel] [PULL 0/9] various Travis and Docker fixes, add gitlab CI

[Alex Bennée]

The following changes since commit fc3dbb90f2eb069801bfb4cfe9cbc83cf9c5f4a9:

  Merge remote-tracking branch 'remotes/jnsnow/tags/bitmaps-pull-request' into staging (2019-02-21 13:09:33 +0000)

are available in the Git repository at:

  https://github.com/stsquad/qemu.git tags/pull-testing-next-220219-1

for you to fetch changes up to 4300dadc00bec6d75283ffbe1fd85c2b442bef2d:

  tests/cdrom-test: only include isapc cdrom test when g_test_slow() (2019-02-22 09:32:32 +0000)

----------------------------------------------------------------
Various testing fixes:

  - Travis updates (inc disable isapc cdrom test)
  - Add gitlab control
  - Fix docker image
  - keep softloat tests short

----------------------------------------------------------------
Alex Bennée (5):
      .travis.yml: split debug builds
      tests/docker: squash initial update and install step for debian9
      tests/docker: peg netmap code to a specific version
      tests/softfloat: always do quick softfloat tests
      tests/cdrom-test: only include isapc cdrom test when g_test_slow()

Dr. David Alan Gilbert (2):
      .travis.yml: Test with disable-replication
      .travis.yml: Remove disable-uuid

Paolo Bonzini (1):
      .travis.yml: the xcode10 image seems to be hosed

Thomas Huth (1):
      Add a gitlab-ci file for Continuous Integration testing on Gitlab

 .gitlab-ci.yml                               | 73 ++++++++++++++++++++++++++++
 .travis.yml                                  | 16 +++---
 MAINTAINERS                                  |  5 ++
 tests/Makefile.include                       |  6 +--
 tests/cdrom-test.c                           | 10 +++-
 tests/docker/dockerfiles/debian-amd64.docker |  1 +
 tests/docker/dockerfiles/debian9.docker      |  4 +-
 7 files changed, 98 insertions(+), 17 deletions(-)
 create mode 100644 .gitlab-ci.yml

-- 
2.20.1

[Qemu-devel] [PULL 1/9] .travis.yml: the xcode10 image seems to be hosed

[Alex Bennée]

From: Paolo Bonzini <pbonzini@redhat.com>

It fails to install homebrew.  Unfortunately we cannot mark
it as an expected failure because Travis does not match
allow_failures rows against include rows (only against the
main test matrix, which we do not use at all), so just disable
it.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20190220105131.23479-1-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/.travis.yml b/.travis.yml
index baa06b976a..866a1872b1 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -174,13 +174,6 @@ matrix:
       compiler: clang
 
 
-    - env:
-        - CONFIG="--target-list=i386-softmmu,ppc-softmmu,ppc64-softmmu,m68k-softmmu,x86_64-softmmu"
-      os: osx
-      osx_image: xcode10
-      compiler: clang
-
-
     # Python builds
     - env:
         - CONFIG="--target-list=x86_64-softmmu"
-- 
2.20.1

[Qemu-devel] [PULL 2/9] .travis.yml: split debug builds

[Alex Bennée]

The builds are reaching the magic 50 minute limit with regularity so
lets split them up. Rather than doing a full debug build on both just
enable debug tcg for linux-user.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

diff --git a/.travis.yml b/.travis.yml
index 866a1872b1..36ed7ec3f9 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -86,7 +86,12 @@ matrix:
 
 
     - env:
-        - CONFIG="--enable-debug --enable-debug-tcg"
+        - CONFIG="--enable-debug --enable-debug-tcg --disable-user"
+
+
+    # TCG debug can be run just on it's own and is mostly agnostic to user/softmmu distinctions
+    - env:
+        - CONFIG="--enable-debug-tcg --disable-system"
 
 
     - env:
-- 
2.20.1

[Qemu-devel] [PULL 3/9] .travis.yml: Test with disable-replication

[Alex Bennée]

From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>

We've had the build break with replication disabled, so lets
test that case in travis.

Suggsted-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20190215094502.32149-1-dgilbert@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/.travis.yml b/.travis.yml
index 36ed7ec3f9..61b086b8b8 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -95,7 +95,7 @@ matrix:
 
 
     - env:
-        - CONFIG="--disable-linux-aio --disable-cap-ng --disable-attr --disable-brlapi --disable-uuid --disable-libusb --disable-user"
+        - CONFIG="--disable-linux-aio --disable-cap-ng --disable-attr --disable-brlapi --disable-uuid --disable-libusb --disable-user --disable-replication"
 
 
     - env:
-- 
2.20.1

[Qemu-devel] [PULL 4/9] .travis.yml: Remove disable-uuid

[Alex Bennée]

From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>

Commit 315d3184525 turned --disable-uuid into a warning only; remove
the check from Travis.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20190215094502.32149-2-dgilbert@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/.travis.yml b/.travis.yml
index 61b086b8b8..cca57f4314 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -95,7 +95,7 @@ matrix:
 
 
     - env:
-        - CONFIG="--disable-linux-aio --disable-cap-ng --disable-attr --disable-brlapi --disable-uuid --disable-libusb --disable-user --disable-replication"
+        - CONFIG="--disable-linux-aio --disable-cap-ng --disable-attr --disable-brlapi --disable-libusb --disable-user --disable-replication"
 
 
     - env:
-- 
2.20.1

[Qemu-devel] [PULL 5/9] tests/docker: squash initial update and install step for debian9

[Alex Bennée]

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/docker/dockerfiles/debian9.docker b/tests/docker/dockerfiles/debian9.docker
index 154ae2a455..5f23a35404 100644
--- a/tests/docker/dockerfiles/debian9.docker
+++ b/tests/docker/dockerfiles/debian9.docker
@@ -13,8 +13,8 @@ FROM debian:stretch-slim
 RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
 
 # Install common build utilities
-RUN apt update
-RUN DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata
+RUN apt-get update && \
+    DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata
 RUN DEBIAN_FRONTEND=noninteractive eatmydata \
     apt install -y --no-install-recommends \
         bison \
-- 
2.20.1

[Qemu-devel] [PULL 6/9] tests/docker: peg netmap code to a specific version

[Alex Bennée]

Tracking head is always going to be at the whims of the upstream.
Let's use a defined release so things don't magically change under us.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

diff --git a/tests/docker/dockerfiles/debian-amd64.docker b/tests/docker/dockerfiles/debian-amd64.docker
index 954fcf9606..d770a11a52 100644
--- a/tests/docker/dockerfiles/debian-amd64.docker
+++ b/tests/docker/dockerfiles/debian-amd64.docker
@@ -33,6 +33,7 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
     apt-get install -y --no-install-recommends \
         linux-headers-amd64
 RUN git clone https://github.com/luigirizzo/netmap.git /usr/src/netmap
+RUN cd /usr/src/netmap && git checkout v11.3
 RUN cd /usr/src/netmap/LINUX && ./configure --no-drivers --no-apps --kernel-dir=$(ls -d /usr/src/linux-headers-*-amd64) && make install
 ENV QEMU_CONFIGURE_OPTS --enable-netmap
 
-- 
2.20.1

[Qemu-devel] [PULL 7/9] Add a gitlab-ci file for Continuous Integration testing on Gitlab

[Alex Bennée]

From: Thomas Huth <thuth@redhat.com>

This is very convenient for people like me who store their QEMU git trees
on gitlab.com: Automatic CI pipelines are now run for each branch that is
pushed to the server - useful for some extra-testing before sending PULL-
requests for example. Since the runtime of the jobs is limited to 1h, the
jobs are distributed into multiple pipelines - this way everything finishs
fine within time (ca. 30 minutes currently).

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <1550058881-16351-1-git-send-email-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
new file mode 100644
index 0000000000..79d02cf740
--- /dev/null
+++ b/.gitlab-ci.yml
@@ -0,0 +1,73 @@
+before_script:
+ - apt-get update -qq
+ - apt-get install -y -qq flex bison libglib2.0-dev libpixman-1-dev genisoimage
+
+build-system1:
+ script:
+ - apt-get install -y -qq libgtk-3-dev libvte-dev nettle-dev libcacard-dev
+      libusb-dev libvde-dev libspice-protocol-dev libgl1-mesa-dev
+ - ./configure --enable-werror --target-list="aarch64-softmmu alpha-softmmu
+      cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu microblazeel-softmmu
+      mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu sparc-softmmu"
+ - make -j2
+ - make -j2 check
+
+build-system2:
+ script:
+ - apt-get install -y -qq libsdl2-dev libgcrypt-dev libbrlapi-dev libaio-dev
+      libfdt-dev liblzo2-dev librdmacm-dev libibverbs-dev libibumad-dev
+ - ./configure --enable-werror --target-list="tricore-softmmu unicore32-softmmu
+      microblaze-softmmu mips-softmmu riscv32-softmmu s390x-softmmu sh4-softmmu
+      sparc64-softmmu x86_64-softmmu xtensa-softmmu nios2-softmmu or1k-softmmu"
+ - make -j2
+ - make -j2 check
+
+build-disabled:
+ script:
+ - ./configure --enable-werror --disable-rdma --disable-slirp --disable-curl
+      --disable-capstone --disable-live-block-migration --disable-glusterfs
+      --disable-replication --disable-coroutine-pool --disable-smartcard
+      --disable-guest-agent --disable-curses --disable-libxml2 --disable-tpm
+      --disable-qom-cast-debug --disable-spice --disable-vhost-vsock
+      --disable-vhost-net --disable-vhost-crypto --disable-vhost-user
+      --target-list="i386-softmmu ppc64-softmmu mips64-softmmu i386-linux-user"
+ - make -j2
+ - make -j2 check-qtest SPEED=slow
+
+build-tcg-disabled:
+ script:
+ - apt-get install -y -qq clang libgtk-3-dev libbluetooth-dev libusb-dev
+ - ./configure --cc=clang --enable-werror --disable-tcg --audio-drv-list=""
+ - make -j2
+ - make check-unit
+ - make check-qapi-schema
+ - cd tests/qemu-iotests/
+ - ./check -raw 001 002 003 004 005 008 009 010 011 012 021 025 032 033 048
+            052 063 077 086 101 104 106 113 147 148 150 151 152 157 159 160
+            163 170 171 183 184 192 194 197 205 208 215 221 222 226 227 236
+ - ./check -qcow2 001 002 003 004 005 007 008 009 010 011 012 013 017 018 019
+            020 021 022 024 025 027 028 029 031 032 033 034 035 036 037 038
+            039 040 042 043 046 047 048 049 050 051 052 053 054 056 057 058
+            060 061 062 063 065 066 067 068 069 071 072 073 074 079 080 082
+            085 086 089 090 091 095 096 097 098 099 102 103 104 105 107 108
+            110 111 114 117 120 122 124 126 127 129 130 132 133 134 137 138
+            139 140 141 142 143 144 145 147 150 151 152 154 155 156 157 158
+            161 165 170 172 174 176 177 179 184 186 187 190 192 194 195 196
+            197 200 202 203 205 208 209 214 215 216 217 218 222 226 227 229 234
+
+build-user:
+ script:
+ - ./configure --enable-werror --disable-system --disable-guest-agent
+               --disable-capstone --disable-slirp --disable-fdt
+ - make -j2
+ - make run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-linux-user
+
+build-clang:
+ script:
+ - apt-get install -y -qq clang libsdl2-dev
+      xfslibs-dev libiscsi-dev libnfs-dev libseccomp-dev gnutls-dev librbd-dev
+ - ./configure --cc=clang --cxx=clang++ --enable-werror
+      --target-list="alpha-softmmu arm-softmmu m68k-softmmu mips64-softmmu
+                     ppc-softmmu s390x-softmmu x86_64-softmmu arm-linux-user"
+ - make -j2
+ - make -j2 check
diff --git a/MAINTAINERS b/MAINTAINERS
index b1d786cfd8..c6f2976c18 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2479,6 +2479,11 @@ S: Maintained
 F: .cirrus.yml
 W: https://cirrus-ci.com/github/qemu/qemu
 
+GitLab Continuous Integration
+M: Thomas Huth <thuth@redhat.com>
+S: Maintained
+F: .gitlab-ci.yml
+
 Guest Test Compilation Support
 M: Alex Bennée <alex.bennee@linaro.org>
 R: Philippe Mathieu-Daudé <f4bug@amsat.org>
-- 
2.20.1

[Qemu-devel] [PULL 8/9] tests/softfloat: always do quick softfloat tests

[Alex Bennée]

Some operations take a long time and enabling "-l 2 -r all" can take
more than a day which is stretching the definition of a "slow" test.
Lets default to the quick test and leave a note for those who wish to
run by hand.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

diff --git a/tests/Makefile.include b/tests/Makefile.include
index b39e989f72..3741f8f6dd 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -896,11 +896,9 @@ $(FP_TEST_BIN):
 	         "BUILD", "$(notdir $@)")
 
 # The full test suite can take a bit of time, default to a quick run
-ifeq ($(SPEED), quick)
+# "-l 2 -r all" can take more than a day for some operations and is best
+# run manually
 FP_TL=-l 1
-else
-FP_TL=-l 2 -r all
-endif
 
 # $1 = tests, $2 = description
 test-softfloat = $(call quiet-command, \
-- 
2.20.1

[Qemu-devel] [PULL 9/9] tests/cdrom-test: only include isapc cdrom test when g_test_slow()

[Alex Bennée]

We are seeing instability on our CI runs which has been there since
the test was introduced. I suspect it triggers more on Travis due to
their heavy load.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Thomas Huth <thuth@redhat.com>

diff --git a/tests/cdrom-test.c b/tests/cdrom-test.c
index 14bd981336..05611da648 100644
--- a/tests/cdrom-test.c
+++ b/tests/cdrom-test.c
@@ -132,8 +132,14 @@ static void add_x86_tests(void)
     qtest_add_data_func("cdrom/boot/virtio-scsi",
                         "-device virtio-scsi -device scsi-cd,drive=cdr "
                         "-blockdev file,node-name=cdr,filename=", test_cdboot);
-    qtest_add_data_func("cdrom/boot/isapc", "-M isapc "
-                        "-drive if=ide,media=cdrom,file=", test_cdboot);
+    /*
+     * Unstable CI test under load
+     * See https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg05509.html
+     */
+    if (g_test_slow()) {
+        qtest_add_data_func("cdrom/boot/isapc", "-M isapc "
+                            "-drive if=ide,media=cdrom,file=", test_cdboot);
+    }
     qtest_add_data_func("cdrom/boot/am53c974",
                         "-device am53c974 -device scsi-cd,drive=cd1 "
                         "-drive if=none,id=cd1,format=raw,file=", test_cdboot);
-- 
2.20.1

Re: [Qemu-devel] [PULL 0/9] various Travis and Docker fixes, add gitlab CI

[Peter Maydell]

On Fri, 22 Feb 2019 at 09:54, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> The following changes since commit fc3dbb90f2eb069801bfb4cfe9cbc83cf9c5f4a9:
>
>   Merge remote-tracking branch 'remotes/jnsnow/tags/bitmaps-pull-request' into staging (2019-02-21 13:09:33 +0000)
>
> are available in the Git repository at:
>
>   https://github.com/stsquad/qemu.git tags/pull-testing-next-220219-1
>
> for you to fetch changes up to 4300dadc00bec6d75283ffbe1fd85c2b442bef2d:
>
>   tests/cdrom-test: only include isapc cdrom test when g_test_slow() (2019-02-22 09:32:32 +0000)
>
> ----------------------------------------------------------------
> Various testing fixes:
>
>   - Travis updates (inc disable isapc cdrom test)
>   - Add gitlab control
>   - Fix docker image
>   - keep softloat tests short


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/4.0
for any user-visible changes.

-- PMM