[Qemu-devel] [PULL 0/7] Travis updates

[Qemu-devel] [PULL 0/7] Travis updates

[Alex Bennée]

The following changes since commit c25e8bba1f546ea72744ccfab77f8a9e8a323be8:

  Merge remote-tracking branch 'remotes/otubo/tags/pull-seccomp-20180601' into staging (2018-06-01 13:11:30 +0100)

are available in the Git repository at:

  https://github.com/stsquad/qemu.git tags/pull-travis-updates-010618-1

for you to fetch changes up to ac07ffc65472ec3b1255f427051481fc9050a2f8:

  .travis.yml: update GCC sanitizer build to GCC 7 (2018-06-01 15:00:54 +0100)

----------------------------------------------------------------
Some Travis & Docker updates

 - simplification of the build matrix
 - usability tweaks for docker
 - update Travis docker image

----------------------------------------------------------------
Alex Bennée (5):
      .travis.yml: disable linux-user build for gcov
      docker: update Travis docker image
      .travis.yml: rationalise clang testing
      .travis.yml: make current setup explicit
      .travis.yml: update GCC sanitizer build to GCC 7

Philippe Mathieu-Daudé (2):
      docker: sort images list displayed by 'make docker'
      docker: do not display deprecated images in 'make docker' help

 .travis.yml                            | 84 +++++++---------------------------
 tests/docker/Makefile.include          |  5 +-
 tests/docker/dockerfiles/travis.docker |  9 +++-
 3 files changed, 27 insertions(+), 71 deletions(-)

--
2.17.0

[Qemu-devel] [PULL 1/7] .travis.yml: disable linux-user build for gcov

[Alex Bennée]

Currently the default testing doesn't exercise the linux-user builds
so there is no point spending time building them. We may want to
enable a separate gcov build once linux-user testing is re-enabled
although it's likely to report very low coverage.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

diff --git a/.travis.yml b/.travis.yml
index c1e99237b2..aa83e9aed7 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -74,7 +74,7 @@ matrix:
     - env: CONFIG=""
       compiler: clang
     # gprof/gcov are GCC features
-    - env: CONFIG="--enable-gprof --enable-gcov --disable-pie"
+    - env: CONFIG="--enable-gprof --enable-gcov --disable-pie --disable-linux-user"
       compiler: gcc
     # We manually include builds which we disable "make check" for
     - env: CONFIG="--enable-debug --enable-tcg-interpreter"
-- 
2.17.0

[Qemu-devel] [PULL 2/7] docker: sort images list displayed by 'make docker'

[Alex Bennée]

From: Philippe Mathieu-Daudé <f4bug@amsat.org>

we can now directly see different version sort consecutively.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index ef1a3e62eb..50cd51a54e 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -4,7 +4,7 @@
 
 DOCKER_SUFFIX := .docker
 DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
-DOCKER_IMAGES := $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker)))
+DOCKER_IMAGES := $(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker))))
 DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES))
 # Use a global constant ccache directory to speed up repetitive builds
 DOCKER_CCACHE_DIR := $$HOME/.cache/qemu-docker-ccache
-- 
2.17.0

[Qemu-devel] [PULL 3/7] docker: do not display deprecated images in 'make docker' help

[Alex Bennée]

From: Philippe Mathieu-Daudé <f4bug@amsat.org>

the 'debian' base image is deprecated since 3e11974988d8

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 50cd51a54e..31f21a43f5 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -4,7 +4,8 @@
 
 DOCKER_SUFFIX := .docker
 DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
-DOCKER_IMAGES := $(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker))))
+DOCKER_DEPRECATED_IMAGES := debian
+DOCKER_IMAGES := $(filter-out $(DOCKER_DEPRECATED_IMAGES),$(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker)))))
 DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES))
 # Use a global constant ccache directory to speed up repetitive builds
 DOCKER_CCACHE_DIR := $$HOME/.cache/qemu-docker-ccache
@@ -63,7 +64,7 @@ docker-image-debian-win64-cross: docker-image-debian8-mxe
 docker-image-travis: NOUSER=1
 
 # Expand all the pre-requistes for each docker image and test combination
-$(foreach i,$(DOCKER_IMAGES), \
+$(foreach i,$(DOCKER_IMAGES) $(DOCKER_DEPRECATED_IMAGES), \
 	$(foreach t,$(DOCKER_TESTS) $(DOCKER_TOOLS), \
 		$(eval .PHONY: docker-$t@$i) \
 		$(eval docker-$t@$i: docker-image-$i docker-run-$t@$i) \
-- 
2.17.0

[Qemu-devel] [PULL 4/7] docker: update Travis docker image

[Alex Bennée]

This is still poorly documented by Travis but according to:

  https://docs.travis-ci.com/user/common-build-problems/#Running-a-Container-Based-Docker-Image-Locally

their reference images are now hosted on Docker Hub. So we update the
FROM line to refer to the new default image. We also need a few
additional tweaks:

  - re-enable deb-src lines for our build-dep install
  - add explicit PATH definition for tools
  - force the build USER to be Travis
  - add clang to FEATURES for our test-clang machinery

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

diff --git a/tests/docker/dockerfiles/travis.docker b/tests/docker/dockerfiles/travis.docker
index 605b6e429b..c5ad39b533 100644
--- a/tests/docker/dockerfiles/travis.docker
+++ b/tests/docker/dockerfiles/travis.docker
@@ -1,8 +1,13 @@
-FROM quay.io/travisci/travis-ruby
+FROM travisci/ci-garnet:packer-1512502276-986baf0
 ENV DEBIAN_FRONTEND noninteractive
 ENV LANG en_US.UTF-8
 ENV LC_ALL en_US.UTF-8
+RUN cat /etc/apt/sources.list | sed "s/# deb-src/deb-src/" >> /etc/apt/sources.list
 RUN apt-get update
 RUN apt-get -y build-dep qemu
 RUN apt-get -y install device-tree-compiler python2.7 python-yaml dh-autoreconf gdb strace lsof net-tools
-ENV FEATURES pyyaml
+# Travis tools require PhantomJS / Neo4j / Maven accessible
+# in their PATH (QEMU build won't access them).
+ENV PATH /usr/local/phantomjs/bin:/usr/local/phantomjs:/usr/local/neo4j-3.2.7/bin:/usr/local/maven-3.5.2/bin:/usr/local/cmake-3.9.2/bin:/usr/local/clang-5.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ENV FEATURES clang pyyaml
+USER travis
-- 
2.17.0

[Qemu-devel] [PULL 5/7] .travis.yml: rationalise clang testing

[Alex Bennée]

As Travis includes Clang 5.0 in its own build environment there is no
point manually building with older Clangs. We still need to test with
the two pythons though so we leave them as minimal system only builds.
We also split the clang build into two as it often exceeds the 40
minute build time limit.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

diff --git a/.travis.yml b/.travis.yml
index aa83e9aed7..f91db3ab1a 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -70,8 +70,10 @@ script:
   - make ${MAKEFLAGS} && ${TEST_CMD}
 matrix:
   include:
-    # Test with CLang for compile portability
-    - env: CONFIG=""
+    # Test with Clang for compile portability (Travis uses clang-5.0)
+    - env: CONFIG="--disable-system"
+      compiler: clang
+    - env: CONFIG="--disable-user"
       compiler: clang
     # gprof/gcov are GCC features
     - env: CONFIG="--enable-gprof --enable-gcov --disable-pie --disable-linux-user"
@@ -95,70 +97,13 @@ matrix:
     - env: CONFIG=""
       os: osx
       compiler: clang
-    # Plain Trusty System Build
-    - env: CONFIG="--disable-linux-user"
-      sudo: required
-      addons:
-      dist: trusty
-      compiler: gcc
-      before_install:
-        - sudo apt-get update -qq
-        - sudo apt-get build-dep -qq qemu
-        - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
-        - git submodule update --init --recursive
-    # Plain Trusty Linux User Build
-    - env: CONFIG="--disable-system"
-      sudo: required
-      addons:
-      dist: trusty
-      compiler: gcc
-      before_install:
-        - sudo apt-get update -qq
-        - sudo apt-get build-dep -qq qemu
-        - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
-        - git submodule update --init --recursive
-    # Trusty System build with latest stable clang & python 3.0
-    - sudo: required
-      addons:
-      dist: trusty
-      language: generic
-      compiler: none
+    # Python builds
+    - env: CONFIG="--target-list=x86_64-softmmu"
       python:
         - "3.0"
-      env:
-        - COMPILER_NAME=clang CXX=clang++-3.9 CC=clang-3.9
-        - CONFIG="--disable-linux-user --cc=clang-3.9 --cxx=clang++-3.9 --python=/usr/bin/python3"
-      before_install:
-        - wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
-        - sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
-        - sudo apt-get update -qq
-        - sudo apt-get install -qq -y clang-3.9
-        - sudo apt-get build-dep -qq qemu
-        - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
-        - git submodule update --init --recursive
-      before_script:
-        - ./configure ${CONFIG} || cat config.log
-    # Trusty Linux User build with latest stable clang & python 3.6
-    - sudo: required
-      addons:
-      dist: trusty
-      language: generic
-      compiler: none
+    - env: CONFIG="--target-list=x86_64-softmmu"
       python:
         - "3.6"
-      env:
-        - COMPILER_NAME=clang CXX=clang++-3.9 CC=clang-3.9
-        - CONFIG="--disable-system --cc=clang-3.9 --cxx=clang++-3.9 --python=/usr/bin/python3"
-      before_install:
-        - wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
-        - sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
-        - sudo apt-get update -qq
-        - sudo apt-get install -qq -y clang-3.9
-        - sudo apt-get build-dep -qq qemu
-        - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
-        - git submodule update --init --recursive
-      before_script:
-        - ./configure ${CONFIG} || cat config.log
     # Using newer GCC with sanitizers
     - addons:
         apt:
-- 
2.17.0

[Qemu-devel] [PULL 6/7] .travis.yml: make current setup explicit

[Alex Bennée]

Add some commentary and make the selection of Container based Trusty
build explicit. We will need to add VM builds later when using docker.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

diff --git a/.travis.yml b/.travis.yml
index f91db3ab1a..7197247a34 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,4 +1,8 @@
+# The current Travis default is a container based 14.04 Trust on EC2
+# Additional builds with specific requirements for a full VM need to
+# be added as additional matrix: entries later on
 sudo: false
+dist: trusty
 language: c
 python:
   - "2.6"
-- 
2.17.0

[Qemu-devel] [PULL 7/7] .travis.yml: update GCC sanitizer build to GCC 7

[Alex Bennée]

GCC has moved on and so should we. We also enable apt update to ensure
we get the latest build from the toolchain PPA.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

diff --git a/.travis.yml b/.travis.yml
index 7197247a34..814be151f4 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -111,13 +111,14 @@ matrix:
     # Using newer GCC with sanitizers
     - addons:
         apt:
+          update: true
           sources:
             # PPAs for newer toolchains
             - ubuntu-toolchain-r-test
           packages:
             # Extra toolchains
-            - gcc-5
-            - g++-5
+            - gcc-7
+            - g++-7
             # Build dependencies
             - libaio-dev
             - libattr1-dev
@@ -146,8 +147,8 @@ matrix:
       language: generic
       compiler: none
       env:
-        - COMPILER_NAME=gcc CXX=g++-5 CC=gcc-5
-        - CONFIG="--cc=gcc-5 --cxx=g++-5 --disable-pie --disable-linux-user"
+        - COMPILER_NAME=gcc CXX=g++-7 CC=gcc-7
+        - CONFIG="--cc=gcc-7 --cxx=g++-7 --disable-pie --disable-linux-user"
         - TEST_CMD=""
       before_script:
         - ./configure ${CONFIG} --extra-cflags="-g3 -O0 -fsanitize=thread -fuse-ld=gold" || cat config.log
-- 
2.17.0

Re: [Qemu-devel] [PULL 0/7] Travis updates

[Peter Maydell]

On 1 June 2018 at 17:20, Alex Bennée <alex.bennee@linaro.org> wrote:
> The following changes since commit c25e8bba1f546ea72744ccfab77f8a9e8a323be8:
>
>   Merge remote-tracking branch 'remotes/otubo/tags/pull-seccomp-20180601' into staging (2018-06-01 13:11:30 +0100)
>
> are available in the Git repository at:
>
>   https://github.com/stsquad/qemu.git tags/pull-travis-updates-010618-1
>
> for you to fetch changes up to ac07ffc65472ec3b1255f427051481fc9050a2f8:
>
>   .travis.yml: update GCC sanitizer build to GCC 7 (2018-06-01 15:00:54 +0100)
>
> ----------------------------------------------------------------
> Some Travis & Docker updates
>
>  - simplification of the build matrix
>  - usability tweaks for docker
>  - update Travis docker image
>

Applied, thanks.

-- PMM