[Qemu-devel] [PATCH for-2.11 v3 00/16] travis: speedup to reduce failures

[Qemu-devel] [PATCH for-2.11 v3 00/16] travis: speedup to reduce failures

[Philippe Mathieu-Daudé]

These patches try to improve our Travis CI usage (few failures the last days):
- lower false negative rate (mostly network issues),
- improved code coverage,
- speedup failure detection.

dropped since v2 (will follow as a separate series)
- multicore parallelism use
- cache use

Patch 1 is expected to enter /master as of yesterday.
Travis warned few months ago:
  "On Wednesday, June 21st 2017, we are going to update all our Ubuntu Trusty
  14.04 images."

Patches 4-6 try to avoid aborting a job on network failure.

Patches 7-9 intend to speedup jobs.

Patch 10 gives hint when ./configure fails, patch 12 silent build warnings.

RFC patch 13 uses upgraded GCC v5 -> v6.
RFC patch 14 don't fails the build if the slow gprof/gcov job fail.

Patches 15, 16 add more OSX VMs/toolchains (older, newer, bleeding edge).

Regards,

Phil.

Peter Maydell (1):
  travis: install more library dependencies

Philippe Mathieu-Daudé (15):
  travis: update sudo-enabled Trusty images
  travis: enable multiple caching features
  travis: increase S3 cache timeout
  travis: retry if llvm.org timeouts
  travis: retry when git submodules initialization fails
  travis: split the gprof/gcov job
  travis: reorder matrix to speedup failure
  travis: reduce git clone depth
  travis: check ./configure outcome, dump config.log on failure
  travis/osx: use readable YAML
  travis/osx: silent texinfo warnings
  RFC travis: use gcc-6 sanitizers
  RFC travis: allow_failures for gprof/gcov
  RFC travis/osx: build using more Xcode versions
  NOTFORMERGE travis/osx: build using bleeding Xcode

 .travis.yml | 134 +++++++++++++++++++++++++++++++++++++++++++-----------------
 1 file changed, 97 insertions(+), 37 deletions(-)

-- 
2.13.3

[Qemu-devel] [PATCH for-2.11 v3 01/16] travis: update sudo-enabled Trusty images

[Philippe Mathieu-Daudé]

see https://blog.travis-ci.com/2017-06-19-trusty-updates-2017-Q2
and https://blog.travis-ci.com/2017-06-21-trusty-updates-2017-Q2-launch

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index f583839755..b9cafe3d40 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -98,6 +98,7 @@ matrix:
       sudo: required
       addons:
       dist: trusty
+      group: deprecated-2017Q2
       compiler: gcc
       before_install:
         - sudo apt-get update -qq
@@ -109,6 +110,7 @@ matrix:
       sudo: required
       addons:
       dist: trusty
+      group: deprecated-2017Q2
       compiler: gcc
       before_install:
         - sudo apt-get update -qq
@@ -119,6 +121,7 @@ matrix:
     - sudo: required
       addons:
       dist: trusty
+      group: deprecated-2017Q2
       language: generic
       compiler: none
       env:
@@ -138,6 +141,7 @@ matrix:
     - sudo: required
       addons:
       dist: trusty
+      group: deprecated-2017Q2
       language: generic
       compiler: none
       env:
-- 
2.13.3

[Qemu-devel] [PATCH for-2.11 v3 02/16] travis: install more library dependencies

[Philippe Mathieu-Daudé]

From: Peter Maydell <peter.maydell@linaro.org>

Update the travis list of library packages to install so that
our build tests cover more of our code base.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
[PMD: added more dependencies:
 glusterfs-common, libbz2-dev, libncursesw5-dev, libnfs-dev]
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
---
 .travis.yml | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index b9cafe3d40..c8fecb584a 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -9,30 +9,43 @@ addons:
   apt:
     packages:
       # Build dependencies
+      - glusterfs-common
       - libaio-dev
       - libattr1-dev
+      - libbluetooth-dev
       - libbrlapi-dev
+      - libbz2-dev
+      - libcap-dev
       - libcap-ng-dev
       - libgnutls-dev
       - libgtk-3-dev
       - libiscsi-dev
       - liblttng-ust-dev
+      - liblzo2-dev
       - libnfs-dev
       - libncurses5-dev
+      - libncursesw5-dev
+      - libnfs-dev
       - libnss3-dev
       - libpixman-1-dev
       - libpng12-dev
       - librados-dev
+      - librdmacm-dev
       - libsdl1.2-dev
       - libseccomp-dev
+      - libsnappy-dev
       - libspice-protocol-dev
       - libspice-server-dev
       - libssh2-1-dev
       - liburcu-dev
       - libusb-1.0-0-dev
+      - libvde-dev
       - libvte-2.90-dev
+      - libxen-dev
+      - nettle-dev
       - sparse
       - uuid-dev
+      - xfslibs-dev
 
 # The channel name "irc.oftc.net#qemu" is encrypted against qemu/qemu
 # to prevent IRC notifications from forks. This was created using:
@@ -168,30 +181,43 @@ matrix:
             - gcc-5
             - g++-5
             # Build dependencies
+            - glusterfs-common
             - libaio-dev
             - libattr1-dev
+            - libbluetooth-dev
             - libbrlapi-dev
+            - libbz2-dev
+            - libcap-dev
             - libcap-ng-dev
             - libgnutls-dev
             - libgtk-3-dev
             - libiscsi-dev
             - liblttng-ust-dev
+            - liblzo2-dev
             - libnfs-dev
             - libncurses5-dev
+            - libncursesw5-dev
+            - libnfs-dev
             - libnss3-dev
             - libpixman-1-dev
             - libpng12-dev
             - librados-dev
+            - librdmacm-dev
             - libsdl1.2-dev
             - libseccomp-dev
+            - libsnappy-dev
             - libspice-protocol-dev
             - libspice-server-dev
             - libssh2-1-dev
             - liburcu-dev
             - libusb-1.0-0-dev
+            - libvde-dev
             - libvte-2.90-dev
+            - libxen-dev
+            - nettle-dev
             - sparse
             - uuid-dev
+            - xfslibs-dev
       language: generic
       compiler: none
       env:
-- 
2.13.3

[Qemu-devel] [PATCH for-2.11 v3 03/16] travis: enable multiple caching features

[Philippe Mathieu-Daudé]

(this eases git workflow)

see https://docs.travis-ci.com/user/caching#enabling-multiple-caching-features

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
 .travis.yml | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index c8fecb584a..311f25eb28 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -4,7 +4,8 @@ python:
   - "2.4"
 compiler:
   - gcc
-cache: ccache
+cache:
+  ccache: true
 addons:
   apt:
     packages:
-- 
2.13.3

[Qemu-devel] [PATCH for-2.11 v3 04/16] travis: increase S3 cache timeout

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
 .travis.yml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.travis.yml b/.travis.yml
index 311f25eb28..d622d8c433 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -6,6 +6,7 @@ compiler:
   - gcc
 cache:
   ccache: true
+  timeout: 1200 # https://docs.travis-ci.com/user/caching#setting-the-timeout
 addons:
   apt:
     packages:
-- 
2.13.3

[Qemu-devel] [PATCH for-2.11 v3 05/16] travis: retry if llvm.org timeouts

[Philippe Mathieu-Daudé]

example of failure: https://travis-ci.org/qemu/qemu/jobs/243232857

    $ sudo apt-get update -qq
    W: Failed to fetch http://llvm.org/apt/trusty/dists/llvm-toolchain-trusty-3.9/Release.gpg  Connection failed
    E: Some index files failed to download. They have been ignored, or old ones used instead.
    The command "sudo apt-get update -qq" failed and exited with 100 during .
    Your build has been stopped.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
 .travis.yml | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index d622d8c433..82c1819c93 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -143,10 +143,10 @@ matrix:
         - COMPILER_NAME=clang CXX=clang++-3.9 CC=clang-3.9
         - CONFIG="--disable-linux-user --cc=clang-3.9 --cxx=clang++-3.9"
       before_install:
-        - wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
-        - sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
-        - sudo apt-get update -qq
-        - sudo apt-get install -qq -y clang-3.9
+        - travis_retry wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
+        - travis_retry sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
+        - travis_retry sudo apt-get update -qq
+        - travis_retry sudo apt-get install -qq -y clang-3.9
         - sudo apt-get build-dep -qq qemu
         - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
         - git submodule update --init --recursive
@@ -163,10 +163,10 @@ matrix:
         - COMPILER_NAME=clang CXX=clang++-3.9 CC=clang-3.9
         - CONFIG="--disable-system --cc=clang-3.9 --cxx=clang++-3.9"
       before_install:
-        - wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
-        - sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
-        - sudo apt-get update -qq
-        - sudo apt-get install -qq -y clang-3.9
+        - travis_retry wget -nv -O - http://llvm.org/apt/llvm-snapshot.gpg.key | sudo apt-key add -
+        - travis_retry sudo apt-add-repository -y 'deb http://llvm.org/apt/trusty llvm-toolchain-trusty-3.9 main'
+        - travis_retry sudo apt-get update -qq
+        - travis_retry sudo apt-get install -qq -y clang-3.9
         - sudo apt-get build-dep -qq qemu
         - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
         - git submodule update --init --recursive
-- 
2.13.3

[Qemu-devel] [PATCH for-2.11 v3 06/16] travis: retry when git submodules initialization fails

[Philippe Mathieu-Daudé]

example of failure: https://travis-ci.org/philmd/qemu/jobs/245612939

  $ git submodule update --init --recursive
  [...]
  Submodule 'pixman' (git://anongit.freedesktop.org/pixman) registered for path 'pixman'
  Cloning into 'pixman'...
  fatal: unable to connect to anongit.freedesktop.org:
  anongit.freedesktop.org[0: 131.252.210.161]: errno=Connection timed out
  anongit.freedesktop.org[1: 2610:10:20:722:a800:ff:fe24:61cf]: errno=Network is unreachable
  Clone of 'git://anongit.freedesktop.org/pixman' into submodule path 'pixman' failed
  The command "git submodule update --init --recursive" failed and exited with 1 during .
  Your build has been stopped.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 82c1819c93..196412f5be 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -76,7 +76,7 @@ before_install:
   - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update ; fi
   - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
   - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
-  - git submodule update --init --recursive
+  - travis_retry git submodule update --init --recursive
 before_script:
   - ./configure ${CONFIG}
 script:
@@ -119,7 +119,7 @@ matrix:
         - sudo apt-get update -qq
         - sudo apt-get build-dep -qq qemu
         - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
-        - git submodule update --init --recursive
+        - travis_retry git submodule update --init --recursive
     # Plain Trusty Linux User Build
     - env: CONFIG="--disable-system"
       sudo: required
@@ -131,7 +131,7 @@ matrix:
         - sudo apt-get update -qq
         - sudo apt-get build-dep -qq qemu
         - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
-        - git submodule update --init --recursive
+        - travis_retry git submodule update --init --recursive
     # Trusty System build with latest stable clang
     - sudo: required
       addons:
@@ -149,7 +149,7 @@ matrix:
         - travis_retry sudo apt-get install -qq -y clang-3.9
         - sudo apt-get build-dep -qq qemu
         - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
-        - git submodule update --init --recursive
+        - travis_retry git submodule update --init --recursive
       before_script:
         - ./configure ${CONFIG} || cat config.log
     # Trusty Linux User build with latest stable clang
@@ -169,7 +169,7 @@ matrix:
         - travis_retry sudo apt-get install -qq -y clang-3.9
         - sudo apt-get build-dep -qq qemu
         - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
-        - git submodule update --init --recursive
+        - travis_retry git submodule update --init --recursive
       before_script:
         - ./configure ${CONFIG} || cat config.log
     # Using newer GCC with sanitizers
-- 
2.13.3

[Qemu-devel] [PATCH for-2.11 v3 07/16] travis: split the gprof/gcov job

[Philippe Mathieu-Daudé]

gcov generates lot of output and often trigger job limit timeout

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
 .travis.yml | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index 196412f5be..e56c928bef 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -87,7 +87,9 @@ matrix:
     - env: CONFIG=""
       compiler: clang
     # gprof/gcov are GCC features
-    - env: CONFIG="--enable-gprof --enable-gcov --disable-pie"
+    - env: CONFIG="--enable-gprof --disable-pie"
+      compiler: gcc
+    - env: CONFIG="--enable-gcov --disable-pie"
       compiler: gcc
     # We manually include builds which we disable "make check" for
     - env: CONFIG="--enable-debug --enable-tcg-interpreter"
-- 
2.13.3

[Qemu-devel] [PATCH for-2.11 v3 08/16] travis: reorder matrix to speedup failure

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 36 +++++++++++++++++++-----------------
 1 file changed, 19 insertions(+), 17 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index e56c928bef..57462b0471 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -64,11 +64,11 @@ env:
     - MAKEFLAGS="-j3"
   matrix:
     - CONFIG=""
-    - CONFIG="--enable-debug --enable-debug-tcg --enable-trace-backends=log"
     - CONFIG="--disable-linux-aio --disable-cap-ng --disable-attr --disable-brlapi --disable-uuid --disable-libusb"
     - CONFIG="--enable-modules"
     - CONFIG="--with-coroutine=ucontext"
     - CONFIG="--with-coroutine=sigaltstack"
+    - CONFIG="--enable-debug --enable-debug-tcg --enable-trace-backends=log"
 git:
   # we want to do this ourselves
   submodules: false
@@ -83,30 +83,32 @@ script:
   - make ${MAKEFLAGS} && ${TEST_CMD}
 matrix:
   include:
+    # We manually include builds which we disable "make check" for, they also
+    # are the faster jobs (no testing).
+    - env: CONFIG="--disable-tcg"
+           TEST_CMD=""
+      compiler: gcc
+    - env: CONFIG="--enable-trace-backends=ust"
+           TEST_CMD=""
+      compiler: gcc
+    - env: CONFIG="--enable-debug --enable-tcg-interpreter"
+           TEST_CMD=""
+      compiler: gcc
+    - env: CONFIG="--enable-trace-backends=simple"
+           TEST_CMD=""
+      compiler: gcc
+    - env: CONFIG="--enable-trace-backends=ftrace"
+           TEST_CMD=""
+      compiler: gcc
     # Test with CLang for compile portability
     - env: CONFIG=""
       compiler: clang
+
     # gprof/gcov are GCC features
     - env: CONFIG="--enable-gprof --disable-pie"
       compiler: gcc
     - env: CONFIG="--enable-gcov --disable-pie"
       compiler: gcc
-    # We manually include builds which we disable "make check" for
-    - env: CONFIG="--enable-debug --enable-tcg-interpreter"
-           TEST_CMD=""
-      compiler: gcc
-    - env: CONFIG="--enable-trace-backends=simple"
-           TEST_CMD=""
-      compiler: gcc
-    - env: CONFIG="--enable-trace-backends=ftrace"
-           TEST_CMD=""
-      compiler: gcc
-    - env: CONFIG="--enable-trace-backends=ust"
-           TEST_CMD=""
-      compiler: gcc
-    - env: CONFIG="--disable-tcg"
-           TEST_CMD=""
-      compiler: gcc
     - env: CONFIG=""
       os: osx
       compiler: clang
-- 
2.13.3

[Qemu-devel] [PATCH for-2.11 v3 09/16] travis: reduce git clone depth

[Philippe Mathieu-Daudé]

see https://docs.travis-ci.com/user/customizing-the-build#Git-Clone-Depth

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.travis.yml b/.travis.yml
index 57462b0471..72aad5ca35 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -70,6 +70,7 @@ env:
     - CONFIG="--with-coroutine=sigaltstack"
     - CONFIG="--enable-debug --enable-debug-tcg --enable-trace-backends=log"
 git:
+  depth: 3
   # we want to do this ourselves
   submodules: false
 before_install:
-- 
2.13.3

[Qemu-devel] [PATCH for-2.11 v3 10/16] travis: check ./configure outcome, dump config.log on failure

[Philippe Mathieu-Daudé]

also de-duplicate 'before_script'

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 72aad5ca35..88ca78c775 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -79,7 +79,7 @@ before_install:
   - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
   - travis_retry git submodule update --init --recursive
 before_script:
-  - ./configure ${CONFIG}
+  - ./configure ${CONFIG} || (cat config.log; exit 1)
 script:
   - make ${MAKEFLAGS} && ${TEST_CMD}
 matrix:
@@ -155,8 +155,6 @@ matrix:
         - sudo apt-get build-dep -qq qemu
         - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
         - travis_retry git submodule update --init --recursive
-      before_script:
-        - ./configure ${CONFIG} || cat config.log
     # Trusty Linux User build with latest stable clang
     - sudo: required
       addons:
@@ -175,8 +173,6 @@ matrix:
         - sudo apt-get build-dep -qq qemu
         - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
         - travis_retry git submodule update --init --recursive
-      before_script:
-        - ./configure ${CONFIG} || cat config.log
     # Using newer GCC with sanitizers
     - addons:
         apt:
@@ -232,4 +228,4 @@ matrix:
         - CONFIG="--cc=gcc-5 --cxx=g++-5 --disable-pie --disable-linux-user"
         - TEST_CMD=""
       before_script:
-        - ./configure ${CONFIG} --extra-cflags="-g3 -O0 -fsanitize=thread -fuse-ld=gold" || cat config.log
+        - ./configure ${CONFIG} --extra-cflags="-g3 -O0 -fsanitize=thread -fuse-ld=gold" || (cat config.log; exit 1)
-- 
2.13.3

[Qemu-devel] [PATCH for-2.11 v3 11/16] travis/osx: use readable YAML

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 88ca78c775..4bf69b0116 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -74,8 +74,10 @@ git:
   # we want to do this ourselves
   submodules: false
 before_install:
-  - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update ; fi
-  - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
+  - if [ "$TRAVIS_OS_NAME" == "osx" ]; then
+        brew update ;
+        brew install libffi gettext glib pixman ;
+    fi
   - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
   - travis_retry git submodule update --init --recursive
 before_script:
-- 
2.13.3

[Qemu-devel] [PATCH for-2.11 v3 12/16] travis/osx: silent texinfo warnings

[Philippe Mathieu-Daudé]

  $ make info
    GEN     qemu-doc.html
  qemu-doc.texi:8: warning: unrecognized encoding name `UTF-8'.
    GEN     qemu-doc.txt
  qemu-doc.texi:8: warning: unrecognized encoding name `UTF-8'.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index 4bf69b0116..24f62fb7cf 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -74,9 +74,11 @@ git:
   # we want to do this ourselves
   submodules: false
 before_install:
+  # silent texinfo warnings (https://github.com/xiaohanyu/oh-my-emacs/blob/c664894e2f1c1cb0f95a9f2da88d41b00f190856/core/ome-basic.org#homebrew)
   - if [ "$TRAVIS_OS_NAME" == "osx" ]; then
         brew update ;
-        brew install libffi gettext glib pixman ;
+        brew install libffi gettext glib pixman texinfo ;
+        brew link texinfo --force ;
     fi
   - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
   - travis_retry git submodule update --init --recursive
-- 
2.13.3

[Qemu-devel] [RFC PATCH for-2.11 v3 13/16] travis: use gcc-6 sanitizers

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 24f62fb7cf..1da542f99f 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -185,8 +185,8 @@ matrix:
             - ubuntu-toolchain-r-test
           packages:
             # Extra toolchains
-            - gcc-5
-            - g++-5
+            - gcc-6
+            - g++-6
             # Build dependencies
             - glusterfs-common
             - libaio-dev
@@ -228,8 +228,8 @@ matrix:
       language: generic
       compiler: none
       env:
-        - COMPILER_NAME=gcc CXX=g++-5 CC=gcc-5
-        - CONFIG="--cc=gcc-5 --cxx=g++-5 --disable-pie --disable-linux-user"
+        - COMPILER_NAME=gcc CXX=g++-6 CC=gcc-6
+        - CONFIG="--disable-pie --disable-linux-user"
         - TEST_CMD=""
       before_script:
         - ./configure ${CONFIG} --extra-cflags="-g3 -O0 -fsanitize=thread -fuse-ld=gold" || (cat config.log; exit 1)
-- 
2.13.3

[Qemu-devel] [RFC PATCH for-2.11 v3 14/16] travis: allow_failures for gprof/gcov

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index 1da542f99f..7c93a10c5f 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -87,6 +87,7 @@ before_script:
 script:
   - make ${MAKEFLAGS} && ${TEST_CMD}
 matrix:
+  fast_finish: true
   include:
     # We manually include builds which we disable "make check" for, they also
     # are the faster jobs (no testing).
@@ -233,3 +234,8 @@ matrix:
         - TEST_CMD=""
       before_script:
         - ./configure ${CONFIG} --extra-cflags="-g3 -O0 -fsanitize=thread -fuse-ld=gold" || (cat config.log; exit 1)
+  allow_failures:
+    - env: CONFIG="--enable-gprof --disable-pie"
+      compiler: gcc
+    - env: CONFIG="--enable-gcov --disable-pie"
+      compiler: gcc
-- 
2.13.3

[Qemu-devel] [RFC PATCH for-2.11 v3 15/16] travis/osx: build using more Xcode versions

[Philippe Mathieu-Daudé]

currently default builder is based on Xcode 7.3.1:
    OS X 10.11 (darwin14.5.0)
    Apple LLVM version 7.3.0

add the oldest available Xcode (6.4):
    OS X 10.10 (darwin14.5.0)
    Apple LLVM 6.1.0 (based on LLVM 3.6.0svn)

and the newer available Xcode (8.3.3):
    OS X 10.12 (darwin16.6.0)
    Apple LLVM version 8.1.0

see https://docs.travis-ci.com/user/reference/osx/#OS-X-Version

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index 7c93a10c5f..cd59570062 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -117,6 +117,18 @@ matrix:
       compiler: gcc
     - env: CONFIG=""
       os: osx
+      # older available: OS X 10.10
+      osx_image: xcode6.4
+      compiler: clang
+    - env: CONFIG=""
+      os: osx
+      # default: OS X 10.11.6
+      osx_image: xcode7.3
+      compiler: clang
+    - env: CONFIG=""
+      os: osx
+      # newer stable available, OS X 10.12
+      osx_image: xcode8.3
       compiler: clang
     # Plain Trusty System Build
     - env: CONFIG="--disable-linux-user"
-- 
2.13.3

[Qemu-devel] [NOTFORMERGE PATCH for-2.11 v3 16/16] travis/osx: build using bleeding Xcode

[Philippe Mathieu-Daudé]

test 'unstable' Xcode (9beta4) ...
    OS X 10.13 (darwin16.7.0)
    Apple LLVM version 9.0.0 (clang-900.0.31)

see https://docs.travis-ci.com/user/reference/osx/#OS-X-Version

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index cd59570062..16c1dee150 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -130,6 +130,11 @@ matrix:
       # newer stable available, OS X 10.12
       osx_image: xcode8.3
       compiler: clang
+    - env: CONFIG=""
+      os: osx
+      # unstable
+      osx_image: xcode9
+      compiler: clang
     # Plain Trusty System Build
     - env: CONFIG="--disable-linux-user"
       sudo: required
-- 
2.13.3

Re: [Qemu-devel] [PATCH for-2.11 v3 01/16] travis: update sudo-enabled Trusty images

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> see https://blog.travis-ci.com/2017-06-19-trusty-updates-2017-Q2
> and
> https://blog.travis-ci.com/2017-06-21-trusty-updates-2017-Q2-launch

I guess we drop this now? Either we are working or we are not :-/

>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>  .travis.yml | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/.travis.yml b/.travis.yml
> index f583839755..b9cafe3d40 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -98,6 +98,7 @@ matrix:
>        sudo: required
>        addons:
>        dist: trusty
> +      group: deprecated-2017Q2
>        compiler: gcc
>        before_install:
>          - sudo apt-get update -qq
> @@ -109,6 +110,7 @@ matrix:
>        sudo: required
>        addons:
>        dist: trusty
> +      group: deprecated-2017Q2
>        compiler: gcc
>        before_install:
>          - sudo apt-get update -qq
> @@ -119,6 +121,7 @@ matrix:
>      - sudo: required
>        addons:
>        dist: trusty
> +      group: deprecated-2017Q2
>        language: generic
>        compiler: none
>        env:
> @@ -138,6 +141,7 @@ matrix:
>      - sudo: required
>        addons:
>        dist: trusty
> +      group: deprecated-2017Q2
>        language: generic
>        compiler: none
>        env:


--
Alex Bennée

Re: [Qemu-devel] [PATCH for-2.11 v3 08/16] travis: reorder matrix to speedup failure

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

> ---
>  .travis.yml | 36 +++++++++++++++++++-----------------
>  1 file changed, 19 insertions(+), 17 deletions(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index e56c928bef..57462b0471 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -64,11 +64,11 @@ env:
>      - MAKEFLAGS="-j3"
>    matrix:
>      - CONFIG=""
> -    - CONFIG="--enable-debug --enable-debug-tcg --enable-trace-backends=log"
>      - CONFIG="--disable-linux-aio --disable-cap-ng --disable-attr --disable-brlapi --disable-uuid --disable-libusb"
>      - CONFIG="--enable-modules"
>      - CONFIG="--with-coroutine=ucontext"
>      - CONFIG="--with-coroutine=sigaltstack"
> +    - CONFIG="--enable-debug --enable-debug-tcg --enable-trace-backends=log"
>  git:
>    # we want to do this ourselves
>    submodules: false
> @@ -83,30 +83,32 @@ script:
>    - make ${MAKEFLAGS} && ${TEST_CMD}
>  matrix:
>    include:
> +    # We manually include builds which we disable "make check" for, they also
> +    # are the faster jobs (no testing).
> +    - env: CONFIG="--disable-tcg"
> +           TEST_CMD=""
> +      compiler: gcc
> +    - env: CONFIG="--enable-trace-backends=ust"
> +           TEST_CMD=""
> +      compiler: gcc
> +    - env: CONFIG="--enable-debug --enable-tcg-interpreter"
> +           TEST_CMD=""
> +      compiler: gcc
> +    - env: CONFIG="--enable-trace-backends=simple"
> +           TEST_CMD=""
> +      compiler: gcc
> +    - env: CONFIG="--enable-trace-backends=ftrace"
> +           TEST_CMD=""
> +      compiler: gcc
>      # Test with CLang for compile portability
>      - env: CONFIG=""
>        compiler: clang
> +
>      # gprof/gcov are GCC features
>      - env: CONFIG="--enable-gprof --disable-pie"
>        compiler: gcc
>      - env: CONFIG="--enable-gcov --disable-pie"
>        compiler: gcc
> -    # We manually include builds which we disable "make check" for
> -    - env: CONFIG="--enable-debug --enable-tcg-interpreter"
> -           TEST_CMD=""
> -      compiler: gcc
> -    - env: CONFIG="--enable-trace-backends=simple"
> -           TEST_CMD=""
> -      compiler: gcc
> -    - env: CONFIG="--enable-trace-backends=ftrace"
> -           TEST_CMD=""
> -      compiler: gcc
> -    - env: CONFIG="--enable-trace-backends=ust"
> -           TEST_CMD=""
> -      compiler: gcc
> -    - env: CONFIG="--disable-tcg"
> -           TEST_CMD=""
> -      compiler: gcc
>      - env: CONFIG=""
>        os: osx
>        compiler: clang


--
Alex Bennée

Re: [Qemu-devel] [PATCH for-2.11 v3 10/16] travis: check ./configure outcome, dump config.log on failure

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> also de-duplicate 'before_script'
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

> ---
>  .travis.yml | 8 ++------
>  1 file changed, 2 insertions(+), 6 deletions(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index 72aad5ca35..88ca78c775 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -79,7 +79,7 @@ before_install:
>    - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
>    - travis_retry git submodule update --init --recursive
>  before_script:
> -  - ./configure ${CONFIG}
> +  - ./configure ${CONFIG} || (cat config.log; exit 1)
>  script:
>    - make ${MAKEFLAGS} && ${TEST_CMD}
>  matrix:
> @@ -155,8 +155,6 @@ matrix:
>          - sudo apt-get build-dep -qq qemu
>          - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
>          - travis_retry git submodule update --init --recursive
> -      before_script:
> -        - ./configure ${CONFIG} || cat config.log
>      # Trusty Linux User build with latest stable clang
>      - sudo: required
>        addons:
> @@ -175,8 +173,6 @@ matrix:
>          - sudo apt-get build-dep -qq qemu
>          - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
>          - travis_retry git submodule update --init --recursive
> -      before_script:
> -        - ./configure ${CONFIG} || cat config.log
>      # Using newer GCC with sanitizers
>      - addons:
>          apt:
> @@ -232,4 +228,4 @@ matrix:
>          - CONFIG="--cc=gcc-5 --cxx=g++-5 --disable-pie --disable-linux-user"
>          - TEST_CMD=""
>        before_script:
> -        - ./configure ${CONFIG} --extra-cflags="-g3 -O0 -fsanitize=thread -fuse-ld=gold" || cat config.log
> +        - ./configure ${CONFIG} --extra-cflags="-g3 -O0 -fsanitize=thread -fuse-ld=gold" || (cat config.log; exit 1)


--
Alex Bennée

Re: [Qemu-devel] [RFC PATCH for-2.11 v3 14/16] travis: allow_failures for gprof/gcov

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

why? We should document the reason in the commit.

> ---
>  .travis.yml | 6 ++++++
>  1 file changed, 6 insertions(+)
>
> diff --git a/.travis.yml b/.travis.yml
> index 1da542f99f..7c93a10c5f 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -87,6 +87,7 @@ before_script:
>  script:
>    - make ${MAKEFLAGS} && ${TEST_CMD}
>  matrix:
> +  fast_finish: true
>    include:
>      # We manually include builds which we disable "make check" for, they also
>      # are the faster jobs (no testing).
> @@ -233,3 +234,8 @@ matrix:
>          - TEST_CMD=""
>        before_script:
>          - ./configure ${CONFIG} --extra-cflags="-g3 -O0 -fsanitize=thread -fuse-ld=gold" || (cat config.log; exit 1)
> +  allow_failures:
> +    - env: CONFIG="--enable-gprof --disable-pie"
> +      compiler: gcc
> +    - env: CONFIG="--enable-gcov --disable-pie"
> +      compiler: gcc


--
Alex Bennée

Re: [Qemu-devel] [RFC PATCH for-2.11 v3 15/16] travis/osx: build using more Xcode versions

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> currently default builder is based on Xcode 7.3.1:
>     OS X 10.11 (darwin14.5.0)
>     Apple LLVM version 7.3.0
>
> add the oldest available Xcode (6.4):
>     OS X 10.10 (darwin14.5.0)
>     Apple LLVM 6.1.0 (based on LLVM 3.6.0svn)
>
> and the newer available Xcode (8.3.3):
>     OS X 10.12 (darwin16.6.0)
>     Apple LLVM version 8.1.0

I reckon oldest stable and newest stable is enough for coverage.

>
> see https://docs.travis-ci.com/user/reference/osx/#OS-X-Version
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>  .travis.yml | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
>
> diff --git a/.travis.yml b/.travis.yml
> index 7c93a10c5f..cd59570062 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -117,6 +117,18 @@ matrix:
>        compiler: gcc
>      - env: CONFIG=""
>        os: osx
> +      # older available: OS X 10.10
> +      osx_image: xcode6.4
> +      compiler: clang
> +    - env: CONFIG=""
> +      os: osx
> +      # default: OS X 10.11.6
> +      osx_image: xcode7.3
> +      compiler: clang
> +    - env: CONFIG=""
> +      os: osx
> +      # newer stable available, OS X 10.12
> +      osx_image: xcode8.3
>        compiler: clang
>      # Plain Trusty System Build
>      - env: CONFIG="--disable-linux-user"


--
Alex Bennée

Re: [Qemu-devel] [PATCH for-2.11 v3 12/16] travis/osx: silent texinfo warnings

[Philippe Mathieu-Daudé]

Hey Alex, Peter, what do you think about this patch?

On Wed, Aug 9, 2017 at 10:27 PM Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
>
>   $ make info
>     GEN     qemu-doc.html
>   qemu-doc.texi:8: warning: unrecognized encoding name `UTF-8'.
>     GEN     qemu-doc.txt
>   qemu-doc.texi:8: warning: unrecognized encoding name `UTF-8'.
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>  .travis.yml | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index 4bf69b0116..24f62fb7cf 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -74,9 +74,11 @@ git:
>    # we want to do this ourselves
>    submodules: false
>  before_install:
> +  # silent texinfo warnings (https://github.com/xiaohanyu/oh-my-emacs/blob/c664894e2f1c1cb0f95a9f2da88d41b00f190856/core/ome-basic.org#homebrew)
>    - if [ "$TRAVIS_OS_NAME" == "osx" ]; then
>          brew update ;
> -        brew install libffi gettext glib pixman ;
> +        brew install libffi gettext glib pixman texinfo ;
> +        brew link texinfo --force ;
>      fi
>    - wget -O - http://people.linaro.org/~alex.bennee/qemu-submodule-git-seed.tar.xz | tar -xvJ
>    - travis_retry git submodule update --init --recursive
> --
> 2.13.3
>

Re: [Qemu-devel] [PATCH for-2.11 v3 12/16] travis/osx: silent texinfo warnings

[Peter Maydell]

On Wed, 19 Dec 2018 at 21:34, Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
>
> Hey Alex, Peter, what do you think about this patch?
>
> On Wed, Aug 9, 2017 at 10:27 PM Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
> >
> >   $ make info
> >     GEN     qemu-doc.html
> >   qemu-doc.texi:8: warning: unrecognized encoding name `UTF-8'.
> >     GEN     qemu-doc.txt
> >   qemu-doc.texi:8: warning: unrecognized encoding name `UTF-8'.
> >
> > Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> > ---
> >  .travis.yml | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/.travis.yml b/.travis.yml
> > index 4bf69b0116..24f62fb7cf 100644
> > --- a/.travis.yml
> > +++ b/.travis.yml
> > @@ -74,9 +74,11 @@ git:
> >    # we want to do this ourselves
> >    submodules: false
> >  before_install:
> > +  # silent texinfo warnings (https://github.com/xiaohanyu/oh-my-emacs/blob/c664894e2f1c1cb0f95a9f2da88d41b00f190856/core/ome-basic.org#homebrew)
> >    - if [ "$TRAVIS_OS_NAME" == "osx" ]; then
> >          brew update ;
> > -        brew install libffi gettext glib pixman ;
> > +        brew install libffi gettext glib pixman texinfo ;
> > +        brew link texinfo --force ;
> >      fi

The commit message is a bit lacking in detail about how it's
fixing things. IIRC this message is caused because the stock
OSX texinfo is too old, so installing a newer version via
brew makes sense. But what does the "brew link" command do?

thanks
-- PMM

Re: [Qemu-devel] [PATCH for-2.11 v3 12/16] travis/osx: silent texinfo warnings

[Philippe Mathieu-Daudé]

On Wed, Dec 19, 2018 at 11:42 PM Peter Maydell <peter.maydell@linaro.org> wrote:
> On Wed, 19 Dec 2018 at 21:34, Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
> >
> > Hey Alex, Peter, what do you think about this patch?
> >
> > On Wed, Aug 9, 2017 at 10:27 PM Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
> > >
> > >   $ make info
> > >     GEN     qemu-doc.html
> > >   qemu-doc.texi:8: warning: unrecognized encoding name `UTF-8'.
> > >     GEN     qemu-doc.txt
> > >   qemu-doc.texi:8: warning: unrecognized encoding name `UTF-8'.
> > >
> > > Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> > > ---
> > >  .travis.yml | 4 +++-
> > >  1 file changed, 3 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/.travis.yml b/.travis.yml
> > > index 4bf69b0116..24f62fb7cf 100644
> > > --- a/.travis.yml
> > > +++ b/.travis.yml
> > > @@ -74,9 +74,11 @@ git:
> > >    # we want to do this ourselves
> > >    submodules: false
> > >  before_install:
> > > +  # silent texinfo warnings (https://github.com/xiaohanyu/oh-my-emacs/blob/c664894e2f1c1cb0f95a9f2da88d41b00f190856/core/ome-basic.org#homebrew)
> > >    - if [ "$TRAVIS_OS_NAME" == "osx" ]; then
> > >          brew update ;
> > > -        brew install libffi gettext glib pixman ;
> > > +        brew install libffi gettext glib pixman texinfo ;
> > > +        brew link texinfo --force ;
> > >      fi
>
> The commit message is a bit lacking in detail about how it's
> fixing things. IIRC this message is caused because the stock
> OSX texinfo is too old, so installing a newer version via
> brew makes sense. But what does the "brew link" command do?

This was more than 1 year ago, now I try/force myself to write more
verbose commit message...
I don't remember exactly why I added the 'link' and will have to check
again (another thing I learned, write commit message for your self,
else you won't remember in 1 year ahead...).
I'm trying to reduce OSX warnings, as of current QEMU master I count
80 warnings using Xcode 10, which result in me not looking at them (or
worst, at new ones...).
I'll include this patch reworded in an OSX cleanup series.
Thanks,
Phil.

Re: [Qemu-devel] [PATCH for-2.11 v3 12/16] travis/osx: silent texinfo warnings

[Peter Maydell]

On Wed, 19 Dec 2018 at 23:13, Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
> I'm trying to reduce OSX warnings, as of current QEMU master I count
> 80 warnings using Xcode 10, which result in me not looking at them (or
> worst, at new ones...).

A lot of them are the address-of-packed-member warnings, which
I've been gradually looking at -- I think we're now down to
just the slirp ones, which I've been leaving until the big
refactor that's pending has landed.
Other than that there are a bunch of deprecated-declaration
warnings about SASL, which I never figured out an approach
for and so just suppress in my "grep logs for warnings"
script. I don't think there's too many other than that?

thanks
-- PMM