[Qemu-devel] [PULL 0/5] Docker testing patches

[Qemu-devel] [PULL 0/5] Docker testing patches

[Fam Zheng]

The following changes since commit ab161529261928ae7f3556e3220829c34b2686ec:

  Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20170927a' into staging (2017-09-27 22:44:51 +0100)

are available in the git repository at:

  git://github.com/famz/qemu.git tags/docker-testing-pull-request

for you to fetch changes up to 36ac78e65a0695f0e397865cceaf4ab5c4e52c64:

  docker: Don't mount ccache db if NOUSER=1 (2017-09-29 11:14:15 +0800)

----------------------------------------------------------------

----------------------------------------------------------------

Alex Bennée (1):
  tests/docker/run: don't source /etc/profile

Fam Zheng (3):
  docker: Fix test-mingw
  docker: test-block: Don't continue if build fails
  docker: Don't mount ccache db if NOUSER=1

Paolo Bonzini (1):
  docker: add installation to build tests

 tests/docker/Makefile.include                | 6 ++++--
 tests/docker/common.rc                       | 8 ++++++++
 tests/docker/dockerfiles/centos6.docker      | 1 +
 tests/docker/dockerfiles/centos7.docker      | 1 +
 tests/docker/dockerfiles/debian-ports.docker | 1 +
 tests/docker/dockerfiles/debian8.docker      | 1 +
 tests/docker/dockerfiles/debian9.docker      | 1 +
 tests/docker/dockerfiles/fedora.docker       | 2 +-
 tests/docker/dockerfiles/min-glib.docker     | 2 +-
 tests/docker/dockerfiles/ubuntu.docker       | 2 +-
 tests/docker/run                             | 4 +++-
 tests/docker/test-block                      | 2 +-
 tests/docker/test-build                      | 1 +
 tests/docker/test-clang                      | 1 +
 tests/docker/test-full                       | 2 +-
 tests/docker/test-mingw                      | 1 +
 tests/docker/test-quick                      | 1 +
 17 files changed, 29 insertions(+), 8 deletions(-)

-- 
2.13.5

[Qemu-devel] [PULL 1/5] docker: add installation to build tests

[Fam Zheng]

From: Paolo Bonzini <pbonzini@redhat.com>

Basic test that "make install" works; this requires msgfmt so add
gettext to the packages.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <1506095371-23160-1-git-send-email-pbonzini@redhat.com>
[Rebase to master. - Fam]
Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/common.rc                       | 8 ++++++++
 tests/docker/dockerfiles/centos6.docker      | 1 +
 tests/docker/dockerfiles/centos7.docker      | 1 +
 tests/docker/dockerfiles/debian-ports.docker | 1 +
 tests/docker/dockerfiles/debian8.docker      | 1 +
 tests/docker/dockerfiles/debian9.docker      | 1 +
 tests/docker/dockerfiles/fedora.docker       | 2 +-
 tests/docker/dockerfiles/min-glib.docker     | 2 +-
 tests/docker/dockerfiles/ubuntu.docker       | 2 +-
 tests/docker/test-build                      | 1 +
 tests/docker/test-clang                      | 1 +
 tests/docker/test-full                       | 2 +-
 tests/docker/test-mingw                      | 1 +
 tests/docker/test-quick                      | 1 +
 14 files changed, 21 insertions(+), 4 deletions(-)

diff --git a/tests/docker/common.rc b/tests/docker/common.rc
index 87f5263757..7951555e3f 100755
--- a/tests/docker/common.rc
+++ b/tests/docker/common.rc
@@ -44,3 +44,11 @@ prep_fail()
     echo "$@"
     exit 2
 }
+
+install_qemu()
+{
+    make install $MAKEFLAGS DESTDIR=$PWD/=destdir
+    ret=$?
+    rm -rf $PWD/=destdir
+    return $ret
+}
diff --git a/tests/docker/dockerfiles/centos6.docker b/tests/docker/dockerfiles/centos6.docker
index f6aae13f29..ad24319582 100644
--- a/tests/docker/dockerfiles/centos6.docker
+++ b/tests/docker/dockerfiles/centos6.docker
@@ -8,6 +8,7 @@ ENV PACKAGES \
     flex \
     g++ \
     gcc \
+    gettext \
     git \
     glib2-devel \
     libepoxy-devel \
diff --git a/tests/docker/dockerfiles/centos7.docker b/tests/docker/dockerfiles/centos7.docker
index 0b59aa2f26..575de29a0a 100644
--- a/tests/docker/dockerfiles/centos7.docker
+++ b/tests/docker/dockerfiles/centos7.docker
@@ -9,6 +9,7 @@ ENV PACKAGES \
     flex \
     g++ \
     gcc \
+    gettext \
     git \
     glib2-devel \
     libepoxy-devel \
diff --git a/tests/docker/dockerfiles/debian-ports.docker b/tests/docker/dockerfiles/debian-ports.docker
index fba224f760..e05a9a9802 100644
--- a/tests/docker/dockerfiles/debian-ports.docker
+++ b/tests/docker/dockerfiles/debian-ports.docker
@@ -27,6 +27,7 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
         clang \
         debian-ports-archive-keyring \
         flex \
+        gettext \
         git \
         pkg-config \
         psmisc \
diff --git a/tests/docker/dockerfiles/debian8.docker b/tests/docker/dockerfiles/debian8.docker
index 3d09b4b462..1bcf2e3d2f 100644
--- a/tests/docker/dockerfiles/debian8.docker
+++ b/tests/docker/dockerfiles/debian8.docker
@@ -26,6 +26,7 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
         clang \
         curl \
         flex \
+        gettext \
         git \
         gnupg \
         pkg-config \
diff --git a/tests/docker/dockerfiles/debian9.docker b/tests/docker/dockerfiles/debian9.docker
index a4509950e6..154ae2a455 100644
--- a/tests/docker/dockerfiles/debian9.docker
+++ b/tests/docker/dockerfiles/debian9.docker
@@ -22,6 +22,7 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
         ca-certificates \
         clang \
         flex \
+        gettext \
         git \
         pkg-config \
         psmisc \
diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker
index 27e8201c54..4b26c3aded 100644
--- a/tests/docker/dockerfiles/fedora.docker
+++ b/tests/docker/dockerfiles/fedora.docker
@@ -1,6 +1,6 @@
 FROM fedora:latest
 ENV PACKAGES \
-    ccache git tar PyYAML sparse flex bison python2 bzip2 hostname \
+    ccache gettext git tar PyYAML sparse flex bison python2 bzip2 hostname \
     glib2-devel pixman-devel zlib-devel SDL-devel libfdt-devel \
     gcc gcc-c++ clang make perl which bc findutils libaio-devel \
     nettle-devel \
diff --git a/tests/docker/dockerfiles/min-glib.docker b/tests/docker/dockerfiles/min-glib.docker
index 9f542d5e9c..f2eed97d35 100644
--- a/tests/docker/dockerfiles/min-glib.docker
+++ b/tests/docker/dockerfiles/min-glib.docker
@@ -1,6 +1,6 @@
 FROM centos:6
 RUN yum install -y \
-    tar git make gcc g++ \
+    tar gettext git make gcc g++ \
     zlib-devel SDL-devel pixman-devel \
     epel-release
 RUN yum install -y libfdt-devel ccache
diff --git a/tests/docker/dockerfiles/ubuntu.docker b/tests/docker/dockerfiles/ubuntu.docker
index d73ce02246..dabbf2a8a4 100644
--- a/tests/docker/dockerfiles/ubuntu.docker
+++ b/tests/docker/dockerfiles/ubuntu.docker
@@ -12,7 +12,7 @@ ENV PACKAGES flex bison \
     libbluetooth-dev librbd-dev libaio-dev glusterfs-common libnuma-dev libepoxy-dev libdrm-dev libgbm-dev \
     libjemalloc-dev libcacard-dev libusbredirhost-dev libnfs-dev libcap-dev libattr1-dev \
     texinfo \
-    git make ccache python-yaml gcc clang sparse
+    gettext git make ccache python-yaml gcc clang sparse
 RUN apt-get -y install $PACKAGES
 RUN dpkg -l $PACKAGES | sort > /packages.txt
 ENV FEATURES clang pyyaml
diff --git a/tests/docker/test-build b/tests/docker/test-build
index 031a7d9d30..22766cfacc 100755
--- a/tests/docker/test-build
+++ b/tests/docker/test-build
@@ -18,3 +18,4 @@ cd "$BUILD_DIR"
 DEF_TARGET_LIST="x86_64-softmmu,aarch64-softmmu"
 TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
 build_qemu
+install_qemu
diff --git a/tests/docker/test-clang b/tests/docker/test-clang
index 16485e6b7e..1eb61a3af7 100755
--- a/tests/docker/test-clang
+++ b/tests/docker/test-clang
@@ -24,3 +24,4 @@ OPTS="--enable-debug --cxx=clang++ --cc=clang --host-cc=clang"
     #--extra-cflags=-fno-sanitize=float-divide-by-zero"
 build_qemu $OPTS
 make $MAKEFLAGS check
+install_qemu
diff --git a/tests/docker/test-full b/tests/docker/test-full
index d71bf9d275..816d5a3eec 100755
--- a/tests/docker/test-full
+++ b/tests/docker/test-full
@@ -86,4 +86,4 @@ build_qemu \
     --enable-xen-pci-passthrough \
     --enable-xen-pv-domain-build \
     --enable-xfsctl \
-&& make check $MAKEFLAGS
+&& make check $MAKEFLAGS && install_qemu
diff --git a/tests/docker/test-mingw b/tests/docker/test-mingw
index 2adadcb58d..39a1da448e 100755
--- a/tests/docker/test-mingw
+++ b/tests/docker/test-mingw
@@ -31,6 +31,7 @@ for prefix in x86_64-w64-mingw32- i686-w64-mingw32-; do
         --enable-guest-agent \
         --with-sdlabi=1.2 \
         --with-gtkabi=2.0
+    install_qemu
     make clean
 
 done
diff --git a/tests/docker/test-quick b/tests/docker/test-quick
index c465dc06d8..3b7bce6105 100755
--- a/tests/docker/test-quick
+++ b/tests/docker/test-quick
@@ -19,3 +19,4 @@ DEF_TARGET_LIST="x86_64-softmmu,aarch64-softmmu"
 TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
 build_qemu
 make check $MAKEFLAGS
+install_qemu
-- 
2.13.5

[Qemu-devel] [PULL 2/5] docker: Fix test-mingw

[Fam Zheng]

Feature "dtc" is explicitly required by test-mingw, but is not detected
by the run script since we switched to archive-source.sh in b7f404201e4.
Since it isn't available in the Fedora image which runs this test on
patchew, the way we get dtc is still from submodule.

archive-source.sh takes care of bundling the submodule files already, so
what we need to do is just checking if files are there. Makefile is
chosen because it is one that is unlikely to get renamed in the future.

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20170925082913.22089-1-famz@redhat.com>
Reviewed-by: Alistair Francis <alistair.francis@xilinx.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/run | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/tests/docker/run b/tests/docker/run
index c8f940de15..0fd2f358ce 100755
--- a/tests/docker/run
+++ b/tests/docker/run
@@ -31,6 +31,9 @@ mkdir -p $TEST_DIR/{src,build,install}
 
 # Extract the source tarballs
 tar -C $TEST_DIR/src -xf $BASE/qemu.tar || prep_fail "Failed to untar source"
+if test -f $TEST_DIR/src/Makefile; then
+    export FEATURES="$FEATURES dtc"
+fi
 
 if test -n "$SHOW_ENV"; then
     if test -f /packages.txt; then
-- 
2.13.5

[Qemu-devel] [PULL 3/5] tests/docker/run: don't source /etc/profile

[Fam Zheng]

From: Alex Bennée <alex.bennee@linaro.org>

The usual behaviour of /etc/profile is to set the default PATH for
users. This runs into problems when we have updated PATH in our
dockerfile e.g. to access a cross-compiler in a non-standard
location. It shouldn't be needed anyway as we inherit the env from the
image when it was setup.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
CC: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20170926133622.14991-1-alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/run | 1 -
 1 file changed, 1 deletion(-)

diff --git a/tests/docker/run b/tests/docker/run
index 0fd2f358ce..642084bcb8 100755
--- a/tests/docker/run
+++ b/tests/docker/run
@@ -18,7 +18,6 @@ fi
 BASE="$(dirname $(readlink -e $0))"
 
 # Prepare the environment
-. /etc/profile
 export PATH=/usr/lib/ccache:$PATH
 
 if test -n "$J"; then
-- 
2.13.5

[Qemu-devel] [PULL 4/5] docker: test-block: Don't continue if build fails

[Fam Zheng]

Report error and exit upon compiling error, otherwise the iotests output
will be pure noise.

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20170926110134.2786-1-famz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/test-block | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/docker/test-block b/tests/docker/test-block
index 2ca1ce54f6..5624b81827 100755
--- a/tests/docker/test-block
+++ b/tests/docker/test-block
@@ -14,7 +14,7 @@
 
 cd "$BUILD_DIR"
 
-build_qemu --target-list=x86_64-softmmu
+build_qemu --target-list=x86_64-softmmu || test_fail "Build failed"
 cd tests/qemu-iotests
 for t in raw qcow2 nbd luks; do
     ./check -g quick -$t || test_fail "Test failed: iotests $t"
-- 
2.13.5

[Qemu-devel] [PULL 5/5] docker: Don't mount ccache db if NOUSER=1

[Fam Zheng]

With NOUSER=1 the container runs code as root, which may create
privileged files that will not be be accssible next time. Skip ccache
dir mount in this case.

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20170925075458.18047-1-famz@redhat.com>
Acked-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/Makefile.include | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 0e4f159619..6f9ea196a7 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -143,9 +143,11 @@ docker-run: docker-qemu-src
 			-e EXTRA_CONFIGURE_OPTS="$(EXTRA_CONFIGURE_OPTS)" \
 			-e V=$V -e J=$J -e DEBUG=$(DEBUG)		\
 			-e SHOW_ENV=$(SHOW_ENV) 			\
-			-e CCACHE_DIR=/var/tmp/ccache 			\
+			$(if $(NOUSER),,				\
+				-e CCACHE_DIR=/var/tmp/ccache 		\
+				-v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z \
+			)						\
 			-v $$(readlink -e $(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
-			-v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z 	\
 			$(IMAGE) 					\
 			/var/tmp/qemu/run 				\
 			$(TEST), "  RUN $(TEST) in ${IMAGE}")
-- 
2.13.5

Re: [Qemu-devel] [PULL 0/5] Docker testing patches

[Peter Maydell]

On 29 September 2017 at 04:18, Fam Zheng <famz@redhat.com> wrote:
> The following changes since commit ab161529261928ae7f3556e3220829c34b2686ec:
>
>   Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20170927a' into staging (2017-09-27 22:44:51 +0100)
>
> are available in the git repository at:
>
>   git://github.com/famz/qemu.git tags/docker-testing-pull-request
>
> for you to fetch changes up to 36ac78e65a0695f0e397865cceaf4ab5c4e52c64:
>
>   docker: Don't mount ccache db if NOUSER=1 (2017-09-29 11:14:15 +0800)
>
> ----------------------------------------------------------------
>
> ----------------------------------------------------------------
>
> Alex Bennée (1):
>   tests/docker/run: don't source /etc/profile
>
> Fam Zheng (3):
>   docker: Fix test-mingw
>   docker: test-block: Don't continue if build fails
>   docker: Don't mount ccache db if NOUSER=1
>
> Paolo Bonzini (1):
>   docker: add installation to build tests
>

Applied, thanks.

-- PMM