[Qemu-devel] [PATCH RFC 00/10] docker on non-x86 hosts

[Qemu-devel] [PATCH RFC 00/10] docker on non-x86 hosts

[Alex Bennée]

Hi,

Our existing support for docker is fairly x86 centric. While docker
itself has support for multiple architectures not all architectures
are equal. For example Debian only packages the widest range of
cross-compilers in it's x86 images (although for example armhf is
available on aarch64 based images).

As the binfmt support is getting more solid we always have that fall
back option and I've converted a number of the guests that way.
Unfortunately I've overloaded the meaning of DOCKER_PARTIAL_IMAGES but
I was loathed to throw in yet another magic variable without some
discussion first. I'm not sure if my current path will end up with a
maze of twisty if/else statements all subtly alike.

Thoughts?

Alex Bennée (10):
  docker: rename docker-amd64 to docker-host
  docker: change docker-image to docker-all-images target
  docker: add a placeholder for handling non-x86 hosts
  docker: don't include docker-arm64-cross on aarch64 hosts
  docker: fall-back to binfmt_misc debian-mips64el-user-cross on non-x86
  docker: fall-back to binfmt_misc debian-ppc64el-user-cross on non-x86
  docker: fall-back to binfmt_misc debian-s390x-user-cross on non-x86
  docker: disable additional non-x86 images
  tests: tcg skip docker images we can't build
  tests/tcg: debian-mips64el-user-cross fallback

 Makefile                                      |  4 +-
 tests/docker/Makefile.include                 | 58 +++++++++++++++++--
 ...debian-amd64.docker => debian-host.docker} |  4 +-
 .../debian-mips64el-user-cross.docker         | 16 +++++
 .../debian-ppc64el-user-cross.docker          | 16 +++++
 .../debian-s390x-user-cross.docker            | 16 +++++
 tests/tcg/Makefile.include                    |  3 +
 tests/tcg/Makefile.probe                      |  2 +-
 tests/tcg/mips/Makefile.include               |  5 ++
 9 files changed, 116 insertions(+), 8 deletions(-)
 rename tests/docker/dockerfiles/{debian-amd64.docker => debian-host.docker} (91%)
 create mode 100644 tests/docker/dockerfiles/debian-mips64el-user-cross.docker
 create mode 100644 tests/docker/dockerfiles/debian-ppc64el-user-cross.docker
 create mode 100644 tests/docker/dockerfiles/debian-s390x-user-cross.docker

-- 
2.17.1

[Qemu-devel] [PATCH RFC 01/10] docker: rename docker-amd64 to docker-host

[Alex Bennée]

When building on non-x86 systems the base system will be correct so if
we avoid too many x86'isms in the install we can still use the image.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 .../dockerfiles/{debian-amd64.docker => debian-host.docker}   | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
 rename tests/docker/dockerfiles/{debian-amd64.docker => debian-host.docker} (91%)

diff --git a/tests/docker/dockerfiles/debian-amd64.docker b/tests/docker/dockerfiles/debian-host.docker
similarity index 91%
rename from tests/docker/dockerfiles/debian-amd64.docker
rename to tests/docker/dockerfiles/debian-host.docker
index eb13f06ed1..3605cc4658 100644
--- a/tests/docker/dockerfiles/debian-amd64.docker
+++ b/tests/docker/dockerfiles/debian-host.docker
@@ -30,9 +30,9 @@ RUN cd /usr/src/virglrenderer && ./autogen.sh && ./configure --with-glx --disabl
 # netmap
 RUN DEBIAN_FRONTEND=noninteractive eatmydata \
     apt-get install -y --no-install-recommends \
-        linux-headers-amd64
+        linux-headers-$(dpkg --print-architecture)
 RUN git clone https://github.com/luigirizzo/netmap.git /usr/src/netmap
-RUN cd /usr/src/netmap/LINUX && ./configure --no-drivers --no-apps --kernel-dir=$(ls -d /usr/src/linux-headers-*-amd64) && make install
+RUN cd /usr/src/netmap/LINUX && ./configure --no-drivers --no-apps --kernel-dir=$(ls -d /usr/src/linux-headers-*) && make install
 ENV QEMU_CONFIGURE_OPTS --enable-netmap
 
 # gcrypt
-- 
2.17.1

Re: [Qemu-devel] [PATCH RFC 01/10] docker: rename docker-amd64 to docker-host

[Fam Zheng]

On Wed, 07/18 11:04, Alex Bennée wrote:
> When building on non-x86 systems the base system will be correct so if
> we avoid too many x86'isms in the install we can still use the image.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  .../dockerfiles/{debian-amd64.docker => debian-host.docker}   | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>  rename tests/docker/dockerfiles/{debian-amd64.docker => debian-host.docker} (91%)
> 
> diff --git a/tests/docker/dockerfiles/debian-amd64.docker b/tests/docker/dockerfiles/debian-host.docker
> similarity index 91%
> rename from tests/docker/dockerfiles/debian-amd64.docker
> rename to tests/docker/dockerfiles/debian-host.docker
> index eb13f06ed1..3605cc4658 100644
> --- a/tests/docker/dockerfiles/debian-amd64.docker
> +++ b/tests/docker/dockerfiles/debian-host.docker
> @@ -30,9 +30,9 @@ RUN cd /usr/src/virglrenderer && ./autogen.sh && ./configure --with-glx --disabl
>  # netmap
>  RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>      apt-get install -y --no-install-recommends \
> -        linux-headers-amd64
> +        linux-headers-$(dpkg --print-architecture)
>  RUN git clone https://github.com/luigirizzo/netmap.git /usr/src/netmap
> -RUN cd /usr/src/netmap/LINUX && ./configure --no-drivers --no-apps --kernel-dir=$(ls -d /usr/src/linux-headers-*-amd64) && make install
> +RUN cd /usr/src/netmap/LINUX && ./configure --no-drivers --no-apps --kernel-dir=$(ls -d /usr/src/linux-headers-*) && make install

Curious: why isn't this "ls -d linux-headers-*-$(dpkg --print-architecture)", like above?

>  ENV QEMU_CONFIGURE_OPTS --enable-netmap
>  
>  # gcrypt
> -- 
> 2.17.1
> 

Re: [Qemu-devel] [PATCH RFC 01/10] docker: rename docker-amd64 to docker-host

[Alex Bennée]

Fam Zheng <famz@redhat.com> writes:

> On Wed, 07/18 11:04, Alex Bennée wrote:
>> When building on non-x86 systems the base system will be correct so if
>> we avoid too many x86'isms in the install we can still use the image.
>>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> ---
>>  .../dockerfiles/{debian-amd64.docker => debian-host.docker}   | 4 ++--
>>  1 file changed, 2 insertions(+), 2 deletions(-)
>>  rename tests/docker/dockerfiles/{debian-amd64.docker => debian-host.docker} (91%)
>>
>> diff --git a/tests/docker/dockerfiles/debian-amd64.docker b/tests/docker/dockerfiles/debian-host.docker
>> similarity index 91%
>> rename from tests/docker/dockerfiles/debian-amd64.docker
>> rename to tests/docker/dockerfiles/debian-host.docker
>> index eb13f06ed1..3605cc4658 100644
>> --- a/tests/docker/dockerfiles/debian-amd64.docker
>> +++ b/tests/docker/dockerfiles/debian-host.docker
>> @@ -30,9 +30,9 @@ RUN cd /usr/src/virglrenderer && ./autogen.sh && ./configure --with-glx --disabl
>>  # netmap
>>  RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>>      apt-get install -y --no-install-recommends \
>> -        linux-headers-amd64
>> +        linux-headers-$(dpkg --print-architecture)
>>  RUN git clone https://github.com/luigirizzo/netmap.git /usr/src/netmap
>> -RUN cd /usr/src/netmap/LINUX && ./configure --no-drivers --no-apps --kernel-dir=$(ls -d /usr/src/linux-headers-*-amd64) && make install
>> +RUN cd /usr/src/netmap/LINUX && ./configure --no-drivers --no-apps --kernel-dir=$(ls -d /usr/src/linux-headers-*) && make install
>
> Curious: why isn't this "ls -d linux-headers-*-$(dpkg
> --print-architecture)", like above?

Well there are only one set of headers, but you are right it would be
cleaner to be explicit.

>
>>  ENV QEMU_CONFIGURE_OPTS --enable-netmap
>>
>>  # gcrypt
>> --
>> 2.17.1
>>


--
Alex Bennée

[Qemu-devel] [PATCH RFC 02/10] docker: change docker-image to docker-all-images target

[Alex Bennée]

We also evaluate this list later in the build rule after all decisions
about what will and won't be built have been made.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/Makefile.include | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index c9e412f9d0..22adf6ac73 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -8,7 +8,6 @@ DOCKER_DEPRECATED_IMAGES := debian
 # we don't run tests on intermediate images (used as base by another image)
 DOCKER_PARTIAL_IMAGES := debian debian8 debian9 debian8-mxe debian-ports debian-sid debian-bootstrap
 DOCKER_IMAGES := $(filter-out $(DOCKER_DEPRECATED_IMAGES),$(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker)))))
-DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES))
 # Use a global constant ccache directory to speed up repetitive builds
 DOCKER_CCACHE_DIR := $$HOME/.cache/qemu-docker-ccache
 
@@ -34,8 +33,6 @@ $(DOCKER_SRC_COPY):
 
 docker-qemu-src: $(DOCKER_SRC_COPY)
 
-docker-image: ${DOCKER_TARGETS}
-
 # General rule for building docker images. If we are a sub-make
 # invoked with SKIP_DOCKER_BUILD we still check the image is upto date
 # though
@@ -154,6 +151,10 @@ $(foreach i,$(filter-out $(DOCKER_PARTIAL_IMAGES),$(DOCKER_IMAGES) $(DOCKER_DEPR
 	) \
 )
 
+# Expand a rule to build all useable images
+DOCKER_TARGETS := $(patsubst %,docker-image-%,$(filter-out $(DOCKER_PARTIAL_IMAGES),$(DOCKER_IMAGES)))
+docker-all-images: $(DOCKER_TARGETS)
+
 docker:
 	@echo 'Build QEMU and run tests inside Docker containers'
 	@echo
@@ -167,7 +168,7 @@ docker:
 	@echo '                         Note: "TEST" is one of the listed test name,'
 	@echo '                         or a script name under $$QEMU_SRC/tests/docker/;'
 	@echo '                         "IMAGE" is one of the listed container name."'
-	@echo '    docker-image:        Build all images.'
+	@echo '    docker-all-images:   Build all images.'
 	@echo '    docker-image-IMAGE:  Build image "IMAGE".'
 	@echo '    docker-run:          For manually running a "TEST" with "IMAGE"'
 	@echo
-- 
2.17.1

[Qemu-devel] [PATCH RFC 03/10] docker: add a placeholder for handling non-x86 hosts

[Alex Bennée]

We want to keep all our variations in one place. This is that place.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/Makefile.include | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 22adf6ac73..7d13ddd497 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -139,6 +139,18 @@ DOCKER_USER_IMAGES += debian-powerpc-user
 # tests due to missing system call: 249 which causes an abort
 DOCKER_PARTIAL_IMAGES += debian-powerpc-user-cross
 
+# Modifications for non-x86 hosts
+#
+# While docker itself is now multiarch aware and will generally do the
+# right thing (assuming the source also supports multiple
+# architectures) there are still variations we need to take into
+# account. These are all done here.
+
+ifneq ($(ARCH),x86_64)
+
+
+endif
+
 # Expand all the pre-requistes for each docker image and test combination
 $(foreach i,$(filter-out $(DOCKER_PARTIAL_IMAGES),$(DOCKER_IMAGES) $(DOCKER_DEPRECATED_IMAGES)), \
 	$(foreach t,$(DOCKER_TESTS) $(DOCKER_TOOLS), \
-- 
2.17.1

[Qemu-devel] [PATCH RFC 04/10] docker: don't include docker-arm64-cross on aarch64 hosts

[Alex Bennée]

Attempting to install the native compiler as a cross compiler rarely
goes well.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/Makefile.include | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 7d13ddd497..2cb180ff6c 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -148,6 +148,10 @@ DOCKER_PARTIAL_IMAGES += debian-powerpc-user-cross
 
 ifneq ($(ARCH),x86_64)
 
+ifeq ($(ARCH),aarch64)
+# no point having a cross compiler when it is the native one
+DOCKER_PARTIAL_IMAGES += debian-arm64-cross
+endif
 
 endif
 
-- 
2.17.1

[Qemu-devel] [PATCH RFC 05/10] docker: fall-back to binfmt_misc debian-mips64el-user-cross on non-x86

[Alex Bennée]

The Debian archives don't package their large array of cross compilers
for all host architectures. For MIPS let's fall back to a linux-user
based image when we can't use the native ones.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/Makefile.include                    | 11 +++++++++++
 .../debian-mips64el-user-cross.docker            | 16 ++++++++++++++++
 2 files changed, 27 insertions(+)
 create mode 100644 tests/docker/dockerfiles/debian-mips64el-user-cross.docker

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 2cb180ff6c..5cfb1a57bb 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -153,8 +153,19 @@ ifeq ($(ARCH),aarch64)
 DOCKER_PARTIAL_IMAGES += debian-arm64-cross
 endif
 
+# No MIPS cross-compilers in non-x86 Debian images, so use linux-user
+docker-binfmt-image-debian-mips64el-user: DEB_ARCH = mips64el
+docker-binfmt-image-debian-mips64el-user: DEB_TYPE = stretch
+docker-binfmt-image-debian-mips64el-user: EXECUTABLE = ${BUILD_DIR}/mips64el-linux-user/qemu-mips64el
+docker-image-debian-mips64el-user-cross: docker-binfmt-image-debian-mips64el-user
+DOCKER_USER_IMAGES += debian-mips64el-user-cross
+DOCKER_PARTIAL_IMAGES += debian-mips-cross debian-mipsel-cross debian-mips64el-cross
+
+else
+DOCKER_PARTIAL_IMAGES += debian-mips64el-user-cross
 endif
 
+
 # Expand all the pre-requistes for each docker image and test combination
 $(foreach i,$(filter-out $(DOCKER_PARTIAL_IMAGES),$(DOCKER_IMAGES) $(DOCKER_DEPRECATED_IMAGES)), \
 	$(foreach t,$(DOCKER_TESTS) $(DOCKER_TOOLS), \
diff --git a/tests/docker/dockerfiles/debian-mips64el-user-cross.docker b/tests/docker/dockerfiles/debian-mips64el-user-cross.docker
new file mode 100644
index 0000000000..98284bc635
--- /dev/null
+++ b/tests/docker/dockerfiles/debian-mips64el-user-cross.docker
@@ -0,0 +1,16 @@
+#
+# Docker mips64el user cross-compiler target
+#
+# This is a binfmt image for when the MIPS cross compilers are not
+# available in the host architecture archives. This image is based of
+# the debootstrapped qemu:debian-mips64el-user but doesn't need any
+# extra magic once it is setup.
+#
+
+FROM qemu:debian-mips64el-user
+
+MAINTAINER Alex Bennee <alex.bennee@linaro.org>
+
+RUN echo man-db man-db/auto-update boolean false | debconf-set-selections
+RUN apt-get update && \
+    DEBIAN_FRONTEND=noninteractive apt-get build-dep -yy qemu
-- 
2.17.1

[Qemu-devel] [PATCH RFC 06/10] docker: fall-back to binfmt_misc debian-ppc64el-user-cross on non-x86

[Alex Bennée]

The Debian archives don't package their large array of cross compilers
for all host architectures. For ppc64el let's fall back to a linux-user
based image when we can't use the native ones.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/Makefile.include                    | 10 +++++++++-
 .../dockerfiles/debian-ppc64el-user-cross.docker | 16 ++++++++++++++++
 2 files changed, 25 insertions(+), 1 deletion(-)
 create mode 100644 tests/docker/dockerfiles/debian-ppc64el-user-cross.docker

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 5cfb1a57bb..879bacde93 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -161,8 +161,16 @@ docker-image-debian-mips64el-user-cross: docker-binfmt-image-debian-mips64el-use
 DOCKER_USER_IMAGES += debian-mips64el-user-cross
 DOCKER_PARTIAL_IMAGES += debian-mips-cross debian-mipsel-cross debian-mips64el-cross
 
+# No ppc64el cross-compilers in non-x86 Debian images, so use linux-user
+docker-binfmt-image-debian-ppc64el-user: DEB_ARCH = ppc64el
+docker-binfmt-image-debian-ppc64el-user: DEB_TYPE = stretch
+docker-binfmt-image-debian-ppc64el-user: EXECUTABLE = ${BUILD_DIR}/ppc64le-linux-user/qemu-ppc64le
+docker-image-debian-ppc64el-user-cross: docker-binfmt-image-debian-ppc64el-user
+DOCKER_USER_IMAGES += debian-ppc64el-user-cross
+DOCKER_PARTIAL_IMAGES += debian-ppc64el-cross
+
 else
-DOCKER_PARTIAL_IMAGES += debian-mips64el-user-cross
+DOCKER_PARTIAL_IMAGES += debian-mips64el-user-cross debian-ppc64el-user-cross
 endif
 
 
diff --git a/tests/docker/dockerfiles/debian-ppc64el-user-cross.docker b/tests/docker/dockerfiles/debian-ppc64el-user-cross.docker
new file mode 100644
index 0000000000..aeb207e9c2
--- /dev/null
+++ b/tests/docker/dockerfiles/debian-ppc64el-user-cross.docker
@@ -0,0 +1,16 @@
+#
+# Docker ppc64el user cross-compiler target
+#
+# This is a binfmt image for when the ppc64el cross compilers are not
+# available in the host architecture archives. This image is based of
+# the debootstrapped qemu:debian-ppc64el-user but doesn't need any
+# extra magic once it is setup.
+#
+
+FROM qemu:debian-ppc64el-user
+
+MAINTAINER Alex Bennee <alex.bennee@linaro.org>
+
+RUN echo man-db man-db/auto-update boolean false | debconf-set-selections
+RUN apt-get update && \
+    DEBIAN_FRONTEND=noninteractive apt-get build-dep -yy qemu
-- 
2.17.1

[Qemu-devel] [PATCH RFC 07/10] docker: fall-back to binfmt_misc debian-s390x-user-cross on non-x86

[Alex Bennée]

The Debian archives don't package their large array of cross compilers
for all host architectures. For s390x let's fall back to a linux-user
based image when we can't use the native ones.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/Makefile.include                    |  9 ++++++++-
 .../dockerfiles/debian-s390x-user-cross.docker   | 16 ++++++++++++++++
 2 files changed, 24 insertions(+), 1 deletion(-)
 create mode 100644 tests/docker/dockerfiles/debian-s390x-user-cross.docker

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 879bacde93..e1eba21f06 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -169,8 +169,15 @@ docker-image-debian-ppc64el-user-cross: docker-binfmt-image-debian-ppc64el-user
 DOCKER_USER_IMAGES += debian-ppc64el-user-cross
 DOCKER_PARTIAL_IMAGES += debian-ppc64el-cross
 
+# No s390x cross-compilers in non-x86 Debian images, so use linux-user
+docker-binfmt-image-debian-s390x-user: DEB_ARCH = s390x
+docker-binfmt-image-debian-s390x-user: DEB_TYPE = stretch
+docker-binfmt-image-debian-s390x-user: EXECUTABLE = ${BUILD_DIR}/s390x-linux-user/qemu-s390x
+docker-image-debian-s390x-user-cross: docker-binfmt-image-debian-s390x-user
+DOCKER_USER_IMAGES += debian-s390x-user-cross
+DOCKER_PARTIAL_IMAGES += debian-s390x-cross
 else
-DOCKER_PARTIAL_IMAGES += debian-mips64el-user-cross debian-ppc64el-user-cross
+DOCKER_PARTIAL_IMAGES += debian-mips64el-user-cross debian-ppc64el-user-cross debian-s390x-user-cross
 endif
 
 
diff --git a/tests/docker/dockerfiles/debian-s390x-user-cross.docker b/tests/docker/dockerfiles/debian-s390x-user-cross.docker
new file mode 100644
index 0000000000..396763f10e
--- /dev/null
+++ b/tests/docker/dockerfiles/debian-s390x-user-cross.docker
@@ -0,0 +1,16 @@
+#
+# Docker s390x user cross-compiler target
+#
+# This is a binfmt image for when the s390x cross compilers are not
+# available in the host architecture archives. This image is based of
+# the debootstrapped qemu:debian-s390x-user but doesn't need any
+# extra magic once it is setup.
+#
+
+FROM qemu:debian-s390x-user
+
+MAINTAINER Alex Bennee <alex.bennee@linaro.org>
+
+RUN echo man-db man-db/auto-update boolean false | debconf-set-selections
+RUN apt-get update && \
+    DEBIAN_FRONTEND=noninteractive apt-get build-dep -yy qemu
-- 
2.17.1

[Qemu-devel] [PATCH RFC 08/10] docker: disable additional non-x86 images

[Alex Bennée]

 - travis - doesn't support non-x8
 - centos7 - only AltArch, next release should have support
 - debian-8 based - docker hub issue?
 - ubuntu - image a bit old, maybe update to 18.04

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/Makefile.include | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index e1eba21f06..2226abd092 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -148,9 +148,16 @@ DOCKER_PARTIAL_IMAGES += debian-powerpc-user-cross
 
 ifneq ($(ARCH),x86_64)
 
+# Don't include non-multiarch images
+DOCKER_PARTIAL_IMAGES += travis centos7
+
 ifeq ($(ARCH),aarch64)
 # no point having a cross compiler when it is the native one
 DOCKER_PARTIAL_IMAGES += debian-arm64-cross
+# hub.docker.com's older Debian images are not multiarch
+DOCKER_PARTIAL_IMAGES += debian-win32-cross debian-win64-cross
+# older Ubuntu's need fixing up as well
+DOCKER_PARTIAL_IMAGES += ubuntu
 endif
 
 # No MIPS cross-compilers in non-x86 Debian images, so use linux-user
-- 
2.17.1

[Qemu-devel] [PATCH RFC 09/10] tests: tcg skip docker images we can't build

[Alex Bennée]

We have a mechansim (DOCKER_PARTIAL_IMAGES) for marking stuff that is
incomplete or won't build. We can use this to avoid attempting to
build them at the top of a check-tcg build. We need to move the
inclusion of docker/Makefile.include to ensure everything is resolved
by this point.

We also have to check this in tests/tcg/Makefile.include to cover the
case where users are building directly out of the target tree.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 Makefile                   | 4 +++-
 tests/tcg/Makefile.include | 3 +++
 tests/tcg/Makefile.probe   | 2 +-
 3 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 2da686be33..6bb8035557 100644
--- a/Makefile
+++ b/Makefile
@@ -434,6 +434,9 @@ dummy := $(call unnest-vars,, \
                 audio-obj-m \
                 trace-obj-y)
 
+# we need to know what Docker images can be built before we can
+# decide on all the tests that can be built.
+include $(SRC_PATH)/tests/docker/Makefile.include
 include $(SRC_PATH)/tests/Makefile.include
 
 all: $(DOCS) $(TOOLS) $(HELPERS-y) recurse-all modules
@@ -1069,7 +1072,6 @@ endif
 # Dependencies in Makefile.objs files come from our recursive subdir rules
 -include $(wildcard *.d tests/*.d)
 
-include $(SRC_PATH)/tests/docker/Makefile.include
 include $(SRC_PATH)/tests/vm/Makefile.include
 
 .PHONY: help
diff --git a/tests/tcg/Makefile.include b/tests/tcg/Makefile.include
index 57470b2a2c..e5f0298375 100644
--- a/tests/tcg/Makefile.include
+++ b/tests/tcg/Makefile.include
@@ -41,6 +41,8 @@ ifneq ($(DOCKER_IMAGE),)
 # We also need the Docker make rules to depend on
 include $(SRC_PATH)/tests/docker/Makefile.include
 
+ifneq ($(filter-out $(DOCKER_PARTIAL_IMAGES),$(DOCKER_IMAGE)),)
+
 DOCKER_COMPILE_CMD="$(DOCKER_SCRIPT) cc --user $(shell id -u) \
 		--cc $(DOCKER_CROSS_COMPILER) \
 		-i qemu:$(DOCKER_IMAGE) \
@@ -58,6 +60,7 @@ docker-build-guest-tests: $(DOCKER_PREREQ)
 
 GUEST_BUILD=docker-build-guest-tests
 
+endif
 endif
 endif
 
diff --git a/tests/tcg/Makefile.probe b/tests/tcg/Makefile.probe
index 15c0412657..22c77d6b88 100644
--- a/tests/tcg/Makefile.probe
+++ b/tests/tcg/Makefile.probe
@@ -20,7 +20,7 @@ DOCKER_PREREQ:=
 -include $(SRC_PATH)/tests/tcg/$(TARGET_NAME)/Makefile.include
 
 ifndef CROSS_CC_GUEST
-ifneq ($(DOCKER_IMAGE),)
+ifneq ($(filter-out $(DOCKER_PARTIAL_IMAGES),$(DOCKER_IMAGE)),)
 DOCKER_PREREQ:=docker-image-$(DOCKER_IMAGE)
 endif
 endif
-- 
2.17.1

[Qemu-devel] [PATCH RFC 10/10] tests/tcg: debian-mips64el-user-cross fallback

[Alex Bennée]

When we can't use the debian-mips64el-cross to build our tests we can
use the user-cross fallback.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/tcg/mips/Makefile.include | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/tests/tcg/mips/Makefile.include b/tests/tcg/mips/Makefile.include
index 4a14fc078d..cc82e586fd 100644
--- a/tests/tcg/mips/Makefile.include
+++ b/tests/tcg/mips/Makefile.include
@@ -6,8 +6,13 @@
 #
 
 ifeq ($(TARGET_NAME),mips64el)
+ifneq ($(ARCH),x86_64)
+DOCKER_IMAGE=debian-mips64el-user-cross
+DOCKER_CROSS_COMPILER=gcc
+else
 DOCKER_IMAGE=debian-mips64el-cross
 DOCKER_CROSS_COMPILER=mips64el-linux-gnuabi64-gcc
+endif
 else ifeq ($(TARGET_NAME),mips64)
 DOCKER_IMAGE=debian-mips64-cross
 DOCKER_CROSS_COMPILER=mips64-linux-gnuabi64-gcc
-- 
2.17.1

Re: [Qemu-devel] [PATCH RFC 00/10] docker on non-x86 hosts

[Fam Zheng]

On Wed, 07/18 11:04, Alex Bennée wrote:
> Hi,
> 
> Our existing support for docker is fairly x86 centric. While docker
> itself has support for multiple architectures not all architectures
> are equal. For example Debian only packages the widest range of
> cross-compilers in it's x86 images (although for example armhf is
> available on aarch64 based images).
> 
> As the binfmt support is getting more solid we always have that fall
> back option and I've converted a number of the guests that way.
> Unfortunately I've overloaded the meaning of DOCKER_PARTIAL_IMAGES but
> I was loathed to throw in yet another magic variable without some
> discussion first. I'm not sure if my current path will end up with a
> maze of twisty if/else statements all subtly alike.

Just my two cents.  So far it looks reasonable. Though I guess we could give
DOCKER_PARTIAL_IMAGES a better name or at least better documentation.

Would a per-arch variable list look cleaner? Like

    DOCKER_IMAGES_GENERIC = debian
    DOCKER_IMAGES_X86_64 = ubuntu fedora centos7
    DOCKER_PARTIAL_IMAGES_X86_64 = ...
    DOCKER_IMAGES_AARCH64 = ...
    DOCKER_PARTIAL_IMAGES_AARCH64 = ...

Then the actual host arch can be used to expand rules:

    docker-all-images: $(DOCKER_IMAGES_GENERIC)
    docker-all-images: $(DOCKER_IMAGES_$(shell uname -m | tr '[a-z]' '[A-Z]))

Fam

> 
> Thoughts?
> 
> Alex Bennée (10):
>   docker: rename docker-amd64 to docker-host
>   docker: change docker-image to docker-all-images target
>   docker: add a placeholder for handling non-x86 hosts
>   docker: don't include docker-arm64-cross on aarch64 hosts
>   docker: fall-back to binfmt_misc debian-mips64el-user-cross on non-x86
>   docker: fall-back to binfmt_misc debian-ppc64el-user-cross on non-x86
>   docker: fall-back to binfmt_misc debian-s390x-user-cross on non-x86
>   docker: disable additional non-x86 images
>   tests: tcg skip docker images we can't build
>   tests/tcg: debian-mips64el-user-cross fallback
> 
>  Makefile                                      |  4 +-
>  tests/docker/Makefile.include                 | 58 +++++++++++++++++--
>  ...debian-amd64.docker => debian-host.docker} |  4 +-
>  .../debian-mips64el-user-cross.docker         | 16 +++++
>  .../debian-ppc64el-user-cross.docker          | 16 +++++
>  .../debian-s390x-user-cross.docker            | 16 +++++
>  tests/tcg/Makefile.include                    |  3 +
>  tests/tcg/Makefile.probe                      |  2 +-
>  tests/tcg/mips/Makefile.include               |  5 ++
>  9 files changed, 116 insertions(+), 8 deletions(-)
>  rename tests/docker/dockerfiles/{debian-amd64.docker => debian-host.docker} (91%)
>  create mode 100644 tests/docker/dockerfiles/debian-mips64el-user-cross.docker
>  create mode 100644 tests/docker/dockerfiles/debian-ppc64el-user-cross.docker
>  create mode 100644 tests/docker/dockerfiles/debian-s390x-user-cross.docker
> 
> -- 
> 2.17.1
> 

Re: [Qemu-devel] [PATCH RFC 00/10] docker on non-x86 hosts

[Alex Bennée]

Fam Zheng <famz@redhat.com> writes:

> On Wed, 07/18 11:04, Alex Bennée wrote:
>> Hi,
>>
>> Our existing support for docker is fairly x86 centric. While docker
>> itself has support for multiple architectures not all architectures
>> are equal. For example Debian only packages the widest range of
>> cross-compilers in it's x86 images (although for example armhf is
>> available on aarch64 based images).
>>
>> As the binfmt support is getting more solid we always have that fall
>> back option and I've converted a number of the guests that way.
>> Unfortunately I've overloaded the meaning of DOCKER_PARTIAL_IMAGES but
>> I was loathed to throw in yet another magic variable without some
>> discussion first. I'm not sure if my current path will end up with a
>> maze of twisty if/else statements all subtly alike.
>
> Just my two cents.  So far it looks reasonable. Though I guess we could give
> DOCKER_PARTIAL_IMAGES a better name or at least better documentation.
>
> Would a per-arch variable list look cleaner? Like
>
>     DOCKER_IMAGES_GENERIC = debian
>     DOCKER_IMAGES_X86_64 = ubuntu fedora centos7
>     DOCKER_PARTIAL_IMAGES_X86_64 = ...
>     DOCKER_IMAGES_AARCH64 = ...
>     DOCKER_PARTIAL_IMAGES_AARCH64 = ...
>
> Then the actual host arch can be used to expand rules:
>
>     docker-all-images: $(DOCKER_IMAGES_GENERIC)
>     docker-all-images: $(DOCKER_IMAGES_$(shell uname -m | tr '[a-z]'
>     '[A-Z]))

That might work better... I'll give that a go on v2

>
> Fam
>
>>
>> Thoughts?
>>
>> Alex Bennée (10):
>>   docker: rename docker-amd64 to docker-host
>>   docker: change docker-image to docker-all-images target
>>   docker: add a placeholder for handling non-x86 hosts
>>   docker: don't include docker-arm64-cross on aarch64 hosts
>>   docker: fall-back to binfmt_misc debian-mips64el-user-cross on non-x86
>>   docker: fall-back to binfmt_misc debian-ppc64el-user-cross on non-x86
>>   docker: fall-back to binfmt_misc debian-s390x-user-cross on non-x86
>>   docker: disable additional non-x86 images
>>   tests: tcg skip docker images we can't build
>>   tests/tcg: debian-mips64el-user-cross fallback
>>
>>  Makefile                                      |  4 +-
>>  tests/docker/Makefile.include                 | 58 +++++++++++++++++--
>>  ...debian-amd64.docker => debian-host.docker} |  4 +-
>>  .../debian-mips64el-user-cross.docker         | 16 +++++
>>  .../debian-ppc64el-user-cross.docker          | 16 +++++
>>  .../debian-s390x-user-cross.docker            | 16 +++++
>>  tests/tcg/Makefile.include                    |  3 +
>>  tests/tcg/Makefile.probe                      |  2 +-
>>  tests/tcg/mips/Makefile.include               |  5 ++
>>  9 files changed, 116 insertions(+), 8 deletions(-)
>>  rename tests/docker/dockerfiles/{debian-amd64.docker => debian-host.docker} (91%)
>>  create mode 100644 tests/docker/dockerfiles/debian-mips64el-user-cross.docker
>>  create mode 100644 tests/docker/dockerfiles/debian-ppc64el-user-cross.docker
>>  create mode 100644 tests/docker/dockerfiles/debian-s390x-user-cross.docker
>>
>> --
>> 2.17.1
>>


--
Alex Bennée