From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46362) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bCSiM-0004CC-Po for qemu-devel@nongnu.org; Mon, 13 Jun 2016 10:19:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bCSiF-0002WJ-V1 for qemu-devel@nongnu.org; Mon, 13 Jun 2016 10:19:41 -0400 Received: from mail-wm0-x231.google.com ([2a00:1450:400c:c09::231]:35511) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bCSiF-0002VU-5i for qemu-devel@nongnu.org; Mon, 13 Jun 2016 10:19:35 -0400 Received: by mail-wm0-x231.google.com with SMTP id v199so80605461wmv.0 for ; Mon, 13 Jun 2016 07:19:35 -0700 (PDT) From: Alex =?utf-8?Q?Benn=C3=A9e?= In-reply-to: Date: Mon, 13 Jun 2016 15:19:37 +0100 Message-ID: <87eg81gohy.fsf@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Subject: Re: [Qemu-devel] [PATCH v2 0/3] Support building qemu-user powered docker test images List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Riku Voipio Cc: Fam Zheng , qemu-devel qemu-devel Riku Voipio writes: > On 13 June 2016 at 15:30, Alex Bennée wrote: >> >> Riku Voipio writes: >> >>> On 13 June 2016 at 12:22, Alex Bennée wrote: >>>> >>>> Fam Zheng writes: >>>> >>>>> On Wed, 06/08 17:35, Alex Bennée wrote: >>>>>> Hi, >>>>>> >>>>>> This is a re-spin of the previous series built on top of >>>>>> fam/docker.next. I've made the changes suggested in the last review >>>>>> and split the first patch apart to separate (and fix) the build >>>>>> directory changes first. >>>>>> >>>>>> Now it no longer messes with the docker file you can actually >>>>>> cross-build tests. First ensure you build the debian-bootstrap image: >>>>>> >>>>>> DEB_ARCH=armhf DEB_TYPE=testing \ >>>>>> ./tests/docker/docker.py build qemu:debian-bootstrap \ >>>>>> ./tests/docker/dockerfiles/debian-bootstrap.docker \ >>>>>> --include-executable=./arm-linux-user/qemu-arm >>>>> >>>>> This can take long depending on the network - I had to explicitly use a close >>>>> mirror in the pre script to test this, not sure why the mirror redirector >>>>> doesn't work. Eventually I get this error once the pre script succeeds and >>>>> docker build starts: >>>>> >>>>> Sending build context to Docker daemon 235.1 MB >>>>> Step 1 : FROM scratch >>>>> ---> >>>>> Step 2 : ADD . / >>>>> ---> 807bfa810b0c >>>>> Removing intermediate container e57ded00b227 >>>>> Step 3 : RUN sed -i 's/in_target mount/echo not for docker in_target mount/g' /debootstrap/functions >>>>> ---> Running in fd80232b38fc >>>>> rpc error: code = 2 desc = "oci runtime error: exec format error" >>>>> >>>>> IIUC the sed is an armhf binary? Is something missing in the docker file? I >>>>> cannot find where the copied qemu-arm is used... >>>> >>>> Ahh I realise I missed out the implicit binfmt_misc needs to be set up. >>>> So on my Ubuntu system just having qemu-user installed means the host >>>> system binfmt_misc is set up for armhf binaries to run >>>> /usr/bin/qemu-arm. I haven't experimented with explicitly setting up >>>> binfmt_misc in the container because it was already working :-/ >>> >>> The containment leaks here. If your host has registered binfmt_misc, >>> identical settings will be visible in container. You can't change the >>> settings inside docker, unless you start with --privileged. And >>> changing binfmt_misc settings inside docker will also change the >>> settings in the host. So one needs to tiptoe around setting it up... >> >> Hmm well that makes it interesting. FWIW I'd stick to setting up binfmt >> outside the container and defaulting to the "just works" setup depending >> on how $HOST sets up binfmt_misc. The original patch actually queried >> binfmt_misc to work out the exact path it needed to install qemu-arm in >> the $GUEST container. For now we just default to /usr/bin which works on >> Debian-types without problem. > > For some value of without problem - you have qemu-user installed, so > you have /usr/bin/qemu-arm registered, while I have qemu-user-static > installed and thus /usr/bin/qemu-arm-static registered. > > We have the ./scripts/qemu-binfmt-conf.sh we could run if no binfmt's > appear registered, or at least print an error message for the enduser > to run it. I noticed that has been updated recently. I shall have a look and see if I can make the pre-script return an error if the binfmt conditions aren't met. Hopefully it doesn't require root permissions to check all this out ;-) > >>> >>>>> >>>>> Fam >>>>> >>>>>> >>>>>> And then run the test quick target: >>>>>> >>>>>> make docker-test-quick@debian-bootstrap J=9 V=1 >>>>>> >>>>>> I'll leave it up to you how we cleanly integrate multi-arch builds >>>>>> into the Make system ;-) >>>>>> >>>>>> Alex Bennée (3): >>>>>> tests/docker/docker.py: docker_dir outside build >>>>>> tests/docker/docker.py: support --include-executable >>>>>> add debian-bootstrap.docker target (and pre script) >>>>>> >>>>>> tests/docker/docker.py | 68 +++++++++++++++++++++--- >>>>>> tests/docker/dockerfiles/debian-bootstrap.docker | 21 ++++++++ >>>>>> tests/docker/dockerfiles/debian-bootstrap.pre | 5 ++ >>>>>> 3 files changed, 86 insertions(+), 8 deletions(-) >>>>>> create mode 100644 tests/docker/dockerfiles/debian-bootstrap.docker >>>>>> create mode 100755 tests/docker/dockerfiles/debian-bootstrap.pre >>>>>> >>>>>> -- >>>>>> 2.7.4 >>>>>> >>>> >>>> >>>> -- >>>> Alex Bennée >> >> >> -- >> Alex Bennée -- Alex Bennée