[Qemu-devel] [PATCH v7 00/10] docker: Support building qemu-user powered docker test images

[Qemu-devel] [PATCH v7 00/10] docker: Support building qemu-user powered docker test images

[Fam Zheng]

v7: Pick up Alex's branch, and add the "docker.py images" patch. Changes since
    v6:
    - Added the EXECUTABLE patch.
    - Fix rc == 0 case of pre script.
    - Add docker.py images subcommand.
    - Use docker.py instead of sudo.
    - More details in message of the debian bootstrap pre script.

v6: A few tweaks to make this more mergable. Including:
    - Add and catch magic pre script exit code "3" to allow skipping image.
    - Tweak debian-bootstrap.pre to return '3' when appropriate (It should
      still return 1 for debootstrap command failure).
    - Fix "realpath" to "readlink -e" to be compatible with centos6.
    - Add the last patch to skip test if the image is skipped by .pre.

Alex, if this looks good to you, I can send a pull req today.

Alex Bennée (6):
  tests/docker/docker.py: docker_dir outside build
  tests/docker/docker.py: support --include-executable
  tests/docker/docker.py: check and run .pre script
  tests/docker/dockerfiles: new debian-bootstrap.docker
  tests/docker/docker.py: add update operation
  docker: pass EXECUTABLE to build script

Fam Zheng (4):
  docker: More sensible run script
  docker: Fix exit code if $CMD failed
  docker: Add "images" subcommand to docker.py
  docker: Don't start a container that doesn't exist

 tests/docker/Makefile.include                    |  14 +-
 tests/docker/docker.py                           | 159 +++++++++++++++++++++--
 tests/docker/dockerfiles/debian-bootstrap.docker |  21 +++
 tests/docker/dockerfiles/debian-bootstrap.pre    |  87 +++++++++++++
 tests/docker/run                                 |  16 ++-
 5 files changed, 280 insertions(+), 17 deletions(-)
 create mode 100644 tests/docker/dockerfiles/debian-bootstrap.docker
 create mode 100755 tests/docker/dockerfiles/debian-bootstrap.pre

-- 
2.7.4

[Qemu-devel] [PATCH v7 01/10] tests/docker/docker.py: docker_dir outside build

[Fam Zheng]

From: Alex Bennée <alex.bennee@linaro.org>

Instead of letting the build_image create the temporary working dir we
move the creation to the build command. This is preparation for the
later patches where additional files can be added to the build context
before the build step is run.

We also ensure we remove the build context after we are done (mkdtemp
doesn't do this automatically for you).

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/docker.py | 21 +++++++++++++--------
 1 file changed, 13 insertions(+), 8 deletions(-)

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index 0151362..ae40bb3 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -20,7 +20,7 @@ import atexit
 import uuid
 import argparse
 import tempfile
-from shutil import copy
+from shutil import copy, rmtree
 
 def _text_checksum(text):
     """Calculate a digest string unique to the text content"""
@@ -87,20 +87,20 @@ class Docker(object):
         labels = json.loads(resp)[0]["Config"].get("Labels", {})
         return labels.get("com.qemu.dockerfile-checksum", "")
 
-    def build_image(self, tag, dockerfile, df_path, quiet=True, argv=None):
+    def build_image(self, tag, docker_dir, dockerfile, quiet=True, argv=None):
         if argv == None:
             argv = []
-        tmp_dir = tempfile.mkdtemp(prefix="docker_build")
 
-        tmp_df = tempfile.NamedTemporaryFile(dir=tmp_dir, suffix=".docker")
+        tmp_df = tempfile.NamedTemporaryFile(dir=docker_dir, suffix=".docker")
         tmp_df.write(dockerfile)
 
         tmp_df.write("\n")
         tmp_df.write("LABEL com.qemu.dockerfile-checksum=%s" %
                      _text_checksum(dockerfile))
         tmp_df.flush()
+
         self._do(["build", "-t", tag, "-f", tmp_df.name] + argv + \
-                 [tmp_dir],
+                 [docker_dir],
                  quiet=quiet)
 
     def image_matches_dockerfile(self, tag, dockerfile):
@@ -164,10 +164,15 @@ class BuildCommand(SubCommand):
         if dkr.image_matches_dockerfile(tag, dockerfile):
             if not args.quiet:
                 print "Image is up to date."
-            return 0
+        else:
+            # Create a docker context directory for the build
+            docker_dir = tempfile.mkdtemp(prefix="docker_build")
+
+            dkr.build_image(tag, docker_dir, dockerfile,
+                            quiet=args.quiet, argv=argv)
+
+            rmtree(docker_dir)
 
-        dkr.build_image(tag, dockerfile, args.dockerfile,
-                        quiet=args.quiet, argv=argv)
         return 0
 
 class CleanCommand(SubCommand):
-- 
2.7.4

[Qemu-devel] [PATCH v7 02/10] tests/docker/docker.py: support --include-executable

[Fam Zheng]

From: Alex Bennée <alex.bennee@linaro.org>

When passed the path to a binary we copy it and any linked libraries (if
it is dynamically linked) into the docker build context. These can then
be included by a dockerfile with the line:

  # Copy all of context into container
  ADD . /

This is mainly intended for setting up foreign architecture docker
images which use qemu-$arch to do cross-architecture linux-user
execution. It also relies on the host and guest file-system following
reasonable multi-arch layouts so the copied libraries don't clash with
the guest ones.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/docker.py | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 58 insertions(+)

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index ae40bb3..96d906e 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -20,6 +20,7 @@ import atexit
 import uuid
 import argparse
 import tempfile
+import re
 from shutil import copy, rmtree
 
 def _text_checksum(text):
@@ -38,6 +39,54 @@ def _guess_docker_command():
     raise Exception("Cannot find working docker command. Tried:\n%s" % \
                     commands_txt)
 
+def _copy_with_mkdir(src, root_dir, sub_path):
+    """Copy src into root_dir, creating sub_path as needed."""
+    dest_dir = os.path.normpath("%s/%s" % (root_dir, sub_path))
+    try:
+        os.makedirs(dest_dir)
+    except OSError:
+        # we can safely ignore already created directories
+        pass
+
+    dest_file = "%s/%s" % (dest_dir, os.path.basename(src))
+    copy(src, dest_file)
+
+
+def _get_so_libs(executable):
+    """Return a list of libraries associated with an executable.
+
+    The paths may be symbolic links which would need to be resolved to
+    ensure theright data is copied."""
+
+    libs = []
+    ldd_re = re.compile(r"(/.*/)(\S*)")
+    try:
+        ldd_output = subprocess.check_output(["ldd", executable])
+        for line in ldd_output.split("\n"):
+            search = ldd_re.search(line)
+            if search and len(search.groups()) == 2:
+                so_path = search.groups()[0]
+                so_lib = search.groups()[1]
+                libs.append("%s/%s" % (so_path, so_lib))
+    except subprocess.CalledProcessError:
+        print "%s had no associated libraries (static build?)" % (executable)
+
+    return libs
+
+def _copy_binary_with_libs(src, dest_dir):
+    """Copy a binary executable and all its dependant libraries.
+
+    This does rely on the host file-system being fairly multi-arch
+    aware so the file don't clash with the guests layout."""
+
+    _copy_with_mkdir(src, dest_dir, "/usr/bin")
+
+    libs = _get_so_libs(src)
+    if libs:
+        for l in libs:
+            so_path = os.path.dirname(l)
+            _copy_with_mkdir(l , dest_dir, so_path)
+
 class Docker(object):
     """ Running Docker commands """
     def __init__(self):
@@ -151,6 +200,10 @@ class BuildCommand(SubCommand):
     """ Build docker image out of a dockerfile. Arguments: <tag> <dockerfile>"""
     name = "build"
     def args(self, parser):
+        parser.add_argument("--include-executable", "-e",
+                            help="""Specify a binary that will be copied to the
+                            container together with all its dependent
+                            libraries""")
         parser.add_argument("tag",
                             help="Image Tag")
         parser.add_argument("dockerfile",
@@ -168,6 +221,11 @@ class BuildCommand(SubCommand):
             # Create a docker context directory for the build
             docker_dir = tempfile.mkdtemp(prefix="docker_build")
 
+            # Do we include a extra binary?
+            if args.include_executable:
+                _copy_binary_with_libs(args.include_executable,
+                                       docker_dir)
+
             dkr.build_image(tag, docker_dir, dockerfile,
                             quiet=args.quiet, argv=argv)
 
-- 
2.7.4

[Qemu-devel] [PATCH v7 03/10] tests/docker/docker.py: check and run .pre script

[Fam Zheng]

From: Alex Bennée <alex.bennee@linaro.org>

The docker script will now search for an associated $dockerfile.pre
script which gets run in the same build context as the dockerfile will
be. This is to support pre-seeding the build context before running the
docker build.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/docker.py | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index 96d906e..76750c4 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -221,6 +221,18 @@ class BuildCommand(SubCommand):
             # Create a docker context directory for the build
             docker_dir = tempfile.mkdtemp(prefix="docker_build")
 
+            # Is there a .pre file to run in the build context?
+            docker_pre = os.path.splitext(args.dockerfile)[0]+".pre"
+            if os.path.exists(docker_pre):
+                rc = subprocess.call(os.path.realpath(docker_pre),
+                                     cwd=docker_dir)
+                if rc == 3:
+                    print "Skip"
+                    return 0
+                elif rc != 0:
+                    print "%s exited with code %d" % (docker_pre, rc)
+                    return 1
+
             # Do we include a extra binary?
             if args.include_executable:
                 _copy_binary_with_libs(args.include_executable,
-- 
2.7.4

[Qemu-devel] [PATCH v7 04/10] tests/docker/dockerfiles: new debian-bootstrap.docker

[Fam Zheng]

From: Alex Bennée <alex.bennee@linaro.org>

Together with the debian-bootstrap.pre script can now build an arbitrary
architecture of Debian using debootstrap. This allows debootstrap to set
up its first stage before the container is built.

To build a container you need a command line like:

  DEB_ARCH=armhf DEB_TYPE=testing \
    ./tests/docker/docker.py build \
    --include-executable=arm-linux-user/qemu-arm debian:armhf \
    ./tests/docker/dockerfiles/debian-bootstrap.docker

Although a number of non-debian systems package the debootstrap script
it is fairly portable in itself. Assuming we have some sort of fakeroot
implementation we can just clone the upstream repository and use the
script from there.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/dockerfiles/debian-bootstrap.docker | 21 ++++++
 tests/docker/dockerfiles/debian-bootstrap.pre    | 87 ++++++++++++++++++++++++
 2 files changed, 108 insertions(+)
 create mode 100644 tests/docker/dockerfiles/debian-bootstrap.docker
 create mode 100755 tests/docker/dockerfiles/debian-bootstrap.pre

diff --git a/tests/docker/dockerfiles/debian-bootstrap.docker b/tests/docker/dockerfiles/debian-bootstrap.docker
new file mode 100644
index 0000000..3a9125e
--- /dev/null
+++ b/tests/docker/dockerfiles/debian-bootstrap.docker
@@ -0,0 +1,21 @@
+# Create Debian Bootstrap Image
+#
+# This is intended to be pre-poluated by:
+#  - a first stage debootstrap (see debian-bootstrap.pre)
+#  - a native qemu-$arch that binfmt_misc will run
+FROM scratch
+
+# Add everything from the context into the container
+ADD . /
+
+# Patch all mounts as docker already has stuff set up
+RUN sed -i 's/in_target mount/echo not for docker in_target mount/g' /debootstrap/functions
+
+# Run stage 2
+RUN /debootstrap/debootstrap --second-stage
+
+# At this point we can install additional packages if we want
+# Duplicate deb line as deb-src
+RUN cat /etc/apt/sources.list | sed "s/deb/deb-src/" >> /etc/apt/sources.list
+RUN apt-get update
+RUN apt-get -y build-dep qemu
diff --git a/tests/docker/dockerfiles/debian-bootstrap.pre b/tests/docker/dockerfiles/debian-bootstrap.pre
new file mode 100755
index 0000000..5d9c8d5
--- /dev/null
+++ b/tests/docker/dockerfiles/debian-bootstrap.pre
@@ -0,0 +1,87 @@
+#!/bin/sh
+#
+# Simple wrapper for debootstrap, run in the docker build context
+#
+FAKEROOT=`which fakeroot 2> /dev/null`
+
+exit_and_skip()
+{
+    exit 3
+}
+
+#
+# fakeroot is needed to run the bootstrap stage
+#
+if [ -z $FAKEROOT ]; then
+    echo "Please install fakeroot to enable bootstraping"
+    exit_and_skip
+fi
+
+# We check in order for
+#
+#  - DEBOOTSTRAP_DIR pointing at a development checkout
+#  - PATH for the debootstrap script (installed)
+#
+# If neither option works then we checkout debootstrap from its
+# upstream SCM and run it from there.
+#
+
+if [ -z $DEBOOTSTRAP_DIR ]; then
+    DEBOOTSTRAP=`which debootstrap 2> /dev/null`
+    if [ -z $DEBOOTSTRAP ]; then
+        echo "No debootstrap installed, attempting to install from SCM"
+        DEBOOTSTRAP_SOURCE=https://anonscm.debian.org/git/d-i/debootstrap.git
+        git clone ${DEBOOTSTRAP_SOURCE} ./debootstrap.git
+        export DEBOOTSTRAP_DIR=./debootstrap.git
+        DEBOOTSTRAP=./debootstrap.git/debootstrap
+    fi
+else
+    DEBOOTSTRAP=${DEBOOTSTRAP_DIR}/debootstrap
+    if [ ! -f $DEBOOTSTRAP ]; then
+        echo "Couldn't find script at ${DEBOOTSTRAP}"
+        exit_and_skip
+    fi
+fi
+
+#
+# Finally check to see if any qemu's are installed
+#
+BINFMT_DIR=/proc/sys/fs/binfmt_misc
+if [ ! -e $BINFMT_DIR ]; then
+   echo "binfmt_misc needs enabling for a QEMU bootstrap to work"
+   exit_and_skip
+else
+    # DEB_ARCH and QEMU arch names are not totally aligned
+    case "${DEB_ARCH}" in
+        amd64)
+            QEMU=qemu-i386
+            ;;
+        armel|armhf)
+            QEMU=qemu-arm
+            ;;
+        arm64)
+            QEMU=qemu-aarch64
+            ;;
+        powerpc)
+            QEMU=qemu-ppc
+            ;;
+        ppc64el)
+            QEMU=qemu-ppc64le
+            ;;
+        s390)
+            QEMU=qemu-s390x
+            ;;
+        *)
+            QEMU=qemu-${DEB_ARCH}
+        ;;
+    esac
+    if [ ! -e "${BINFMT_DIR}/$QEMU" ]; then
+        echo "No binfmt_misc rule to run $QEMU, can't bootstrap"
+        exit_and_skip
+    fi
+fi
+
+echo "Building a rootfs using ${FAKEROOT} and ${DEBOOTSTRAP} ${DEB_ARCH}/${DEB_TYPE}"
+
+${FAKEROOT} ${DEBOOTSTRAP} --variant=buildd --foreign --arch=$DEB_ARCH $DEB_TYPE . http://httpredir.debian.org/debian || exit 1
+exit 0
-- 
2.7.4

[Qemu-devel] [PATCH v7 05/10] tests/docker/docker.py: add update operation

[Fam Zheng]

From: Alex Bennée <alex.bennee@linaro.org>

This adds a new operation to the docker script to allow updating of
binaries in an existing container. This is because it would be
inefficient to re-build the whole container just for an update to the
QEMU binary.

To update the executable run:

    ./tests/docker/docker.py update \
        debian:armhf ./arm-linux-user/qemu-arm

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/docker.py | 59 +++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 58 insertions(+), 1 deletion(-)

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index 76750c4..40bda9d 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -21,6 +21,8 @@ import uuid
 import argparse
 import tempfile
 import re
+from tarfile import TarFile, TarInfo
+from StringIO import StringIO
 from shutil import copy, rmtree
 
 def _text_checksum(text):
@@ -94,9 +96,11 @@ class Docker(object):
         self._instances = []
         atexit.register(self._kill_instances)
 
-    def _do(self, cmd, quiet=True, **kwargs):
+    def _do(self, cmd, quiet=True, infile=None, **kwargs):
         if quiet:
             kwargs["stdout"] = subprocess.PIPE
+        if infile:
+            kwargs["stdin"] = infile
         return subprocess.call(self._command + cmd, **kwargs)
 
     def _do_kill_instances(self, only_known, only_active=True):
@@ -152,6 +156,11 @@ class Docker(object):
                  [docker_dir],
                  quiet=quiet)
 
+    def update_image(self, tag, tarball, quiet=True):
+        "Update a tagged image using "
+
+        self._do(["build", "-t", tag, "-"], quiet=quiet, infile=tarball)
+
     def image_matches_dockerfile(self, tag, dockerfile):
         try:
             checksum = self.get_image_dockerfile_checksum(tag)
@@ -245,6 +254,54 @@ class BuildCommand(SubCommand):
 
         return 0
 
+class UpdateCommand(SubCommand):
+    """ Update a docker image with new executables. Arguments: <tag> <executable>"""
+    name = "update"
+    def args(self, parser):
+        parser.add_argument("tag",
+                            help="Image Tag")
+        parser.add_argument("executable",
+                            help="Executable to copy")
+
+    def run(self, args, argv):
+        # Create a temporary tarball with our whole build context and
+        # dockerfile for the update
+        tmp = tempfile.NamedTemporaryFile(suffix="dckr.tar.gz")
+        tmp_tar = TarFile(fileobj=tmp, mode='w')
+
+        # Add the executable to the tarball
+        bn = os.path.basename(args.executable)
+        ff = "/usr/bin/%s" % bn
+        tmp_tar.add(args.executable, arcname=ff)
+
+        # Add any associated libraries
+        libs = _get_so_libs(args.executable)
+        if libs:
+            for l in libs:
+                tmp_tar.add(os.path.realpath(l), arcname=l)
+
+        # Create a Docker buildfile
+        df = StringIO()
+        df.write("FROM %s\n" % args.tag)
+        df.write("ADD . /\n")
+        df.seek(0)
+
+        df_tar = TarInfo(name="Dockerfile")
+        df_tar.size = len(df.buf)
+        tmp_tar.addfile(df_tar, fileobj=df)
+
+        tmp_tar.close()
+
+        # reset the file pointers
+        tmp.flush()
+        tmp.seek(0)
+
+        # Run the build with our tarball context
+        dkr = Docker()
+        dkr.update_image(args.tag, tmp, quiet=args.quiet)
+
+        return 0
+
 class CleanCommand(SubCommand):
     """Clean up docker instances"""
     name = "clean"
-- 
2.7.4

[Qemu-devel] [PATCH v7 06/10] docker: More sensible run script

[Fam Zheng]

It is very easy to figure out current directory and bash option from the
execution, so do less in the Makefile invocation command line, and
figure both options in the script.

This makes the next patch easier.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/Makefile.include |  4 +---
 tests/docker/run              | 12 +++++++++---
 2 files changed, 10 insertions(+), 6 deletions(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index f88c0a7..c5546ee 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -114,10 +114,8 @@ docker-run-%: docker-qemu-src
 				-e CCACHE_DIR=/var/tmp/ccache \
 				-v $$(realpath $(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
 				-v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z \
-				-w /var/tmp/qemu \
 				qemu:$(IMAGE) \
-				$(if $V,/bin/bash -x ,) \
-				./run \
+				/var/tmp/qemu/run \
 				$(CMD); \
 			, "  RUN $(CMD) in $(IMAGE)")))
 
diff --git a/tests/docker/run b/tests/docker/run
index ec3d119..ddfac05 100755
--- a/tests/docker/run
+++ b/tests/docker/run
@@ -11,6 +11,12 @@
 # or (at your option) any later version. See the COPYING file in
 # the top-level directory.
 
+if test -n "$V"; then
+    set -x
+fi
+
+BASE="$(dirname $(readlink -e $0))"
+
 # Prepare the environment
 . /etc/profile || true
 export PATH=/usr/lib/ccache:$PATH
@@ -24,10 +30,10 @@ export TEST_DIR=/tmp/qemu-test
 mkdir -p $TEST_DIR/{src,build,install}
 
 # Extract the source tarballs
-tar -C $TEST_DIR/src -xzf qemu.tgz
+tar -C $TEST_DIR/src -xzf $BASE/qemu.tgz
 for p in dtc pixman; do
-    if test -f $p.tgz; then
-        tar -C $TEST_DIR/src/$p -xzf $p.tgz
+    if test -f $BASE/$p.tgz; then
+        tar -C $TEST_DIR/src/$p -xzf $BASE/$p.tgz
         export FEATURES="$FEATURES $p"
     fi
 done
-- 
2.7.4

[Qemu-devel] [PATCH v7 07/10] docker: Fix exit code if $CMD failed

[Fam Zheng]

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/run | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tests/docker/run b/tests/docker/run
index ddfac05..d85d49a 100755
--- a/tests/docker/run
+++ b/tests/docker/run
@@ -11,6 +11,8 @@
 # or (at your option) any later version. See the COPYING file in
 # the top-level directory.
 
+set -e
+
 if test -n "$V"; then
     set -x
 fi
@@ -61,4 +63,6 @@ elif test -n "$DEBUG"; then
     echo
     # Force error after shell exits
     $SHELL && exit 1
+else
+    exit 1
 fi
-- 
2.7.4

[Qemu-devel] [PATCH v7 08/10] docker: Add "images" subcommand to docker.py

[Fam Zheng]

This is a wrapper for the 'docker images' command.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/docker.py | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index 40bda9d..222a105 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -179,6 +179,9 @@ class Docker(object):
             self._instances.remove(label)
         return ret
 
+    def command(self, cmd, argv, quiet):
+        return self._do([cmd] + argv, quiet=quiet)
+
 class SubCommand(object):
     """A SubCommand template base class"""
     name = None # Subcommand name
@@ -309,6 +312,12 @@ class CleanCommand(SubCommand):
         Docker().clean()
         return 0
 
+class ImagesCommand(SubCommand):
+    """Run "docker images" command"""
+    name = "images"
+    def run(self, args, argv):
+        return Docker().command("images", argv, args.quiet)
+
 def main():
     parser = argparse.ArgumentParser(description="A Docker helper",
             usage="%s <subcommand> ..." % os.path.basename(sys.argv[0]))
-- 
2.7.4

Re: [Qemu-devel] [PATCH v7 08/10] docker: Add "images" subcommand to docker.py

[Alex Bennée]

Fam Zheng <famz@redhat.com> writes:

> This is a wrapper for the 'docker images' command.
>
> Signed-off-by: Fam Zheng <famz@redhat.com>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

> ---
>  tests/docker/docker.py | 9 +++++++++
>  1 file changed, 9 insertions(+)
>
> diff --git a/tests/docker/docker.py b/tests/docker/docker.py
> index 40bda9d..222a105 100755
> --- a/tests/docker/docker.py
> +++ b/tests/docker/docker.py
> @@ -179,6 +179,9 @@ class Docker(object):
>              self._instances.remove(label)
>          return ret
>
> +    def command(self, cmd, argv, quiet):
> +        return self._do([cmd] + argv, quiet=quiet)
> +
>  class SubCommand(object):
>      """A SubCommand template base class"""
>      name = None # Subcommand name
> @@ -309,6 +312,12 @@ class CleanCommand(SubCommand):
>          Docker().clean()
>          return 0
>
> +class ImagesCommand(SubCommand):
> +    """Run "docker images" command"""
> +    name = "images"
> +    def run(self, args, argv):
> +        return Docker().command("images", argv, args.quiet)
> +
>  def main():
>      parser = argparse.ArgumentParser(description="A Docker helper",
>              usage="%s <subcommand> ..." % os.path.basename(sys.argv[0]))


--
Alex Bennée

[Qemu-devel] [PATCH v7 09/10] docker: Don't start a container that doesn't exist

[Fam Zheng]

Image building targets are dependencies of test running targets, so when
a docker image doesn't exist, it means it's skipped (due to dependency
checks in pre script). Therefore, skip the test too.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/docker/Makefile.include | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index c5546ee..e7f0023 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -105,7 +105,10 @@ docker-run-%: docker-qemu-src
 	fi
 	$(if $(filter $(TESTS),$(CMD)),$(if $(filter $(IMAGES),$(IMAGE)), \
 		$(call quiet-command,\
-			$(SRC_PATH)/tests/docker/docker.py run $(if $V,,--rm) \
+			if $(SRC_PATH)/tests/docker/docker.py images \
+				--format={{.Repository}}:{{.Tag}} | \
+					grep -qx qemu:$(IMAGE); then \
+				$(SRC_PATH)/tests/docker/docker.py run $(if $V,,--rm) \
 				-t \
 				$(if $(DEBUG),-i,--net=none) \
 				-e TARGET_LIST=$(TARGET_LIST) \
@@ -117,6 +120,7 @@ docker-run-%: docker-qemu-src
 				qemu:$(IMAGE) \
 				/var/tmp/qemu/run \
 				$(CMD); \
+			fi \
 			, "  RUN $(CMD) in $(IMAGE)")))
 
 docker-clean:
-- 
2.7.4

[Qemu-devel] [PATCH v7 10/10] docker: pass EXECUTABLE to build script

[Fam Zheng]

From: Alex Bennée <alex.bennee@linaro.org>

To build a docker image with which needs qemu linux-user emulation we
need to pass --include-executable to the build script. Using the same
mechanism as for other container controls we enable the option is
EXECUTABLE is set on the make command line e.g:

    make docker-image-debian-bootstrap V=1 J=9 DEB_ARCH=armhf \
        DEB_TYPE=stable EXECUTABLE=./arm-linux-user/qemu-arm

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/Makefile.include | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index e7f0023..78af468 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -46,7 +46,8 @@ docker-image: ${DOCKER_TARGETS}
 docker-image-%: $(DOCKER_FILES_DIR)/%.docker
 	$(call quiet-command,\
 		$(SRC_PATH)/tests/docker/docker.py build qemu:$* $< \
-		$(if $V,,--quiet) $(if $(NOCACHE),--no-cache),\
+		$(if $V,,--quiet) $(if $(NOCACHE),--no-cache) \
+		$(if $(EXECUTABLE),--include-executable=$(EXECUTABLE)),\
 		"  BUILD $*")
 
 # Expand all the pre-requistes for each docker image and test combination
@@ -95,6 +96,7 @@ docker:
 	@echo '    DEBUG=1              Stop and drop to shell in the created container'
 	@echo '                         before running the command.'
 	@echo '    NOCACHE=1            Ignore cache when build images.'
+	@echo '    EXECUTABLE=<path>    Include executable in image.'
 
 docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/')
 docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/')
-- 
2.7.4