qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: "Alex Bennée" <alex.bennee@linaro.org>
To: peter.maydell@linaro.org
Cc: qemu-devel@nongnu.org, "Philippe Mathieu-Daudé" <f4bug@amsat.org>,
	"Alex Bennée" <alex.bennee@linaro.org>,
	"Fam Zheng" <famz@redhat.com>
Subject: [Qemu-devel] [PULL 04/21] docker: rebuild image if 'extra files' checksum does not match
Date: Wed, 21 Jun 2017 15:47:57 +0100	[thread overview]
Message-ID: <20170621144814.15324-5-alex.bennee@linaro.org> (raw)
In-Reply-To: <20170621144814.15324-1-alex.bennee@linaro.org>

From: Philippe Mathieu-Daudé <f4bug@amsat.org>

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index 10fa907905..e707e5bcca 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -38,6 +38,9 @@ def _text_checksum(text):
     """Calculate a digest string unique to the text content"""
     return hashlib.sha1(text).hexdigest()
 
+def _file_checksum(filename):
+    return _text_checksum(open(filename, 'rb').read())
+
 def _guess_docker_command():
     """ Guess a working docker command or raise exception if not found"""
     commands = [["docker"], ["sudo", "-n", "docker"]]
@@ -154,7 +157,7 @@ class Docker(object):
         return labels.get("com.qemu.dockerfile-checksum", "")
 
     def build_image(self, tag, docker_dir, dockerfile,
-                    quiet=True, user=False, argv=None):
+                    quiet=True, user=False, argv=None, extra_files_cksum=[]):
         if argv == None:
             argv = []
 
@@ -170,7 +173,8 @@ class Docker(object):
 
         tmp_df.write("\n")
         tmp_df.write("LABEL com.qemu.dockerfile-checksum=%s" %
-                     _text_checksum(dockerfile))
+                     _text_checksum("\n".join([dockerfile] +
+                                    extra_files_cksum)))
         tmp_df.flush()
 
         self._do(["build", "-t", tag, "-f", tmp_df.name] + argv + \
@@ -276,16 +280,22 @@ class BuildCommand(SubCommand):
 
             # Copy any extra files into the Docker context. These can be
             # included by the use of the ADD directive in the Dockerfile.
+            cksum = []
             if args.include_executable:
+                # FIXME: there is no checksum of this executable and the linked
+                # libraries, once the image built any change of this executable
+                # or any library won't trigger another build.
                 _copy_binary_with_libs(args.include_executable, docker_dir)
             for filename in args.extra_files or []:
                 _copy_with_mkdir(filename, docker_dir)
+                cksum += [_file_checksum(filename)]
 
             argv += ["--build-arg=" + k.lower() + "=" + v
                         for k, v in os.environ.iteritems()
                         if k.lower() in FILTERED_ENV_NAMES]
             dkr.build_image(tag, docker_dir, dockerfile,
-                            quiet=args.quiet, user=args.user, argv=argv)
+                            quiet=args.quiet, user=args.user, argv=argv,
+                            extra_files_cksum=cksum)
 
             rmtree(docker_dir)
 
-- 
2.13.0

  parent reply	other threads:[~2017-06-21 14:47 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-06-21 14:47 [Qemu-devel] [PULL 00/21] Docker and shippable updates Alex Bennée
2017-06-21 14:47 ` [Qemu-devel] [PULL 01/21] docker: update qemu:debian base following stretch release Alex Bennée
2017-06-21 14:47 ` [Qemu-devel] [PULL 02/21] docker: let _copy_with_mkdir() sub_path argument be optional Alex Bennée
2017-06-21 14:47 ` [Qemu-devel] [PULL 03/21] docker: add --include-files argument to 'build' command Alex Bennée
2017-06-21 14:47 ` Alex Bennée [this message]
2017-06-21 14:47 ` [Qemu-devel] [PULL 05/21] docker: install ca-certificates package in base image Alex Bennée
2017-06-21 14:47 ` [Qemu-devel] [PULL 06/21] docker: use better regex to generate deb-src entries Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 07/21] docker: use eatmydata, install common build packages in base image Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 08/21] docker: use eatmydata in debian armhf image Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 09/21] docker: use eatmydata in debian arm64 image Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 10/21] docker: add extra libs to armhf target to extend codebase coverage Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 11/21] docker: add extra libs to arm64 " Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 12/21] docker: add extra libs to s390x " Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 13/21] docker: add mipsel build target Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 14/21] shippable: use C locale to simplify console output Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 15/21] shippable: build using all available cpus Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 16/21] shippable: do not initialize submodules automatically Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 17/21] shippable: be verbose while building docker images Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 18/21] shippable: add armeb-linux-user target Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 19/21] shippable: add mipsel target Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 20/21] MAINTAINERS: add Shippable automation platform URL Alex Bennée
2017-06-21 14:48 ` [Qemu-devel] [PULL 21/21] MAINTAINERS: self-appoint me as reviewer in build/test automation Alex Bennée
2017-06-22 14:09 ` [Qemu-devel] [PULL 00/21] Docker and shippable updates Peter Maydell
2017-06-22 14:58   ` Philippe Mathieu-Daudé
2017-06-22 15:13     ` Alex Bennée
2017-06-22 15:42       ` Philippe Mathieu-Daudé
2017-07-10 12:04         ` Philippe Mathieu-Daudé
2017-07-10 14:17           ` Alex Bennée
2017-07-10 14:22             ` Peter Maydell
2017-07-12 12:17               ` Philippe Mathieu-Daudé

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170621144814.15324-5-alex.bennee@linaro.org \
    --to=alex.bennee@linaro.org \
    --cc=f4bug@amsat.org \
    --cc=famz@redhat.com \
    --cc=peter.maydell@linaro.org \
    --cc=qemu-devel@nongnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).