qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: "Philippe Mathieu-Daudé" <f4bug@amsat.org>
To: qemu-devel@nongnu.org, "Alex Bennée" <alex.bennee@linaro.org>,
	"Fam Zheng" <famz@redhat.com>
Cc: "Philippe Mathieu-Daudé" <f4bug@amsat.org>
Subject: [Qemu-devel] [PATCH v4 03/24] docker: rebuild image if 'extra files' checksum does not match
Date: Sat, 27 May 2017 13:17:34 -0300	[thread overview]
Message-ID: <20170527161755.7761-4-f4bug@amsat.org> (raw)
In-Reply-To: <20170527161755.7761-1-f4bug@amsat.org>

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 tests/docker/docker.py | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index 5401e58fce..b5ee8bd6e3 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -38,6 +38,9 @@ def _text_checksum(text):
     """Calculate a digest string unique to the text content"""
     return hashlib.sha1(text).hexdigest()
 
+def _file_checksum(filename):
+    return _text_checksum(open(filename, 'rb').read())
+
 def _guess_docker_command():
     """ Guess a working docker command or raise exception if not found"""
     commands = [["docker"], ["sudo", "-n", "docker"]]
@@ -154,7 +157,7 @@ class Docker(object):
         return labels.get("com.qemu.dockerfile-checksum", "")
 
     def build_image(self, tag, docker_dir, dockerfile,
-                    quiet=True, user=False, argv=None):
+                    quiet=True, user=False, argv=None, extra_files_cksum=[]):
         if argv == None:
             argv = []
 
@@ -170,7 +173,8 @@ class Docker(object):
 
         tmp_df.write("\n")
         tmp_df.write("LABEL com.qemu.dockerfile-checksum=%s" %
-                     _text_checksum(dockerfile))
+                     _text_checksum("\n".join([dockerfile] +
+                                    extra_files_cksum)))
         tmp_df.flush()
 
         self._do(["build", "-t", tag, "-f", tmp_df.name] + argv + \
@@ -275,16 +279,22 @@ class BuildCommand(SubCommand):
                     return 1
 
             # Include files used by ADD directives found within the Dockerfile.
+            cksum = []
             if args.include_executable:
+                # FIXME: there is no checksum of this executable and the linked
+                # libraries, once the image built any change of this executable
+                # or any library won't trigger another build.
                 _copy_binary_with_libs(args.include_executable, docker_dir)
             for filename in args.extra_files or []:
                 _copy_with_mkdir(filename, docker_dir)
+                cksum += [_file_checksum(filename)]
 
             argv += ["--build-arg=" + k.lower() + "=" + v
                         for k, v in os.environ.iteritems()
                         if k.lower() in FILTERED_ENV_NAMES]
             dkr.build_image(tag, docker_dir, dockerfile,
-                            quiet=args.quiet, user=args.user, argv=argv)
+                            quiet=args.quiet, user=args.user, argv=argv,
+                            extra_files_cksum=cksum)
 
             rmtree(docker_dir)
 
-- 
2.11.0

  parent reply	other threads:[~2017-05-27 16:18 UTC|newest]

Thread overview: 28+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-05-27 16:17 [Qemu-devel] [PATCH v4 00/24] docker/shippable: cross-build mipsel and powerpc targets Philippe Mathieu-Daudé
2017-05-27 16:17 ` [Qemu-devel] [PATCH v4 01/24] docker: let _copy_with_mkdir() sub_path argument be optional Philippe Mathieu-Daudé
2017-05-27 16:17 ` [Qemu-devel] [PATCH v4 02/24] docker: add --include-files argument to 'build' command Philippe Mathieu-Daudé
2017-05-31 16:31   ` Alex Bennée
2017-06-01 10:00     ` Fam Zheng
2017-05-27 16:17 ` Philippe Mathieu-Daudé [this message]
2017-05-31 16:32   ` [Qemu-devel] [PATCH v4 03/24] docker: rebuild image if 'extra files' checksum does not match Alex Bennée
2017-05-27 16:17 ` [Qemu-devel] [PATCH v4 04/24] docker: add 'apt-fake' script which generate fake debian packages Philippe Mathieu-Daudé
2017-05-27 16:17 ` [Qemu-devel] [PATCH v4 05/24] docker: install ca-certificates package in base image Philippe Mathieu-Daudé
2017-05-27 16:17 ` [Qemu-devel] [PATCH v4 06/24] docker: create deb-src entry and setup Emdebian in the same layer Philippe Mathieu-Daudé
2017-05-27 16:17 ` [Qemu-devel] [PATCH v4 07/24] docker: use better regex to generate deb-src entries Philippe Mathieu-Daudé
2017-05-27 16:17 ` [Qemu-devel] [PATCH v4 08/24] docker: use eatmydata, install common build packages in base image Philippe Mathieu-Daudé
2017-05-27 16:17 ` [Qemu-devel] [PATCH v4 09/24] docker: use eatmydata in debian armhf image Philippe Mathieu-Daudé
2017-05-27 16:17 ` [Qemu-devel] [PATCH v4 10/24] docker: use eatmydata in debian arm64 image Philippe Mathieu-Daudé
2017-05-27 16:17 ` [Qemu-devel] [PATCH v4 11/24] docker: add extra libs to armhf target to extend codebase coverage Philippe Mathieu-Daudé
2017-05-27 16:17 ` [Qemu-devel] [PATCH v4 12/24] docker: add extra libs to arm64 " Philippe Mathieu-Daudé
2017-05-27 16:17 ` [Qemu-devel] [PATCH v4 13/24] docker: add extra libs to s390x " Philippe Mathieu-Daudé
2017-05-27 16:17 ` [Qemu-devel] [PATCH v4 14/24] docker: add mipsel build target Philippe Mathieu-Daudé
2017-05-27 16:17 ` [Qemu-devel] [PATCH v4 15/24] docker: add powerpc " Philippe Mathieu-Daudé
2017-05-27 16:17 ` [Qemu-devel] [PATCH v4 16/24] shippable: use C locale to simplify console output Philippe Mathieu-Daudé
2017-05-27 16:17 ` [Qemu-devel] [PATCH v4 17/24] shippable: build using all available cpus Philippe Mathieu-Daudé
2017-05-27 16:17 ` [Qemu-devel] [PATCH v4 18/24] shippable: do not initialize submodules automatically Philippe Mathieu-Daudé
2017-05-27 16:17 ` [Qemu-devel] [PATCH v4 19/24] shippable: be verbose while building docker images Philippe Mathieu-Daudé
2017-05-27 16:17 ` [Qemu-devel] [PATCH v4 20/24] shippable: add armeb-linux-user target Philippe Mathieu-Daudé
2017-05-27 16:17 ` [Qemu-devel] [PATCH v4 21/24] shippable: add powerpc target Philippe Mathieu-Daudé
2017-05-27 16:17 ` [Qemu-devel] [PATCH v4 22/24] shippable: add mipsel target Philippe Mathieu-Daudé
2017-05-27 16:17 ` [Qemu-devel] [PATCH v4 23/24] MAINTAINERS: add Shippable automation platform URL Philippe Mathieu-Daudé
2017-05-27 16:17 ` [Qemu-devel] [PATCH v4 24/24] MAINTAINERS: self-appoint me as reviewer in build/test automation Philippe Mathieu-Daudé

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170527161755.7761-4-f4bug@amsat.org \
    --to=f4bug@amsat.org \
    --cc=alex.bennee@linaro.org \
    --cc=famz@redhat.com \
    --cc=qemu-devel@nongnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).