public inbox for igt-dev@lists.freedesktop.org
 help / color / mirror / Atom feed
From: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
To: igt-dev@lists.freedesktop.org
Cc: Petri Latvala <petri.latvala@intel.com>
Subject: [igt-dev] [PATCH i-g-t 4/8] .gitalb-ci: Use only the necessary tags
Date: Tue, 3 Sep 2019 11:36:10 +0300	[thread overview]
Message-ID: <20190903083614.31305-4-arkadiusz.hiler@intel.com> (raw)
In-Reply-To: <20190903083614.31305-1-arkadiusz.hiler@intel.com>

Tag build-* images only with dockerfile-$FILE_SHA1 for changes detection
and commit-$COMMIT_SHA1, so we know which set of images we have to use
for any given commit.

The igt container needs only ref (e.g. master) and commit-$COMMIT_SHA1
tags, as the dockerfile-$FILE_SHA1 one is irrelevant - we build build
every time anyway.

Signed-off-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
---
 .gitlab-ci.yml                | 14 +++++------
 .gitlab-ci/pull-or-rebuild.sh | 47 +++++++++++++++++++++--------------
 2 files changed, 36 insertions(+), 25 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index c5611138..5b983618 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -30,10 +30,10 @@ build-containers:build-debian:
     DOCKER_DRIVER: overlay2
   script:
     - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
-    - .gitlab-ci/pull-or-rebuild.sh Dockerfile.build-debian-minimal build-debian-minimal
+    - .gitlab-ci/pull-or-rebuild.sh base Dockerfile.build-debian-minimal build-debian-minimal
     - docker pull $CI_REGISTRY/$CI_PROJECT_PATH/build-debian-minimal:commit-$CI_COMMIT_SHA
     - docker tag $CI_REGISTRY/$CI_PROJECT_PATH/build-debian-minimal:commit-$CI_COMMIT_SHA build-debian-minimal:commit-$CI_COMMIT_SHA
-    - .gitlab-ci/pull-or-rebuild.sh Dockerfile.build-debian build-debian
+    - .gitlab-ci/pull-or-rebuild.sh base Dockerfile.build-debian build-debian
 
 build-containers:build-debian-armhf:
   stage: build-containers
@@ -45,7 +45,7 @@ build-containers:build-debian-armhf:
     DOCKER_DRIVER: overlay2
   script:
     - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
-    - .gitlab-ci/pull-or-rebuild.sh Dockerfile.build-debian-armhf build-debian-armhf
+    - .gitlab-ci/pull-or-rebuild.sh base Dockerfile.build-debian-armhf build-debian-armhf
 
 build-containers:build-debian-arm64:
   stage: build-containers
@@ -57,7 +57,7 @@ build-containers:build-debian-arm64:
     DOCKER_DRIVER: overlay2
   script:
     - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
-    - .gitlab-ci/pull-or-rebuild.sh Dockerfile.build-debian-arm64 build-debian-arm64
+    - .gitlab-ci/pull-or-rebuild.sh base Dockerfile.build-debian-arm64 build-debian-arm64
 
 build-containers:build-debian-mips:
   stage: build-containers
@@ -69,7 +69,7 @@ build-containers:build-debian-mips:
     DOCKER_DRIVER: overlay2
   script:
     - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
-    - .gitlab-ci/pull-or-rebuild.sh Dockerfile.build-debian-mips build-debian-mips
+    - .gitlab-ci/pull-or-rebuild.sh base Dockerfile.build-debian-mips build-debian-mips
 
 build-containers:build-fedora:
   stage: build-containers
@@ -81,7 +81,7 @@ build-containers:build-fedora:
     DOCKER_DRIVER: overlay2
   script:
     - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
-    - .gitlab-ci/pull-or-rebuild.sh Dockerfile.build-fedora build-fedora
+    - .gitlab-ci/pull-or-rebuild.sh base Dockerfile.build-fedora build-fedora
 
 #################### BUILD #########################
 
@@ -296,4 +296,4 @@ containers:igt:
     - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
     - docker pull $CI_REGISTRY/$CI_PROJECT_PATH/build-fedora:commit-$CI_COMMIT_SHA
     - docker tag $CI_REGISTRY/$CI_PROJECT_PATH/build-fedora:commit-$CI_COMMIT_SHA build-fedora:commit-$CI_COMMIT_SHA
-    - .gitlab-ci/pull-or-rebuild.sh Dockerfile igt
+    - .gitlab-ci/pull-or-rebuild.sh igt Dockerfile igt
diff --git a/.gitlab-ci/pull-or-rebuild.sh b/.gitlab-ci/pull-or-rebuild.sh
index 7e69477e..3f00e831 100755
--- a/.gitlab-ci/pull-or-rebuild.sh
+++ b/.gitlab-ci/pull-or-rebuild.sh
@@ -21,9 +21,10 @@
 # FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
 # IN THE SOFTWARE.
 
-DOCKERFILE=$1
-NAME=$2
-REF=${3:-${CI_COMMIT_REF_NAME:-latest}}
+TYPE=$1
+DOCKERFILE=$2
+NAME=$3
+REF=${4:-${CI_COMMIT_REF_NAME:-latest}}
 
 REF=$(echo $REF | tr / - )
 IMAGENAME=$CI_REGISTRY/$CI_PROJECT_PATH/$NAME
@@ -33,21 +34,31 @@ REFNAME=$IMAGENAME:$REF
 DOCKERNAME=$IMAGENAME:dockerfile-$DOCKERFILE_CHECKSUM
 COMMITNAME=$IMAGENAME:commit-$CI_COMMIT_SHA
 
-docker pull $DOCKERNAME
-IMAGE_PRESENT=$?
+if [ "$TYPE" = "base" ]; then
+	# base container (building, etc) - we rebuild only if changed or forced
+	docker pull $DOCKERNAME
+	IMAGE_PRESENT=$?
 
-set -e
-if [ $IMAGE_PRESENT -eq 0 ] && [ ${FORCE_REBUILD:-0} -eq 0 ] ; then
-	echo "Skipping, already built"
-	docker tag $DOCKERNAME $NAME
-	docker tag $DOCKERNAME $REFNAME
-	docker tag $DOCKERNAME $COMMITNAME
-else
-	echo "Building!"
+	set -e
+	if [ $IMAGE_PRESENT -eq 0 ] && [ ${FORCE_REBUILD:-0} -eq 0 ] ; then
+		echo "Skipping, already built"
+		docker tag $DOCKERNAME $COMMITNAME
+	else
+		echo "Building!"
+		docker build --build-arg=CI_COMMIT_SHA=$CI_COMMIT_SHA \
+			     -t $DOCKERNAME -t $COMMITNAME -f $DOCKERFILE .
+		docker push $DOCKERNAME
+    fi
+    docker push $COMMITNAME
+elif [ "$TYPE" = "igt" ]; then
+	# container with IGT, we don't care about Dockerfile changes
+	# we always rebuild
+	set -e
 	docker build --build-arg=CI_COMMIT_SHA=$CI_COMMIT_SHA \
-			 -t $DOCKERNAME -t $NAME \
-			 -t $REFNAME -t $COMMITNAME -f $DOCKERFILE .
-	docker push $DOCKERNAME
+		     -t $REFNAME -t $COMMITNAME -f $DOCKERFILE .
+	docker push $REFNAME
+	docker push $COMMITNAME
+else
+	echo "unknown build type $TYPE"
+	exit 1
 fi
-docker push $REFNAME
-docker push $COMMITNAME
-- 
2.21.0

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

  parent reply	other threads:[~2019-09-03  8:36 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-09-03  8:36 [igt-dev] [PATCH i-g-t 1/8] Dockerfile: Avoid messing with WORKDIR Arkadiusz Hiler
2019-09-03  8:36 ` [igt-dev] [PATCH i-g-t 2/8] Dockerfile: Make sure that we use correct base image Arkadiusz Hiler
2019-09-03  8:36 ` [igt-dev] [PATCH i-g-t 3/8] .gitlab-ci: Remove dead code Arkadiusz Hiler
2019-09-03  8:36 ` Arkadiusz Hiler [this message]
2019-09-03  8:36 ` [igt-dev] [PATCH i-g-t 5/8] .gitlab-ci: Switch to podman/buildah/skopeo Arkadiusz Hiler
2019-09-03  8:37 ` [igt-dev] [PATCH i-g-t 6/8] Dockerfile: Clean package manager caches and squash images Arkadiusz Hiler
2019-09-03  8:38 ` [igt-dev] [PATCH i-g-t 7/8] .gitlab-ci: Don't pull images until they are needed Arkadiusz Hiler
2019-09-03  8:38 ` [igt-dev] [PATCH i-g-t 8/8] .gitlab-ci: Use true --prefix Arkadiusz Hiler
2019-09-03  9:10 ` [igt-dev] ✓ Fi.CI.BAT: success for series starting with [i-g-t,1/8] Dockerfile: Avoid messing with WORKDIR Patchwork
2019-09-03  9:35 ` [igt-dev] [PATCH i-g-t 1/8] " Petri Latvala
2019-09-03 11:06   ` Arkadiusz Hiler
2019-09-03 11:41 ` [igt-dev] ✗ Fi.CI.IGT: failure for series starting with [i-g-t,1/8] " Patchwork
2019-09-03 12:46   ` Arkadiusz Hiler

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190903083614.31305-4-arkadiusz.hiler@intel.com \
    --to=arkadiusz.hiler@intel.com \
    --cc=igt-dev@lists.freedesktop.org \
    --cc=petri.latvala@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox