public inbox for docs@lists.yoctoproject.org
 help / color / mirror / Atom feed
From: Antonin Godard <antonin.godard@bootlin.com>
To: docs@lists.yoctoproject.org
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>,
	 Quentin Schulz <quentin.schulz@cherry.de>,
	 Antonin Godard <antonin.godard@bootlin.com>
Subject: [PATCH 1/8] tools/build-docs-container: guarantee the image to run matches the just-built image
Date: Mon, 22 Dec 2025 13:27:47 +0100	[thread overview]
Message-ID: <20251222-concurrent-safety-v1-1-e3d86e44cd38@bootlin.com> (raw)
In-Reply-To: <20251222-concurrent-safety-v1-0-e3d86e44cd38@bootlin.com>

From: Quentin Schulz <quentin.schulz@cherry.de>

We aren't that interested in tags actually, the only thing it brings is
a belief that we are going to run exactly what we just built. The issue
is that this is incorrect.

Indeed, one could simply run the script in parallel for the same image.
Script runtime A will build the image A and tag it X, Script runtime B
will build the image B and tag it X, Script runtime B will run the tag X
(image B), Script runtime A will run the tag X (image B). Note that this
problem exists whether we are building from the same tree concurrently
or from different yocto-docs trees concurrently.
One way to fix this could be to introduce random numbers in the tag so
that it's always unique, but we would be flooding the system with
useless tags.

Instead, we can use the sha of the generated image and run that sha
directly. If it's the same across rebuilds, it'll stay the same. If it's
different, the sha will be different and thus we are safe from
concurrent use.

The only downside is that we cannot infer from the image sha the
underlying distro we're testing.

Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de>
---
 documentation/tools/build-docs-container | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/documentation/tools/build-docs-container b/documentation/tools/build-docs-container
index 23c3dfe33..2f31295ed 100755
--- a/documentation/tools/build-docs-container
+++ b/documentation/tools/build-docs-container
@@ -80,10 +80,6 @@ main ()
 
   OCI=$(which "$CONTAINERCMD")
 
-  # docker build doesn't accept 2 colons, so "sanitize" the name
-  local sanitized_dockername
-  sanitized_dockername=$(echo "$image" | tr ':.' '-')
-
   local version
   version=$(echo "$image" | awk -F: '{print $NF}')
 
@@ -159,8 +155,13 @@ main ()
       ;;
   esac
 
+  local image_sha
+  image_id_file=$(mktemp)
+  # Don't clutter tmpfs on fails
+  trap 'rm -f "$image_id_file"' EXIT
+
   $OCI build \
-    --tag "yocto-docs-$sanitized_dockername:latest" \
+    --iidfile "$image_id_file" \
     --build-arg ARG_FROM="$repo/$image" \
     --build-arg INCLUDE_ESSENTIAL_PACKAGES="${INCLUDE_ESSENTIAL_PACKAGES}" \
     --build-arg ESSENTIAL="host_packages_scripts/$essential" \
@@ -170,6 +171,9 @@ main ()
     --file "$SCRIPT_DIR/containerfiles/$containerfile" \
     "$SCRIPT_DIR"
 
+  image_sha="$(< "$image_id_file")"
+  rm "$image_id_file"
+
   local -a args_run=(
     --rm
     --interactive
@@ -193,7 +197,7 @@ main ()
 
   $OCI run \
     "${args_run[@]}" \
-    "yocto-docs-$sanitized_dockername" \
+    "$image_sha" \
     "$@"
 }
 

-- 
2.51.0



  reply	other threads:[~2025-12-22 12:28 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-12-22 12:27 [PATCH 0/8] Parallel docs build improvements Antonin Godard
2025-12-22 12:27 ` Antonin Godard [this message]
2025-12-22 12:27 ` [PATCH 2/8] Makefile: allow to specify build directory Antonin Godard
2025-12-22 12:27 ` [PATCH 3/8] tools/build-docs-container: build in separate directory for each distro Antonin Godard
2025-12-22 12:27 ` [PATCH 4/8] tools/build-docs-container: create symlink to latest build output Antonin Godard
2026-01-12 11:24   ` Quentin Schulz
2025-12-22 12:27 ` [PATCH 5/8] Makefile: wrap set_versions.py with flock Antonin Godard
2026-01-12 11:40   ` [docs] " Quentin Schulz
2025-12-22 12:27 ` [PATCH 6/8] tools/build-docs-container: allow running non-interactively Antonin Godard
2026-01-12 11:43   ` Quentin Schulz
2025-12-22 12:27 ` [PATCH 7/8] tools/build-docs-container: allow passing extra args to $OCI run Antonin Godard
2026-01-12 13:20   ` Quentin Schulz
2025-12-22 12:27 ` [PATCH 8/8] tools/build-docs-container: make it possible to print the distro list Antonin Godard
2026-01-12 13:25   ` Quentin Schulz
2026-01-09 17:06 ` [PATCH 0/8] Parallel docs build improvements Quentin Schulz

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20251222-concurrent-safety-v1-1-e3d86e44cd38@bootlin.com \
    --to=antonin.godard@bootlin.com \
    --cc=docs@lists.yoctoproject.org \
    --cc=quentin.schulz@cherry.de \
    --cc=thomas.petazzoni@bootlin.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox