* [PATCH 0/4] tests/docker/dockerfiles: add missing apt update
@ 2026-07-07 21:46 Pierrick Bouvier
2026-07-07 21:46 ` [PATCH 1/4] tests/lcitool/refresh: use raw string literal for trailers Pierrick Bouvier
` (4 more replies)
0 siblings, 5 replies; 18+ messages in thread
From: Pierrick Bouvier @ 2026-07-07 21:46 UTC (permalink / raw)
To: qemu-devel
Cc: mark.cave-ayland, richard.henderson, Pierrick Bouvier,
Alex Bennée, berrange, mjt
Some of our dockerfiles include custom statements, and those are missing
explicit apt update when doing an apt install.
Since docker caches layers when building an image, not including an apt update
may result in error, since packages file will not be up to date.
This is also documented in official docker documentation:
https://docs.docker.com/build/building/best-practices/#apt-get
This issue was first reported on:
https://lore.kernel.org/qemu-devel/4b08f4e2-2274-409a-86d8-469c73e21579@linaro.org/
Pierrick Bouvier (4):
tests/lcitool/refresh: use raw string literal for trailers
tests/docker/dockerfiles/ubuntu2404.docker: add missing apt update
tests/docker/dockerfiles/debian.docker: add missing apt update
tests/docker/dockerfiles/debian-all-test-cross.docker: add missing apt
update
.../dockerfiles/debian-all-test-cross.docker | 4 +-
tests/docker/dockerfiles/debian.docker | 3 +-
tests/docker/dockerfiles/ubuntu2404.docker | 3 +-
tests/lcitool/refresh | 213 +++++++++---------
4 files changed, 116 insertions(+), 107 deletions(-)
--
2.47.3
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH 1/4] tests/lcitool/refresh: use raw string literal for trailers
2026-07-07 21:46 [PATCH 0/4] tests/docker/dockerfiles: add missing apt update Pierrick Bouvier
@ 2026-07-07 21:46 ` Pierrick Bouvier
2026-07-08 8:13 ` Daniel P. Berrangé
2026-07-07 21:46 ` [PATCH 2/4] tests/docker/dockerfiles/ubuntu2404.docker: add missing apt update Pierrick Bouvier
` (3 subsequent siblings)
4 siblings, 1 reply; 18+ messages in thread
From: Pierrick Bouvier @ 2026-07-07 21:46 UTC (permalink / raw)
To: qemu-devel
Cc: mark.cave-ayland, richard.henderson, Pierrick Bouvier,
Alex Bennée, berrange, mjt
Using a list of string is not convenient, as users need to escape it
manually and handle end of lines. By using a raw string literal,
what you see is what you get.
Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
tests/lcitool/refresh | 209 +++++++++++++++++++++---------------------
1 file changed, 105 insertions(+), 104 deletions(-)
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index e0da0d2de1d..1fdd37440c9 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -72,6 +72,7 @@ def generate_dockerfile(host, target, project="qemu", cross=None, trailer=None,
cmd.extend([target, project])
if trailer is not None:
+ trailer = trailer.strip() + "\n"
trailer += "\n".join(add_user_mapping)
else:
trailer = "\n".join(add_user_mapping)
@@ -100,113 +101,113 @@ def generate_yaml(os, target, arch, trailer=None):
generate(filename, cmd, trailer)
-alpine_extras = [
- "# https://gitlab.alpinelinux.org/alpine/aports/-/issues/17463\n",
- "RUN apk add clang19-libclang\n",
-]
+alpine_extras = r"""
+# https://gitlab.alpinelinux.org/alpine/aports/-/issues/17463
+RUN apk add clang19-libclang
+"""
# Netmap still needs to be manually built as it is yet to be packaged
# into a distro. We also add cscope and gtags which are used in the CI
# test
-debian13_extras = [
- "# netmap/cscope/global\n",
- "RUN DEBIAN_FRONTEND=noninteractive eatmydata \\\n",
- " apt install -y --no-install-recommends \\\n",
- " cscope\\\n",
- " global\\\n",
- " linux-headers-generic\n",
- "RUN git clone https://github.com/luigirizzo/netmap.git /usr/src/netmap\n",
- "RUN cd /usr/src/netmap && git checkout v11.3\n",
- "RUN cd /usr/src/netmap/LINUX && \\\n",
- " ./configure --no-drivers --no-apps \\\n",
- " --kernel-dir=$(ls -d /usr/src/linux-headers-*-$(dpkg --print-architecture)) \\\n",
- " && make install\n",
- "ENV QEMU_CONFIGURE_OPTS=--enable-netmap\n"
-]
+debian13_extras = r"""
+# netmap/cscope/global
+RUN DEBIAN_FRONTEND=noninteractive eatmydata \
+ apt install -y --no-install-recommends \
+ cscope\
+ global\
+ linux-headers-generic
+RUN git clone https://github.com/luigirizzo/netmap.git /usr/src/netmap
+RUN cd /usr/src/netmap && git checkout v11.3
+RUN cd /usr/src/netmap/LINUX && \
+ ./configure --no-drivers --no-apps \
+ --kernel-dir=$(ls -d /usr/src/linux-headers-*-$(dpkg --print-architecture)) \
+ && make install
+ENV QEMU_CONFIGURE_OPTS=--enable-netmap
+"""
# Based on the hub.docker.com/library/rust Dockerfiles
-fedora_rustup_nightly_extras = [
- "RUN dnf install -y wget\n",
- "ENV RUSTUP_HOME=/usr/local/rustup CARGO_HOME=/usr/local/cargo\n",
- "ENV RUSTC=/usr/local/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/rustc\n",
- "ENV RUSTDOC=/usr/local/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/rustdoc\n",
- "ENV CARGO=/usr/local/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/cargo\n",
- "RUN set -eux && \\\n",
- " rustArch='x86_64-unknown-linux-gnu' && \\\n",
- " rustupSha256='6aeece6993e902708983b209d04c0d1dbb14ebb405ddb87def578d41f920f56d' && \\\n",
- ' url="https://static.rust-lang.org/rustup/archive/1.27.1/${rustArch}/rustup-init" && \\\n',
- ' wget "$url" && \\\n',
- ' echo "${rustupSha256} *rustup-init" | sha256sum -c - && \\\n',
- " chmod +x rustup-init && \\\n",
- " ./rustup-init -y --no-modify-path --profile default --default-toolchain nightly --default-host ${rustArch} && \\\n",
- " chmod -R a+w $RUSTUP_HOME $CARGO_HOME && \\\n",
- " /usr/local/cargo/bin/rustup --version && \\\n",
- " /usr/local/cargo/bin/rustup run nightly cargo --version && \\\n",
- " /usr/local/cargo/bin/rustup run nightly rustc --version && \\\n",
- ' test "$CARGO" = "$(/usr/local/cargo/bin/rustup +nightly which cargo)" && \\\n',
- ' test "$RUSTDOC" = "$(/usr/local/cargo/bin/rustup +nightly which rustdoc)" && \\\n',
- ' test "$RUSTC" = "$(/usr/local/cargo/bin/rustup +nightly which rustc)"\n',
- 'ENV PATH=$CARGO_HOME/bin:$PATH\n',
- 'RUN /usr/local/cargo/bin/rustup run nightly cargo install --locked bindgen-cli\n',
- 'RUN $CARGO --list\n',
-]
+fedora_rustup_nightly_extras = r"""
+RUN dnf install -y wget
+ENV RUSTUP_HOME=/usr/local/rustup CARGO_HOME=/usr/local/cargo
+ENV RUSTC=/usr/local/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/rustc
+ENV RUSTDOC=/usr/local/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/rustdoc
+ENV CARGO=/usr/local/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/cargo
+RUN set -eux && \
+ rustArch='x86_64-unknown-linux-gnu' && \
+ rustupSha256='6aeece6993e902708983b209d04c0d1dbb14ebb405ddb87def578d41f920f56d' && \
+ url="https://static.rust-lang.org/rustup/archive/1.27.1/${rustArch}/rustup-init" && \
+ wget "$url" && \
+ echo "${rustupSha256} *rustup-init" | sha256sum -c - && \
+ chmod +x rustup-init && \
+ ./rustup-init -y --no-modify-path --profile default --default-toolchain nightly --default-host ${rustArch} && \
+ chmod -R a+w $RUSTUP_HOME $CARGO_HOME && \
+ /usr/local/cargo/bin/rustup --version && \
+ /usr/local/cargo/bin/rustup run nightly cargo --version && \
+ /usr/local/cargo/bin/rustup run nightly rustc --version && \
+ test "$CARGO" = "$(/usr/local/cargo/bin/rustup +nightly which cargo)" && \
+ test "$RUSTDOC" = "$(/usr/local/cargo/bin/rustup +nightly which rustdoc)" && \
+ test "$RUSTC" = "$(/usr/local/cargo/bin/rustup +nightly which rustc)"
+ENV PATH=$CARGO_HOME/bin:$PATH
+RUN /usr/local/cargo/bin/rustup run nightly cargo install --locked bindgen-cli
+RUN $CARGO --list
+"""
-ubuntu2404_rust_extras = [
- "ENV RUSTC=/usr/bin/rustc-1.83\n",
- "ENV RUSTDOC=/usr/bin/rustdoc-1.83\n",
- "ENV CARGO_HOME=/usr/local/cargo\n",
- 'ENV PATH=$CARGO_HOME/bin:$PATH\n',
- "RUN DEBIAN_FRONTEND=noninteractive eatmydata \\\n",
- " apt install -y --no-install-recommends cargo\n",
- 'RUN cargo install --locked bindgen-cli\n',
-]
+ubuntu2404_rust_extras = r"""
+ENV RUSTC=/usr/bin/rustc-1.83
+ENV RUSTDOC=/usr/bin/rustdoc-1.83
+ENV CARGO_HOME=/usr/local/cargo
+ENV PATH=$CARGO_HOME/bin:$PATH
+RUN DEBIAN_FRONTEND=noninteractive eatmydata \
+ apt install -y --no-install-recommends cargo
+RUN cargo install --locked bindgen-cli
+"""
-debian_all_test_cross_compilers = [
- "# extras for cross and alternate toolchains\n",
- "RUN DEBIAN_FRONTEND=noninteractive eatmydata \\\n",
- " apt install -y --no-install-recommends \\\n",
- " clang\\\n",
- " dpkg-dev\\\n",
- " gdb-multiarch\\\n",
- " libclang-rt-dev\n",
- "ENV AVAILABLE_COMPILERS=\"\\\n",
- " gcc-aarch64-linux-gnu \\\n",
- " libc6-dev-arm64-cross \\\n",
- " gcc-arm-linux-gnueabihf \\\n",
- " libc6-dev-armhf-cross \\\n",
- " gcc-alpha-linux-gnu \\\n"
- " libc6.1-dev-alpha-cross \\\n"
- " gcc-mips-linux-gnu \\\n",
- " libc6-dev-mips-cross \\\n",
- " gcc-mips64-linux-gnuabi64 \\\n",
- " libc6-dev-mips64-cross \\\n",
- " gcc-mips64el-linux-gnuabi64 \\\n",
- " libc6-dev-mips64el-cross \\\n",
- " gcc-mipsel-linux-gnu \\\n",
- " libc6-dev-mipsel-cross \\\n",
- " gcc-powerpc64le-linux-gnu \\\n",
- " libc6-dev-ppc64el-cross \\\n",
- " gcc-riscv64-linux-gnu \\\n",
- " libc6-dev-riscv64-cross \\\n",
- " gcc-s390x-linux-gnu \\\n",
- " libc6-dev-s390x-cross\\\n",
- " gcc-sh4-linux-gnu \\\n",
- " libc6-dev-sh4-cross \\\n",
- " \"\n",
- "RUN if dpkg-architecture -e amd64; then \\\n",
- " export AVAILABLE_COMPILERS=\"${AVAILABLE_COMPILERS} gcc-hppa-linux-gnu libc6-dev-hppa-cross\"; \\\n",
- " export AVAILABLE_COMPILERS=\"${AVAILABLE_COMPILERS} gcc-m68k-linux-gnu libc6-dev-m68k-cross\"; \\\n",
- " export AVAILABLE_COMPILERS=\"${AVAILABLE_COMPILERS} gcc-powerpc-linux-gnu libc6-dev-powerpc-cross\"; \\\n",
- " export AVAILABLE_COMPILERS=\"${AVAILABLE_COMPILERS} gcc-powerpc64-linux-gnu libc6-dev-ppc64-cross\"; \\\n",
- " export AVAILABLE_COMPILERS=\"${AVAILABLE_COMPILERS} gcc-sparc64-linux-gnu libc6-dev-sparc64-cross\"; \\\n",
- "fi && \\\n",
- "DEBIAN_FRONTEND=noninteractive eatmydata \\\n",
- "apt install -y --no-install-recommends \\\n",
- "${AVAILABLE_COMPILERS} && \\\n",
- "dpkg-query --showformat '${Package}_${Version}_${Architecture}' --show > /packages.txt\n",
- "ENV QEMU_CONFIGURE_OPTS=--disable-docs\n",
- "ENV DEF_TARGET_LIST=aarch64-linux-user,alpha-linux-user,arm-linux-user,hppa-linux-user,i386-linux-user,m68k-linux-user,mips-linux-user,mips64-linux-user,mips64el-linux-user,mipsel-linux-user,ppc-linux-user,ppc64-linux-user,ppc64le-linux-user,riscv64-linux-user,s390x-linux-user,sh4-linux-user,sparc64-linux-user\n",
-]
+debian_all_test_cross_compilers = r"""
+# extras for cross and alternate toolchains
+RUN DEBIAN_FRONTEND=noninteractive eatmydata \
+ apt install -y --no-install-recommends \
+ clang\
+ dpkg-dev\
+ gdb-multiarch\
+ libclang-rt-dev
+ENV AVAILABLE_COMPILERS="\
+ gcc-aarch64-linux-gnu \
+ libc6-dev-arm64-cross \
+ gcc-arm-linux-gnueabihf \
+ libc6-dev-armhf-cross \
+ gcc-alpha-linux-gnu \
+ libc6.1-dev-alpha-cross \
+ gcc-mips-linux-gnu \
+ libc6-dev-mips-cross \
+ gcc-mips64-linux-gnuabi64 \
+ libc6-dev-mips64-cross \
+ gcc-mips64el-linux-gnuabi64 \
+ libc6-dev-mips64el-cross \
+ gcc-mipsel-linux-gnu \
+ libc6-dev-mipsel-cross \
+ gcc-powerpc64le-linux-gnu \
+ libc6-dev-ppc64el-cross \
+ gcc-riscv64-linux-gnu \
+ libc6-dev-riscv64-cross \
+ gcc-s390x-linux-gnu \
+ libc6-dev-s390x-cross\
+ gcc-sh4-linux-gnu \
+ libc6-dev-sh4-cross \
+ "
+RUN if dpkg-architecture -e amd64; then \
+ export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-hppa-linux-gnu libc6-dev-hppa-cross"; \
+ export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-m68k-linux-gnu libc6-dev-m68k-cross"; \
+ export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-powerpc-linux-gnu libc6-dev-powerpc-cross"; \
+ export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-powerpc64-linux-gnu libc6-dev-ppc64-cross"; \
+ export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-sparc64-linux-gnu libc6-dev-sparc64-cross"; \
+fi && \
+DEBIAN_FRONTEND=noninteractive eatmydata \
+apt install -y --no-install-recommends \
+${AVAILABLE_COMPILERS} && \
+dpkg-query --showformat '${Package}_${Version}_${Architecture}' --show > /packages.txt
+ENV QEMU_CONFIGURE_OPTS=--disable-docs
+ENV DEF_TARGET_LIST=aarch64-linux-user,alpha-linux-user,arm-linux-user,hppa-linux-user,i386-linux-user,m68k-linux-user,mips-linux-user,mips64-linux-user,mips64el-linux-user,mipsel-linux-user,ppc-linux-user,ppc64-linux-user,ppc64le-linux-user,riscv64-linux-user,s390x-linux-user,sh4-linux-user,sparc64-linux-user
+"""
def cross_build(prefix, targets):
conf = "ENV QEMU_CONFIGURE_OPTS=--cross-prefix=%s\n" % (prefix)
@@ -223,20 +224,20 @@ try:
# Standard native builds
#
generate_dockerfile("alpine", "alpine-323",
- trailer="".join(alpine_extras))
+ trailer=alpine_extras)
generate_dockerfile("centos9", "centos-stream-9")
generate_dockerfile("debian", "debian-13",
- trailer="".join(debian13_extras))
+ trailer=debian13_extras)
generate_dockerfile("fedora", "fedora-43")
generate_dockerfile("opensuse-leap", "opensuse-leap-16")
generate_dockerfile("ubuntu2404", "ubuntu-2404",
- trailer="".join(ubuntu2404_rust_extras))
+ trailer=ubuntu2404_rust_extras)
#
# Non-fatal Rust-enabled build
#
generate_dockerfile("fedora-rust-nightly", "fedora-43",
- trailer="".join(fedora_rustup_nightly_extras))
+ trailer=fedora_rustup_nightly_extras)
#
# Cross compiling builds
@@ -293,7 +294,7 @@ try:
generate_dockerfile("debian-all-test-cross", "debian-13",
project="qemu-minimal",
enable_rust=False,
- trailer="".join(debian_all_test_cross_compilers))
+ trailer=debian_all_test_cross_compilers)
#
# GitLab packages lists
--
2.47.3
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH 2/4] tests/docker/dockerfiles/ubuntu2404.docker: add missing apt update
2026-07-07 21:46 [PATCH 0/4] tests/docker/dockerfiles: add missing apt update Pierrick Bouvier
2026-07-07 21:46 ` [PATCH 1/4] tests/lcitool/refresh: use raw string literal for trailers Pierrick Bouvier
@ 2026-07-07 21:46 ` Pierrick Bouvier
2026-07-08 8:19 ` Daniel P. Berrangé
` (2 more replies)
2026-07-07 21:46 ` [PATCH 3/4] tests/docker/dockerfiles/debian.docker: " Pierrick Bouvier
` (2 subsequent siblings)
4 siblings, 3 replies; 18+ messages in thread
From: Pierrick Bouvier @ 2026-07-07 21:46 UTC (permalink / raw)
To: qemu-devel
Cc: mark.cave-ayland, richard.henderson, Pierrick Bouvier,
Alex Bennée, berrange, mjt
RUN statements should always use apt update && apt install together:
https://docs.docker.com/build/building/best-practices/#apt-get
Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
tests/docker/dockerfiles/ubuntu2404.docker | 3 ++-
tests/lcitool/refresh | 3 ++-
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/tests/docker/dockerfiles/ubuntu2404.docker b/tests/docker/dockerfiles/ubuntu2404.docker
index 26bc2dd94f4..27992e34fd9 100644
--- a/tests/docker/dockerfiles/ubuntu2404.docker
+++ b/tests/docker/dockerfiles/ubuntu2404.docker
@@ -160,7 +160,8 @@ ENV RUSTC=/usr/bin/rustc-1.83
ENV RUSTDOC=/usr/bin/rustdoc-1.83
ENV CARGO_HOME=/usr/local/cargo
ENV PATH=$CARGO_HOME/bin:$PATH
-RUN DEBIAN_FRONTEND=noninteractive eatmydata \
+RUN apt update && \
+ DEBIAN_FRONTEND=noninteractive eatmydata \
apt install -y --no-install-recommends cargo
RUN cargo install --locked bindgen-cli
# As a final step configure the user (if env is defined)
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index 1fdd37440c9..bbbc6d3c46a 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -157,7 +157,8 @@ ENV RUSTC=/usr/bin/rustc-1.83
ENV RUSTDOC=/usr/bin/rustdoc-1.83
ENV CARGO_HOME=/usr/local/cargo
ENV PATH=$CARGO_HOME/bin:$PATH
-RUN DEBIAN_FRONTEND=noninteractive eatmydata \
+RUN apt update && \
+ DEBIAN_FRONTEND=noninteractive eatmydata \
apt install -y --no-install-recommends cargo
RUN cargo install --locked bindgen-cli
"""
--
2.47.3
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH 3/4] tests/docker/dockerfiles/debian.docker: add missing apt update
2026-07-07 21:46 [PATCH 0/4] tests/docker/dockerfiles: add missing apt update Pierrick Bouvier
2026-07-07 21:46 ` [PATCH 1/4] tests/lcitool/refresh: use raw string literal for trailers Pierrick Bouvier
2026-07-07 21:46 ` [PATCH 2/4] tests/docker/dockerfiles/ubuntu2404.docker: add missing apt update Pierrick Bouvier
@ 2026-07-07 21:46 ` Pierrick Bouvier
2026-07-08 8:25 ` Daniel P. Berrangé
2026-07-07 21:46 ` [PATCH 4/4] tests/docker/dockerfiles/debian-all-test-cross.docker: " Pierrick Bouvier
2026-07-10 5:13 ` [PATCH 0/4] tests/docker/dockerfiles: " Pierrick Bouvier
4 siblings, 1 reply; 18+ messages in thread
From: Pierrick Bouvier @ 2026-07-07 21:46 UTC (permalink / raw)
To: qemu-devel
Cc: mark.cave-ayland, richard.henderson, Pierrick Bouvier,
Alex Bennée, berrange, mjt
RUN statements should always use apt update && apt install together:
https://docs.docker.com/build/building/best-practices/#apt-get
Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
tests/docker/dockerfiles/debian.docker | 3 ++-
tests/lcitool/refresh | 3 ++-
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/tests/docker/dockerfiles/debian.docker b/tests/docker/dockerfiles/debian.docker
index ee8b313d654..413986d3f07 100644
--- a/tests/docker/dockerfiles/debian.docker
+++ b/tests/docker/dockerfiles/debian.docker
@@ -157,7 +157,8 @@ ENV MAKE="/usr/bin/make"
ENV NINJA="/usr/bin/ninja"
ENV PYTHON="/usr/bin/python3"
# netmap/cscope/global
-RUN DEBIAN_FRONTEND=noninteractive eatmydata \
+RUN apt update && \
+ DEBIAN_FRONTEND=noninteractive eatmydata \
apt install -y --no-install-recommends \
cscope\
global\
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index bbbc6d3c46a..8def84d946e 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -111,7 +111,8 @@ RUN apk add clang19-libclang
# test
debian13_extras = r"""
# netmap/cscope/global
-RUN DEBIAN_FRONTEND=noninteractive eatmydata \
+RUN apt update && \
+ DEBIAN_FRONTEND=noninteractive eatmydata \
apt install -y --no-install-recommends \
cscope\
global\
--
2.47.3
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH 4/4] tests/docker/dockerfiles/debian-all-test-cross.docker: add missing apt update
2026-07-07 21:46 [PATCH 0/4] tests/docker/dockerfiles: add missing apt update Pierrick Bouvier
` (2 preceding siblings ...)
2026-07-07 21:46 ` [PATCH 3/4] tests/docker/dockerfiles/debian.docker: " Pierrick Bouvier
@ 2026-07-07 21:46 ` Pierrick Bouvier
2026-07-08 8:50 ` Daniel P. Berrangé
2026-07-10 5:13 ` [PATCH 0/4] tests/docker/dockerfiles: " Pierrick Bouvier
4 siblings, 1 reply; 18+ messages in thread
From: Pierrick Bouvier @ 2026-07-07 21:46 UTC (permalink / raw)
To: qemu-devel
Cc: mark.cave-ayland, richard.henderson, Pierrick Bouvier,
Alex Bennée, berrange, mjt
RUN statements should always use apt update && apt install together:
https://docs.docker.com/build/building/best-practices/#apt-get
Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
---
tests/docker/dockerfiles/debian-all-test-cross.docker | 4 +++-
tests/lcitool/refresh | 4 +++-
2 files changed, 6 insertions(+), 2 deletions(-)
diff --git a/tests/docker/dockerfiles/debian-all-test-cross.docker b/tests/docker/dockerfiles/debian-all-test-cross.docker
index a10ffc73500..2dc580552e3 100644
--- a/tests/docker/dockerfiles/debian-all-test-cross.docker
+++ b/tests/docker/dockerfiles/debian-all-test-cross.docker
@@ -55,7 +55,8 @@ ENV MAKE="/usr/bin/make"
ENV NINJA="/usr/bin/ninja"
ENV PYTHON="/usr/bin/python3"
# extras for cross and alternate toolchains
-RUN DEBIAN_FRONTEND=noninteractive eatmydata \
+RUN apt update && \
+ DEBIAN_FRONTEND=noninteractive eatmydata \
apt install -y --no-install-recommends \
clang\
dpkg-dev\
@@ -92,6 +93,7 @@ RUN if dpkg-architecture -e amd64; then \
export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-powerpc64-linux-gnu libc6-dev-ppc64-cross"; \
export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-sparc64-linux-gnu libc6-dev-sparc64-cross"; \
fi && \
+apt update && \
DEBIAN_FRONTEND=noninteractive eatmydata \
apt install -y --no-install-recommends \
${AVAILABLE_COMPILERS} && \
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index 8def84d946e..6bc611b0e6c 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -166,7 +166,8 @@ RUN cargo install --locked bindgen-cli
debian_all_test_cross_compilers = r"""
# extras for cross and alternate toolchains
-RUN DEBIAN_FRONTEND=noninteractive eatmydata \
+RUN apt update && \
+ DEBIAN_FRONTEND=noninteractive eatmydata \
apt install -y --no-install-recommends \
clang\
dpkg-dev\
@@ -203,6 +204,7 @@ RUN if dpkg-architecture -e amd64; then \
export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-powerpc64-linux-gnu libc6-dev-ppc64-cross"; \
export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-sparc64-linux-gnu libc6-dev-sparc64-cross"; \
fi && \
+apt update && \
DEBIAN_FRONTEND=noninteractive eatmydata \
apt install -y --no-install-recommends \
${AVAILABLE_COMPILERS} && \
--
2.47.3
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH 1/4] tests/lcitool/refresh: use raw string literal for trailers
2026-07-07 21:46 ` [PATCH 1/4] tests/lcitool/refresh: use raw string literal for trailers Pierrick Bouvier
@ 2026-07-08 8:13 ` Daniel P. Berrangé
2026-07-08 15:56 ` Pierrick Bouvier
0 siblings, 1 reply; 18+ messages in thread
From: Daniel P. Berrangé @ 2026-07-08 8:13 UTC (permalink / raw)
To: Pierrick Bouvier
Cc: qemu-devel, mark.cave-ayland, richard.henderson, Alex Bennée,
mjt
On Tue, Jul 07, 2026 at 02:46:52PM -0700, Pierrick Bouvier wrote:
> Using a list of string is not convenient, as users need to escape it
> manually and handle end of lines. By using a raw string literal,
> what you see is what you get.
>
> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
> ---
> tests/lcitool/refresh | 209 +++++++++++++++++++++---------------------
> 1 file changed, 105 insertions(+), 104 deletions(-)
Does what it says on the tin so....
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
...but given the size of the trailer we have inline, I wonder if we
would be better off just having that trailer in a plain text file
alongside the refresh script, such that 'refresh' just auto-appends
any <foo>.trailer file contents to any docker file it emits called
"<foo>".
With regards,
Daniel
--
|: https://berrange.com ~~ https://hachyderm.io/@berrange :|
|: https://libvirt.org ~~ https://entangle-photo.org :|
|: https://pixelfed.art/berrange ~~ https://fstop138.berrange.com :|
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 2/4] tests/docker/dockerfiles/ubuntu2404.docker: add missing apt update
2026-07-07 21:46 ` [PATCH 2/4] tests/docker/dockerfiles/ubuntu2404.docker: add missing apt update Pierrick Bouvier
@ 2026-07-08 8:19 ` Daniel P. Berrangé
2026-07-08 9:27 ` Michael Tokarev
2026-07-08 16:14 ` Pierrick Bouvier
2026-07-08 16:38 ` Pierrick Bouvier
2026-07-08 20:46 ` Philippe Mathieu-Daudé
2 siblings, 2 replies; 18+ messages in thread
From: Daniel P. Berrangé @ 2026-07-08 8:19 UTC (permalink / raw)
To: Pierrick Bouvier
Cc: qemu-devel, mark.cave-ayland, richard.henderson, Alex Bennée,
mjt
On Tue, Jul 07, 2026 at 02:46:53PM -0700, Pierrick Bouvier wrote:
> RUN statements should always use apt update && apt install together:
> https://docs.docker.com/build/building/best-practices/#apt-get
>
> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
> ---
> tests/docker/dockerfiles/ubuntu2404.docker | 3 ++-
> tests/lcitool/refresh | 3 ++-
> 2 files changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/tests/docker/dockerfiles/ubuntu2404.docker b/tests/docker/dockerfiles/ubuntu2404.docker
> index 26bc2dd94f4..27992e34fd9 100644
> --- a/tests/docker/dockerfiles/ubuntu2404.docker
> +++ b/tests/docker/dockerfiles/ubuntu2404.docker
> @@ -160,7 +160,8 @@ ENV RUSTC=/usr/bin/rustc-1.83
> ENV RUSTDOC=/usr/bin/rustdoc-1.83
> ENV CARGO_HOME=/usr/local/cargo
> ENV PATH=$CARGO_HOME/bin:$PATH
> -RUN DEBIAN_FRONTEND=noninteractive eatmydata \
> +RUN apt update && \
> + DEBIAN_FRONTEND=noninteractive eatmydata \
> apt install -y --no-install-recommends cargo
> RUN cargo install --locked bindgen-cli
> # As a final step configure the user (if env is defined)
> diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
> index 1fdd37440c9..bbbc6d3c46a 100755
> --- a/tests/lcitool/refresh
> +++ b/tests/lcitool/refresh
> @@ -157,7 +157,8 @@ ENV RUSTC=/usr/bin/rustc-1.83
> ENV RUSTDOC=/usr/bin/rustdoc-1.83
> ENV CARGO_HOME=/usr/local/cargo
> ENV PATH=$CARGO_HOME/bin:$PATH
> -RUN DEBIAN_FRONTEND=noninteractive eatmydata \
> +RUN apt update && \
> + DEBIAN_FRONTEND=noninteractive eatmydata \
> apt install -y --no-install-recommends cargo
> RUN cargo install --locked bindgen-cli
So lcitool already ran 'apt update' earlier, but if we're trying to
avoid caching issues, I guess running it again is needed. Should we
also run 'apt dist-upgrade' though, as already installed content in
the cached layer can be outdated.
With regards,
Daniel
--
|: https://berrange.com ~~ https://hachyderm.io/@berrange :|
|: https://libvirt.org ~~ https://entangle-photo.org :|
|: https://pixelfed.art/berrange ~~ https://fstop138.berrange.com :|
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 3/4] tests/docker/dockerfiles/debian.docker: add missing apt update
2026-07-07 21:46 ` [PATCH 3/4] tests/docker/dockerfiles/debian.docker: " Pierrick Bouvier
@ 2026-07-08 8:25 ` Daniel P. Berrangé
0 siblings, 0 replies; 18+ messages in thread
From: Daniel P. Berrangé @ 2026-07-08 8:25 UTC (permalink / raw)
To: Pierrick Bouvier
Cc: qemu-devel, mark.cave-ayland, richard.henderson, Alex Bennée,
mjt
On Tue, Jul 07, 2026 at 02:46:54PM -0700, Pierrick Bouvier wrote:
> RUN statements should always use apt update && apt install together:
> https://docs.docker.com/build/building/best-practices/#apt-get
>
> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
> ---
> tests/docker/dockerfiles/debian.docker | 3 ++-
> tests/lcitool/refresh | 3 ++-
> 2 files changed, 4 insertions(+), 2 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com ~~ https://hachyderm.io/@berrange :|
|: https://libvirt.org ~~ https://entangle-photo.org :|
|: https://pixelfed.art/berrange ~~ https://fstop138.berrange.com :|
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 4/4] tests/docker/dockerfiles/debian-all-test-cross.docker: add missing apt update
2026-07-07 21:46 ` [PATCH 4/4] tests/docker/dockerfiles/debian-all-test-cross.docker: " Pierrick Bouvier
@ 2026-07-08 8:50 ` Daniel P. Berrangé
0 siblings, 0 replies; 18+ messages in thread
From: Daniel P. Berrangé @ 2026-07-08 8:50 UTC (permalink / raw)
To: Pierrick Bouvier
Cc: qemu-devel, mark.cave-ayland, richard.henderson, Alex Bennée,
mjt
On Tue, Jul 07, 2026 at 02:46:55PM -0700, Pierrick Bouvier wrote:
> RUN statements should always use apt update && apt install together:
> https://docs.docker.com/build/building/best-practices/#apt-get
>
> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
> ---
> tests/docker/dockerfiles/debian-all-test-cross.docker | 4 +++-
> tests/lcitool/refresh | 4 +++-
> 2 files changed, 6 insertions(+), 2 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com ~~ https://hachyderm.io/@berrange :|
|: https://libvirt.org ~~ https://entangle-photo.org :|
|: https://pixelfed.art/berrange ~~ https://fstop138.berrange.com :|
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 2/4] tests/docker/dockerfiles/ubuntu2404.docker: add missing apt update
2026-07-08 8:19 ` Daniel P. Berrangé
@ 2026-07-08 9:27 ` Michael Tokarev
2026-07-08 14:28 ` Philippe Mathieu-Daudé
2026-07-08 16:14 ` Pierrick Bouvier
1 sibling, 1 reply; 18+ messages in thread
From: Michael Tokarev @ 2026-07-08 9:27 UTC (permalink / raw)
To: Daniel P. Berrangé, Pierrick Bouvier
Cc: qemu-devel, mark.cave-ayland, richard.henderson, Alex Bennée
On 08.07.2026 11:19, Daniel P. Berrangé wrote:
> On Tue, Jul 07, 2026 at 02:46:53PM -0700, Pierrick Bouvier wrote:
>> RUN statements should always use apt update && apt install together:
>> https://docs.docker.com/build/building/best-practices/#apt-get
>>
>> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>> ---
>> tests/docker/dockerfiles/ubuntu2404.docker | 3 ++-
>> tests/lcitool/refresh | 3 ++-
>> 2 files changed, 4 insertions(+), 2 deletions(-)
>>
>> diff --git a/tests/docker/dockerfiles/ubuntu2404.docker b/tests/docker/dockerfiles/ubuntu2404.docker
>> index 26bc2dd94f4..27992e34fd9 100644
>> --- a/tests/docker/dockerfiles/ubuntu2404.docker
>> +++ b/tests/docker/dockerfiles/ubuntu2404.docker
>> @@ -160,7 +160,8 @@ ENV RUSTC=/usr/bin/rustc-1.83
>> ENV RUSTDOC=/usr/bin/rustdoc-1.83
>> ENV CARGO_HOME=/usr/local/cargo
>> ENV PATH=$CARGO_HOME/bin:$PATH
>> -RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>> +RUN apt update && \
>> + DEBIAN_FRONTEND=noninteractive eatmydata \
>> apt install -y --no-install-recommends cargo
>> RUN cargo install --locked bindgen-cli
>> # As a final step configure the user (if env is defined)
>> diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
>> index 1fdd37440c9..bbbc6d3c46a 100755
>> --- a/tests/lcitool/refresh
>> +++ b/tests/lcitool/refresh
>> @@ -157,7 +157,8 @@ ENV RUSTC=/usr/bin/rustc-1.83
>> ENV RUSTDOC=/usr/bin/rustdoc-1.83
>> ENV CARGO_HOME=/usr/local/cargo
>> ENV PATH=$CARGO_HOME/bin:$PATH
>> -RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>> +RUN apt update && \
>> + DEBIAN_FRONTEND=noninteractive eatmydata \
>> apt install -y --no-install-recommends cargo
>> RUN cargo install --locked bindgen-cli
>
> So lcitool already ran 'apt update' earlier, but if we're trying to
> avoid caching issues, I guess running it again is needed. Should we
No, running it again should not be needed. Unless we change
/etc/apt/sources.list.
> also run 'apt dist-upgrade' though, as already installed content in
> the cached layer can be outdated.
Um. Or maybe when we add more layers which contains outdated cache?
Apt has just one cache - /var/lib/apt/lists/ - of the packages which
are available online. This cache needs to be kept up to date for
installation from network to work. It is not, in any way, related to
the installed software - apt knows which packages are installed without
the cache, it always looks at the current dpkg database for that.
Thanks,
/mjt
> With regards,
> Daniel
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 2/4] tests/docker/dockerfiles/ubuntu2404.docker: add missing apt update
2026-07-08 9:27 ` Michael Tokarev
@ 2026-07-08 14:28 ` Philippe Mathieu-Daudé
2026-07-08 14:54 ` Michael Tokarev
0 siblings, 1 reply; 18+ messages in thread
From: Philippe Mathieu-Daudé @ 2026-07-08 14:28 UTC (permalink / raw)
To: Michael Tokarev, Daniel P. Berrangé, Pierrick Bouvier
Cc: qemu-devel, mark.cave-ayland, richard.henderson, Alex Bennée
On 8/7/26 11:27, Michael Tokarev wrote:
> On 08.07.2026 11:19, Daniel P. Berrangé wrote:
>> On Tue, Jul 07, 2026 at 02:46:53PM -0700, Pierrick Bouvier wrote:
>>> RUN statements should always use apt update && apt install together:
>>> https://docs.docker.com/build/building/best-practices/#apt-get
>>>
>>> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>>> ---
>>> tests/docker/dockerfiles/ubuntu2404.docker | 3 ++-
>>> tests/lcitool/refresh | 3 ++-
>>> 2 files changed, 4 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/tests/docker/dockerfiles/ubuntu2404.docker b/tests/
>>> docker/dockerfiles/ubuntu2404.docker
>>> index 26bc2dd94f4..27992e34fd9 100644
>>> --- a/tests/docker/dockerfiles/ubuntu2404.docker
>>> +++ b/tests/docker/dockerfiles/ubuntu2404.docker
>>> @@ -160,7 +160,8 @@ ENV RUSTC=/usr/bin/rustc-1.83
>>> ENV RUSTDOC=/usr/bin/rustdoc-1.83
>>> ENV CARGO_HOME=/usr/local/cargo
>>> ENV PATH=$CARGO_HOME/bin:$PATH
>>> -RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>>> +RUN apt update && \
>>> + DEBIAN_FRONTEND=noninteractive eatmydata \
>>> apt install -y --no-install-recommends cargo
>>> RUN cargo install --locked bindgen-cli
>>> # As a final step configure the user (if env is defined)
>>> diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
>>> index 1fdd37440c9..bbbc6d3c46a 100755
>>> --- a/tests/lcitool/refresh
>>> +++ b/tests/lcitool/refresh
>>> @@ -157,7 +157,8 @@ ENV RUSTC=/usr/bin/rustc-1.83
>>> ENV RUSTDOC=/usr/bin/rustdoc-1.83
>>> ENV CARGO_HOME=/usr/local/cargo
>>> ENV PATH=$CARGO_HOME/bin:$PATH
>>> -RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>>> +RUN apt update && \
>>> + DEBIAN_FRONTEND=noninteractive eatmydata \
>>> apt install -y --no-install-recommends cargo
>>> RUN cargo install --locked bindgen-cli
>>
>> So lcitool already ran 'apt update' earlier, but if we're trying to
>> avoid caching issues, I guess running it again is needed. Should we
>
> No, running it again should not be needed. Unless we change
> /etc/apt/sources.list.
The problem we want to address is wrt remote content going out of
sync; we need to run 'apt update' previous to any 'apt' call which
use the remote.
>
>> also run 'apt dist-upgrade' though, as already installed content in
>> the cached layer can be outdated.
>
> Um. Or maybe when we add more layers which contains outdated cache?
>
> Apt has just one cache - /var/lib/apt/lists/ - of the packages which
> are available online. This cache needs to be kept up to date for
> installation from network to work. It is not, in any way, related to
> the installed software - apt knows which packages are installed without
> the cache, it always looks at the current dpkg database for that.
>
> Thanks,
>
> /mjt
>
>
>> With regards,
>> Daniel
>
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 2/4] tests/docker/dockerfiles/ubuntu2404.docker: add missing apt update
2026-07-08 14:28 ` Philippe Mathieu-Daudé
@ 2026-07-08 14:54 ` Michael Tokarev
2026-07-08 16:11 ` Pierrick Bouvier
0 siblings, 1 reply; 18+ messages in thread
From: Michael Tokarev @ 2026-07-08 14:54 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, Daniel P. Berrangé,
Pierrick Bouvier
Cc: qemu-devel, mark.cave-ayland, richard.henderson, Alex Bennée
On 08.07.2026 17:28, Philippe Mathieu-Daudé wrote:
> On 8/7/26 11:27, Michael Tokarev wrote:
>> No, running it again should not be needed. Unless we change
>> /etc/apt/sources.list.
>
> The problem we want to address is wrt remote content going out of
> sync; we need to run 'apt update' previous to any 'apt' call which
> use the remote.
It's definitely not required to be this way. Running `apt update' once,
when creating containers, is enough. Any subsequent apt install &Co
will be run "soon enough" to avoid worrying about cache going out of
sync. Even if meanwhile a new version of some package has been uploaded,
old version is available for some time (usually at least a day).
So no, running `apt update' before every `apt install' and similar
is not necessary.
And sure, you can do it - it's just some wasted time.
Thanks,
/mjt
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 1/4] tests/lcitool/refresh: use raw string literal for trailers
2026-07-08 8:13 ` Daniel P. Berrangé
@ 2026-07-08 15:56 ` Pierrick Bouvier
0 siblings, 0 replies; 18+ messages in thread
From: Pierrick Bouvier @ 2026-07-08 15:56 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: qemu-devel, mark.cave-ayland, richard.henderson, Alex Bennée,
mjt
On 7/8/2026 1:13 AM, Daniel P. Berrangé wrote:
> On Tue, Jul 07, 2026 at 02:46:52PM -0700, Pierrick Bouvier wrote:
>> Using a list of string is not convenient, as users need to escape it
>> manually and handle end of lines. By using a raw string literal,
>> what you see is what you get.
>>
>> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>> ---
>> tests/lcitool/refresh | 209 +++++++++++++++++++++---------------------
>> 1 file changed, 105 insertions(+), 104 deletions(-)
>
> Does what it says on the tin so....
>
> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
>
> ...but given the size of the trailer we have inline, I wonder if we
> would be better off just having that trailer in a plain text file
> alongside the refresh script, such that 'refresh' just auto-appends
> any <foo>.trailer file contents to any docker file it emits called
> "<foo>".
>
IMHO a simple raw string literal in an existing script is enough, it's
perfectly readable and maintainable this way. Adding yet another file
means that you need to look in another place, which is already complex
enough between the dockerfiles, lcitool itself, and refresh script.
> With regards,
> Daniel
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 2/4] tests/docker/dockerfiles/ubuntu2404.docker: add missing apt update
2026-07-08 14:54 ` Michael Tokarev
@ 2026-07-08 16:11 ` Pierrick Bouvier
0 siblings, 0 replies; 18+ messages in thread
From: Pierrick Bouvier @ 2026-07-08 16:11 UTC (permalink / raw)
To: Michael Tokarev, Philippe Mathieu-Daudé,
Daniel P. Berrangé
Cc: qemu-devel, mark.cave-ayland, richard.henderson, Alex Bennée
On 7/8/2026 7:54 AM, Michael Tokarev wrote:
> On 08.07.2026 17:28, Philippe Mathieu-Daudé wrote:
>> On 8/7/26 11:27, Michael Tokarev wrote:
>
>>> No, running it again should not be needed. Unless we change
>>> /etc/apt/sources.list.
>>
>> The problem we want to address is wrt remote content going out of
>> sync; we need to run 'apt update' previous to any 'apt' call which
>> use the remote.
>
> It's definitely not required to be this way. Running `apt update' once,
> when creating containers, is enough. Any subsequent apt install &Co
> will be run "soon enough" to avoid worrying about cache going out of
> sync. Even if meanwhile a new version of some package has been uploaded,
> old version is available for some time (usually at least a day).
>
> So no, running `apt update' before every `apt install' and similar
> is not necessary.
>
> And sure, you can do it - it's just some wasted time.
>
Sorry but that's an incorrect view of how docker layer is working.
There is no guarantee of how "soon enough" a previous RUN statement will
be ran, before a next one. As long as command line associated to RUN is
not modified, or that base image is not pulled again, all previous RUN
statements are cached by default.
That's why Richard observed that his container was broken for weeks, and
it could have been broken until the end of times.
The recommendation given to him was to make a build without cache, which
is not the right way to fix it.
If you want a good analogy, let's imagine you have a build system with a
missing dependency: you can either fix it, or suggest to run "clean" and
start over. The latter was suggested.
In the case of apt install, concerned package might not be available
anymore between previous and current RUN statement. That's why you need
apt update with every apt install.
Those links, including official docker documentation, explain the same:
- https://docs.docker.com/build/building/best-practices/#apt-get
-
https://stackoverflow.com/questions/34248439/in-docker-why-is-it-recommended-to-run-apt-get-update-in-the-dockerfile
-
https://medium.com/@janscheffler/lessons-learned-apt-install-in-docker-images-bfad0e026f6d
Another common example is a previous RUN statement that downloads
something from a (now) broken link. New builds will fail, while cached
one will work. That's why building containers from scratch daily CI is a
good practice. It's similar to what we observe with QEMU functional
tests assets that are cached in our CI, but are missing regularly, until
someone unfortunate discovers that.
Running apt upgrade/dist-upgrade manually is not recommended, and the
best way to have updated containers (especially for production purpose)
is to rebuild them from scratch.
> Thanks,
>
> /mjt
Regards,
Pierrick
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 2/4] tests/docker/dockerfiles/ubuntu2404.docker: add missing apt update
2026-07-08 8:19 ` Daniel P. Berrangé
2026-07-08 9:27 ` Michael Tokarev
@ 2026-07-08 16:14 ` Pierrick Bouvier
1 sibling, 0 replies; 18+ messages in thread
From: Pierrick Bouvier @ 2026-07-08 16:14 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: qemu-devel, mark.cave-ayland, richard.henderson, Alex Bennée,
mjt
On 7/8/2026 1:19 AM, Daniel P. Berrangé wrote:
> On Tue, Jul 07, 2026 at 02:46:53PM -0700, Pierrick Bouvier wrote:
>> RUN statements should always use apt update && apt install together:
>> https://docs.docker.com/build/building/best-practices/#apt-get
>>
>> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>> ---
>> tests/docker/dockerfiles/ubuntu2404.docker | 3 ++-
>> tests/lcitool/refresh | 3 ++-
>> 2 files changed, 4 insertions(+), 2 deletions(-)
>>
>> diff --git a/tests/docker/dockerfiles/ubuntu2404.docker b/tests/docker/dockerfiles/ubuntu2404.docker
>> index 26bc2dd94f4..27992e34fd9 100644
>> --- a/tests/docker/dockerfiles/ubuntu2404.docker
>> +++ b/tests/docker/dockerfiles/ubuntu2404.docker
>> @@ -160,7 +160,8 @@ ENV RUSTC=/usr/bin/rustc-1.83
>> ENV RUSTDOC=/usr/bin/rustdoc-1.83
>> ENV CARGO_HOME=/usr/local/cargo
>> ENV PATH=$CARGO_HOME/bin:$PATH
>> -RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>> +RUN apt update && \
>> + DEBIAN_FRONTEND=noninteractive eatmydata \
>> apt install -y --no-install-recommends cargo
>> RUN cargo install --locked bindgen-cli
>> # As a final step configure the user (if env is defined)
>> diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
>> index 1fdd37440c9..bbbc6d3c46a 100755
>> --- a/tests/lcitool/refresh
>> +++ b/tests/lcitool/refresh
>> @@ -157,7 +157,8 @@ ENV RUSTC=/usr/bin/rustc-1.83
>> ENV RUSTDOC=/usr/bin/rustdoc-1.83
>> ENV CARGO_HOME=/usr/local/cargo
>> ENV PATH=$CARGO_HOME/bin:$PATH
>> -RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>> +RUN apt update && \
>> + DEBIAN_FRONTEND=noninteractive eatmydata \
>> apt install -y --no-install-recommends cargo
>> RUN cargo install --locked bindgen-cli
>
> So lcitool already ran 'apt update' earlier, but if we're trying to
> avoid caching issues, I guess running it again is needed. Should we
> also run 'apt dist-upgrade' though, as already installed content in
> the cached layer can be outdated.
>
upgrade/dist-upgrade are not needed, we just address the packages
listing issue, not the fact they are up to date or not.
In fact, if you want an updated container, the good practice is to
rebuild them from scratch. For our use case, it's definitely not needed
as we don't have security concerns to address for local containers.
> With regards,
> Daniel
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 2/4] tests/docker/dockerfiles/ubuntu2404.docker: add missing apt update
2026-07-07 21:46 ` [PATCH 2/4] tests/docker/dockerfiles/ubuntu2404.docker: add missing apt update Pierrick Bouvier
2026-07-08 8:19 ` Daniel P. Berrangé
@ 2026-07-08 16:38 ` Pierrick Bouvier
2026-07-08 20:46 ` Philippe Mathieu-Daudé
2 siblings, 0 replies; 18+ messages in thread
From: Pierrick Bouvier @ 2026-07-08 16:38 UTC (permalink / raw)
To: qemu-devel
Cc: mark.cave-ayland, richard.henderson, Alex Bennée, berrange,
mjt
On 7/7/2026 2:46 PM, Pierrick Bouvier wrote:
> RUN statements should always use apt update && apt install together:
> https://docs.docker.com/build/building/best-practices/#apt-get
>
> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
> ---
> tests/docker/dockerfiles/ubuntu2404.docker | 3 ++-
> tests/lcitool/refresh | 3 ++-
> 2 files changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/tests/docker/dockerfiles/ubuntu2404.docker b/tests/docker/dockerfiles/ubuntu2404.docker
> index 26bc2dd94f4..27992e34fd9 100644
> --- a/tests/docker/dockerfiles/ubuntu2404.docker
> +++ b/tests/docker/dockerfiles/ubuntu2404.docker
> @@ -160,7 +160,8 @@ ENV RUSTC=/usr/bin/rustc-1.83
> ENV RUSTDOC=/usr/bin/rustdoc-1.83
> ENV CARGO_HOME=/usr/local/cargo
> ENV PATH=$CARGO_HOME/bin:$PATH
> -RUN DEBIAN_FRONTEND=noninteractive eatmydata \
> +RUN apt update && \
> + DEBIAN_FRONTEND=noninteractive eatmydata \
> apt install -y --no-install-recommends cargo
> RUN cargo install --locked bindgen-cli
> # As a final step configure the user (if env is defined)
> diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
> index 1fdd37440c9..bbbc6d3c46a 100755
> --- a/tests/lcitool/refresh
> +++ b/tests/lcitool/refresh
> @@ -157,7 +157,8 @@ ENV RUSTC=/usr/bin/rustc-1.83
> ENV RUSTDOC=/usr/bin/rustdoc-1.83
> ENV CARGO_HOME=/usr/local/cargo
> ENV PATH=$CARGO_HOME/bin:$PATH
> -RUN DEBIAN_FRONTEND=noninteractive eatmydata \
> +RUN apt update && \
> + DEBIAN_FRONTEND=noninteractive eatmydata \
> apt install -y --no-install-recommends cargo
> RUN cargo install --locked bindgen-cli
> """
Adding missing reviewed-by to be able to send a PR for this.
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
Regards,
Pierrick
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 2/4] tests/docker/dockerfiles/ubuntu2404.docker: add missing apt update
2026-07-07 21:46 ` [PATCH 2/4] tests/docker/dockerfiles/ubuntu2404.docker: add missing apt update Pierrick Bouvier
2026-07-08 8:19 ` Daniel P. Berrangé
2026-07-08 16:38 ` Pierrick Bouvier
@ 2026-07-08 20:46 ` Philippe Mathieu-Daudé
2 siblings, 0 replies; 18+ messages in thread
From: Philippe Mathieu-Daudé @ 2026-07-08 20:46 UTC (permalink / raw)
To: Pierrick Bouvier, qemu-devel
Cc: mark.cave-ayland, richard.henderson, Alex Bennée, berrange,
mjt
On 7/7/26 23:46, Pierrick Bouvier wrote:
> RUN statements should always use apt update && apt install together:
> https://docs.docker.com/build/building/best-practices/#apt-get
>
> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
> ---
> tests/docker/dockerfiles/ubuntu2404.docker | 3 ++-
> tests/lcitool/refresh | 3 ++-
> 2 files changed, 4 insertions(+), 2 deletions(-)
Reviewed-by: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 0/4] tests/docker/dockerfiles: add missing apt update
2026-07-07 21:46 [PATCH 0/4] tests/docker/dockerfiles: add missing apt update Pierrick Bouvier
` (3 preceding siblings ...)
2026-07-07 21:46 ` [PATCH 4/4] tests/docker/dockerfiles/debian-all-test-cross.docker: " Pierrick Bouvier
@ 2026-07-10 5:13 ` Pierrick Bouvier
4 siblings, 0 replies; 18+ messages in thread
From: Pierrick Bouvier @ 2026-07-10 5:13 UTC (permalink / raw)
To: qemu-devel
Cc: mark.cave-ayland, richard.henderson, Alex Bennée, berrange,
mjt
On 7/7/2026 2:46 PM, Pierrick Bouvier wrote:
> Some of our dockerfiles include custom statements, and those are missing
> explicit apt update when doing an apt install.
>
> Since docker caches layers when building an image, not including an apt update
> may result in error, since packages file will not be up to date.
> This is also documented in official docker documentation:
> https://docs.docker.com/build/building/best-practices/#apt-get
>
> This issue was first reported on:
> https://lore.kernel.org/qemu-devel/4b08f4e2-2274-409a-86d8-469c73e21579@linaro.org/
>
> Pierrick Bouvier (4):
> tests/lcitool/refresh: use raw string literal for trailers
> tests/docker/dockerfiles/ubuntu2404.docker: add missing apt update
> tests/docker/dockerfiles/debian.docker: add missing apt update
> tests/docker/dockerfiles/debian-all-test-cross.docker: add missing apt
> update
>
> .../dockerfiles/debian-all-test-cross.docker | 4 +-
> tests/docker/dockerfiles/debian.docker | 3 +-
> tests/docker/dockerfiles/ubuntu2404.docker | 3 +-
> tests/lcitool/refresh | 213 +++++++++---------
> 4 files changed, 116 insertions(+), 107 deletions(-)
>
This was merged into master (4faab9a11f8be37c869894cda821823ef6ca7f58).
Regards,
Pierrick
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2026-07-10 5:14 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-07-07 21:46 [PATCH 0/4] tests/docker/dockerfiles: add missing apt update Pierrick Bouvier
2026-07-07 21:46 ` [PATCH 1/4] tests/lcitool/refresh: use raw string literal for trailers Pierrick Bouvier
2026-07-08 8:13 ` Daniel P. Berrangé
2026-07-08 15:56 ` Pierrick Bouvier
2026-07-07 21:46 ` [PATCH 2/4] tests/docker/dockerfiles/ubuntu2404.docker: add missing apt update Pierrick Bouvier
2026-07-08 8:19 ` Daniel P. Berrangé
2026-07-08 9:27 ` Michael Tokarev
2026-07-08 14:28 ` Philippe Mathieu-Daudé
2026-07-08 14:54 ` Michael Tokarev
2026-07-08 16:11 ` Pierrick Bouvier
2026-07-08 16:14 ` Pierrick Bouvier
2026-07-08 16:38 ` Pierrick Bouvier
2026-07-08 20:46 ` Philippe Mathieu-Daudé
2026-07-07 21:46 ` [PATCH 3/4] tests/docker/dockerfiles/debian.docker: " Pierrick Bouvier
2026-07-08 8:25 ` Daniel P. Berrangé
2026-07-07 21:46 ` [PATCH 4/4] tests/docker/dockerfiles/debian-all-test-cross.docker: " Pierrick Bouvier
2026-07-08 8:50 ` Daniel P. Berrangé
2026-07-10 5:13 ` [PATCH 0/4] tests/docker/dockerfiles: " Pierrick Bouvier
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.