Re: [PATCH for 10.1 13/13] tests/docker: handle host-arch selection for all-test-cross

["Alex Bennée" <alex.bennee@linaro.org>]

Manos Pitsidianakis <manos.pitsidianakis@linaro.org> writes:

> On Thu, Jul 24, 2025 at 3:36 PM Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>> Manos Pitsidianakis <manos.pitsidianakis@linaro.org> writes:
>>
>> > On Thu, Jul 24, 2025 at 2:00 PM Alex Bennée <alex.bennee@linaro.org> wrote:
>> >>
>> >> When building on non-x86 we get a bunch but not all of the compilers.
>> >> Handle this in the Dockerfile by probing the arch and expanding the
>> >> list available.
>> >>
>> >> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> >> ---
>> >>  .../dockerfiles/debian-all-test-cross.docker  | 31 ++++++++++---------
>> >>  1 file changed, 17 insertions(+), 14 deletions(-)
>> >>
>> >> diff --git a/tests/docker/dockerfiles/debian-all-test-cross.docker b/tests/docker/dockerfiles/debian-all-test-cross.docker
>> >> index 5aa43749ebe..16a83241270 100644
>> >> --- a/tests/docker/dockerfiles/debian-all-test-cross.docker
>> >> +++ b/tests/docker/dockerfiles/debian-all-test-cross.docker
>> >> @@ -23,7 +23,9 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>> >>          bison \
>> >>          ccache \
>> >>          clang  \
>> >> +        dpkg-dev \
>> >>          flex \
>> >> +        gcc \
>> >>          git \
>> >>          libclang-rt-dev \
>> >>          ninja-build \
>> >> @@ -33,16 +35,11 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>> >>          python3-venv \
>> >>          python3-wheel
>> >>
>> >> -RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>> >> -        apt install -y --no-install-recommends \
>> >> -        gcc-aarch64-linux-gnu \
>> >> +# All the generally available compilers
>> >> +ENV AVAILABLE_COMPILERS gcc-aarch64-linux-gnu \
>> >>          libc6-dev-arm64-cross \
>> >>          gcc-arm-linux-gnueabihf \
>> >>          libc6-dev-armhf-cross \
>> >> -        gcc-hppa-linux-gnu \
>> >> -        libc6-dev-hppa-cross \
>> >> -        gcc-m68k-linux-gnu \
>> >> -        libc6-dev-m68k-cross \
>> >>          gcc-mips-linux-gnu \
>> >>          libc6-dev-mips-cross \
>> >>          gcc-mips64-linux-gnuabi64 \
>> >> @@ -51,18 +48,24 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>> >>          libc6-dev-mips64el-cross \
>> >>          gcc-mipsel-linux-gnu \
>> >>          libc6-dev-mipsel-cross \
>> >> -        gcc-powerpc-linux-gnu \
>> >> -        libc6-dev-powerpc-cross \
>> >> -        gcc-powerpc64-linux-gnu \
>> >> -        libc6-dev-ppc64-cross \
>> >>          gcc-powerpc64le-linux-gnu \
>> >>          libc6-dev-ppc64el-cross \
>> >>          gcc-riscv64-linux-gnu \
>> >>          libc6-dev-riscv64-cross \
>> >>          gcc-s390x-linux-gnu \
>> >> -        libc6-dev-s390x-cross \
>> >> -        gcc-sparc64-linux-gnu \
>> >> -        libc6-dev-sparc64-cross && \
>> >> +        libc6-dev-s390x-cross
>> >> +
>> >> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-hppa-linux-gnu libc6-dev-hppa-cross"; fi
>> >> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-m68k-linux-gnu libc6-dev-m68k-cross"; fi
>> >> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-powerpc-linux-gnu libc6-dev-powerpc-cross"; fi
>> >> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-powerpc64-linux-gnu libc6-dev-ppc64-cross"; fi
>> >> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-sparc64-linux-gnu libc6-dev-sparc64-cross"; fi
>> >> +
>> >> +RUN echo "compilers: ${AVAILABLE_COMPILERS}"
>> >
>> > Nitpick, each `RUN` command will create a new cached layer for the
>> > container build. It makes more sense to fold them in a single `RUN`
>> > step to avoid unnecessary layers. Does not make a big difference so
>> > feel free to ignore.
>>
>> I did try to figure out how to do a multi-line shell with an env
>> expansion but wasn't able to get the escaping right. If you can suggest
>> the right runes please do ;-)
>
> Like this?
>
> RUN if dpkg-architecture -e amd64; then \
>   export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}
> gcc-hppa-linux-gnu libc6-dev-hppa-cross"; \
>   export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}
> gcc-m68k-linux-gnu libc6-dev-m68k-cross"; \
>   export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}
> gcc-powerpc-linux-gnu libc6-dev-powerpc-cross"; \
>   export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}
> gcc-powerpc64-linux-gnu libc6-dev-ppc64-cross"; \
>   export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}
> gcc-sparc64-linux-gnu libc6-dev-sparc64-cross"; \
>   fi && \
>   echo "compilers: ${AVAILABLE_COMPILERS}"
>

Nope, the continuation has to run for all the lines of the RUN
statement.

  Processing triggers for libc-bin (2.36-9+deb12u10) ...
  --> 703d5c21c6e4
  STEP 5/18: ENV AVAILABLE_COMPILERS gcc-aarch64-linux-gnu         libc6-dev-arm64-cross         gcc-arm-linux-gnueabihf         libc6-dev-armhf-cross         gcc-mips-linux-gnu         libc6-dev-mips-cross         gcc-mips64-linux-gnuabi64         libc6-dev-mips64-cross         gcc-mips64el-linux-gnuabi64         libc6-dev-mips64el-cross         gcc-mipsel-linux-gnu         libc6-dev-mipsel-cross         gcc-powerpc64le-linux-gnu         libc6-dev-ppc64el-cross         gcc-riscv64-linux-gnu         libc6-dev-riscv64-cross         gcc-s390x-linux-gnu         libc6-dev-s390x-cross
  --> e6fa99881c94
  STEP 6/18: RUN if dpkg-architecture -e amd64; then   export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}
  /bin/sh: 1: Syntax error: Unterminated quoted string
  Error: building at STEP "RUN if dpkg-architecture -e amd64; then   export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}": while running runtime: exit status 2
  make: *** [tests/docker/Makefile.include:40: docker-image-debian-all-test-cross] Error 2

>
>>
>> >
>> > Reviewed-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
>> >
>> >> +
>> >> +RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>> >> +        apt install -y --no-install-recommends \
>> >> +        ${AVAILABLE_COMPILERS} && \
>> >>          dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt
>> >>
>> >>
>> >> --
>> >> 2.47.2
>> >>
>> >>
>>
>> --
>> Alex Bennée
>> Virtualisation Tech Lead @ Linaro

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro