Re: [PULL 1/1] Revert "configure: build ROMs with container-based cross compilers"

["Alex Bennée" <alex.bennee@linaro.org>]

Daniel Henrique Barboza <danielhb413@gmail.com> writes:

> On 10/12/22 09:13, Alex Bennée wrote:
>> Daniel Henrique Barboza <danielhb413@gmail.com> writes:
>> 
>>> On 10/12/22 03:46, Paolo Bonzini wrote:
>>>> Il mar 11 ott 2022, 21:29 Alex Bennée <alex.bennee@linaro.org
>>>> <mailto:alex.bennee@linaro.org>> ha scritto:
>>>>      This reverts commit 730fe750fba63023e294ff0acf0f874369f1946f.
>>>>      Unconditionally building all the bios for all arches was a
>>>> little too
>>>>      far too fast.
>>>> I would like to understand the issue better, because chances are
>>>> that it is preexisting and applies to the TCG tests as well.
>>>> Daniel, does building the TCG tests work for you? If not, I think we
>>>> should just disable containers by default.
>>>
>>>
>>> 'make check-tcg' never worked in this particular Xeon host I use. I never
>>> had the curiosity to find out why because I have access to a Power9 host
>>> that runs 'make check-tcg'.
>>>
>>> Using this revert patch on top of master in this Xeon box makes 'make -j'
>>> successful and 'make check-tcg' fails with the following error:

What are you running on the Xeon machine? I think it indicates you have
podman but it can't do rootless builds (which I think is the whole point
of podman). I can try an replicate your setup and make the docker.py
probe a bit more robust.

>>>
>>>
>>> $ make -j
>>>    GIT     ui/keycodemapdb meson tests/fp/berkeley-testfloat-3 tests/fp/berkeley-softfloat-3 dtc
>>> [1/24] Generating qemu-version.h with a custom command (wrapped by meson to capture output)
>>>
>>> $ make check-tcg
>>>    GIT     ui/keycodemapdb meson tests/fp/berkeley-testfloat-3 tests/fp/berkeley-softfloat-3 dtc
>>>    BUILD   debian-powerpc-test-cross
>>>    BUILD   ppc64-linux-user guest-tests
>>> Traceback (most recent call last):
>>>    File "/home/danielhb/qemu/tests/docker/docker.py", line 683, in <module>
>>>      sys.exit(main())
>>>    File "/home/danielhb/qemu/tests/docker/docker.py", line 679, in main
>>>      return args.cmdobj.run(args, argv)
>>>    File "/home/danielhb/qemu/tests/docker/docker.py", line 657, in run
>>>      return Docker().run(cmd, False, quiet=args.quiet,
>>>    File "/home/danielhb/qemu/tests/docker/docker.py", line 370, in run
>>>      ret = self._do_check(["run", "--rm", "--label",
>>>    File "/home/danielhb/qemu/tests/docker/docker.py", line 247, in _do_check
>>>      return subprocess.check_call(self._command + cmd, **kwargs)
>>>    File "/usr/lib64/python3.9/subprocess.py", line 373, in check_call
>>>      raise CalledProcessError(retcode, cmd)
>>> subprocess.CalledProcessError: Command '['podman', 'run', '--rm', '--label', 'com.qemu.instance.uuid=cf15761c98884d0a9b4e37f631ba593f', '--userns=keep-id', '-u', '1005', '-w', '/home/danielhb/qemu/build/tests/tcg/ppc64-linux-user', '-v', '/home/danielhb/qemu/build/tests/tcg/ppc64-linux-user:/home/danielhb/qemu/build/tests/tcg/ppc64-linux-user:rw', '-v', '/home/danielhb/qemu:/home/danielhb/qemu:ro,z', 'qemu/debian-powerpc-test-cross', 'powerpc64-linux-gnu-gcc-10', '-Wall', '-Werror', '-O0', '-g', '-fno-strict-aliasing', '-m64', '-mbig-endian', '/home/danielhb/qemu/tests/tcg/multiarch/float_convd.c', '/home/danielhb/qemu/tests/tcg/multiarch/libs/float_helpers.c', '-o', 'float_convd', '-static', '-lm']' returned non-zero exit status 127.
>>> filter=--filter=label=com.qemu.instance.uuid=cf15761c98884d0a9b4e37f631ba593f
>>> make[1]: *** [/home/danielhb/qemu/tests/tcg/multiarch/Makefile.target:26: float_convd] Error 1
>>> make: *** [/home/danielhb/qemu/tests/Makefile.include:50: build-tcg-tests-ppc64-linux-user] Error 2
>>>
>>>
>>> This is very similar to the error message I get when running 'make -j' on mainline
>>> without this revert.
>>>
>>> So yeah, I guess we can say this is a preexisting condition that I always saw with
>>> 'make check-tcg' in this particular host, and 730fe750fba just made it manifest when
>>> running a plain 'make'.
>>>
>>>
>>> Thanks,
>>>
>>>
>>> Daniel
>>>
>>>
>>>>      Signed-off-by: Alex Bennée <alex.bennee@linaro.org
>>>> <mailto:alex.bennee@linaro.org>>
>>>>      Cc: Paolo Bonzini <pbonzini@redhat.com <mailto:pbonzini@redhat.com>>
>>>>      Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com <mailto:danielhb413@gmail.com>>
>>>>      Tested-by: Daniel Henrique Barboza <danielhb413@gmail.com <mailto:danielhb413@gmail.com>>
>>>>      Message-Id: <20221011113417.794841-4-alex.bennee@linaro.org <mailto:20221011113417.794841-4-alex.bennee@linaro.org>>
>>>>      diff --git a/configure b/configure
>>>>      index baa69189f0..45ee6f4eb3 100755
>>>>      --- a/configure
>>>>      +++ b/configure
>>>>      @@ -2121,7 +2121,7 @@ probe_target_compiler() {
>>>>            target_ranlib=
>>>>            target_strip=
>>>>          fi
>>>>      -  test -n "$target_cc" || test -n "$container_image"
>>>>      +  test -n "$target_cc"
>>>>        }
>>>>        write_target_makefile() {
>>>>      @@ -2268,7 +2268,7 @@ if test "$targetos" != "darwin" && test "$targetos" != "sunos" && \
>>>>            config_mak=pc-bios/optionrom/config.mak
>>>>            echo "# Automatically generated by configure - do not modify" > $config_mak
>>>>            echo "TOPSRC_DIR=$source_path" >> $config_mak
>>>>      -    write_target_makefile pc-bios/optionrom/all >> $config_mak
>>>>      +    write_target_makefile >> $config_mak
>>>>        fi
>>>>        if test "$softmmu" = yes && probe_target_compiler ppc-softmmu;
>>>> then
>>>>      @@ -2276,31 +2276,25 @@ if test "$softmmu" = yes && probe_target_compiler ppc-softmmu; then
>>>>            config_mak=pc-bios/vof/config.mak
>>>>            echo "# Automatically generated by configure - do not modify" > $config_mak
>>>>            echo "SRC_DIR=$source_path/pc-bios/vof" >> $config_mak
>>>>      -    write_target_makefile pc-bios/vof/all >> $config_mak
>>>>      +    write_target_makefile >> $config_mak
>>>>        fi
>>>>        # Only build s390-ccw bios if the compiler has -march=z900 or
>>>> -march=z10
>>>>        # (which is the lowest architecture level that Clang supports)
>>>>        if test "$softmmu" = yes && probe_target_compiler s390x-softmmu; then
>>>>      -  got_cross_cc=no
>>>>      -  if test -n "$target_cc"; then
>>>>      -    write_c_skeleton
>>>>      -    do_compiler "$target_cc" $target_cc_cflags -march=z900 -o $TMPO -c $TMPC
>>>>      -    has_z900=$?
>>>>      -    if [ $has_z900 = 0 ] || do_compiler "$target_cc" $target_cc_cflags -march=z10 -msoft-float -Werror -o $TMPO -c $TMPC; then
>>>>      -      if [ $has_z900 != 0 ]; then
>>>>      -        echo "WARNING: Your compiler does not support the z900!"
>>>>      -        echo "         The s390-ccw bios will only work with guest CPUs >= z10."
>>>>      -      fi
>>>>      -      got_cross_cc=yes
>>>>      +  write_c_skeleton
>>>>      +  do_compiler "$target_cc" $target_cc_cflags -march=z900 -o $TMPO -c $TMPC
>>>>      +  has_z900=$?
>>>>      +  if [ $has_z900 = 0 ] || do_compiler "$target_cc" $target_cc_cflags -march=z10 -msoft-float -Werror -o $TMPO -c $TMPC; then
>>>>      +    if [ $has_z900 != 0 ]; then
>>>>      +      echo "WARNING: Your compiler does not support the z900!"
>>>>      +      echo "         The s390-ccw bios will only work with guest CPUs >= z10."
>>>>            fi
>>>>      -  fi
>>>>      -  if test "$got_cross_cc" = yes || test -n "$container_image"; then
>>>>            roms="$roms pc-bios/s390-ccw"
>>>>            config_mak=pc-bios/s390-ccw/config-host.mak
>>>>            echo "# Automatically generated by configure - do not modify" > $config_mak
>>>>            echo "SRC_PATH=$source_path/pc-bios/s390-ccw" >> $config_mak
>>>>      -    write_target_makefile pc-bios/s390-ccw/all >> $config_mak
>>>>      +    write_target_makefile >> $config_mak
>>>>            # SLOF is required for building the s390-ccw firmware on s390x,
>>>>            # since it is using the libnet code from SLOF for network booting.
>>>>            git_submodules="${git_submodules} roms/SLOF"
>>>>      @@ -2488,7 +2482,7 @@ for target in $target_list; do
>>>>              ;;
>>>>          esac
>>>>      -  if probe_target_compiler $target; then
>>>>      +  if probe_target_compiler $target || test -n "$container_image"; then
>>>>              test -n "$container_image" && build_static=y
>>>>              mkdir -p "tests/tcg/$target"
>>>>              config_target_mak=tests/tcg/$target/config-target.mak
>>>>      --     2.34.1
>>>>
>> Can you check what ENGINE is in your config-host.mak? If no
>> container
>> engine is defined we should be gating against running docker.
>
> Yeah. Running the case I mentioned above (revert is applied, 'make check-tcg'
> fails):
>
> [danielhb@kal1 build]$ grep ENGINE config-host.mak
> ENGINE=podman
>
>
> And with current master where 'make' is failing:
>
> [danielhb@kal1 build]$ grep ENGINE config-host.mak
> ENGINE=podman
>
>
> podman version:
>
> $ podman --version
> podman version 4.0.2
>
>
>
> Daniel
>
>> 


-- 
Alex Bennée