Re: [PULL 1/1] Revert "configure: build ROMs with container-based cross compilers"

[Daniel Henrique Barboza <danielhb413@gmail.com>]

On 10/12/22 09:13, Alex Bennée wrote:
> 
> Daniel Henrique Barboza <danielhb413@gmail.com> writes:
> 
>> On 10/12/22 03:46, Paolo Bonzini wrote:
>>> Il mar 11 ott 2022, 21:29 Alex Bennée <alex.bennee@linaro.org
>>> <mailto:alex.bennee@linaro.org>> ha scritto:
>>>      This reverts commit 730fe750fba63023e294ff0acf0f874369f1946f.
>>>      Unconditionally building all the bios for all arches was a
>>> little too
>>>      far too fast.
>>> I would like to understand the issue better, because chances are
>>> that it is preexisting and applies to the TCG tests as well.
>>> Daniel, does building the TCG tests work for you? If not, I think we
>>> should just disable containers by default.
>>
>>
>> 'make check-tcg' never worked in this particular Xeon host I use. I never
>> had the curiosity to find out why because I have access to a Power9 host
>> that runs 'make check-tcg'.
>>
>> Using this revert patch on top of master in this Xeon box makes 'make -j'
>> successful and 'make check-tcg' fails with the following error:
>>
>>
>> $ make -j
>>    GIT     ui/keycodemapdb meson tests/fp/berkeley-testfloat-3 tests/fp/berkeley-softfloat-3 dtc
>> [1/24] Generating qemu-version.h with a custom command (wrapped by meson to capture output)
>>
>> $ make check-tcg
>>    GIT     ui/keycodemapdb meson tests/fp/berkeley-testfloat-3 tests/fp/berkeley-softfloat-3 dtc
>>    BUILD   debian-powerpc-test-cross
>>    BUILD   ppc64-linux-user guest-tests
>> Traceback (most recent call last):
>>    File "/home/danielhb/qemu/tests/docker/docker.py", line 683, in <module>
>>      sys.exit(main())
>>    File "/home/danielhb/qemu/tests/docker/docker.py", line 679, in main
>>      return args.cmdobj.run(args, argv)
>>    File "/home/danielhb/qemu/tests/docker/docker.py", line 657, in run
>>      return Docker().run(cmd, False, quiet=args.quiet,
>>    File "/home/danielhb/qemu/tests/docker/docker.py", line 370, in run
>>      ret = self._do_check(["run", "--rm", "--label",
>>    File "/home/danielhb/qemu/tests/docker/docker.py", line 247, in _do_check
>>      return subprocess.check_call(self._command + cmd, **kwargs)
>>    File "/usr/lib64/python3.9/subprocess.py", line 373, in check_call
>>      raise CalledProcessError(retcode, cmd)
>> subprocess.CalledProcessError: Command '['podman', 'run', '--rm', '--label', 'com.qemu.instance.uuid=cf15761c98884d0a9b4e37f631ba593f', '--userns=keep-id', '-u', '1005', '-w', '/home/danielhb/qemu/build/tests/tcg/ppc64-linux-user', '-v', '/home/danielhb/qemu/build/tests/tcg/ppc64-linux-user:/home/danielhb/qemu/build/tests/tcg/ppc64-linux-user:rw', '-v', '/home/danielhb/qemu:/home/danielhb/qemu:ro,z', 'qemu/debian-powerpc-test-cross', 'powerpc64-linux-gnu-gcc-10', '-Wall', '-Werror', '-O0', '-g', '-fno-strict-aliasing', '-m64', '-mbig-endian', '/home/danielhb/qemu/tests/tcg/multiarch/float_convd.c', '/home/danielhb/qemu/tests/tcg/multiarch/libs/float_helpers.c', '-o', 'float_convd', '-static', '-lm']' returned non-zero exit status 127.
>> filter=--filter=label=com.qemu.instance.uuid=cf15761c98884d0a9b4e37f631ba593f
>> make[1]: *** [/home/danielhb/qemu/tests/tcg/multiarch/Makefile.target:26: float_convd] Error 1
>> make: *** [/home/danielhb/qemu/tests/Makefile.include:50: build-tcg-tests-ppc64-linux-user] Error 2
>>
>>
>> This is very similar to the error message I get when running 'make -j' on mainline
>> without this revert.
>>
>> So yeah, I guess we can say this is a preexisting condition that I always saw with
>> 'make check-tcg' in this particular host, and 730fe750fba just made it manifest when
>> running a plain 'make'.
>>
>>
>> Thanks,
>>
>>
>> Daniel
>>
>>
>>>      Signed-off-by: Alex Bennée <alex.bennee@linaro.org
>>> <mailto:alex.bennee@linaro.org>>
>>>      Cc: Paolo Bonzini <pbonzini@redhat.com <mailto:pbonzini@redhat.com>>
>>>      Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com <mailto:danielhb413@gmail.com>>
>>>      Tested-by: Daniel Henrique Barboza <danielhb413@gmail.com <mailto:danielhb413@gmail.com>>
>>>      Message-Id: <20221011113417.794841-4-alex.bennee@linaro.org <mailto:20221011113417.794841-4-alex.bennee@linaro.org>>
>>>      diff --git a/configure b/configure
>>>      index baa69189f0..45ee6f4eb3 100755
>>>      --- a/configure
>>>      +++ b/configure
>>>      @@ -2121,7 +2121,7 @@ probe_target_compiler() {
>>>            target_ranlib=
>>>            target_strip=
>>>          fi
>>>      -  test -n "$target_cc" || test -n "$container_image"
>>>      +  test -n "$target_cc"
>>>        }
>>>        write_target_makefile() {
>>>      @@ -2268,7 +2268,7 @@ if test "$targetos" != "darwin" && test "$targetos" != "sunos" && \
>>>            config_mak=pc-bios/optionrom/config.mak
>>>            echo "# Automatically generated by configure - do not modify" > $config_mak
>>>            echo "TOPSRC_DIR=$source_path" >> $config_mak
>>>      -    write_target_makefile pc-bios/optionrom/all >> $config_mak
>>>      +    write_target_makefile >> $config_mak
>>>        fi
>>>        if test "$softmmu" = yes && probe_target_compiler ppc-softmmu;
>>> then
>>>      @@ -2276,31 +2276,25 @@ if test "$softmmu" = yes && probe_target_compiler ppc-softmmu; then
>>>            config_mak=pc-bios/vof/config.mak
>>>            echo "# Automatically generated by configure - do not modify" > $config_mak
>>>            echo "SRC_DIR=$source_path/pc-bios/vof" >> $config_mak
>>>      -    write_target_makefile pc-bios/vof/all >> $config_mak
>>>      +    write_target_makefile >> $config_mak
>>>        fi
>>>        # Only build s390-ccw bios if the compiler has -march=z900 or
>>> -march=z10
>>>        # (which is the lowest architecture level that Clang supports)
>>>        if test "$softmmu" = yes && probe_target_compiler s390x-softmmu; then
>>>      -  got_cross_cc=no
>>>      -  if test -n "$target_cc"; then
>>>      -    write_c_skeleton
>>>      -    do_compiler "$target_cc" $target_cc_cflags -march=z900 -o $TMPO -c $TMPC
>>>      -    has_z900=$?
>>>      -    if [ $has_z900 = 0 ] || do_compiler "$target_cc" $target_cc_cflags -march=z10 -msoft-float -Werror -o $TMPO -c $TMPC; then
>>>      -      if [ $has_z900 != 0 ]; then
>>>      -        echo "WARNING: Your compiler does not support the z900!"
>>>      -        echo "         The s390-ccw bios will only work with guest CPUs >= z10."
>>>      -      fi
>>>      -      got_cross_cc=yes
>>>      +  write_c_skeleton
>>>      +  do_compiler "$target_cc" $target_cc_cflags -march=z900 -o $TMPO -c $TMPC
>>>      +  has_z900=$?
>>>      +  if [ $has_z900 = 0 ] || do_compiler "$target_cc" $target_cc_cflags -march=z10 -msoft-float -Werror -o $TMPO -c $TMPC; then
>>>      +    if [ $has_z900 != 0 ]; then
>>>      +      echo "WARNING: Your compiler does not support the z900!"
>>>      +      echo "         The s390-ccw bios will only work with guest CPUs >= z10."
>>>            fi
>>>      -  fi
>>>      -  if test "$got_cross_cc" = yes || test -n "$container_image"; then
>>>            roms="$roms pc-bios/s390-ccw"
>>>            config_mak=pc-bios/s390-ccw/config-host.mak
>>>            echo "# Automatically generated by configure - do not modify" > $config_mak
>>>            echo "SRC_PATH=$source_path/pc-bios/s390-ccw" >> $config_mak
>>>      -    write_target_makefile pc-bios/s390-ccw/all >> $config_mak
>>>      +    write_target_makefile >> $config_mak
>>>            # SLOF is required for building the s390-ccw firmware on s390x,
>>>            # since it is using the libnet code from SLOF for network booting.
>>>            git_submodules="${git_submodules} roms/SLOF"
>>>      @@ -2488,7 +2482,7 @@ for target in $target_list; do
>>>              ;;
>>>          esac
>>>      -  if probe_target_compiler $target; then
>>>      +  if probe_target_compiler $target || test -n "$container_image"; then
>>>              test -n "$container_image" && build_static=y
>>>              mkdir -p "tests/tcg/$target"
>>>              config_target_mak=tests/tcg/$target/config-target.mak
>>>      --     2.34.1
>>>
> 
> Can you check what ENGINE is in your config-host.mak? If no container
> engine is defined we should be gating against running docker.

Yeah. Running the case I mentioned above (revert is applied, 'make check-tcg'
fails):

[danielhb@kal1 build]$ grep ENGINE config-host.mak
ENGINE=podman


And with current master where 'make' is failing:

[danielhb@kal1 build]$ grep ENGINE config-host.mak
ENGINE=podman


podman version:

$ podman --version
podman version 4.0.2



Daniel

>