Re: [Qemu-devel] [PATCH 7/8] docker: Cross build QEMU on ppc32 hosts

["Alex Bennée" <alex.bennee@linaro.org>]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> On 06/29/2018 11:29 AM, Alex Bennée wrote:
>> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
>>
>>> Since 08f56d8c9c the powerpc-cross image is no more based on Emdebian
>>> but on the Sid release.
>>>
>>> This patch restore the ability to cross build QEMU on powerpc, which
>>> is interesting since it is a 32-bit host.
>>>
>>> The ugly apt-fake script is, however, still required.
>>>
>>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>>
>>
>> Hmm I couldn't get this to work with:
>>
>>    make docker-test-build@debian-powerpc-cross J=9 V=1 NOCACHE=1
>>
>>
>> Step 9/13 : RUN apt-fake install         glusterfs-common:powerpc=4.0.2-fake
>>  ---> Running in 0bf7bf5e268a
>> Reading package lists...
>> Building dependency tree...
>> Reading state information...
>> E: Version '4.0.2-fake' for 'glusterfs-common:powerpc' was not found
>> Generating fake glusterfs-common powerpc 4.0.2-fake ...
>> Selecting previously unselected package glusterfs-common:powerpc.
>> (Reading database ... 19461 files and directories currently installed.)
>> Preparing to unpack glusterfs-common_4.0.2-fake_powerpc.deb ...
>> Unpacking glusterfs-common:powerpc (4.0.2-fake) ...
>> Setting up glusterfs-common:powerpc (4.0.2-fake) ...
>> Removing intermediate container 0bf7bf5e268a
>>  ---> 9f903850b104
>> Step 10/13 : RUN DEBIAN_FRONTEND=noninteractive eatmydata     apt-get build-dep -yy -a powerpc qemu
>>  ---> Running in 973b5c17668d
>> Reading package lists...
>> Building dependency tree...
>> Reading state information...
>> Some packages could not be installed. This may mean that you have
>> requested an impossible situation or if you are using the unstable
>> distribution that some required packages have not yet been created
>> or been moved out of Incoming.
>> The following information may help to resolve the situation:
>>
>> The following packages have unmet dependencies:
>>  builddeps:qemu:powerpc : Depends: libcapstone-dev:powerpc but it is not going to be installed
>>                           Depends: libaio-dev:powerpc but it is not going to be installed
>>                           Depends: libpulse-dev:powerpc but it is not going to be installed
>>                           Depends: libasound2-dev:powerpc but it is not going to be installed
>>                           Depends: libattr1-dev:powerpc but it is not going to be installed
>>                           Depends: libbluetooth-dev:powerpc but it is not going to be installed
>>                           Depends: libbrlapi-dev:powerpc but it is not going to be installed
>>                           Depends: libcap-dev:powerpc but it is not going to be installed
>>                           Depends: libcap-ng-dev:powerpc but it is not going to be installed
>>                           Depends: libcurl4-gnutls-dev:powerpc but it is not going to be installed
>>                           Depends: libfdt-dev:powerpc but it is not going to be installed
>>                           Depends: gnutls-dev:powerpc
>>                           Depends: libgtk-3-dev:powerpc but it is not going to be installed
>>                           Depends: libvte-2.91-dev:powerpc but it is not going to be installed
>>                           Depends: libiscsi-dev:powerpc (> 1.9.0~) but it is not going to be installed
>>                           Depends: libncursesw5-dev:powerpc
>>                           Depends: libnfs-dev:powerpc (> 1.9.3) but it is not going to be installed
>>                           Depends: libnuma-dev:powerpc but it is not going to be installed
>>                           Depends: libcacard-dev:powerpc but it is not going to be installed
>>                           Depends: libpixman-1-dev:powerpc but it is not going to be installed
>>                           Depends: librados-dev:powerpc but it is not going to be installed
>>                           Depends: librbd-dev:powerpc but it is not going to be installed
>>                           Depends: libsasl2-dev:powerpc but it is not going to be installed
>>                           Depends: libseccomp-dev:powerpc (>= 2.3.0) but it is not going to be installed
>>                           Depends: libusb-1.0-0-dev:powerpc (>= 2:1.0.13~) but it is not going to be installed
>>                           Depends: libusbredirparser-dev:powerpc (>= 0.6~) but it is not going to be installed
>>                           Depends: libssh2-1-dev:powerpc but it is not going to be installed
>>                           Depends: libvdeplug-dev:powerpc but it is not going to be installed
>>                           Depends: uuid-dev:powerpc but it is not going to be installed
>>                           Depends: xfslibs-dev:powerpc but it is not going to be installed
>>                           Depends: zlib1g-dev:powerpc but it is not going to be installed
>>                           Depends: libjpeg-dev:powerpc
>>                           Depends: libpng-dev:powerpc but it is not going to be installed
>> E: Unable to correct problems, you have held broken packages.
>> The command '/bin/sh -c DEBIAN_FRONTEND=noninteractive eatmydata     apt-get build-dep -yy -a powerpc qemu' returned a non-zero code: 100
>> Traceback (most recent call last):
>>   File "/home/alex/lsrc/qemu/qemu.git/tests/docker/docker.py", line 526, in <module>
>>     sys.exit(main())
>>   File "/home/alex/lsrc/qemu/qemu.git/tests/docker/docker.py", line 523, in main
>>     return args.cmdobj.run(args, argv)
>>   File "/home/alex/lsrc/qemu/qemu.git/tests/docker/docker.py", line 344, in run
>>     extra_files_cksum=cksum)
>>   File "/home/alex/lsrc/qemu/qemu.git/tests/docker/docker.py", line 227, in build_image
>>     quiet=quiet)
>>   File "/home/alex/lsrc/qemu/qemu.git/tests/docker/docker.py", line 154, in _do_check
>>     return subprocess.check_call(self._command + cmd, **kwargs)
>>   File "/usr/lib/python2.7/subprocess.py", line 190, in check_call
>>     raise CalledProcessError(retcode, cmd)
>> subprocess.CalledProcessError: Command '['docker', 'build', '-t', 'qemu:debian-powerpc-cross', '-f', '/tmp/docker_buildjXN8x4/tmpJASaT1.docker', '--no-cache', '/tmp/docker_buildjXN8x4']' returned non-zero exit status 100
>> /home/alex/lsrc/qemu/qemu.git/tests/docker/Makefile.include:50: recipe for target 'docker-image-debian-powerpc-cross' failed
>> make: *** [docker-image-debian-powerpc-cross] Error 1
>
> Caveats of using Debian/Sid again :(
>
> It worked for me 5 days ago, and I'v been using this image since, but
> without rebuilding it, so did not notice the upstream packages change
> again... whew :(
>
> Let's postpone this for the next development window.

Yeah I think all we'll ever be doing is papering over the cracks with
this one. I think to support powerpc as a QEMU host we need to find a
stable host OS which actually supports it and cross compiling.

It's one thing hoping sid's gcc/libc stay functional but at soon as more
packages are involved it becomes a lot more dicey. Once gcc/libc bitrot
away for the platform then it is truly dead....

>
> Thanks for giving it a try!
>
> Phil.
>
>>
>>> ---
>>>  tests/docker/Makefile.include                 |  2 +
>>>  tests/docker/dockerfiles/debian-apt-fake.sh   | 46 +++++++++++++++++++
>>>  .../dockerfiles/debian-powerpc-cross.docker   | 43 +++++++++++++++++
>>>  3 files changed, 91 insertions(+)
>>>  create mode 100755 tests/docker/dockerfiles/debian-apt-fake.sh
>>>
>>> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
>>> index 301ee68ae6..d73d60cfbb 100644
>>> --- a/tests/docker/Makefile.include
>>> +++ b/tests/docker/Makefile.include
>>> @@ -58,6 +58,8 @@ docker-image-%: $(DOCKER_FILES_DIR)/%.docker
>>>  		"BUILD","$*")
>>>  endif
>>>
>>> +docker-image-debian-powerpc-cross: EXTRA_FILES:=$(SRC_PATH)/tests/docker/dockerfiles/debian-apt-fake.sh
>>> +
>>>  # Enforce dependencies for composite images
>>>  docker-image-debian: docker-image-debian9
>>>  docker-image-debian8-mxe: docker-image-debian8
>>> diff --git a/tests/docker/dockerfiles/debian-apt-fake.sh b/tests/docker/dockerfiles/debian-apt-fake.sh
>>> new file mode 100755
>>> index 0000000000..2ec0fdf47a
>>> --- /dev/null
>>> +++ b/tests/docker/dockerfiles/debian-apt-fake.sh
>>> @@ -0,0 +1,46 @@
>>> +#! /bin/sh
>>> +#
>>> +# Generate fake debian package to resolve unimportant unmet dependencies held
>>> +# by upstream multiarch broken packages.
>>> +#
>>> +# Copyright (c) 2017 Philippe Mathieu-Daudé <f4bug@amsat.org>
>>> +#
>>> +# This work is licensed under the terms of the GNU GPL, version 2
>>> +# or (at your option) any later version. See the COPYING file in
>>> +# the top-level directory.
>>> +
>>> +test $1 = "install" && shift 1
>>> +
>>> +fake_install()
>>> +{
>>> +    echo "Generating fake $2 $1 $3 ..."
>>> +    (cd /var/cache/apt/archives
>>> +        (cat << 'EOF'
>>> +Section: misc
>>> +Priority: optional
>>> +Standards-Version: 3.9.2
>>> +
>>> +Package: NAME
>>> +Version: VERSION
>>> +Maintainer: qemu-devel@nongnu.org
>>> +Architecture: any
>>> +Multi-Arch: same
>>> +Description: fake NAME
>>> +EOF
>>> +        ) | sed s/NAME/$2/g | sed s/VERSION/$3/g > $2.control
>>> +        equivs-build -a $1 $2.control 1>/dev/null 2>/dev/null
>>> +        dpkg -i --force-overwrite $2_$3_$1.deb
>>> +    )
>>> +}
>>> +
>>> +try_install()
>>> +{
>>> +    name=$(echo $1|sed "s/\(.*\):\(.*\)=\(.*\)/\1/")
>>> +    arch=$(echo $1|sed "s/\(.*\):\(.*\)=\(.*\)/\2/")
>>> +    vers=$(echo $1|sed "s/\(.*\):\(.*\)=\(.*\)/\3/")
>>> +    apt-get install -q -yy $1 || fake_install $arch $name $vers
>>> +}
>>> +
>>> +for package in $*; do
>>> +    try_install $package
>>> +done
>>> diff --git a/tests/docker/dockerfiles/debian-powerpc-cross.docker b/tests/docker/dockerfiles/debian-powerpc-cross.docker
>>> index 5e62ca0df1..4ccd17d9d5 100644
>>> --- a/tests/docker/dockerfiles/debian-powerpc-cross.docker
>>> +++ b/tests/docker/dockerfiles/debian-powerpc-cross.docker
>>> @@ -11,3 +11,46 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>>>      apt-get install -y --no-install-recommends \
>>>          gcc-powerpc-linux-gnu \
>>>          libc6-dev-powerpc-cross || { echo "Failed to build - see debian-sid.docker notes"; exit 1; }
>>> +
>>> +# Setup some basic tools we need
>>> +RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>>> +    apt-get install -y --no-install-recommends \
>>> +        bison \
>>> +        binutils-multiarch \
>>> +        build-essential \
>>> +        ca-certificates \
>>> +        curl \
>>> +        flex \
>>> +        gettext \
>>> +        git \
>>> +        gnupg \
>>> +        pkg-config \
>>> +        python-minimal
>>> +
>>> +RUN dpkg --add-architecture powerpc && \
>>> +    apt-get update
>>> +
>>> +ENV PKG_CONFIG_PATH /usr/lib/powerpc-linux-gnu/pkgconfig
>>> +
>>> +# Specify the cross prefix for this image (see tests/docker/common.rc)
>>> +ENV QEMU_CONFIGURE_OPTS --cross-prefix=powerpc-linux-gnu-
>>> +
>>> +# <kludge> to fix "following packages have unmet dependencies" ...
>>> +ADD debian-apt-fake.sh /usr/local/bin/apt-fake
>>> +RUN apt-get install -y --no-install-recommends \
>>> +        equivs
>>> +RUN apt-fake install \
>>> +        glusterfs-common:powerpc=4.0.2-fake
>>> +# </kludge>
>>> +
>>> +RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>>> +    apt-get build-dep -yy -a powerpc qemu
>>> +RUN DEBIAN_FRONTEND=noninteractive \
>>> +    apt-get install -y --no-install-recommends \
>>> +        glusterfs-common:powerpc \
>>> +        libbz2-dev:powerpc \
>>> +        liblzo2-dev:powerpc \
>>> +        libncursesw5-dev:powerpc \
>>> +        libnfs-dev:powerpc \
>>> +        librdmacm-dev:powerpc \
>>> +        libsnappy-dev:powerpc
>>
>>
>> --
>> Alex Bennée
>>


--
Alex Bennée