Re: [Qemu-devel] [PATCH 2/2] scripts/coverity-scan: Add Docker support

["Philippe Mathieu-Daudé" <philmd@redhat.com>]

On 14/11/18 12:25, Alex Bennée wrote:
> Philippe Mathieu-Daudé <philmd@redhat.com> writes:
>> On 13/11/18 19:46, Peter Maydell wrote:
>>> Add support for running the Coverity Scan tools inside a Docker
>>> container rather than directly on the host system.
>>>
>>> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
>>> ---
>>>    scripts/coverity-scan/coverity-scan.docker | 120 +++++++++++++++++++++
>>>    scripts/coverity-scan/run-coverity-scan    |  58 ++++++++++
>>>    2 files changed, 178 insertions(+)
>>>    create mode 100644 scripts/coverity-scan/coverity-scan.docker
>>>
>>> diff --git a/scripts/coverity-scan/coverity-scan.docker b/scripts/coverity-scan/coverity-scan.docker
>>> new file mode 100644
>>> index 00000000000..81f69459954
>>> --- /dev/null
>>> +++ b/scripts/coverity-scan/coverity-scan.docker
>>> @@ -0,0 +1,120 @@
>>> +# syntax=docker/dockerfile:1.0.0-experimental
>>> +#
>>> +# Docker setup for running the "Coverity Scan" tools over the source
>>> +# tree and uploading them to the website, as per
>>> +# https://scan.coverity.com/projects/qemu/builds/new
>>> +# We do this on a fixed config (currently Fedora 28 with a known
>>> +# set of dependencies and a configure command that enables a specific
>>> +# set of options) so that random changes don't result in our accidentally
>>> +# dropping some files from the scan.
>>> +# The work of actually doing the build is handled by the
>>> +# run-coverity-scan script.
>>> +
>>> +
>>> +FROM fedora:28
>>> +ENV PACKAGES \
>>> +    alsa-lib-devel \
>>> +    bc \
>>> +    bison \
>>> +    bluez-libs-devel \
>>> +    brlapi-devel \
>>> +    bzip2 \
>>> +    bzip2-devel \
>>> +    ccache \
>>> +    clang \
>>> +    curl \
>>> +    cyrus-sasl-devel \
>>> +    device-mapper-multipath-devel \
>>> +    findutils \
>>> +    flex \
>>> +    gcc \
>>> +    gcc-c++ \
>>> +    gettext \
>>> +    git \
>>> +    glib2-devel \
>>> +    glusterfs-api-devel \
>>> +    gnutls-devel \
>>> +    gtk3-devel \
>>> +    hostname \
>>> +    libaio-devel \
>>> +    libasan \
>>> +    libattr-devel \
>>> +    libcap-devel \
>>> +    libcap-ng-devel \
>>> +    libcurl-devel \
>>> +    libepoxy-devel \
>>> +    libfdt-devel \
>>> +    libgbm-devel \
>>> +    libiscsi-devel \
>>> +    libjpeg-devel \
>>> +    libnfs-devel \
>>> +    libpng-devel \
>>> +    librbd-devel \
>>> +    libseccomp-devel \
>>> +    libssh2-devel \
>>> +    libubsan \
>>> +    libudev-devel \
>>> +    libusbx-devel \
>>> +    libxml2-devel \
>>> +    llvm \
>>> +    lzo-devel \
>>> +    make \
>>> +    mingw32-bzip2 \
>>> +    mingw32-curl \
>>> +    mingw32-glib2 \
>>> +    mingw32-gmp \
>>> +    mingw32-gnutls \
>>> +    mingw32-gtk3 \
>>> +    mingw32-libjpeg-turbo \
>>> +    mingw32-libpng \
>>> +    mingw32-libssh2 \
>>> +    mingw32-libtasn1 \
>>> +    mingw32-nettle \
>>> +    mingw32-pixman \
>>> +    mingw32-pkg-config \
>>> +    mingw32-SDL2 \
>>> +    mingw64-bzip2 \
>>> +    mingw64-curl \
>>> +    mingw64-glib2 \
>>> +    mingw64-gmp \
>>> +    mingw64-gnutls \
>>> +    mingw64-gtk3 \
>>> +    mingw64-libjpeg-turbo \
>>> +    mingw64-libpng \
>>> +    mingw64-libssh2 \
>>> +    mingw64-libtasn1 \
>>> +    mingw64-nettle \
>>> +    mingw64-pixman \
>>> +    mingw64-pkg-config \
>>> +    mingw64-SDL2 \
>>> +    ncurses-devel \
>>> +    nettle-devel \
>>> +    nss-devel \
>>> +    numactl-devel \
>>> +    perl \
>>> +    pixman-devel \
>>> +    pulseaudio-libs-devel \
>>> +    python3 \
>>> +    PyYAML \
>>> +    rdma-core-devel \
>>> +    SDL2-devel \
>>> +    snappy-devel \
>>> +    sparse \
>>> +    spice-server-devel \
>>> +    systemtap-sdt-devel \
>>> +    tar \
>>> +    usbredir-devel \
>>> +    virglrenderer-devel \
>>> +    vte3-devel \
>>> +    wget \
>>> +    which \
>>> +    xen-devel \
>>> +    xfsprogs-devel \
>>> +    zlib-devel
>>> +ENV QEMU_CONFIGURE_OPTS --python=/usr/bin/python3
>>> +
>>> +RUN dnf install -y $PACKAGES
>>> +RUN rpm -q $PACKAGES | sort > /packages.txt
>>> +ENV COVERITY_TOOL_BASE=/coverity-tools
>>> +COPY run-coverity-scan run-coverity-scan
>>> +RUN --mount=type=secret,id=coverity.token,required ./run-coverity-scan --update-tools-only --tokenfile /run/secrets/coverity.token
>>
>> Calling "make docket-image-fedora" you can reduce this script to:
> 
> Remember for this to work we need to enforce the dependencies in the
> tests/docker/Makefile.include and integrate into our make machinery.
> Currently this dockerfile lives outside of the rest of our make
> machinery.

Yes, but since this image is ran via a script which calls "docker build 
..." it could previously call "make docket-image-fedora".

Currenty the qemu:fedora layer takes a bit more than 2GB, space worth on 
laptop SSD ;)

> 
> We've talked about having Docker environments for building test pieces
> before so I wonder if this is a good fit for expanding the make system
> support for these sort of jobs?

I am not sure which of the various Docker talk you are thinking of...

For this particular case this is probably not worth integrating it into 
the make system.

However it makes sense to me to have the qemu:fedora and this image 
pushed. Probably worth another thread although.

> 
>>
>> -- >8 --
>> FROM qemu:fedora
>> ENV PACKAGES \
>>      $PACKAGES \
>>      alsa-lib-devel \
>>      curl \
>>      cyrus-sasl-devel \
>>      libepoxy-devel \
>>      libgbm-devel \
>>      libiscsi-devel \
>>      libnfs-devel \
>>      libseccomp-devel \
>>      libudev-devel \
>>      pulseaudio-libs-devel \
>>      rdma-core-devel \
>>      wget \
>>      xfsprogs-devel
>>
>> RUN dnf install -y $PACKAGES
>> RUN rpm -q $PACKAGES | sort > /packages.txt
>> ENV COVERITY_TOOL_BASE=/coverity-tools
>> COPY run-coverity-scan run-coverity-scan
>> RUN --mount=type=secret,id=coverity.token,required ./run-coverity-scan
>> --update-tools-only --tokenfile /run/secrets/coverity.token
>> ---
>>
>> sharing a big docker layer.
> 
> 
> --
> Alex Bennée
>