[Qemu-devel] [PATCH v2] Add a gitlab-ci file for Continuous Integration testing on Gitlab

[Qemu-devel] [PATCH v2] Add a gitlab-ci file for Continuous Integration testing on Gitlab

[Thomas Huth]

This is very convenient for people like me who store their QEMU git trees
on gitlab.com: Automatic CI pipelines are now run for each branch that is
pushed to the server - useful for some extra-testing before sending PULL-
requests for example. Since the runtime of the jobs is limited to 1h, the
jobs are distributed into multiple pipelines - this way everything finishs
fine within time (ca. 30 minutes currently).

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 An example can be seen here: https://gitlab.com/huth/qemu/pipelines/

 I'd really like to get this into the main QEMU repository, so that I don't
 have to cherry-pick this patch onto my testing branches anymore each time
 I want to test before sending a PULL request...

 .gitlab-ci.yml | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 MAINTAINERS    |  5 ++++
 2 files changed, 78 insertions(+)
 create mode 100644 .gitlab-ci.yml

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
new file mode 100644
index 0000000..79d02cf
--- /dev/null
+++ b/.gitlab-ci.yml
@@ -0,0 +1,73 @@
+before_script:
+ - apt-get update -qq
+ - apt-get install -y -qq flex bison libglib2.0-dev libpixman-1-dev genisoimage
+
+build-system1:
+ script:
+ - apt-get install -y -qq libgtk-3-dev libvte-dev nettle-dev libcacard-dev
+      libusb-dev libvde-dev libspice-protocol-dev libgl1-mesa-dev
+ - ./configure --enable-werror --target-list="aarch64-softmmu alpha-softmmu
+      cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu microblazeel-softmmu
+      mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu sparc-softmmu"
+ - make -j2
+ - make -j2 check
+
+build-system2:
+ script:
+ - apt-get install -y -qq libsdl2-dev libgcrypt-dev libbrlapi-dev libaio-dev
+      libfdt-dev liblzo2-dev librdmacm-dev libibverbs-dev libibumad-dev
+ - ./configure --enable-werror --target-list="tricore-softmmu unicore32-softmmu
+      microblaze-softmmu mips-softmmu riscv32-softmmu s390x-softmmu sh4-softmmu
+      sparc64-softmmu x86_64-softmmu xtensa-softmmu nios2-softmmu or1k-softmmu"
+ - make -j2
+ - make -j2 check
+
+build-disabled:
+ script:
+ - ./configure --enable-werror --disable-rdma --disable-slirp --disable-curl
+      --disable-capstone --disable-live-block-migration --disable-glusterfs
+      --disable-replication --disable-coroutine-pool --disable-smartcard
+      --disable-guest-agent --disable-curses --disable-libxml2 --disable-tpm
+      --disable-qom-cast-debug --disable-spice --disable-vhost-vsock
+      --disable-vhost-net --disable-vhost-crypto --disable-vhost-user
+      --target-list="i386-softmmu ppc64-softmmu mips64-softmmu i386-linux-user"
+ - make -j2
+ - make -j2 check-qtest SPEED=slow
+
+build-tcg-disabled:
+ script:
+ - apt-get install -y -qq clang libgtk-3-dev libbluetooth-dev libusb-dev
+ - ./configure --cc=clang --enable-werror --disable-tcg --audio-drv-list=""
+ - make -j2
+ - make check-unit
+ - make check-qapi-schema
+ - cd tests/qemu-iotests/
+ - ./check -raw 001 002 003 004 005 008 009 010 011 012 021 025 032 033 048
+            052 063 077 086 101 104 106 113 147 148 150 151 152 157 159 160
+            163 170 171 183 184 192 194 197 205 208 215 221 222 226 227 236
+ - ./check -qcow2 001 002 003 004 005 007 008 009 010 011 012 013 017 018 019
+            020 021 022 024 025 027 028 029 031 032 033 034 035 036 037 038
+            039 040 042 043 046 047 048 049 050 051 052 053 054 056 057 058
+            060 061 062 063 065 066 067 068 069 071 072 073 074 079 080 082
+            085 086 089 090 091 095 096 097 098 099 102 103 104 105 107 108
+            110 111 114 117 120 122 124 126 127 129 130 132 133 134 137 138
+            139 140 141 142 143 144 145 147 150 151 152 154 155 156 157 158
+            161 165 170 172 174 176 177 179 184 186 187 190 192 194 195 196
+            197 200 202 203 205 208 209 214 215 216 217 218 222 226 227 229 234
+
+build-user:
+ script:
+ - ./configure --enable-werror --disable-system --disable-guest-agent
+               --disable-capstone --disable-slirp --disable-fdt
+ - make -j2
+ - make run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-linux-user
+
+build-clang:
+ script:
+ - apt-get install -y -qq clang libsdl2-dev
+      xfslibs-dev libiscsi-dev libnfs-dev libseccomp-dev gnutls-dev librbd-dev
+ - ./configure --cc=clang --cxx=clang++ --enable-werror
+      --target-list="alpha-softmmu arm-softmmu m68k-softmmu mips64-softmmu
+                     ppc-softmmu s390x-softmmu x86_64-softmmu arm-linux-user"
+ - make -j2
+ - make -j2 check
diff --git a/MAINTAINERS b/MAINTAINERS
index a2da141..191138c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2484,6 +2484,11 @@ S: Maintained
 F: .cirrus.yml
 W: https://cirrus-ci.com/github/qemu/qemu
 
+GitLab Continuous Integration
+M: Thomas Huth <thuth@redhat.com>
+S: Maintained
+F: .gitlab-ci.yml
+
 Guest Test Compilation Support
 M: Alex Bennée <alex.bennee@linaro.org>
 R: Philippe Mathieu-Daudé <f4bug@amsat.org>
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH v2] Add a gitlab-ci file for Continuous Integration testing on Gitlab

[Marc-André Lureau]

Hi

On Wed, Feb 13, 2019 at 12:55 PM Thomas Huth <thuth@redhat.com> wrote:
>
> This is very convenient for people like me who store their QEMU git trees
> on gitlab.com: Automatic CI pipelines are now run for each branch that is
> pushed to the server - useful for some extra-testing before sending PULL-
> requests for example. Since the runtime of the jobs is limited to 1h, the
> jobs are distributed into multiple pipelines - this way everything finishs
> fine within time (ca. 30 minutes currently).
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  An example can be seen here: https://gitlab.com/huth/qemu/pipelines/
>
>  I'd really like to get this into the main QEMU repository, so that I don't
>  have to cherry-pick this patch onto my testing branches anymore each time
>  I want to test before sending a PULL request...
>
>  .gitlab-ci.yml | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  MAINTAINERS    |  5 ++++
>  2 files changed, 78 insertions(+)
>  create mode 100644 .gitlab-ci.yml
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> new file mode 100644
> index 0000000..79d02cf
> --- /dev/null
> +++ b/.gitlab-ci.yml
> @@ -0,0 +1,73 @@
> +before_script:
> + - apt-get update -qq
> + - apt-get install -y -qq flex bison libglib2.0-dev libpixman-1-dev genisoimage
> +
> +build-system1:
> + script:
> + - apt-get install -y -qq libgtk-3-dev libvte-dev nettle-dev libcacard-dev
> +      libusb-dev libvde-dev libspice-protocol-dev libgl1-mesa-dev
> + - ./configure --enable-werror --target-list="aarch64-softmmu alpha-softmmu
> +      cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu microblazeel-softmmu
> +      mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu sparc-softmmu"
> + - make -j2

Have you tried $(nproc) ?

> + - make -j2 check
> +
> +build-system2:
> + script:
> + - apt-get install -y -qq libsdl2-dev libgcrypt-dev libbrlapi-dev libaio-dev
> +      libfdt-dev liblzo2-dev librdmacm-dev libibverbs-dev libibumad-dev
> + - ./configure --enable-werror --target-list="tricore-softmmu unicore32-softmmu
> +      microblaze-softmmu mips-softmmu riscv32-softmmu s390x-softmmu sh4-softmmu
> +      sparc64-softmmu x86_64-softmmu xtensa-softmmu nios2-softmmu or1k-softmmu"
> + - make -j2
> + - make -j2 check
> +
> +build-disabled:
> + script:
> + - ./configure --enable-werror --disable-rdma --disable-slirp --disable-curl
> +      --disable-capstone --disable-live-block-migration --disable-glusterfs
> +      --disable-replication --disable-coroutine-pool --disable-smartcard
> +      --disable-guest-agent --disable-curses --disable-libxml2 --disable-tpm
> +      --disable-qom-cast-debug --disable-spice --disable-vhost-vsock
> +      --disable-vhost-net --disable-vhost-crypto --disable-vhost-user
> +      --target-list="i386-softmmu ppc64-softmmu mips64-softmmu i386-linux-user"
> + - make -j2
> + - make -j2 check-qtest SPEED=slow
> +
> +build-tcg-disabled:
> + script:
> + - apt-get install -y -qq clang libgtk-3-dev libbluetooth-dev libusb-dev
> + - ./configure --cc=clang --enable-werror --disable-tcg --audio-drv-list=""
> + - make -j2
> + - make check-unit
> + - make check-qapi-schema
> + - cd tests/qemu-iotests/
> + - ./check -raw 001 002 003 004 005 008 009 010 011 012 021 025 032 033 048
> +            052 063 077 086 101 104 106 113 147 148 150 151 152 157 159 160
> +            163 170 171 183 184 192 194 197 205 208 215 221 222 226 227 236
> + - ./check -qcow2 001 002 003 004 005 007 008 009 010 011 012 013 017 018 019
> +            020 021 022 024 025 027 028 029 031 032 033 034 035 036 037 038
> +            039 040 042 043 046 047 048 049 050 051 052 053 054 056 057 058
> +            060 061 062 063 065 066 067 068 069 071 072 073 074 079 080 082
> +            085 086 089 090 091 095 096 097 098 099 102 103 104 105 107 108
> +            110 111 114 117 120 122 124 126 127 129 130 132 133 134 137 138
> +            139 140 141 142 143 144 145 147 150 151 152 154 155 156 157 158
> +            161 165 170 172 174 176 177 179 184 186 187 190 192 194 195 196
> +            197 200 202 203 205 208 209 214 215 216 217 218 222 226 227 229 234
> +
> +build-user:
> + script:
> + - ./configure --enable-werror --disable-system --disable-guest-agent
> +               --disable-capstone --disable-slirp --disable-fdt
> + - make -j2
> + - make run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-linux-user
> +
> +build-clang:
> + script:
> + - apt-get install -y -qq clang libsdl2-dev
> +      xfslibs-dev libiscsi-dev libnfs-dev libseccomp-dev gnutls-dev librbd-dev
> + - ./configure --cc=clang --cxx=clang++ --enable-werror
> +      --target-list="alpha-softmmu arm-softmmu m68k-softmmu mips64-softmmu
> +                     ppc-softmmu s390x-softmmu x86_64-softmmu arm-linux-user"
> + - make -j2
> + - make -j2 check
> diff --git a/MAINTAINERS b/MAINTAINERS
> index a2da141..191138c 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -2484,6 +2484,11 @@ S: Maintained
>  F: .cirrus.yml
>  W: https://cirrus-ci.com/github/qemu/qemu
>
> +GitLab Continuous Integration
> +M: Thomas Huth <thuth@redhat.com>
> +S: Maintained
> +F: .gitlab-ci.yml
> +
>  Guest Test Compilation Support
>  M: Alex Bennée <alex.bennee@linaro.org>
>  R: Philippe Mathieu-Daudé <f4bug@amsat.org>
> --
> 1.8.3.1
>
>

Despite the duplication of CI files, which is not really a problem imho,

lgtm,
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>



-- 
Marc-André Lureau

Re: [Qemu-devel] [PATCH v2] Add a gitlab-ci file for Continuous Integration testing on Gitlab

[Thomas Huth]

On 2019-02-13 13:06, Marc-André Lureau wrote:
> Hi
> 
> On Wed, Feb 13, 2019 at 12:55 PM Thomas Huth <thuth@redhat.com> wrote:
>>
>> This is very convenient for people like me who store their QEMU git trees
>> on gitlab.com: Automatic CI pipelines are now run for each branch that is
>> pushed to the server - useful for some extra-testing before sending PULL-
>> requests for example. Since the runtime of the jobs is limited to 1h, the
>> jobs are distributed into multiple pipelines - this way everything finishs
>> fine within time (ca. 30 minutes currently).
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>>  An example can be seen here: https://gitlab.com/huth/qemu/pipelines/
>>
>>  I'd really like to get this into the main QEMU repository, so that I don't
>>  have to cherry-pick this patch onto my testing branches anymore each time
>>  I want to test before sending a PULL request...
>>
>>  .gitlab-ci.yml | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>  MAINTAINERS    |  5 ++++
>>  2 files changed, 78 insertions(+)
>>  create mode 100644 .gitlab-ci.yml
>>
>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>> new file mode 100644
>> index 0000000..79d02cf
>> --- /dev/null
>> +++ b/.gitlab-ci.yml
>> @@ -0,0 +1,73 @@
>> +before_script:
>> + - apt-get update -qq
>> + - apt-get install -y -qq flex bison libglib2.0-dev libpixman-1-dev genisoimage
>> +
>> +build-system1:
>> + script:
>> + - apt-get install -y -qq libgtk-3-dev libvte-dev nettle-dev libcacard-dev
>> +      libusb-dev libvde-dev libspice-protocol-dev libgl1-mesa-dev
>> + - ./configure --enable-werror --target-list="aarch64-softmmu alpha-softmmu
>> +      cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu microblazeel-softmmu
>> +      mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu sparc-softmmu"
>> + - make -j2
> 
> Have you tried $(nproc) ?

The containers are only single CPU there. I'm using -j2 just in case one
of the processes is blocked by waiting for IO, the other one can still
continue compiling.

> Despite the duplication of CI files, which is not really a problem imho,
> 
> lgtm,
> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>

Thanks!

And yes, I also think the duplication of the CI files should not really
be a problem - it's rather a plus since we also got test variations this
way. I already found some bugs with this here that were not detected by
the other CI setups yet (e.g. since one of the pipelines uses
--disable-replication and other --disable-xxx flags).

 Thomas

Re: [Qemu-devel] [PATCH v2] Add a gitlab-ci file for Continuous Integration testing on Gitlab

[Alex Bennée]

Thomas Huth <thuth@redhat.com> writes:

> On 2019-02-13 13:06, Marc-André Lureau wrote:
>> Hi
>>
>> On Wed, Feb 13, 2019 at 12:55 PM Thomas Huth <thuth@redhat.com> wrote:
>>>
>>> This is very convenient for people like me who store their QEMU git trees
>>> on gitlab.com: Automatic CI pipelines are now run for each branch that is
>>> pushed to the server - useful for some extra-testing before sending PULL-
>>> requests for example. Since the runtime of the jobs is limited to 1h, the
>>> jobs are distributed into multiple pipelines - this way everything finishs
>>> fine within time (ca. 30 minutes currently).
>>>
>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>> ---
>>>  An example can be seen here: https://gitlab.com/huth/qemu/pipelines/
>>>
>>>  I'd really like to get this into the main QEMU repository, so that I don't
>>>  have to cherry-pick this patch onto my testing branches anymore each time
>>>  I want to test before sending a PULL request...
>>>
>>>  .gitlab-ci.yml | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>  MAINTAINERS    |  5 ++++
>>>  2 files changed, 78 insertions(+)
>>>  create mode 100644 .gitlab-ci.yml
>>>
>>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>>> new file mode 100644
>>> index 0000000..79d02cf
>>> --- /dev/null
>>> +++ b/.gitlab-ci.yml
>>> @@ -0,0 +1,73 @@
>>> +before_script:
>>> + - apt-get update -qq
>>> + - apt-get install -y -qq flex bison libglib2.0-dev libpixman-1-dev genisoimage
>>> +
>>> +build-system1:
>>> + script:
>>> + - apt-get install -y -qq libgtk-3-dev libvte-dev nettle-dev libcacard-dev
>>> +      libusb-dev libvde-dev libspice-protocol-dev libgl1-mesa-dev
>>> + - ./configure --enable-werror --target-list="aarch64-softmmu alpha-softmmu
>>> +      cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu microblazeel-softmmu
>>> +      mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu sparc-softmmu"
>>> + - make -j2
>>
>> Have you tried $(nproc) ?
>
> The containers are only single CPU there. I'm using -j2 just in case one
> of the processes is blocked by waiting for IO, the other one can still
> continue compiling.
>
>> Despite the duplication of CI files, which is not really a problem imho,
>>
>> lgtm,
>> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>
> Thanks!
>
> And yes, I also think the duplication of the CI files should not really
> be a problem - it's rather a plus since we also got test variations this
> way. I already found some bugs with this here that were not detected by
> the other CI setups yet (e.g. since one of the pipelines uses
> --disable-replication and other --disable-xxx flags).

Sure I'm happy to have a thousand flowers bloom in this regard.

Do you know if gitlab does build status buttons like the others?

--
Alex Bennée

Re: [Qemu-devel] [PATCH v2] Add a gitlab-ci file for Continuous Integration testing on Gitlab

[Marc-André Lureau]

On Wed, Feb 13, 2019 at 3:03 PM Alex Bennée <alex.bennee@linaro.org> wrote:
>
>
> Thomas Huth <thuth@redhat.com> writes:
>
> > On 2019-02-13 13:06, Marc-André Lureau wrote:
> >> Hi
> >>
> >> On Wed, Feb 13, 2019 at 12:55 PM Thomas Huth <thuth@redhat.com> wrote:
> >>>
> >>> This is very convenient for people like me who store their QEMU git trees
> >>> on gitlab.com: Automatic CI pipelines are now run for each branch that is
> >>> pushed to the server - useful for some extra-testing before sending PULL-
> >>> requests for example. Since the runtime of the jobs is limited to 1h, the
> >>> jobs are distributed into multiple pipelines - this way everything finishs
> >>> fine within time (ca. 30 minutes currently).
> >>>
> >>> Signed-off-by: Thomas Huth <thuth@redhat.com>
> >>> ---
> >>>  An example can be seen here: https://gitlab.com/huth/qemu/pipelines/
> >>>
> >>>  I'd really like to get this into the main QEMU repository, so that I don't
> >>>  have to cherry-pick this patch onto my testing branches anymore each time
> >>>  I want to test before sending a PULL request...
> >>>
> >>>  .gitlab-ci.yml | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>>  MAINTAINERS    |  5 ++++
> >>>  2 files changed, 78 insertions(+)
> >>>  create mode 100644 .gitlab-ci.yml
> >>>
> >>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> >>> new file mode 100644
> >>> index 0000000..79d02cf
> >>> --- /dev/null
> >>> +++ b/.gitlab-ci.yml
> >>> @@ -0,0 +1,73 @@
> >>> +before_script:
> >>> + - apt-get update -qq
> >>> + - apt-get install -y -qq flex bison libglib2.0-dev libpixman-1-dev genisoimage
> >>> +
> >>> +build-system1:
> >>> + script:
> >>> + - apt-get install -y -qq libgtk-3-dev libvte-dev nettle-dev libcacard-dev
> >>> +      libusb-dev libvde-dev libspice-protocol-dev libgl1-mesa-dev
> >>> + - ./configure --enable-werror --target-list="aarch64-softmmu alpha-softmmu
> >>> +      cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu microblazeel-softmmu
> >>> +      mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu sparc-softmmu"
> >>> + - make -j2
> >>
> >> Have you tried $(nproc) ?
> >
> > The containers are only single CPU there. I'm using -j2 just in case one
> > of the processes is blocked by waiting for IO, the other one can still
> > continue compiling.
> >
> >> Despite the duplication of CI files, which is not really a problem imho,
> >>
> >> lgtm,
> >> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> >
> > Thanks!
> >
> > And yes, I also think the duplication of the CI files should not really
> > be a problem - it's rather a plus since we also got test variations this
> > way. I already found some bugs with this here that were not detected by
> > the other CI setups yet (e.g. since one of the pipelines uses
> > --disable-replication and other --disable-xxx flags).
>
> Sure I'm happy to have a thousand flowers bloom in this regard.
>
> Do you know if gitlab does build status buttons like the others?

I think that's called badges:
https://docs.gitlab.com/ee/user/project/pipelines/settings.html#pipeline-badges




-- 
Marc-André Lureau

Re: [Qemu-devel] [PATCH v2] Add a gitlab-ci file for Continuous Integration testing on Gitlab

[Cleber Rosa]

On 2/13/19 6:54 AM, Thomas Huth wrote:
> This is very convenient for people like me who store their QEMU git trees
> on gitlab.com: Automatic CI pipelines are now run for each branch that is
> pushed to the server - useful for some extra-testing before sending PULL-
> requests for example. Since the runtime of the jobs is limited to 1h, the
> jobs are distributed into multiple pipelines - this way everything finishs
> fine within time (ca. 30 minutes currently).
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  An example can be seen here: https://gitlab.com/huth/qemu/pipelines/
> 
>  I'd really like to get this into the main QEMU repository, so that I don't
>  have to cherry-pick this patch onto my testing branches anymore each time
>  I want to test before sending a PULL request...
> 
>  .gitlab-ci.yml | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  MAINTAINERS    |  5 ++++
>  2 files changed, 78 insertions(+)
>  create mode 100644 .gitlab-ci.yml
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> new file mode 100644
> index 0000000..79d02cf
> --- /dev/null
> +++ b/.gitlab-ci.yml
> @@ -0,0 +1,73 @@
> +before_script:
> + - apt-get update -qq
> + - apt-get install -y -qq flex bison libglib2.0-dev libpixman-1-dev genisoimage
> +
> +build-system1:
> + script:
> + - apt-get install -y -qq libgtk-3-dev libvte-dev nettle-dev libcacard-dev
> +      libusb-dev libvde-dev libspice-protocol-dev libgl1-mesa-dev
> + - ./configure --enable-werror --target-list="aarch64-softmmu alpha-softmmu
> +      cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu microblazeel-softmmu
> +      mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu sparc-softmmu"
> + - make -j2
> + - make -j2 check
> +
> +build-system2:
> + script:
> + - apt-get install -y -qq libsdl2-dev libgcrypt-dev libbrlapi-dev libaio-dev
> +      libfdt-dev liblzo2-dev librdmacm-dev libibverbs-dev libibumad-dev
> + - ./configure --enable-werror --target-list="tricore-softmmu unicore32-softmmu
> +      microblaze-softmmu mips-softmmu riscv32-softmmu s390x-softmmu sh4-softmmu
> +      sparc64-softmmu x86_64-softmmu xtensa-softmmu nios2-softmmu or1k-softmmu"
> + - make -j2
> + - make -j2 check
> +
> +build-disabled:
> + script:
> + - ./configure --enable-werror --disable-rdma --disable-slirp --disable-curl
> +      --disable-capstone --disable-live-block-migration --disable-glusterfs
> +      --disable-replication --disable-coroutine-pool --disable-smartcard
> +      --disable-guest-agent --disable-curses --disable-libxml2 --disable-tpm
> +      --disable-qom-cast-debug --disable-spice --disable-vhost-vsock
> +      --disable-vhost-net --disable-vhost-crypto --disable-vhost-user
> +      --target-list="i386-softmmu ppc64-softmmu mips64-softmmu i386-linux-user"
> + - make -j2
> + - make -j2 check-qtest SPEED=slow
> +
> +build-tcg-disabled:
> + script:
> + - apt-get install -y -qq clang libgtk-3-dev libbluetooth-dev libusb-dev
> + - ./configure --cc=clang --enable-werror --disable-tcg --audio-drv-list=""
> + - make -j2
> + - make check-unit
> + - make check-qapi-schema
> + - cd tests/qemu-iotests/
> + - ./check -raw 001 002 003 004 005 008 009 010 011 012 021 025 032 033 048
> +            052 063 077 086 101 104 106 113 147 148 150 151 152 157 159 160
> +            163 170 171 183 184 192 194 197 205 208 215 221 222 226 227 236
> + - ./check -qcow2 001 002 003 004 005 007 008 009 010 011 012 013 017 018 019
> +            020 021 022 024 025 027 028 029 031 032 033 034 035 036 037 038
> +            039 040 042 043 046 047 048 049 050 051 052 053 054 056 057 058
> +            060 061 062 063 065 066 067 068 069 071 072 073 074 079 080 082
> +            085 086 089 090 091 095 096 097 098 099 102 103 104 105 107 108
> +            110 111 114 117 120 122 124 126 127 129 130 132 133 134 137 138
> +            139 140 141 142 143 144 145 147 150 151 152 154 155 156 157 158
> +            161 165 170 172 174 176 177 179 184 186 187 190 192 194 195 196
> +            197 200 202 203 205 208 209 214 215 216 217 218 222 226 227 229 234
> +

On the matter of "make check-block" or "./check xxx", I've seen
arguments for both sides.  I've sent a similar (but using make
check-block) patch for Travis:

https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg02334.html

I'd be nice to hear from the qemu-iotest's maintainers which approach
is preferable for upstream + shared computing resources kind of environment.

> +build-user:
> + script:
> + - ./configure --enable-werror --disable-system --disable-guest-agent
> +               --disable-capstone --disable-slirp --disable-fdt
> + - make -j2
> + - make run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-linux-user
> +
> +build-clang:
> + script:
> + - apt-get install -y -qq clang libsdl2-dev
> +      xfslibs-dev libiscsi-dev libnfs-dev libseccomp-dev gnutls-dev librbd-dev
> + - ./configure --cc=clang --cxx=clang++ --enable-werror
> +      --target-list="alpha-softmmu arm-softmmu m68k-softmmu mips64-softmmu
> +                     ppc-softmmu s390x-softmmu x86_64-softmmu arm-linux-user"
> + - make -j2
> + - make -j2 check
> diff --git a/MAINTAINERS b/MAINTAINERS
> index a2da141..191138c 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -2484,6 +2484,11 @@ S: Maintained
>  F: .cirrus.yml
>  W: https://cirrus-ci.com/github/qemu/qemu
>  
> +GitLab Continuous Integration
> +M: Thomas Huth <thuth@redhat.com>
> +S: Maintained
> +F: .gitlab-ci.yml
> +
>  Guest Test Compilation Support
>  M: Alex Bennée <alex.bennee@linaro.org>
>  R: Philippe Mathieu-Daudé <f4bug@amsat.org>
> 

Even though I'm not a GitLab user (yet?), I've gone through the file,
and job results, and they LGTM.

Reviewed-by: Cleber Rosa <crosa@redhat.com>

Re: [Qemu-devel] [PATCH v2] Add a gitlab-ci file for Continuous Integration testing on Gitlab

[Thomas Huth]

On 18/02/2019 19.22, Cleber Rosa wrote:
> 
> 
> On 2/13/19 6:54 AM, Thomas Huth wrote:
>> This is very convenient for people like me who store their QEMU git trees
>> on gitlab.com: Automatic CI pipelines are now run for each branch that is
>> pushed to the server - useful for some extra-testing before sending PULL-
>> requests for example. Since the runtime of the jobs is limited to 1h, the
>> jobs are distributed into multiple pipelines - this way everything finishs
>> fine within time (ca. 30 minutes currently).
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>>  An example can be seen here: https://gitlab.com/huth/qemu/pipelines/
>>
>>  I'd really like to get this into the main QEMU repository, so that I don't
>>  have to cherry-pick this patch onto my testing branches anymore each time
>>  I want to test before sending a PULL request...
>>
>>  .gitlab-ci.yml | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>  MAINTAINERS    |  5 ++++
>>  2 files changed, 78 insertions(+)
>>  create mode 100644 .gitlab-ci.yml
>>
>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>> new file mode 100644
>> index 0000000..79d02cf
>> --- /dev/null
>> +++ b/.gitlab-ci.yml
>> @@ -0,0 +1,73 @@
>> +before_script:
>> + - apt-get update -qq
>> + - apt-get install -y -qq flex bison libglib2.0-dev libpixman-1-dev genisoimage
>> +
>> +build-system1:
>> + script:
>> + - apt-get install -y -qq libgtk-3-dev libvte-dev nettle-dev libcacard-dev
>> +      libusb-dev libvde-dev libspice-protocol-dev libgl1-mesa-dev
>> + - ./configure --enable-werror --target-list="aarch64-softmmu alpha-softmmu
>> +      cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu microblazeel-softmmu
>> +      mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu sparc-softmmu"
>> + - make -j2
>> + - make -j2 check
>> +
>> +build-system2:
>> + script:
>> + - apt-get install -y -qq libsdl2-dev libgcrypt-dev libbrlapi-dev libaio-dev
>> +      libfdt-dev liblzo2-dev librdmacm-dev libibverbs-dev libibumad-dev
>> + - ./configure --enable-werror --target-list="tricore-softmmu unicore32-softmmu
>> +      microblaze-softmmu mips-softmmu riscv32-softmmu s390x-softmmu sh4-softmmu
>> +      sparc64-softmmu x86_64-softmmu xtensa-softmmu nios2-softmmu or1k-softmmu"
>> + - make -j2
>> + - make -j2 check
>> +
>> +build-disabled:
>> + script:
>> + - ./configure --enable-werror --disable-rdma --disable-slirp --disable-curl
>> +      --disable-capstone --disable-live-block-migration --disable-glusterfs
>> +      --disable-replication --disable-coroutine-pool --disable-smartcard
>> +      --disable-guest-agent --disable-curses --disable-libxml2 --disable-tpm
>> +      --disable-qom-cast-debug --disable-spice --disable-vhost-vsock
>> +      --disable-vhost-net --disable-vhost-crypto --disable-vhost-user
>> +      --target-list="i386-softmmu ppc64-softmmu mips64-softmmu i386-linux-user"
>> + - make -j2
>> + - make -j2 check-qtest SPEED=slow
>> +
>> +build-tcg-disabled:
>> + script:
>> + - apt-get install -y -qq clang libgtk-3-dev libbluetooth-dev libusb-dev
>> + - ./configure --cc=clang --enable-werror --disable-tcg --audio-drv-list=""
>> + - make -j2
>> + - make check-unit
>> + - make check-qapi-schema
>> + - cd tests/qemu-iotests/
>> + - ./check -raw 001 002 003 004 005 008 009 010 011 012 021 025 032 033 048
>> +            052 063 077 086 101 104 106 113 147 148 150 151 152 157 159 160
>> +            163 170 171 183 184 192 194 197 205 208 215 221 222 226 227 236
>> + - ./check -qcow2 001 002 003 004 005 007 008 009 010 011 012 013 017 018 019
>> +            020 021 022 024 025 027 028 029 031 032 033 034 035 036 037 038
>> +            039 040 042 043 046 047 048 049 050 051 052 053 054 056 057 058
>> +            060 061 062 063 065 066 067 068 069 071 072 073 074 079 080 082
>> +            085 086 089 090 091 095 096 097 098 099 102 103 104 105 107 108
>> +            110 111 114 117 120 122 124 126 127 129 130 132 133 134 137 138
>> +            139 140 141 142 143 144 145 147 150 151 152 154 155 156 157 158
>> +            161 165 170 172 174 176 177 179 184 186 187 190 192 194 195 196
>> +            197 200 202 203 205 208 209 214 215 216 217 218 222 226 227 229 234
>> +
> 
> On the matter of "make check-block" or "./check xxx", I've seen
> arguments for both sides.  I've sent a similar (but using make
> check-block) patch for Travis:
> 
> https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg02334.html
> 
> I'd be nice to hear from the qemu-iotest's maintainers which approach
> is preferable for upstream + shared computing resources kind of environment.

I think "make check-block" should likely be the preferred way to run the
tests for normal builds (and by the way, why don't we always run this
during "make check" yet?).

But in this CI file here, I'm running the tests for the "--disable-tcg"
build - simply because this test finishes way earlier than the other
tests otherwise. Unfortunately --disable-tcg causes some of the iotests
to fail - expected, since you also can not use KVM on the CI server, so
there is no way to execute guest code with this build (e.g. I'm also not
running the qtests in this pipe). Thus I had to limit the iotests to the
ones that also work without TCG, i.e. I have to run the check script
directly here with a list of tests that seem to be working in this
environment.

 Thomas

Re: [Qemu-devel] [Qemu-block] [PATCH v2] Add a gitlab-ci file for Continuous Integration testing on Gitlab

[Kevin Wolf]

Am 19.02.2019 um 07:44 hat Thomas Huth geschrieben:
> On 18/02/2019 19.22, Cleber Rosa wrote:
> > 
> > 
> > On 2/13/19 6:54 AM, Thomas Huth wrote:
> >> This is very convenient for people like me who store their QEMU git trees
> >> on gitlab.com: Automatic CI pipelines are now run for each branch that is
> >> pushed to the server - useful for some extra-testing before sending PULL-
> >> requests for example. Since the runtime of the jobs is limited to 1h, the
> >> jobs are distributed into multiple pipelines - this way everything finishs
> >> fine within time (ca. 30 minutes currently).
> >>
> >> Signed-off-by: Thomas Huth <thuth@redhat.com>
> >> ---
> >>  An example can be seen here: https://gitlab.com/huth/qemu/pipelines/
> >>
> >>  I'd really like to get this into the main QEMU repository, so that I don't
> >>  have to cherry-pick this patch onto my testing branches anymore each time
> >>  I want to test before sending a PULL request...
> >>
> >>  .gitlab-ci.yml | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>  MAINTAINERS    |  5 ++++
> >>  2 files changed, 78 insertions(+)
> >>  create mode 100644 .gitlab-ci.yml
> >>
> >> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> >> new file mode 100644
> >> index 0000000..79d02cf
> >> --- /dev/null
> >> +++ b/.gitlab-ci.yml
> >> @@ -0,0 +1,73 @@
> >> +before_script:
> >> + - apt-get update -qq
> >> + - apt-get install -y -qq flex bison libglib2.0-dev libpixman-1-dev genisoimage
> >> +
> >> +build-system1:
> >> + script:
> >> + - apt-get install -y -qq libgtk-3-dev libvte-dev nettle-dev libcacard-dev
> >> +      libusb-dev libvde-dev libspice-protocol-dev libgl1-mesa-dev
> >> + - ./configure --enable-werror --target-list="aarch64-softmmu alpha-softmmu
> >> +      cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu microblazeel-softmmu
> >> +      mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu sparc-softmmu"
> >> + - make -j2
> >> + - make -j2 check
> >> +
> >> +build-system2:
> >> + script:
> >> + - apt-get install -y -qq libsdl2-dev libgcrypt-dev libbrlapi-dev libaio-dev
> >> +      libfdt-dev liblzo2-dev librdmacm-dev libibverbs-dev libibumad-dev
> >> + - ./configure --enable-werror --target-list="tricore-softmmu unicore32-softmmu
> >> +      microblaze-softmmu mips-softmmu riscv32-softmmu s390x-softmmu sh4-softmmu
> >> +      sparc64-softmmu x86_64-softmmu xtensa-softmmu nios2-softmmu or1k-softmmu"
> >> + - make -j2
> >> + - make -j2 check
> >> +
> >> +build-disabled:
> >> + script:
> >> + - ./configure --enable-werror --disable-rdma --disable-slirp --disable-curl
> >> +      --disable-capstone --disable-live-block-migration --disable-glusterfs
> >> +      --disable-replication --disable-coroutine-pool --disable-smartcard
> >> +      --disable-guest-agent --disable-curses --disable-libxml2 --disable-tpm
> >> +      --disable-qom-cast-debug --disable-spice --disable-vhost-vsock
> >> +      --disable-vhost-net --disable-vhost-crypto --disable-vhost-user
> >> +      --target-list="i386-softmmu ppc64-softmmu mips64-softmmu i386-linux-user"
> >> + - make -j2
> >> + - make -j2 check-qtest SPEED=slow
> >> +
> >> +build-tcg-disabled:
> >> + script:
> >> + - apt-get install -y -qq clang libgtk-3-dev libbluetooth-dev libusb-dev
> >> + - ./configure --cc=clang --enable-werror --disable-tcg --audio-drv-list=""
> >> + - make -j2
> >> + - make check-unit
> >> + - make check-qapi-schema
> >> + - cd tests/qemu-iotests/
> >> + - ./check -raw 001 002 003 004 005 008 009 010 011 012 021 025 032 033 048
> >> +            052 063 077 086 101 104 106 113 147 148 150 151 152 157 159 160
> >> +            163 170 171 183 184 192 194 197 205 208 215 221 222 226 227 236
> >> + - ./check -qcow2 001 002 003 004 005 007 008 009 010 011 012 013 017 018 019
> >> +            020 021 022 024 025 027 028 029 031 032 033 034 035 036 037 038
> >> +            039 040 042 043 046 047 048 049 050 051 052 053 054 056 057 058
> >> +            060 061 062 063 065 066 067 068 069 071 072 073 074 079 080 082
> >> +            085 086 089 090 091 095 096 097 098 099 102 103 104 105 107 108
> >> +            110 111 114 117 120 122 124 126 127 129 130 132 133 134 137 138
> >> +            139 140 141 142 143 144 145 147 150 151 152 154 155 156 157 158
> >> +            161 165 170 172 174 176 177 179 184 186 187 190 192 194 195 196
> >> +            197 200 202 203 205 208 209 214 215 216 217 218 222 226 227 229 234
> >> +
> > 
> > On the matter of "make check-block" or "./check xxx", I've seen
> > arguments for both sides.  I've sent a similar (but using make
> > check-block) patch for Travis:
> > 
> > https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg02334.html
> > 
> > I'd be nice to hear from the qemu-iotest's maintainers which approach
> > is preferable for upstream + shared computing resources kind of environment.
> 
> I think "make check-block" should likely be the preferred way to run the
> tests for normal builds (and by the way, why don't we always run this
> during "make check" yet?).
> 
> But in this CI file here, I'm running the tests for the "--disable-tcg"
> build - simply because this test finishes way earlier than the other
> tests otherwise. Unfortunately --disable-tcg causes some of the iotests
> to fail - expected, since you also can not use KVM on the CI server, so
> there is no way to execute guest code with this build (e.g. I'm also not
> running the qtests in this pipe). Thus I had to limit the iotests to the
> ones that also work without TCG, i.e. I have to run the check script
> directly here with a list of tests that seem to be working in this
> environment.

qemu-iotests doesn't run any guest code and I think almost all tests
only use the qtest or none machines. We can make sure that the remaining
one or two tests that do need TCG/KVM for some reason are not in 'quick'
group, so that './check -T -qcow2 -g quick' does the right thing (which
is also what 'make check-block' does internally).

Which are the cases that fail for you with '--disable-tcg'?

Kevin

Re: [Qemu-devel] Failing iotests in CI (was: Add a gitlab-ci file for Continuous Integration testing on Gitlab)

[Thomas Huth]

On 19/02/2019 08.53, Kevin Wolf wrote:
> Am 19.02.2019 um 07:44 hat Thomas Huth geschrieben:
>> On 18/02/2019 19.22, Cleber Rosa wrote:
>>>
>>>
>>> On 2/13/19 6:54 AM, Thomas Huth wrote:
>>>> This is very convenient for people like me who store their QEMU git trees
>>>> on gitlab.com: Automatic CI pipelines are now run for each branch that is
>>>> pushed to the server - useful for some extra-testing before sending PULL-
>>>> requests for example. Since the runtime of the jobs is limited to 1h, the
>>>> jobs are distributed into multiple pipelines - this way everything finishs
>>>> fine within time (ca. 30 minutes currently).
>>>>
>>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>>> ---
>>>>  An example can be seen here: https://gitlab.com/huth/qemu/pipelines/
>>>>
>>>>  I'd really like to get this into the main QEMU repository, so that I don't
>>>>  have to cherry-pick this patch onto my testing branches anymore each time
>>>>  I want to test before sending a PULL request...
>>>>
>>>>  .gitlab-ci.yml | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>  MAINTAINERS    |  5 ++++
>>>>  2 files changed, 78 insertions(+)
>>>>  create mode 100644 .gitlab-ci.yml
>>>>
>>>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>>>> new file mode 100644
>>>> index 0000000..79d02cf
>>>> --- /dev/null
>>>> +++ b/.gitlab-ci.yml
>>>> @@ -0,0 +1,73 @@
>>>> +before_script:
>>>> + - apt-get update -qq
>>>> + - apt-get install -y -qq flex bison libglib2.0-dev libpixman-1-dev genisoimage
>>>> +
>>>> +build-system1:
>>>> + script:
>>>> + - apt-get install -y -qq libgtk-3-dev libvte-dev nettle-dev libcacard-dev
>>>> +      libusb-dev libvde-dev libspice-protocol-dev libgl1-mesa-dev
>>>> + - ./configure --enable-werror --target-list="aarch64-softmmu alpha-softmmu
>>>> +      cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu microblazeel-softmmu
>>>> +      mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu sparc-softmmu"
>>>> + - make -j2
>>>> + - make -j2 check
>>>> +
>>>> +build-system2:
>>>> + script:
>>>> + - apt-get install -y -qq libsdl2-dev libgcrypt-dev libbrlapi-dev libaio-dev
>>>> +      libfdt-dev liblzo2-dev librdmacm-dev libibverbs-dev libibumad-dev
>>>> + - ./configure --enable-werror --target-list="tricore-softmmu unicore32-softmmu
>>>> +      microblaze-softmmu mips-softmmu riscv32-softmmu s390x-softmmu sh4-softmmu
>>>> +      sparc64-softmmu x86_64-softmmu xtensa-softmmu nios2-softmmu or1k-softmmu"
>>>> + - make -j2
>>>> + - make -j2 check
>>>> +
>>>> +build-disabled:
>>>> + script:
>>>> + - ./configure --enable-werror --disable-rdma --disable-slirp --disable-curl
>>>> +      --disable-capstone --disable-live-block-migration --disable-glusterfs
>>>> +      --disable-replication --disable-coroutine-pool --disable-smartcard
>>>> +      --disable-guest-agent --disable-curses --disable-libxml2 --disable-tpm
>>>> +      --disable-qom-cast-debug --disable-spice --disable-vhost-vsock
>>>> +      --disable-vhost-net --disable-vhost-crypto --disable-vhost-user
>>>> +      --target-list="i386-softmmu ppc64-softmmu mips64-softmmu i386-linux-user"
>>>> + - make -j2
>>>> + - make -j2 check-qtest SPEED=slow
>>>> +
>>>> +build-tcg-disabled:
>>>> + script:
>>>> + - apt-get install -y -qq clang libgtk-3-dev libbluetooth-dev libusb-dev
>>>> + - ./configure --cc=clang --enable-werror --disable-tcg --audio-drv-list=""
>>>> + - make -j2
>>>> + - make check-unit
>>>> + - make check-qapi-schema
>>>> + - cd tests/qemu-iotests/
>>>> + - ./check -raw 001 002 003 004 005 008 009 010 011 012 021 025 032 033 048
>>>> +            052 063 077 086 101 104 106 113 147 148 150 151 152 157 159 160
>>>> +            163 170 171 183 184 192 194 197 205 208 215 221 222 226 227 236
>>>> + - ./check -qcow2 001 002 003 004 005 007 008 009 010 011 012 013 017 018 019
>>>> +            020 021 022 024 025 027 028 029 031 032 033 034 035 036 037 038
>>>> +            039 040 042 043 046 047 048 049 050 051 052 053 054 056 057 058
>>>> +            060 061 062 063 065 066 067 068 069 071 072 073 074 079 080 082
>>>> +            085 086 089 090 091 095 096 097 098 099 102 103 104 105 107 108
>>>> +            110 111 114 117 120 122 124 126 127 129 130 132 133 134 137 138
>>>> +            139 140 141 142 143 144 145 147 150 151 152 154 155 156 157 158
>>>> +            161 165 170 172 174 176 177 179 184 186 187 190 192 194 195 196
>>>> +            197 200 202 203 205 208 209 214 215 216 217 218 222 226 227 229 234
>>>> +
>>>
>>> On the matter of "make check-block" or "./check xxx", I've seen
>>> arguments for both sides.  I've sent a similar (but using make
>>> check-block) patch for Travis:
>>>
>>> https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg02334.html
>>>
>>> I'd be nice to hear from the qemu-iotest's maintainers which approach
>>> is preferable for upstream + shared computing resources kind of environment.
>>
>> I think "make check-block" should likely be the preferred way to run the
>> tests for normal builds (and by the way, why don't we always run this
>> during "make check" yet?).
>>
>> But in this CI file here, I'm running the tests for the "--disable-tcg"
>> build - simply because this test finishes way earlier than the other
>> tests otherwise. Unfortunately --disable-tcg causes some of the iotests
>> to fail - expected, since you also can not use KVM on the CI server, so
>> there is no way to execute guest code with this build (e.g. I'm also not
>> running the qtests in this pipe). Thus I had to limit the iotests to the
>> ones that also work without TCG, i.e. I have to run the check script
>> directly here with a list of tests that seem to be working in this
>> environment.
> 
> qemu-iotests doesn't run any guest code and I think almost all tests
> only use the qtest or none machines. We can make sure that the remaining
> one or two tests that do need TCG/KVM for some reason are not in 'quick'
> group, so that './check -T -qcow2 -g quick' does the right thing (which
> is also what 'make check-block' does internally).
> 
> Which are the cases that fail for you with '--disable-tcg'?

These tests are failing: 087 169 188 232 235 238

235 and 238 apparently try to use accel=kvm, so they likely should not
be in the "quick" group.

The other tests seem to fail for different reasons, see:

 https://gitlab.com/huth/qemu/-/jobs/163680780

Some of them apparently need encryption to be enabled (as already
mentioned by Cleber in his patch) - thus should they really be in the
quick check, too? Or could they at least check whether QEMU has been
built with encryption?

By the way, 235 and 238 also fail on my normal laptop with RHEL7:

235 2s ...        [07:29:31] [07:29:31] [failed, exit status 1] - output
mismatch (see 235.out.bad)
--- /home/thuth/devel/qemu/tests/qemu-iotests/235.out	2019-02-19
07:14:45.000000000 +0100
+++ /home/thuth/tmp/qemu-build/tests/qemu-iotests/235.out.bad	2019-02-19
07:29:31.000000000 +0100
@@ -1,3 +1,14 @@
-{"return": {}}
-{"return": {}}
-{"return": {}}
+Traceback (most recent call last):
+  File "235", line 56, in <module>
+    vm.launch()
+  File
"/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qemu.py", line
303, in launch
+    self._launch()
+  File
"/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qemu.py", line
329, in _launch
+    self._post_launch()
+  File
"/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qemu.py", line
274, in _post_launch
+    self._qmp.accept()
+  File
"/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qmp/qmp.py",
line 157, in accept
+    return self.__negotiate_capabilities()
+  File
"/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qmp/qmp.py",
line 73, in __negotiate_capabilities
+    raise QMPConnectError
+qmp.qmp.QMPConnectError
236 0s ...        [07:29:31] [07:29:31]
237                [07:29:31] [07:29:31] [not run]
        237 -- not suitable for this image format: qcow2
238                [07:29:31] [07:29:31] [failed, exit status 1] -
output mismatch (see 238.out.bad)
--- /home/thuth/devel/qemu/tests/qemu-iotests/238.out	2019-02-19
07:17:39.000000000 +0100
+++ /home/thuth/tmp/qemu-build/tests/qemu-iotests/238.out.bad	2019-02-19
07:29:31.000000000 +0100
@@ -1,6 +1,14 @@
-{"return": {}}
-{"return": {}}
-{"return": {}}
-{"return": {}}
-{"return": {}}
-{"return": {}}
+Traceback (most recent call last):
+  File "238", line 37, in <module>
+    vm.launch()
+  File
"/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qemu.py", line
303, in launch
+    self._launch()
+  File
"/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qemu.py", line
329, in _launch
+    self._post_launch()
+  File
"/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qemu.py", line
274, in _post_launch
+    self._qmp.accept()
+  File
"/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qmp/qmp.py",
line 157, in accept
+    return self.__negotiate_capabilities()
+  File
"/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qmp/qmp.py",
line 73, in __negotiate_capabilities
+    raise QMPConnectError
+qmp.qmp.QMPConnectError

Any ideas what might be going on here?

 Thomas

Re: [Qemu-devel] Failing iotests in CI (was: Add a gitlab-ci file for Continuous Integration testing on Gitlab)

[Kevin Wolf]

Am 19.02.2019 um 10:04 hat Thomas Huth geschrieben:
> On 19/02/2019 08.53, Kevin Wolf wrote:
> > Am 19.02.2019 um 07:44 hat Thomas Huth geschrieben:
> >> On 18/02/2019 19.22, Cleber Rosa wrote:
> >>>
> >>>
> >>> On 2/13/19 6:54 AM, Thomas Huth wrote:
> >>>> This is very convenient for people like me who store their QEMU git trees
> >>>> on gitlab.com: Automatic CI pipelines are now run for each branch that is
> >>>> pushed to the server - useful for some extra-testing before sending PULL-
> >>>> requests for example. Since the runtime of the jobs is limited to 1h, the
> >>>> jobs are distributed into multiple pipelines - this way everything finishs
> >>>> fine within time (ca. 30 minutes currently).
> >>>>
> >>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
> >>>> ---
> >>>>  An example can be seen here: https://gitlab.com/huth/qemu/pipelines/
> >>>>
> >>>>  I'd really like to get this into the main QEMU repository, so that I don't
> >>>>  have to cherry-pick this patch onto my testing branches anymore each time
> >>>>  I want to test before sending a PULL request...
> >>>>
> >>>>  .gitlab-ci.yml | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>>>  MAINTAINERS    |  5 ++++
> >>>>  2 files changed, 78 insertions(+)
> >>>>  create mode 100644 .gitlab-ci.yml
> >>>>
> >>>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> >>>> new file mode 100644
> >>>> index 0000000..79d02cf
> >>>> --- /dev/null
> >>>> +++ b/.gitlab-ci.yml
> >>>> @@ -0,0 +1,73 @@
> >>>> +before_script:
> >>>> + - apt-get update -qq
> >>>> + - apt-get install -y -qq flex bison libglib2.0-dev libpixman-1-dev genisoimage
> >>>> +
> >>>> +build-system1:
> >>>> + script:
> >>>> + - apt-get install -y -qq libgtk-3-dev libvte-dev nettle-dev libcacard-dev
> >>>> +      libusb-dev libvde-dev libspice-protocol-dev libgl1-mesa-dev
> >>>> + - ./configure --enable-werror --target-list="aarch64-softmmu alpha-softmmu
> >>>> +      cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu microblazeel-softmmu
> >>>> +      mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu sparc-softmmu"
> >>>> + - make -j2
> >>>> + - make -j2 check
> >>>> +
> >>>> +build-system2:
> >>>> + script:
> >>>> + - apt-get install -y -qq libsdl2-dev libgcrypt-dev libbrlapi-dev libaio-dev
> >>>> +      libfdt-dev liblzo2-dev librdmacm-dev libibverbs-dev libibumad-dev
> >>>> + - ./configure --enable-werror --target-list="tricore-softmmu unicore32-softmmu
> >>>> +      microblaze-softmmu mips-softmmu riscv32-softmmu s390x-softmmu sh4-softmmu
> >>>> +      sparc64-softmmu x86_64-softmmu xtensa-softmmu nios2-softmmu or1k-softmmu"
> >>>> + - make -j2
> >>>> + - make -j2 check
> >>>> +
> >>>> +build-disabled:
> >>>> + script:
> >>>> + - ./configure --enable-werror --disable-rdma --disable-slirp --disable-curl
> >>>> +      --disable-capstone --disable-live-block-migration --disable-glusterfs
> >>>> +      --disable-replication --disable-coroutine-pool --disable-smartcard
> >>>> +      --disable-guest-agent --disable-curses --disable-libxml2 --disable-tpm
> >>>> +      --disable-qom-cast-debug --disable-spice --disable-vhost-vsock
> >>>> +      --disable-vhost-net --disable-vhost-crypto --disable-vhost-user
> >>>> +      --target-list="i386-softmmu ppc64-softmmu mips64-softmmu i386-linux-user"
> >>>> + - make -j2
> >>>> + - make -j2 check-qtest SPEED=slow
> >>>> +
> >>>> +build-tcg-disabled:
> >>>> + script:
> >>>> + - apt-get install -y -qq clang libgtk-3-dev libbluetooth-dev libusb-dev
> >>>> + - ./configure --cc=clang --enable-werror --disable-tcg --audio-drv-list=""
> >>>> + - make -j2
> >>>> + - make check-unit
> >>>> + - make check-qapi-schema
> >>>> + - cd tests/qemu-iotests/
> >>>> + - ./check -raw 001 002 003 004 005 008 009 010 011 012 021 025 032 033 048
> >>>> +            052 063 077 086 101 104 106 113 147 148 150 151 152 157 159 160
> >>>> +            163 170 171 183 184 192 194 197 205 208 215 221 222 226 227 236
> >>>> + - ./check -qcow2 001 002 003 004 005 007 008 009 010 011 012 013 017 018 019
> >>>> +            020 021 022 024 025 027 028 029 031 032 033 034 035 036 037 038
> >>>> +            039 040 042 043 046 047 048 049 050 051 052 053 054 056 057 058
> >>>> +            060 061 062 063 065 066 067 068 069 071 072 073 074 079 080 082
> >>>> +            085 086 089 090 091 095 096 097 098 099 102 103 104 105 107 108
> >>>> +            110 111 114 117 120 122 124 126 127 129 130 132 133 134 137 138
> >>>> +            139 140 141 142 143 144 145 147 150 151 152 154 155 156 157 158
> >>>> +            161 165 170 172 174 176 177 179 184 186 187 190 192 194 195 196
> >>>> +            197 200 202 203 205 208 209 214 215 216 217 218 222 226 227 229 234
> >>>> +
> >>>
> >>> On the matter of "make check-block" or "./check xxx", I've seen
> >>> arguments for both sides.  I've sent a similar (but using make
> >>> check-block) patch for Travis:
> >>>
> >>> https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg02334.html
> >>>
> >>> I'd be nice to hear from the qemu-iotest's maintainers which approach
> >>> is preferable for upstream + shared computing resources kind of environment.
> >>
> >> I think "make check-block" should likely be the preferred way to run the
> >> tests for normal builds (and by the way, why don't we always run this
> >> during "make check" yet?).
> >>
> >> But in this CI file here, I'm running the tests for the "--disable-tcg"
> >> build - simply because this test finishes way earlier than the other
> >> tests otherwise. Unfortunately --disable-tcg causes some of the iotests
> >> to fail - expected, since you also can not use KVM on the CI server, so
> >> there is no way to execute guest code with this build (e.g. I'm also not
> >> running the qtests in this pipe). Thus I had to limit the iotests to the
> >> ones that also work without TCG, i.e. I have to run the check script
> >> directly here with a list of tests that seem to be working in this
> >> environment.
> > 
> > qemu-iotests doesn't run any guest code and I think almost all tests
> > only use the qtest or none machines. We can make sure that the remaining
> > one or two tests that do need TCG/KVM for some reason are not in 'quick'
> > group, so that './check -T -qcow2 -g quick' does the right thing (which
> > is also what 'make check-block' does internally).
> > 
> > Which are the cases that fail for you with '--disable-tcg'?
> 
> These tests are failing: 087 169 188 232 235 238

Hm, 087 and 232 just do something like:

    $QEMU_PROG -nodefaults -machine accel=qtest -nographic \
               -qmp stdio -serial none \
               ...some -drive and -object options...

This should be fine with --disable-tcg, I think?

169 runs a VM, but I don't see anything that makes it use TCG.

188 doesn't even run QEMU at all, it's only qemu-io. I don't see how
this could be possibly related to --disable-tcg.

> 235 and 238 apparently try to use accel=kvm, so they likely should not
> be in the "quick" group.

Yes, I agree.

Or maybe we should check again why they need kvm at all, in theory they
shouldn't. They are throttling tests, so the obvious reason seems to be
that they need a running clock. Using -M qtest and advancing the clock
manually could do the trick then (and would even be more reliable).

> The other tests seem to fail for different reasons, see:
> 
>  https://gitlab.com/huth/qemu/-/jobs/163680780
> 
> Some of them apparently need encryption to be enabled (as already
> mentioned by Cleber in his patch) - thus should they really be in the
> quick check, too? Or could they at least check whether QEMU has been
> built with encryption?

The correct solution would be that they detect the situation
automatically and skip the test by calling _notrun.

I'm not sure how to detect if a given QEMU binary supports encryption,
but Dan might know.

> By the way, 235 and 238 also fail on my normal laptop with RHEL7:
> 
> 235 2s ...        [07:29:31] [07:29:31] [failed, exit status 1] - output
> mismatch (see 235.out.bad)
> --- /home/thuth/devel/qemu/tests/qemu-iotests/235.out	2019-02-19
> 07:14:45.000000000 +0100
> +++ /home/thuth/tmp/qemu-build/tests/qemu-iotests/235.out.bad	2019-02-19
> 07:29:31.000000000 +0100
> @@ -1,3 +1,14 @@
> -{"return": {}}
> -{"return": {}}
> -{"return": {}}
> +Traceback (most recent call last):
> +  File "235", line 56, in <module>
> +    vm.launch()
> +  File
> "/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qemu.py", line
> 303, in launch
> +    self._launch()
> +  File
> "/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qemu.py", line
> 329, in _launch
> +    self._post_launch()
> +  File
> "/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qemu.py", line
> 274, in _post_launch
> +    self._qmp.accept()
> +  File
> "/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qmp/qmp.py",
> line 157, in accept
> +    return self.__negotiate_capabilities()
> +  File
> "/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qmp/qmp.py",
> line 73, in __negotiate_capabilities
> +    raise QMPConnectError
> +qmp.qmp.QMPConnectError
> 236 0s ...        [07:29:31] [07:29:31]
> 237                [07:29:31] [07:29:31] [not run]
>         237 -- not suitable for this image format: qcow2
> 238                [07:29:31] [07:29:31] [failed, exit status 1] -
> output mismatch (see 238.out.bad)
> --- /home/thuth/devel/qemu/tests/qemu-iotests/238.out	2019-02-19
> 07:17:39.000000000 +0100
> +++ /home/thuth/tmp/qemu-build/tests/qemu-iotests/238.out.bad	2019-02-19
> 07:29:31.000000000 +0100
> @@ -1,6 +1,14 @@
> -{"return": {}}
> -{"return": {}}
> -{"return": {}}
> -{"return": {}}
> -{"return": {}}
> -{"return": {}}
> +Traceback (most recent call last):
> +  File "238", line 37, in <module>
> +    vm.launch()
> +  File
> "/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qemu.py", line
> 303, in launch
> +    self._launch()
> +  File
> "/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qemu.py", line
> 329, in _launch
> +    self._post_launch()
> +  File
> "/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qemu.py", line
> 274, in _post_launch
> +    self._qmp.accept()
> +  File
> "/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qmp/qmp.py",
> line 157, in accept
> +    return self.__negotiate_capabilities()
> +  File
> "/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qmp/qmp.py",
> line 73, in __negotiate_capabilities
> +    raise QMPConnectError
> +qmp.qmp.QMPConnectError
> 
> Any ideas what might be going on here?

I think it's most likely that QEMU just prints an error message on
startup and exits. You could probably get more information from the log
file if on shutdown QEMUMachine didn't helpfully delete all the nice
logs it gathered.

I usually comment out the deletion part when I have something to debug,
but someone (TM) should maybe improve this so that the output of QEMU is
at least displayed with -d without having to modify the code...

Kevin

Re: [Qemu-devel] Failing iotests in CI (was: Add a gitlab-ci file for Continuous Integration testing on Gitlab)

[Thomas Huth]

On 19/02/2019 10.37, Kevin Wolf wrote:
> Am 19.02.2019 um 10:04 hat Thomas Huth geschrieben:
>> On 19/02/2019 08.53, Kevin Wolf wrote:
[...]
>>> Which are the cases that fail for you with '--disable-tcg'?
>>
>> These tests are failing: 087 169 188 232 235 238
> 
> Hm, 087 and 232 just do something like:
> 
>     $QEMU_PROG -nodefaults -machine accel=qtest -nographic \
>                -qmp stdio -serial none \
>                ...some -drive and -object options...
> 
> This should be fine with --disable-tcg, I think?
> 
> 169 runs a VM, but I don't see anything that makes it use TCG.
> 
> 188 doesn't even run QEMU at all, it's only qemu-io. I don't see how
> this could be possibly related to --disable-tcg.

087 and 188 obviously simply lack a check for the required crypto support.


Output of 087:

087                [08:33:55] [08:33:56] - output mismatch (see 087.out.bad)
--- /builds/huth/qemu/tests/qemu-iotests/087.out	2019-02-19 08:23:54.000000000 +0000
+++ /builds/huth/qemu/tests/qemu-iotests/087.out.bad	2019-02-19 08:33:56.000000000 +0000
@@ -46,12 +46,13 @@
 
 === Encrypted image LUKS ===
 
+qemu-img: TEST_DIR/t.IMGFMT: No crypto library supporting PBKDF in this build: Function not implemented
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encrypt.format=luks encrypt.key-secret=sec0
 Testing:
 QMP_VERSION
 {"return": {}}
 {"return": {}}
-{"return": {}}
+{"error": {"class": "GenericError", "desc": "No encryption in image header, but options specified format 'luks'"}}
 {"return": {}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}}


And the output of 188:

188                [08:34:53] [08:34:54] - output mismatch (see 188.out.bad)
--- /builds/huth/qemu/tests/qemu-iotests/188.out	2019-02-19 08:23:54.000000000 +0000
+++ /builds/huth/qemu/tests/qemu-iotests/188.out.bad	2019-02-19 08:34:54.000000000 +0000
@@ -1,4 +1,5 @@
 QA output created by 188
+qemu-img: TEST_DIR/t.IMGFMT: No crypto library supporting PBKDF in this build: Function not implemented
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
 
 == reading whole image ==
@@ -14,5 +15,6 @@
 16 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
 == verify open failure with wrong password ==
-can't open: Invalid password, cannot unlock any keyslot
+read 16777216/16777216 bytes at offset 0
+16 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 *** done


169 got killed via abort():

169                [08:34:39] [08:34:46] [failed, exit status 1] - output mismatch (see 169.out.bad)
--- /builds/huth/qemu/tests/qemu-iotests/169.out	2019-02-19 08:23:54.000000000 +0000
+++ /builds/huth/qemu/tests/qemu-iotests/169.out.bad	2019-02-19 08:34:46.000000000 +0000
@@ -1,5 +1,29 @@
-....................
+WARNING:qemu:qemu received signal 6: /builds/huth/qemu/tests/qemu-iotests/../../x86_64-softmmu/qemu-system-x86_64 -chardev socket,id=mon,path=/tmp/qemu-iotests-quick-25045/tmpGQOExQ/qemua-13044-monitor.sock -mon chardev=mon,mode=control -display none -vga none -qtest unix:path=/tmp/qemu-iotests-quick-25045/qemua-13044-qtest.sock -machine accel=qtest -nodefaults -machine accel=qtest -drive if=virtio,id=drive0,file=/tmp/qemu-iotests-quick-25045/disk_a,format=qcow2,cache=writeback
[...]

No clue why.


232 is also strange, no idea what is going on here:

232                [08:38:53] [08:38:56] - output mismatch (see 232.out.bad)
--- /builds/huth/qemu/tests/qemu-iotests/232.out	2019-02-19 08:23:54.000000000 +0000
+++ /builds/huth/qemu/tests/qemu-iotests/232.out.bad	2019-02-19 08:38:56.000000000 +0000
@@ -21,13 +21,13 @@
 NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only)
 NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only)
 
-QEMU_PROG: -drive driver=file,file=TEST_DIR/t.IMGFMT,if=none,read-only=off,auto-read-only=off: Could not open 'TEST_DIR/t.IMGFMT': Permission denied
-NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only)
-NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only)
-
-QEMU_PROG: -drive driver=file,file=TEST_DIR/t.IMGFMT,if=none,auto-read-only=off: Could not open 'TEST_DIR/t.IMGFMT': Permission denied
-NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only)
-NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only)
+NODE_NAME: TEST_DIR/t.IMGFMT (file)
+NODE_NAME: TEST_DIR/t.IMGFMT (file)
+NODE_NAME: TEST_DIR/t.IMGFMT (file)
+
+NODE_NAME: TEST_DIR/t.IMGFMT (file)
+NODE_NAME: TEST_DIR/t.IMGFMT (file)
+NODE_NAME: TEST_DIR/t.IMGFMT (file)

>> 235 and 238 apparently try to use accel=kvm, so they likely should not
>> be in the "quick" group.
> 
> Yes, I agree.
> 
> Or maybe we should check again why they need kvm at all, in theory they
> shouldn't. They are throttling tests, so the obvious reason seems to be
> that they need a running clock. Using -M qtest and advancing the clock
> manually could do the trick then (and would even be more reliable).
> 
>> The other tests seem to fail for different reasons, see:
>>
>>  https://gitlab.com/huth/qemu/-/jobs/163680780
>>
>> Some of them apparently need encryption to be enabled (as already
>> mentioned by Cleber in his patch) - thus should they really be in the
>> quick check, too? Or could they at least check whether QEMU has been
>> built with encryption?
> 
> The correct solution would be that they detect the situation
> automatically and skip the test by calling _notrun.
> 
> I'm not sure how to detect if a given QEMU binary supports encryption,
> but Dan might know.
> 
>> By the way, 235 and 238 also fail on my normal laptop with RHEL7:
>>
>> 235 2s ...        [07:29:31] [07:29:31] [failed, exit status 1] - output
>> mismatch (see 235.out.bad)
>> --- /home/thuth/devel/qemu/tests/qemu-iotests/235.out	2019-02-19
>> 07:14:45.000000000 +0100
>> +++ /home/thuth/tmp/qemu-build/tests/qemu-iotests/235.out.bad	2019-02-19
>> 07:29:31.000000000 +0100
>> @@ -1,3 +1,14 @@
>> -{"return": {}}
>> -{"return": {}}
>> -{"return": {}}
>> +Traceback (most recent call last):
>> +  File "235", line 56, in <module>
>> +    vm.launch()
>> +  File
>> "/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qemu.py", line
>> 303, in launch
>> +    self._launch()
>> +  File
>> "/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qemu.py", line
>> 329, in _launch
>> +    self._post_launch()
>> +  File
>> "/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qemu.py", line
>> 274, in _post_launch
>> +    self._qmp.accept()
>> +  File
>> "/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qmp/qmp.py",
>> line 157, in accept
>> +    return self.__negotiate_capabilities()
>> +  File
>> "/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qmp/qmp.py",
>> line 73, in __negotiate_capabilities
>> +    raise QMPConnectError
>> +qmp.qmp.QMPConnectError
>> 236 0s ...        [07:29:31] [07:29:31]
>> 237                [07:29:31] [07:29:31] [not run]
>>         237 -- not suitable for this image format: qcow2
>> 238                [07:29:31] [07:29:31] [failed, exit status 1] -
>> output mismatch (see 238.out.bad)
>> --- /home/thuth/devel/qemu/tests/qemu-iotests/238.out	2019-02-19
>> 07:17:39.000000000 +0100
>> +++ /home/thuth/tmp/qemu-build/tests/qemu-iotests/238.out.bad	2019-02-19
>> 07:29:31.000000000 +0100
>> @@ -1,6 +1,14 @@
>> -{"return": {}}
>> -{"return": {}}
>> -{"return": {}}
>> -{"return": {}}
>> -{"return": {}}
>> -{"return": {}}
>> +Traceback (most recent call last):
>> +  File "238", line 37, in <module>
>> +    vm.launch()
>> +  File
>> "/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qemu.py", line
>> 303, in launch
>> +    self._launch()
>> +  File
>> "/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qemu.py", line
>> 329, in _launch
>> +    self._post_launch()
>> +  File
>> "/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qemu.py", line
>> 274, in _post_launch
>> +    self._qmp.accept()
>> +  File
>> "/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qmp/qmp.py",
>> line 157, in accept
>> +    return self.__negotiate_capabilities()
>> +  File
>> "/home/thuth/devel/qemu/tests/qemu-iotests/../../scripts/qmp/qmp.py",
>> line 73, in __negotiate_capabilities
>> +    raise QMPConnectError
>> +qmp.qmp.QMPConnectError
>>
>> Any ideas what might be going on here?
> 
> I think it's most likely that QEMU just prints an error message on
> startup and exits.
Right, I finally found the issue:

qemu-system-x86_64: -machine accel=kvm: No accelerator found

I apparently compiled my QEMU with --disable-kvm at one point in time and
forgot to enable it later again. ==> These tests should really check whether
KVM is available in QEMU before they blindly use this feature.

 Thomas

Re: [Qemu-devel] Failing iotests in CI (was: Add a gitlab-ci file for Continuous Integration testing on Gitlab)

[Kevin Wolf]

Am 19.02.2019 um 11:11 hat Thomas Huth geschrieben:
> On 19/02/2019 10.37, Kevin Wolf wrote:
> > Am 19.02.2019 um 10:04 hat Thomas Huth geschrieben:
> >> On 19/02/2019 08.53, Kevin Wolf wrote:
> [...]
> >>> Which are the cases that fail for you with '--disable-tcg'?
> >>
> >> These tests are failing: 087 169 188 232 235 238
> > 
> > Hm, 087 and 232 just do something like:
> > 
> >     $QEMU_PROG -nodefaults -machine accel=qtest -nographic \
> >                -qmp stdio -serial none \
> >                ...some -drive and -object options...
> > 
> > This should be fine with --disable-tcg, I think?
> > 
> > 169 runs a VM, but I don't see anything that makes it use TCG.
> > 
> > 188 doesn't even run QEMU at all, it's only qemu-io. I don't see how
> > this could be possibly related to --disable-tcg.
> 
> 087 and 188 obviously simply lack a check for the required crypto
> support.

Ah, okay, I thought you meant your whole list fails because of
--disable-tcg. Makes sense then.

> 169 got killed via abort():
> 
> 169                [08:34:39] [08:34:46] [failed, exit status 1] - output mismatch (see 169.out.bad)
> --- /builds/huth/qemu/tests/qemu-iotests/169.out	2019-02-19 08:23:54.000000000 +0000
> +++ /builds/huth/qemu/tests/qemu-iotests/169.out.bad	2019-02-19 08:34:46.000000000 +0000
> @@ -1,5 +1,29 @@
> -....................
> +WARNING:qemu:qemu received signal 6: /builds/huth/qemu/tests/qemu-iotests/../../x86_64-softmmu/qemu-system-x86_64 -chardev socket,id=mon,path=/tmp/qemu-iotests-quick-25045/tmpGQOExQ/qemua-13044-monitor.sock -mon chardev=mon,mode=control -display none -vga none -qtest unix:path=/tmp/qemu-iotests-quick-25045/qemua-13044-qtest.sock -machine accel=qtest -nodefaults -machine accel=qtest -drive if=virtio,id=drive0,file=/tmp/qemu-iotests-quick-25045/disk_a,format=qcow2,cache=writeback
> [...]
> 
> No clue why.

In order to debug failures like this, I usually enable core dumps.

> 232 is also strange, no idea what is going on here:
> 
> 232                [08:38:53] [08:38:56] - output mismatch (see 232.out.bad)
> --- /builds/huth/qemu/tests/qemu-iotests/232.out	2019-02-19 08:23:54.000000000 +0000
> +++ /builds/huth/qemu/tests/qemu-iotests/232.out.bad	2019-02-19 08:38:56.000000000 +0000
> @@ -21,13 +21,13 @@
>  NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only)
>  NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only)
>  
> -QEMU_PROG: -drive driver=file,file=TEST_DIR/t.IMGFMT,if=none,read-only=off,auto-read-only=off: Could not open 'TEST_DIR/t.IMGFMT': Permission denied
> -NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only)
> -NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only)
> -
> -QEMU_PROG: -drive driver=file,file=TEST_DIR/t.IMGFMT,if=none,auto-read-only=off: Could not open 'TEST_DIR/t.IMGFMT': Permission denied
> -NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only)
> -NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only)
> +NODE_NAME: TEST_DIR/t.IMGFMT (file)
> +NODE_NAME: TEST_DIR/t.IMGFMT (file)
> +NODE_NAME: TEST_DIR/t.IMGFMT (file)
> +
> +NODE_NAME: TEST_DIR/t.IMGFMT (file)
> +NODE_NAME: TEST_DIR/t.IMGFMT (file)
> +NODE_NAME: TEST_DIR/t.IMGFMT (file)

Do you run the tests as root? This messes with any tests that try to
test read-only cases because root can write to everything, no matter
what the permissions say.

> >> By the way, 235 and 238 also fail on my normal laptop with RHEL7:
> >> [...]
> >> Any ideas what might be going on here?
> > 
> > I think it's most likely that QEMU just prints an error message on
> > startup and exits.
> Right, I finally found the issue:
> 
> qemu-system-x86_64: -machine accel=kvm: No accelerator found
> 
> I apparently compiled my QEMU with --disable-kvm at one point in time and
> forgot to enable it later again. ==> These tests should really check whether
> KVM is available in QEMU before they blindly use this feature.

Ideally, they shouldn't use it at all.

Kevin

Re: [Qemu-devel] Failing iotests in CI (was: Add a gitlab-ci file for Continuous Integration testing on Gitlab)

[Thomas Huth]

On 19/02/2019 12.38, Kevin Wolf wrote:
> Am 19.02.2019 um 11:11 hat Thomas Huth geschrieben:
>> On 19/02/2019 10.37, Kevin Wolf wrote:
>>> Am 19.02.2019 um 10:04 hat Thomas Huth geschrieben:
>>>> On 19/02/2019 08.53, Kevin Wolf wrote:
[...]
>> 169 got killed via abort():
>>
>> 169                [08:34:39] [08:34:46] [failed, exit status 1] - output mismatch (see 169.out.bad)
>> --- /builds/huth/qemu/tests/qemu-iotests/169.out	2019-02-19 08:23:54.000000000 +0000
>> +++ /builds/huth/qemu/tests/qemu-iotests/169.out.bad	2019-02-19 08:34:46.000000000 +0000
>> @@ -1,5 +1,29 @@
>> -....................
>> +WARNING:qemu:qemu received signal 6: /builds/huth/qemu/tests/qemu-iotests/../../x86_64-softmmu/qemu-system-x86_64 -chardev socket,id=mon,path=/tmp/qemu-iotests-quick-25045/tmpGQOExQ/qemua-13044-monitor.sock -mon chardev=mon,mode=control -display none -vga none -qtest unix:path=/tmp/qemu-iotests-quick-25045/qemua-13044-qtest.sock -machine accel=qtest -nodefaults -machine accel=qtest -drive if=virtio,id=drive0,file=/tmp/qemu-iotests-quick-25045/disk_a,format=qcow2,cache=writeback
>> [...]
>>
>> No clue why.
> 
> In order to debug failures like this, I usually enable core dumps.

It's only failing for me on the gitlab-CI system. I don't think that
I've got the possibility to grab coredumps there...

>> 232 is also strange, no idea what is going on here:
>>
>> 232                [08:38:53] [08:38:56] - output mismatch (see 232.out.bad)
>> --- /builds/huth/qemu/tests/qemu-iotests/232.out	2019-02-19 08:23:54.000000000 +0000
>> +++ /builds/huth/qemu/tests/qemu-iotests/232.out.bad	2019-02-19 08:38:56.000000000 +0000
>> @@ -21,13 +21,13 @@
>>  NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only)
>>  NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only)
>>  
>> -QEMU_PROG: -drive driver=file,file=TEST_DIR/t.IMGFMT,if=none,read-only=off,auto-read-only=off: Could not open 'TEST_DIR/t.IMGFMT': Permission denied
>> -NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only)
>> -NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only)
>> -
>> -QEMU_PROG: -drive driver=file,file=TEST_DIR/t.IMGFMT,if=none,auto-read-only=off: Could not open 'TEST_DIR/t.IMGFMT': Permission denied
>> -NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only)
>> -NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only)
>> +NODE_NAME: TEST_DIR/t.IMGFMT (file)
>> +NODE_NAME: TEST_DIR/t.IMGFMT (file)
>> +NODE_NAME: TEST_DIR/t.IMGFMT (file)
>> +
>> +NODE_NAME: TEST_DIR/t.IMGFMT (file)
>> +NODE_NAME: TEST_DIR/t.IMGFMT (file)
>> +NODE_NAME: TEST_DIR/t.IMGFMT (file)
> 
> Do you run the tests as root? This messes with any tests that try to
> test read-only cases because root can write to everything, no matter
> what the permissions say.

I don't know whether gitlab runs the CI pipelines as root, but I doubt
it. Looking at the shell "prompts" that they show inbetween ("$") it
looks like they are running the CI tests as a normal user, though.

>>>> By the way, 235 and 238 also fail on my normal laptop with RHEL7:
>>>> [...]
>>>> Any ideas what might be going on here?
>>>
>>> I think it's most likely that QEMU just prints an error message on
>>> startup and exits.
>> Right, I finally found the issue:
>>
>> qemu-system-x86_64: -machine accel=kvm: No accelerator found
>>
>> I apparently compiled my QEMU with --disable-kvm at one point in time and
>> forgot to enable it later again. ==> These tests should really check whether
>> KVM is available in QEMU before they blindly use this feature.
> 
> Ideally, they shouldn't use it at all.

235 seems to use KVM on purpose according to the commit description...?

But maybe we can change it to "-M accel=kvm:qtest" somehow ... though
this currently seems to hang the test, as far as I can see... Maybe
"accel=kvm:tcg", so that it works at least without kvm?

According to Stefan, 238 does not need KVM at all (see his patch that he
just sent).

 Thomas

Re: [Qemu-devel] Failing iotests in CI (was: Add a gitlab-ci file for Continuous Integration testing on Gitlab)

[Daniel P. Berrangé]

On Tue, Feb 19, 2019 at 10:37:16AM +0100, Kevin Wolf wrote:
> Am 19.02.2019 um 10:04 hat Thomas Huth geschrieben:
> > 
> >  https://gitlab.com/huth/qemu/-/jobs/163680780
> > 
> > Some of them apparently need encryption to be enabled (as already
> > mentioned by Cleber in his patch) - thus should they really be in the
> > quick check, too? Or could they at least check whether QEMU has been
> > built with encryption?
> 
> The correct solution would be that they detect the situation
> automatically and skip the test by calling _notrun.
> 
> I'm not sure how to detect if a given QEMU binary supports encryption,
> but Dan might know.

It isn't easy & depends which encryption feature you're trying to use.

For TLS related features you can do something gross like

    qemu-img info --object tls-creds-anon,id=dummy README 2>&1
    test $? != 0 && exit 0

This relies on fact that 'tls-creds-anon' object type will report a
runtime error during initialization if gnutls isn't enabled.

For more general ciphers you pretty much have to just try the higher level
feature and see if it fails.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] Failing iotests in CI (was: Add a gitlab-ci file for Continuous Integration testing on Gitlab)

[Kevin Wolf]

Am 19.02.2019 um 12:06 hat Daniel P. Berrangé geschrieben:
> On Tue, Feb 19, 2019 at 10:37:16AM +0100, Kevin Wolf wrote:
> > Am 19.02.2019 um 10:04 hat Thomas Huth geschrieben:
> > > 
> > >  https://gitlab.com/huth/qemu/-/jobs/163680780
> > > 
> > > Some of them apparently need encryption to be enabled (as already
> > > mentioned by Cleber in his patch) - thus should they really be in the
> > > quick check, too? Or could they at least check whether QEMU has been
> > > built with encryption?
> > 
> > The correct solution would be that they detect the situation
> > automatically and skip the test by calling _notrun.
> > 
> > I'm not sure how to detect if a given QEMU binary supports encryption,
> > but Dan might know.
> 
> It isn't easy & depends which encryption feature you're trying to use.
> 
> For TLS related features you can do something gross like
> 
>     qemu-img info --object tls-creds-anon,id=dummy README 2>&1
>     test $? != 0 && exit 0
> 
> This relies on fact that 'tls-creds-anon' object type will report a
> runtime error during initialization if gnutls isn't enabled.
> 
> For more general ciphers you pretty much have to just try the higher level
> feature and see if it fails.

Actually, I think for test cases we should see 'qemu-img create' failing
and could just skip the test if it returns a non-zero exit code.

But then I looked at Thomas' output again:

    --- /builds/huth/qemu/tests/qemu-iotests/188.out	2019-02-19 08:23:54.000000000 +0000
    +++ /builds/huth/qemu/tests/qemu-iotests/188.out.bad	2019-02-19 08:34:54.000000000 +0000
    @@ -1,4 +1,5 @@
     QA output created by 188
    +qemu-img: TEST_DIR/t.IMGFMT: No crypto library supporting PBKDF in this build: Function not implemented
     Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10

     == reading whole image ==--- /builds/huth/qemu/tests/qemu-iotests/188.out	2019-02-19 08:23:54.000000000 +0000

What is it actually doing there? There's clearly an error message, but
it almost looks like it's creating some kind of image anyway? The
following I/O works fine (i.e. this created image can even be opened
again with the luks driver), except that you can also access the image
with the wrong password.

Is this a real bug in either qcow2 or luks?

Kevin

Re: [Qemu-devel] Failing iotests in CI (was: Add a gitlab-ci file for Continuous Integration testing on Gitlab)

[Daniel P. Berrangé]

On Tue, Feb 19, 2019 at 12:31:41PM +0100, Kevin Wolf wrote:
> Am 19.02.2019 um 12:06 hat Daniel P. Berrangé geschrieben:
> > On Tue, Feb 19, 2019 at 10:37:16AM +0100, Kevin Wolf wrote:
> > > Am 19.02.2019 um 10:04 hat Thomas Huth geschrieben:
> > > > 
> > > >  https://gitlab.com/huth/qemu/-/jobs/163680780
> > > > 
> > > > Some of them apparently need encryption to be enabled (as already
> > > > mentioned by Cleber in his patch) - thus should they really be in the
> > > > quick check, too? Or could they at least check whether QEMU has been
> > > > built with encryption?
> > > 
> > > The correct solution would be that they detect the situation
> > > automatically and skip the test by calling _notrun.
> > > 
> > > I'm not sure how to detect if a given QEMU binary supports encryption,
> > > but Dan might know.
> > 
> > It isn't easy & depends which encryption feature you're trying to use.
> > 
> > For TLS related features you can do something gross like
> > 
> >     qemu-img info --object tls-creds-anon,id=dummy README 2>&1
> >     test $? != 0 && exit 0
> > 
> > This relies on fact that 'tls-creds-anon' object type will report a
> > runtime error during initialization if gnutls isn't enabled.
> > 
> > For more general ciphers you pretty much have to just try the higher level
> > feature and see if it fails.
> 
> Actually, I think for test cases we should see 'qemu-img create' failing
> and could just skip the test if it returns a non-zero exit code.
> 
> But then I looked at Thomas' output again:
> 
>     --- /builds/huth/qemu/tests/qemu-iotests/188.out	2019-02-19 08:23:54.000000000 +0000
>     +++ /builds/huth/qemu/tests/qemu-iotests/188.out.bad	2019-02-19 08:34:54.000000000 +0000
>     @@ -1,4 +1,5 @@
>      QA output created by 188
>     +qemu-img: TEST_DIR/t.IMGFMT: No crypto library supporting PBKDF in this build: Function not implemented
>      Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
> 
>      == reading whole image ==--- /builds/huth/qemu/tests/qemu-iotests/188.out	2019-02-19 08:23:54.000000000 +0000
> 
> What is it actually doing there? There's clearly an error message, but
> it almost looks like it's creating some kind of image anyway? The
> following I/O works fine (i.e. this created image can even be opened
> again with the luks driver), except that you can also access the image
> with the wrong password.
> 
> Is this a real bug in either qcow2 or luks?

It is an artifact of the way qcow2 image creation happens in multiple
phases. qcow2_co_create first creates a minimal qcow2 file, and then
opens it and updates it to add in the various extra features, including
luks encryption. We fail to create the luks encryption, but enough of
the qcow2 file has been created that it is able to still do plain text
I/O.

Essentially the problem is that qcow2_co_create() doesn't unlink() the
partially created image when things fail. This is a generic problem
which can affect any part of qcow2_co_create that might fail, but it
is especially problematic with luks.

The complication in fixing this is that can't just do an unlink() as
we can't assume a local file. We need to have a bdrv_unlink() driver
callback we can use to delegate to the right block driver APIs for
deletion.

This is something I think could be useful in general, to expose as a
"qemu-img delete" command - especially for non-local formats like
rbd, sheepdog, glusterfs the mgmt app can't simply run "rm $img".
QEMU already knows how to talk to the native APis for these network
drivers so can make life easier by exposing a 'delete' op.

For luks it would be desirable to allow options to the 'delete'
operation, for example to request scrubbing of the key material
headers.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] Failing iotests in CI (was: Add a gitlab-ci file for Continuous Integration testing on Gitlab)

[Daniel P. Berrangé]

On Tue, Feb 19, 2019 at 12:01:28PM +0000, Daniel P. Berrangé wrote:
> On Tue, Feb 19, 2019 at 12:31:41PM +0100, Kevin Wolf wrote:
> > Am 19.02.2019 um 12:06 hat Daniel P. Berrangé geschrieben:
> > > On Tue, Feb 19, 2019 at 10:37:16AM +0100, Kevin Wolf wrote:
> > > > Am 19.02.2019 um 10:04 hat Thomas Huth geschrieben:
> > > > > 
> > > > >  https://gitlab.com/huth/qemu/-/jobs/163680780
> > > > > 
> > > > > Some of them apparently need encryption to be enabled (as already
> > > > > mentioned by Cleber in his patch) - thus should they really be in the
> > > > > quick check, too? Or could they at least check whether QEMU has been
> > > > > built with encryption?
> > > > 
> > > > The correct solution would be that they detect the situation
> > > > automatically and skip the test by calling _notrun.
> > > > 
> > > > I'm not sure how to detect if a given QEMU binary supports encryption,
> > > > but Dan might know.
> > > 
> > > It isn't easy & depends which encryption feature you're trying to use.
> > > 
> > > For TLS related features you can do something gross like
> > > 
> > >     qemu-img info --object tls-creds-anon,id=dummy README 2>&1
> > >     test $? != 0 && exit 0
> > > 
> > > This relies on fact that 'tls-creds-anon' object type will report a
> > > runtime error during initialization if gnutls isn't enabled.
> > > 
> > > For more general ciphers you pretty much have to just try the higher level
> > > feature and see if it fails.
> > 
> > Actually, I think for test cases we should see 'qemu-img create' failing
> > and could just skip the test if it returns a non-zero exit code.
> > 
> > But then I looked at Thomas' output again:
> > 
> >     --- /builds/huth/qemu/tests/qemu-iotests/188.out	2019-02-19 08:23:54.000000000 +0000
> >     +++ /builds/huth/qemu/tests/qemu-iotests/188.out.bad	2019-02-19 08:34:54.000000000 +0000
> >     @@ -1,4 +1,5 @@
> >      QA output created by 188
> >     +qemu-img: TEST_DIR/t.IMGFMT: No crypto library supporting PBKDF in this build: Function not implemented
> >      Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
> > 
> >      == reading whole image ==--- /builds/huth/qemu/tests/qemu-iotests/188.out	2019-02-19 08:23:54.000000000 +0000
> > 
> > What is it actually doing there? There's clearly an error message, but
> > it almost looks like it's creating some kind of image anyway? The
> > following I/O works fine (i.e. this created image can even be opened
> > again with the luks driver), except that you can also access the image
> > with the wrong password.
> > 
> > Is this a real bug in either qcow2 or luks?
> 
> It is an artifact of the way qcow2 image creation happens in multiple
> phases. qcow2_co_create first creates a minimal qcow2 file, and then
> opens it and updates it to add in the various extra features, including
> luks encryption. We fail to create the luks encryption, but enough of
> the qcow2 file has been created that it is able to still do plain text
> I/O.
> 
> Essentially the problem is that qcow2_co_create() doesn't unlink() the
> partially created image when things fail. This is a generic problem
> which can affect any part of qcow2_co_create that might fail, but it
> is especially problematic with luks.
> 
> The complication in fixing this is that can't just do an unlink() as
> we can't assume a local file. We need to have a bdrv_unlink() driver
> callback we can use to delegate to the right block driver APIs for
> deletion.

As a quick hack we could perhaps overwrite the qcow2 header with
garbage or all-zeros on failure to avoid it being mistakenly
interpreted as a valid qcow2 file on failure (could still be
mistakenly probed as raw but we tell people not to allow probing)


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] Failing iotests in CI (was: Add a gitlab-ci file for Continuous Integration testing on Gitlab)

[Kevin Wolf]

Am 19.02.2019 um 13:01 hat Daniel P. Berrangé geschrieben:
> On Tue, Feb 19, 2019 at 12:31:41PM +0100, Kevin Wolf wrote:
> > Am 19.02.2019 um 12:06 hat Daniel P. Berrangé geschrieben:
> > > On Tue, Feb 19, 2019 at 10:37:16AM +0100, Kevin Wolf wrote:
> > > > Am 19.02.2019 um 10:04 hat Thomas Huth geschrieben:
> > > > > 
> > > > >  https://gitlab.com/huth/qemu/-/jobs/163680780
> > > > > 
> > > > > Some of them apparently need encryption to be enabled (as already
> > > > > mentioned by Cleber in his patch) - thus should they really be in the
> > > > > quick check, too? Or could they at least check whether QEMU has been
> > > > > built with encryption?
> > > > 
> > > > The correct solution would be that they detect the situation
> > > > automatically and skip the test by calling _notrun.
> > > > 
> > > > I'm not sure how to detect if a given QEMU binary supports encryption,
> > > > but Dan might know.
> > > 
> > > It isn't easy & depends which encryption feature you're trying to use.
> > > 
> > > For TLS related features you can do something gross like
> > > 
> > >     qemu-img info --object tls-creds-anon,id=dummy README 2>&1
> > >     test $? != 0 && exit 0
> > > 
> > > This relies on fact that 'tls-creds-anon' object type will report a
> > > runtime error during initialization if gnutls isn't enabled.
> > > 
> > > For more general ciphers you pretty much have to just try the higher level
> > > feature and see if it fails.
> > 
> > Actually, I think for test cases we should see 'qemu-img create' failing
> > and could just skip the test if it returns a non-zero exit code.
> > 
> > But then I looked at Thomas' output again:
> > 
> >     --- /builds/huth/qemu/tests/qemu-iotests/188.out	2019-02-19 08:23:54.000000000 +0000
> >     +++ /builds/huth/qemu/tests/qemu-iotests/188.out.bad	2019-02-19 08:34:54.000000000 +0000
> >     @@ -1,4 +1,5 @@
> >      QA output created by 188
> >     +qemu-img: TEST_DIR/t.IMGFMT: No crypto library supporting PBKDF in this build: Function not implemented
> >      Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
> > 
> >      == reading whole image ==--- /builds/huth/qemu/tests/qemu-iotests/188.out	2019-02-19 08:23:54.000000000 +0000
> > 
> > What is it actually doing there? There's clearly an error message, but
> > it almost looks like it's creating some kind of image anyway? The
> > following I/O works fine (i.e. this created image can even be opened
> > again with the luks driver), except that you can also access the image
> > with the wrong password.
> > 
> > Is this a real bug in either qcow2 or luks?
> 
> It is an artifact of the way qcow2 image creation happens in multiple
> phases. qcow2_co_create first creates a minimal qcow2 file, and then
> opens it and updates it to add in the various extra features, including
> luks encryption. We fail to create the luks encryption, but enough of
> the qcow2 file has been created that it is able to still do plain text
> I/O.

But why doesn't qcow2_update_options_prepare() fail then? If you pass
encryption options like 188 does, but the image isn't encrypted, then
the function at least attempts to error out. Where is this failing?

> Essentially the problem is that qcow2_co_create() doesn't unlink() the
> partially created image when things fail. This is a generic problem
> which can affect any part of qcow2_co_create that might fail, but it
> is especially problematic with luks.
> 
> The complication in fixing this is that can't just do an unlink() as
> we can't assume a local file. We need to have a bdrv_unlink() driver
> callback we can use to delegate to the right block driver APIs for
> deletion.

.bdrv_co_create can't unlink at all, because that would be undoing
something that it didn't even do itself. If I created a file (local or
on a remote server that is accessed over a network protocol) and my
blockdev-create command to create the qcow2 layer fails, I certainly
don't expect the resource that I manually created to go away.

What .bdrv_co_create could and probably should do is make the header
invalid so that it's still recognised as qcow2 (to avoid probing it as
raw), but opening it fails.

> This is something I think could be useful in general, to expose as a
> "qemu-img delete" command - especially for non-local formats like
> rbd, sheepdog, glusterfs the mgmt app can't simply run "rm $img".
> QEMU already knows how to talk to the native APis for these network
> drivers so can make life easier by exposing a 'delete' op.
> 
> For luks it would be desirable to allow options to the 'delete'
> operation, for example to request scrubbing of the key material
> headers.

This could still be useful, just not necessarily in the context of
blockdev-create. It could actually be used in 'qemu-img create', which
creates both the protocol layer and the format layer.

Kevin

Re: [Qemu-devel] Failing iotests in CI (was: Add a gitlab-ci file for Continuous Integration testing on Gitlab)

[Daniel P. Berrangé]

On Tue, Feb 19, 2019 at 01:16:57PM +0100, Kevin Wolf wrote:
> Am 19.02.2019 um 13:01 hat Daniel P. Berrangé geschrieben:
> > On Tue, Feb 19, 2019 at 12:31:41PM +0100, Kevin Wolf wrote:
> > > Am 19.02.2019 um 12:06 hat Daniel P. Berrangé geschrieben:
> > > > On Tue, Feb 19, 2019 at 10:37:16AM +0100, Kevin Wolf wrote:
> > > > > Am 19.02.2019 um 10:04 hat Thomas Huth geschrieben:
> > > > > > 
> > > > > >  https://gitlab.com/huth/qemu/-/jobs/163680780
> > > > > > 
> > > > > > Some of them apparently need encryption to be enabled (as already
> > > > > > mentioned by Cleber in his patch) - thus should they really be in the
> > > > > > quick check, too? Or could they at least check whether QEMU has been
> > > > > > built with encryption?
> > > > > 
> > > > > The correct solution would be that they detect the situation
> > > > > automatically and skip the test by calling _notrun.
> > > > > 
> > > > > I'm not sure how to detect if a given QEMU binary supports encryption,
> > > > > but Dan might know.
> > > > 
> > > > It isn't easy & depends which encryption feature you're trying to use.
> > > > 
> > > > For TLS related features you can do something gross like
> > > > 
> > > >     qemu-img info --object tls-creds-anon,id=dummy README 2>&1
> > > >     test $? != 0 && exit 0
> > > > 
> > > > This relies on fact that 'tls-creds-anon' object type will report a
> > > > runtime error during initialization if gnutls isn't enabled.
> > > > 
> > > > For more general ciphers you pretty much have to just try the higher level
> > > > feature and see if it fails.
> > > 
> > > Actually, I think for test cases we should see 'qemu-img create' failing
> > > and could just skip the test if it returns a non-zero exit code.
> > > 
> > > But then I looked at Thomas' output again:
> > > 
> > >     --- /builds/huth/qemu/tests/qemu-iotests/188.out	2019-02-19 08:23:54.000000000 +0000
> > >     +++ /builds/huth/qemu/tests/qemu-iotests/188.out.bad	2019-02-19 08:34:54.000000000 +0000
> > >     @@ -1,4 +1,5 @@
> > >      QA output created by 188
> > >     +qemu-img: TEST_DIR/t.IMGFMT: No crypto library supporting PBKDF in this build: Function not implemented
> > >      Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
> > > 
> > >      == reading whole image ==--- /builds/huth/qemu/tests/qemu-iotests/188.out	2019-02-19 08:23:54.000000000 +0000
> > > 
> > > What is it actually doing there? There's clearly an error message, but
> > > it almost looks like it's creating some kind of image anyway? The
> > > following I/O works fine (i.e. this created image can even be opened
> > > again with the luks driver), except that you can also access the image
> > > with the wrong password.
> > > 
> > > Is this a real bug in either qcow2 or luks?
> > 
> > It is an artifact of the way qcow2 image creation happens in multiple
> > phases. qcow2_co_create first creates a minimal qcow2 file, and then
> > opens it and updates it to add in the various extra features, including
> > luks encryption. We fail to create the luks encryption, but enough of
> > the qcow2 file has been created that it is able to still do plain text
> > I/O.
> 
> But why doesn't qcow2_update_options_prepare() fail then? If you pass
> encryption options like 188 does, but the image isn't encrypted, then
> the function at least attempts to error out. Where is this failing?

Oh, its because we check for "encrypt.format", but we made that field
optional when opening the image, allowing other encrypt.* fields to be
set and just pulling format from the header. I'v sent a patch for this.

> > Essentially the problem is that qcow2_co_create() doesn't unlink() the
> > partially created image when things fail. This is a generic problem
> > which can affect any part of qcow2_co_create that might fail, but it
> > is especially problematic with luks.
> > 
> > The complication in fixing this is that can't just do an unlink() as
> > we can't assume a local file. We need to have a bdrv_unlink() driver
> > callback we can use to delegate to the right block driver APIs for
> > deletion.
> 
> .bdrv_co_create can't unlink at all, because that would be undoing
> something that it didn't even do itself. If I created a file (local or
> on a remote server that is accessed over a network protocol) and my
> blockdev-create command to create the qcow2 layer fails, I certainly
> don't expect the resource that I manually created to go away.

Oh, I forgot that we unconditionally call bdrv_create_file even if
we might have a pre-existing file, expecting it to be a no-op.

> What .bdrv_co_create could and probably should do is make the header
> invalid so that it's still recognised as qcow2 (to avoid probing it as
> raw), but opening it fails.

Yes, good idea. I've sent a patch for this too

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH v2] Add a gitlab-ci file for Continuous Integration testing on Gitlab

[Paolo Bonzini]

On 13/02/19 12:54, Thomas Huth wrote:
> This is very convenient for people like me who store their QEMU git trees
> on gitlab.com: Automatic CI pipelines are now run for each branch that is
> pushed to the server - useful for some extra-testing before sending PULL-
> requests for example. Since the runtime of the jobs is limited to 1h, the
> jobs are distributed into multiple pipelines - this way everything finishs
> fine within time (ca. 30 minutes currently).
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  An example can be seen here: https://gitlab.com/huth/qemu/pipelines/
> 
>  I'd really like to get this into the main QEMU repository, so that I don't
>  have to cherry-pick this patch onto my testing branches anymore each time
>  I want to test before sending a PULL request...

Very good idea.  In fact, I wouldn't mind "forcing" maintainers to place
their mirrors and signed tags on GitLab so that they get their pipelines
for free and with less setup hassles than GitHub+Travis.

Paolo

>  .gitlab-ci.yml | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  MAINTAINERS    |  5 ++++
>  2 files changed, 78 insertions(+)
>  create mode 100644 .gitlab-ci.yml
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> new file mode 100644
> index 0000000..79d02cf
> --- /dev/null
> +++ b/.gitlab-ci.yml
> @@ -0,0 +1,73 @@
> +before_script:
> + - apt-get update -qq
> + - apt-get install -y -qq flex bison libglib2.0-dev libpixman-1-dev genisoimage
> +
> +build-system1:
> + script:
> + - apt-get install -y -qq libgtk-3-dev libvte-dev nettle-dev libcacard-dev
> +      libusb-dev libvde-dev libspice-protocol-dev libgl1-mesa-dev
> + - ./configure --enable-werror --target-list="aarch64-softmmu alpha-softmmu
> +      cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu microblazeel-softmmu
> +      mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu sparc-softmmu"
> + - make -j2
> + - make -j2 check
> +
> +build-system2:
> + script:
> + - apt-get install -y -qq libsdl2-dev libgcrypt-dev libbrlapi-dev libaio-dev
> +      libfdt-dev liblzo2-dev librdmacm-dev libibverbs-dev libibumad-dev
> + - ./configure --enable-werror --target-list="tricore-softmmu unicore32-softmmu
> +      microblaze-softmmu mips-softmmu riscv32-softmmu s390x-softmmu sh4-softmmu
> +      sparc64-softmmu x86_64-softmmu xtensa-softmmu nios2-softmmu or1k-softmmu"
> + - make -j2
> + - make -j2 check
> +
> +build-disabled:
> + script:
> + - ./configure --enable-werror --disable-rdma --disable-slirp --disable-curl
> +      --disable-capstone --disable-live-block-migration --disable-glusterfs
> +      --disable-replication --disable-coroutine-pool --disable-smartcard
> +      --disable-guest-agent --disable-curses --disable-libxml2 --disable-tpm
> +      --disable-qom-cast-debug --disable-spice --disable-vhost-vsock
> +      --disable-vhost-net --disable-vhost-crypto --disable-vhost-user
> +      --target-list="i386-softmmu ppc64-softmmu mips64-softmmu i386-linux-user"
> + - make -j2
> + - make -j2 check-qtest SPEED=slow
> +
> +build-tcg-disabled:
> + script:
> + - apt-get install -y -qq clang libgtk-3-dev libbluetooth-dev libusb-dev
> + - ./configure --cc=clang --enable-werror --disable-tcg --audio-drv-list=""
> + - make -j2
> + - make check-unit
> + - make check-qapi-schema
> + - cd tests/qemu-iotests/
> + - ./check -raw 001 002 003 004 005 008 009 010 011 012 021 025 032 033 048
> +            052 063 077 086 101 104 106 113 147 148 150 151 152 157 159 160
> +            163 170 171 183 184 192 194 197 205 208 215 221 222 226 227 236
> + - ./check -qcow2 001 002 003 004 005 007 008 009 010 011 012 013 017 018 019
> +            020 021 022 024 025 027 028 029 031 032 033 034 035 036 037 038
> +            039 040 042 043 046 047 048 049 050 051 052 053 054 056 057 058
> +            060 061 062 063 065 066 067 068 069 071 072 073 074 079 080 082
> +            085 086 089 090 091 095 096 097 098 099 102 103 104 105 107 108
> +            110 111 114 117 120 122 124 126 127 129 130 132 133 134 137 138
> +            139 140 141 142 143 144 145 147 150 151 152 154 155 156 157 158
> +            161 165 170 172 174 176 177 179 184 186 187 190 192 194 195 196
> +            197 200 202 203 205 208 209 214 215 216 217 218 222 226 227 229 234
> +
> +build-user:
> + script:
> + - ./configure --enable-werror --disable-system --disable-guest-agent
> +               --disable-capstone --disable-slirp --disable-fdt
> + - make -j2
> + - make run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-linux-user
> +
> +build-clang:
> + script:
> + - apt-get install -y -qq clang libsdl2-dev
> +      xfslibs-dev libiscsi-dev libnfs-dev libseccomp-dev gnutls-dev librbd-dev
> + - ./configure --cc=clang --cxx=clang++ --enable-werror
> +      --target-list="alpha-softmmu arm-softmmu m68k-softmmu mips64-softmmu
> +                     ppc-softmmu s390x-softmmu x86_64-softmmu arm-linux-user"
> + - make -j2
> + - make -j2 check
> diff --git a/MAINTAINERS b/MAINTAINERS
> index a2da141..191138c 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -2484,6 +2484,11 @@ S: Maintained
>  F: .cirrus.yml
>  W: https://cirrus-ci.com/github/qemu/qemu
>  
> +GitLab Continuous Integration
> +M: Thomas Huth <thuth@redhat.com>
> +S: Maintained
> +F: .gitlab-ci.yml
> +
>  Guest Test Compilation Support
>  M: Alex Bennée <alex.bennee@linaro.org>
>  R: Philippe Mathieu-Daudé <f4bug@amsat.org>
> 

Re: [Qemu-devel] [PATCH v2] Add a gitlab-ci file for Continuous Integration testing on Gitlab

[Alex Bennée]

Thomas Huth <thuth@redhat.com> writes:

> This is very convenient for people like me who store their QEMU git trees
> on gitlab.com: Automatic CI pipelines are now run for each branch that is
> pushed to the server - useful for some extra-testing before sending PULL-
> requests for example. Since the runtime of the jobs is limited to 1h, the
> jobs are distributed into multiple pipelines - this way everything finishs
> fine within time (ca. 30 minutes currently).
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>

Queued to testing/next, thanks.

> ---
>  An example can be seen here: https://gitlab.com/huth/qemu/pipelines/
>
>  I'd really like to get this into the main QEMU repository, so that I don't
>  have to cherry-pick this patch onto my testing branches anymore each time
>  I want to test before sending a PULL request...
>
>  .gitlab-ci.yml | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  MAINTAINERS    |  5 ++++
>  2 files changed, 78 insertions(+)
>  create mode 100644 .gitlab-ci.yml
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> new file mode 100644
> index 0000000..79d02cf
> --- /dev/null
> +++ b/.gitlab-ci.yml
> @@ -0,0 +1,73 @@
> +before_script:
> + - apt-get update -qq
> + - apt-get install -y -qq flex bison libglib2.0-dev libpixman-1-dev genisoimage
> +
> +build-system1:
> + script:
> + - apt-get install -y -qq libgtk-3-dev libvte-dev nettle-dev libcacard-dev
> +      libusb-dev libvde-dev libspice-protocol-dev libgl1-mesa-dev
> + - ./configure --enable-werror --target-list="aarch64-softmmu alpha-softmmu
> +      cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu microblazeel-softmmu
> +      mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu sparc-softmmu"
> + - make -j2
> + - make -j2 check
> +
> +build-system2:
> + script:
> + - apt-get install -y -qq libsdl2-dev libgcrypt-dev libbrlapi-dev libaio-dev
> +      libfdt-dev liblzo2-dev librdmacm-dev libibverbs-dev libibumad-dev
> + - ./configure --enable-werror --target-list="tricore-softmmu unicore32-softmmu
> +      microblaze-softmmu mips-softmmu riscv32-softmmu s390x-softmmu sh4-softmmu
> +      sparc64-softmmu x86_64-softmmu xtensa-softmmu nios2-softmmu or1k-softmmu"
> + - make -j2
> + - make -j2 check
> +
> +build-disabled:
> + script:
> + - ./configure --enable-werror --disable-rdma --disable-slirp --disable-curl
> +      --disable-capstone --disable-live-block-migration --disable-glusterfs
> +      --disable-replication --disable-coroutine-pool --disable-smartcard
> +      --disable-guest-agent --disable-curses --disable-libxml2 --disable-tpm
> +      --disable-qom-cast-debug --disable-spice --disable-vhost-vsock
> +      --disable-vhost-net --disable-vhost-crypto --disable-vhost-user
> +      --target-list="i386-softmmu ppc64-softmmu mips64-softmmu i386-linux-user"
> + - make -j2
> + - make -j2 check-qtest SPEED=slow
> +
> +build-tcg-disabled:
> + script:
> + - apt-get install -y -qq clang libgtk-3-dev libbluetooth-dev libusb-dev
> + - ./configure --cc=clang --enable-werror --disable-tcg --audio-drv-list=""
> + - make -j2
> + - make check-unit
> + - make check-qapi-schema
> + - cd tests/qemu-iotests/
> + - ./check -raw 001 002 003 004 005 008 009 010 011 012 021 025 032 033 048
> +            052 063 077 086 101 104 106 113 147 148 150 151 152 157 159 160
> +            163 170 171 183 184 192 194 197 205 208 215 221 222 226 227 236
> + - ./check -qcow2 001 002 003 004 005 007 008 009 010 011 012 013 017 018 019
> +            020 021 022 024 025 027 028 029 031 032 033 034 035 036 037 038
> +            039 040 042 043 046 047 048 049 050 051 052 053 054 056 057 058
> +            060 061 062 063 065 066 067 068 069 071 072 073 074 079 080 082
> +            085 086 089 090 091 095 096 097 098 099 102 103 104 105 107 108
> +            110 111 114 117 120 122 124 126 127 129 130 132 133 134 137 138
> +            139 140 141 142 143 144 145 147 150 151 152 154 155 156 157 158
> +            161 165 170 172 174 176 177 179 184 186 187 190 192 194 195 196
> +            197 200 202 203 205 208 209 214 215 216 217 218 222 226 227 229 234
> +
> +build-user:
> + script:
> + - ./configure --enable-werror --disable-system --disable-guest-agent
> +               --disable-capstone --disable-slirp --disable-fdt
> + - make -j2
> + - make run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-linux-user
> +
> +build-clang:
> + script:
> + - apt-get install -y -qq clang libsdl2-dev
> +      xfslibs-dev libiscsi-dev libnfs-dev libseccomp-dev gnutls-dev librbd-dev
> + - ./configure --cc=clang --cxx=clang++ --enable-werror
> +      --target-list="alpha-softmmu arm-softmmu m68k-softmmu mips64-softmmu
> +                     ppc-softmmu s390x-softmmu x86_64-softmmu arm-linux-user"
> + - make -j2
> + - make -j2 check
> diff --git a/MAINTAINERS b/MAINTAINERS
> index a2da141..191138c 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -2484,6 +2484,11 @@ S: Maintained
>  F: .cirrus.yml
>  W: https://cirrus-ci.com/github/qemu/qemu
>
> +GitLab Continuous Integration
> +M: Thomas Huth <thuth@redhat.com>
> +S: Maintained
> +F: .gitlab-ci.yml
> +
>  Guest Test Compilation Support
>  M: Alex Bennée <alex.bennee@linaro.org>
>  R: Philippe Mathieu-Daudé <f4bug@amsat.org>


--
Alex Bennée