[Qemu-devel] [PATCH 0/2] make check: Add qemu-iotests subset

[Qemu-devel] [PATCH 0/2] make check: Add qemu-iotests subset

[Kevin Wolf]

Kevin Wolf (2):
  qemu-iotests: Mark some tests as quick
  make check: Add qemu-iotests subset

 tests/Makefile           |    1 +
 tests/qemu-iotests.sh    |   18 ++++++++++++++++++
 tests/qemu-iotests/group |   24 ++++++++++++------------
 3 files changed, 31 insertions(+), 12 deletions(-)
 create mode 100755 tests/qemu-iotests.sh

-- 
1.7.6.5

[Qemu-devel] [PATCH 1/2] qemu-iotests: Mark some tests as quick

[Kevin Wolf]

This creates a new test group 'quick' for some test case that take at
most a couple of seconds each, so that the group can be run during a
quick 'make check'

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 tests/qemu-iotests/group |   24 ++++++++++++------------
 1 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index fcf869d..b549f10 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -8,32 +8,32 @@
 # test-group association ... one line per test
 #
 001 rw auto
-002 rw auto
+002 rw auto quick
 003 rw auto
-004 rw auto
+004 rw auto quick
 005 img auto
 006 img auto
 007 snapshot auto
 008 rw auto
 009 rw auto
 010 rw auto
-011 rw auto
-012 auto
+011 rw auto quick
+012 auto quick
 013 rw auto
 014 rw auto
 015 rw snapshot auto
-016 rw auto
-017 rw backing auto
+016 rw auto quick
+017 rw backing auto quick
 018 rw backing auto
-019 rw backing auto
-020 rw backing auto
+019 rw backing auto quick
+020 rw backing auto quick
 021 io auto
 022 rw snapshot auto
 023 rw auto
-024 rw backing auto
-025 rw auto
+024 rw backing auto quick
+025 rw auto quick
 026 rw blkdbg auto
-027 rw auto
+027 rw auto quick
 028 rw backing auto
-029 rw auto
+029 rw auto quick
 030 rw auto
-- 
1.7.6.5

[Qemu-devel] [PATCH 2/2] make check: Add qemu-iotests subset

[Kevin Wolf]

Run the 'quick' group from qemu-iotests during 'make check'.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 tests/Makefile        |    1 +
 tests/qemu-iotests.sh |   18 ++++++++++++++++++
 2 files changed, 19 insertions(+), 0 deletions(-)
 create mode 100755 tests/qemu-iotests.sh

diff --git a/tests/Makefile b/tests/Makefile
index 74b29dc..155ecbd 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -1,6 +1,7 @@
 CHECKS = check-qdict check-qfloat check-qint check-qstring check-qlist
 CHECKS += check-qjson test-qmp-output-visitor test-qmp-input-visitor
 CHECKS += test-string-input-visitor test-string-output-visitor test-coroutine
+CHECKS += tests/qemu-iotests.sh
 
 check-qint.o check-qstring.o check-qdict.o check-qlist.o check-qfloat.o check-qjson.o test-coroutine.o: $(GENERATED_HEADERS)
 
diff --git a/tests/qemu-iotests.sh b/tests/qemu-iotests.sh
new file mode 100755
index 0000000..af99635
--- /dev/null
+++ b/tests/qemu-iotests.sh
@@ -0,0 +1,18 @@
+#!/bin/sh
+
+# We don't know which of the system emulator binaries there is (or if there is
+# any at all), so the 'quick' group doesn't contain any tests that require
+# running qemu proper. Assign a fake binary name so that qemu-iotests doesn't
+# complain about the missing binary.
+export QEMU_PROG="this_should_be_unused"
+
+export QEMU_IMG_PROG="$(pwd)/qemu-img"
+export QEMU_IO_PROG="$(pwd)/qemu-io"
+
+cd tests/qemu-iotests
+
+ret=0
+./check -T -nocache -raw -g quick || ret=1
+./check -T -nocache -qcow2 -g quick || ret=1
+
+exit $ret
-- 
1.7.6.5

Re: [Qemu-devel] [PATCH 2/2] make check: Add qemu-iotests subset

[Stefan Hajnoczi]

On Thu, Mar 8, 2012 at 11:32 AM, Kevin Wolf <kwolf@redhat.com> wrote:
> Run the 'quick' group from qemu-iotests during 'make check'.
>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
>  tests/Makefile        |    1 +
>  tests/qemu-iotests.sh |   18 ++++++++++++++++++

I think tests/qemu-iotests-quick.sh would be a clearer name since this
is not a general-purpose qemu-iotests wrapper - it only works when
called from QEMU's root directory and only invokes the 'quick' group.

> +./check -T -nocache -raw -g quick || ret=1
> +./check -T -nocache -qcow2 -g quick || ret=1

Some love for qed?  It adds 10s on my box but we're already up at 26s.

Stefan

Re: [Qemu-devel] [PATCH 2/2] make check: Add qemu-iotests subset

[Kevin Wolf]

Am 08.03.2012 16:51, schrieb Stefan Hajnoczi:
> On Thu, Mar 8, 2012 at 11:32 AM, Kevin Wolf <kwolf@redhat.com> wrote:
>> Run the 'quick' group from qemu-iotests during 'make check'.
>>
>> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
>> ---
>>  tests/Makefile        |    1 +
>>  tests/qemu-iotests.sh |   18 ++++++++++++++++++
> 
> I think tests/qemu-iotests-quick.sh would be a clearer name since this
> is not a general-purpose qemu-iotests wrapper - it only works when
> called from QEMU's root directory and only invokes the 'quick' group.

Ok, I'll rename it.

>> +./check -T -nocache -raw -g quick || ret=1
>> +./check -T -nocache -qcow2 -g quick || ret=1
> 
> Some love for qed?  It adds 10s on my box but we're already up at 26s.

I'm used to high expectations, but loving QED is a bit too much... ;-)

Not sure what to do about all the formats. Ideally we would test all of
them (at least those with better implementations, VMDK, VHD, VDI), but
that would definitely take too long. I hope that in the not too distant
future, QED will have a similar position to qcow1, but we may consider
adding it for now.

However, 'make check' is really the quick test that you run when you
don't change anything in the image formats. If you do, you should do a
full qemu-iotests run. So the important question is whether QED is
likely to reveal any breakage outside block/* that the qcow2 tests
wouldn't find.

With the same reason we can probably drop the raw tests: qcow2 is the
most featureful format, so any breakage should be visible there. Most of
raw-posix.c should be part of the qcow2 tests already.

Kevin

Re: [Qemu-devel] [PATCH 2/2] make check: Add qemu-iotests subset

[Stefan Hajnoczi]

On Thu, Mar 8, 2012 at 4:10 PM, Kevin Wolf <kwolf@redhat.com> wrote:
> Am 08.03.2012 16:51, schrieb Stefan Hajnoczi:
>> On Thu, Mar 8, 2012 at 11:32 AM, Kevin Wolf <kwolf@redhat.com> wrote:
>>> Run the 'quick' group from qemu-iotests during 'make check'.
>>>
>>> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
>>> ---
>>>  tests/Makefile        |    1 +
>>>  tests/qemu-iotests.sh |   18 ++++++++++++++++++
>>
>> I think tests/qemu-iotests-quick.sh would be a clearer name since this
>> is not a general-purpose qemu-iotests wrapper - it only works when
>> called from QEMU's root directory and only invokes the 'quick' group.
>
> Ok, I'll rename it.
>
>>> +./check -T -nocache -raw -g quick || ret=1
>>> +./check -T -nocache -qcow2 -g quick || ret=1
>>
>> Some love for qed?  It adds 10s on my box but we're already up at 26s.
>
> I'm used to high expectations, but loving QED is a bit too much... ;-)
>
> Not sure what to do about all the formats. Ideally we would test all of
> them (at least those with better implementations, VMDK, VHD, VDI), but
> that would definitely take too long. I hope that in the not too distant
> future, QED will have a similar position to qcow1, but we may consider
> adding it for now.
>
> However, 'make check' is really the quick test that you run when you
> don't change anything in the image formats. If you do, you should do a
> full qemu-iotests run. So the important question is whether QED is
> likely to reveal any breakage outside block/* that the qcow2 tests
> wouldn't find.
>
> With the same reason we can probably drop the raw tests: qcow2 is the
> most featureful format, so any breakage should be visible there. Most of
> raw-posix.c should be part of the qcow2 tests already.

In another thread I think Anthony suggested check-block.  So if we
just test qcow2 here to make sure the block layer works, then we can
do a full run in check-block with raw, qed, and friends.

Stefan

Re: [Qemu-devel] [PATCH 2/2] make check: Add qemu-iotests subset

[Kevin Wolf]

Am 08.03.2012 17:52, schrieb Stefan Hajnoczi:
> On Thu, Mar 8, 2012 at 4:10 PM, Kevin Wolf <kwolf@redhat.com> wrote:
>> Am 08.03.2012 16:51, schrieb Stefan Hajnoczi:
>>> On Thu, Mar 8, 2012 at 11:32 AM, Kevin Wolf <kwolf@redhat.com> wrote:
>>>> Run the 'quick' group from qemu-iotests during 'make check'.
>>>>
>>>> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
>>>> ---
>>>>  tests/Makefile        |    1 +
>>>>  tests/qemu-iotests.sh |   18 ++++++++++++++++++
>>>
>>> I think tests/qemu-iotests-quick.sh would be a clearer name since this
>>> is not a general-purpose qemu-iotests wrapper - it only works when
>>> called from QEMU's root directory and only invokes the 'quick' group.
>>
>> Ok, I'll rename it.
>>
>>>> +./check -T -nocache -raw -g quick || ret=1
>>>> +./check -T -nocache -qcow2 -g quick || ret=1
>>>
>>> Some love for qed?  It adds 10s on my box but we're already up at 26s.
>>
>> I'm used to high expectations, but loving QED is a bit too much... ;-)
>>
>> Not sure what to do about all the formats. Ideally we would test all of
>> them (at least those with better implementations, VMDK, VHD, VDI), but
>> that would definitely take too long. I hope that in the not too distant
>> future, QED will have a similar position to qcow1, but we may consider
>> adding it for now.
>>
>> However, 'make check' is really the quick test that you run when you
>> don't change anything in the image formats. If you do, you should do a
>> full qemu-iotests run. So the important question is whether QED is
>> likely to reveal any breakage outside block/* that the qcow2 tests
>> wouldn't find.
>>
>> With the same reason we can probably drop the raw tests: qcow2 is the
>> most featureful format, so any breakage should be visible there. Most of
>> raw-posix.c should be part of the qcow2 tests already.
> 
> In another thread I think Anthony suggested check-block.  So if we
> just test qcow2 here to make sure the block layer works, then we can
> do a full run in check-block with raw, qed, and friends.

Yes, I think that would make a lot of sense. Do you want me to include a
check-block in v2 (that would be similar to qemu-iotests-quick.sh, just
without -g quick and for more formats) or should we introduce something
more sophisticated later?

Kevin

Re: [Qemu-devel] [PATCH 2/2] make check: Add qemu-iotests subset

[Stefan Hajnoczi]

On Thu, Mar 8, 2012 at 5:01 PM, Kevin Wolf <kwolf@redhat.com> wrote:
> Am 08.03.2012 17:52, schrieb Stefan Hajnoczi:
>> On Thu, Mar 8, 2012 at 4:10 PM, Kevin Wolf <kwolf@redhat.com> wrote:
>>> Am 08.03.2012 16:51, schrieb Stefan Hajnoczi:
>>>> On Thu, Mar 8, 2012 at 11:32 AM, Kevin Wolf <kwolf@redhat.com> wrote:
>>>>> Run the 'quick' group from qemu-iotests during 'make check'.
>>>>>
>>>>> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
>>>>> ---
>>>>>  tests/Makefile        |    1 +
>>>>>  tests/qemu-iotests.sh |   18 ++++++++++++++++++
>>>>
>>>> I think tests/qemu-iotests-quick.sh would be a clearer name since this
>>>> is not a general-purpose qemu-iotests wrapper - it only works when
>>>> called from QEMU's root directory and only invokes the 'quick' group.
>>>
>>> Ok, I'll rename it.
>>>
>>>>> +./check -T -nocache -raw -g quick || ret=1
>>>>> +./check -T -nocache -qcow2 -g quick || ret=1
>>>>
>>>> Some love for qed?  It adds 10s on my box but we're already up at 26s.
>>>
>>> I'm used to high expectations, but loving QED is a bit too much... ;-)
>>>
>>> Not sure what to do about all the formats. Ideally we would test all of
>>> them (at least those with better implementations, VMDK, VHD, VDI), but
>>> that would definitely take too long. I hope that in the not too distant
>>> future, QED will have a similar position to qcow1, but we may consider
>>> adding it for now.
>>>
>>> However, 'make check' is really the quick test that you run when you
>>> don't change anything in the image formats. If you do, you should do a
>>> full qemu-iotests run. So the important question is whether QED is
>>> likely to reveal any breakage outside block/* that the qcow2 tests
>>> wouldn't find.
>>>
>>> With the same reason we can probably drop the raw tests: qcow2 is the
>>> most featureful format, so any breakage should be visible there. Most of
>>> raw-posix.c should be part of the qcow2 tests already.
>>
>> In another thread I think Anthony suggested check-block.  So if we
>> just test qcow2 here to make sure the block layer works, then we can
>> do a full run in check-block with raw, qed, and friends.
>
> Yes, I think that would make a lot of sense. Do you want me to include a
> check-block in v2 (that would be similar to qemu-iotests-quick.sh, just
> without -g quick and for more formats) or should we introduce something
> more sophisticated later?

If you can add it right away that would be convenient - it will save
us from having to run qemu-iotests ./check manually.

Stefan