[Qemu-devel] How to use qtest

[Qemu-devel] How to use qtest

[Gonglei (Arei)]

Hi,

I know qemu has a test framework, and I get some information
from http://wiki.qemu.org/Features/QTest

Recently I want to use qtest for some modules, but I have no idea
how to run those tests. Such as tests/ e1000-test

Any guidance? Or how to? Thanks in advance!


Best regards,
-Gonglei

Re: [Qemu-devel] How to use qtest

[Markus Armbruster]

"Gonglei (Arei)" <arei.gonglei@huawei.com> writes:

> Hi,
>
> I know qemu has a test framework, and I get some information
> from http://wiki.qemu.org/Features/QTest
>
> Recently I want to use qtest for some modules, but I have no idea
> how to run those tests. Such as tests/ e1000-test
>
> Any guidance? Or how to? Thanks in advance!
>
>
> Best regards,
> -Gonglei

Try "make check-help".

Re: [Qemu-devel] How to use qtest

[Gonglei (Arei)]

> -----Original Message-----
> From: Markus Armbruster [mailto:armbru@redhat.com]
> Sent: Wednesday, May 14, 2014 3:34 PM
> To: Gonglei (Arei)
> Cc: qemu-devel@nongnu.org; Andreas Färber
> Subject: Re: [Qemu-devel] How to use qtest
> 
> "Gonglei (Arei)" <arei.gonglei@huawei.com> writes:
> 
> > Hi,
> >
> > I know qemu has a test framework, and I get some information
> > from http://wiki.qemu.org/Features/QTest
> >
> > Recently I want to use qtest for some modules, but I have no idea
> > how to run those tests. Such as tests/ e1000-test
> >
> > Any guidance? Or how to? Thanks in advance!
> >
> >
> > Best regards,
> > -Gonglei
> 
> Try "make check-help".

Thanks!
Is there two steps for qtest?
First, "make check" for running tests, 
Second, make "check-report.html" for getting results.

If I want to write some qtest by myself, any guidance? Thanks.

Best regards,
-Gonglei

Re: [Qemu-devel] How to use qtest

[Andreas Färber]

Hi,

Am 14.05.2014 09:50, schrieb Gonglei (Arei):
>> -----Original Message-----
>> From: Markus Armbruster [mailto:armbru@redhat.com]
>> Sent: Wednesday, May 14, 2014 3:34 PM
>> To: Gonglei (Arei)
>> Cc: qemu-devel@nongnu.org; Andreas Färber
>> Subject: Re: [Qemu-devel] How to use qtest
>>
>> "Gonglei (Arei)" <arei.gonglei@huawei.com> writes:
>>
>>> I know qemu has a test framework, and I get some information
>>> from http://wiki.qemu.org/Features/QTest
>>>
>>> Recently I want to use qtest for some modules, but I have no idea
>>> how to run those tests. Such as tests/ e1000-test
>>>
>>> Any guidance? Or how to? Thanks in advance!
>>
>> Try "make check-help".
> 
> Thanks!
> Is there two steps for qtest?
> First, "make check" for running tests, 
> Second, make "check-report.html" for getting results.

No, "make check V=1" shows you the progress as well. The HTML or XML
reports are only necessary for some more verbose info/warning messages
and for automated reporting purposes.

> If I want to write some qtest by myself, any guidance? Thanks.

Use out-of-tree builds. One for your regular development and testing,
one for gcov coverage testing. The latter (--enable-gcov) will dump
statistics about test coverage and allow you to verify you got the paths
for new tests right (it's not the source code paths but the paths to the
.o files actually, with target-i386 as placeholder for target-*).

There's a lot of stubs that I've been adding recently that you can copy
(see git history for how to add them to Makefile); for functional tests
see libqtest.h and the libqos/ subdirectory and grep for existing tests
that use those functions. More concrete questions will be easier to
answer. :)

Thanks in advance for your efforts!

Best regards,

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] How to use qtest

[Gonglei (Arei)]

Hi, Andreas

> -----Original Message-----
> From: Andreas Färber [mailto:afaerber@suse.de]
> Sent: Wednesday, May 14, 2014 5:06 PM
> To: Gonglei (Arei)
> Cc: Markus Armbruster; qemu-devel@nongnu.org
> Subject: Re: [Qemu-devel] How to use qtest
> 
> Hi,
> 
> Am 14.05.2014 09:50, schrieb Gonglei (Arei):
> >> -----Original Message-----
> >> From: Markus Armbruster [mailto:armbru@redhat.com]
> >> Sent: Wednesday, May 14, 2014 3:34 PM
> >> To: Gonglei (Arei)
> >> Cc: qemu-devel@nongnu.org; Andreas Färber
> >> Subject: Re: [Qemu-devel] How to use qtest
> >>
> >> "Gonglei (Arei)" <arei.gonglei@huawei.com> writes:
> >>
> >>> I know qemu has a test framework, and I get some information
> >>> from http://wiki.qemu.org/Features/QTest
> >>>
> >>> Recently I want to use qtest for some modules, but I have no idea
> >>> how to run those tests. Such as tests/ e1000-test
> >>>
> >>> Any guidance? Or how to? Thanks in advance!
> >>
> >> Try "make check-help".
> >
> > Thanks!
> > Is there two steps for qtest?
> > First, "make check" for running tests,
> > Second, make "check-report.html" for getting results.
> 
> No, "make check V=1" shows you the progress as well. The HTML or XML
> reports are only necessary for some more verbose info/warning messages
> and for automated reporting purposes.
> 
Good. Thanks!

> > If I want to write some qtest by myself, any guidance? Thanks.
> 
> Use out-of-tree builds. One for your regular development and testing,
> one for gcov coverage testing. The latter (--enable-gcov) will dump
> statistics about test coverage and allow you to verify you got the paths
> for new tests right (it's not the source code paths but the paths to the
> .o files actually, with target-i386 as placeholder for target-*).
> 
> There's a lot of stubs that I've been adding recently that you can copy
> (see git history for how to add them to Makefile); for functional tests
> see libqtest.h and the libqos/ subdirectory and grep for existing tests
> that use those functions. 
>
Okay, I have noticed that patch sets you pulled.

>More concrete questions will be easier to answer. :)
> 
Yep. Thank you so much!


Best regards,
-Gonglei

Re: [Qemu-devel] How to use qtest

[Markus Armbruster]

"Gonglei (Arei)" <arei.gonglei@huawei.com> writes:

>> -----Original Message-----
>> From: Markus Armbruster [mailto:armbru@redhat.com]
>> Sent: Wednesday, May 14, 2014 3:34 PM
>> To: Gonglei (Arei)
>> Cc: qemu-devel@nongnu.org; Andreas Färber
>> Subject: Re: [Qemu-devel] How to use qtest
>> 
>> "Gonglei (Arei)" <arei.gonglei@huawei.com> writes:
>> 
>> > Hi,
>> >
>> > I know qemu has a test framework, and I get some information
>> > from http://wiki.qemu.org/Features/QTest
>> >
>> > Recently I want to use qtest for some modules, but I have no idea
>> > how to run those tests. Such as tests/ e1000-test
>> >
>> > Any guidance? Or how to? Thanks in advance!
>> >
>> >
>> > Best regards,
>> > -Gonglei
>> 
>> Try "make check-help".
>
> Thanks!
> Is there two steps for qtest?
> First, "make check" for running tests, 
> Second, make "check-report.html" for getting results.

I run just "make check".  Failing tests fail the make, which is usually
all I need.

> If I want to write some qtest by myself, any guidance? Thanks.

You figure out it from existing tests and libqtest.h.  There's no
tutorial, sorry.

To find existing tests, try "git-grep libqtest.h tests".  Which ones to
read first depends on what kind of test you want to write.