questions about kvm test

questions about kvm test

[Zhu Yijun]

Hi all,

  I am getting start with kvm, and interest in kvm auto test. Currently I have found two related projects, one is kvm-unit-tests, and the other is avocado-vt.

  So my question is:

  1) If there is relationship between above two test suite? We should use them alone or together?

  2) Is there any other kvm test suite available? And which one is mainly used at present?

  Thanks in advance!

Zhu Yijun

Re: questions about kvm test

[Radim Krčmář]

2016-02-24 17:45+0800, Zhu Yijun:
> Hi all,
> 
>   I am getting start with kvm, and interest in kvm auto test. Currently I have found two related projects, one is kvm-unit-tests, and the other is avocado-vt.
> 
>   So my question is:
> 
>   1) If there is relationship between above two test suite?

They have different goals, but it makes sense to run kvm-unit-tests
inside avocado.

kvm-unit-tests is a collection of single-purpose guests that test some
parts of KVM (the kernel).  kvm-unit-tests also contain a simple
framework to let you run those guests without hassle.

avocado-vt is a module for avocado testing framework that allows you to
conveniently automate tests about and inside guests.  avocado-vt also
collects tests for guest that run on KVM.

>   1)                                                        We should use them alone or together?

If you would be using avocado anyway, then it's a good idea to run
kvm-unit-tests from it, instead of running them separately.

(I presume that you'll run something, like jenkins, on top of avocado,
 so having just one set of results will be simpler.)

>   2) Is there any other kvm test suite available?

That depends on what you count and what you actually want to test.
Working with KVM guests is testing KVM. :)

Virtualization mostly emulates real hardware, so pretty much any
hardware test suite is worthwhile and other components in your virt
stack (QEMU, libvirt, ...) have a specialized test suites that also
exercise KVM.

AFAIK avocado(-vt) is the latest try at THE testing framework, so using
it should be more convenient than writing your own framework to test a
huge cartesian product of variables that KVM+QEMU+libvirt+... provides.

>   2)                                              And which one is mainly used at present?

I guess that developers mostly use the test suite for their project or
custom tests/benchmarks that cover their current work.

Autotest maybe still is the most popular framework, but avocado
obsoletes it.

Btw. the avocado team will be able to provide much better information.

Re: questions about kvm test

[Zhu Yijun]

Thanks for giving me so much information!

Currently I have installed redhat 7.0 on my machine, and run kun-unit-tests. However, several tests case failed, such as apic, ioapic, smap, pku, idt_test.

So should I use the latest kvm-kmod and qemu? Or is there something requirement on hardware?

Here is my cpu info showed by /proc/cpuinfo:

model name      : Intel(R) Xeon(R) CPU E5-2643 0 @ 3.30GHz

flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx
smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm t
pr_shadow vnmi flexpriority ept vpid

I had look the error info in test.log, and will check the source code these days. Thanks again!

Zhu Yijun


On 2016/2/26 22:26, Radim Krčmář wrote:

> 2016-02-24 17:45+0800, Zhu Yijun:
>> Hi all,
>>
>>   I am getting start with kvm, and interest in kvm auto test. Currently I have found two related projects, one is kvm-unit-tests, and the other is avocado-vt.
>>
>>   So my question is:
>>
>>   1) If there is relationship between above two test suite?
> 
> They have different goals, but it makes sense to run kvm-unit-tests
> inside avocado.
> 
> kvm-unit-tests is a collection of single-purpose guests that test some
> parts of KVM (the kernel).  kvm-unit-tests also contain a simple
> framework to let you run those guests without hassle.
> 
> avocado-vt is a module for avocado testing framework that allows you to
> conveniently automate tests about and inside guests.  avocado-vt also
> collects tests for guest that run on KVM.
> 
>>   1)                                                        We should use them alone or together?
> 
> If you would be using avocado anyway, then it's a good idea to run
> kvm-unit-tests from it, instead of running them separately.
> 
> (I presume that you'll run something, like jenkins, on top of avocado,
>  so having just one set of results will be simpler.)
> 
>>   2) Is there any other kvm test suite available?
> 
> That depends on what you count and what you actually want to test.
> Working with KVM guests is testing KVM. :)
> 
> Virtualization mostly emulates real hardware, so pretty much any
> hardware test suite is worthwhile and other components in your virt
> stack (QEMU, libvirt, ...) have a specialized test suites that also
> exercise KVM.
> 
> AFAIK avocado(-vt) is the latest try at THE testing framework, so using
> it should be more convenient than writing your own framework to test a
> huge cartesian product of variables that KVM+QEMU+libvirt+... provides.
> 
>>   2)                                              And which one is mainly used at present?
> 
> I guess that developers mostly use the test suite for their project or
> custom tests/benchmarks that cover their current work.
> 
> Autotest maybe still is the most popular framework, but avocado
> obsoletes it.
> 
> Btw. the avocado team will be able to provide much better information.
> 
> .
> 

Re: questions about kvm test

[Amos Jianjun Kong]

On Sat, Feb 27, 2016 at 9:01 AM, Zhu Yijun <zhuyijun@huawei.com> wrote:
> Thanks for giving me so much information!
>
> Currently I have installed redhat 7.0 on my machine, and run kun-unit-tests. However, several tests case failed, such as apic, ioapic, smap, pku, idt_test.
>
> So should I use the latest kvm-kmod and qemu? Or is there something requirement on hardware?
>
> Here is my cpu info showed by /proc/cpuinfo:
>
> model name      : Intel(R) Xeon(R) CPU E5-2643 0 @ 3.30GHz
>
> flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
> pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx
> smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm t
> pr_shadow vnmi flexpriority ept vpid
>
> I had look the error info in test.log, and will check the source code these days. Thanks again!
>
> Zhu Yijun

Hi Yijun,

It's ok to run latest kvm-unit-test code with released RHEL (Red Hat
Enterprise Linux). But some case doesn't work, because of the
compatibility. This Upstream test project wasn't maintained to always
work with RHEL.

Document : Running QEMU unittests with virt-test
http://avocado-vt.readthedocs.org/en/0.29.0/extra/RunQemuUnittests.html

You can attached your log here.


> On 2016/2/26 22:26, Radim Krčmář wrote:
>
>> 2016-02-24 17:45+0800, Zhu Yijun:
>>> Hi all,
>>>
>>>   I am getting start with kvm, and interest in kvm auto test. Currently I have found two related projects, one is kvm-unit-tests, and the other is avocado-vt.
>>>
>>>   So my question is:
>>>
>>>   1) If there is relationship between above two test suite?
>>
>> They have different goals, but it makes sense to run kvm-unit-tests
>> inside avocado.
>>
>> kvm-unit-tests is a collection of single-purpose guests that test some
>> parts of KVM (the kernel).  kvm-unit-tests also contain a simple
>> framework to let you run those guests without hassle.
>>
>> avocado-vt is a module for avocado testing framework that allows you to
>> conveniently automate tests about and inside guests.  avocado-vt also
>> collects tests for guest that run on KVM.
>>
>>>   1)                                                        We should use them alone or together?
>>
>> If you would be using avocado anyway, then it's a good idea to run
>> kvm-unit-tests from it, instead of running them separately.
>>
>> (I presume that you'll run something, like jenkins, on top of avocado,
>>  so having just one set of results will be simpler.)
>>
>>>   2) Is there any other kvm test suite available?
>>
>> That depends on what you count and what you actually want to test.
>> Working with KVM guests is testing KVM. :)
>>
>> Virtualization mostly emulates real hardware, so pretty much any
>> hardware test suite is worthwhile and other components in your virt
>> stack (QEMU, libvirt, ...) have a specialized test suites that also
>> exercise KVM.
>>
>> AFAIK avocado(-vt) is the latest try at THE testing framework, so using
>> it should be more convenient than writing your own framework to test a
>> huge cartesian product of variables that KVM+QEMU+libvirt+... provides.
>>
>>>   2)                                              And which one is mainly used at present?
>>
>> I guess that developers mostly use the test suite for their project or
>> custom tests/benchmarks that cover their current work.
>>
>> Autotest maybe still is the most popular framework, but avocado
>> obsoletes it.
>>
>> Btw. the avocado team will be able to provide much better information.