native kvm tool hrtimer problem

native kvm tool hrtimer problem

[Daniele Carollo]

Hi all,
for an university study, I'm doing some network test between two vm
using native linux kvm tool and connected via tap/virtio/vhost.
When I run my script (several consecutive iperf tcp and udp
execution), the first vm freeze with a message like: "hrtimer:
interrupt took * ns".
Is this a bug? (In order to complete the test i had to set the number
of guest cpu to 1)

Do you know what is the expected throughput between two vm using virtio/vhost?

Thanks,
Daniele

Re: native kvm tool hrtimer problem

[Pekka Enberg]

On Thu, Mar 8, 2012 at 4:32 PM, Daniele Carollo <carollo.dani@gmail.com> wrote:
> for an university study, I'm doing some network test between two vm
> using native linux kvm tool and connected via tap/virtio/vhost.
> When I run my script (several consecutive iperf tcp and udp
> execution), the first vm freeze with a message like: "hrtimer:
> interrupt took * ns".
> Is this a bug? (In order to complete the test i had to set the number
> of guest cpu to 1)
>
> Do you know what is the expected throughput between two vm using virtio/vhost?

I don't really use the tap interface so lets CC Asias. Which guest
kernel are you using, btw?

Re: native kvm tool hrtimer problem

[Cyrill Gorcunov]

On Thu, Mar 08, 2012 at 05:11:16PM +0200, Pekka Enberg wrote:
> On Thu, Mar 8, 2012 at 4:32 PM, Daniele Carollo <carollo.dani@gmail.com> wrote:
> > for an university study, I'm doing some network test between two vm
> > using native linux kvm tool and connected via tap/virtio/vhost.
> > When I run my script (several consecutive iperf tcp and udp
> > execution), the first vm freeze with a message like: "hrtimer:
> > interrupt took * ns".
> > Is this a bug? (In order to complete the test i had to set the number
> > of guest cpu to 1)
> >
> > Do you know what is the expected throughput between two vm using virtio/vhost?
> 
> I don't really use the tap interface so lets CC Asias. Which guest
> kernel are you using, btw?
> 

Yup, Asias was using it, if my memory doesn't betray me. Also both -- host
and guest kernel versions might be useful to know. iirc we were emulating
rtc only while anything else passes through to kvm kernel driver.

	Cyrill

Re: native kvm tool hrtimer problem

[Daniele Carollo]

Il giorno 08 marzo 2012 16:13, Cyrill Gorcunov <gorcunov@openvz.org> ha scritto:
>
> On Thu, Mar 08, 2012 at 05:11:16PM +0200, Pekka Enberg wrote:
> > On Thu, Mar 8, 2012 at 4:32 PM, Daniele Carollo <carollo.dani@gmail.com>
> > wrote:
> > > for an university study, I'm doing some network test between two vm
> > > using native linux kvm tool and connected via tap/virtio/vhost.
> > > When I run my script (several consecutive iperf tcp and udp
> > > execution), the first vm freeze with a message like: "hrtimer:
> > > interrupt took * ns".
> > > Is this a bug? (In order to complete the test i had to set the number
> > > of guest cpu to 1)
> > >
> > > Do you know what is the expected throughput between two vm using
> > > virtio/vhost?
> >
> > I don't really use the tap interface so lets CC Asias. Which guest
> > kernel are you using, btw?
> >
>
> Yup, Asias was using it, if my memory doesn't betray me. Also both -- host
> and guest kernel versions might be useful to know. iirc we were emulating
> rtc only while anything else passes through to kvm kernel driver.
>
>        Cyrill

As guest I'm using debian 6.0 squeeze found here
http://people.debian.org/~aurel32/qemu/i386/debian_squeeze_i386_standard.qcow2
while on the host I'm using Opensuse 11.2 with kernel 3.3.0-rc1KVM
gitted from here http://git.kernel.org/pub/scm/virt/kvm/kvm.git

Daniele

Re: native kvm tool hrtimer problem

[Cyrill Gorcunov]

On Thu, Mar 08, 2012 at 04:31:17PM +0100, Daniele Carollo wrote:
> > > I don't really use the tap interface so lets CC Asias. Which guest
> > > kernel are you using, btw?
> > >
> >
> > Yup, Asias was using it, if my memory doesn't betray me. Also both -- host
> > and guest kernel versions might be useful to know. iirc we were emulating
> > rtc only while anything else passes through to kvm kernel driver.
> >
> >        Cyrill
> 
> As guest I'm using debian 6.0 squeeze found here
> http://people.debian.org/~aurel32/qemu/i386/debian_squeeze_i386_standard.qcow2
> while on the host I'm using Opensuse 11.2 with kernel 3.3.0-rc1KVM
> gitted from here http://git.kernel.org/pub/scm/virt/kvm/kvm.git
> 

OK, lets see what Asias say.

	Cyrill

Re: native kvm tool hrtimer problem

[Asias He]

On 03/08/2012 11:51 PM, Cyrill Gorcunov wrote:
> On Thu, Mar 08, 2012 at 04:31:17PM +0100, Daniele Carollo wrote:
>>>> I don't really use the tap interface so lets CC Asias. Which guest
>>>> kernel are you using, btw?
>>>>
>>>
>>> Yup, Asias was using it, if my memory doesn't betray me. Also both -- host
>>> and guest kernel versions might be useful to know. iirc we were emulating
>>> rtc only while anything else passes through to kvm kernel driver.
>>>
>>>        Cyrill
>>
>> As guest I'm using debian 6.0 squeeze found here
>> http://people.debian.org/~aurel32/qemu/i386/debian_squeeze_i386_standard.qcow2
>> while on the host I'm using Opensuse 11.2 with kernel 3.3.0-rc1KVM
>> gitted from here http://git.kernel.org/pub/scm/virt/kvm/kvm.git
>>
> 
> OK, lets see what Asias say.

Hi, Daniele

The guest should be fine, I use that image as well.

Can you post the command line which you started the kvm tool?
Is tap/virtio or vhost/virtio working before the iperf stress testing,
like ping guestA in guesgB?

Can you try it on host with stock kernel from your distro?

-- 
Asias He

Re: native kvm tool hrtimer problem

[Daniele Carollo]

The command I use is something like:
sudo /home/kvm/linux-kvm/tools/kvm/lkvm run -p "root=/dev/vda1" -d
/home/kvm/Internato/debian_squeeze_i386_standard.raw -n
vhost=on,guest_mac=00:16:35:AF:94:4B,guest_ip=192.168.0.3,host_ip=192.168.0.9

and then in order to connect the tap interface to the host bridge:
/usr/bin/sudo /sbin/brctl addif br0 tap0

Usually the error comes up when the vm is sendig udp packets to the second vm.
It's possible to reobtain the control of the console by pressing
ctrl-z but the network becomes unusable. (even trying pinging the
host)
The second vm, acting as server on my test, is correctly connected the host.

Same story for the stock kernel (3.1.9-1.4-desktop on opensuse 11.2)
but here the problems come up in the first part of the test, when it's
sending udp packets to the host.
To start kvm tool on stock kernel i have to use -p "idle=halt" due to
a kernel bug.

Daniele

Re: native kvm tool hrtimer problem

[Asias He]

On Mon, Mar 12, 2012 at 11:03 PM, Daniele Carollo
<carollo.dani@gmail.com> wrote:
> The command I use is something like:
> sudo /home/kvm/linux-kvm/tools/kvm/lkvm run -p "root=/dev/vda1" -d
> /home/kvm/Internato/debian_squeeze_i386_standard.raw -n
> vhost=on,guest_mac=00:16:35:AF:94:4B,guest_ip=192.168.0.3,host_ip=192.168.0.9
>
> and then in order to connect the tap interface to the host bridge:
> /usr/bin/sudo /sbin/brctl addif br0 tap0
>
> Usually the error comes up when the vm is sendig udp packets to the second vm.
> It's possible to reobtain the control of the console by pressing
> ctrl-z but the network becomes unusable. (even trying pinging the
> host)
> The second vm, acting as server on my test, is correctly connected the host.
>
> Same story for the stock kernel (3.1.9-1.4-desktop on opensuse 11.2)
> but here the problems come up in the first part of the test, when it's
> sending udp packets to the host.
> To start kvm tool on stock kernel i have to use -p "idle=halt" due to
> a kernel bug.

I tried both virtio-net/without vhost and virtio-net/with vhost with
UDP and TCP stress between two VMs. I did not see any problems.
Can you try stable kernel as your guest kernel,  e.g. 3.2.X, 3.0.X?

-- 
Asias He

Re: native kvm tool hrtimer problem

[Daniele Carollo]

I tried using the kernel 3.2.11 and i had different results:
-one time it worked correctly without errors
-one time i had the hrtimer message but the network seemed continue working
-one time the network stopped working (during udp test)

If you have time, try using these scripts:
http://pastebin.com/K4sJq724 for the client on vm with ip address 192.168.0.x
http://pastebin.com/xLfR94EZ for the server on a vm with ip address 192.168.0.3