[Qemu-devel] any known virtio-net regressions in Qemu 2.7?

[Qemu-devel] any known virtio-net regressions in Qemu 2.7?

[Stefan Priebe - Profihost AG]

Hello,

after upgrading a cluster OS, Qemu, ... i'm experiencing slow and
volatile network speeds inside my VMs.

Currently I've no idea what causes this but it's related to the host
upgrades. Before i was running Qemu 2.6.2.

I'm using virtio for the network cards.

Greets,
Stefan

Re: [Qemu-devel] any known virtio-net regressions in Qemu 2.7?

[Alexandre DERUMIER]

Hi Stefan,

do you have upgraded kernel ?

maybe it could be related to vhost-net module too.


----- Mail original -----
De: "Stefan Priebe, Profihost AG" <s.priebe@profihost.ag>
À: "qemu-devel" <qemu-devel@nongnu.org>
Envoyé: Mercredi 14 Décembre 2016 16:04:08
Objet: [Qemu-devel] any known virtio-net regressions in Qemu 2.7?

Hello, 

after upgrading a cluster OS, Qemu, ... i'm experiencing slow and 
volatile network speeds inside my VMs. 

Currently I've no idea what causes this but it's related to the host 
upgrades. Before i was running Qemu 2.6.2. 

I'm using virtio for the network cards. 

Greets, 
Stefan 

Re: [Qemu-devel] any known virtio-net regressions in Qemu 2.7?

[Stefan Priebe - Profihost AG]

Am 14.12.2016 um 16:33 schrieb Alexandre DERUMIER:
> Hi Stefan,
> 
> do you have upgraded kernel ?

Yes sure. But i'm out of ideas how to debug. Sometimes it gives me
constant 80MB/s, sometimes 125 and sometimes only 6. While on the host
the cards are not busy.

Greets,
Stefan

> 
> maybe it could be related to vhost-net module too.
> 
> 
> ----- Mail original -----
> De: "Stefan Priebe, Profihost AG" <s.priebe@profihost.ag>
> À: "qemu-devel" <qemu-devel@nongnu.org>
> Envoyé: Mercredi 14 Décembre 2016 16:04:08
> Objet: [Qemu-devel] any known virtio-net regressions in Qemu 2.7?
> 
> Hello, 
> 
> after upgrading a cluster OS, Qemu, ... i'm experiencing slow and 
> volatile network speeds inside my VMs. 
> 
> Currently I've no idea what causes this but it's related to the host 
> upgrades. Before i was running Qemu 2.6.2. 
> 
> I'm using virtio for the network cards. 
> 
> Greets, 
> Stefan 
> 

Re: [Qemu-devel] any known virtio-net regressions in Qemu 2.7?

[Alexandre DERUMIER]

does rollbacking the kernel to previous version fix the problem ?

i'm not sure if "perf" could give you some hints
----- Mail original -----
De: "Stefan Priebe, Profihost AG" <s.priebe@profihost.ag>
À: "aderumier" <aderumier@odiso.com>
Cc: "qemu-devel" <qemu-devel@nongnu.org>
Envoyé: Mercredi 14 Décembre 2016 21:36:23
Objet: Re: [Qemu-devel] any known virtio-net regressions in Qemu 2.7?

Am 14.12.2016 um 16:33 schrieb Alexandre DERUMIER: 
> Hi Stefan, 
> 
> do you have upgraded kernel ? 

Yes sure. But i'm out of ideas how to debug. Sometimes it gives me 
constant 80MB/s, sometimes 125 and sometimes only 6. While on the host 
the cards are not busy. 

Greets, 
Stefan 

> 
> maybe it could be related to vhost-net module too. 
> 
> 
> ----- Mail original ----- 
> De: "Stefan Priebe, Profihost AG" <s.priebe@profihost.ag> 
> À: "qemu-devel" <qemu-devel@nongnu.org> 
> Envoyé: Mercredi 14 Décembre 2016 16:04:08 
> Objet: [Qemu-devel] any known virtio-net regressions in Qemu 2.7? 
> 
> Hello, 
> 
> after upgrading a cluster OS, Qemu, ... i'm experiencing slow and 
> volatile network speeds inside my VMs. 
> 
> Currently I've no idea what causes this but it's related to the host 
> upgrades. Before i was running Qemu 2.6.2. 
> 
> I'm using virtio for the network cards. 
> 
> Greets, 
> Stefan 
> 

Re: [Qemu-devel] any known virtio-net regressions in Qemu 2.7?

[Stefan Priebe - Profihost AG]

Am 15.12.2016 um 07:46 schrieb Alexandre DERUMIER:
> does rollbacking the kernel to previous version fix the problem ?

The culprit is the used tuned agent from Redhat
(https://github.com/redhat-performance/tuned). The used profile
virtual-host results in these problems. Stopping tuned or using another
profile like throughput-performance everything is fine again.

Geets,
Stefan

> 
> i'm not sure if "perf" could give you some hints
> ----- Mail original -----
> De: "Stefan Priebe, Profihost AG" <s.priebe@profihost.ag>
> À: "aderumier" <aderumier@odiso.com>
> Cc: "qemu-devel" <qemu-devel@nongnu.org>
> Envoyé: Mercredi 14 Décembre 2016 21:36:23
> Objet: Re: [Qemu-devel] any known virtio-net regressions in Qemu 2.7?
> 
> Am 14.12.2016 um 16:33 schrieb Alexandre DERUMIER: 
>> Hi Stefan, 
>>
>> do you have upgraded kernel ? 
> 
> Yes sure. But i'm out of ideas how to debug. Sometimes it gives me 
> constant 80MB/s, sometimes 125 and sometimes only 6. While on the host 
> the cards are not busy. 
> 
> Greets, 
> Stefan 
> 
>>
>> maybe it could be related to vhost-net module too. 
>>
>>
>> ----- Mail original ----- 
>> De: "Stefan Priebe, Profihost AG" <s.priebe@profihost.ag> 
>> À: "qemu-devel" <qemu-devel@nongnu.org> 
>> Envoyé: Mercredi 14 Décembre 2016 16:04:08 
>> Objet: [Qemu-devel] any known virtio-net regressions in Qemu 2.7? 
>>
>> Hello, 
>>
>> after upgrading a cluster OS, Qemu, ... i'm experiencing slow and 
>> volatile network speeds inside my VMs. 
>>
>> Currently I've no idea what causes this but it's related to the host 
>> upgrades. Before i was running Qemu 2.6.2. 
>>
>> I'm using virtio for the network cards. 
>>
>> Greets, 
>> Stefan 
>>
> 

Re: [Qemu-devel] any known virtio-net regressions in Qemu 2.7?

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 825 bytes --]

On Fri, Dec 16, 2016 at 10:00:36PM +0100, Stefan Priebe - Profihost AG wrote:
> 
> Am 15.12.2016 um 07:46 schrieb Alexandre DERUMIER:
> > does rollbacking the kernel to previous version fix the problem ?
> 
> The culprit is the used tuned agent from Redhat
> (https://github.com/redhat-performance/tuned). The used profile
> virtual-host results in these problems. Stopping tuned or using another
> profile like throughput-performance everything is fine again.

Interesting discovery.  Have you filed a bug report about it?

> after upgrading a cluster OS, Qemu, ... i'm experiencing slow and
> volatile network speeds inside my VMs.
>
> Currently I've no idea what causes this but it's related to the host
> upgrades. Before i was running Qemu 2.6.2.
>
> I'm using virtio for the network cards.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [Qemu-devel] any known virtio-net regressions in Qemu 2.7?

[Stefan Priebe - Profihost AG]

Am 19.12.2016 um 12:03 schrieb Stefan Hajnoczi:
> On Fri, Dec 16, 2016 at 10:00:36PM +0100, Stefan Priebe - Profihost AG wrote:
>>
>> Am 15.12.2016 um 07:46 schrieb Alexandre DERUMIER:
>>> does rollbacking the kernel to previous version fix the problem ?
>>
>> The culprit is the used tuned agent from Redhat
>> (https://github.com/redhat-performance/tuned). The used profile
>> virtual-host results in these problems. Stopping tuned or using another
>> profile like throughput-performance everything is fine again.
> 
> Interesting discovery.  Have you filed a bug report about it?

As we're not using RHEL, nor CentOS i didn't expect this to make sense
;-) I just believed that something in the kernel changed.

Greets,
Stefan

>> after upgrading a cluster OS, Qemu, ... i'm experiencing slow and
>> volatile network speeds inside my VMs.
>>
>> Currently I've no idea what causes this but it's related to the host
>> upgrades. Before i was running Qemu 2.6.2.
>>
>> I'm using virtio for the network cards.
> 
> Stefan
> 

Re: [Qemu-devel] any known virtio-net regressions in Qemu 2.7?

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1306 bytes --]

On Mon, Dec 19, 2016 at 07:37:30PM +0100, Stefan Priebe - Profihost AG wrote:
> Am 19.12.2016 um 12:03 schrieb Stefan Hajnoczi:
> > On Fri, Dec 16, 2016 at 10:00:36PM +0100, Stefan Priebe - Profihost AG wrote:
> >>
> >> Am 15.12.2016 um 07:46 schrieb Alexandre DERUMIER:
> >>> does rollbacking the kernel to previous version fix the problem ?
> >>
> >> The culprit is the used tuned agent from Redhat
> >> (https://github.com/redhat-performance/tuned). The used profile
> >> virtual-host results in these problems. Stopping tuned or using another
> >> profile like throughput-performance everything is fine again.
> > 
> > Interesting discovery.  Have you filed a bug report about it?
> 
> As we're not using RHEL, nor CentOS i didn't expect this to make sense
> ;-) I just believed that something in the kernel changed.

I see.  At least the tuned maintainers are aware of it now.  If others
report the same problem maybe it can be solved.

Thanks!

Stefan

> >> after upgrading a cluster OS, Qemu, ... i'm experiencing slow and
> >> volatile network speeds inside my VMs.
> >>
> >> Currently I've no idea what causes this but it's related to the host
> >> upgrades. Before i was running Qemu 2.6.2.
> >>
> >> I'm using virtio for the network cards.
> > 
> > Stefan
> > 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]