DPDK - VIRTIO performance problems

DPDK - VIRTIO performance problems

[Yan Freedland]

Hi

I am checking performance DPDK VIRTIO mode running on KVM (Linux ubuntu 3.11.0-15-generic).
The maximum throughput I reached was 4Gbps and then I saw an interesting phenomena.

Every ~2min traffic stopped completely and then immediately came back. This happened in a periodic fashion.

I have never seen such thing in a pass-through mode, where I reached very much higher rates of course.

Can you please help in resolving this problem in VIRTIO ?


Thank you
Yan

Re: DPDK - VIRTIO performance problems

[Matthew Hall]

On Sun, Oct 12, 2014 at 12:37:37PM +0000, Yan Freedland wrote:
> Every ~2min traffic stopped completely and then immediately came back. This 
> happened in a periodic fashion.

To me it sounds like it could be similar to what I've seen when I ran out of 
mbuf's or ran out of RX / TX descriptor entries. It could be worth checking 
the error counters on the interfaces with DPDK and Linux OS / ethtool to see 
what might be incrementing during the failed time periods.

Matthew.

Re: DPDK - VIRTIO performance problems

[Ouyang, Changchun]

Hi ,

> -----Original Message-----
> From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Matthew Hall
> Sent: Sunday, October 12, 2014 9:18 PM
> To: Yan Freedland
> Cc: dev-VfR2kkLFssw@public.gmane.org
> Subject: Re: [dpdk-dev] DPDK - VIRTIO performance problems
> 
> On Sun, Oct 12, 2014 at 12:37:37PM +0000, Yan Freedland wrote:
> > Every ~2min traffic stopped completely and then immediately came back.
> > This happened in a periodic fashion.
> 
> To me it sounds like it could be similar to what I've seen when I ran out of
> mbuf's or ran out of RX / TX descriptor entries. It could be worth checking the
> error counters on the interfaces with DPDK and Linux OS / ethtool to see
> what might be incrementing during the failed time periods.
> 

I didn't meet this issue before,
I am not sure if the following patch will fix this issue or not. Please try it.
http://dpdk.org/dev/patchwork/patch/779/

By the way, what kind of backend did you use?
User space vhost, or other backend?

Thanks 
Changchun

Re: DPDK - VIRTIO performance problems

[Yan Freedland]

Hi

I found that what blocked me was actually the "nf_conntrack",
So enlarging the maximum there solved the issue.

Thanks 
Yan

-----Original Message-----
From: Ouyang, Changchun [mailto:changchun.ouyang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org] 
Sent: Monday, October 13, 2014 6:10 AM
To: Matthew Hall; Yan Freedland
Cc: dev-VfR2kkLFssw@public.gmane.org; Ouyang, Changchun
Subject: RE: [dpdk-dev] DPDK - VIRTIO performance problems

Hi ,

> -----Original Message-----
> From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Matthew Hall
> Sent: Sunday, October 12, 2014 9:18 PM
> To: Yan Freedland
> Cc: dev-VfR2kkLFssw@public.gmane.org
> Subject: Re: [dpdk-dev] DPDK - VIRTIO performance problems
> 
> On Sun, Oct 12, 2014 at 12:37:37PM +0000, Yan Freedland wrote:
> > Every ~2min traffic stopped completely and then immediately came back.
> > This happened in a periodic fashion.
> 
> To me it sounds like it could be similar to what I've seen when I ran 
> out of mbuf's or ran out of RX / TX descriptor entries. It could be 
> worth checking the error counters on the interfaces with DPDK and 
> Linux OS / ethtool to see what might be incrementing during the failed time periods.
> 

I didn't meet this issue before,
I am not sure if the following patch will fix this issue or not. Please try it.
http://dpdk.org/dev/patchwork/patch/779/

By the way, what kind of backend did you use?
User space vhost, or other backend?

Thanks
Changchun