Re: Poor network performance between DomU with multiqueue support

[Zoltan Kiss <zoltan.kiss@linaro.org>]

On 02/12/14 11:53, Zhangleiqiang (Trump) wrote:
>> -----Original Message-----
>> From: xen-devel-bounces@lists.xen.org
>> [mailto:xen-devel-bounces@lists.xen.org] On Behalf Of David Vrabel
>> Sent: Tuesday, December 02, 2014 6:57 PM
>> To: zhangleiqiang; xen-devel@lists.xen.org
>> Subject: Re: [Xen-devel] Poor network performance between DomU with
>> multiqueue support
>>
>> On 02/12/14 08:30, zhangleiqiang wrote:
>>> Hi, all
>>>      I am testing the performance of xen netfront-netback driver that
>>> with multi-queues support. The throughput from domU to remote dom0 is
>>> 9.2Gb/s, but the throughput from domU to remote domU is only 3.6Gb/s,
>>> I think the bottleneck is the throughput from dom0 to local domU.
>>> However, we have done some testing and found the throughput from dom0
>>> to local domU is 5.8Gb/s.
>>>      And if we send packets from one DomU to other 3 DomUs on different
>>> host simultaneously, the sum of throughout can reach 9Gbps. It seems
>>> like the bottleneck is the receiver?
>>>      After some analysis, I found that even the max_queue of
>>> netfront/back is set to 4, there are some strange results as follows:
>>>      1. In domU, only one rx queue deal with softirq
>>>      2. In dom0, only two netback queues process are scheduled, other
>>> two process aren't scheduled.
>>
>> Multiqueue only has benefits if you have multiple flows since the
>> source/destination addresses are hashed to a queue number.  This probably
>> explains why only some of the queues are being used in your test.
>
> The hash method you mentioned is used for selection of netback process or netfront rx queue?
It's used in both direction to select the queue.

> Indeed, there are 4 netback processes running in Dom0, because there are only one DomU running in Dom0 and so four netback processes are running in Dom0 (the max_queue param of netback kernel module is set to 4).
> The phenomenon is that only 2 of these four netback process were running with about 70% cpu usage, and another two use little CPU.
>
>> David
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xen.org
>> http://lists.xen.org/xen-devel
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>