Re: [Qemu-devel] [QA-virtio]:Why vring size is limited to 1024?

[Avi Kivity <avi@cloudius-systems.com>]

On 10/08/2014 01:14 PM, Michael S. Tsirkin wrote:
> On Wed, Oct 08, 2014 at 12:51:21PM +0300, Avi Kivity wrote:
>> On 10/08/2014 12:15 PM, Michael S. Tsirkin wrote:
>>> On Wed, Oct 08, 2014 at 10:43:07AM +0300, Avi Kivity wrote:
>>>> On 09/30/2014 12:33 PM, Michael S. Tsirkin wrote:
>>>>> a single descriptor might use all of
>>>>> the virtqueue. In this case we wont to be able to pass the
>>>>> descriptor directly to linux as a single iov, since
>>>>>
>>>> You could separate maximum request scatter/gather list size from the
>>>> virtqueue size.  They are totally unrelated - even now you can have a larger
>>>> request by using indirect descriptors.
>>> We could add a feature to have a smaller or larger S/G length limit.
>>> Is this something useful?
>>>
>> Having a larger ring size is useful, esp. with zero-copy transmit, and you
>> would need the sglist length limit in order to not require linearization on
>> linux hosts.  So the limit is not useful in itself, only indirectly.
>>
>> Google cloud engine exposes virtio ring sizes of 16384.
> OK this sounds useful, I'll queue this up for consideration.
> Thanks!

Thanks.

>> Even more useful is getting rid of the desc array and instead passing descs
>> inline in avail and used.
> You expect this to improve performance?
> Quite possibly but this will have to be demonstrated.
>

The top vhost function in small packet workloads is vhost_get_vq_desc, 
and the top instruction within that (50%) is the one that reads the 
first 8 bytes of desc.  It's a guaranteed cache line miss (and again on 
the guest side when it's time to reuse).

Inline descriptors will amortize the cache miss over 4 descriptors, and 
will allow the hardware to prefetch, since the descriptors are linear in 
memory.