From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46376) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Xbocj-0001Yg-K0 for qemu-devel@nongnu.org; Wed, 08 Oct 2014 06:37:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Xbocc-0004DK-3d for qemu-devel@nongnu.org; Wed, 08 Oct 2014 06:37:37 -0400 Received: from mail-wg0-f46.google.com ([74.125.82.46]:49043) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Xbocb-0004DG-U8 for qemu-devel@nongnu.org; Wed, 08 Oct 2014 06:37:30 -0400 Received: by mail-wg0-f46.google.com with SMTP id l18so11204310wgh.17 for ; Wed, 08 Oct 2014 03:37:29 -0700 (PDT) Message-ID: <543513E5.8010507@cloudius-systems.com> Date: Wed, 08 Oct 2014 13:37:25 +0300 From: Avi Kivity MIME-Version: 1.0 References: <542A6B70.7090607@huawei.com> <20140930093356.GA3673@redhat.com> <5434EB0B.8010800@cloudius-systems.com> <20141008091547.GB3872@redhat.com> <54350919.8050401@cloudius-systems.com> <20141008101443.GA4291@redhat.com> In-Reply-To: <20141008101443.GA4291@redhat.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [QA-virtio]:Why vring size is limited to 1024? List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Michael S. Tsirkin" Cc: Jason Wang , qemu-devel@nongnu.org, liuyongan@huawei.com, qinchuanyu@huawei.com, "Zhangjie (HZ)" , akong@redhat.com On 10/08/2014 01:14 PM, Michael S. Tsirkin wrote: > On Wed, Oct 08, 2014 at 12:51:21PM +0300, Avi Kivity wrote: >> On 10/08/2014 12:15 PM, Michael S. Tsirkin wrote: >>> On Wed, Oct 08, 2014 at 10:43:07AM +0300, Avi Kivity wrote: >>>> On 09/30/2014 12:33 PM, Michael S. Tsirkin wrote: >>>>> a single descriptor might use all of >>>>> the virtqueue. In this case we wont to be able to pass the >>>>> descriptor directly to linux as a single iov, since >>>>> >>>> You could separate maximum request scatter/gather list size from the >>>> virtqueue size. They are totally unrelated - even now you can have a larger >>>> request by using indirect descriptors. >>> We could add a feature to have a smaller or larger S/G length limit. >>> Is this something useful? >>> >> Having a larger ring size is useful, esp. with zero-copy transmit, and you >> would need the sglist length limit in order to not require linearization on >> linux hosts. So the limit is not useful in itself, only indirectly. >> >> Google cloud engine exposes virtio ring sizes of 16384. > OK this sounds useful, I'll queue this up for consideration. > Thanks! Thanks. >> Even more useful is getting rid of the desc array and instead passing descs >> inline in avail and used. > You expect this to improve performance? > Quite possibly but this will have to be demonstrated. > The top vhost function in small packet workloads is vhost_get_vq_desc, and the top instruction within that (50%) is the one that reads the first 8 bytes of desc. It's a guaranteed cache line miss (and again on the guest side when it's time to reuse). Inline descriptors will amortize the cache miss over 4 descriptors, and will allow the hardware to prefetch, since the descriptors are linear in memory.