From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48593) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Xboqa-0004d8-RM for qemu-devel@nongnu.org; Wed, 08 Oct 2014 06:52:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XboqU-0000Tw-Nc for qemu-devel@nongnu.org; Wed, 08 Oct 2014 06:51:56 -0400 Received: from mx1.redhat.com ([209.132.183.28]:45579) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XboqU-0000Sj-Eb for qemu-devel@nongnu.org; Wed, 08 Oct 2014 06:51:50 -0400 Date: Wed, 8 Oct 2014 13:55:15 +0300 From: "Michael S. Tsirkin" Message-ID: <20141008105515.GA4429@redhat.com> References: <542A6B70.7090607@huawei.com> <20140930093356.GA3673@redhat.com> <5434EB0B.8010800@cloudius-systems.com> <20141008091547.GB3872@redhat.com> <54350919.8050401@cloudius-systems.com> <20141008101443.GA4291@redhat.com> <543513E5.8010507@cloudius-systems.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <543513E5.8010507@cloudius-systems.com> Subject: Re: [Qemu-devel] [QA-virtio]:Why vring size is limited to 1024? List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Avi Kivity Cc: Jason Wang , qemu-devel@nongnu.org, liuyongan@huawei.com, qinchuanyu@huawei.com, "Zhangjie (HZ)" , akong@redhat.com On Wed, Oct 08, 2014 at 01:37:25PM +0300, Avi Kivity wrote: > > On 10/08/2014 01:14 PM, Michael S. Tsirkin wrote: > >On Wed, Oct 08, 2014 at 12:51:21PM +0300, Avi Kivity wrote: > >>On 10/08/2014 12:15 PM, Michael S. Tsirkin wrote: > >>>On Wed, Oct 08, 2014 at 10:43:07AM +0300, Avi Kivity wrote: > >>>>On 09/30/2014 12:33 PM, Michael S. Tsirkin wrote: > >>>>>a single descriptor might use all of > >>>>>the virtqueue. In this case we wont to be able to pass the > >>>>>descriptor directly to linux as a single iov, since > >>>>> > >>>>You could separate maximum request scatter/gather list size from the > >>>>virtqueue size. They are totally unrelated - even now you can have a larger > >>>>request by using indirect descriptors. > >>>We could add a feature to have a smaller or larger S/G length limit. > >>>Is this something useful? > >>> > >>Having a larger ring size is useful, esp. with zero-copy transmit, and you > >>would need the sglist length limit in order to not require linearization on > >>linux hosts. So the limit is not useful in itself, only indirectly. > >> > >>Google cloud engine exposes virtio ring sizes of 16384. > >OK this sounds useful, I'll queue this up for consideration. > >Thanks! > > Thanks. > > >>Even more useful is getting rid of the desc array and instead passing descs > >>inline in avail and used. > >You expect this to improve performance? > >Quite possibly but this will have to be demonstrated. > > > > The top vhost function in small packet workloads is vhost_get_vq_desc, and > the top instruction within that (50%) is the one that reads the first 8 > bytes of desc. It's a guaranteed cache line miss (and again on the guest > side when it's time to reuse). OK so basically what you are pointing out is that we get 5 accesses: read of available head, read of available ring, read of descriptor, write of used ring, write of used ring head. If processing is in-order, we could build a much simpler design, with a valid bit in the descriptor, cleared by host as descriptors are consumed. Basically get rid of both used and available ring. Sounds good in theory. > Inline descriptors will amortize the cache miss over 4 descriptors, and will > allow the hardware to prefetch, since the descriptors are linear in memory. If descriptors are used in order (as they are with current qemu) then aren't they amortized already? -- MST