From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37610) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WJIOd-0005C4-W8 for qemu-devel@nongnu.org; Fri, 28 Feb 2014 03:02:17 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WJIOa-0002nz-9y for qemu-devel@nongnu.org; Fri, 28 Feb 2014 03:02:15 -0500 Received: from mail-ea0-x22c.google.com ([2a00:1450:4013:c01::22c]:61022) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WJIOZ-0002mX-UV for qemu-devel@nongnu.org; Fri, 28 Feb 2014 03:02:12 -0500 Received: by mail-ea0-f172.google.com with SMTP id l9so2400753eaj.17 for ; Fri, 28 Feb 2014 00:02:10 -0800 (PST) MIME-Version: 1.0 Sender: lukego@gmail.com In-Reply-To: <20140227144949.GA21933@redhat.com> References: <20140224152004.GC23185@stefanha-thinkpad.hitronhub.home> <20140227141744.GG30387@stefanha-thinkpad.redhat.com> <20140227144949.GA21933@redhat.com> Date: Fri, 28 Feb 2014 09:02:10 +0100 Message-ID: From: Luke Gorrie Content-Type: multipart/alternative; boundary=047d7beb9394dad7ee04f372da31 Subject: Re: [Qemu-devel] [snabb-devel] Re: Make virtio-net.c ring size configurable? List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "snabb-devel@googlegroups.com" Cc: Stefan Hajnoczi , qemu-devel --047d7beb9394dad7ee04f372da31 Content-Type: text/plain; charset=ISO-8859-1 On 27 February 2014 15:49, Michael S. Tsirkin wrote: > > Michael: Luke has asked to increase the virtio-net virtqueue size. > > Thoughts? > > > > Stefan > > Heh you want to increase the bufferbloat? > I'm sensitive to this. (I have actually built a commercial anti-bufferbloat network device for ISPs in the recent past.) I will go to great lengths to keep latency below 1 millisecond but beyond that I'm more flexible. Each buffer pointer takes up 16 bytes so we are using order-2 > allocations as it is, anything more and it'll start to fail > if hotplug happens long after boot. > (Sorry I don't have the background to understand this issue.) > AFAIK baremetal does not push line rate with 1 byte payload > either. > To me it feels normal to do this in the commercial networking industry. Many networking vendors will sell you a NIC with a software interface to drive it at line rate from userspace: Intel, Myricom, SolarFlare, Chelsio, Mellanox. They really work. Lots of high-end commercial network devices are built on these simple and cheap components. Here's one detailed performance test that Luca Deri did based on standard Intel CPU and NIC and all packet sizes: http://www.ntop.org/wp-content/uploads/2012/04/DNA_ip_forward_RFC2544.pdf For my project now I need to drive 6x10G ports worth of network traffic through Virtio-net to KVM guests. That's the ballpark of what ISPs I'm talking with require to be able to use Virtio-net instead of SR-IOV+Passthrough. They really want to use Virtio-net for a variety of reasons and the only barrier is performance for router-like workloads. I'm working on Deutsche Telekom's TeraStream project [1] [2] and success will mean that Virtio-net drives all internet traffic for national ISPs. That would be really cool imo :-). [1] TeraStream blurb http://blog.ipspace.net/2013/11/deutsche-telekom-terastream-designed.html [2] TeraStream talk http://ripe67.ripe.net/archives/video/3/ --047d7beb9394dad7ee04f372da31 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
On 27 February 2014 15:49, Michael S. Tsirkin <mst@redhat.co= m> wrote:
> Michael: Luke has asked to increase the virtio-net vir= tqueue size.
> Thoughts?
>
> Stefan

Heh you want to increase the bufferbloat?

I'm sensitive to this. (I have actually built a commercial ant= i-bufferbloat network device for ISPs in the recent past.) I will go to gre= at lengths to keep latency below 1 millisecond but beyond that I'm more= flexible.

Each buffer pointer takes up 16 bytes so we= are using order-2
allocations as it is, anything more and it'll start to fail
if hotplug happens long after boot.

(So= rry I don't have the background to understand this issue.)
= =A0
AFAIK baremetal does not push line rate with 1 byte payload
either.

To me it feels normal to do thi= s in the commercial networking industry. Many networking vendors will sell = you a NIC with a software interface to drive it at line rate from userspace= : Intel, Myricom, SolarFlare, Chelsio, Mellanox. They really work. Lots of = high-end commercial network devices are built on these simple and cheap com= ponents.

Here's one detailed performance test that Luca Deri= did based on standard Intel CPU and NIC and all packet sizes:=A0http://www.ntop.org/wp-content/uploads/2012/04/DNA_ip_forward_RFC2544.pdf<= /a>

--047d7beb9394dad7ee04f372da31--