Re: copyless virtio net thoughts?

[Anthony Liguori <anthony@codemonkey.ws>]

Avi Kivity wrote:
> Chris Wright wrote:
>> There's been a number of different discussions re: getting copyless 
>> virtio
>> net (esp. for KVM).  This is just a poke in that general direction to
>> stir the discussion.  I'm interested to hear current thoughts
>
> I believe that copyless networking is absolutely essential.
>
> For transmit, copyless is needed to properly support sendfile() type 
> workloads - http/ftp/nfs serving.  These are usually high-bandwidth, 
> cache-cold workloads where a copy is most expensive.
>
> For receive, the guest will almost always do an additional copy, but 
> it will most likely do the copy from another cpu.  Xen netchannel2 
> mitigates this somewhat by having the guest request the hypervisor to 
> perform the copy when the rx interrupt is processed, but this may 
> still be too early (the packet may be destined to a process that is on 
> another vcpu), and the extra hypercall is expensive.
>
> In my opinion, it would be ideal to linux-aio enable taps and packet 
> sockets.  io_submit() allows submitting multiple buffers in one 
> syscall and supports scatter/gather.  io_getevents() supports 
> dequeuing multiple packet completions in one syscall.

splice() has some nice properties too.  It disconnects the notion of 
moving around packets from the actually copy them.   It also fits well 
into a more performant model of interguest IO.  You can't publish 
multiple buffers with splice but I don't think we can do that today 
practically speaking because of mergable RX buffers.  You would have to 
extend the linux-aio interface to hand it a bunch of buffers and for it 
to tell you where the packet boundaries were.

Regards,

Anthony Liguroi