Re: copyless virtio net thoughts?

[Rusty Russell <rusty@rustcorp.com.au>]

On Thursday 19 February 2009 02:54:06 Arnd Bergmann wrote:
> On Wednesday 18 February 2009, Rusty Russell wrote:
> 
> > 2) Direct NIC attachment
> > This is particularly interesting with SR-IOV or other multiqueue nics,
> > but for boutique cases or benchmarks, could be for normal NICs.  So
> > far I have some very sketched-out patches: for the attached nic 
> > dev_alloc_skb() gets an skb from the guest (which supplies them via
> > some kind of AIO interface), and a branch in netif_receive_skb()
> > which returned it to the guest.  This bypasses all firewalling in
> > the host though; we're basically having the guest process drive
> > the NIC directly.       
> 
> If this is not passing the PCI device directly to the guest, but
> uses your concept, wouldn't it still be possible to use the firewalling
> in the host? You can always inspect the headers, drop the frame, etc
> without copying the whole frame at any point.

It's possible, but you don't want routing or parsing, etc: the NIC
is just "directly" attached to the guest.

You could do it in qemu or whatever, but it would not be the kernel scheme
(netfilter/iptables).

> > 3) Direct interguest networking
> > Anthony has been thinking here: vmsplice has already been mentioned.
> > The idea of passing directly from one guest to another is an
> > interesting one: using dma engines might be possible too.  Again,
> > host can't firewall this traffic.  Simplest as a dedicated "internal
> > lan" NIC, but we could theoretically do a fast-path for certain MAC
> > addresses on a general guest NIC.     
> 
> Another option would be to use an SR-IOV adapter from multiple guests,
> with a virtual ethernet bridge in the adapter. This moves the overhead
> from the CPU to the bus and/or adapter, so it may or may not be a real
> benefit depending on the workload.

Yes, I guess this should work.  Even different SR-IOV adapters will simply
send to one another.  I'm not sure this obviates the desire to have direct
inter-guest which is more generic though.

Thanks!
Rusty.