Re: [PATCH net-next RFC 0/5] xen-netback: TX grant mapping instead of copy

[Zoltan Kiss <zoltan.kiss@citrix.com>]

On 01/11/13 10:50, Ian Campbell wrote:
> Does this always avoid copying when bridging/openvswitching/forwarding
> (e.g. masquerading etc)? For both domU->domU and domU->physical NIC?
I've tested the domU->domU, domU->physical with bridge and openvswitch 
usecase, and now I've created a new stat counter to see how often copy 
happens (the callback's second parameter tells you whether the skb was 
freed or copied). It doesn't do copy in all of these scenarios.
What do you mean by forwarding? The scenario when you use bridge and 
iptables mangling with the packet, not just filtering?

> How does it deal with broadcast traffic?
Most of the real broadcast traffic actually small packets fit in the 
PKT_PROT_LEN sized linear space, so it doesn't make any difference, 
apart from doing a mapping before copy. But that will be eliminated 
later on, I plan to add an incremental improvement to grant copy the 
linear part.
I haven't spent too much time on that, but I couldn't find any broadcast 
protocol which use large enough packets and easy to test, so I'm open to 
ideas.
What I already know, skb_clone trigger a copy, and if the caller use the 
original skb for every cloning, it will do several copy. I think that 
could be fixed by using the first clone to do any further clones.

> Do you have any numbers for the dom0 cpu usage impact?
DomU->NIC: the vif took 40% according to top, I guess the bottleneck 
there is the TLB flushing.
DomU->DomU: the vif of the RX side cause the bottleneck due to grant 
copy to the guest

> Aggregate throughput for many guests would be a useful datapoint too.
I will do measurements about that.

 >> Based on my investigations the packet get only copied if it is 
delivered to
 >>Dom0 stack, which is due to this patch:
 >>https://lkml.org/lkml/2012/7/20/363
 >>That's a bit unfortunate, but as far as I know for the huge majority 
this use
 >>case is not too important.
> Likely to be true, but it would still be interesting to know how badly
> this use case suffers with this change, and any increase in CPU usage
> would be interesting to know about as well.
I can't find my numbers, but as far as I remember it wasn't 
significantly worse than grant copy. I will check that again.

Zoli