Re: xennet: skb rides the rocket: 20 slots

[ANNIE LI <annie.li@oracle.com>]

On 2013-1-10 20:26, Ian Campbell wrote:
> On Thu, 2013-01-10 at 11:22 +0000, ANNIE LI wrote:
>> I am thinking to do re-fragment in netfront for these skbs like following,
>>
>> Create a new skb, copy linear data and frag data from original skb into
>> this one, and make every frags data size is PAGE_SIZE except for the
>> last fragment. It is possible that the last fragment length is less than
>> PAGE_SIZE, then free the original skb. The skb packet is large, and
>> there will be lots of copys.
> You don't need (or I suspect want) to copy, you can directly add pages
> from the source skb's frags to the destination skb's frags, with
> appropriate refcount frobbing. You can also share a page between two (or
> more) skbs in the case where the boundary between two skbs happens to be
> in the middle of a page.
>
> But more importantly than all that you need to do more than just
> refragment, you actually need to resegment i.e. you need to duplicate
> the headers (Ethernet, IP, TCP) at the front of each new skb and adjust
> the (psuedo-)checksums as appropriate (which will depend on whether the
> SKB is GSO, or just has checksum offload or nothing).
>
> So you need to go from
>    <Ether><IP><TCP><...data... ...data...>
> to
>    <Ether><IP><TCP><...data...>  <Ether><IP><TCP><...data...>
>
> Where the headers are adjusted to cope with this. Some of data might be
> in frags but equally some might be in skb->data.
>
> I'm guessing that Linux already has code which can do this for you,
> since it has a software fallback for GSO.

There is "skb_gso_segment" existing to perform segmentation on skb and 
return a list of segments. From the code, it seems this function is 
specific to GSO packets, maybe it is not available to other non-gso 
large packets which requires slots is larger than MAX_SKB_FRAGS + 1". So 
for non-gso offload packets, I would go forward to write a function to 
do resegment.

Thanks
Annie
>
> Ian.
>
>> struct skbuff *xennet_refrag_skb(skb)
>> {
>>      create newskb
>>      copying data and doing fragmentation
>>      return newskb
>> }
>>
>> .......
>>
>> if (unlikely(slots>   MAX_SKB_FRAGS + 1)) {
>>           net_alert_ratelimited(
>>                 "xennet: skb rides the rocket: %d slots\n", slots);
>>           skb = xennet_refrag_skb(skb);
>> }
>> .....
>>
>> Thanks
>> Annie
>>>> "netchannel vs MAX_SKB_FRAGS". Maybe these two mechanism are all
>>>> necessary?
>>> Lets see first if this is indeed the problem. Perhaps a simple debug
>>> patch that just does:
>>>
>>> 	s/MAX_SKB_FRAGS/DEBUG_MAX_FRAGS/
>>> 	#define DEBUG_MAX_FRAGS 21
>>>
>>> in both netback and netfront to set the maximum number of frags we can
>>> handle to 21? If that works with Sander test - then yes, it looks like
>>> we really need to get this 'feature-max-skb-frags' done.
>>>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel