[RFC] network driver skb allocations

[RFC] network driver skb allocations

[Eric Dumazet]

Some performance idea about drivers and skb allocations :

-----------------------------------------------------------------------------------------------------------
   PerfTop:     954 irqs/sec  kernel:99.5% [1000Hz cycles],  (all, cpu: 0)
-----------------------------------------------------------------------------------------------------------

             samples  pcnt function                      DSO
             _______ _____ _____________________________ _________________

             2378.00 16.3% __alloc_skb                   [kernel.kallsyms]
             1962.00 13.5% eth_type_trans                [kernel.kallsyms]
             1472.00 10.1% __kmalloc_track_caller        [kernel.kallsyms]
              991.00  6.8% __slab_alloc                  [kernel.kallsyms]
              938.00  6.4% _raw_spin_lock                [kernel.kallsyms]
              914.00  6.3% __netdev_alloc_skb            [kernel.kallsyms]
              876.00  6.0% kmem_cache_alloc              [kernel.kallsyms]
              519.00  3.6% tg3_poll_work                 [kernel.kallsyms]
              416.00  2.9% tg3_read32                    [kernel.kallsyms]
              394.00  2.7% get_rps_cpu                   [kernel.kallsyms]

Current logic for drivers is to :

allocate skbs (sk_buff + data) and put them in a ring buffer.

When rx interrupt comes, get the skb and give it to stack.

Allocate a new skb (sk_buff + data) and put it in rx fat ring buffer (511 entries for tg3 )

This is suboptimal, because sk_buff will probably be cold 512 rx later...
Also, NUMA info might be wrong : sk_buff should be allocated on current node,
not on the device preferred node.

Drivers should allocate only the data part for NIC, and at the time of interrupt,
allocate the skb_buff and link it to buffer filled by NIC.

With a prefetch(first_cache_line_of_data) before doing sk_buff allocation and init,
eth_type_trans() / get_rps_cpus() would be much faster.

Re: [RFC] network driver skb allocations

[Ben Hutchings]

On Mon, 2010-05-03 at 19:06 +0200, Eric Dumazet wrote:
[...]
> Current logic for drivers is to :
> 
> allocate skbs (sk_buff + data) and put them in a ring buffer.

Not all of them.

> When rx interrupt comes, get the skb and give it to stack.
> 
> Allocate a new skb (sk_buff + data) and put it in rx fat ring buffer (511 entries for tg3 )
> 
> This is suboptimal, because sk_buff will probably be cold 512 rx later...
> Also, NUMA info might be wrong : sk_buff should be allocated on current node,
> not on the device preferred node.

This also avoids allocating sk_buffs that are never needed due to GRO or
scattering of jumbo frames.

> Drivers should allocate only the data part for NIC, and at the time of interrupt,
> allocate the skb_buff and link it to buffer filled by NIC.

I think we found that this increases latency, so sfc switches between
page and skb allocations dynamically.

Ben.

> With a prefetch(first_cache_line_of_data) before doing sk_buff allocation and init,
> eth_type_trans() / get_rps_cpus() would be much faster.

-- 
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

Re: [RFC] network driver skb allocations

[David Miller]

From: Ben Hutchings <bhutchings@solarflare.com>
Date: Mon, 03 May 2010 20:49:26 +0100

> On Mon, 2010-05-03 at 19:06 +0200, Eric Dumazet wrote:
> [...]
>> Current logic for drivers is to :
>> 
>> allocate skbs (sk_buff + data) and put them in a ring buffer.
> 
> Not all of them.

In particular NIU always allocates SKBs at the time that it passes the
packet up to the stack.