Re: [net-next PATCH V2 5/9] net: frag, per CPU resource, mem limit and LRU list accounting

[Jesper Dangaard Brouer <brouer@redhat.com>]

On Thu, 2012-11-29 at 09:06 -0800, Eric Dumazet wrote:
> On Thu, 2012-11-29 at 17:13 +0100, Jesper Dangaard Brouer wrote:
> > The major performance bottleneck on NUMA systems, is the mem limit
> > counter which is based an atomic counter.  This patch removes the
> > cache-bouncing of the atomic counter, by moving this accounting to be
> > bound to each CPU.  The LRU list also need to be done per CPU,
> > in-order to keep the accounting straight.
> > 
> > If fragments belonging together is "sprayed" across CPUs, performance
> > will still suffer, but due to NIC rxhashing this is not very common.
> > Correct accounting in this situation is maintained by recording and
> > "assigning" a CPU to a frag queue when its allocated (caused by the
> > first packet associated packet).
> > 
[...]
> > +/* Need to maintain these resource limits per CPU, else we will kill
> > + * performance due to cache-line bouncing
> > + */
> > +struct frag_cpu_limit {
> > +	atomic_t                mem;
> > +	struct list_head        lru_list;
> > +	spinlock_t              lru_lock;
> > +} ____cacheline_aligned_in_smp;
> > +
> 
> This looks like a big patch introducing a specific infrastructure, while
> we already have lib/percpu_counter.c

For the record, I cannot use the lib/percpu_counter, because this
accounting is not kept strictly per CPU, if the fragments are "sprayed"
across CPUs (as described in the commit message above).