Re: [PATCH net-next] net: percpu net_device refcount

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Sat, Oct 09, 2010 at 08:23:16AM +0200, Eric Dumazet wrote:
> Le vendredi 08 octobre 2010 à 14:56 -0700, Paul E. McKenney a écrit :
> > On Thu, Oct 07, 2010 at 10:30:51AM -0700, Stephen Hemminger wrote:
> > > On Thu, 07 Oct 2010 19:12:35 +0200
> > > Eric Dumazet <eric.dumazet@gmail.com> wrote:
> > > 
> > > > We tried very hard to remove all possible dev_hold()/dev_put() pairs in
> > > > network stack, using RCU conversions.
> > > > 
> > > > There is still an unavoidable device refcount change for every dst we
> > > > create/destroy, and this can slow down some workloads (routers or some
> > > > app servers)
> > > > 
> > > > We can switch to a percpu refcount implementation, now dynamic per_cpu
> > > > infrastructure is mature. On a 64 cpus machine, this consumes 256 bytes
> > > > per device.
> > > 
> > > It makes sense, but what about 256 cores and 1024 Vlans?
> > > That adds up to 4M of memory which is might be noticeable.
> > 
> > I bet that systems that have 256 cores have >100GB of memory, at which
> > point 4MB is way down in the noise.
> 
> Well, first its 1MB added, and secondly we added percpu stats for vlan
> devices, and this consumed 8x more :
> 
> (struct vlan_rx_stats is 32 bytes per cpu and per vlan
> 32*256*1024  ->  8 Mbytes
> 
> Some strange machines have many cores sharing a small amount of memory,
> but I am not sure they want to run many net devices ;)

I do have to admit that the rapid growth rate in the data required might
well be cause for concern.  But only if it continues.  ;-)

							Thanx, Paul