Re: [patch] Cache align futex hash buckets

[Ravikiran G Thirumalai <kiran@scalex86.org>]

On Mon, Feb 20, 2006 at 04:23:17PM -0800, Andrew Morton wrote:
> Ravikiran G Thirumalai <kiran@scalex86.org> wrote:
> >
> > On Mon, Feb 20, 2006 at 03:34:19PM -0800, Andrew Morton wrote:
> > > Andrew Morton <akpm@osdl.org> wrote:
> > > >
> > > > > @@ -100,9 +100,10 @@ struct futex_q {
> > > > >  struct futex_hash_bucket {
> > > > >         spinlock_t              lock;
> > > > >         struct list_head       chain;
> > > > > -};
> > > > > +} ____cacheline_internodealigned_in_smp;
> > > > >  
> > > > > -static struct futex_hash_bucket futex_queues[1<<FUTEX_HASHBITS];
> > > > > +static struct futex_hash_bucket futex_queues[1<<FUTEX_HASHBITS] 
> > > > > +				__cacheline_aligned_in_smp;
> > > > >  
> > > > 
> > > > How much memory does that thing end up consuming?
> > > 
> > > I think a megabyte?
> > 
> > On most machines it would be 256 * 128 = 32k. or 16k on arches with 64B 
> > cachelines.  This looked like a simpler solution for spinlocks falling on
> > the same cacheline.  So is 16/32k unreasonable?
> > 
> 
> CONFIG_X86_VSMP enables 4096-byte padding for
> ____cacheline_internodealigned_in_smp.    It's a megabyte.

Yes, only on vSMPowered systems.  Well, we have a large 
internode cacheline, but these machines have lots of memory too.  I 
thought a  simpler padding solution might be acceptable as futex_queues 
would be large only on our boxes.
Maybe we can dynamically allocate spinlocks, but it would be difficult to
say which node the spinlock should come from as yet.