Re: [patch 12/14] fs: dcache per-bucket dcache hash locking

[Nick Piggin <npiggin@suse.de>]

On Mon, Mar 30, 2009 at 02:00:31PM -0400, Christoph Hellwig wrote:
> On Mon, Mar 30, 2009 at 02:14:08PM +0200, Andi Kleen wrote:
> > npiggin@suse.de writes:
> > 
> > > We can turn the dcache hash locking from a global dcache_hash_lock into
> > > per-bucket locking.
> > 
> > Per bucket locking is typically a bad idea because you get far too
> > many locks and you increase cache footprint with all of them. It's
> > typically better to use a second much smaller hash table that only has
> > locks (by just shifting the hash value down some more) 
> > Just need to be careful to avoid too much false sharing.
> 
> Yeah, I'm also not too happy about it.  I think we're better off
> replacing the global hash with more fine-grained structures.  That might
> even some sort of simplistic tree hanging of directory dentries.
> 
> The hard part in getting anything like this done was always the amount
> of different things dcache_lock protects besiseds the hash, but Nick's
> earlier patches when actually fully implemented should sort that out.

Well yes that's the thing, dcache_hash_lock becomes inner-most and
very simply protecting dentry hash and d_hash, so it should now
be quite trivial (locking-wise) to experiment with other data
structures.

Of course any proposed new data structure will have to compete with
a scaled hash table... whether we scale it the dumb way like I did
or something like Andi suggests, I don't really know. But just to
be clear, the point of that patch is more to show how it is possible
to easily change locking. The really important parts of my series for
the moment are the first half of actually breaking up dcache_lock.

Anyway, no real objections yet, so I'll continue down this path and
see what I come up with.

If anybody knows of workloads or benchmarks to try, also let me know.
The patchset posted improved dbench-in-ram performance by about 1.6x
on my 2s8c system here so it is encouraging but not the most realistic
test :)