Re: [RFC, PATCH] Avoid hot statistics cache line in ext4 extent cache

[Ted Ts'o <tytso@mit.edu>]

On Mon, Mar 26, 2012 at 04:00:47PM -0700, Andi Kleen wrote:
> On 3/26/2012 3:26 PM, Vivek Haldar wrote:
> >Andi --
> >
> >I realized the problem soon after the original patch, and submitted
> >another patch to make these per cpu  counters.
> 
> Is there a clear use case having these counters on every production system?

Today, with the current single entry extent cache, I don't think
there's a good justification for it, no.

Vivek has worked on a rather more sophisticated extent cache which
could cache several extent entries (and indeed, combine multiple
on-disk extent entries into a single in-memory extent).  There are a
variety of reasons that hasn't gone upstream yet; one of which is
there are some interesting questions about how to control memory usage
of the extent cache; how do we trim it back in the case of memory
pressure?

One of the other things that we need to consider as we think about
getting this upstream is the "status" or "delayed" extents patches
which Allison and Yongqiang were looking at.  Does it make sense to
have two parallel datastructures which are indexed by logical block
number?  On the one hand, using an in-memory tree structure is pretty
expensive, just because of all of the 64-bit logical block numbers and
64-bit pointers.  On the other hand, would that make things too
complicated?

Once we start having multiple knobs to adjust, having these counters
available does make sense.  For now, using a per-cpu counter is
relatively low cost, except on extreme SGI Altix-like machines with
hundreds of CPU's, where the memory utilization is something to think
about.  Given that Vivek has submitted a patch to convert to per-cpu,
I can see applying it just to fix it; or just removing the stats for
now until we get the more sophisticated extent cache merged in.

    	     	     	  		       - Ted