Re: [PATCH 00/18] Make common x86 arch area for i386 and x86_64 - Take 2

[Dave Hansen <hansendc@us.ibm.com>]

On Fri, 2007-03-16 at 13:15 -0700, Christoph Lameter wrote:
> On Fri, 16 Mar 2007, Andi Kleen wrote:
> > > x86_64 is going to acquire more functionality that will not be available 
> > > for i386. We plan f.e. to add virtual memmap support for x86_64. Virtual 
> > 
> > What advantage would that have over the current setup?
> > We already should handle holes between nodes reasonably efficiently
> > and with nonlinear memory even holes inside nodes shouldn't be a problem.
> 
> It is primarily a performance improvement since the sparsemem table 
> lookups would no longer be necessary and it also streamlines other 
> frequent cacheline uses. These page -> page_struct and vice versa 
> operations are key to the performance of various subsystem among them 
> the slab allocator.

Hi Christoph,

Yeah, those are horribly common operations.  But, have we actually
quantified how sparsemem hurts here?  I know that it look on the surface
like those lookups should kill you but, as far as I can tell, we've
never been able to show that they actually do.  We were worried on the
NUMAQ that we would see some performance regressions, but it somehow
slightly outperformed discontigmem.

Do you have any hard data that shows vmemmap killing sparsemem on some
of your workloads?  I'd love to try and fix some of the issues if we can
dig them up.

One of the reasons I really like sparsemem is that ports to new
architectures are something like 100 lines of code, including all of the
#defines.  It's really ridiculously easy to do.  One of the things that
I really worry about for vmemmap implementations is how complicated they
get to implement.  The fact that we can't easily do it for both i386 and
x86_64 at the same time speaks to this a bit.

-- Dave