Re: Avoid allocating during interleave from almost full nodes

[Andrew Morton <akpm@osdl.org>]

On Fri, 3 Nov 2006 16:28:31 -0800 (PST)
Christoph Lameter <clameter@sgi.com> wrote:

> On Fri, 3 Nov 2006, Andrew Morton wrote:
> 
> > But in this application which you are proposing, any correlation with
> > elapsed walltime is very slight.  It's just the wrong baseline to use. 
> > What is the *sense* in it?
> 
> You just accepted Paul's use of a similar mechanism to void cached 
> zonelists. He has a one second timeout for the cache there it seems.

With complaints.

> The sense is that memory on nodes may be freed and then we need to 
> allocate from those nodes again.

This has almost nothing to do with elapsed time.

How about doing, in free_pages_bulk():

	if (zone->over_interleave_pages) {
		zone->over_interleave_pages = 0;
		node_clear(zone_to_nid(zone), full_interleave_nodes);
	}

?

> > Yes.  And it is wrong to do so.  Because a node may well have no "free"
> > pages but plenty of completely stale ones which should be reclaimed.
> 
> But there may be other nodes that have more free pages. If we allocate 
> from those then we can avoid reclaim.
> 
> > Reclaim isn't expensive.
> 
> It is needlessly expensive if its done for an allocation that is not bound 
> to a specific node and there are other nodes with free pages. We may throw 
> out pages that we need later.

Well it grossly changes the meaning of "interleaving".  We might as well
call it something else.  It's not necessarily worse, but it's not
interleaved any more.

Actually by staying on the same node for a string of successive allocations
it could well be quicker.  How come MPOL_INTERLEAVE doesn't already do some
batching?   Or does it, and I missed it?