Re: [PATCH 1/5] cpuset memory spread basic implementation

[Ingo Molnar <mingo@elte.hu>]

* Paul Jackson <pj@sgi.com> wrote:

> Ingo asked:
> > what type of objects need to be spread (currently)? It seems that your 
> > current focus is on filesystem related objects: 
> 
> In addition to the filesystem related objects called out in this 
> current patch set, we also have some xfs directory and inode caches.  
> An xfs patch is winding its way toward lkml that will enhance the xfs 
> cache creation calls a little, so that we can pick off the particular 
> slab caches we need to be able to spread, while leaving other xfs slab 
> caches with the default node-local policy.
> 
> >  does any userspace mapped memory need to be spread 
> 
> I don't think so, but I am not entirely confident of my answer 
> tonight.  I would expect the applications I care about to place mapped 
> pages by being careful to make the first access (load or store) of 
> that page from a cpu on the node where they wanted that page placed.
> 
> So, yes, either mostly filesystem related objects, or all such.
> 
> I'm not sure which.

if that's the case, then i think the best way to express this would be 
to categorize file objects into two groups: "global" (spread-out) and 
"local". Since filesystem space is already categorized per-project, this 
is also practical for the admin to do.

i.e. mountpoints/directories/files would get a 'locality of reference' 
attribute, and whenever the VFS allocates memory related to those files, 
it will do so based on the attribute. (The attribute is inherited deeper 
in the hierarchy - i.e. setting a 'global' attribute for a mountpoint 
makes all files within that filesystem spread-out.)

this is much cleaner i think, and easy/intuitive to configure. This 
would have performance advantages over your current approach as well: 
e.g. /tmp would always stay "local", in all cpusets - while with your 
current patch they are spread out. Bigger applications (like databases) 
would set this attribute themselves - but sysadmins would do it too, on 
shared boxes.

	Ingo