Re: [Lse-tech] [PATCH] cpusets - big numa cpu and memory placement

Martin J. Bligh
2004-08-06T03:26:40Z

>> is the point to add back virtualized memory and
>> CPU numbering sets specific to each process or group of them,
>> a la cpumemsets thing you were posting a year or two ago?
>
> To answer the easy question first, no. No virtual numbering anymore. We
> might do some virtualizing in user library code, but so far as this
> patch and the kernel are aware, cpu number 17 is cpu number 17, all the
> time, using the same cpu and node numberings as used in the other kernel
> API's (setaffinity, mbind and set_mempolicy) and in the kernel cpumasks
> and nodemasks.

OK, good ;-) I don't think the kernel should have to deal with that stuff.
Sorry, it's just a little difficult to dive into a large patch without
a higher level idea what it's trying to do (which after your last email,
I think I have a much better grasp on).

...

> The existing cpu and memory placement facilities, added in 2.6,
> set_schedaffinity (for cpus) and mbind/set_mempolicy (for memory) are
> just the right thing for an individual task to manage in detail its
> placement across the resources available to it (the online cpus and
> nodes if CONFIG_CPUSET is disabled, or within the cpuset if cpusets
> are enabled).

I agree that the current mechanisms are not wholly sufficient - the
most obvious failing being that whilst you can bind a process to a
resource, there's very little support for making a resource exclusively
available to a process or set thereof.

> But they do not provide the named hierarchy with kernel enforced
> permissions and resource support required to sanely manage the
> largest multi-use systems.

Right ... but I'm kind of shocked by the size of the patch to fix what
seems like a fairly simple problem. The other thing that seems to glare
at me is the overlap between what you have here and PAGG/CKRM. Does
either cpusets depend on PAGG/CKRM or vice versa? They seem to have
similar goals, and it'd be strange to have two independant mechanisms.

> Three additional ways to approach this patch:
>
> 1) The first file in the patch, Documentation/cpusets.txt,
> describes this facility, and its purpose.
>
> 2) Look at the hooks in the rest of the kernel. I have spent much
> time minimizing these hooks, so that they are few in number,
> placed as best I could in low maintenace code in the kernel,
> and vanish if CONFIG_CPUSETS is disabled. But in addition
> to evaluating the risk and impact of the hooks, you can get
> a further sense of how cpusets works from these hooks.
> These hooks are listed in Documentation/cpusets.txt.
>
> 3) Look at the include/linux/cpusets.h header file. It shows
> the tiny interface with the rest of the kernel, which
> pretty much evaporates if CONFIG_CPUSET is disabled.

Thanks ... that'll help me. I'll try to look through it in some more
detail.

M.