Re: [patch -mm] cpusets: add memory_slab_hardwall flag

[Matt Mackall <mpm@selenic.com>]

On Tue, 2009-03-10 at 16:50 -0400, Christoph Lameter wrote:
> On Mon, 9 Mar 2009, David Rientjes wrote:
> 
> > On Mon, 9 Mar 2009, Christoph Lameter wrote:
> >
> > > > On large NUMA machines, it is currently possible for a very large
> > > > percentage (if not all) of your slab allocations to come from memory that
> > > > is distant from your application's set of allowable cpus.  Such
> > > > allocations that are long-lived would benefit from having affinity to
> > > > those processors.  Again, this is the typical use case for cpusets: to
> > > > bind memory nodes to groups of cpus with affinity to it for the tasks
> > > > attached to the cpuset.
> > >
> > > Can you show us a real workload that suffers from this issue?
> > >
> >
> > We're more interested in the isolation characteristic, but that also
> > benefits large NUMA machines by keeping nodes free of egregious amounts of
> > slab allocated for remote cpus.
> 
> So no real workload just some isolation idea.
> 
> > > If you want to make sure that an allocation comes from a certain node then
> > > specifying the node in kmalloc_node() will give you what you want.
> > >
> >
> > That's essentially what the change does implicitly: it changes all
> > kmalloc() calls to kmalloc_node() for current->mems_allowed.
> 
> Ok then you can use kmalloc_node?

Yes, he certainly could change every single kmalloc that a process might
ever reach to kmalloc_node. But I don't think that's optimal.

> 
> > > The usage of kernel objects may not be cpuset specific. This is true for
> > > other objects than inode and dentries well.
> > >
> >
> > Yes, and that's why we require the cpuset hardwall on a configurable
> > per-cpuset basis.  If a cpuset has set this option for its workload, then
> > it is demanding object allocations from local memory.  Other cpusets that
> > do not have memory_slab_hardwall set can still allocate from any cpu slab
> > or partial slab, including those allocated for the hardwall cpuset.
> 
> You cannot hardwall something that is used in a shared way by processes in
> multiple cpusets.

He can enforce that every allocation made when a given task is current
conforms. His patch demonstrates that.

-- 
http://selenic.com : development and support for Mercurial and Linux