Re: [RFC PATCH 0/3] Fix SLQB on memoryless configurations V2

David Rientjes
2009-09-22T17:31:56Z

On Tue, 22 Sep 2009, Mel Gorman wrote:

> Lumping the per-cpu allocator to cover per-cpu and per-node feels a bit
> confusing. Maybe it would have been easier if there simply were never
> memoryless nodes and cpus were always mapped to their closest, instead of
> their local, node. There likely would be a few corner cases though and memory
> hotplug would add to the mess. I haven't been able to decide on a sensible
> way forward that doesn't involve a number of invasive changes.
>

I strongly agree with removing memoryless node support from the kernel,
but I don't think we should substitute it with a multiple cpu binding
approach because it doesn't export the true physical topology of the
system.

If we treat all cpus equally with respect to a region of memory when one
happens to be more remote, userspace can no longer use sched_setaffinity()
for latency-sensitive apps nor can it correctly interleave with other
nodes. Reduced to its simplest form, a machine now with a single node
because memoryless nodes have been obsoleted would incorrectly report that
all cpus are true siblings.

While memoryless nodes are inconvenient for the implementation, they do
have the benefit of being able to represent the actual physical topology
when binding cpus to their nearest node, even though it may not be local,
would not.

It's important to accurately represent the physical topology, and that can
be done with the device class abstraction model as I described in
http://lkml.org/lkml/2009/9/22/97 of which a node is only a locality of a
particular type.