Re: [RFC][PATCH 1/1] Expose per-node reclaim and migration to userspace

Paul Menage
2006-11-30T10:40:17Z

On 11/30/06, Nick Piggin <nickpiggin@yahoo.com.au> wrote:
>
> I know it doesn't do what you want. It is an example of using page
> migration under a higher level API, which I thought is what you
> wanted to see.

I'd been talking about the possibility of doing "try to move all
memory from this node to this other set of nodes"; that wasn't an
example of such an API.

>
> Now you're talking about physical nodes as well, which is definitely
> a problem you get when mixing the two.
>
> But there is no reason why you shouldn't be able to specify physical
> nodes, while also altering the reservation. Even if that does mean
> hiding the fake nodes from the cpuset interface.

I think it should be possible to expose the real numa topology via the
fake topology (e.g. all fake nodes on the same real node appear to be
fairly close together, compared to any fake nodes on a different real
node). So I don't think it's necessary to have a separate abstraction
for fake vs physical nodes.

>
> >> If it is exporting any kind of implementation details, then it needs
> >> to be justified with a specific user that can't be implemented in a
> >> better way, IMO.
> >
> >
> > It's not really exporting any more implementation details than the
> > existing cpusets API (i.e. explicitly binding a job to a set of nodes
> > chosen by userspace). The only true exposed implementation detail is
> > the "priority" value from try_to_free_pages, and that could be
> > abstracted away as a value in some range 0-N where 0 means "try very
> > hard" and N means "hardly try at all", and it wouldn't have to be
> > directly linked to the try_to_free_pages() priority.
>
> Or the fact that memory reservation is implemented with nodes.

Right, but to me that's a pretty fundamental design decision, rather
than an implementation detail.

> I'm
> still not convinced that idea is the best way to export memory
> control to userspace, regardless of whether it is quick and easy to
> develop (or even deploy, at google).

Maybe not the best way for all memory control, but it has certain big
advantages, such as leveraging the existing numa support, and not
requiring additional per-page overhead or LRU complexity.

Paul