Re: [rfc][patch] Memory Binding API v0.3 2.5.41

Eric W. Biederman
2002-10-15T17:17:05Z

Matthew Dobson <colpatch@us.ibm.com> writes:

> Eric W. Biederman wrote:
> > Matthew Dobson <colpatch@us.ibm.com> writes:
> >>Greetings & Salutations,
> >> Here's a wonderful patch that I know you're all dying for... Memory
> >>Binding! It works just like CPU Affinity (binding) except that it binds a
> >>processes memory allocations (just buddy allocator for now) to specific memory
>
> >>blocks.
> > Due we want this per numa area or simply per zone? My suspicion is that
> > internally at least we want this per zone.
> I think that per memory block is better.
[snip]
> I'm not fanatically
> opposed to per zone binding, though, and if there is a general agreement that it
> would be better that way, I don't think it would be unreasonably difficult to
> change it.

My only feeling with zones is that it could be useful in the non numa cases,
if it was per zone.

But unless this API becomes is a pure hint we need at least one specifier that
says writing to swap is o.k.

> > The API doesn't make much sense at the moment.
> Hmm.. That is unfortunate, I'd aimed to make it as simple as possible.

Simple is good only if the proper pieces are connected.

> > 1) You are operating on tasks and not mm's, or preferably vmas.
> Correct. There are plans (somewhere inside my cranium) to allow binding at that
>
> granularity. For now, per task seemed an appropriate level.

It makes it terribly unpredictable. If you have two threads each bound
to a different location there are race conditions which area the memory
is allocated from.

> > 2) sys_mem_setbinding does not move the mm to the new binding.
> Also correct. A task may wish to allocate several large data structures from
> one memory area, rebind, do more allocations, rebind, ad nauseum. There are
> plans to have a flag that, if set, would force relocation of all currently
> allocated memory.

Actually the bindings need to stick to the vma or to the struct address_space.
Otherwise you are talking about an allocation hint, as swapping can trivially
undue it and nothing happens when the actual call is made. A hint is a very
different thing from a binding.

And if we stick this to struct address_space for the non anonymous cases
having a fmem_setbinding(struct fd) that works on files would be a useful
thing as well.

> > 5) mprotect is the more natural model rather than set_cpu_affinity.
> Well, I think that may be true for the API you are imagining (per zone, per
> mm/vma, etc), not the one that I've written.

For a binding with respect to memory I imagine things like mlock(). For
anything else you are talking a future hint to the memory allocators, which
feels less much useful.

Eric