Re: [RFC 2.6.11-rc2-mm2 0/7] mm: manual page migration -- overview

[Marcelo Tosatti <marcelo.tosatti@cyclades.com>]

On Sat, Feb 12, 2005 at 01:54:26PM -0200, Marcelo Tosatti wrote:
> On Sat, Feb 12, 2005 at 12:17:25PM +0100, Andi Kleen wrote:
> > Ray Bryant <raybry@sgi.com> writes:
> > > set of pages associated with a particular process need to be moved.
> > > The kernel interface that we are proposing is the following:
> > >
> > > page_migrate(pid, va_start, va_end, count, old_nodes, new_nodes);
> > 
> > [Only commenting on the interface, haven't read your patches at all]
> > 
> > This is basically mbind() with MPOL_F_STRICT, except that it has a pid 
> > argument. I assume that's for the benefit of your batch scheduler.
> 
> As far as I understand mbind() is used to set policies to given memory 
> regions, not move memory regions?
> 
> > But it's not clear to me how and why the batch scheduler should know about
> > virtual addresses of different processes anyways. Walking
> > /proc/pid/maps? That's all inherently racy when the process is doing
> > mmap in parallel. The only way I can think of to do this would be to
> > check for changes in maps after a full move and loop, but then you risk
> > livelock.
> 
> True. 
> 
> There is no problem, however, if all threads beloging to the process are stopped, 
> as Ray mentions. 
> 
> So, there wont be memory mapping changes happening at the same time. 
> 
> Note that the memory migration code which sys_page_migrate() uses moves
> running processes to other memory zones, handling truncate, etc.
> 
> > And you cannot also just specify va_start=0, va_end=~0UL because that
> > would make the node arrays grow infinitely. 
> > 
> > Also is there a good use case why the batch scheduler should only
> > move individual areas in a process around, not the full process?
> 
> Quoting him:
> 
> "In addition to its use by batch schedulers, we also envision that
> this facility could be used by a program to re-arrange the allocation
> of its own pages on various nodes of the NUMA system, most likely
> to optimize performance of the application during different phases
> of its computation."
> 
> Seems doable. 
> 
> Are there any good xamples of optimizations that could be made by 
> moving pages around except for NUMA?

If you have virtually indexed caches moving pages around can optimize cache behaviour 
if program access pattern is well known? That is not a thing common thing 
to do - and is architecture dependant anyway.