Re: manual page migration -- issue list

Robin Holt
2005-02-16T19:56:25Z

On Wed, Feb 16, 2005 at 11:23:35AM -0800, Paul Jackson wrote:
> Robin wrote:
> > Reading /proc/<pid>maps just scans through the vmas and not the
> > address space.
>
> Yes - you're right.
>
> So the number of system calls in your example of a few hours ago, using
> your preferred array API, if you include the reads of each tasks
> /proc/<pid>/maps file, is about equal to the number of tasks, right?
>
> And I take it that the user code you asked Ray about looks at these
> maps files for each of the tasks to be migrated, identifies each
> mapped range of each mapped object (mapped file or whatever) and
> calculates a fairly minimum set of tasks and virtual address ranges
> therein, sufficient to cover all the mapped objects that should
> be migrated, thus minimizing the amount of scanning that needs
> to be done of individual pages.
>
> And further I take it that you recommend the above described code [to
> find a fairly minimum set of tasks and address ranges to scan that will
> cover any page of interest] be put in user space, not in the kernel (a
> quite reasonable recommendation).

I think user space for a few reasons. The code in the kernel will be
much easier to digest and ensure it is a bug-free as possible. If bugs
are found or issues arise in the portions that are in userland, we are
left with a maximum amount of flexibility to correct the issue without
needing kernel code change. In a different direction, if I am a support
person trying to figure out why an application is performing poorly,
I can try migrating portions of the applications address space to a node
closer to the cpu and hopefully see a performance improvement.

>
> Why didn't your example have some writable private pages? Wouldn't such
> pages be commonplace, and wouldn't they have to be migrated for each
> thread, resulting in at least N calls to the new sys_page_migrate()
> system call, for N tasks, rather than the 3 calls in your example?

You are right about everything above. The calls to migrate the private
regions will be small in comparison to the typical large shared mapping.
The real work horse is going to always be walking the page tables and
that will take time. I am advocating for a system call which covers the
needs and also remains flexible enough to correct short comings in our
thinking about all the possible permutations of user virtual address
spaces.

Thanks,
Robin