Re: [patch 1/4] mmu_notifier: Core code

Christoph Lameter
2008-01-25T20:10:41Z

On Fri, 25 Jan 2008, Robin Holt wrote:

> Keep in mind that on a 2048p SSI MPI job starting up, we have 2048 ranks
> doing this at the same time 6 times withing their address range. That
> seems like a lock which could get hot fairly quickly. It may be for a
> short period during startup and shutdown, but it is there.

Ok. I guess we need to have a __register_mmu_notifier that expects the
mmap_sem to be held then?

> > 1. invalidate_all()
>
> That will be fine as long as we can unregister the ops notifier and free
> the structure. Otherwise, we end up being called needlessly.

No you cannot do that because there are still callbacks that come later.
The invalidate_all may lead to invalidate_range() doing nothing for this
mm. The ops notifier and the freeing of the structure has to wait until
release().

> > 2. invalidate_range() for each vma
> >
> > 3. release()
> >
> > We cannot simply move the call up because there will be future range
> > callbacks on vma invalidation.
>
> I am not sure what this means. Right now, if you were to notify XPMEM
> the process is exiting, we would take care of all the recalling of pages
> exported by this process, clearing those pages cache lines from cache,
> and raising memory protections. I would assume that moving the callout
> earlier would expect the same of every driver.

That does not sync with the current scheme of the invalidate_range()
hooks. We would have to do a global invalidate early and then place the
other invalidate_range hooks in such a way that none is called in later in
process exit handling.