Re: Ugly rmap NULL ptr deref oopsie on hibernate (was Linux 2.6.34-rc3)

[Rik van Riel <riel@redhat.com>]

On 04/06/2010 05:27 PM, Linus Torvalds wrote:

> I still don't see _how_ it happens, though. That 'struct anon_vma' is very
> simple, and contains literally just the lock and that list_head.

It gets more fun.  It looks like the anon_vma is only
allocated through anon_vma_alloc() and only handled
by the functions in rmap.c

By themselves, all of those functions look alright.

However, I think I may have found a possible bug in
the interplay between anon_vma_prepare() and vma_adjust(),
across several mprotect invocations.

Let me explain what I think may be going on in small
steps, since it is quite subtle (assuming I am right).

1) a process forks, creating a second "layer" of
    anon_vma objects for the VMAs that have anon pages

2) a new VMA is created adjacant to an existing one,
    with different permissions

3) anon_vma_prepare is called on the new VMA, this
    only links the "top" anon_vma to the new VMA, since
    that is the anon_vma where all new pages get
    instantiated anyway   (this would be part of the bug)

4) mprotect changes the permission of one of the VMAs,
    causing the old and the new VMAs to get merged

5) vma_adjust calls anon_vma_merge, causing the anon_vma
    chain of one of the VMAs to get nuked - with bad luck,
    this is the original one, leaving just the new anon_vma
    attached to the VMA

6) if the parent process quits, the old anon_vma structs
    get freed

7) meanwhile, we may still have some anonymous pages
    stick around in memory that have their page->mapping
    point to a freed anon_vma struct

Does this look like it could happen?

If so, I'll cook up a patch to change anon_vma_prepare
and find_mergeable_anon_vma to attach the whole chain
of anon_vmas to the new VMA, using anon_vma_clone().