Re: Checking page_count(page) in invalidate_complete_page

Andrew Morton
2006-09-25T21:10:36Z

(Added linux-mm)

On Mon, 25 Sep 2006 15:51:26 -0400
Chuck Lever <chuck.lever@oracle.com> wrote:

> Hi Andrew-
>
> Steve Dickson and I have independently discovered some cache
> invalidation problems in 2.6.18's NFS client. Using git bisect, I was
> able to track it back to this commit:
>
> > commit 016eb4a0ed06a3677d67a584da901f0e9a63c666
> > Author: Andrew Morton <akpm@osdl.org>
> > Date: Fri Sep 8 09:48:38 2006 -0700
> >
> > [PATCH] invalidate_complete_page() race fix
> >
> > If a CPU faults this page into pagetables after invalidate_mapping_pages()
> > checked page_mapped(), invalidate_complete_page() will still proceed to remove
> > the page from pagecache. This leaves the page-faulting process with a
> > detached page. If it was MAP_SHARED then file data loss will ensue.
> >
> > Fix that up by checking the page's refcount after taking tree_lock.
> >
> > Cc: Nick Piggin <nickpiggin@yahoo.com.au>
> > Cc: Hugh Dickins <hugh@veritas.com>
> > Cc: <stable@kernel.org>
> > Signed-off-by: Andrew Morton <akpm@osdl.org>
> > Signed-off-by: Linus Torvalds <torvalds@osdl.org>
>
> Instrumenting get_page and put_page has shown that the page reclaim
> logic is temporarily bumping the page count on otherwise active but idle
> pages, racing with the new page_count check in invalidate_complete_page
> and preventing pages from being invalidated.
>
> One problem for the NFS client is that invalidate_inode_pages2 is being
> used to invalidate the pages associated with a cached directory. If the
> directory pages can't be invalidated because of this race, the contents
> of the directory pages don't match the dentry cache, and all kinds of
> strange behavior results.

NFS is presently ignoring the return value from invalidate_inode_pages2(),
in two places. Could I suggest you fix that? Then we'd at least not be
seeing "strange behaviour" and things will be easier to diagnose next time.

> I haven't checked the data invalidation behavior for regular files, but
> the result will be file data corruption that comes and goes.
>
> Would it be acceptable to revert that page_count(page) != 2 check in
> invalidate_complete_page ?

Unfortunately not - that patch fixes cramfs failures and potential file
corruption.

The way to keep memory reclaim away from that page is to take
zone->lru_lock, but that's quite impractical for several reasons.

We could retry the invalidation a few times, but that stinks.

I think invalidate_inode_pages2() is sufficiently different from (ie:
stronger than) invalidate_inode_pages() to justify the addition of a new
invalidate_complete_page2(), which skips the page refcount check.