[PATCH] mm/memory-failure.c: fix memory leak by race between poison and unpoison

[PATCH] mm/memory-failure.c: fix memory leak by race between poison and unpoison

[Naoya Horiguchi]

When a memory error happens on an in-use page or (free and in-use) hugepage,
the victim page is isolated with its refcount set to one. When you try to
unpoison it later, unpoison_memory() calls put_page() for it twice in order to
bring the page back to free page pool (buddy or free hugepage list.)
However, if another memory error occurs on the page which we are unpoisoning,
memory_failure() returns without releasing the refcount which was incremented
in the same call at first, which results in memory leak and unconsistent
num_poisoned_pages statistics. This patch fixes it.

Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Cc: <stable@vger.kernel.org>    [2.6.32+]
---
 mm/memory-failure.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git next-20140512.orig/mm/memory-failure.c next-20140512/mm/memory-failure.c
index 9872af1b1e9d..93a08bd78c78 100644
--- next-20140512.orig/mm/memory-failure.c
+++ next-20140512/mm/memory-failure.c
@@ -1153,6 +1153,8 @@ int memory_failure(unsigned long pfn, int trapno, int flags)
 	 */
 	if (!PageHWPoison(p)) {
 		printk(KERN_ERR "MCE %#lx: just unpoisoned\n", pfn);
+		atomic_long_sub(nr_pages, &num_poisoned_pages);
+		put_page(hpage);
 		res = 0;
 		goto out;
 	}
-- 
1.9.0

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm/memory-failure.c: fix memory leak by race between poison and unpoison

[Andrew Morton]

On Wed, 14 May 2014 11:21:31 -0400 Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> wrote:

> When a memory error happens on an in-use page or (free and in-use) hugepage,
> the victim page is isolated with its refcount set to one. When you try to
> unpoison it later, unpoison_memory() calls put_page() for it twice in order to
> bring the page back to free page pool (buddy or free hugepage list.)
> However, if another memory error occurs on the page which we are unpoisoning,
> memory_failure() returns without releasing the refcount which was incremented
> in the same call at first, which results in memory leak and unconsistent
> num_poisoned_pages statistics. This patch fixes it.
> 
> ...
>
> --- next-20140512.orig/mm/memory-failure.c
> +++ next-20140512/mm/memory-failure.c
> @@ -1153,6 +1153,8 @@ int memory_failure(unsigned long pfn, int trapno, int flags)
>  	 */
>  	if (!PageHWPoison(p)) {
>  		printk(KERN_ERR "MCE %#lx: just unpoisoned\n", pfn);
> +		atomic_long_sub(nr_pages, &num_poisoned_pages);
> +		put_page(hpage);
>  		res = 0;
>  		goto out;
>  	}

Looking at the surrounding code...

	/*
	 * Lock the page and wait for writeback to finish.
	 * It's very difficult to mess with pages currently under IO
	 * and in many cases impossible, so we just avoid it here.
	 */
	lock_page(hpage);


lock_page() doesn't wait for writeback to finish -
wait_on_page_writeback() does that.  Either the code or the comment
could do with fixing.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm/memory-failure.c: fix memory leak by race between poison and unpoison

[Naoya Horiguchi]

On Wed, May 14, 2014 at 03:10:37PM -0700, Andrew Morton wrote:
...
> Looking at the surrounding code...
> 
> 	/*
> 	 * Lock the page and wait for writeback to finish.
> 	 * It's very difficult to mess with pages currently under IO
> 	 * and in many cases impossible, so we just avoid it here.
> 	 */
> 	lock_page(hpage);
> 
> 
> lock_page() doesn't wait for writeback to finish -
> wait_on_page_writeback() does that.  Either the code or the comment
> could do with fixing.

OK, here is the patch to move the comment.

---
Subject: [PATCH] mm/memory-failure.c: move comment

The comment about pages under writeback is far from the relevant code,
so let's move it to the right place.

Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
---
 mm/memory-failure.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 93a08bd78c78..e3154d99b87f 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -1132,11 +1132,6 @@ int memory_failure(unsigned long pfn, int trapno, int flags)
 		}
 	}
 
-	/*
-	 * Lock the page and wait for writeback to finish.
-	 * It's very difficult to mess with pages currently under IO
-	 * and in many cases impossible, so we just avoid it here.
-	 */
 	lock_page(hpage);
 
 	/*
@@ -1186,6 +1181,10 @@ int memory_failure(unsigned long pfn, int trapno, int flags)
 	if (PageHuge(p))
 		set_page_hwpoison_huge_page(hpage);
 
+	/*
+	 * It's very difficult to mess with pages currently under IO
+	 * and in many cases impossible, so we just avoid it here.
+	 */
 	wait_on_page_writeback(p);
 
 	/*
-- 
1.9.0

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm/memory-failure.c: fix memory leak by race between poison and unpoison

[cyc]

a?? 2014-05-14a,?c?? 11:21 -0400i 1/4 ?Naoya Horiguchia??e??i 1/4 ?
> When a memory error happens on an in-use page or (free and in-use) hugepage,
> the victim page is isolated with its refcount set to one. When you try to
> unpoison it later, unpoison_memory() calls put_page() for it twice in order to
> bring the page back to free page pool (buddy or free hugepage list.)
> However, if another memory error occurs on the page which we are unpoisoning,
> memory_failure() returns without releasing the refcount which was incremented
> in the same call at first, which results in memory leak and unconsistent
> num_poisoned_pages statistics. This patch fixes it.

We assume that a new memory error occurs on the hugepage which we are
unpoisoning. 

          A   unpoisoned  B    poisoned    C          
hugepage: |---------------+++++++++++++++++|

There are two cases, so shown.
  1. the victim page belongs to A-B, the memory_failure will be blocked
by lock_page() until unlock_page() invoked by unpoison_memory().
  2. the victim page belongs to B-C, the memory_failure() will return
very soon at the beginning of this function.

So the new memory error will have no effect what you say so.

thx!
cyc 

> 
> Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> Cc: <stable@vger.kernel.org>    [2.6.32+]
> ---
>  mm/memory-failure.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git next-20140512.orig/mm/memory-failure.c next-20140512/mm/memory-failure.c
> index 9872af1b1e9d..93a08bd78c78 100644
> --- next-20140512.orig/mm/memory-failure.c
> +++ next-20140512/mm/memory-failure.c
> @@ -1153,6 +1153,8 @@ int memory_failure(unsigned long pfn, int trapno, int flags)
>  	 */
>  	if (!PageHWPoison(p)) {
>  		printk(KERN_ERR "MCE %#lx: just unpoisoned\n", pfn);
> +		atomic_long_sub(nr_pages, &num_poisoned_pages);
> +		put_page(hpage);
>  		res = 0;
>  		goto out;
>  	}


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm/memory-failure.c: fix memory leak by race between poison and unpoison

[Naoya Horiguchi]

On Thu, May 15, 2014 at 11:34:26AM +0800, cyc wrote:
> 在 2014-05-14三的 11:21 -0400，Naoya Horiguchi写道：
> > When a memory error happens on an in-use page or (free and in-use) hugepage,
> > the victim page is isolated with its refcount set to one. When you try to
> > unpoison it later, unpoison_memory() calls put_page() for it twice in order to
> > bring the page back to free page pool (buddy or free hugepage list.)
> > However, if another memory error occurs on the page which we are unpoisoning,
> > memory_failure() returns without releasing the refcount which was incremented
> > in the same call at first, which results in memory leak and unconsistent
> > num_poisoned_pages statistics. This patch fixes it.
> 
> We assume that a new memory error occurs on the hugepage which we are
> unpoisoning. 
> 
>           A   unpoisoned  B    poisoned    C          
> hugepage: |---------------+++++++++++++++++|
> 
> There are two cases, so shown.
>   1. the victim page belongs to A-B, the memory_failure will be blocked
> by lock_page() until unlock_page() invoked by unpoison_memory().

No. memory_failure() set PageHWPoison at first before taking page lock.
This is a design choice based on the idea that we need detect errors ASAP.
What happens in this race is like below:

    CPU 0 (poison)                 CPU 1 (unpoison)
                                   lock_page
    TestSetPageHWPoison
                                   TestClearPageHWPoison
    lock_page (wait)
                                   unlock_page
    check PageHWPoison
      printk("just unpoisoned")

>   2. the victim page belongs to B-C, the memory_failure() will return
> very soon at the beginning of this function.

Right.

Thanks,
Naoya Horiguchi

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm/memory-failure.c: fix memory leak by race between poison and unpoison

[Naoya Horiguchi]

On Thu, May 15, 2014 at 08:23:10AM -0400, Naoya Horiguchi wrote:
> On Thu, May 15, 2014 at 11:34:26AM +0800, cyc wrote:
> > 在 2014-05-14三的 11:21 -0400，Naoya Horiguchi写道：
> > > When a memory error happens on an in-use page or (free and in-use) hugepage,
> > > the victim page is isolated with its refcount set to one. When you try to
> > > unpoison it later, unpoison_memory() calls put_page() for it twice in order to
> > > bring the page back to free page pool (buddy or free hugepage list.)
> > > However, if another memory error occurs on the page which we are unpoisoning,
> > > memory_failure() returns without releasing the refcount which was incremented
> > > in the same call at first, which results in memory leak and unconsistent
> > > num_poisoned_pages statistics. This patch fixes it.
> > 
> > We assume that a new memory error occurs on the hugepage which we are
> > unpoisoning. 
> > 
> >           A   unpoisoned  B    poisoned    C          
> > hugepage: |---------------+++++++++++++++++|
> > 
> > There are two cases, so shown.
> >   1. the victim page belongs to A-B, the memory_failure will be blocked
> > by lock_page() until unlock_page() invoked by unpoison_memory().
> 
> No. memory_failure() set PageHWPoison at first before taking page lock.
> This is a design choice based on the idea that we need detect errors ASAP.

I might have not caught you, sorry. With this patch, we can properly cancel
poisoning operation when it races with unpoisoning, so no effect as you said
for both case.

Thanks,
Naoya


> What happens in this race is like below:
> 
>     CPU 0 (poison)                 CPU 1 (unpoison)
>                                    lock_page
>     TestSetPageHWPoison
>                                    TestClearPageHWPoison
>     lock_page (wait)
>                                    unlock_page
>     check PageHWPoison
>       printk("just unpoisoned")
> 
> >   2. the victim page belongs to B-C, the memory_failure() will return
> > very soon at the beginning of this function.
> 
> Right.
> 
> Thanks,
> Naoya Horiguchi
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
> 
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>