Re: [BUG] Bad page states

Re: [BUG] Bad page states

[Linus Torvalds]

Adding more people, and linux-mm.

On Fri, Aug 5, 2016 at 10:13 AM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
>
> Bisected to nowhere :(
>
> Anyone has an idea ?

How easy is it for you to reproduce? It must be *fairly* easy since
you tried bisecting it, and presumably all the ones you marked "bad"
really are reliably bad.

Which means that I would expect that since the bisect failed, some of
the "good" ones (particularly at the end) might really be bad, just
didn't have the time/load to reproduce.

So maybe you could re-test the good ones for a bit longer? Trust all
the ones you've marked bad (and presumably trust v4.7 itself as good),
and re-try the bisection.

That said, it looks pretty bleak. If you don't trust any of the good
ones, you'd start out with

  git bisect start
  git bisect bad 5bbea66bf8d9ba898abbe5499f06998a993364f6
  git bisect good v4.7

and that's still almost 6000 commits.

So let's narrow it down by looking at the details:

> [   32.666450] BUG: Bad page state in process swapper/0  pfn:1fd945c
> [   32.672542] page:ffffea007f651700 count:0 mapcount:-511 mapping:          (null) index:0x0
> [   32.680823] flags: 0x1000000000000000()
> [   32.684655] page dumped because: nonzero mapcount
> ...
> [   43.477693] BUG: Bad page state in process S05containers  pfn:1ff02a3
> [   43.484417] page:ffffea007fc0a8c0 count:0 mapcount:-511 mapping:          (null) index:0x0
> [   43.492737] flags: 0x1000000000000000()
> [   43.496602] page dumped because: nonzero mapcount

Hmm. The _mapcount field is a union with other fields, but that number
doesn't make sense for any of the other fields.

So it's almost certainly related to "PAGE_KMEMCG_MAPCOUNT_VALUE". So
something presumably mapped such a page into an address space, and
incremented the number. That should never have happened, of course.

Oh. Actually, I guess it *is* PAGE_KMEMCG_MAPCOUNT_VALUE, and what
happens is that __page_mapcount() returns "_mapcount+1", so no other
increment needed.

The fact that one of the trces comes from tlb_flush_mmu_free() still
does mean that it has been mapped into a VM, though.

Unrelated to that, the "flags" field has bit 60 set, which is
presumably just part of the zone/node/section number. Maybe the
page_debug() code should print out that information too, not just the
flag bit names?

Anyway, the PAGE_KMEMCG_MAPCOUNT_VALUE connection makes me blame
Vladimir Davydov and commit 4949148ad433. Maybe you could center your
testing around that one (ie rather than bisection, try the immediate
parent, and then that commit).

And maybe the page mapping code could have some debug code for "am I
mapping a page that has a mapcount < -1", and alert people to that? To
more easily find the path that triggers this?

Vladimir?

                      Linus

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [BUG] Bad page states

[Vladimir Davydov]

On Mon, Aug 08, 2016 at 10:48:45AM -0700, Linus Torvalds wrote:
...
> > [   43.477693] BUG: Bad page state in process S05containers  pfn:1ff02a3
> > [   43.484417] page:ffffea007fc0a8c0 count:0 mapcount:-511 mapping:          (null) index:0x0
> > [   43.492737] flags: 0x1000000000000000()
> > [   43.496602] page dumped because: nonzero mapcount
> 
> Hmm. The _mapcount field is a union with other fields, but that number
> doesn't make sense for any of the other fields.
> 
> So it's almost certainly related to "PAGE_KMEMCG_MAPCOUNT_VALUE". So

Yes, it is - my bad. The thing is I set/clear PAGE_KMEMCG_MAPCOUNT_VALUE
for pages allocated with __GFP_ACCOUNT iff memcg_kmem_enabled() is true
(see __alloc_pages_nodemask and free_pages_prepare), while the latter
gets disabled when the last cgroup gets destroyed. So if you do

 mkdir /sys/fs/cgroup/memory/test
 # run something in the root cgroup that allocates pages with
 # __GFP_ACCOUNT, e.g. a program using pipe
 rmdir /sys/fs/cgroup/memory/test

Then, if there are no other memory cgroups, you'll see the bug.

Sorry about that :-(

Obviously, the PageKmemcg flag should only be set for pages that are
actually accounted to a non-root kmemcg and hence pin memcg_kmem_enabled
static key. I'll fix that.

Thanks,
Vladimir

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH] mm: memcontrol: only mark charged pages with PageKmemcg

[Vladimir Davydov]

To distinguish non-slab pages charged to kmemcg we mark them PageKmemcg,
which sets page->_mapcount to -512. Currently, we set/clear PageKmemcg in
__alloc_pages_nodemask()/free_pages_prepare() for any page allocated
with __GFP_ACCOUNT, including those that aren't actually charged to any
cgroup, i.e. allocated from the root cgroup context. To avoid overhead
in case cgroups are not used, we only do that if memcg_kmem_enabled() is
true. The latter is set iff there are kmem-enabled memory cgroups
(online or offline). The root cgroup is not considered kmem-enabled.

As a result, if a page is allocated with __GFP_ACCOUNT for the root
cgroup when there are kmem-enabled memory cgroups and is freed after all
kmem-enabled memory cgroups were removed, e.g.

  # no memory cgroups has been created yet, create one
  mkdir /sys/fs/cgroup/memory/test
  # run something allocating pages with __GFP_ACCOUNT, e.g.
  # a program using pipe
  dmesg | tail
  # remove the memory cgroup
  rmdir /sys/fs/cgroup/memory/test

we'll get bad page state bug complaining about page->_mapcount != -1:

  BUG: Bad page state in process swapper/0  pfn:1fd945c
  page:ffffea007f651700 count:0 mapcount:-511 mapping:          (null) index:0x0
  flags: 0x1000000000000000()

To avoid that, let's mark with PageKmemcg only those pages that are
actually charged to and hence pin a non-root memory cgroup.

Fixes: 4949148ad433 ("mm: charge/uncharge kmemcg from generic page allocator paths")
Reported-by: Eric Dumazet <eric.dumazet@gmail.com>
Signed-off-by: Vladimir Davydov <vdavydov@virtuozzo.com>
Cc: <stable@vger.kernel.org>	[4.7+]
---
 fs/pipe.c       |  4 +---
 mm/memcontrol.c | 14 ++++++++++++--
 mm/page_alloc.c | 14 +++++---------
 3 files changed, 18 insertions(+), 14 deletions(-)

diff --git a/fs/pipe.c b/fs/pipe.c
index 4b32928f5426..4ebe6b2e5217 100644
--- a/fs/pipe.c
+++ b/fs/pipe.c
@@ -144,10 +144,8 @@ static int anon_pipe_buf_steal(struct pipe_inode_info *pipe,
 	struct page *page = buf->page;
 
 	if (page_count(page) == 1) {
-		if (memcg_kmem_enabled()) {
+		if (memcg_kmem_enabled())
 			memcg_kmem_uncharge(page, 0);
-			__ClearPageKmemcg(page);
-		}
 		__SetPageLocked(page);
 		return 0;
 	}
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 1c0aa59fd333..7c9c0bd475f0 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -2308,8 +2308,11 @@ int memcg_kmem_charge(struct page *page, gfp_t gfp, int order)
 		return 0;
 
 	memcg = get_mem_cgroup_from_mm(current->mm);
-	if (!mem_cgroup_is_root(memcg))
+	if (!mem_cgroup_is_root(memcg)) {
 		ret = memcg_kmem_charge_memcg(page, gfp, order, memcg);
+		if (!ret)
+			__SetPageKmemcg(page);
+	}
 	css_put(&memcg->css);
 	return ret;
 }
@@ -2336,6 +2339,11 @@ void memcg_kmem_uncharge(struct page *page, int order)
 		page_counter_uncharge(&memcg->memsw, nr_pages);
 
 	page->mem_cgroup = NULL;
+
+	/* slab pages do not have PageKmemcg flag set */
+	if (PageKmemcg(page))
+		__ClearPageKmemcg(page);
+
 	css_put_many(&memcg->css, nr_pages);
 }
 #endif /* !CONFIG_SLOB */
@@ -5533,8 +5541,10 @@ static void uncharge_list(struct list_head *page_list)
 			else
 				nr_file += nr_pages;
 			pgpgout++;
-		} else
+		} else {
 			nr_kmem += 1 << compound_order(page);
+			__ClearPageKmemcg(page);
+		}
 
 		page->mem_cgroup = NULL;
 	} while (next != page_list);
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index ee5a4b20daf4..3a7d3a1fb6a0 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1009,10 +1009,8 @@ static __always_inline bool free_pages_prepare(struct page *page,
 	}
 	if (PageMappingFlags(page))
 		page->mapping = NULL;
-	if (memcg_kmem_enabled() && PageKmemcg(page)) {
+	if (memcg_kmem_enabled() && PageKmemcg(page))
 		memcg_kmem_uncharge(page, order);
-		__ClearPageKmemcg(page);
-	}
 	if (check_free)
 		bad += free_pages_check(page);
 	if (bad)
@@ -3788,12 +3786,10 @@ no_zone:
 	}
 
 out:
-	if (memcg_kmem_enabled() && (gfp_mask & __GFP_ACCOUNT) && page) {
-		if (unlikely(memcg_kmem_charge(page, gfp_mask, order))) {
-			__free_pages(page, order);
-			page = NULL;
-		} else
-			__SetPageKmemcg(page);
+	if (memcg_kmem_enabled() && (gfp_mask & __GFP_ACCOUNT) && page &&
+	    unlikely(memcg_kmem_charge(page, gfp_mask, order) != 0)) {
+		__free_pages(page, order);
+		page = NULL;
 	}
 
 	if (kmemcheck_enabled && page)
-- 
2.1.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm: memcontrol: only mark charged pages with PageKmemcg

[Linus Torvalds]

On Mon, Aug 8, 2016 at 1:03 PM, Vladimir Davydov <vdavydov@virtuozzo.com> wrote:
> To distinguish non-slab pages charged to kmemcg we mark them PageKmemcg [..]

Eric, can you confirm that this fixes your issue?

             Linus

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm: memcontrol: only mark charged pages with PageKmemcg

[Eric Dumazet]

On Mon, 2016-08-08 at 14:20 -0700, Linus Torvalds wrote:
> On Mon, Aug 8, 2016 at 1:03 PM, Vladimir Davydov <vdavydov@virtuozzo.com> wrote:
> > To distinguish non-slab pages charged to kmemcg we mark them PageKmemcg [..]
> 
> Eric, can you confirm that this fixes your issue?
> 

It is fixing the issue for me, thanks a lot Vladimir and Linus.

(I also checked that commit 4949148ad433 was indeed the bug origin)

Tested-by: Eric Dumazet <edumazet@google.com>




--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>