mm: kernel BUG at include/linux/swapops.h:131!

mm: kernel BUG at include/linux/swapops.h:131!

[Sasha Levin]

Hi all,

While fuzzing with trinity inside a KVM tools guest running latest -next kernel, I've stumbled on 
the following spew.

The code is in zap_pte_range():

                 if (!non_swap_entry(entry))
                         rss[MM_SWAPENTS]--;
                 else if (is_migration_entry(entry)) {
                         struct page *page;

                         page = migration_entry_to_page(entry);	<==== HERE

                         if (PageAnon(page))
                                 rss[MM_ANONPAGES]--;
                         else
                                 rss[MM_FILEPAGES]--;


[ 2622.589064] kernel BUG at include/linux/swapops.h:131!
[ 2622.589064] invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
[ 2622.589064] Dumping ftrace buffer:
[ 2622.589064]    (ftrace buffer empty)
[ 2622.589064] Modules linked in:
[ 2622.589064] CPU: 9 PID: 15984 Comm: trinity-child16 Tainted: G        W    3.13.0-rc
4-next-20131217-sasha-00013-ga878504-dirty #4150
[ 2622.589064] task: ffff88168346b000 ti: ffff8816561d8000 task.ti: ffff8816561d8000
[ 2622.589064] RIP: 0010:[<ffffffff8127c730>]  [<ffffffff8127c730>] zap_pte_range+0x360
/0x4a0
[ 2622.589064] RSP: 0018:ffff8816561d9c18  EFLAGS: 00010246
[ 2622.589064] RAX: ffffea00736a6600 RBX: ffff88200299d068 RCX: 0000000000000009
[ 2622.589064] RDX: 022fffff80380000 RSI: ffffea0000000000 RDI: 3c00000001cda998
[ 2622.589064] RBP: ffff8816561d9cb8 R08: 0000000000000000 R09: 0000000000000000
[ 2622.589064] R10: 0000000000000001 R11: 0000000000000000 R12: 00007fc7ee20d000
[ 2622.589064] R13: ffff8816561d9de8 R14: 000000039b53303c R15: 00007fc7ee29b000
[ 2622.589064] FS:  00007fc7eeceb700(0000) GS:ffff882011a00000(0000) knlGS:000000000000
0000
[ 2622.589064] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 2622.589064] CR2: 000000000068c000 CR3: 0000000005c26000 CR4: 00000000000006e0
[ 2622.589064] Stack:
[ 2622.589064]  ffff8816561d9c58 0000000000000286 ffff88168327b060 ffff88168327b060
[ 2622.589064]  00007fc700000160 ffff881667067b88 00000000c8f4b120 00ff88168327b060
[ 2622.589064]  ffff88168327b060 ffff8820051a8600 0000000000000000 ffff88168327b000
[ 2622.589064] Call Trace:
[ 2622.589064]  [<ffffffff8127cc5e>] unmap_page_range+0x3ee/0x400
[ 2622.589064]  [<ffffffff8127cd71>] unmap_single_vma+0x101/0x120
[ 2622.589064]  [<ffffffff8127cdf1>] unmap_vmas+0x61/0xa0
[ 2622.589064]  [<ffffffff81283980>] exit_mmap+0xd0/0x170
[ 2622.589064]  [<ffffffff8112d430>] mmput+0x70/0xe0
[ 2622.589064]  [<ffffffff8113144d>] exit_mm+0x18d/0x1a0
[ 2622.589064]  [<ffffffff811defb5>] ? acct_collect+0x175/0x1b0
[ 2622.589064]  [<ffffffff8113389f>] do_exit+0x24f/0x500
[ 2622.589064]  [<ffffffff81133bf9>] do_group_exit+0xa9/0xe0
[ 2622.589064]  [<ffffffff81133c47>] SyS_exit_group+0x17/0x20
[ 2622.589064]  [<ffffffff843a6150>] tracesys+0xdd/0xe2
[ 2622.589064] Code: 83 f8 1f 75 46 48 b8 ff ff ff ff ff ff ff 01 48 be 00 00 00 00 00 ea ff ff 48 
21 f8 48 c1 e0 06 48 01 f0 48 8b 10 80 e2 01 75 0a <0f> 0b 66 0f 1f 44 00 00 eb fe f6 40 08 01 74 05 
ff 4d c4 eb 0b
[ 2622.589064] RIP  [<ffffffff8127c730>] zap_pte_range+0x360/0x4a0
[ 2622.589064]  RSP <ffff8816561d9c18>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: mm: kernel BUG at include/linux/swapops.h:131!

[Cyrill Gorcunov]

On Wed, Dec 18, 2013 at 10:37:39AM -0500, Sasha Levin wrote:
> Hi all,
> 
> While fuzzing with trinity inside a KVM tools guest running latest
> -next kernel, I've stumbled on the following spew.
> 
> The code is in zap_pte_range():
> 
>                 if (!non_swap_entry(entry))
>                         rss[MM_SWAPENTS]--;
>                 else if (is_migration_entry(entry)) {
>                         struct page *page;
> 
>                         page = migration_entry_to_page(entry);	<==== HERE
> 
>                         if (PageAnon(page))
>                                 rss[MM_ANONPAGES]--;
>                         else
>                                 rss[MM_FILEPAGES]--;

This I think is my issue, I'll take a look, thanks Sasha!

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: mm: kernel BUG at include/linux/swapops.h:131!

[Cyrill Gorcunov]

On Wed, Dec 18, 2013 at 07:41:45PM +0400, Cyrill Gorcunov wrote:
> On Wed, Dec 18, 2013 at 10:37:39AM -0500, Sasha Levin wrote:
> > Hi all,
> > 
> > While fuzzing with trinity inside a KVM tools guest running latest
> > -next kernel, I've stumbled on the following spew.
> > 
> > The code is in zap_pte_range():
> > 
> >                 if (!non_swap_entry(entry))
> >                         rss[MM_SWAPENTS]--;
> >                 else if (is_migration_entry(entry)) {
> >                         struct page *page;
> > 
> >                         page = migration_entry_to_page(entry);	<==== HERE
> > 
> >                         if (PageAnon(page))
> >                                 rss[MM_ANONPAGES]--;
> >                         else
> >                                 rss[MM_FILEPAGES]--;
> 
> This I think is my issue, I'll take a look, thanks Sasha!

Ah, no. I thought it somehow related to dirty tracking, but
it's not, different issue.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: mm: kernel BUG at include/linux/swapops.h:131!

[Sasha Levin]

Ping?

I've also Cc'ed the "this page shouldn't be locked at all" team.

On 12/18/2013 10:37 AM, Sasha Levin wrote:
> Hi all,
>
> While fuzzing with trinity inside a KVM tools guest running latest -next kernel, I've stumbled on
> the following spew.
>
> The code is in zap_pte_range():
>
>                  if (!non_swap_entry(entry))
>                          rss[MM_SWAPENTS]--;
>                  else if (is_migration_entry(entry)) {
>                          struct page *page;
>
>                          page = migration_entry_to_page(entry);    <==== HERE
>
>                          if (PageAnon(page))
>                                  rss[MM_ANONPAGES]--;
>                          else
>                                  rss[MM_FILEPAGES]--;
>
>
> [ 2622.589064] kernel BUG at include/linux/swapops.h:131!
> [ 2622.589064] invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
> [ 2622.589064] Dumping ftrace buffer:
> [ 2622.589064]    (ftrace buffer empty)
> [ 2622.589064] Modules linked in:
> [ 2622.589064] CPU: 9 PID: 15984 Comm: trinity-child16 Tainted: G        W    3.13.0-rc
> 4-next-20131217-sasha-00013-ga878504-dirty #4150
> [ 2622.589064] task: ffff88168346b000 ti: ffff8816561d8000 task.ti: ffff8816561d8000
> [ 2622.589064] RIP: 0010:[<ffffffff8127c730>]  [<ffffffff8127c730>] zap_pte_range+0x360
> /0x4a0
> [ 2622.589064] RSP: 0018:ffff8816561d9c18  EFLAGS: 00010246
> [ 2622.589064] RAX: ffffea00736a6600 RBX: ffff88200299d068 RCX: 0000000000000009
> [ 2622.589064] RDX: 022fffff80380000 RSI: ffffea0000000000 RDI: 3c00000001cda998
> [ 2622.589064] RBP: ffff8816561d9cb8 R08: 0000000000000000 R09: 0000000000000000
> [ 2622.589064] R10: 0000000000000001 R11: 0000000000000000 R12: 00007fc7ee20d000
> [ 2622.589064] R13: ffff8816561d9de8 R14: 000000039b53303c R15: 00007fc7ee29b000
> [ 2622.589064] FS:  00007fc7eeceb700(0000) GS:ffff882011a00000(0000) knlGS:000000000000
> 0000
> [ 2622.589064] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 2622.589064] CR2: 000000000068c000 CR3: 0000000005c26000 CR4: 00000000000006e0
> [ 2622.589064] Stack:
> [ 2622.589064]  ffff8816561d9c58 0000000000000286 ffff88168327b060 ffff88168327b060
> [ 2622.589064]  00007fc700000160 ffff881667067b88 00000000c8f4b120 00ff88168327b060
> [ 2622.589064]  ffff88168327b060 ffff8820051a8600 0000000000000000 ffff88168327b000
> [ 2622.589064] Call Trace:
> [ 2622.589064]  [<ffffffff8127cc5e>] unmap_page_range+0x3ee/0x400
> [ 2622.589064]  [<ffffffff8127cd71>] unmap_single_vma+0x101/0x120
> [ 2622.589064]  [<ffffffff8127cdf1>] unmap_vmas+0x61/0xa0
> [ 2622.589064]  [<ffffffff81283980>] exit_mmap+0xd0/0x170
> [ 2622.589064]  [<ffffffff8112d430>] mmput+0x70/0xe0
> [ 2622.589064]  [<ffffffff8113144d>] exit_mm+0x18d/0x1a0
> [ 2622.589064]  [<ffffffff811defb5>] ? acct_collect+0x175/0x1b0
> [ 2622.589064]  [<ffffffff8113389f>] do_exit+0x24f/0x500
> [ 2622.589064]  [<ffffffff81133bf9>] do_group_exit+0xa9/0xe0
> [ 2622.589064]  [<ffffffff81133c47>] SyS_exit_group+0x17/0x20
> [ 2622.589064]  [<ffffffff843a6150>] tracesys+0xdd/0xe2
> [ 2622.589064] Code: 83 f8 1f 75 46 48 b8 ff ff ff ff ff ff ff 01 48 be 00 00 00 00 00 ea ff ff 48
> 21 f8 48 c1 e0 06 48 01 f0 48 8b 10 80 e2 01 75 0a <0f> 0b 66 0f 1f 44 00 00 eb fe f6 40 08 01 74 05
> ff 4d c4 eb 0b
> [ 2622.589064] RIP  [<ffffffff8127c730>] zap_pte_range+0x360/0x4a0
> [ 2622.589064]  RSP <ffff8816561d9c18>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: mm: kernel BUG at include/linux/swapops.h:131!

[Joonsoo Kim]

On Mon, Dec 23, 2013 at 12:24:02PM -0500, Sasha Levin wrote:
> Ping?
> 
> I've also Cc'ed the "this page shouldn't be locked at all" team.

Hello,

I can't find the reason of this problem.
If it is reproducible, how about bisecting?

Thanks.

> 
> On 12/18/2013 10:37 AM, Sasha Levin wrote:
> >Hi all,
> >
> >While fuzzing with trinity inside a KVM tools guest running latest -next kernel, I've stumbled on
> >the following spew.
> >
> >The code is in zap_pte_range():
> >
> >                 if (!non_swap_entry(entry))
> >                         rss[MM_SWAPENTS]--;
> >                 else if (is_migration_entry(entry)) {
> >                         struct page *page;
> >
> >                         page = migration_entry_to_page(entry);    <==== HERE
> >
> >                         if (PageAnon(page))
> >                                 rss[MM_ANONPAGES]--;
> >                         else
> >                                 rss[MM_FILEPAGES]--;
> >
> >
> >[ 2622.589064] kernel BUG at include/linux/swapops.h:131!
> >[ 2622.589064] invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
> >[ 2622.589064] Dumping ftrace buffer:
> >[ 2622.589064]    (ftrace buffer empty)
> >[ 2622.589064] Modules linked in:
> >[ 2622.589064] CPU: 9 PID: 15984 Comm: trinity-child16 Tainted: G        W    3.13.0-rc
> >4-next-20131217-sasha-00013-ga878504-dirty #4150
> >[ 2622.589064] task: ffff88168346b000 ti: ffff8816561d8000 task.ti: ffff8816561d8000
> >[ 2622.589064] RIP: 0010:[<ffffffff8127c730>]  [<ffffffff8127c730>] zap_pte_range+0x360
> >/0x4a0
> >[ 2622.589064] RSP: 0018:ffff8816561d9c18  EFLAGS: 00010246
> >[ 2622.589064] RAX: ffffea00736a6600 RBX: ffff88200299d068 RCX: 0000000000000009
> >[ 2622.589064] RDX: 022fffff80380000 RSI: ffffea0000000000 RDI: 3c00000001cda998
> >[ 2622.589064] RBP: ffff8816561d9cb8 R08: 0000000000000000 R09: 0000000000000000
> >[ 2622.589064] R10: 0000000000000001 R11: 0000000000000000 R12: 00007fc7ee20d000
> >[ 2622.589064] R13: ffff8816561d9de8 R14: 000000039b53303c R15: 00007fc7ee29b000
> >[ 2622.589064] FS:  00007fc7eeceb700(0000) GS:ffff882011a00000(0000) knlGS:000000000000
> >0000
> >[ 2622.589064] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> >[ 2622.589064] CR2: 000000000068c000 CR3: 0000000005c26000 CR4: 00000000000006e0
> >[ 2622.589064] Stack:
> >[ 2622.589064]  ffff8816561d9c58 0000000000000286 ffff88168327b060 ffff88168327b060
> >[ 2622.589064]  00007fc700000160 ffff881667067b88 00000000c8f4b120 00ff88168327b060
> >[ 2622.589064]  ffff88168327b060 ffff8820051a8600 0000000000000000 ffff88168327b000
> >[ 2622.589064] Call Trace:
> >[ 2622.589064]  [<ffffffff8127cc5e>] unmap_page_range+0x3ee/0x400
> >[ 2622.589064]  [<ffffffff8127cd71>] unmap_single_vma+0x101/0x120
> >[ 2622.589064]  [<ffffffff8127cdf1>] unmap_vmas+0x61/0xa0
> >[ 2622.589064]  [<ffffffff81283980>] exit_mmap+0xd0/0x170
> >[ 2622.589064]  [<ffffffff8112d430>] mmput+0x70/0xe0
> >[ 2622.589064]  [<ffffffff8113144d>] exit_mm+0x18d/0x1a0
> >[ 2622.589064]  [<ffffffff811defb5>] ? acct_collect+0x175/0x1b0
> >[ 2622.589064]  [<ffffffff8113389f>] do_exit+0x24f/0x500
> >[ 2622.589064]  [<ffffffff81133bf9>] do_group_exit+0xa9/0xe0
> >[ 2622.589064]  [<ffffffff81133c47>] SyS_exit_group+0x17/0x20
> >[ 2622.589064]  [<ffffffff843a6150>] tracesys+0xdd/0xe2
> >[ 2622.589064] Code: 83 f8 1f 75 46 48 b8 ff ff ff ff ff ff ff 01 48 be 00 00 00 00 00 ea ff ff 48
> >21 f8 48 c1 e0 06 48 01 f0 48 8b 10 80 e2 01 75 0a <0f> 0b 66 0f 1f 44 00 00 eb fe f6 40 08 01 74 05
> >ff 4d c4 eb 0b
> >[ 2622.589064] RIP  [<ffffffff8127c730>] zap_pte_range+0x360/0x4a0
> >[ 2622.589064]  RSP <ffff8816561d9c18>
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: mm: kernel BUG at include/linux/swapops.h:131!

[Sasha Levin]

On 12/23/2013 09:51 PM, Joonsoo Kim wrote:
> On Mon, Dec 23, 2013 at 12:24:02PM -0500, Sasha Levin wrote:
>> >Ping?
>> >
>> >I've also Cc'ed the "this page shouldn't be locked at all" team.
> Hello,
>
> I can't find the reason of this problem.
> If it is reproducible, how about bisecting?

While it reproduces under fuzzing it's pretty hard to bisect it with
the amount of issues uncovered by trinity recently.

I can add any debug code to the site of the BUG if that helps.


Thanks,
Sasha

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: mm: kernel BUG at include/linux/swapops.h:131!

[Joonsoo Kim]

On Mon, Dec 23, 2013 at 10:01:10PM -0500, Sasha Levin wrote:
> On 12/23/2013 09:51 PM, Joonsoo Kim wrote:
> >On Mon, Dec 23, 2013 at 12:24:02PM -0500, Sasha Levin wrote:
> >>>Ping?
> >>>
> >>>I've also Cc'ed the "this page shouldn't be locked at all" team.
> >Hello,
> >
> >I can't find the reason of this problem.
> >If it is reproducible, how about bisecting?
> 
> While it reproduces under fuzzing it's pretty hard to bisect it with
> the amount of issues uncovered by trinity recently.
> 
> I can add any debug code to the site of the BUG if that helps.

Good!
It will be helpful to add dump_page() in migration_entry_to_page().

Thanks.

--------8<------
diff --git a/include/linux/swapops.h b/include/linux/swapops.h
index c0f7526..f695abc 100644
--- a/include/linux/swapops.h
+++ b/include/linux/swapops.h
@@ -3,6 +3,7 @@
 
 #include <linux/radix-tree.h>
 #include <linux/bug.h>
+#include <linux/mm.h>
 
 /*
  * swapcache pages are stored in the swapper_space radix tree.  We want to
@@ -128,6 +129,8 @@ static inline struct page *migration_entry_to_page(swp_entry_t entry)
         * Any use of migration entries may only occur while the
         * corresponding page is locked
         */
+       if (!PageLocked(p))
+               dump_page(p);
        BUG_ON(!PageLocked(p));
        return p;
 }

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: mm: kernel BUG at include/linux/swapops.h:131!

[Joonsoo Kim]

On Tue, Dec 24, 2013 at 03:07:05PM +0900, Joonsoo Kim wrote:
> On Mon, Dec 23, 2013 at 10:01:10PM -0500, Sasha Levin wrote:
> > On 12/23/2013 09:51 PM, Joonsoo Kim wrote:
> > >On Mon, Dec 23, 2013 at 12:24:02PM -0500, Sasha Levin wrote:
> > >>>Ping?
> > >>>
> > >>>I've also Cc'ed the "this page shouldn't be locked at all" team.
> > >Hello,
> > >
> > >I can't find the reason of this problem.
> > >If it is reproducible, how about bisecting?
> > 
> > While it reproduces under fuzzing it's pretty hard to bisect it with
> > the amount of issues uncovered by trinity recently.
> > 
> > I can add any debug code to the site of the BUG if that helps.
> 
> Good!
> It will be helpful to add dump_page() in migration_entry_to_page().
> 
> Thanks.
> 

Minchan teaches me that there is possible race condition between
fork and migration.

Please consider following situation.


Process A (do migration)			Process B (parents) Process C (child)

try_to_unmap() for migration <begin>		fork
setup migration entry to B's vma
...
try_to_unmap() for migration <end>
move_to_new_page()

						link new vma
						    into interval tree
remove_migration_ptes() <begin>
check and clear migration entry on C's vma
...						copy_one_pte:
...						    now, B and C have migration entry
...
...
check and clear migration entry on B's vma
...
...
remove_migration_ptes() <end>


Eventually, migration entry on C's vma is left.
And then, when C exits, above BUG_ON() can be triggered.

I'm not sure the I am right, so please think of it together. :)
And I'm not sure again that above assumption is related to this trigger report,
since this may exist for a long time.

So my question to mm folks is is above assumption possible and do we have
any protection mechanism on this race?

Thanks.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: mm: kernel BUG at include/linux/swapops.h:131!

[Wanpeng Li]

Hi Joonsoo,
On Tue, Dec 24, 2013 at 04:45:46PM +0900, Joonsoo Kim wrote:
>On Tue, Dec 24, 2013 at 03:07:05PM +0900, Joonsoo Kim wrote:
>> On Mon, Dec 23, 2013 at 10:01:10PM -0500, Sasha Levin wrote:
>> > On 12/23/2013 09:51 PM, Joonsoo Kim wrote:
>> > >On Mon, Dec 23, 2013 at 12:24:02PM -0500, Sasha Levin wrote:
>> > >>>Ping?
>> > >>>
>> > >>>I've also Cc'ed the "this page shouldn't be locked at all" team.
>> > >Hello,
>> > >
>> > >I can't find the reason of this problem.
>> > >If it is reproducible, how about bisecting?
>> > 
>> > While it reproduces under fuzzing it's pretty hard to bisect it with
>> > the amount of issues uncovered by trinity recently.
>> > 
>> > I can add any debug code to the site of the BUG if that helps.
>> 
>> Good!
>> It will be helpful to add dump_page() in migration_entry_to_page().
>> 
>> Thanks.
>> 
>
>Minchan teaches me that there is possible race condition between
>fork and migration.
>
>Please consider following situation.
>
>
>Process A (do migration)			Process B (parents) Process C (child)
>
>try_to_unmap() for migration <begin>		fork
>setup migration entry to B's vma
>...
>try_to_unmap() for migration <end>
>move_to_new_page()
>
>						link new vma
>						    into interval tree
>remove_migration_ptes() <begin>
>check and clear migration entry on C's vma
>...						copy_one_pte:
>...						    now, B and C have migration entry
>...

>From Sasha's report:

| [ 3800.520039] page:ffffea0000245800 count:12 mapcount:4 mapping:ffff88001d0c3668 index:0x7de
| [ 3800.521404] page flags: 0x1fffff8038003c(referenced|uptodate|dirty|lru|swapbacked|unevictable|mlocked)
| [ 3800.522585] pc:ffff88001ed91600 pc->flags:2 pc->mem_cgroup:ffffc90000c0a000

IIUC, C's mapcount should be 0 as B's in the race condition you mentioned. 

Regards,
Wanpeng Li 

>...
>check and clear migration entry on B's vma
>...
>...
>remove_migration_ptes() <end>
>
>
>Eventually, migration entry on C's vma is left.
>And then, when C exits, above BUG_ON() can be triggered.
>
>I'm not sure the I am right, so please think of it together. :)
>And I'm not sure again that above assumption is related to this trigger report,
>since this may exist for a long time.
>
>So my question to mm folks is is above assumption possible and do we have
>any protection mechanism on this race?
>
>Thanks.
>
>--
>To unsubscribe, send a message with 'unsubscribe linux-mm' in
>the body to majordomo@kvack.org.  For more info on Linux MM,
>see: http://www.linux-mm.org/ .
>Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: mm: kernel BUG at include/linux/swapops.h:131!

[Bob Liu]

On 12/24/2013 03:45 PM, Joonsoo Kim wrote:
> On Tue, Dec 24, 2013 at 03:07:05PM +0900, Joonsoo Kim wrote:
>> On Mon, Dec 23, 2013 at 10:01:10PM -0500, Sasha Levin wrote:
>>> On 12/23/2013 09:51 PM, Joonsoo Kim wrote:
>>>> On Mon, Dec 23, 2013 at 12:24:02PM -0500, Sasha Levin wrote:
>>>>>> Ping?
>>>>>>
>>>>>> I've also Cc'ed the "this page shouldn't be locked at all" team.
>>>> Hello,
>>>>
>>>> I can't find the reason of this problem.
>>>> If it is reproducible, how about bisecting?
>>>
>>> While it reproduces under fuzzing it's pretty hard to bisect it with
>>> the amount of issues uncovered by trinity recently.
>>>
>>> I can add any debug code to the site of the BUG if that helps.
>>
>> Good!
>> It will be helpful to add dump_page() in migration_entry_to_page().
>>
>> Thanks.
>>
> 
> Minchan teaches me that there is possible race condition between
> fork and migration.
> 
> Please consider following situation.
> 
> 
> Process A (do migration)			Process B (parents) Process C (child)
> 
> try_to_unmap() for migration <begin>		fork
> setup migration entry to B's vma
> ...
> try_to_unmap() for migration <end>
> move_to_new_page()
> 
> 						link new vma
> 						    into interval tree
> remove_migration_ptes() <begin>
> check and clear migration entry on C's vma
> ...						copy_one_pte:
> ...						    now, B and C have migration entry
> ...
> ...
> check and clear migration entry on B's vma
> ...
> ...
> remove_migration_ptes() <end>
> 
> 
> Eventually, migration entry on C's vma is left.
> And then, when C exits, above BUG_ON() can be triggered.
> 

Yes, Looks like this is a potential race condition.

> I'm not sure the I am right, so please think of it together. :)
> And I'm not sure again that above assumption is related to this trigger report,
> since this may exist for a long time.
> 
> So my question to mm folks is is above assumption possible and do we have
> any protection mechanism on this race?
> 

I think we can down_read(&mm->mmap_sem) before remove_migration_ptes()
to fix this issue, but I don't have time to verify it currently.

-- 
Regards,
-Bob

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: mm: kernel BUG at include/linux/swapops.h:131!

[Konstantin Khlebnikov]

[-- Attachment #1: Type: text/plain, Size: 3273 bytes --]

Hmm. This kind of race looks impossible: dup_mmap() always places child's
vma in into rmap tree after parent's one. For file-vma it's done explicitly
(vma_interval_tree_insert_after), for anon vma it's true because rb-tree
insert function goes to right branch if elements are equal.

Thus remove_migration_ptes() sees parent's pte first:
If child has the copy this function will check it after that.
And they are already synchronized with parent's and child's pte locks.

On Dec 26, 2013 10:21 AM, "Bob Liu" <bob.liu@oracle.com> wrote:
>
> On 12/24/2013 03:45 PM, Joonsoo Kim wrote:
> > On Tue, Dec 24, 2013 at 03:07:05PM +0900, Joonsoo Kim wrote:
> >> On Mon, Dec 23, 2013 at 10:01:10PM -0500, Sasha Levin wrote:
> >>> On 12/23/2013 09:51 PM, Joonsoo Kim wrote:
> >>>> On Mon, Dec 23, 2013 at 12:24:02PM -0500, Sasha Levin wrote:
> >>>>>> Ping?
> >>>>>>
> >>>>>> I've also Cc'ed the "this page shouldn't be locked at all" team.
> >>>> Hello,
> >>>>
> >>>> I can't find the reason of this problem.
> >>>> If it is reproducible, how about bisecting?
> >>>
> >>> While it reproduces under fuzzing it's pretty hard to bisect it with
> >>> the amount of issues uncovered by trinity recently.
> >>>
> >>> I can add any debug code to the site of the BUG if that helps.
> >>
> >> Good!
> >> It will be helpful to add dump_page() in migration_entry_to_page().
> >>
> >> Thanks.
> >>
> >
> > Minchan teaches me that there is possible race condition between
> > fork and migration.
> >
> > Please consider following situation.
> >
> >
> > Process A (do migration)                      Process B (parents)
Process C (child)
> >
> > try_to_unmap() for migration <begin>          fork
> > setup migration entry to B's vma
> > ...
> > try_to_unmap() for migration <end>
> > move_to_new_page()
> >
> >                                               link new vma
> >                                                   into interval tree
> > remove_migration_ptes() <begin>
> > check and clear migration entry on C's vma
> > ...                                           copy_one_pte:
> > ...                                               now, B and C have
migration entry
> > ...
> > ...
> > check and clear migration entry on B's vma
> > ...
> > ...
> > remove_migration_ptes() <end>
> >
> >
> > Eventually, migration entry on C's vma is left.
> > And then, when C exits, above BUG_ON() can be triggered.
> >
>
> Yes, Looks like this is a potential race condition.
>
> > I'm not sure the I am right, so please think of it together. :)
> > And I'm not sure again that above assumption is related to this trigger
report,
> > since this may exist for a long time.
> >
> > So my question to mm folks is is above assumption possible and do we
have
> > any protection mechanism on this race?
> >
>
> I think we can down_read(&mm->mmap_sem) before remove_migration_ptes()
> to fix this issue, but I don't have time to verify it currently.
>
> --
> Regards,
> -Bob
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[-- Attachment #2: Type: text/html, Size: 4694 bytes --]

Re: mm: kernel BUG at include/linux/swapops.h:131!

[Konstantin Khlebnikov]

Bob Liu <bob.liu@oracle.com> wrote:
>On 12/24/2013 03:45 PM, Joonsoo Kim wrote:
>> On Tue, Dec 24, 2013 at 03:07:05PM +0900, Joonsoo Kim wrote:
>>> On Mon, Dec 23, 2013 at 10:01:10PM -0500, Sasha Levin wrote:
>>>> On 12/23/2013 09:51 PM, Joonsoo Kim wrote:
>>>>> On Mon, Dec 23, 2013 at 12:24:02PM -0500, Sasha Levin wrote:
>>>>>>> Ping?
>>>>>>>
>>>>>>> I've also Cc'ed the "this page shouldn't be locked at all" team.
>>>>> Hello,
>>>>>
>>>>> I can't find the reason of this problem.
>>>>> If it is reproducible, how about bisecting?
>>>>
>>>> While it reproduces under fuzzing it's pretty hard to bisect it
>with
>>>> the amount of issues uncovered by trinity recently.
>>>>
>>>> I can add any debug code to the site of the BUG if that helps.
>>>
>>> Good!
>>> It will be helpful to add dump_page() in migration_entry_to_page().
>>>
>>> Thanks.
>>>
>> 
>> Minchan teaches me that there is possible race condition between
>> fork and migration.
>> 
>> Please consider following situation.
>> 
>> 
>> Process A (do migration)			Process B (parents) Process C (child)
>> 
>> try_to_unmap() for migration <begin>		fork
>> setup migration entry to B's vma
>> ...
>> try_to_unmap() for migration <end>
>> move_to_new_page()
>> 
>> 						link new vma
>> 						    into interval tree
>> remove_migration_ptes() <begin>
>> check and clear migration entry on C's vma
>> ...						copy_one_pte:
>> ...						    now, B and C have migration entry
>> ...
>> ...
>> check and clear migration entry on B's vma
>> ...
>> ...
>> remove_migration_ptes() <end>
>> 
>> 
>> Eventually, migration entry on C's vma is left.
>> And then, when C exits, above BUG_ON() can be triggered.
>> 
>
>Yes, Looks like this is a potential race condition.
>
>> I'm not sure the I am right, so please think of it together. :)
>> And I'm not sure again that above assumption is related to this
>trigger report,
>> since this may exist for a long time.
>> 
>> So my question to mm folks is is above assumption possible and do we
>have
>> any protection mechanism on this race?
>> 
>
>I think we can down_read(&mm->mmap_sem) before remove_migration_ptes()
>to fix this issue, but I don't have time to verify it currently.

Hmm. This kind of race looks impossible: dup_mmap() always places child's
vma in into rmap tree after parent's one. For file-vma it's done explicitly
(vma_interval_tree_insert_after), for anon vma it's true because rb-tree
insert function goes to right branch if elements are equal.

Thus remove_migration_ptes() sees parent's pte first:
If child has the copy this function will check it after that.
And they are already synchronized with parent's and child's pte locks.i>>?

Sorry for double posting, gmail cannot into plain text =)

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: mm: kernel BUG at include/linux/swapops.h:131!

[Wanpeng Li]

On Thu, Dec 26, 2013 at 03:18:58PM +0900, Konstantin Khlebnikov wrote:
>Bob Liu <bob.liu@oracle.com> wrote:
>>On 12/24/2013 03:45 PM, Joonsoo Kim wrote:
>>> On Tue, Dec 24, 2013 at 03:07:05PM +0900, Joonsoo Kim wrote:
>>>> On Mon, Dec 23, 2013 at 10:01:10PM -0500, Sasha Levin wrote:
>>>>> On 12/23/2013 09:51 PM, Joonsoo Kim wrote:
>>>>>> On Mon, Dec 23, 2013 at 12:24:02PM -0500, Sasha Levin wrote:
>>>>>>>> Ping?
>>>>>>>>
>>>>>>>> I've also Cc'ed the "this page shouldn't be locked at all" team.
>>>>>> Hello,
>>>>>>
>>>>>> I can't find the reason of this problem.
>>>>>> If it is reproducible, how about bisecting?
>>>>>
>>>>> While it reproduces under fuzzing it's pretty hard to bisect it
>>with
>>>>> the amount of issues uncovered by trinity recently.
>>>>>
>>>>> I can add any debug code to the site of the BUG if that helps.
>>>>
>>>> Good!
>>>> It will be helpful to add dump_page() in migration_entry_to_page().
>>>>
>>>> Thanks.
>>>>
>>> 
>>> Minchan teaches me that there is possible race condition between
>>> fork and migration.
>>> 
>>> Please consider following situation.
>>> 
>>> 
>>> Process A (do migration)			Process B (parents) Process C (child)
>>> 
>>> try_to_unmap() for migration <begin>		fork
>>> setup migration entry to B's vma
>>> ...
>>> try_to_unmap() for migration <end>
>>> move_to_new_page()
>>> 
>>> 						link new vma
>>> 						    into interval tree
>>> remove_migration_ptes() <begin>
>>> check and clear migration entry on C's vma
>>> ...						copy_one_pte:
>>> ...						    now, B and C have migration entry
>>> ...
>>> ...
>>> check and clear migration entry on B's vma
>>> ...
>>> ...
>>> remove_migration_ptes() <end>
>>> 
>>> 
>>> Eventually, migration entry on C's vma is left.
>>> And then, when C exits, above BUG_ON() can be triggered.
>>> 
>>
>>Yes, Looks like this is a potential race condition.
>>
>>> I'm not sure the I am right, so please think of it together. :)
>>> And I'm not sure again that above assumption is related to this
>>trigger report,
>>> since this may exist for a long time.
>>> 
>>> So my question to mm folks is is above assumption possible and do we
>>have
>>> any protection mechanism on this race?
>>> 
>>
>>I think we can down_read(&mm->mmap_sem) before remove_migration_ptes()
>>to fix this issue, but I don't have time to verify it currently.
>
>Hmm. This kind of race looks impossible: dup_mmap() always places child's
>vma in into rmap tree after parent's one. For file-vma it's done explicitly
>(vma_interval_tree_insert_after), for anon vma it's true because rb-tree
>insert function goes to right branch if elements are equal.
>
>Thus remove_migration_ptes() sees parent's pte first:
>If child has the copy this function will check it after that.
>And they are already synchronized with parent's and child's pte locks.i>>?
>

Agreed. 

>Sorry for double posting, gmail cannot into plain text =)
>
>-- 
>Sent from my Android device with K-9 Mail. Please excuse my brevity.
>
>--
>To unsubscribe, send a message with 'unsubscribe linux-mm' in
>the body to majordomo@kvack.org.  For more info on Linux MM,
>see: http://www.linux-mm.org/ .
>Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: mm: kernel BUG at include/linux/swapops.h:131!

[Sasha Levin]

On 12/24/2013 01:07 AM, Joonsoo Kim wrote:
> On Mon, Dec 23, 2013 at 10:01:10PM -0500, Sasha Levin wrote:
>> On 12/23/2013 09:51 PM, Joonsoo Kim wrote:
>>> On Mon, Dec 23, 2013 at 12:24:02PM -0500, Sasha Levin wrote:
>>>>> Ping?
>>>>>
>>>>> I've also Cc'ed the "this page shouldn't be locked at all" team.
>>> Hello,
>>>
>>> I can't find the reason of this problem.
>>> If it is reproducible, how about bisecting?
>>
>> While it reproduces under fuzzing it's pretty hard to bisect it with
>> the amount of issues uncovered by trinity recently.
>>
>> I can add any debug code to the site of the BUG if that helps.
>
> Good!
> It will be helpful to add dump_page() in migration_entry_to_page().


[ 3800.520039] page:ffffea0000245800 count:12 mapcount:4 mapping:ffff88001d0c3668 index:0x7de
[ 3800.521404] page flags: 
0x1fffff8038003c(referenced|uptodate|dirty|lru|swapbacked|unevictable|mlocked)
[ 3800.522585] pc:ffff88001ed91600 pc->flags:2 pc->mem_cgroup:ffffc90000c0a000


Thanks,
Sasha

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: mm: kernel BUG at include/linux/swapops.h:131!

[Bob Liu]

On 12/24/2013 01:24 AM, Sasha Levin wrote:
> Ping?
> 
> I've also Cc'ed the "this page shouldn't be locked at all" team.
> 

I have no idea why this BUG_ON was triggered.
And it looks like 'mm: kernel BUG at mm/huge_memory.c:1440!' have the
same call trace with this one. Perhaps they were introduced by the same
reason.
Could you confirm whether those issues exist in v3.13-rc6?

Thanks,
-Bob

> On 12/18/2013 10:37 AM, Sasha Levin wrote:
>> Hi all,
>>
>> While fuzzing with trinity inside a KVM tools guest running latest
>> -next kernel, I've stumbled on
>> the following spew.
>>
>> The code is in zap_pte_range():
>>
>>                  if (!non_swap_entry(entry))
>>                          rss[MM_SWAPENTS]--;
>>                  else if (is_migration_entry(entry)) {
>>                          struct page *page;
>>
>>                          page = migration_entry_to_page(entry);   
>> <==== HERE
>>
>>                          if (PageAnon(page))
>>                                  rss[MM_ANONPAGES]--;
>>                          else
>>                                  rss[MM_FILEPAGES]--;
>>
>>
>> [ 2622.589064] kernel BUG at include/linux/swapops.h:131!
>> [ 2622.589064] invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
>> [ 2622.589064] Dumping ftrace buffer:
>> [ 2622.589064]    (ftrace buffer empty)
>> [ 2622.589064] Modules linked in:
>> [ 2622.589064] CPU: 9 PID: 15984 Comm: trinity-child16 Tainted:
>> G        W    3.13.0-rc
>> 4-next-20131217-sasha-00013-ga878504-dirty #4150
>> [ 2622.589064] task: ffff88168346b000 ti: ffff8816561d8000 task.ti:
>> ffff8816561d8000
>> [ 2622.589064] RIP: 0010:[<ffffffff8127c730>]  [<ffffffff8127c730>]
>> zap_pte_range+0x360
>> /0x4a0
>> [ 2622.589064] RSP: 0018:ffff8816561d9c18  EFLAGS: 00010246
>> [ 2622.589064] RAX: ffffea00736a6600 RBX: ffff88200299d068 RCX:
>> 0000000000000009
>> [ 2622.589064] RDX: 022fffff80380000 RSI: ffffea0000000000 RDI:
>> 3c00000001cda998
>> [ 2622.589064] RBP: ffff8816561d9cb8 R08: 0000000000000000 R09:
>> 0000000000000000
>> [ 2622.589064] R10: 0000000000000001 R11: 0000000000000000 R12:
>> 00007fc7ee20d000
>> [ 2622.589064] R13: ffff8816561d9de8 R14: 000000039b53303c R15:
>> 00007fc7ee29b000
>> [ 2622.589064] FS:  00007fc7eeceb700(0000) GS:ffff882011a00000(0000)
>> knlGS:000000000000
>> 0000
>> [ 2622.589064] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
>> [ 2622.589064] CR2: 000000000068c000 CR3: 0000000005c26000 CR4:
>> 00000000000006e0
>> [ 2622.589064] Stack:
>> [ 2622.589064]  ffff8816561d9c58 0000000000000286 ffff88168327b060
>> ffff88168327b060
>> [ 2622.589064]  00007fc700000160 ffff881667067b88 00000000c8f4b120
>> 00ff88168327b060
>> [ 2622.589064]  ffff88168327b060 ffff8820051a8600 0000000000000000
>> ffff88168327b000
>> [ 2622.589064] Call Trace:
>> [ 2622.589064]  [<ffffffff8127cc5e>] unmap_page_range+0x3ee/0x400
>> [ 2622.589064]  [<ffffffff8127cd71>] unmap_single_vma+0x101/0x120
>> [ 2622.589064]  [<ffffffff8127cdf1>] unmap_vmas+0x61/0xa0
>> [ 2622.589064]  [<ffffffff81283980>] exit_mmap+0xd0/0x170
>> [ 2622.589064]  [<ffffffff8112d430>] mmput+0x70/0xe0
>> [ 2622.589064]  [<ffffffff8113144d>] exit_mm+0x18d/0x1a0
>> [ 2622.589064]  [<ffffffff811defb5>] ? acct_collect+0x175/0x1b0
>> [ 2622.589064]  [<ffffffff8113389f>] do_exit+0x24f/0x500
>> [ 2622.589064]  [<ffffffff81133bf9>] do_group_exit+0xa9/0xe0
>> [ 2622.589064]  [<ffffffff81133c47>] SyS_exit_group+0x17/0x20
>> [ 2622.589064]  [<ffffffff843a6150>] tracesys+0xdd/0xe2
>> [ 2622.589064] Code: 83 f8 1f 75 46 48 b8 ff ff ff ff ff ff ff 01 48
>> be 00 00 00 00 00 ea ff ff 48
>> 21 f8 48 c1 e0 06 48 01 f0 48 8b 10 80 e2 01 75 0a <0f> 0b 66 0f 1f 44
>> 00 00 eb fe f6 40 08 01 74 05
>> ff 4d c4 eb 0b
>> [ 2622.589064] RIP  [<ffffffff8127c730>] zap_pte_range+0x360/0x4a0
>> [ 2622.589064]  RSP <ffff8816561d9c18>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: mm: kernel BUG at include/linux/swapops.h:131!

[Sasha Levin]

On 01/02/2014 01:36 AM, Bob Liu wrote:
> I have no idea why this BUG_ON was triggered.
> And it looks like 'mm: kernel BUG at mm/huge_memory.c:1440!' have the
> same call trace with this one. Perhaps they were introduced by the same
> reason.
> Could you confirm whether those issues exist in v3.13-rc6?

Yes, this is reproducible in 3.13-rc6.


Thanks,
Sasha

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>