linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
* Early test: hangs in mm/compact.c w. Linus's 12d7aacab56e9ef185c
@ 2014-11-04  7:26 P. Christeas
  2014-11-04  8:55 ` Vlastimil Babka
  0 siblings, 1 reply; 20+ messages in thread
From: P. Christeas @ 2014-11-04  7:26 UTC (permalink / raw)
  To: linux-mm; +Cc: Joonsoo Kim, Vlastimil Babka

TL;DR: I'm testing Linus's 3.18-rcX in my desktop (x86_64, full load), 
experiencing mm races about every day. Current -rc starves the canary of 
stablity

Will keep testing (should I try some -mm tree, please? ) , provide you 
feedback about the issue.

Not an active kernel-developer.

Long:

Since 26 Oct. upgraded my everything-on-it laptop to new distro (systemd -
based, all new glibc etc.) and switched from 3.17 to 3.18-pre . First time in 
years, kernel got unstable.

This machine is occasionaly under heavy load, doing I/O and serving random 
desktop applications. (machine is Intel x86_64, dual core, mechanical SATA 
disk).
Now, I have a race about once a day, have narrowed them down (guess) to:
 
        [<ffffffff813b1025>] preempt_schedule_irq+0x3c/0x59
        [<ffffffff813b4810>] retint_kernel+0x20/0x30
        [<ffffffff810d7481>] ? __zone_watermark_ok+0x77/0x85
        [<ffffffff810d8256>] zone_watermark_ok+0x1a/0x1c
        [<ffffffff810eee56>] compact_zone+0x215/0x4b2
        [<ffffffff810ef13f>] compact_zone_order+0x4c/0x5f
        [<ffffffff810ef2fe>] try_to_compact_pages+0xc4/0x1e8
        [<ffffffff813ad7f8>] __alloc_pages_direct_compact+0x61/0x1bf
        [<ffffffff810da299>] __alloc_pages_nodemask+0x409/0x799
        [<ffffffff8110d3fd>] new_slab+0x5f/0x21c
       ...

Sometimes is a less critical process, that I can safely kill, otherwise I have 
to drop everything and reboot.

Unless you are already aware of this case, please accept this feedback.
I'm pulling from Linus, should I also try some of your trees for an early 
solution?



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: Early test: hangs in mm/compact.c w. Linus's 12d7aacab56e9ef185c
  2014-11-04  7:26 Early test: hangs in mm/compact.c w. Linus's 12d7aacab56e9ef185c P. Christeas
@ 2014-11-04  8:55 ` Vlastimil Babka
  2014-11-04  9:36   ` P. Christeas
  0 siblings, 1 reply; 20+ messages in thread
From: Vlastimil Babka @ 2014-11-04  8:55 UTC (permalink / raw)
  To: P. Christeas, linux-mm; +Cc: Joonsoo Kim, lkml

On 11/04/2014 08:26 AM, P. Christeas wrote:
> TL;DR: I'm testing Linus's 3.18-rcX in my desktop (x86_64, full load),
> experiencing mm races about every day. Current -rc starves the canary of
> stablity
>
> Will keep testing (should I try some -mm tree, please? ) , provide you
> feedback about the issue.

Hello,

Please do keep testing (and see below what we need), and don't try 
another tree - it's 3.18 we need to fix!

> Not an active kernel-developer.
>
> Long:
>
> Since 26 Oct. upgraded my everything-on-it laptop to new distro (systemd -
> based, all new glibc etc.) and switched from 3.17 to 3.18-pre . First time in
> years, kernel got unstable.
>
> This machine is occasionaly under heavy load, doing I/O and serving random
> desktop applications. (machine is Intel x86_64, dual core, mechanical SATA
> disk).
> Now, I have a race about once a day, have narrowed them down (guess) to:
>
>          [<ffffffff813b1025>] preempt_schedule_irq+0x3c/0x59
>          [<ffffffff813b4810>] retint_kernel+0x20/0x30
>          [<ffffffff810d7481>] ? __zone_watermark_ok+0x77/0x85
>          [<ffffffff810d8256>] zone_watermark_ok+0x1a/0x1c
>          [<ffffffff810eee56>] compact_zone+0x215/0x4b2
>          [<ffffffff810ef13f>] compact_zone_order+0x4c/0x5f
>          [<ffffffff810ef2fe>] try_to_compact_pages+0xc4/0x1e8
>          [<ffffffff813ad7f8>] __alloc_pages_direct_compact+0x61/0x1bf
>          [<ffffffff810da299>] __alloc_pages_nodemask+0x409/0x799
>          [<ffffffff8110d3fd>] new_slab+0x5f/0x21c
>         ...

I'm not sure what you mean by "race" here and your snippet is 
unfortunately just a small portion of the output which could be a BUG, 
OOPS, lockdep, soft-lockup, hardlock and possibly many other things. But 
the backtrace itself is not enough, please send the whole error output 
(it should stard and end with something like:
-----[ cut here ]------
Thanks in advance.

> Sometimes is a less critical process, that I can safely kill, otherwise I have
> to drop everything and reboot.

OK so the process is not dead due to the problem? That probably rules 
out some kinds of errors but we still need the full output. Thanks in 
advance.

> Unless you are already aware of this case, please accept this feedback.
> I'm pulling from Linus, should I also try some of your trees for an early
> solution?

I'm not aware of this, CCing lkml for wider coverage.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: Early test: hangs in mm/compact.c w. Linus's 12d7aacab56e9ef185c
  2014-11-04  8:55 ` Vlastimil Babka
@ 2014-11-04  9:36   ` P. Christeas
  2014-11-05 15:26     ` Vlastimil Babka
  0 siblings, 1 reply; 20+ messages in thread
From: P. Christeas @ 2014-11-04  9:36 UTC (permalink / raw)
  To: Vlastimil Babka; +Cc: linux-mm, Joonsoo Kim, lkml

[-- Attachment #1: Type: text/plain, Size: 1736 bytes --]

On Tuesday 04 November 2014, Vlastimil Babka wrote:
> Please do keep testing (and see below what we need), and don't try
> another tree - it's 3.18 we need to fix!
Let me apologize/warn you about the poor quality of this report (and debug 
data).
It is on a system meant for everyday desktop usage, not kernel development. 
Thus, it is tuned to be "slightly" debuggable ; mostly for performance.

> I'm not sure what you mean by "race" here and your snippet is
> unfortunately just a small portion of the output ...

It is a shot in the dark. System becomes non-responsive (narrowed to desktop 
apps waiting each other, or the X+kwin blocking), I can feel the CPU heating 
and /sometimes/ disk I/O.

No BUG, Oops or any kernel message. (is printk level 4 adequate? )

Then, I try to drop to a console and collect as much data as possible with 
SysRq.

The snippet I'd sent you is from all-cpus-backtrace (l), trying to see which 
traces appear consistently during the lockup. There is also the huge traces of 
"task-states" (t), but I reckon they are too noisy.
That trace also matches the usage profile, because AFAICG[uess] the issue 
appears when allocating during I/O load. 

After turning on full-preemption, I have been able to terminate/kill all tasks 
and continue with same kernel but new userspace.

> OK so the process is not dead due to the problem? That probably rules
> out some kinds of errors but we still need the full output. Thanks in
> advance. 
> I'm not aware of this, CCing lkml for wider coverage.

Thank you. As I've told in the first mail, this is an early report of possible 
3.18 regression. I'm trying to narrow down the case and make it reproducible 
or get a good trace.

Attached is my current .config



[-- Attachment #2: config-3.18.gz --]
[-- Type: application/gzip, Size: 35515 bytes --]

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: Early test: hangs in mm/compact.c w. Linus's 12d7aacab56e9ef185c
  2014-11-04  9:36   ` P. Christeas
@ 2014-11-05 15:26     ` Vlastimil Babka
  2014-11-05 16:02       ` P. Christeas
  2014-11-06 19:23       ` P. Christeas
  0 siblings, 2 replies; 20+ messages in thread
From: Vlastimil Babka @ 2014-11-05 15:26 UTC (permalink / raw)
  To: P. Christeas; +Cc: linux-mm, Joonsoo Kim, lkml

On 11/04/2014 10:36 AM, P. Christeas wrote:
> On Tuesday 04 November 2014, Vlastimil Babka wrote:
>> Please do keep testing (and see below what we need), and don't try
>> another tree - it's 3.18 we need to fix!
> Let me apologize/warn you about the poor quality of this report (and debug 
> data).
> It is on a system meant for everyday desktop usage, not kernel development. 
> Thus, it is tuned to be "slightly" debuggable ; mostly for performance.
> 
>> I'm not sure what you mean by "race" here and your snippet is
>> unfortunately just a small portion of the output ...
> 
> It is a shot in the dark. System becomes non-responsive (narrowed to desktop 
> apps waiting each other, or the X+kwin blocking), I can feel the CPU heating 
> and /sometimes/ disk I/O.
> 
> No BUG, Oops or any kernel message. (is printk level 4 adequate? )
> 
> Then, I try to drop to a console and collect as much data as possible with 
> SysRq.
> 
> The snippet I'd sent you is from all-cpus-backtrace (l), trying to see which 
> traces appear consistently during the lockup. There is also the huge traces of 
> "task-states" (t), but I reckon they are too noisy.
> That trace also matches the usage profile, because AFAICG[uess] the issue 
> appears when allocating during I/O load. 
> 
> After turning on full-preemption, I have been able to terminate/kill all tasks 
> and continue with same kernel but new userspace.
> 
>> OK so the process is not dead due to the problem? That probably rules
>> out some kinds of errors but we still need the full output. Thanks in
>> advance. 
>> I'm not aware of this, CCing lkml for wider coverage.
> 
> Thank you. As I've told in the first mail, this is an early report of possible 
> 3.18 regression. I'm trying to narrow down the case and make it reproducible 
> or get a good trace.

I see. I've tried to reproduce such issues with 3.18-rc3 but wasn't successful.
But I noticed a possible issue that could lead to your problem.
Can you please try the following patch?

--------8<-------

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: Early test: hangs in mm/compact.c w. Linus's 12d7aacab56e9ef185c
  2014-11-05 15:26     ` Vlastimil Babka
@ 2014-11-05 16:02       ` P. Christeas
  2014-11-06 19:23       ` P. Christeas
  1 sibling, 0 replies; 20+ messages in thread
From: P. Christeas @ 2014-11-05 16:02 UTC (permalink / raw)
  To: Vlastimil Babka; +Cc: linux-mm, Joonsoo Kim, lkml

On Wednesday 05 November 2014, Vlastimil Babka wrote:
> I see. I've tried to reproduce such issues with 3.18-rc3 but wasn't
> successful. But I noticed a possible issue that could lead to your problem.
> Can you please try the following patch?

OK, I can give it a try.

FYI, the "stability canary" is still alive, my system is on for 28hours, under 
avg. load >=3 all this time, HEAD=980d0d51b1c9617a4

/me goes busy fire-proofing your patch...



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: Early test: hangs in mm/compact.c w. Linus's 12d7aacab56e9ef185c
  2014-11-05 15:26     ` Vlastimil Babka
  2014-11-05 16:02       ` P. Christeas
@ 2014-11-06 19:23       ` P. Christeas
  2014-11-06 21:38         ` Vlastimil Babka
  1 sibling, 1 reply; 20+ messages in thread
From: P. Christeas @ 2014-11-06 19:23 UTC (permalink / raw)
  To: Vlastimil Babka; +Cc: linux-mm, Joonsoo Kim, lkml

[-- Attachment #1: Type: text/plain, Size: 723 bytes --]

On Wednesday 05 November 2014, Vlastimil Babka wrote:
> Can you please try the following patch?
> --- a/mm/compaction.c
> +++ b/mm/compaction.c
> @@ -1325,13 +1325,6 @@ unsigned long try_to_compact_pages(struct zonelist
> -			compaction_defer_reset(zone, order, false);

NACK :(

I just got again into a state that some task was spinning out of control, and 
blocking the rest of the desktop.

You will see me trying a few things, apparently the first OOM managed to 
unblock something, but a few seconds later the system "stepped" on some other 
blocking task.

See attached log, it may only give you some hint; the problem could well be in 
some other part of the kernel.

In the meanwhile, I'm pulling linus/master ...


[-- Attachment #2: kcrash.log --]
[-- Type: text/x-log, Size: 39919 bytes --]

SysRq : Show backtrace of all active CPUs
sending NMI to all CPUs:
NMI backtrace for cpu 1
CPU: 1 PID: 13544 Comm: python Not tainted 3.18.0-rc3+ #46
Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
task: ffff88000c78ee40 ti: ffff88000e5f8000 task.ti: ffff88000e5f8000
RIP: 0010:[<ffffffff811df888>]  [<ffffffff811df888>] delay_tsc+0x28/0xa2
RSP: 0000:ffff8800bf303b28  EFLAGS: 00000002
RAX: 000000006bd322e8 RBX: 0000000000002710 RCX: 0000000000000007
RDX: 000000000000021d RSI: ffffffff8151623e RDI: ffffffff8152fea5
RBP: ffff8800bf303b48 R08: 0000000000000400 R09: 00000000ffffffff
R10: 0000000000000046 R11: 0000000000000046 R12: 0000000000185ac0
R13: 0000000000000001 R14: 0000000000000001 R15: ffffffff81668f90
FS:  00007f1570ed1700(0000) GS:ffff8800bf300000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f7cd966b000 CR3: 00000000740c9000 CR4: 00000000000007e0
Stack:
 0000000000002710 0000000000000003 000000000000006c 0000000000000001
 ffff8800bf303b58 ffffffff811df814 ffff8800bf303b68 ffffffff811df83d
 ffff8800bf303b88 ffffffff81025de1 0000000080010002 ffffffff816692b0
Call Trace:
 <IRQ> 

 [<ffffffff811df814>] __delay+0xa/0xc
 [<ffffffff811df83d>] __const_udelay+0x27/0x29
 [<ffffffff81025de1>] arch_trigger_all_cpu_backtrace+0xa8/0xd2
 [<ffffffff8126bc4f>] sysrq_handle_showallcpus+0xe/0x10
 [<ffffffff8126c186>] __handle_sysrq+0x94/0x126
 [<ffffffff8126c329>] sysrq_filter+0xee/0x287
 [<ffffffff812c0fd8>] input_to_handler+0x5e/0xcb
 [<ffffffff812c1de2>] input_pass_values.part.3+0x76/0x134
 [<ffffffff812c3eae>] input_handle_event+0x457/0x46d
 [<ffffffff812c3f19>] input_event+0x55/0x6f
 [<ffffffff812c6fb5>] input_sync+0xf/0x11
 [<ffffffff812c7f47>] atkbd_interrupt+0x4d5/0x595
 [<ffffffff812bf2c3>] serio_interrupt+0x43/0x7d
 [<ffffffff812bfa2e>] i8042_interrupt+0x292/0x2a8
 [<ffffffff8108b64b>] ? tick_sched_do_timer+0x33/0x33
 [<ffffffff810729a6>] handle_irq_event_percpu+0x44/0x19f
 [<ffffffff81072b3d>] handle_irq_event+0x3c/0x5c
 [<ffffffff81025e49>] ? apic_eoi+0x18/0x1a
 [<ffffffff810752b2>] handle_edge_irq+0x95/0xae
 [<ffffffff81004679>] handle_irq+0x158/0x16d
 [<ffffffff8105683f>] ? get_parent_ip+0xe/0x3e
 [<ffffffff81003f71>] do_IRQ+0x58/0xda
 [<ffffffff813ba1ea>] common_interrupt+0x6a/0x6a
 <EOI> 

 [<ffffffff810ee503>] ? rcu_read_unlock_sched_notrace+0x17/0x17
 [<ffffffff810ef46a>] ? compact_zone+0x2a8/0x4b2
 [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
 [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
 [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
 [<ffffffff810da3e1>] __alloc_pages_nodemask+0x409/0x799
 [<ffffffff810fdd84>] ? anon_vma_prepare+0xf5/0x12c
 [<ffffffff811163bb>] do_huge_pmd_anonymous_page+0x13c/0x255
 [<ffffffff810faa65>] ? mmap_region+0x171/0x458
 [<ffffffff810f65bd>] handle_mm_fault+0x112/0x808
 [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
 [<ffffffff810fb004>] ? do_mmap_pgoff+0x2b8/0x306
 [<ffffffff810e88dd>] ? vm_mmap_pgoff+0x82/0xaa
 [<ffffffff810f9997>] ? SyS_mmap_pgoff+0x183/0x1cf
 [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
 [<ffffffff813bb162>] page_fault+0x22/0x30
Code: ff 5d c3 55 48 89 e5 41 56 41 55 41 54 41 89 fc bf 01 00 00 00 53 e8 f7 6f e7 ff e8 9a 9c 00 00 41 89 c5 0f 1f 00 0f ae e8 0f 31 <89> c3 0f 1f 00 0f ae e8 0f 31 48 c1 e2 20 89 c0 48 09 c2 41 89 
NMI backtrace for cpu 0
CPU: 0 PID: 13788 Comm: net_applet Not tainted 3.18.0-rc3+ #46
Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
task: ffff8800067a3720 ti: ffff88000e20c000 task.ti: ffff88000e20c000
RIP: 0010:[<ffffffff810ef586>]  [<ffffffff810ef586>] compact_zone+0x3c4/0x4b2
RSP: 0000:ffff88000e20fa18  EFLAGS: 00000202
RAX: 00000000ffffffff RBX: ffffffff8168be40 RCX: 0000000000000008
RDX: 0000000000000380 RSI: 0000000000000009 RDI: ffffffff8168be40
RBP: ffff88000e20fa78 R08: 0000000000000000 R09: fffffffffffffef5
R10: 0000000000000038 R11: ffffffff8168be40 R12: 00000000000bf800
R13: 00000000000bf600 R14: ffff88000e20fa98 R15: 0000160000000000
FS:  00007ff9cbe92700(0000) GS:ffff8800bf200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007ff3a52f0000 CR3: 000000000af17000 CR4: 00000000000007f0
Stack:
 ffff88000e20fa18 ffffea0002fd0000 0000000000000020 ffff8800067a3720
 0000000000000004 ffff88000e20faa8 0000000000000000 0000000000000000
 0000000000000009 ffff88000e20fccc 0000000000000002 0000000000000000
Call Trace:
 [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
 [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
 [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
 [<ffffffff810da3e1>] __alloc_pages_nodemask+0x409/0x799
 [<ffffffff810d458c>] ? unlock_page+0x1f/0x23
 [<ffffffff81116903>] do_huge_pmd_wp_page+0x127/0x4eb
 [<ffffffff810f65fc>] handle_mm_fault+0x151/0x808
 [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
 [<ffffffff8102ddf8>] ? do_page_fault+0xc/0xe
 [<ffffffff813bb162>] ? page_fault+0x22/0x30
 [<ffffffff811e0740>] ? __put_user_4+0x20/0x30
 [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
 [<ffffffff813bb162>] page_fault+0x22/0x30
Code: 8b 7b 08 44 89 e6 ff 13 48 83 c3 10 48 83 3b 00 eb eb 41 83 7e 40 01 4d 8b 6e 38 19 c0 89 45 c0 4d 8d a5 00 02 00 00 83 65 c0 04 <49> 81 e4 00 fe ff ff e9 b2 fe ff ff 41 80 7e 44 00 74 09 41 83 

SysRq : Changing Loglevel
Loglevel set to 8

SysRq : Show backtrace of all active CPUs
sending NMI to all CPUs:
NMI backtrace for cpu 1
CPU: 1 PID: 13544 Comm: python Not tainted 3.18.0-rc3+ #46
Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
task: ffff88000c78ee40 ti: ffff88000e5f8000 task.ti: ffff88000e5f8000
RIP: 0010:[<ffffffff811df817>]  [<ffffffff811df817>] __const_udelay+0x1/0x29
RSP: 0000:ffff8800bf303b68  EFLAGS: 00000006
RAX: 0000000000000000 RBX: 0000000000002710 RCX: 0000000000000007
RDX: 0000000080010003 RSI: 0000000000000c00 RDI: 0000000000418958
RBP: ffff8800bf303b88 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000046 R11: 0000000000000046 R12: 0000000000000008
R13: 000000000000006c R14: 0000000000000001 R15: ffffffff81668f90
FS:  00007f1570ed1700(0000) GS:ffff8800bf300000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f7cd966b000 CR3: 00000000740c9000 CR4: 00000000000007e0
Stack:
 ffff8800bf303b88 ffffffff81025de1 0000000080010002 ffffffff816692b0
 ffff8800bf303b98 ffffffff8126bc4f ffff8800bf303bc8 ffffffff8126c186
 ffff88003781c200 0000000000000001 0000000000000026 ffff88003781c201
Call Trace:
 <IRQ> 

 [<ffffffff81025de1>] ? arch_trigger_all_cpu_backtrace+0xa8/0xd2
 [<ffffffff8126bc4f>] sysrq_handle_showallcpus+0xe/0x10
 [<ffffffff8126c186>] __handle_sysrq+0x94/0x126
 [<ffffffff8126c329>] sysrq_filter+0xee/0x287
 [<ffffffff812c0fd8>] input_to_handler+0x5e/0xcb
 [<ffffffff812c1de2>] input_pass_values.part.3+0x76/0x134
 [<ffffffff812c3eae>] input_handle_event+0x457/0x46d
 [<ffffffff812c3f19>] input_event+0x55/0x6f
 [<ffffffff812c6fb5>] input_sync+0xf/0x11
 [<ffffffff812c7f47>] atkbd_interrupt+0x4d5/0x595
 [<ffffffff812bf2c3>] serio_interrupt+0x43/0x7d
 [<ffffffff812bfa2e>] i8042_interrupt+0x292/0x2a8
 [<ffffffff8108b64b>] ? tick_sched_do_timer+0x33/0x33
 [<ffffffff810729a6>] handle_irq_event_percpu+0x44/0x19f
 [<ffffffff81072b3d>] handle_irq_event+0x3c/0x5c
 [<ffffffff81025e49>] ? apic_eoi+0x18/0x1a
 [<ffffffff810752b2>] handle_edge_irq+0x95/0xae
 [<ffffffff81004679>] handle_irq+0x158/0x16d
 [<ffffffff8105683f>] ? get_parent_ip+0xe/0x3e
 [<ffffffff81003f71>] do_IRQ+0x58/0xda
 [<ffffffff813ba1ea>] common_interrupt+0x6a/0x6a
 <EOI> 

 [<ffffffff813b6b95>] ? preempt_schedule_irq+0x3c/0x59
 [<ffffffff810d75b5>] ? __zone_watermark_ok+0x63/0x85
 [<ffffffff810d839e>] zone_watermark_ok+0x1a/0x1c
 [<ffffffff810ef3d7>] compact_zone+0x215/0x4b2
 [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
 [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
 [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
 [<ffffffff810da3e1>] __alloc_pages_nodemask+0x409/0x799
 [<ffffffff810fdd84>] ? anon_vma_prepare+0xf5/0x12c
 [<ffffffff811163bb>] do_huge_pmd_anonymous_page+0x13c/0x255
 [<ffffffff810faa65>] ? mmap_region+0x171/0x458
 [<ffffffff810f65bd>] handle_mm_fault+0x112/0x808
 [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
 [<ffffffff810fb004>] ? do_mmap_pgoff+0x2b8/0x306
 [<ffffffff810e88dd>] ? vm_mmap_pgoff+0x82/0xaa
 [<ffffffff810f9997>] ? SyS_mmap_pgoff+0x183/0x1cf
 [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
 [<ffffffff813bb162>] page_fault+0x22/0x30
Code: eb 02 66 90 eb 0e 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 48 ff c8 75 fb 48 ff c8 5d c3 55 48 89 e5 ff 15 a4 0c 48 00 5d c3 55 <48> 8d 04 bd 00 00 00 00 48 89 e5 65 48 8b 14 25 20 26 01 00 48 
NMI backtrace for cpu 0
CPU: 0 PID: 11733 Comm: kwin Not tainted 3.18.0-rc3+ #46
Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
task: ffff8800055e7620 ti: ffff88009dc78000 task.ti: ffff88009dc78000
RIP: 0010:[<ffffffff810ee567>]  [<ffffffff810ee567>] acct_isolated+0x64/0x6b
RSP: 0000:ffff88009dc7ba08  EFLAGS: 00000246
RAX: ffff88009dc7bac8 RBX: ffffffff8168be40 RCX: ffff88009dc7bac8
RDX: 0000000000000380 RSI: ffff88009dc7bab8 RDI: ffffffff8168be40
RBP: ffff88009dc7ba28 R08: 0000000000000000 R09: ffffffffffffff01
R10: 0000000000000038 R11: ffffffff8168be40 R12: 00000000000bf800
R13: 00000000000bf600 R14: ffff88009dc7bab8 R15: 0000160000000000
FS:  00007facfaf397c0(0000) GS:ffff8800bf200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f9d5781d910 CR3: 0000000025c25000 CR4: 00000000000007f0
Stack:
 0000000000000000 0000000000000000 ffff88009dc7ba28 ffffffff8168be40
 ffff88009dc7ba98 ffffffff810ef46a ffff88008035e660 ffffea0002fd0000
 0000000000000020 ffff8800055e7620 0000000000000004 ffff88009dc7bac8
Call Trace:
 [<ffffffff810ef46a>] compact_zone+0x2a8/0x4b2
 [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
 [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
 [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
 [<ffffffff810da3e1>] __alloc_pages_nodemask+0x409/0x799
 [<ffffffff810fdcba>] ? anon_vma_prepare+0x2b/0x12c
 [<ffffffff811163bb>] do_huge_pmd_anonymous_page+0x13c/0x255
 [<ffffffff810f65bd>] handle_mm_fault+0x112/0x808
 [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
 [<ffffffff810fb004>] ? do_mmap_pgoff+0x2b8/0x306
 [<ffffffff810e88dd>] ? vm_mmap_pgoff+0x82/0xaa
 [<ffffffff813b9028>] ? _raw_spin_unlock_irq+0x14/0x27
 [<ffffffff810f9997>] ? SyS_mmap_pgoff+0x183/0x1cf
 [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
 [<ffffffff813bb162>] page_fault+0x22/0x30
Code: f7 d2 83 e2 01 ff 44 95 e8 eb dd 8b 55 e8 be 16 00 00 00 48 89 df e8 7d b3 ff ff 8b 55 ec be 17 00 00 00 48 89 df e8 6d b3 ff ff <48> 83 c4 18 5b 5d c3 83 7a 40 00 55 48 89 e5 41 54 49 89 d4 53 
INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 91.545 msecs

SysRq : Show backtrace of all active CPUs
sending NMI to all CPUs:

NMI backtrace for cpu 0
CPU: 0 PID: 11733 Comm: kwin Not tainted 3.18.0-rc3+ #46
Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
task: ffff8800055e7620 ti: ffff88009dc78000 task.ti: ffff88009dc78000
RIP: 0010:[<ffffffff810ef586>]  [<ffffffff810ef586>] compact_zone+0x3c4/0x4b2
RSP: 0000:ffff88009dc7ba38  EFLAGS: 00000202
RAX: 00000000ffffffff RBX: ffffffff8168be40 RCX: 0000000000000008
RDX: 0000000000000380 RSI: 0000000000000009 RDI: ffffffff8168be40
RBP: ffff88009dc7ba98 R08: 0000000000000000 R09: ffffffffffffff01
R10: 0000000000000038 R11: ffffffff8168be40 R12: 00000000000bf800
R13: 00000000000bf600 R14: ffff88009dc7bab8 R15: 0000160000000000
FS:  00007facfaf397c0(0000) GS:ffff8800bf200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007fe8952e5000 CR3: 0000000025c25000 CR4: 00000000000007f0
Stack:
 ffff88008035e660 ffffea0002fd0000 0000000000000020 ffff8800055e7620
 0000000000000004 ffff88009dc7bac8 0000000000000234 0000000000000000
 0000000000000009 ffff88009dc7bcec 0000000000000002 0000000000000000
Call Trace:
 [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
 [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
 [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
 [<ffffffff810da3e1>] __alloc_pages_nodemask+0x409/0x799
 [<ffffffff810fdcba>] ? anon_vma_prepare+0x2b/0x12c
 [<ffffffff811163bb>] do_huge_pmd_anonymous_page+0x13c/0x255
 [<ffffffff810f65bd>] handle_mm_fault+0x112/0x808
 [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
 [<ffffffff810fb004>] ? do_mmap_pgoff+0x2b8/0x306
 [<ffffffff810e88dd>] ? vm_mmap_pgoff+0x82/0xaa
 [<ffffffff813b9028>] ? _raw_spin_unlock_irq+0x14/0x27
 [<ffffffff810f9997>] ? SyS_mmap_pgoff+0x183/0x1cf
 [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
 [<ffffffff813bb162>] page_fault+0x22/0x30
Code: 8b 7b 08 44 89 e6 ff 13 48 83 c3 10 48 83 3b 00 eb eb 41 83 7e 40 01 4d 8b 6e 38 19 c0 89 45 c0 4d 8d a5 00 02 00 00 83 65 c0 04 <49> 81 e4 00 fe ff ff e9 b2 fe ff ff 41 80 7e 44 00 74 09 41 83 
NMI backtrace for cpu 1
CPU: 1 PID: 13544 Comm: python Not tainted 3.18.0-rc3+ #46
Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
task: ffff88000c78ee40 ti: ffff88000e5f8000 task.ti: ffff88000e5f8000
RIP: 0010:[<ffffffff811df82b>]  [<ffffffff811df82b>] __const_udelay+0x15/0x29
RSP: 0000:ffff8800bf303b68  EFLAGS: 00000006
RAX: 0000000001062560 RBX: 0000000000002710 RCX: 0000000000000007
RDX: 0000000000185ab3 RSI: 0000000000000c00 RDI: 0000000000418958
RBP: ffff8800bf303b68 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000046 R11: 0000000000000046 R12: 0000000000000008
R13: 000000000000006c R14: 0000000000000001 R15: ffffffff81668f90
FS:  00007f1570ed1700(0000) GS:ffff8800bf300000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f38f594a000 CR3: 00000000740c9000 CR4: 00000000000007e0
Stack:
 ffff8800bf303b88 ffffffff81025de1 0000000080010002 ffffffff816692b0
 ffff8800bf303b98 ffffffff8126bc4f ffff8800bf303bc8 ffffffff8126c186
 ffff88003781c200 0000000000000001 0000000000000026 ffff88003781c201
Call Trace:
 <IRQ> 

 [<ffffffff81025de1>] arch_trigger_all_cpu_backtrace+0xa8/0xd2
 [<ffffffff8126bc4f>] sysrq_handle_showallcpus+0xe/0x10
 [<ffffffff8126c186>] __handle_sysrq+0x94/0x126
 [<ffffffff8126c329>] sysrq_filter+0xee/0x287
 [<ffffffff812c0fd8>] input_to_handler+0x5e/0xcb
 [<ffffffff812c1de2>] input_pass_values.part.3+0x76/0x134
 [<ffffffff812c3eae>] input_handle_event+0x457/0x46d
 [<ffffffff812c3f19>] input_event+0x55/0x6f
 [<ffffffff812c6fb5>] input_sync+0xf/0x11
 [<ffffffff812c7f47>] atkbd_interrupt+0x4d5/0x595
 [<ffffffff812bf2c3>] serio_interrupt+0x43/0x7d
 [<ffffffff812bfa2e>] i8042_interrupt+0x292/0x2a8
 [<ffffffff8108b64b>] ? tick_sched_do_timer+0x33/0x33
 [<ffffffff810729a6>] handle_irq_event_percpu+0x44/0x19f
 [<ffffffff81072b3d>] handle_irq_event+0x3c/0x5c
 [<ffffffff81025e49>] ? apic_eoi+0x18/0x1a
 [<ffffffff810752b2>] handle_edge_irq+0x95/0xae
 [<ffffffff81004679>] handle_irq+0x158/0x16d
 [<ffffffff8105683f>] ? get_parent_ip+0xe/0x3e
 [<ffffffff81003f71>] do_IRQ+0x58/0xda
 [<ffffffff813ba1ea>] common_interrupt+0x6a/0x6a
 <EOI> 

 [<ffffffff810d75ad>] ? __zone_watermark_ok+0x5b/0x85
 [<ffffffff810d839e>] zone_watermark_ok+0x1a/0x1c
 [<ffffffff810ef3d7>] compact_zone+0x215/0x4b2
 [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
 [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
 [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
 [<ffffffff810da3e1>] __alloc_pages_nodemask+0x409/0x799
 [<ffffffff810fdd84>] ? anon_vma_prepare+0xf5/0x12c
 [<ffffffff811163bb>] do_huge_pmd_anonymous_page+0x13c/0x255
 [<ffffffff810faa65>] ? mmap_region+0x171/0x458
 [<ffffffff810f65bd>] handle_mm_fault+0x112/0x808
 [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
 [<ffffffff810fb004>] ? do_mmap_pgoff+0x2b8/0x306
 [<ffffffff810e88dd>] ? vm_mmap_pgoff+0x82/0xaa
 [<ffffffff810f9997>] ? SyS_mmap_pgoff+0x183/0x1cf
 [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
 [<ffffffff813bb162>] page_fault+0x22/0x30
Code: 48 ff c8 75 fb 48 ff c8 5d c3 55 48 89 e5 ff 15 a4 0c 48 00 5d c3 55 48 8d 04 bd 00 00 00 00 48 89 e5 65 48 8b 14 25 20 26 01 00 <48> 69 d2 fa 00 00 00 f7 e2 48 8d 7a 01 e8 cd ff ff ff 5d c3 48 
INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 163.331 msecs

SysRq : Manual OOM execution
Purging GPU memory, 4186 bytes freed, 4243456 bytes still pinned.

SysRq : HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k) show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q) unraw(r) sync(s) show-task-states(t) unmount(u) force-fb(V) show-blocked-tasks(w) dump-ftrace-buffer(z) 
SysRq : Manual OOM execution

Purging GPU memory, 0 bytes freed, 4243456 bytes still pinned.

kworker/1:0 invoked oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=0
kworker/1:0 cpuset=/ mems_allowed=0
CPU: 1 PID: 13984 Comm: kworker/1:0 Not tainted 3.18.0-rc3+ #46
Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
Workqueue: events moom_callback
 ffff8800067a1600 ffff88000dadfc58 ffffffff813b4868 0000000000000001
 ffff8800067a0fc0 ffff88000dadfce8 ffffffff813b2e37 000000000023222e
 ffffffff8163e380 000000000023222e 0000000000000206 ffff88000dadfca8
Call Trace:
 [<ffffffff813b4868>] dump_stack+0x4f/0x7c
 [<ffffffff813b2e37>] dump_header.isra.11+0x71/0x1d7
 [<ffffffff813b8ffe>] ? _raw_spin_unlock_irqrestore+0x1b/0x31
 [<ffffffff811db724>] ? ___ratelimit+0xb9/0xc7
 [<ffffffff810d6ca8>] oom_kill_process+0x60/0x310
 [<ffffffff810d6b5a>] ? oom_badness+0xb1/0xfb
 [<ffffffff810d733d>] out_of_memory+0x282/0x29b
 [<ffffffff8126bd74>] moom_callback+0x1f/0x21
 [<ffffffff8104b077>] process_one_work+0x156/0x29c
 [<ffffffff8104b978>] worker_thread+0x1eb/0x2c2
 [<ffffffff8104b78d>] ? cancel_delayed_work_sync+0x10/0x10
 [<ffffffff8104f502>] kthread+0xbb/0xc3
 [<ffffffff8104f447>] ? __kthread_parkme+0x5c/0x5c
 [<ffffffff813b96ac>] ret_from_fork+0x7c/0xb0
 [<ffffffff8104f447>] ? __kthread_parkme+0x5c/0x5c
Mem-Info:
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  97
CPU    1: hi:  186, btch:  31 usd: 183
active_anon:286387 inactive_anon:87475 isolated_anon:0
 active_file:174069 inactive_file:43465 isolated_file:0
 unevictable:0 dirty:30 writeback:0 unstable:0
 free:80502 slab_reclaimable:58629 slab_unreclaimable:10926
 mapped:98606 shmem:48888 pagetables:17014 bounce:0
 free_cma:0
DMA free:12216kB min:232kB low:288kB high:348kB active_anon:540kB inactive_anon:804kB active_file:1056kB inactive_file:760kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15992kB managed:15908kB mlocked:0kB dirty:0kB writeback:0kB mapped:892kB shmem:372kB slab_reclaimable:148kB slab_unreclaimable:88kB kernel_stack:16kB pagetables:92kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 2984 2984 2984
DMA32 free:309792kB min:44820kB low:56024kB high:67228kB active_anon:1145008kB inactive_anon:349096kB active_file:695220kB inactive_file:173100kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3119936kB managed:3057340kB mlocked:0kB dirty:120kB writeback:0kB mapped:393532kB shmem:195180kB slab_reclaimable:234368kB slab_unreclaimable:43616kB kernel_stack:7360kB pagetables:67964kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 10*4kB (UEM) 6*8kB (UM) 2*16kB (EM) 8*32kB (UEM) 3*64kB (UEM) 1*128kB (E) 1*256kB (M) 2*512kB (EM) 2*1024kB (UE) 2*2048kB (ER) 1*4096kB (M) = 12216kB
DMA32: 3084*4kB (UEMR) 8564*8kB (UEMR) 6169*16kB (UEMR) 2174*32kB (UEMR) 402*64kB (UEM) 139*128kB (UEMR) 49*256kB (UEMR) 7*512kB (UEM) 1*1024kB (M) 0*2048kB 0*4096kB = 309792kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
266800 total pagecache pages
376 pages in swap cache
Swap cache stats: add 12077, delete 11701, find 857/1253
Free swap  = 6092348kB
Total swap = 6136792kB
783982 pages RAM
0 pages HighMem/MovableOnly
15649 pages reserved
0 pages hwpoisoned
[ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
[  492]     0   492    10595     2553      23       19             0 systemd-journal
[  544]     0   544     8460      747      20       22         -1000 systemd-udevd
[  738]     0   738     4743      696      14       19             0 alsactl
[  761]     0   761     5848      855      16       28             0 bluetoothd
[  762]     0   762     8185      723      21       69             0 abrtd
[  766]     0   766     4827      581      15       30             0 irqbalance
[  788]     0   788     3814      709      12       97             0 smartd
[  789]     0   789     1827      365       9      811             0 hddtemp
[  795]     0   795     5423      624      16       11             0 systemd-logind
[  796]     0   796    48410     1889      31       74             0 udisks-daemon
[  797]   499   797     3638      754      12       10          -900 dbus-daemon
[  799]     0   799     1071      382       8       13             0 acpid
[  811]     0   811    26132     4425      53      472             0 cupsd
[  813]     0   813    56404     1572      41      647             0 upowerd
[  815]     0   815     6159      545      17       15             0 kdm
[  831]     0   831   173245      498      34       30             0 nscd
[  904]     0   904    11431        4      26       78             0 udisks-daemon
[ 1005]   497  1005    91379     3112      41     1379             0 polkitd
[ 1113]     0  1113     1586      287       8       22             0 ifplugd
[ 1212]     0  1212     8029      408      18      126             0 wpa_supplicant
[ 1214]     0  1214     1586      288       9       22             0 ifplugd
[ 1324]     0  1324    28065     1115      26      387             0 cf-execd
[ 1333]     0  1333    12113      541      24     1150             0 cf-serverd
[ 1479]     0  1479     3117       98      10     1610             0 dhclient
[ 1579]     0  1579    87422     2054     107      925             0 libvirtd
[ 1584]     0  1584    10579      942      25      135         -1000 sshd
[ 1592]     0  1592     2846      401      11       42             0 xinetd
[ 1598]   491  1598     6250      412      14       70             0 rpcbind
[ 1602]     0  1602     3851        0      12       38             0 rpc.idmapd
[ 1611]   488  1611     8360      544      21      141             0 rpc.statd
[ 1614]    80  1614     7989     1084      20      167             0 pgpool
[ 1615]    80  1615     8029      475      19      181             0 pgpool
[ 1616]    80  1616     8029      475      19      181             0 pgpool
[ 1617]    80  1617     8029      475      20      181             0 pgpool
[ 1618]    80  1618     8029      475      19      181             0 pgpool
[ 1619]    80  1619     7989      327      18      172             0 pgpool
[ 1620]    80  1620     7989       11      18      163             0 pgpool
[ 1622]    80  1622    47318     4356      34      123         -1000 postgres
[ 1636]     0  1636     2941      405      12      120             0 crond
[ 1643]   487  1643     5374      578      15       33             0 ntpd
[ 1647]     0  1647     5408     2340      16       23             0 preload
[ 1680]    80  1680    47350     6430      66      106         -1000 postgres
[ 1681]    80  1681    47318     9645      90      109         -1000 postgres
[ 1682]    80  1682    47318      785      25      146         -1000 postgres
[ 1683]    80  1683    47447     1683      39      148         -1000 postgres
[ 1684]    80  1684    11074      609      23      131         -1000 postgres
[ 1685]    80  1685    11146      954      23      118         -1000 postgres
[ 1803]   493  1803    70606     1317      41      377             0 colord
[ 1812]     0  1812     9046      792      24      234             0 systemd
[ 1815]     0  1815    14697      157      31      453             0 (sd-pam)
[ 2445]     0  2445     4228       12      12       38             0 gpg-agent
[ 5893]     0  5893    12071     1997      29      236             0 cf-monitord
[ 8691]   500  8691     9233     1169      22       19             0 systemd
[ 8692]   500  8692    14769      177      31      487             0 (sd-pam)
[ 8759]   500  8759     3170        7       9       63             0 ssh-agent
[ 8784]   500  8784     3459      393      14       34             0 gpg-agent
[ 9260]     0  9260     1975      400       9        0             0 agetty
[ 9262]     0  9262     1975      393       9        0             0 agetty
[ 9367]   500  9367     3098      684      11        0             0 gam_server
[ 9976]     0  9976    88063     2116      38        0             0 udisksd
[11283]     0 11283    47144    13442      89        0             0 X
[11297]     0 11297    19771     1253      44        6             0 kdm
[11507]   500 11507     3191      735      12        0             0 startkde
[11602]   500 11602     3987      460      13        0             0 dbus-launch
[11603]   500 11603     3907     1011      13        0             0 dbus-daemon
[11634]   500 11634    34515     1861      67        0             0 s2u
[11690]   500 11690     1028       20       6        0             0 start_kdeinit
[11691]   500 11691    95918     8995     150        0             0 kdeinit4
[11692]   500 11692    97137     6446     140        0             0 klauncher
[11694]   500 11694   260253    17122     254        0             0 kded4
[11697]   500 11697   118836     9730     181        0             0 kglobalaccel
[11700]   500 11700     5856      676      17        0             0 obexd
[11707]   500 11707    72764     6751     125        0             0 bluedevil-monol
[11718]   500 11718   228901     9415     160        0             0 kactivitymanage
[11721]   500 11721     1062      162       8        0             0 kwrapper4
[11724]   500 11724   139507     9588     188        0             0 ksmserver
[11733]   500 11733   683159    22362     242        0             0 kwin
[11737]   500 11737   821255    39358     343        0             0 plasma-desktop
[11740]   500 11740     2491      606      10        0             0 ksysguardd
[11742]   500 11742   185670    14918     241        0             0 krunner
[11836]   500 11836    66336     1673      33        0             0 mission-control
[11900]   500 11900    71627     7762     130        0             0 kuiserver
[11902]   500 11902    39315     2928      38        0             0 akonadi_control
[11904]   500 11904     3190      688      12        0             0 akonadiserver
[11905]   500 11905   364072     7154     110        0             0 akonadiserver
[11907]    80 11907    48978    37329      98       90         -1000 postgres
[11916]   500 11916    71126     7723     127        0             0 ksyndaemon
[11924]   500 11924    94819     2524      88        0             0 pulseaudio
[11925]   492 11925    40153      566      15        0             0 rtkit-daemon
[11938]    80 11938    47549     2575      34       95         -1000 postgres
[11939]    80 11939    47549     2619      34       95         -1000 postgres
[11940]    80 11940    47549     2619      34       95         -1000 postgres
[11943]   500 11943    80114     9317     112        0             0 akonadi_agent_l
[11944]   500 11944    80111     9358     108        0             0 akonadi_agent_l
[11945]   500 11945   149309    16150     260        0             0 akonadi_archive
[11948]   500 11948    80738    10447     147        0             0 akonadi_birthda
[11949]   500 11949    80046     9859     108        0             0 akonadi_agent_l
[11950]   500 11950   149456    15918     259        0             0 akonadi_foldera
[11952]   500 11952   119534    12515     160        0             0 akonadi_imap_re
[11953]   500 11953   138083    12936     163        0             0 akonadi_imap_re
[11954]   500 11954   137951    12152     158        0             0 akonadi_imap_re
[11955]   500 11955    74613     8962     134        0             0 akonadi_localbo
[11956]   500 11956    20123      790      42        0             0 gconf-helper
[11958]   500 11958    11393     1256      27        0             0 gconfd-2
[11959]   500 11959    80110     9345     111        0             0 akonadi_agent_l
[11960]   500 11960    80110     9316     112        0             0 akonadi_agent_l
[11961]   500 11961    88890    10322     156        0             0 akonadi_maildis
[11962]   500 11962   150003    16953     260        0             0 akonadi_mailfil
[11963]   500 11963    75882     9862     142        0             0 akonadi_migrati
[11964]   500 11964    80530    10776     142        0             0 akonadi_mixedma
[11965]   500 11965    82835    12607     151        0             0 akonadi_mixedma
[11966]   500 11966   144439    13711     197        0             0 konsole
[11973]   500 11973    76667    10080     139        0             0 akonadi_nepomuk
[11974]   500 11974   100405    11536     184        0             0 akonadi_newmail
[11975]   500 11975    79291     9474     108        0             0 akonadi_agent_l
[11976]   500 11976    76551     9175     140        0             0 akonadi_pop3_re
[11977]   500 11977    76535     9251     139        0             0 akonadi_pop3_re
[11978]   500 11978    77748    10430     141        0             0 akonadi_pop3_re
[11979]   500 11979   138554    13963     242        0             0 akonadi_sendlat
[11981]   500 11981    78493     9240     109        0             0 akonadi_agent_l
[11982]   500 11982    80012     9799     113        0             0 akonadi_agent_l
[11983]   500 11983   177548    11542     195        0             0 kmix
[11984]   500 11984   180599    14445     204        0             0 dolphin
[11993]   500 11993    74732     8349     133        0             0 akonaditray
[11997]   500 11997   107450    11140     188        0             0 knotes
[11999]   500 11999     3649     1186      12        0             0 bash
[12011]   500 12011     3585     1095      12        0             0 bash
[12025]   500 12025     3585     1076      12        0             0 bash
[12047]   500 12047     3585     1116      11        0             0 bash
[12186]   500 12186   116692    12696     175        0             0 kgpg
[12293]   500 12293     3585     1117      12        0             0 bash
[12317]   500 12317     3585     1113      13        0             0 bash
[12356]   500 12356     3585     1102      12        0             0 bash
[12405]   500 12405     3585     1086      13        0             0 bash
[12506]   500 12506     3585     1073      11        0             0 bash
[12645]   500 12645   122416    29665     208        0             0 choqok
[12673]   500 12673     3585     1115      13        0             0 bash
[12718]   500 12718     3585     1104      13        0             0 bash
[12771]   500 12771     3585     1075      12        0             0 bash
[12843]   500 12843     3585     1087      12        0             0 bash
[13020]   500 13020   686067    27224     266        0             0 kate
[13027]   500 13027   121357    10647     186        0             0 kwalletd
[13029]    80 13029    47607     3959      41       92         -1000 postgres
[13033]    80 13033    47549     2573      34       95         -1000 postgres
[13052]   500 13052   138593     9489     158        0             0 knotify4
[13053]    80 13053    47602     3917      40       92         -1000 postgres
[13054]    80 13054    47602     3920      40       93         -1000 postgres
[13055]    80 13055    47835     7071      72       92         -1000 postgres
[13058]    80 13058    47831     4688      51       92         -1000 postgres
[13059]    80 13059    47602     3919      40       92         -1000 postgres
[13061]   500 13061   121241    17287     186        0             0 konversation
[13062]    80 13062    47583     3456      39       95         -1000 postgres
[13076]    80 13076    47745     6832      72       92         -1000 postgres
[13080]    80 13080    47629     4038      42       93         -1000 postgres
[13081]   500 13081   701137    24187     260        0             0 kate
[13082]    80 13082    47866     8228      73       92         -1000 postgres
[13083]    80 13083    47623     3981      40       92         -1000 postgres
[13085]    80 13085    47830     8033      67       92         -1000 postgres
[13087]    80 13087    47549     2587      34       95         -1000 postgres
[13088]    80 13088    47828     5052      52       92         -1000 postgres
[13089]    80 13089    47549     2601      34       95         -1000 postgres
[13090]    80 13090    47684     9326      79       92         -1000 postgres
[13091]    80 13091    47581     3365      39       95         -1000 postgres
[13092]    80 13092    47651     4875      59       92         -1000 postgres
[13093]    80 13093    47769     5336      62       92         -1000 postgres
[13094]    80 13094    47670     4777      55       93         -1000 postgres
[13095]    80 13095    47582     3205      39       95         -1000 postgres
[13096]    80 13096    47596     3887      44       93         -1000 postgres
[13097]    80 13097    47835     4919      50       92         -1000 postgres
[13098]    80 13098    47900    22860      93       89         -1000 postgres
[13100]    80 13100    47581     3364      39       95         -1000 postgres
[13102]    80 13102    47549     2586      34       95         -1000 postgres
[13103]    80 13103    47549     2586      34       95         -1000 postgres
[13105]   500 13105   578961    40138     324        0             0 rekonq
[13106]    80 13106    47595     3889      44       93         -1000 postgres
[13107]    80 13107    47549     2586      34       95         -1000 postgres
[13108]    80 13108    47549     2586      34       95         -1000 postgres
[13109]    80 13109    49319    34543      99       92         -1000 postgres
[13111]    80 13111    47549     2571      34       95         -1000 postgres
[13112]    80 13112    49304    35409      99       92         -1000 postgres
[13114]    80 13114    49304    36076      99       92         -1000 postgres
[13115]    80 13115    47862     5758      65       92         -1000 postgres
[13116]   500 13116   113252    14152     201        0             0 kaddressbook
[13122]   500 13122   493943    33314     323        0             0 kmail
[13126]   500 13126   123010    12244     188        0             0 konqueror
[13140]   500 13140    64238     5816      96        0             0 applet.py
[13141]   500 13141   115594    19636     162        0             0 net_applet
[13143]   500 13143    95177     8297     141        0             0 polkit-kde-auth
[13148]   500 13148    66455     1821      32        0             0 xsettings-kde
[13149]   500 13149    10882     1157      27        0             0 xload
[13155]    80 13155    47803     4595      53       93         -1000 postgres
[13156]    80 13156    47973    11993      86       92         -1000 postgres
[13171]   500 13171    73289     9621     132        0             0 kwalletmanager
[13178]   500 13178    84306     1842      35        0             0 at-spi-bus-laun
[13182]   500 13182     3403      569      12        0             0 dbus-daemon
[13185]   500 13185    31230     1213      31        0             0 at-spi2-registr
[13189]   500 13189    46019     1304      27        0             0 gvfsd
[13231]    80 13231    48340    35725      97       92         -1000 postgres
[13233]    80 13233    47925    24455      91       92         -1000 postgres
[13309]    80 13309    47549     2555      34       95         -1000 postgres
[13442]   500 13442   260391    19500     137        0           200 python
[13447]    80 13447    49761    29978     101       83         -1000 postgres
[13470]   500 13470   122584     8709     157        0             0 kio_http
[13482]    80 13482    48989    18272      99       83         -1000 postgres
[13501]   500 13501   122587     8695     157        0             0 kio_http
[13505]   500 13505   122584     8710     157        0             0 kio_http
[13512]   500 13512   122585     8694     157        0             0 kio_http
[13523]   500 13523   122584     8686     157        0             0 kio_http
[13526]   500 13526   122584     8702     157        0             0 kio_http
[13788]   500 13788   115594    13290     145        0             0 net_applet
[13815]   500 13815    40303     3579      48        0             0 netmon_cron_par
[13830]   500 13830    23067     3637      47        0             0 content_index.p
[13843]   500 13843    21772     3377      46        0             0 decode-hardware
[13863]   500 13863    24181     3735      52        0             0 netmon_decode.p
[13909]   500 13909    21912     3505      47        0             0 python
[13914]   500 13914     2330      599      10        0             0 mails-process-n
[13925]   500 13925    24138     3722      53        0             0 netmon_cflastse
[13989]     0 13989     1975      438       9        0             0 agetty
[13994]   500 13994    34668     3038      66        0             0 kcminit
[13995]   500 13995    43712     4218      78        0             0 kcminit
[13996]   500 13996     4623      678      13        0             0 setxkbmap
[13997]     0 13997     4582      268      13        0             0 xkbcomp
Out of memory: Kill process 13442 (python) score 208 or sacrifice child
Killed process 13442 (python) total-vm:1041564kB, anon-rss:67284kB, file-rss:10716kB

SysRq : HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k) show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q) unraw(r) sync(s) show-task-states(t) unmount(u) force-fb(V) show-blocked-tasks(w) dump-ftrace-buffer(z) 
SysRq : HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k) show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q) unraw(r) sync(s) show-task-states(t) unmount(u) force-fb(V) show-blocked-tasks(w) dump-ftrace-buffer(z) 
SysRq : HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k) show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q) unraw(r) sync(s) show-task-states(t) unmount(u) force-fb(V) show-blocked-tasks(w) dump-ftrace-buffer(z) 
SysRq : Restore framebuffer console
SysRq : HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k) show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q) unraw(r) sync(s) show-task-states(t) unmount(u) force-fb(V) show-blocked-tasks(w) dump-ftrace-buffer(z) 
SysRq : Keyboard mode set to system default

-- Reboot --

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: Early test: hangs in mm/compact.c w. Linus's 12d7aacab56e9ef185c
  2014-11-06 19:23       ` P. Christeas
@ 2014-11-06 21:38         ` Vlastimil Babka
  2014-11-08 13:11           ` P. Christeas
  0 siblings, 1 reply; 20+ messages in thread
From: Vlastimil Babka @ 2014-11-06 21:38 UTC (permalink / raw)
  To: P. Christeas; +Cc: linux-mm, Joonsoo Kim, lkml

On 11/06/2014 08:23 PM, P. Christeas wrote:
> On Wednesday 05 November 2014, Vlastimil Babka wrote:
>> Can you please try the following patch?
>> --- a/mm/compaction.c
>> +++ b/mm/compaction.c
>> @@ -1325,13 +1325,6 @@ unsigned long try_to_compact_pages(struct zonelist
>> -			compaction_defer_reset(zone, order, false);
> 
> NACK :(

Sigh.

> I just got again into a state that some task was spinning out of control, and 
> blocking the rest of the desktop.

Well this is similar to reports [1] and [2] except [1] points to
isolate_freepages_block() and your traces only go as deep as compact_zone. Which
probably inlines isolate_migratepages* but I would expect it cannot inline
isolate_freepages* due to invocation via pointer.

> You will see me trying a few things, apparently the first OOM managed to 
> unblock something, but a few seconds later the system "stepped" on some other 
> blocking task.
> 
> See attached log, it may only give you some hint; the problem could well be in 
> some other part of the kernel.

Well I doubt that but I'd like to be surprised :)

> In the meanwhile, I'm pulling linus/master ...

Could you perhaps bisect the most suspicious part? It's not a lot of commits
and you seem to be reproducing this quite easily?

commit 447f05bb488bff4282088259b04f47f0f9f76760 should be good
commit 6d7ce55940b6ecd463ca044ad241f0122d913293 should be bad

If that's true, then bisection should find the cause rather quickly.

Oh and did I ask in this thread for /proc/zoneinfo yet? :)

Thanks.

> kcrash.log
> 

[1]
http://article.gmane.org/gmane.linux.kernel.mm/124451/match=isolate_freepages_block+very+high+intermittent+overhead

[2] https://lkml.org/lkml/2014/11/4/904

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: Early test: hangs in mm/compact.c w. Linus's 12d7aacab56e9ef185c
  2014-11-06 21:38         ` Vlastimil Babka
@ 2014-11-08 13:11           ` P. Christeas
  2014-11-08 22:18             ` Vlastimil Babka
  0 siblings, 1 reply; 20+ messages in thread
From: P. Christeas @ 2014-11-08 13:11 UTC (permalink / raw)
  To: Vlastimil Babka; +Cc: linux-mm, Joonsoo Kim, lkml

[-- Attachment #1: Type: text/plain, Size: 1495 bytes --]

On Thursday 06 November 2014, Vlastimil Babka wrote:
> > On Wednesday 05 November 2014, Vlastimil Babka wrote:
> >> Can you please try the following patch?
> >> -			compaction_defer_reset(zone, order, false);
> Oh and did I ask in this thread for /proc/zoneinfo yet? :)

Using that same kernel[1], got again into a race, gathered a few more data.

This time, I had 1x "urpmq" process [2] hung at 100% CPU , when "kwin" got 
apparently blocked (100% CPU, too) trying to resize a GUI window. I suppose 
the resizing operation would mean heavy memory alloc/free.

The rest of the system was responsive, I could easily get a console, login, 
gather the files.. Then, I have *killed* -9 the "urpmq" process, which solved 
the race and my system is still alive! "kwin" is still running, returned to 
regular CPU load.

Attached is traces from SysRq+l (pressed a few times, wanted to "snapshot" the 
stack) and /proc/zoneinfo + /proc/vmstat

Bisection is not yet meaningful, IMHO, because I cannot be sure that "good" 
points are really free from this issue. I'd estimate that each test would take 
+3days, unless I really find a deterministic way to reproduce the issue .


Thank you, again.


[1] linus's didn't have any -mm changes, so I haven't compiled anything yet. 
This means it also contains the "- compaction_defer_reset()" change

[2] urpmq is a Mandrake distro Perl script for querying the RPM database. It 
does some disk I/O , loads data into allocated Perl structs and sorts that, 
FYI.

[-- Attachment #2: zoneinfo.log --]
[-- Type: text/x-log, Size: 5002 bytes --]

Node 0, zone      DMA
  pages free     3055
        min      58
        low      72
        high     87
        scanned  0
        spanned  4095
        present  3998
        managed  3977
    nr_free_pages 3055
    nr_alloc_batch 15
    nr_inactive_anon 295
    nr_active_anon 132
    nr_inactive_file 134
    nr_active_file 198
    nr_unevictable 0
    nr_mlock     0
    nr_anon_pages 388
    nr_mapped    84
    nr_file_pages 386
    nr_dirty     0
    nr_writeback 0
    nr_slab_reclaimable 59
    nr_slab_unreclaimable 32
    nr_page_table_pages 57
    nr_kernel_stack 2
    nr_unstable  0
    nr_bounce    0
    nr_vmscan_write 1301
    nr_vmscan_immediate_reclaim 272
    nr_writeback_temp 0
    nr_isolated_anon 0
    nr_isolated_file 0
    nr_shmem     19
    nr_dirtied   21715
    nr_written   20583
    nr_pages_scanned 0
    workingset_refault 7169
    workingset_activate 1604
    workingset_nodereclaim 0
    nr_anon_transparent_hugepages 0
    nr_free_cma  0
        protection: (0, 2984, 2984, 2984)
  pagesets
    cpu: 0
              count: 0
              high:  0
              batch: 1
  vm stats threshold: 4
    cpu: 1
              count: 0
              high:  0
              batch: 1
  vm stats threshold: 4
  all_unreclaimable: 0
  start_pfn:         1
  inactive_ratio:    1
Node 0, zone    DMA32
  pages free     51824
        min      11205
        low      14006
        high     16807
        scanned  0
        spanned  779984
        present  779984
        managed  764335
    nr_free_pages 51824
    nr_alloc_batch 42
    nr_inactive_anon 108284
    nr_active_anon 388047
    nr_inactive_file 28047
    nr_active_file 95328
    nr_unevictable 0
    nr_mlock     0
    nr_anon_pages 444472
    nr_mapped    78178
    nr_file_pages 186535
    nr_dirty     236
    nr_writeback 0
    nr_slab_reclaimable 53697
    nr_slab_unreclaimable 11297
    nr_page_table_pages 18188
    nr_kernel_stack 483
    nr_unstable  0
    nr_bounce    0
    nr_vmscan_write 423678
    nr_vmscan_immediate_reclaim 2915
    nr_writeback_temp 0
    nr_isolated_anon 0
    nr_isolated_file 0
    nr_shmem     50993
    nr_dirtied   10098257
    nr_written   8809535
    nr_pages_scanned 0
    workingset_refault 5683710
    workingset_activate 1087302
    workingset_nodereclaim 1664
    nr_anon_transparent_hugepages 334
    nr_free_cma  0
        protection: (0, 0, 0, 0)
  pagesets
    cpu: 0
              count: 155
              high:  186
              batch: 31
  vm stats threshold: 24
    cpu: 1
              count: 49
              high:  186
              batch: 31
  vm stats threshold: 24
  all_unreclaimable: 0
  start_pfn:         4096
  inactive_ratio:    4

/proc/vmstat:
nr_free_pages 24041
nr_alloc_batch 1364
nr_inactive_anon 108048
nr_active_anon 397021
nr_inactive_file 42071
nr_active_file 102045
nr_unevictable 0
nr_mlock 0
nr_anon_pages 453175
nr_mapped 79221
nr_file_pages 208686
nr_dirty 977
nr_writeback 0
nr_slab_reclaimable 54008
nr_slab_unreclaimable 11475
nr_page_table_pages 19820
nr_kernel_stack 488
nr_unstable 0
nr_bounce 0
nr_vmscan_write 425540
nr_vmscan_immediate_reclaim 3187
nr_writeback_temp 0
nr_isolated_anon 0
nr_isolated_file 0
nr_shmem 50631
nr_dirtied 10151224
nr_written 8851175
nr_pages_scanned 0
workingset_refault 5711048
workingset_activate 1090895
workingset_nodereclaim 1664
nr_anon_transparent_hugepages 331
nr_free_cma 0
nr_dirty_threshold 29656
nr_dirty_background_threshold 14828
pgpgin 26370697
pgpgout 36940756
pswpin 197981
pswpout 424588
pgalloc_dma 379037
pgalloc_dma32 226001662
pgalloc_normal 0
pgalloc_movable 0
pgfree 230530685
pgactivate 8145753
pgdeactivate 9388084
pgfault 205223740
pgmajfault 189721
pgrefill_dma 15435
pgrefill_dma32 10362280
pgrefill_normal 0
pgrefill_movable 0
pgsteal_kswapd_dma 11715
pgsteal_kswapd_dma32 7800447
pgsteal_kswapd_normal 0
pgsteal_kswapd_movable 0
pgsteal_direct_dma 0
pgsteal_direct_dma32 990214
pgsteal_direct_normal 0
pgsteal_direct_movable 0
pgscan_kswapd_dma 15269
pgscan_kswapd_dma32 9268463
pgscan_kswapd_normal 0
pgscan_kswapd_movable 0
pgscan_direct_dma 0
pgscan_direct_dma32 1139388
pgscan_direct_normal 0
pgscan_direct_movable 0
pgscan_direct_throttle 0
pginodesteal 0
slabs_scanned 13515392
kswapd_inodesteal 2787
kswapd_low_wmark_hit_quickly 13889
kswapd_high_wmark_hit_quickly 8171
pageoutrun 24547
allocstall 6766
pgrotated 426791
drop_pagecache 0
drop_slab 0
pgmigrate_success 3318478
pgmigrate_fail 143
compact_migrate_scanned 25250660
compact_free_scanned 1321336375
compact_isolated 8016565
compact_stall 10944
compact_fail 9741
compact_success 1203
htlb_buddy_alloc_success 0
htlb_buddy_alloc_fail 0
unevictable_pgs_culled 934
unevictable_pgs_scanned 0
unevictable_pgs_rescued 3228
unevictable_pgs_mlocked 4060
unevictable_pgs_munlocked 4060
unevictable_pgs_cleared 0
unevictable_pgs_stranded 0
thp_fault_alloc 12914
thp_fault_fallback 9010
thp_collapse_alloc 5147
thp_collapse_alloc_failed 3115
thp_split 217
thp_zero_page_alloc 9
thp_zero_page_alloc_failed 0

[-- Attachment #3: kcrash2.log --]
[-- Type: text/x-log, Size: 39529 bytes --]

SysRq : Changing Loglevel
Loglevel set to 8
SysRq : Show backtrace of all active CPUs
sending NMI to all CPUs:
NMI backtrace for cpu 0
CPU: 0 PID: 7037 Comm: urpmq Not tainted 3.18.0-rc3+ #46
Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
task: ffff88000387de80 ti: ffff88007dddc000 task.ti: ffff88007dddc000
RIP: 0010:[<ffffffff811df82b>]  [<ffffffff811df82b>] __const_udelay+0x15/0x29
RSP: 0000:ffff8800bf203b68  EFLAGS: 00000006
RAX: 0000000001062560 RBX: 0000000000002710 RCX: 0000000000000007
RDX: 00000000001859ed RSI: 0000000000000c00 RDI: 0000000000418958
RBP: ffff8800bf203b68 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000046 R11: 0000000000000046 R12: 0000000000000008
R13: 000000000000006c R14: 0000000000000001 R15: ffffffff81668f90
FS:  00007f20be79b700(0000) GS:ffff8800bf200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f0428984000 CR3: 0000000012f19000 CR4: 00000000000007f0
Stack:
 ffff8800bf203b88 ffffffff81025de1 0000000080010002 ffffffff816692b0
 ffff8800bf203b98 ffffffff8126bc4f ffff8800bf203bc8 ffffffff8126c186
 ffff88003780c000 0000000000000001 0000000000000026 ffff88003780c001
Call Trace:
 <IRQ> 

 [<ffffffff81025de1>] arch_trigger_all_cpu_backtrace+0xa8/0xd2
 [<ffffffff8126bc4f>] sysrq_handle_showallcpus+0xe/0x10
 [<ffffffff8126c186>] __handle_sysrq+0x94/0x126
 [<ffffffff8126c329>] sysrq_filter+0xee/0x287
 [<ffffffff812c0fd8>] input_to_handler+0x5e/0xcb
 [<ffffffff812c1de2>] input_pass_values.part.3+0x76/0x134
 [<ffffffff812c3eae>] input_handle_event+0x457/0x46d
 [<ffffffff812c3f19>] input_event+0x55/0x6f
 [<ffffffff812c6fb5>] input_sync+0xf/0x11
 [<ffffffff812c7f47>] atkbd_interrupt+0x4d5/0x595
 [<ffffffff812bf2c3>] serio_interrupt+0x43/0x7d
 [<ffffffff812bfa2e>] i8042_interrupt+0x292/0x2a8
 [<ffffffff8108b64b>] ? tick_sched_do_timer+0x33/0x33
 [<ffffffff810729a6>] handle_irq_event_percpu+0x44/0x19f
 [<ffffffff81072b3d>] handle_irq_event+0x3c/0x5c
 [<ffffffff81025e49>] ? apic_eoi+0x18/0x1a
 [<ffffffff810752b2>] handle_edge_irq+0x95/0xae
 [<ffffffff81004679>] handle_irq+0x158/0x16d
 [<ffffffff8105683f>] ? get_parent_ip+0xe/0x3e
 [<ffffffff81003f71>] do_IRQ+0x58/0xda
 [<ffffffff813ba1ea>] common_interrupt+0x6a/0x6a
 <EOI> 

 [<ffffffff813b6b95>] ? preempt_schedule_irq+0x3c/0x59
 [<ffffffff810d75cc>] ? __zone_watermark_ok+0x7a/0x85
 [<ffffffff810d839e>] zone_watermark_ok+0x1a/0x1c
 [<ffffffff810ef3d7>] compact_zone+0x215/0x4b2
 [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
 [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
 [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
 [<ffffffff810da5c8>] __alloc_pages_nodemask+0x5f0/0x799
 [<ffffffff811163bb>] do_huge_pmd_anonymous_page+0x13c/0x255
 [<ffffffff810f65bd>] handle_mm_fault+0x112/0x808
 [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
 [<ffffffff8111af2b>] ? new_sync_read+0x74/0x98
 [<ffffffff8111b2c3>] ? fsnotify_access+0x5a/0x63
 [<ffffffff8111b602>] ? vfs_read+0xac/0xd2
 [<ffffffff8111b1f9>] ? fdput_pos.isra.13+0x29/0x30
 [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
 [<ffffffff813bb162>] page_fault+0x22/0x30
Code: 48 ff c8 75 fb 48 ff c8 5d c3 55 48 89 e5 ff 15 a4 0c 48 00 5d c3 55 48 8d 04 bd 00 00 00 00 48 89 e5 65 48 8b 14 25 20 26 01 00 <48> 69 d2 fa 00 00 00 f7 e2 48 8d 7a 01 e8 cd ff ff ff 5d c3 48 
NMI backtrace for cpu 1
CPU: 1 PID: 7072 Comm: ps Not tainted 3.18.0-rc3+ #46
Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
task: ffff88000387c6e0 ti: ffff880059bcc000 task.ti: ffff880059bcc000
RIP: 0010:[<ffffffff81136da3>]  [<ffffffff81136da3>] seq_put_decimal_ull+0x46/0x72
RSP: 0018:ffff880059bcfce8  EFLAGS: 00000206
RAX: 00000000000000ad RBX: ffff880091097480 RCX: 00000000000000ae
RDX: 0000000000000030 RSI: 0000000000000020 RDI: ffff8800b0230000
RBP: ffff880059bcfcf8 R08: 000000000000000a R09: 00000000ffffffff
R10: ffffffff81136749 R11: ffffffff59bcfcb0 R12: ffff880080768000
R13: ffff8800ba205a40 R14: 0000000000000001 R15: 0000000000003d00
FS:  00007f042896a700(0000) GS:ffff8800bf300000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000f0b018 CR3: 000000003abb8000 CR4: 00000000000007e0
Stack:
 00000000ffffffff ffff880091097480 ffff880059bcfe28 ffffffff8116d751
 ffff8800749e8480 0000000000800e06 5300880000000000 ffffffffffffffff
 0000000000000000 0000000000000000 0000000000000000 0000000000000000
Call Trace:
 [<ffffffff8116d751>] do_task_stat+0x7cf/0x980
 [<ffffffff8116e12b>] proc_tgid_stat+0xf/0x11
 [<ffffffff81168d8a>] proc_single_show+0x4c/0x6e
 [<ffffffff81137069>] seq_read+0x163/0x330
 [<ffffffff8111b5d3>] vfs_read+0x7d/0xd2
 [<ffffffff811322c1>] ? __fdget_pos+0xd/0x3c
 [<ffffffff8111bc4c>] SyS_read+0x42/0x79
 [<ffffffff813b9752>] system_call_fastpath+0x12/0x17
Code: 0f 48 8b 0f 48 8d 78 01 48 89 7b 18 40 88 34 01 48 83 fa 09 48 8b 43 18 48 8b 3b 77 10 48 8d 48 01 83 c2 30 48 89 4b 18 88 14 07 <eb> 17 8b 73 08 48 01 c7 29 c6 e8 89 84 0a 00 85 c0 74 0a 48 98 
INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 76.962 msecs
SysRq : HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k) show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q) unraw(r) sync(s) show-task-states(t) unmount(u) force-fb(V) show-blocked-tasks(w) dump-ftrace-buffer(z) 
SysRq : Show backtrace of all active CPUs
sending NMI to all CPUs:
NMI backtrace for cpu 1
CPU: 1 PID: 7037 Comm: urpmq Not tainted 3.18.0-rc3+ #46
Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
task: ffff88000387de80 ti: ffff88007dddc000 task.ti: ffff88007dddc000
RIP: 0010:[<ffffffff810ef578>]  [<ffffffff810ef578>] compact_zone+0x3b6/0x4b2
RSP: 0000:ffff88007dddfa38  EFLAGS: 00000297
RAX: 00000000ffffffff RBX: ffffffff8168be40 RCX: 0000000000000008
RDX: 0000000000000800 RSI: 0000000000000009 RDI: ffffffff8168be40
RBP: ffff88007dddfa98 R08: 0000000000000000 R09: fffffffffffffef1
R10: 0000000000000038 R11: ffffffff8168be40 R12: 00000000000bf800
R13: 00000000000bf600 R14: ffff88007dddfab8 R15: 0000160000000000
FS:  00007f20be79b700(0000) GS:ffff8800bf300000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f143453c000 CR3: 0000000012f19000 CR4: 00000000000007e0
Stack:
 ffff8800bf312e80 ffffea0002fd0000 0000000000000020 ffff88000387de80
 0000000000000004 ffff88007dddfac8 0000000000000000 0000000000000000
 0000000000000009 ffff88007dddfcec 0000000000000002 0000000000000000
Call Trace:
 [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
 [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
 [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
 [<ffffffff810da5c8>] __alloc_pages_nodemask+0x5f0/0x799
 [<ffffffff811163bb>] do_huge_pmd_anonymous_page+0x13c/0x255
 [<ffffffff810f65bd>] handle_mm_fault+0x112/0x808
 [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
 [<ffffffff8111af2b>] ? new_sync_read+0x74/0x98
 [<ffffffff8111b2c3>] ? fsnotify_access+0x5a/0x63
 [<ffffffff8111b602>] ? vfs_read+0xac/0xd2
 [<ffffffff8111b1f9>] ? fdput_pos.isra.13+0x29/0x30
 [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
 [<ffffffff813bb162>] page_fault+0x22/0x30
Code: 00 48 8b 1d 9b 2e 5a 00 48 85 db 74 9e 48 8b 7b 08 44 89 e6 ff 13 48 83 c3 10 48 83 3b 00 eb eb 41 83 7e 40 01 4d 8b 6e 38 19 c0 <89> 45 c0 4d 8d a5 00 02 00 00 83 65 c0 04 49 81 e4 00 fe ff ff 
NMI backtrace for cpu 0
CPU: 0 PID: 7356 Comm: kwin Not tainted 3.18.0-rc3+ #46
Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
task: ffff8800b8c23f00 ti: ffff88009b1f8000 task.ti: ffff88009b1f8000
RIP: 0010:[<ffffffff811df861>]  [<ffffffff811df861>] delay_tsc+0x1/0xa2
RSP: 0000:ffff8800bf203b48  EFLAGS: 00000807
RAX: 0000000026e004c0 RBX: 0000000000002710 RCX: 0000000000000007
RDX: 00000000001859f9 RSI: 0000000000000c00 RDI: 00000000001859fa
RBP: ffff8800bf203b58 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000046 R11: 0000000000000046 R12: 0000000000000008
R13: 000000000000006c R14: 0000000000000001 R15: ffffffff81668f90
FS:  00007f445c0f17c0(0000) GS:ffff8800bf200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000003a5b1d8 CR3: 00000000a6d15000 CR4: 00000000000007f0
Stack:
 ffff8800bf203b58 ffffffff811df814 ffff8800bf203b68 ffffffff811df83d
 ffff8800bf203b88 ffffffff81025de1 0000000080010002 ffffffff816692b0
 ffff8800bf203b98 ffffffff8126bc4f ffff8800bf203bc8 ffffffff8126c186
Call Trace:
 <IRQ> 

 [<ffffffff811df814>] ? __delay+0xa/0xc
 [<ffffffff811df83d>] __const_udelay+0x27/0x29
 [<ffffffff81025de1>] arch_trigger_all_cpu_backtrace+0xa8/0xd2
 [<ffffffff8126bc4f>] sysrq_handle_showallcpus+0xe/0x10
 [<ffffffff8126c186>] __handle_sysrq+0x94/0x126
 [<ffffffff8126c329>] sysrq_filter+0xee/0x287
 [<ffffffff812c0fd8>] input_to_handler+0x5e/0xcb
 [<ffffffff812c1de2>] input_pass_values.part.3+0x76/0x134
 [<ffffffff812c3eae>] input_handle_event+0x457/0x46d
 [<ffffffff812c3f19>] input_event+0x55/0x6f
 [<ffffffff812c6fb5>] input_sync+0xf/0x11
 [<ffffffff812c7f47>] atkbd_interrupt+0x4d5/0x595
 [<ffffffff812bf2c3>] serio_interrupt+0x43/0x7d
 [<ffffffff812bfa2e>] i8042_interrupt+0x292/0x2a8
 [<ffffffff8108b64b>] ? tick_sched_do_timer+0x33/0x33
 [<ffffffff810729a6>] handle_irq_event_percpu+0x44/0x19f
 [<ffffffff81072b3d>] handle_irq_event+0x3c/0x5c
 [<ffffffff81025e49>] ? apic_eoi+0x18/0x1a
 [<ffffffff810752b2>] handle_edge_irq+0x95/0xae
 [<ffffffff81004679>] handle_irq+0x158/0x16d
 [<ffffffff8105683f>] ? get_parent_ip+0xe/0x3e
 [<ffffffff81003f71>] do_IRQ+0x58/0xda
 [<ffffffff813ba1ea>] common_interrupt+0x6a/0x6a
 <EOI> 

 [<ffffffff810ef3d9>] ? compact_zone+0x217/0x4b2
 [<ffffffff810ef3d7>] ? compact_zone+0x215/0x4b2
 [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
 [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
 [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
 [<ffffffff810da3e1>] __alloc_pages_nodemask+0x409/0x799
 [<ffffffff810fdcba>] ? anon_vma_prepare+0x2b/0x12c
 [<ffffffff811163bb>] do_huge_pmd_anonymous_page+0x13c/0x255
 [<ffffffff810f65bd>] handle_mm_fault+0x112/0x808
 [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
 [<ffffffff8105ca95>] ? set_next_entity+0x3a/0x63
 [<ffffffff811e952e>] ? debug_smp_processor_id+0x17/0x19
 [<ffffffff810017c3>] ? __switch_to+0x33f/0x49c
 [<ffffffff813b9028>] ? _raw_spin_unlock_irq+0x14/0x27
 [<ffffffff811e952e>] ? debug_smp_processor_id+0x17/0x19
 [<ffffffff813b66d1>] ? __schedule+0x2d9/0x451
 [<ffffffff810f9997>] ? SyS_mmap_pgoff+0x183/0x1cf
 [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
 [<ffffffff813bb162>] page_fault+0x22/0x30
Code: 7a 01 e8 cd ff ff ff 5d c3 48 69 ff c7 10 00 00 55 48 89 e5 e8 c7 ff ff ff 5d c3 55 48 8d 3c bf 48 89 e5 e8 b8 ff ff ff 5d c3 55 <48> 89 e5 41 56 41 55 41 54 41 89 fc bf 01 00 00 00 53 e8 f7 6f 
INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 182.694 msecs
SysRq : Show backtrace of all active CPUs
sending NMI to all CPUs:
NMI backtrace for cpu 1
CPU: 1 PID: 7037 Comm: urpmq Not tainted 3.18.0-rc3+ #46
Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
task: ffff88000387de80 ti: ffff88007dddc000 task.ti: ffff88007dddc000
RIP: 0010:[<ffffffff810d75c9>]  [<ffffffff810d75c9>] __zone_watermark_ok+0x77/0x85
RSP: 0000:ffff88007dddfa10  EFLAGS: 00000212
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000001
RDX: 000000000000057c RSI: 0000000000000009 RDI: ffffffff8168be40
RBP: ffff88007dddfa18 R08: 0000000000000000 R09: 000000000000ca1f
R10: 0000000000001c5b R11: ffffffff8168be40 R12: 00000000000bf800
R13: 00000000000bf600 R14: ffff88007dddfab8 R15: 0000160000000000
FS:  00007f20be79b700(0000) GS:ffff8800bf300000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f143453c000 CR3: 0000000012f19000 CR4: 00000000000007e0
Stack:
 ffffffff8168be40 ffff88007dddfa28 ffffffff810d839e ffff88007dddfa98
 ffffffff810ef3d7 ffff8800bf312e80 ffffea0002fd0000 0000000000000020
 ffff88000387de80 0000000000000004 ffff88007dddfac8 0000000000000000
Call Trace:
 [<ffffffff810d839e>] zone_watermark_ok+0x1a/0x1c
 [<ffffffff810ef3d7>] compact_zone+0x215/0x4b2
 [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
 [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
 [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
 [<ffffffff810da5c8>] __alloc_pages_nodemask+0x5f0/0x799
 [<ffffffff811163bb>] do_huge_pmd_anonymous_page+0x13c/0x255
 [<ffffffff810f65bd>] handle_mm_fault+0x112/0x808
 [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
 [<ffffffff8111af2b>] ? new_sync_read+0x74/0x98
 [<ffffffff8111b2c3>] ? fsnotify_access+0x5a/0x63
 [<ffffffff8111b602>] ? vfs_read+0xac/0xd2
 [<ffffffff8111b1f9>] ? fdput_pos.isra.13+0x29/0x30
 [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
 [<ffffffff813bb162>] page_fault+0x22/0x30
Code: d2 31 c0 31 c9 48 03 54 df 18 49 39 d1 7e 27 39 ce 76 21 48 6b d1 58 49 d1 fa 48 8b 94 17 28 01 00 00 48 d3 e2 48 ff c1 49 29 d1 <4d> 39 d1 7f df 31 c0 eb 02 b0 01 5b 5d c3 49 b9 13 da 4b 68 2f 
NMI backtrace for cpu 0
CPU: 0 PID: 7356 Comm: kwin Not tainted 3.18.0-rc3+ #46
Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
task: ffff8800b8c23f00 ti: ffff88009b1f8000 task.ti: ffff88009b1f8000
RIP: 0010:[<ffffffff811df86a>]  [<ffffffff811df86a>] delay_tsc+0xa/0xa2
RSP: 0000:ffff8800bf203b30  EFLAGS: 00000807
RAX: 0000000026e004c0 RBX: 0000000000002710 RCX: 0000000000000007
RDX: 00000000001859f9 RSI: 0000000000000c00 RDI: 00000000001859fa
RBP: ffff8800bf203b48 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000046 R11: 0000000000000046 R12: 0000000000000008
R13: 000000000000006c R14: 0000000000000001 R15: ffffffff81668f90
FS:  00007f445c0f17c0(0000) GS:ffff8800bf200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000003a5b1d8 CR3: 00000000a6d15000 CR4: 00000000000007f0
Stack:
 0000000000000008 000000000000006c 0000000000000001 ffff8800bf203b58
 ffffffff811df814 ffff8800bf203b68 ffffffff811df83d ffff8800bf203b88
 ffffffff81025de1 0000000080010002 ffffffff816692b0 ffff8800bf203b98
Call Trace:
 <IRQ> 

 [<ffffffff811df814>] __delay+0xa/0xc
 [<ffffffff811df83d>] __const_udelay+0x27/0x29
 [<ffffffff81025de1>] arch_trigger_all_cpu_backtrace+0xa8/0xd2
 [<ffffffff8126bc4f>] sysrq_handle_showallcpus+0xe/0x10
 [<ffffffff8126c186>] __handle_sysrq+0x94/0x126
 [<ffffffff8126c329>] sysrq_filter+0xee/0x287
 [<ffffffff812c0fd8>] input_to_handler+0x5e/0xcb
 [<ffffffff812c1de2>] input_pass_values.part.3+0x76/0x134
 [<ffffffff812c3eae>] input_handle_event+0x457/0x46d
 [<ffffffff812c3f19>] input_event+0x55/0x6f
 [<ffffffff812c6fb5>] input_sync+0xf/0x11
 [<ffffffff812c7f47>] atkbd_interrupt+0x4d5/0x595
 [<ffffffff812bf2c3>] serio_interrupt+0x43/0x7d
 [<ffffffff812bfa2e>] i8042_interrupt+0x292/0x2a8
 [<ffffffff8108b64b>] ? tick_sched_do_timer+0x33/0x33
 [<ffffffff810729a6>] handle_irq_event_percpu+0x44/0x19f
 [<ffffffff81072b3d>] handle_irq_event+0x3c/0x5c
 [<ffffffff81025e49>] ? apic_eoi+0x18/0x1a
 [<ffffffff810752b2>] handle_edge_irq+0x95/0xae
 [<ffffffff81004679>] handle_irq+0x158/0x16d
 [<ffffffff8105683f>] ? get_parent_ip+0xe/0x3e
 [<ffffffff81003f71>] do_IRQ+0x58/0xda
 [<ffffffff813ba1ea>] common_interrupt+0x6a/0x6a
 <EOI> 

 [<ffffffff810d839e>] ? zone_watermark_ok+0x1a/0x1c
 [<ffffffff810d839e>] ? zone_watermark_ok+0x1a/0x1c
 [<ffffffff810ef3d7>] compact_zone+0x215/0x4b2
 [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
 [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
 [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
 [<ffffffff810da3e1>] __alloc_pages_nodemask+0x409/0x799
 [<ffffffff810fdcba>] ? anon_vma_prepare+0x2b/0x12c
 [<ffffffff811163bb>] do_huge_pmd_anonymous_page+0x13c/0x255
 [<ffffffff810f65bd>] handle_mm_fault+0x112/0x808
 [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
 [<ffffffff8105ca95>] ? set_next_entity+0x3a/0x63
 [<ffffffff811e952e>] ? debug_smp_processor_id+0x17/0x19
 [<ffffffff810017c3>] ? __switch_to+0x33f/0x49c
 [<ffffffff813b9028>] ? _raw_spin_unlock_irq+0x14/0x27
 [<ffffffff811e952e>] ? debug_smp_processor_id+0x17/0x19
 [<ffffffff813b66d1>] ? __schedule+0x2d9/0x451
 [<ffffffff810f9997>] ? SyS_mmap_pgoff+0x183/0x1cf
 [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
 [<ffffffff813bb162>] page_fault+0x22/0x30
Code: 48 69 ff c7 10 00 00 55 48 89 e5 e8 c7 ff ff ff 5d c3 55 48 8d 3c bf 48 89 e5 e8 b8 ff ff ff 5d c3 55 48 89 e5 41 56 41 55 41 54 <41> 89 fc bf 01 00 00 00 53 e8 f7 6f e7 ff e8 9a 9c 00 00 41 89 
SysRq : Show backtrace of all active CPUs
sending NMI to all CPUs:
NMI backtrace for cpu 1
CPU: 1 PID: 7037 Comm: urpmq Not tainted 3.18.0-rc3+ #46
Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
task: ffff88000387de80 ti: ffff88007dddc000 task.ti: ffff88007dddc000
RIP: 0010:[<ffffffff8105686f>]  [<ffffffff8105686f>] preempt_count_add+0x0/0x8b
RSP: 0000:ffff8800bf303b20  EFLAGS: 00000807
RAX: 0000000026e004c0 RBX: 0000000000002710 RCX: 0000000000000007
RDX: 00000000001859f9 RSI: 0000000000000c00 RDI: 0000000000000001
RBP: ffff8800bf303b48 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000046 R11: 0000000000000046 R12: 00000000001859fa
R13: 000000000000006c R14: 0000000000000001 R15: ffffffff81668f90
FS:  00007f20be79b700(0000) GS:ffff8800bf300000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f143453c000 CR3: 0000000012f19000 CR4: 00000000000007e0
Stack:
 ffffffff811df878 0000000000002710 0000000000000008 000000000000006c
 0000000000000001 ffff8800bf303b58 ffffffff811df814 ffff8800bf303b68
 ffffffff811df83d ffff8800bf303b88 ffffffff81025de1 0000000080010002
Call Trace:
 <IRQ> 

 [<ffffffff811df878>] ? delay_tsc+0x18/0xa2
 [<ffffffff811df814>] __delay+0xa/0xc
 [<ffffffff811df83d>] __const_udelay+0x27/0x29
 [<ffffffff81025de1>] arch_trigger_all_cpu_backtrace+0xa8/0xd2
 [<ffffffff8126bc4f>] sysrq_handle_showallcpus+0xe/0x10
 [<ffffffff8126c186>] __handle_sysrq+0x94/0x126
 [<ffffffff8126c329>] sysrq_filter+0xee/0x287
 [<ffffffff812c0fd8>] input_to_handler+0x5e/0xcb
 [<ffffffff812c1de2>] input_pass_values.part.3+0x76/0x134
 [<ffffffff812c3eae>] input_handle_event+0x457/0x46d
 [<ffffffff812c3f19>] input_event+0x55/0x6f
 [<ffffffff812c6fb5>] input_sync+0xf/0x11
 [<ffffffff812c7f47>] atkbd_interrupt+0x4d5/0x595
 [<ffffffff812bf2c3>] serio_interrupt+0x43/0x7d
 [<ffffffff812bfa2e>] i8042_interrupt+0x292/0x2a8
 [<ffffffff8108b64b>] ? tick_sched_do_timer+0x33/0x33
 [<ffffffff810729a6>] handle_irq_event_percpu+0x44/0x19f
 [<ffffffff81072b3d>] handle_irq_event+0x3c/0x5c
 [<ffffffff81025e49>] ? apic_eoi+0x18/0x1a
 [<ffffffff810752b2>] handle_edge_irq+0x95/0xae
 [<ffffffff81004679>] handle_irq+0x158/0x16d
 [<ffffffff8105683f>] ? get_parent_ip+0xe/0x3e
 [<ffffffff81003f71>] do_IRQ+0x58/0xda
 [<ffffffff813ba1ea>] common_interrupt+0x6a/0x6a
 <EOI> 

 [<ffffffff813b6b95>] ? preempt_schedule_irq+0x3c/0x59
 [<ffffffff810d75b8>] ? __zone_watermark_ok+0x66/0x85
 [<ffffffff810d839e>] zone_watermark_ok+0x1a/0x1c
 [<ffffffff810ef3d7>] compact_zone+0x215/0x4b2
 [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
 [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
 [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
 [<ffffffff810da5c8>] __alloc_pages_nodemask+0x5f0/0x799
 [<ffffffff811163bb>] do_huge_pmd_anonymous_page+0x13c/0x255
 [<ffffffff810f65bd>] handle_mm_fault+0x112/0x808
 [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
 [<ffffffff8111af2b>] ? new_sync_read+0x74/0x98
 [<ffffffff8111b2c3>] ? fsnotify_access+0x5a/0x63
 [<ffffffff8111b602>] ? vfs_read+0xac/0xd2
 [<ffffffff8111b1f9>] ? fdput_pos.isra.13+0x29/0x30
 [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
 [<ffffffff813bb162>] page_fault+0x22/0x30
Code: 8b 45 00 48 8b 00 48 8b 58 08 48 89 df e8 15 f4 00 00 85 c0 74 0e 48 8b 45 00 48 8b 00 48 8b 00 48 8b 58 08 5a 48 89 d8 5b 5d c3 <55> 48 89 e5 53 89 fb 41 50 65 01 3c 25 90 b8 00 00 83 3d c1 3d 
NMI backtrace for cpu 0
CPU: 0 PID: 7356 Comm: kwin Not tainted 3.18.0-rc3+ #46
Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
task: ffff8800b8c23f00 ti: ffff88009b1f8000 task.ti: ffff88009b1f8000
RIP: 0010:[<ffffffff810ef3bc>]  [<ffffffff810ef3bc>] compact_zone+0x1fa/0x4b2
RSP: 0000:ffff88009b1fba38  EFLAGS: 00000217
RAX: 00000000000bf600 RBX: ffffffff8168be40 RCX: ffff88009b1fba09
RDX: 0000000000000000 RSI: 0000000000000009 RDI: ffff8800b8c23f00
RBP: ffff88009b1fba98 R08: 0000000000000000 R09: fffffffffffffefb
R10: 0000000000000038 R11: ffffffff8168be40 R12: 00000000000bf800
R13: 00000000000bf600 R14: ffff88009b1fbab8 R15: 0000160000000000
FS:  00007f445c0f17c0(0000) GS:ffff8800bf200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000003a5b1d8 CR3: 00000000a6d15000 CR4: 00000000000007f0
Stack:
 ffff8800354be660 ffffea0002fd0000 0000000000000020 ffff8800b8c23f00
 0000000000000004 ffff88009b1fbac8 ffff8800a6d1a800 0000000000000000
 0000000000000009 ffff88009b1fbcec 0000000000000002 0000000000000000
Call Trace:
 [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
 [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
 [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
 [<ffffffff810da3e1>] __alloc_pages_nodemask+0x409/0x799
 [<ffffffff810fdcba>] ? anon_vma_prepare+0x2b/0x12c
 [<ffffffff811163bb>] do_huge_pmd_anonymous_page+0x13c/0x255
 [<ffffffff810f65bd>] handle_mm_fault+0x112/0x808
 [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
 [<ffffffff8105ca95>] ? set_next_entity+0x3a/0x63
 [<ffffffff811e952e>] ? debug_smp_processor_id+0x17/0x19
 [<ffffffff810017c3>] ? __switch_to+0x33f/0x49c
 [<ffffffff813b9028>] ? _raw_spin_unlock_irq+0x14/0x27
 [<ffffffff811e952e>] ? debug_smp_processor_id+0x17/0x19
 [<ffffffff813b66d1>] ? __schedule+0x2d9/0x451
 [<ffffffff810f9997>] ? SyS_mmap_pgoff+0x183/0x1cf
 [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
 [<ffffffff813bb162>] page_fault+0x22/0x30
Code: 00 48 8b 45 b8 f6 40 16 04 0f 85 3e 01 00 00 c6 83 7c 05 00 00 01 e9 32 01 00 00 41 8b 76 48 83 fe ff 0f 84 b4 01 00 00 40 88 f1 <ba> 01 00 00 00 45 31 c0 d3 e2 48 89 df 31 c9 48 63 d2 48 03 53 
SysRq : Show backtrace of all active CPUs
sending NMI to all CPUs:
NMI backtrace for cpu 1
CPU: 1 PID: 7037 Comm: urpmq Not tainted 3.18.0-rc3+ #46
Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
task: ffff88000387de80 ti: ffff88007dddc000 task.ti: ffff88007dddc000
RIP: 0010:[<ffffffff811df82b>]  [<ffffffff811df82b>] __const_udelay+0x15/0x29
RSP: 0000:ffff8800bf303b68  EFLAGS: 00000006
RAX: 0000000001062560 RBX: 0000000000002710 RCX: 0000000000000007
RDX: 00000000001859ed RSI: 0000000000000c00 RDI: 0000000000418958
RBP: ffff8800bf303b68 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000046 R11: 0000000000000046 R12: 0000000000000008
R13: 000000000000006c R14: 0000000000000001 R15: ffffffff81668f90
FS:  00007f20be79b700(0000) GS:ffff8800bf300000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f143453c000 CR3: 0000000012f19000 CR4: 00000000000007e0
Stack:
 ffff8800bf303b88 ffffffff81025de1 0000000080010002 ffffffff816692b0
 ffff8800bf303b98 ffffffff8126bc4f ffff8800bf303bc8 ffffffff8126c186
 ffff88003780c000 0000000000000001 0000000000000026 ffff88003780c001
Call Trace:
 <IRQ> 

 [<ffffffff81025de1>] arch_trigger_all_cpu_backtrace+0xa8/0xd2
 [<ffffffff8126bc4f>] sysrq_handle_showallcpus+0xe/0x10
 [<ffffffff8126c186>] __handle_sysrq+0x94/0x126
 [<ffffffff8126c329>] sysrq_filter+0xee/0x287
 [<ffffffff812c0fd8>] input_to_handler+0x5e/0xcb
 [<ffffffff812c1de2>] input_pass_values.part.3+0x76/0x134
 [<ffffffff812c3eae>] input_handle_event+0x457/0x46d
 [<ffffffff812c3f19>] input_event+0x55/0x6f
 [<ffffffff812c6fb5>] input_sync+0xf/0x11
 [<ffffffff812c7f47>] atkbd_interrupt+0x4d5/0x595
 [<ffffffff812bf2c3>] serio_interrupt+0x43/0x7d
 [<ffffffff812bfa2e>] i8042_interrupt+0x292/0x2a8
 [<ffffffff8108b64b>] ? tick_sched_do_timer+0x33/0x33
 [<ffffffff810729a6>] handle_irq_event_percpu+0x44/0x19f
 [<ffffffff81072b3d>] handle_irq_event+0x3c/0x5c
 [<ffffffff81025e49>] ? apic_eoi+0x18/0x1a
 [<ffffffff810752b2>] handle_edge_irq+0x95/0xae
 [<ffffffff81004679>] handle_irq+0x158/0x16d
 [<ffffffff8105683f>] ? get_parent_ip+0xe/0x3e
 [<ffffffff81003f71>] do_IRQ+0x58/0xda
 [<ffffffff813ba1ea>] common_interrupt+0x6a/0x6a
 <EOI> 

 [<ffffffff810d75c0>] ? __zone_watermark_ok+0x6e/0x85
 [<ffffffff810d839e>] zone_watermark_ok+0x1a/0x1c
 [<ffffffff810ef3d7>] compact_zone+0x215/0x4b2
 [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
 [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
 [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
 [<ffffffff810da5c8>] __alloc_pages_nodemask+0x5f0/0x799
 [<ffffffff811163bb>] do_huge_pmd_anonymous_page+0x13c/0x255
 [<ffffffff810f65bd>] handle_mm_fault+0x112/0x808
 [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
 [<ffffffff8111af2b>] ? new_sync_read+0x74/0x98
 [<ffffffff8111b2c3>] ? fsnotify_access+0x5a/0x63
 [<ffffffff8111b602>] ? vfs_read+0xac/0xd2
 [<ffffffff8111b1f9>] ? fdput_pos.isra.13+0x29/0x30
 [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
 [<ffffffff813bb162>] page_fault+0x22/0x30
Code: 48 ff c8 75 fb 48 ff c8 5d c3 55 48 89 e5 ff 15 a4 0c 48 00 5d c3 55 48 8d 04 bd 00 00 00 00 48 89 e5 65 48 8b 14 25 20 26 01 00 <48> 69 d2 fa 00 00 00 f7 e2 48 8d 7a 01 e8 cd ff ff ff 5d c3 48 
NMI backtrace for cpu 0
CPU: 0 PID: 7356 Comm: kwin Not tainted 3.18.0-rc3+ #46
Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
task: ffff8800b8c23f00 ti: ffff88009b1f8000 task.ti: ffff88009b1f8000
RIP: 0010:[<ffffffff810d75af>]  [<ffffffff810d75af>] __zone_watermark_ok+0x5d/0x85
RSP: 0000:ffff88009b1fba10  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000007
RDX: 0000000000000f00 RSI: 0000000000000009 RDI: ffffffff8168be40
RBP: ffff88009b1fba18 R08: 0000000000000000 R09: 00000000000006fd
R10: 0000000000000071 R11: ffffffff8168be40 R12: 00000000000bf800
R13: 00000000000bf600 R14: ffff88009b1fbab8 R15: 0000160000000000
FS:  00007f445c0f17c0(0000) GS:ffff8800bf200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f26fa192000 CR3: 00000000a6d15000 CR4: 00000000000007f0
Stack:
 ffffffff8168be40 ffff88009b1fba28 ffffffff810d839e ffff88009b1fba98
 ffffffff810ef3d7 ffff8800354be660 ffffea0002fd0000 0000000000000020
 ffff8800b8c23f00 0000000000000004 ffff88009b1fbac8 ffff8800a6d1a800
Call Trace:
 [<ffffffff810d839e>] zone_watermark_ok+0x1a/0x1c
 [<ffffffff810ef3d7>] compact_zone+0x215/0x4b2
 [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
 [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
 [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
 [<ffffffff810da3e1>] __alloc_pages_nodemask+0x409/0x799
 [<ffffffff810fdcba>] ? anon_vma_prepare+0x2b/0x12c
 [<ffffffff811163bb>] do_huge_pmd_anonymous_page+0x13c/0x255
 [<ffffffff810f65bd>] handle_mm_fault+0x112/0x808
 [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
 [<ffffffff8105ca95>] ? set_next_entity+0x3a/0x63
 [<ffffffff811e952e>] ? debug_smp_processor_id+0x17/0x19
 [<ffffffff810017c3>] ? __switch_to+0x33f/0x49c
 [<ffffffff813b9028>] ? _raw_spin_unlock_irq+0x14/0x27
 [<ffffffff811e952e>] ? debug_smp_processor_id+0x17/0x19
 [<ffffffff813b66d1>] ? __schedule+0x2d9/0x451
 [<ffffffff810f9997>] ? SyS_mmap_pgoff+0x183/0x1cf
 [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
 [<ffffffff813bb162>] page_fault+0x22/0x30
Code: c2 41 80 e0 10 74 11 4c 89 d0 41 b8 04 00 00 00 48 99 49 f7 f8 49 29 c2 4c 89 d2 31 c0 31 c9 48 03 54 df 18 49 39 d1 7e 27 39 ce <76> 21 48 6b d1 58 49 d1 fa 48 8b 94 17 28 01 00 00 48 d3 e2 48 
SysRq : Show backtrace of all active CPUs
sending NMI to all CPUs:
NMI backtrace for cpu 0
CPU: 0 PID: 7356 Comm: kwin Not tainted 3.18.0-rc3+ #46
Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
task: ffff8800b8c23f00 ti: ffff88009b1f8000 task.ti: ffff88009b1f8000
RIP: 0010:[<ffffffff811df861>]  [<ffffffff811df861>] delay_tsc+0x1/0xa2
RSP: 0000:ffff8800bf203b48  EFLAGS: 00000807
RAX: 0000000026e004c0 RBX: 0000000000002710 RCX: 0000000000000007
RDX: 00000000001859f9 RSI: 0000000000000c00 RDI: 00000000001859fa
RBP: ffff8800bf203b58 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000046 R11: 0000000000000046 R12: 0000000000000008
R13: 000000000000006c R14: 0000000000000001 R15: ffffffff81668f90
FS:  00007f445c0f17c0(0000) GS:ffff8800bf200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007ffc189c5640 CR3: 00000000a6d15000 CR4: 00000000000007f0
Stack:
 ffff8800bf203b58 ffffffff811df814 ffff8800bf203b68 ffffffff811df83d
 ffff8800bf203b88 ffffffff81025de1 0000000080010002 ffffffff816692b0
 ffff8800bf203b98 ffffffff8126bc4f ffff8800bf203bc8 ffffffff8126c186
Call Trace:
 <IRQ> 

 [<ffffffff811df814>] ? __delay+0xa/0xc
 [<ffffffff811df83d>] __const_udelay+0x27/0x29
 [<ffffffff81025de1>] arch_trigger_all_cpu_backtrace+0xa8/0xd2
 [<ffffffff8126bc4f>] sysrq_handle_showallcpus+0xe/0x10
 [<ffffffff8126c186>] __handle_sysrq+0x94/0x126
 [<ffffffff8126c329>] sysrq_filter+0xee/0x287
 [<ffffffff812c0fd8>] input_to_handler+0x5e/0xcb
 [<ffffffff812c1de2>] input_pass_values.part.3+0x76/0x134
 [<ffffffff812c3eae>] input_handle_event+0x457/0x46d
 [<ffffffff812c3f19>] input_event+0x55/0x6f
 [<ffffffff812c6fb5>] input_sync+0xf/0x11
 [<ffffffff812c7f47>] atkbd_interrupt+0x4d5/0x595
 [<ffffffff812bf2c3>] serio_interrupt+0x43/0x7d
 [<ffffffff812bfa2e>] i8042_interrupt+0x292/0x2a8
 [<ffffffff8108b64b>] ? tick_sched_do_timer+0x33/0x33
 [<ffffffff810729a6>] handle_irq_event_percpu+0x44/0x19f
 [<ffffffff81072b3d>] handle_irq_event+0x3c/0x5c
 [<ffffffff81025e49>] ? apic_eoi+0x18/0x1a
 [<ffffffff810752b2>] handle_edge_irq+0x95/0xae
 [<ffffffff81004679>] handle_irq+0x158/0x16d
 [<ffffffff8105683f>] ? get_parent_ip+0xe/0x3e
 [<ffffffff81003f71>] do_IRQ+0x58/0xda
 [<ffffffff813ba1ea>] common_interrupt+0x6a/0x6a
 <EOI> 

 [<ffffffff813ba390>] ? retint_kernel+0x20/0x30
 [<ffffffff810d7569>] ? __zone_watermark_ok+0x17/0x85
 [<ffffffff810d839e>] zone_watermark_ok+0x1a/0x1c
 [<ffffffff810ef3d7>] compact_zone+0x215/0x4b2
 [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
 [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
 [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
 [<ffffffff810da3e1>] __alloc_pages_nodemask+0x409/0x799
 [<ffffffff810fdcba>] ? anon_vma_prepare+0x2b/0x12c
 [<ffffffff811163bb>] do_huge_pmd_anonymous_page+0x13c/0x255
 [<ffffffff810f65bd>] handle_mm_fault+0x112/0x808
 [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
 [<ffffffff8105ca95>] ? set_next_entity+0x3a/0x63
 [<ffffffff811e952e>] ? debug_smp_processor_id+0x17/0x19
 [<ffffffff810017c3>] ? __switch_to+0x33f/0x49c
 [<ffffffff813b9028>] ? _raw_spin_unlock_irq+0x14/0x27
 [<ffffffff811e952e>] ? debug_smp_processor_id+0x17/0x19
 [<ffffffff813b66d1>] ? __schedule+0x2d9/0x451
 [<ffffffff810f9997>] ? SyS_mmap_pgoff+0x183/0x1cf
 [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
 [<ffffffff813bb162>] page_fault+0x22/0x30
Code: 7a 01 e8 cd ff ff ff 5d c3 48 69 ff c7 10 00 00 55 48 89 e5 e8 c7 ff ff ff 5d c3 55 48 8d 3c bf 48 89 e5 e8 b8 ff ff ff 5d c3 55 <48> 89 e5 41 56 41 55 41 54 41 89 fc bf 01 00 00 00 53 e8 f7 6f 
NMI backtrace for cpu 1
CPU: 1 PID: 7037 Comm: urpmq Not tainted 3.18.0-rc3+ #46
Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
task: ffff88000387de80 ti: ffff88007dddc000 task.ti: ffff88007dddc000
RIP: 0010:[<ffffffff810d75af>]  [<ffffffff810d75af>] __zone_watermark_ok+0x5d/0x85
RSP: 0000:ffff88007dddfa10  EFLAGS: 00000202
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000007
RDX: 0000000000000f00 RSI: 0000000000000009 RDI: ffffffff8168be40
RBP: ffff88007dddfa18 R08: 0000000000000000 R09: 0000000000000701
R10: 0000000000000071 R11: ffffffff8168be40 R12: 00000000000bf800
R13: 00000000000bf600 R14: ffff88007dddfab8 R15: 0000160000000000
FS:  00007f20be79b700(0000) GS:ffff8800bf300000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f143453c000 CR3: 0000000012f19000 CR4: 00000000000007e0
Stack:
 ffffffff8168be40 ffff88007dddfa28 ffffffff810d839e ffff88007dddfa98
 ffffffff810ef3d7 ffff8800bf312e80 ffffea0002fd0000 0000000000000020
 ffff88000387de80 0000000000000004 ffff88007dddfac8 0000000000000000
Call Trace:
 [<ffffffff810d839e>] zone_watermark_ok+0x1a/0x1c
 [<ffffffff810ef3d7>] compact_zone+0x215/0x4b2
 [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
 [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
 [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
 [<ffffffff810da5c8>] __alloc_pages_nodemask+0x5f0/0x799
 [<ffffffff811163bb>] do_huge_pmd_anonymous_page+0x13c/0x255
 [<ffffffff810f65bd>] handle_mm_fault+0x112/0x808
 [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
 [<ffffffff8111af2b>] ? new_sync_read+0x74/0x98
 [<ffffffff8111b2c3>] ? fsnotify_access+0x5a/0x63
 [<ffffffff8111b602>] ? vfs_read+0xac/0xd2
 [<ffffffff8111b1f9>] ? fdput_pos.isra.13+0x29/0x30
 [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
 [<ffffffff813bb162>] page_fault+0x22/0x30
Code: c2 41 80 e0 10 74 11 4c 89 d0 41 b8 04 00 00 00 48 99 49 f7 f8 49 29 c2 4c 89 d2 31 c0 31 c9 48 03 54 df 18 49 39 d1 7e 27 39 ce <76> 21 48 6b d1 58 49 d1 fa 48 8b 94 17 28 01 00 00 48 d3 e2 48 
SysRq : Show backtrace of all active CPUs
sending NMI to all CPUs:
NMI backtrace for cpu 0
CPU: 0 PID: 7037 Comm: urpmq Not tainted 3.18.0-rc3+ #46
Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
task: ffff88000387de80 ti: ffff88007dddc000 task.ti: ffff88007dddc000
RIP: 0010:[<ffffffff811df82b>]  [<ffffffff811df82b>] __const_udelay+0x15/0x29
RSP: 0000:ffff8800bf203b68  EFLAGS: 00000006
RAX: 0000000001062560 RBX: 0000000000002710 RCX: 0000000000000007
RDX: 00000000001859ed RSI: 0000000000000c00 RDI: 0000000000418958
RBP: ffff8800bf203b68 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000046 R11: 0000000000000046 R12: 0000000000000008
R13: 000000000000006c R14: 0000000000000001 R15: ffffffff81668f90
FS:  00007f20be79b700(0000) GS:ffff8800bf200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007faaeda2a000 CR3: 0000000012f19000 CR4: 00000000000007f0
Stack:
 ffff8800bf203b88 ffffffff81025de1 0000000080010002 ffffffff816692b0
 ffff8800bf203b98 ffffffff8126bc4f ffff8800bf203bc8 ffffffff8126c186
 ffff88003780c000 0000000000000001 0000000000000026 ffff88003780c001
Call Trace:
 <IRQ> 

 [<ffffffff81025de1>] arch_trigger_all_cpu_backtrace+0xa8/0xd2
 [<ffffffff8126bc4f>] sysrq_handle_showallcpus+0xe/0x10
 [<ffffffff8126c186>] __handle_sysrq+0x94/0x126
 [<ffffffff8126c329>] sysrq_filter+0xee/0x287
 [<ffffffff812c0fd8>] input_to_handler+0x5e/0xcb
 [<ffffffff812c1de2>] input_pass_values.part.3+0x76/0x134
 [<ffffffff812c3eae>] input_handle_event+0x457/0x46d
 [<ffffffff812c3f19>] input_event+0x55/0x6f
 [<ffffffff812c6fb5>] input_sync+0xf/0x11
 [<ffffffff812c7f47>] atkbd_interrupt+0x4d5/0x595
 [<ffffffff812bf2c3>] serio_interrupt+0x43/0x7d
 [<ffffffff812bfa2e>] i8042_interrupt+0x292/0x2a8
 [<ffffffff8108b64b>] ? tick_sched_do_timer+0x33/0x33
 [<ffffffff810729a6>] handle_irq_event_percpu+0x44/0x19f
 [<ffffffff81072b3d>] handle_irq_event+0x3c/0x5c
 [<ffffffff81025e49>] ? apic_eoi+0x18/0x1a
 [<ffffffff810752b2>] handle_edge_irq+0x95/0xae
 [<ffffffff81004679>] handle_irq+0x158/0x16d
 [<ffffffff8105683f>] ? get_parent_ip+0xe/0x3e
 [<ffffffff81003f71>] do_IRQ+0x58/0xda
 [<ffffffff813ba1ea>] common_interrupt+0x6a/0x6a
 <EOI> 

 [<ffffffff810ef3d7>] ? compact_zone+0x215/0x4b2
 [<ffffffff810ef3d7>] ? compact_zone+0x215/0x4b2
 [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
 [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
 [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
 [<ffffffff810da5c8>] __alloc_pages_nodemask+0x5f0/0x799
 [<ffffffff811163bb>] do_huge_pmd_anonymous_page+0x13c/0x255
 [<ffffffff810f65bd>] handle_mm_fault+0x112/0x808
 [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
 [<ffffffff8111af2b>] ? new_sync_read+0x74/0x98
 [<ffffffff8111b2c3>] ? fsnotify_access+0x5a/0x63
 [<ffffffff8111b602>] ? vfs_read+0xac/0xd2
 [<ffffffff8111b1f9>] ? fdput_pos.isra.13+0x29/0x30
 [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
 [<ffffffff813bb162>] page_fault+0x22/0x30
Code: 48 ff c8 75 fb 48 ff c8 5d c3 55 48 89 e5 ff 15 a4 0c 48 00 5d c3 55 48 8d 04 bd 00 00 00 00 48 89 e5 65 48 8b 14 25 20 26 01 00 <48> 69 d2 fa 00 00 00 f7 e2 48 8d 7a 01 e8 cd ff ff ff 5d c3 48 
NMI backtrace for cpu 1
CPU: 1 PID: 7356 Comm: kwin Not tainted 3.18.0-rc3+ #46
Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
task: ffff8800b8c23f00 ti: ffff88009b1f8000 task.ti: ffff88009b1f8000
RIP: 0010:[<ffffffff810ef3d9>]  [<ffffffff810ef3d9>] compact_zone+0x217/0x4b2
RSP: 0000:ffff88009b1fba38  EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffffffff8168be40 RCX: 0000000000000008
RDX: 0000000000000800 RSI: 0000000000000009 RDI: ffffffff8168be40
RBP: ffff88009b1fba98 R08: 0000000000000000 R09: ffffffffffffff01
R10: 0000000000000038 R11: ffffffff8168be40 R12: 00000000000bf800
R13: 00000000000bf600 R14: ffff88009b1fbab8 R15: 0000160000000000
FS:  00007f445c0f17c0(0000) GS:ffff8800bf300000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000003a5b1d8 CR3: 00000000a6d15000 CR4: 00000000000007e0
Stack:
 ffff8800354be660 ffffea0002fd0000 0000000000000020 ffff8800b8c23f00
 0000000000000004 ffff88009b1fbac8 ffff8800a6d1a800 0000000000000000
 0000000000000009 ffff88009b1fbcec 0000000000000002 0000000000000000
Call Trace:
 [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
 [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
 [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
 [<ffffffff810da3e1>] __alloc_pages_nodemask+0x409/0x799
 [<ffffffff810fdcba>] ? anon_vma_prepare+0x2b/0x12c
 [<ffffffff811163bb>] do_huge_pmd_anonymous_page+0x13c/0x255
 [<ffffffff810f65bd>] handle_mm_fault+0x112/0x808
 [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
 [<ffffffff8105ca95>] ? set_next_entity+0x3a/0x63
 [<ffffffff811e952e>] ? debug_smp_processor_id+0x17/0x19
 [<ffffffff810017c3>] ? __switch_to+0x33f/0x49c
 [<ffffffff813b9028>] ? _raw_spin_unlock_irq+0x14/0x27
 [<ffffffff811e952e>] ? debug_smp_processor_id+0x17/0x19
 [<ffffffff813b66d1>] ? __schedule+0x2d9/0x451
 [<ffffffff810f9997>] ? SyS_mmap_pgoff+0x183/0x1cf
 [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
 [<ffffffff813bb162>] page_fault+0x22/0x30
Code: 76 48 83 fe ff 0f 84 b4 01 00 00 40 88 f1 ba 01 00 00 00 45 31 c0 d3 e2 48 89 df 31 c9 48 63 d2 48 03 53 08 e8 ad 8f fe ff 84 c0 <0f> 84 8e 01 00 00 41 8b 56 48 89 d0 83 f8 0a 0f 87 7f 01 00 00 

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: Early test: hangs in mm/compact.c w. Linus's 12d7aacab56e9ef185c
  2014-11-08 13:11           ` P. Christeas
@ 2014-11-08 22:18             ` Vlastimil Babka
  2014-11-09  8:27               ` Pavel Machek
                                 ` (2 more replies)
  0 siblings, 3 replies; 20+ messages in thread
From: Vlastimil Babka @ 2014-11-08 22:18 UTC (permalink / raw)
  To: P. Christeas
  Cc: linux-mm, Joonsoo Kim, lkml, David Rientjes, Norbert Preining,
	Markus Trippelsdorf, Pavel Machek

On 11/08/2014 02:11 PM, P. Christeas wrote:
> On Thursday 06 November 2014, Vlastimil Babka wrote:
>>> On Wednesday 05 November 2014, Vlastimil Babka wrote:
>>>> Can you please try the following patch?
>>>> -			compaction_defer_reset(zone, order, false);
>> Oh and did I ask in this thread for /proc/zoneinfo yet? :)
> 
> Using that same kernel[1], got again into a race, gathered a few more data.
> 
> This time, I had 1x "urpmq" process [2] hung at 100% CPU , when "kwin" got 
> apparently blocked (100% CPU, too) trying to resize a GUI window. I suppose 
> the resizing operation would mean heavy memory alloc/free.
> 
> The rest of the system was responsive, I could easily get a console, login, 
> gather the files.. Then, I have *killed* -9 the "urpmq" process, which solved 
> the race and my system is still alive! "kwin" is still running, returned to 
> regular CPU load.
> 
> Attached is traces from SysRq+l (pressed a few times, wanted to "snapshot" the 
> stack) and /proc/zoneinfo + /proc/vmstat
> 
> Bisection is not yet meaningful, IMHO, because I cannot be sure that "good" 
> points are really free from this issue. I'd estimate that each test would take 
> +3days, unless I really find a deterministic way to reproduce the issue .

Hi,

I think I finally found the cause by staring into the code... CCing
people from all 4 separate threads I know about this issue.
The problem with finding the cause was that the first report I got from
Markus was about isolate_freepages_block() overhead, and later Norbert
reported that reverting a patch for isolate_freepages* helped. But the
problem seems to be that although the loop in isolate_migratepages exits
because the scanners almost meet (they are within same pageblock), they
don't truly meet, therefore compact_finished() decides to continue, but
isolate_migratepages() exits immediately... boom! But indeed e14c720efdd7
made this situation possible, as free scaner pfn can now point to a
middle of pageblock.

So I hope the attached patch will fix the soft-lockup issues in
compact_zone. Please apply on 3.18-rc3 or later without any other reverts,
and test. It probably won't help Markus and his isolate_freepages_block()
overhead though...

Thanks,
Vlastimil

------8<------

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: Early test: hangs in mm/compact.c w. Linus's 12d7aacab56e9ef185c
@ 2014-11-09  4:47 Hillf Danton
  2014-11-09  8:22 ` P. Christeas
  0 siblings, 1 reply; 20+ messages in thread
From: Hillf Danton @ 2014-11-09  4:47 UTC (permalink / raw)
  To: 'P. Christeas'
  Cc: 'Vlastimil Babka', linux-mm, 'Joonsoo Kim',
	linux-kernel, Hillf Danton

> > Can you please try the following patch?
> > --- a/mm/compaction.c
> > +++ b/mm/compaction.c
> > @@ -1325,13 +1325,6 @@ unsigned long try_to_compact_pages(struct zonelist
> > -			compaction_defer_reset(zone, order, false);
> 
> NACK :(
> 
> I just got again into a state that some task was spinning out of control, and
> blocking the rest of the desktop.
> 
Would you please try the diff(against 3.18-rc3) if no other progress?

--- a/mm/compaction.c	Sun Nov  9 12:02:59 2014
+++ b/mm/compaction.c	Sun Nov  9 12:07:30 2014
@@ -1070,12 +1070,12 @@ static int compact_finished(struct zone 
 	if (cc->order == -1)
 		return COMPACT_CONTINUE;
 
-	/* Compaction run is not finished if the watermark is not met */
+	/* Compaction run is skipped if the watermark is not met */
 	watermark = low_wmark_pages(zone);
 	watermark += (1 << cc->order);
 
 	if (!zone_watermark_ok(zone, cc->order, watermark, 0, 0))
-		return COMPACT_CONTINUE;
+		return COMPACT_SKIPPED;
 
 	/* Direct compactor: Is a suitable page free? */
 	for (order = cc->order; order < MAX_ORDER; order++) {
--

> You will see me trying a few things, apparently the first OOM managed to
> unblock something, but a few seconds later the system "stepped" on some other
> blocking task.
> 
> See attached log, it may only give you some hint; the problem could well be in
> some other part of the kernel.
> 
> In the meanwhile, I'm pulling linus/master ...
> 
> SysRq : Show backtrace of all active CPUs
> sending NMI to all CPUs:
> NMI backtrace for cpu 1
> CPU: 1 PID: 13544 Comm: python Not tainted 3.18.0-rc3+ #46
> Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
> task: ffff88000c78ee40 ti: ffff88000e5f8000 task.ti: ffff88000e5f8000
> RIP: 0010:[<ffffffff811df888>]  [<ffffffff811df888>] delay_tsc+0x28/0xa2
> RSP: 0000:ffff8800bf303b28  EFLAGS: 00000002
> RAX: 000000006bd322e8 RBX: 0000000000002710 RCX: 0000000000000007
> RDX: 000000000000021d RSI: ffffffff8151623e RDI: ffffffff8152fea5
> RBP: ffff8800bf303b48 R08: 0000000000000400 R09: 00000000ffffffff
> R10: 0000000000000046 R11: 0000000000000046 R12: 0000000000185ac0
> R13: 0000000000000001 R14: 0000000000000001 R15: ffffffff81668f90
> FS:  00007f1570ed1700(0000) GS:ffff8800bf300000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: 00007f7cd966b000 CR3: 00000000740c9000 CR4: 00000000000007e0
> Stack:
>  0000000000002710 0000000000000003 000000000000006c 0000000000000001
>  ffff8800bf303b58 ffffffff811df814 ffff8800bf303b68 ffffffff811df83d
>  ffff8800bf303b88 ffffffff81025de1 0000000080010002 ffffffff816692b0
> Call Trace:
>  <IRQ>
> 
>  [<ffffffff811df814>] __delay+0xa/0xc
>  [<ffffffff811df83d>] __const_udelay+0x27/0x29
>  [<ffffffff81025de1>] arch_trigger_all_cpu_backtrace+0xa8/0xd2
>  [<ffffffff8126bc4f>] sysrq_handle_showallcpus+0xe/0x10
>  [<ffffffff8126c186>] __handle_sysrq+0x94/0x126
>  [<ffffffff8126c329>] sysrq_filter+0xee/0x287
>  [<ffffffff812c0fd8>] input_to_handler+0x5e/0xcb
>  [<ffffffff812c1de2>] input_pass_values.part.3+0x76/0x134
>  [<ffffffff812c3eae>] input_handle_event+0x457/0x46d
>  [<ffffffff812c3f19>] input_event+0x55/0x6f
>  [<ffffffff812c6fb5>] input_sync+0xf/0x11
>  [<ffffffff812c7f47>] atkbd_interrupt+0x4d5/0x595
>  [<ffffffff812bf2c3>] serio_interrupt+0x43/0x7d
>  [<ffffffff812bfa2e>] i8042_interrupt+0x292/0x2a8
>  [<ffffffff8108b64b>] ? tick_sched_do_timer+0x33/0x33
>  [<ffffffff810729a6>] handle_irq_event_percpu+0x44/0x19f
>  [<ffffffff81072b3d>] handle_irq_event+0x3c/0x5c
>  [<ffffffff81025e49>] ? apic_eoi+0x18/0x1a
>  [<ffffffff810752b2>] handle_edge_irq+0x95/0xae
>  [<ffffffff81004679>] handle_irq+0x158/0x16d
>  [<ffffffff8105683f>] ? get_parent_ip+0xe/0x3e
>  [<ffffffff81003f71>] do_IRQ+0x58/0xda
>  [<ffffffff813ba1ea>] common_interrupt+0x6a/0x6a
>  <EOI>
> 
>  [<ffffffff810ee503>] ? rcu_read_unlock_sched_notrace+0x17/0x17
>  [<ffffffff810ef46a>] ? compact_zone+0x2a8/0x4b2
>  [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
>  [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
>  [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
>  [<ffffffff810da3e1>] __alloc_pages_nodemask+0x409/0x799
>  [<ffffffff810fdd84>] ? anon_vma_prepare+0xf5/0x12c
>  [<ffffffff811163bb>] do_huge_pmd_anonymous_page+0x13c/0x255
>  [<ffffffff810faa65>] ? mmap_region+0x171/0x458
>  [<ffffffff810f65bd>] handle_mm_fault+0x112/0x808
>  [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
>  [<ffffffff810fb004>] ? do_mmap_pgoff+0x2b8/0x306
>  [<ffffffff810e88dd>] ? vm_mmap_pgoff+0x82/0xaa
>  [<ffffffff810f9997>] ? SyS_mmap_pgoff+0x183/0x1cf
>  [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
>  [<ffffffff813bb162>] page_fault+0x22/0x30
> Code: ff 5d c3 55 48 89 e5 41 56 41 55 41 54 41 89 fc bf 01 00 00 00 53 e8 f7 6f e7 ff e8 9a 9c 00 00 41 89 c5 0f 1f 00 0f ae e8 0f 31 <89> c3 0f
> 1f 00 0f ae e8 0f 31 48 c1 e2 20 89 c0 48 09 c2 41 89
> NMI backtrace for cpu 0
> CPU: 0 PID: 13788 Comm: net_applet Not tainted 3.18.0-rc3+ #46
> Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
> task: ffff8800067a3720 ti: ffff88000e20c000 task.ti: ffff88000e20c000
> RIP: 0010:[<ffffffff810ef586>]  [<ffffffff810ef586>] compact_zone+0x3c4/0x4b2
> RSP: 0000:ffff88000e20fa18  EFLAGS: 00000202
> RAX: 00000000ffffffff RBX: ffffffff8168be40 RCX: 0000000000000008
> RDX: 0000000000000380 RSI: 0000000000000009 RDI: ffffffff8168be40
> RBP: ffff88000e20fa78 R08: 0000000000000000 R09: fffffffffffffef5
> R10: 0000000000000038 R11: ffffffff8168be40 R12: 00000000000bf800
> R13: 00000000000bf600 R14: ffff88000e20fa98 R15: 0000160000000000
> FS:  00007ff9cbe92700(0000) GS:ffff8800bf200000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: 00007ff3a52f0000 CR3: 000000000af17000 CR4: 00000000000007f0
> Stack:
>  ffff88000e20fa18 ffffea0002fd0000 0000000000000020 ffff8800067a3720
>  0000000000000004 ffff88000e20faa8 0000000000000000 0000000000000000
>  0000000000000009 ffff88000e20fccc 0000000000000002 0000000000000000
> Call Trace:
>  [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
>  [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
>  [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
>  [<ffffffff810da3e1>] __alloc_pages_nodemask+0x409/0x799
>  [<ffffffff810d458c>] ? unlock_page+0x1f/0x23
>  [<ffffffff81116903>] do_huge_pmd_wp_page+0x127/0x4eb
>  [<ffffffff810f65fc>] handle_mm_fault+0x151/0x808
>  [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
>  [<ffffffff8102ddf8>] ? do_page_fault+0xc/0xe
>  [<ffffffff813bb162>] ? page_fault+0x22/0x30
>  [<ffffffff811e0740>] ? __put_user_4+0x20/0x30
>  [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
>  [<ffffffff813bb162>] page_fault+0x22/0x30
> Code: 8b 7b 08 44 89 e6 ff 13 48 83 c3 10 48 83 3b 00 eb eb 41 83 7e 40 01 4d 8b 6e 38 19 c0 89 45 c0 4d 8d a5 00 02 00 00 83 65 c0 04 <49>
> 81 e4 00 fe ff ff e9 b2 fe ff ff 41 80 7e 44 00 74 09 41 83
> 
> SysRq : Changing Loglevel
> Loglevel set to 8
> 
> SysRq : Show backtrace of all active CPUs
> sending NMI to all CPUs:
> NMI backtrace for cpu 1
> CPU: 1 PID: 13544 Comm: python Not tainted 3.18.0-rc3+ #46
> Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
> task: ffff88000c78ee40 ti: ffff88000e5f8000 task.ti: ffff88000e5f8000
> RIP: 0010:[<ffffffff811df817>]  [<ffffffff811df817>] __const_udelay+0x1/0x29
> RSP: 0000:ffff8800bf303b68  EFLAGS: 00000006
> RAX: 0000000000000000 RBX: 0000000000002710 RCX: 0000000000000007
> RDX: 0000000080010003 RSI: 0000000000000c00 RDI: 0000000000418958
> RBP: ffff8800bf303b88 R08: 0000000000000000 R09: 0000000000000000
> R10: 0000000000000046 R11: 0000000000000046 R12: 0000000000000008
> R13: 000000000000006c R14: 0000000000000001 R15: ffffffff81668f90
> FS:  00007f1570ed1700(0000) GS:ffff8800bf300000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: 00007f7cd966b000 CR3: 00000000740c9000 CR4: 00000000000007e0
> Stack:
>  ffff8800bf303b88 ffffffff81025de1 0000000080010002 ffffffff816692b0
>  ffff8800bf303b98 ffffffff8126bc4f ffff8800bf303bc8 ffffffff8126c186
>  ffff88003781c200 0000000000000001 0000000000000026 ffff88003781c201
> Call Trace:
>  <IRQ>
> 
>  [<ffffffff81025de1>] ? arch_trigger_all_cpu_backtrace+0xa8/0xd2
>  [<ffffffff8126bc4f>] sysrq_handle_showallcpus+0xe/0x10
>  [<ffffffff8126c186>] __handle_sysrq+0x94/0x126
>  [<ffffffff8126c329>] sysrq_filter+0xee/0x287
>  [<ffffffff812c0fd8>] input_to_handler+0x5e/0xcb
>  [<ffffffff812c1de2>] input_pass_values.part.3+0x76/0x134
>  [<ffffffff812c3eae>] input_handle_event+0x457/0x46d
>  [<ffffffff812c3f19>] input_event+0x55/0x6f
>  [<ffffffff812c6fb5>] input_sync+0xf/0x11
>  [<ffffffff812c7f47>] atkbd_interrupt+0x4d5/0x595
>  [<ffffffff812bf2c3>] serio_interrupt+0x43/0x7d
>  [<ffffffff812bfa2e>] i8042_interrupt+0x292/0x2a8
>  [<ffffffff8108b64b>] ? tick_sched_do_timer+0x33/0x33
>  [<ffffffff810729a6>] handle_irq_event_percpu+0x44/0x19f
>  [<ffffffff81072b3d>] handle_irq_event+0x3c/0x5c
>  [<ffffffff81025e49>] ? apic_eoi+0x18/0x1a
>  [<ffffffff810752b2>] handle_edge_irq+0x95/0xae
>  [<ffffffff81004679>] handle_irq+0x158/0x16d
>  [<ffffffff8105683f>] ? get_parent_ip+0xe/0x3e
>  [<ffffffff81003f71>] do_IRQ+0x58/0xda
>  [<ffffffff813ba1ea>] common_interrupt+0x6a/0x6a
>  <EOI>
> 
>  [<ffffffff813b6b95>] ? preempt_schedule_irq+0x3c/0x59
>  [<ffffffff810d75b5>] ? __zone_watermark_ok+0x63/0x85
>  [<ffffffff810d839e>] zone_watermark_ok+0x1a/0x1c
>  [<ffffffff810ef3d7>] compact_zone+0x215/0x4b2
>  [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
>  [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
>  [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
>  [<ffffffff810da3e1>] __alloc_pages_nodemask+0x409/0x799
>  [<ffffffff810fdd84>] ? anon_vma_prepare+0xf5/0x12c
>  [<ffffffff811163bb>] do_huge_pmd_anonymous_page+0x13c/0x255
>  [<ffffffff810faa65>] ? mmap_region+0x171/0x458
>  [<ffffffff810f65bd>] handle_mm_fault+0x112/0x808
>  [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
>  [<ffffffff810fb004>] ? do_mmap_pgoff+0x2b8/0x306
>  [<ffffffff810e88dd>] ? vm_mmap_pgoff+0x82/0xaa
>  [<ffffffff810f9997>] ? SyS_mmap_pgoff+0x183/0x1cf
>  [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
>  [<ffffffff813bb162>] page_fault+0x22/0x30
> Code: eb 02 66 90 eb 0e 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 48 ff c8 75 fb 48 ff c8 5d c3 55 48 89 e5 ff 15 a4 0c 48 00 5d c3 55 <48> 8d
> 04 bd 00 00 00 00 48 89 e5 65 48 8b 14 25 20 26 01 00 48
> NMI backtrace for cpu 0
> CPU: 0 PID: 11733 Comm: kwin Not tainted 3.18.0-rc3+ #46
> Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
> task: ffff8800055e7620 ti: ffff88009dc78000 task.ti: ffff88009dc78000
> RIP: 0010:[<ffffffff810ee567>]  [<ffffffff810ee567>] acct_isolated+0x64/0x6b
> RSP: 0000:ffff88009dc7ba08  EFLAGS: 00000246
> RAX: ffff88009dc7bac8 RBX: ffffffff8168be40 RCX: ffff88009dc7bac8
> RDX: 0000000000000380 RSI: ffff88009dc7bab8 RDI: ffffffff8168be40
> RBP: ffff88009dc7ba28 R08: 0000000000000000 R09: ffffffffffffff01
> R10: 0000000000000038 R11: ffffffff8168be40 R12: 00000000000bf800
> R13: 00000000000bf600 R14: ffff88009dc7bab8 R15: 0000160000000000
> FS:  00007facfaf397c0(0000) GS:ffff8800bf200000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: 00007f9d5781d910 CR3: 0000000025c25000 CR4: 00000000000007f0
> Stack:
>  0000000000000000 0000000000000000 ffff88009dc7ba28 ffffffff8168be40
>  ffff88009dc7ba98 ffffffff810ef46a ffff88008035e660 ffffea0002fd0000
>  0000000000000020 ffff8800055e7620 0000000000000004 ffff88009dc7bac8
> Call Trace:
>  [<ffffffff810ef46a>] compact_zone+0x2a8/0x4b2
>  [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
>  [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
>  [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
>  [<ffffffff810da3e1>] __alloc_pages_nodemask+0x409/0x799
>  [<ffffffff810fdcba>] ? anon_vma_prepare+0x2b/0x12c
>  [<ffffffff811163bb>] do_huge_pmd_anonymous_page+0x13c/0x255
>  [<ffffffff810f65bd>] handle_mm_fault+0x112/0x808
>  [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
>  [<ffffffff810fb004>] ? do_mmap_pgoff+0x2b8/0x306
>  [<ffffffff810e88dd>] ? vm_mmap_pgoff+0x82/0xaa
>  [<ffffffff813b9028>] ? _raw_spin_unlock_irq+0x14/0x27
>  [<ffffffff810f9997>] ? SyS_mmap_pgoff+0x183/0x1cf
>  [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
>  [<ffffffff813bb162>] page_fault+0x22/0x30
> Code: f7 d2 83 e2 01 ff 44 95 e8 eb dd 8b 55 e8 be 16 00 00 00 48 89 df e8 7d b3 ff ff 8b 55 ec be 17 00 00 00 48 89 df e8 6d b3 ff ff <48> 83
> c4 18 5b 5d c3 83 7a 40 00 55 48 89 e5 41 54 49 89 d4 53
> INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 91.545 msecs
> 
> SysRq : Show backtrace of all active CPUs
> sending NMI to all CPUs:
> 
> NMI backtrace for cpu 0
> CPU: 0 PID: 11733 Comm: kwin Not tainted 3.18.0-rc3+ #46
> Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
> task: ffff8800055e7620 ti: ffff88009dc78000 task.ti: ffff88009dc78000
> RIP: 0010:[<ffffffff810ef586>]  [<ffffffff810ef586>] compact_zone+0x3c4/0x4b2
> RSP: 0000:ffff88009dc7ba38  EFLAGS: 00000202
> RAX: 00000000ffffffff RBX: ffffffff8168be40 RCX: 0000000000000008
> RDX: 0000000000000380 RSI: 0000000000000009 RDI: ffffffff8168be40
> RBP: ffff88009dc7ba98 R08: 0000000000000000 R09: ffffffffffffff01
> R10: 0000000000000038 R11: ffffffff8168be40 R12: 00000000000bf800
> R13: 00000000000bf600 R14: ffff88009dc7bab8 R15: 0000160000000000
> FS:  00007facfaf397c0(0000) GS:ffff8800bf200000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: 00007fe8952e5000 CR3: 0000000025c25000 CR4: 00000000000007f0
> Stack:
>  ffff88008035e660 ffffea0002fd0000 0000000000000020 ffff8800055e7620
>  0000000000000004 ffff88009dc7bac8 0000000000000234 0000000000000000
>  0000000000000009 ffff88009dc7bcec 0000000000000002 0000000000000000
> Call Trace:
>  [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
>  [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
>  [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
>  [<ffffffff810da3e1>] __alloc_pages_nodemask+0x409/0x799
>  [<ffffffff810fdcba>] ? anon_vma_prepare+0x2b/0x12c
>  [<ffffffff811163bb>] do_huge_pmd_anonymous_page+0x13c/0x255
>  [<ffffffff810f65bd>] handle_mm_fault+0x112/0x808
>  [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
>  [<ffffffff810fb004>] ? do_mmap_pgoff+0x2b8/0x306
>  [<ffffffff810e88dd>] ? vm_mmap_pgoff+0x82/0xaa
>  [<ffffffff813b9028>] ? _raw_spin_unlock_irq+0x14/0x27
>  [<ffffffff810f9997>] ? SyS_mmap_pgoff+0x183/0x1cf
>  [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
>  [<ffffffff813bb162>] page_fault+0x22/0x30
> Code: 8b 7b 08 44 89 e6 ff 13 48 83 c3 10 48 83 3b 00 eb eb 41 83 7e 40 01 4d 8b 6e 38 19 c0 89 45 c0 4d 8d a5 00 02 00 00 83 65 c0 04 <49>
> 81 e4 00 fe ff ff e9 b2 fe ff ff 41 80 7e 44 00 74 09 41 83
> NMI backtrace for cpu 1
> CPU: 1 PID: 13544 Comm: python Not tainted 3.18.0-rc3+ #46
> Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
> task: ffff88000c78ee40 ti: ffff88000e5f8000 task.ti: ffff88000e5f8000
> RIP: 0010:[<ffffffff811df82b>]  [<ffffffff811df82b>] __const_udelay+0x15/0x29
> RSP: 0000:ffff8800bf303b68  EFLAGS: 00000006
> RAX: 0000000001062560 RBX: 0000000000002710 RCX: 0000000000000007
> RDX: 0000000000185ab3 RSI: 0000000000000c00 RDI: 0000000000418958
> RBP: ffff8800bf303b68 R08: 0000000000000000 R09: 0000000000000000
> R10: 0000000000000046 R11: 0000000000000046 R12: 0000000000000008
> R13: 000000000000006c R14: 0000000000000001 R15: ffffffff81668f90
> FS:  00007f1570ed1700(0000) GS:ffff8800bf300000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: 00007f38f594a000 CR3: 00000000740c9000 CR4: 00000000000007e0
> Stack:
>  ffff8800bf303b88 ffffffff81025de1 0000000080010002 ffffffff816692b0
>  ffff8800bf303b98 ffffffff8126bc4f ffff8800bf303bc8 ffffffff8126c186
>  ffff88003781c200 0000000000000001 0000000000000026 ffff88003781c201
> Call Trace:
>  <IRQ>
> 
>  [<ffffffff81025de1>] arch_trigger_all_cpu_backtrace+0xa8/0xd2
>  [<ffffffff8126bc4f>] sysrq_handle_showallcpus+0xe/0x10
>  [<ffffffff8126c186>] __handle_sysrq+0x94/0x126
>  [<ffffffff8126c329>] sysrq_filter+0xee/0x287
>  [<ffffffff812c0fd8>] input_to_handler+0x5e/0xcb
>  [<ffffffff812c1de2>] input_pass_values.part.3+0x76/0x134
>  [<ffffffff812c3eae>] input_handle_event+0x457/0x46d
>  [<ffffffff812c3f19>] input_event+0x55/0x6f
>  [<ffffffff812c6fb5>] input_sync+0xf/0x11
>  [<ffffffff812c7f47>] atkbd_interrupt+0x4d5/0x595
>  [<ffffffff812bf2c3>] serio_interrupt+0x43/0x7d
>  [<ffffffff812bfa2e>] i8042_interrupt+0x292/0x2a8
>  [<ffffffff8108b64b>] ? tick_sched_do_timer+0x33/0x33
>  [<ffffffff810729a6>] handle_irq_event_percpu+0x44/0x19f
>  [<ffffffff81072b3d>] handle_irq_event+0x3c/0x5c
>  [<ffffffff81025e49>] ? apic_eoi+0x18/0x1a
>  [<ffffffff810752b2>] handle_edge_irq+0x95/0xae
>  [<ffffffff81004679>] handle_irq+0x158/0x16d
>  [<ffffffff8105683f>] ? get_parent_ip+0xe/0x3e
>  [<ffffffff81003f71>] do_IRQ+0x58/0xda
>  [<ffffffff813ba1ea>] common_interrupt+0x6a/0x6a
>  <EOI>
> 
>  [<ffffffff810d75ad>] ? __zone_watermark_ok+0x5b/0x85
>  [<ffffffff810d839e>] zone_watermark_ok+0x1a/0x1c
>  [<ffffffff810ef3d7>] compact_zone+0x215/0x4b2
>  [<ffffffff810ef6c0>] compact_zone_order+0x4c/0x5f
>  [<ffffffff810ef87f>] try_to_compact_pages+0xc4/0x1d6
>  [<ffffffff813b3118>] __alloc_pages_direct_compact+0x61/0x1bf
>  [<ffffffff810da3e1>] __alloc_pages_nodemask+0x409/0x799
>  [<ffffffff810fdd84>] ? anon_vma_prepare+0xf5/0x12c
>  [<ffffffff811163bb>] do_huge_pmd_anonymous_page+0x13c/0x255
>  [<ffffffff810faa65>] ? mmap_region+0x171/0x458
>  [<ffffffff810f65bd>] handle_mm_fault+0x112/0x808
>  [<ffffffff8102dced>] __do_page_fault+0x27a/0x358
>  [<ffffffff810fb004>] ? do_mmap_pgoff+0x2b8/0x306
>  [<ffffffff810e88dd>] ? vm_mmap_pgoff+0x82/0xaa
>  [<ffffffff810f9997>] ? SyS_mmap_pgoff+0x183/0x1cf
>  [<ffffffff8102ddf8>] do_page_fault+0xc/0xe
>  [<ffffffff813bb162>] page_fault+0x22/0x30
> Code: 48 ff c8 75 fb 48 ff c8 5d c3 55 48 89 e5 ff 15 a4 0c 48 00 5d c3 55 48 8d 04 bd 00 00 00 00 48 89 e5 65 48 8b 14 25 20 26 01 00 <48> 69
> d2 fa 00 00 00 f7 e2 48 8d 7a 01 e8 cd ff ff ff 5d c3 48
> INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 163.331 msecs
> 
> SysRq : Manual OOM execution
> Purging GPU memory, 4186 bytes freed, 4243456 bytes still pinned.
> 
> SysRq : HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k)
> show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q)
> unraw(r) sync(s) show-task-states(t) unmount(u) force-fb(V) show-blocked-tasks(w) dump-ftrace-buffer(z)
> SysRq : Manual OOM execution
> 
> Purging GPU memory, 0 bytes freed, 4243456 bytes still pinned.
> 
> kworker/1:0 invoked oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=0
> kworker/1:0 cpuset=/ mems_allowed=0
> CPU: 1 PID: 13984 Comm: kworker/1:0 Not tainted 3.18.0-rc3+ #46
> Hardware name: Acer            TravelMate 5720                /Columbia                       , BIOS V1.34           04/15/2008
> Workqueue: events moom_callback
>  ffff8800067a1600 ffff88000dadfc58 ffffffff813b4868 0000000000000001
>  ffff8800067a0fc0 ffff88000dadfce8 ffffffff813b2e37 000000000023222e
>  ffffffff8163e380 000000000023222e 0000000000000206 ffff88000dadfca8
> Call Trace:
>  [<ffffffff813b4868>] dump_stack+0x4f/0x7c
>  [<ffffffff813b2e37>] dump_header.isra.11+0x71/0x1d7
>  [<ffffffff813b8ffe>] ? _raw_spin_unlock_irqrestore+0x1b/0x31
>  [<ffffffff811db724>] ? ___ratelimit+0xb9/0xc7
>  [<ffffffff810d6ca8>] oom_kill_process+0x60/0x310
>  [<ffffffff810d6b5a>] ? oom_badness+0xb1/0xfb
>  [<ffffffff810d733d>] out_of_memory+0x282/0x29b
>  [<ffffffff8126bd74>] moom_callback+0x1f/0x21
>  [<ffffffff8104b077>] process_one_work+0x156/0x29c
>  [<ffffffff8104b978>] worker_thread+0x1eb/0x2c2
>  [<ffffffff8104b78d>] ? cancel_delayed_work_sync+0x10/0x10
>  [<ffffffff8104f502>] kthread+0xbb/0xc3
>  [<ffffffff8104f447>] ? __kthread_parkme+0x5c/0x5c
>  [<ffffffff813b96ac>] ret_from_fork+0x7c/0xb0
>  [<ffffffff8104f447>] ? __kthread_parkme+0x5c/0x5c
> Mem-Info:
> DMA per-cpu:
> CPU    0: hi:    0, btch:   1 usd:   0
> CPU    1: hi:    0, btch:   1 usd:   0
> DMA32 per-cpu:
> CPU    0: hi:  186, btch:  31 usd:  97
> CPU    1: hi:  186, btch:  31 usd: 183
> active_anon:286387 inactive_anon:87475 isolated_anon:0
>  active_file:174069 inactive_file:43465 isolated_file:0
>  unevictable:0 dirty:30 writeback:0 unstable:0
>  free:80502 slab_reclaimable:58629 slab_unreclaimable:10926
>  mapped:98606 shmem:48888 pagetables:17014 bounce:0
>  free_cma:0
> DMA free:12216kB min:232kB low:288kB high:348kB active_anon:540kB inactive_anon:804kB active_file:1056kB inactive_file:760kB
> unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15992kB managed:15908kB mlocked:0kB dirty:0kB writeback:0kB
> mapped:892kB shmem:372kB slab_reclaimable:148kB slab_unreclaimable:88kB kernel_stack:16kB pagetables:92kB unstable:0kB
> bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
> lowmem_reserve[]: 0 2984 2984 2984
> DMA32 free:309792kB min:44820kB low:56024kB high:67228kB active_anon:1145008kB inactive_anon:349096kB active_file:695220kB
> inactive_file:173100kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3119936kB managed:3057340kB mlocked:0kB
> dirty:120kB writeback:0kB mapped:393532kB shmem:195180kB slab_reclaimable:234368kB slab_unreclaimable:43616kB
> kernel_stack:7360kB pagetables:67964kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0
> all_unreclaimable? no
> lowmem_reserve[]: 0 0 0 0
> DMA: 10*4kB (UEM) 6*8kB (UM) 2*16kB (EM) 8*32kB (UEM) 3*64kB (UEM) 1*128kB (E) 1*256kB (M) 2*512kB (EM) 2*1024kB (UE)
> 2*2048kB (ER) 1*4096kB (M) = 12216kB
> DMA32: 3084*4kB (UEMR) 8564*8kB (UEMR) 6169*16kB (UEMR) 2174*32kB (UEMR) 402*64kB (UEM) 139*128kB (UEMR) 49*256kB
> (UEMR) 7*512kB (UEM) 1*1024kB (M) 0*2048kB 0*4096kB = 309792kB
> Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
> 266800 total pagecache pages
> 376 pages in swap cache
> Swap cache stats: add 12077, delete 11701, find 857/1253
> Free swap  = 6092348kB
> Total swap = 6136792kB
> 783982 pages RAM
> 0 pages HighMem/MovableOnly
> 15649 pages reserved
> 0 pages hwpoisoned
> [ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
> [  492]     0   492    10595     2553      23       19             0 systemd-journal
> [  544]     0   544     8460      747      20       22         -1000 systemd-udevd
> [  738]     0   738     4743      696      14       19             0 alsactl
> [  761]     0   761     5848      855      16       28             0 bluetoothd
> [  762]     0   762     8185      723      21       69             0 abrtd
> [  766]     0   766     4827      581      15       30             0 irqbalance
> [  788]     0   788     3814      709      12       97             0 smartd
> [  789]     0   789     1827      365       9      811             0 hddtemp
> [  795]     0   795     5423      624      16       11             0 systemd-logind
> [  796]     0   796    48410     1889      31       74             0 udisks-daemon
> [  797]   499   797     3638      754      12       10          -900 dbus-daemon
> [  799]     0   799     1071      382       8       13             0 acpid
> [  811]     0   811    26132     4425      53      472             0 cupsd
> [  813]     0   813    56404     1572      41      647             0 upowerd
> [  815]     0   815     6159      545      17       15             0 kdm
> [  831]     0   831   173245      498      34       30             0 nscd
> [  904]     0   904    11431        4      26       78             0 udisks-daemon
> [ 1005]   497  1005    91379     3112      41     1379             0 polkitd
> [ 1113]     0  1113     1586      287       8       22             0 ifplugd
> [ 1212]     0  1212     8029      408      18      126             0 wpa_supplicant
> [ 1214]     0  1214     1586      288       9       22             0 ifplugd
> [ 1324]     0  1324    28065     1115      26      387             0 cf-execd
> [ 1333]     0  1333    12113      541      24     1150             0 cf-serverd
> [ 1479]     0  1479     3117       98      10     1610             0 dhclient
> [ 1579]     0  1579    87422     2054     107      925             0 libvirtd
> [ 1584]     0  1584    10579      942      25      135         -1000 sshd
> [ 1592]     0  1592     2846      401      11       42             0 xinetd
> [ 1598]   491  1598     6250      412      14       70             0 rpcbind
> [ 1602]     0  1602     3851        0      12       38             0 rpc.idmapd
> [ 1611]   488  1611     8360      544      21      141             0 rpc.statd
> [ 1614]    80  1614     7989     1084      20      167             0 pgpool
> [ 1615]    80  1615     8029      475      19      181             0 pgpool
> [ 1616]    80  1616     8029      475      19      181             0 pgpool
> [ 1617]    80  1617     8029      475      20      181             0 pgpool
> [ 1618]    80  1618     8029      475      19      181             0 pgpool
> [ 1619]    80  1619     7989      327      18      172             0 pgpool
> [ 1620]    80  1620     7989       11      18      163             0 pgpool
> [ 1622]    80  1622    47318     4356      34      123         -1000 postgres
> [ 1636]     0  1636     2941      405      12      120             0 crond
> [ 1643]   487  1643     5374      578      15       33             0 ntpd
> [ 1647]     0  1647     5408     2340      16       23             0 preload
> [ 1680]    80  1680    47350     6430      66      106         -1000 postgres
> [ 1681]    80  1681    47318     9645      90      109         -1000 postgres
> [ 1682]    80  1682    47318      785      25      146         -1000 postgres
> [ 1683]    80  1683    47447     1683      39      148         -1000 postgres
> [ 1684]    80  1684    11074      609      23      131         -1000 postgres
> [ 1685]    80  1685    11146      954      23      118         -1000 postgres
> [ 1803]   493  1803    70606     1317      41      377             0 colord
> [ 1812]     0  1812     9046      792      24      234             0 systemd
> [ 1815]     0  1815    14697      157      31      453             0 (sd-pam)
> [ 2445]     0  2445     4228       12      12       38             0 gpg-agent
> [ 5893]     0  5893    12071     1997      29      236             0 cf-monitord
> [ 8691]   500  8691     9233     1169      22       19             0 systemd
> [ 8692]   500  8692    14769      177      31      487             0 (sd-pam)
> [ 8759]   500  8759     3170        7       9       63             0 ssh-agent
> [ 8784]   500  8784     3459      393      14       34             0 gpg-agent
> [ 9260]     0  9260     1975      400       9        0             0 agetty
> [ 9262]     0  9262     1975      393       9        0             0 agetty
> [ 9367]   500  9367     3098      684      11        0             0 gam_server
> [ 9976]     0  9976    88063     2116      38        0             0 udisksd
> [11283]     0 11283    47144    13442      89        0             0 X
> [11297]     0 11297    19771     1253      44        6             0 kdm
> [11507]   500 11507     3191      735      12        0             0 startkde
> [11602]   500 11602     3987      460      13        0             0 dbus-launch
> [11603]   500 11603     3907     1011      13        0             0 dbus-daemon
> [11634]   500 11634    34515     1861      67        0             0 s2u
> [11690]   500 11690     1028       20       6        0             0 start_kdeinit
> [11691]   500 11691    95918     8995     150        0             0 kdeinit4
> [11692]   500 11692    97137     6446     140        0             0 klauncher
> [11694]   500 11694   260253    17122     254        0             0 kded4
> [11697]   500 11697   118836     9730     181        0             0 kglobalaccel
> [11700]   500 11700     5856      676      17        0             0 obexd
> [11707]   500 11707    72764     6751     125        0             0 bluedevil-monol
> [11718]   500 11718   228901     9415     160        0             0 kactivitymanage
> [11721]   500 11721     1062      162       8        0             0 kwrapper4
> [11724]   500 11724   139507     9588     188        0             0 ksmserver
> [11733]   500 11733   683159    22362     242        0             0 kwin
> [11737]   500 11737   821255    39358     343        0             0 plasma-desktop
> [11740]   500 11740     2491      606      10        0             0 ksysguardd
> [11742]   500 11742   185670    14918     241        0             0 krunner
> [11836]   500 11836    66336     1673      33        0             0 mission-control
> [11900]   500 11900    71627     7762     130        0             0 kuiserver
> [11902]   500 11902    39315     2928      38        0             0 akonadi_control
> [11904]   500 11904     3190      688      12        0             0 akonadiserver
> [11905]   500 11905   364072     7154     110        0             0 akonadiserver
> [11907]    80 11907    48978    37329      98       90         -1000 postgres
> [11916]   500 11916    71126     7723     127        0             0 ksyndaemon
> [11924]   500 11924    94819     2524      88        0             0 pulseaudio
> [11925]   492 11925    40153      566      15        0             0 rtkit-daemon
> [11938]    80 11938    47549     2575      34       95         -1000 postgres
> [11939]    80 11939    47549     2619      34       95         -1000 postgres
> [11940]    80 11940    47549     2619      34       95         -1000 postgres
> [11943]   500 11943    80114     9317     112        0             0 akonadi_agent_l
> [11944]   500 11944    80111     9358     108        0             0 akonadi_agent_l
> [11945]   500 11945   149309    16150     260        0             0 akonadi_archive
> [11948]   500 11948    80738    10447     147        0             0 akonadi_birthda
> [11949]   500 11949    80046     9859     108        0             0 akonadi_agent_l
> [11950]   500 11950   149456    15918     259        0             0 akonadi_foldera
> [11952]   500 11952   119534    12515     160        0             0 akonadi_imap_re
> [11953]   500 11953   138083    12936     163        0             0 akonadi_imap_re
> [11954]   500 11954   137951    12152     158        0             0 akonadi_imap_re
> [11955]   500 11955    74613     8962     134        0             0 akonadi_localbo
> [11956]   500 11956    20123      790      42        0             0 gconf-helper
> [11958]   500 11958    11393     1256      27        0             0 gconfd-2
> [11959]   500 11959    80110     9345     111        0             0 akonadi_agent_l
> [11960]   500 11960    80110     9316     112        0             0 akonadi_agent_l
> [11961]   500 11961    88890    10322     156        0             0 akonadi_maildis
> [11962]   500 11962   150003    16953     260        0             0 akonadi_mailfil
> [11963]   500 11963    75882     9862     142        0             0 akonadi_migrati
> [11964]   500 11964    80530    10776     142        0             0 akonadi_mixedma
> [11965]   500 11965    82835    12607     151        0             0 akonadi_mixedma
> [11966]   500 11966   144439    13711     197        0             0 konsole
> [11973]   500 11973    76667    10080     139        0             0 akonadi_nepomuk
> [11974]   500 11974   100405    11536     184        0             0 akonadi_newmail
> [11975]   500 11975    79291     9474     108        0             0 akonadi_agent_l
> [11976]   500 11976    76551     9175     140        0             0 akonadi_pop3_re
> [11977]   500 11977    76535     9251     139        0             0 akonadi_pop3_re
> [11978]   500 11978    77748    10430     141        0             0 akonadi_pop3_re
> [11979]   500 11979   138554    13963     242        0             0 akonadi_sendlat
> [11981]   500 11981    78493     9240     109        0             0 akonadi_agent_l
> [11982]   500 11982    80012     9799     113        0             0 akonadi_agent_l
> [11983]   500 11983   177548    11542     195        0             0 kmix
> [11984]   500 11984   180599    14445     204        0             0 dolphin
> [11993]   500 11993    74732     8349     133        0             0 akonaditray
> [11997]   500 11997   107450    11140     188        0             0 knotes
> [11999]   500 11999     3649     1186      12        0             0 bash
> [12011]   500 12011     3585     1095      12        0             0 bash
> [12025]   500 12025     3585     1076      12        0             0 bash
> [12047]   500 12047     3585     1116      11        0             0 bash
> [12186]   500 12186   116692    12696     175        0             0 kgpg
> [12293]   500 12293     3585     1117      12        0             0 bash
> [12317]   500 12317     3585     1113      13        0             0 bash
> [12356]   500 12356     3585     1102      12        0             0 bash
> [12405]   500 12405     3585     1086      13        0             0 bash
> [12506]   500 12506     3585     1073      11        0             0 bash
> [12645]   500 12645   122416    29665     208        0             0 choqok
> [12673]   500 12673     3585     1115      13        0             0 bash
> [12718]   500 12718     3585     1104      13        0             0 bash
> [12771]   500 12771     3585     1075      12        0             0 bash
> [12843]   500 12843     3585     1087      12        0             0 bash
> [13020]   500 13020   686067    27224     266        0             0 kate
> [13027]   500 13027   121357    10647     186        0             0 kwalletd
> [13029]    80 13029    47607     3959      41       92         -1000 postgres
> [13033]    80 13033    47549     2573      34       95         -1000 postgres
> [13052]   500 13052   138593     9489     158        0             0 knotify4
> [13053]    80 13053    47602     3917      40       92         -1000 postgres
> [13054]    80 13054    47602     3920      40       93         -1000 postgres
> [13055]    80 13055    47835     7071      72       92         -1000 postgres
> [13058]    80 13058    47831     4688      51       92         -1000 postgres
> [13059]    80 13059    47602     3919      40       92         -1000 postgres
> [13061]   500 13061   121241    17287     186        0             0 konversation
> [13062]    80 13062    47583     3456      39       95         -1000 postgres
> [13076]    80 13076    47745     6832      72       92         -1000 postgres
> [13080]    80 13080    47629     4038      42       93         -1000 postgres
> [13081]   500 13081   701137    24187     260        0             0 kate
> [13082]    80 13082    47866     8228      73       92         -1000 postgres
> [13083]    80 13083    47623     3981      40       92         -1000 postgres
> [13085]    80 13085    47830     8033      67       92         -1000 postgres
> [13087]    80 13087    47549     2587      34       95         -1000 postgres
> [13088]    80 13088    47828     5052      52       92         -1000 postgres
> [13089]    80 13089    47549     2601      34       95         -1000 postgres
> [13090]    80 13090    47684     9326      79       92         -1000 postgres
> [13091]    80 13091    47581     3365      39       95         -1000 postgres
> [13092]    80 13092    47651     4875      59       92         -1000 postgres
> [13093]    80 13093    47769     5336      62       92         -1000 postgres
> [13094]    80 13094    47670     4777      55       93         -1000 postgres
> [13095]    80 13095    47582     3205      39       95         -1000 postgres
> [13096]    80 13096    47596     3887      44       93         -1000 postgres
> [13097]    80 13097    47835     4919      50       92         -1000 postgres
> [13098]    80 13098    47900    22860      93       89         -1000 postgres
> [13100]    80 13100    47581     3364      39       95         -1000 postgres
> [13102]    80 13102    47549     2586      34       95         -1000 postgres
> [13103]    80 13103    47549     2586      34       95         -1000 postgres
> [13105]   500 13105   578961    40138     324        0             0 rekonq
> [13106]    80 13106    47595     3889      44       93         -1000 postgres
> [13107]    80 13107    47549     2586      34       95         -1000 postgres
> [13108]    80 13108    47549     2586      34       95         -1000 postgres
> [13109]    80 13109    49319    34543      99       92         -1000 postgres
> [13111]    80 13111    47549     2571      34       95         -1000 postgres
> [13112]    80 13112    49304    35409      99       92         -1000 postgres
> [13114]    80 13114    49304    36076      99       92         -1000 postgres
> [13115]    80 13115    47862     5758      65       92         -1000 postgres
> [13116]   500 13116   113252    14152     201        0             0 kaddressbook
> [13122]   500 13122   493943    33314     323        0             0 kmail
> [13126]   500 13126   123010    12244     188        0             0 konqueror
> [13140]   500 13140    64238     5816      96        0             0 applet.py
> [13141]   500 13141   115594    19636     162        0             0 net_applet
> [13143]   500 13143    95177     8297     141        0             0 polkit-kde-auth
> [13148]   500 13148    66455     1821      32        0             0 xsettings-kde
> [13149]   500 13149    10882     1157      27        0             0 xload
> [13155]    80 13155    47803     4595      53       93         -1000 postgres
> [13156]    80 13156    47973    11993      86       92         -1000 postgres
> [13171]   500 13171    73289     9621     132        0             0 kwalletmanager
> [13178]   500 13178    84306     1842      35        0             0 at-spi-bus-laun
> [13182]   500 13182     3403      569      12        0             0 dbus-daemon
> [13185]   500 13185    31230     1213      31        0             0 at-spi2-registr
> [13189]   500 13189    46019     1304      27        0             0 gvfsd
> [13231]    80 13231    48340    35725      97       92         -1000 postgres
> [13233]    80 13233    47925    24455      91       92         -1000 postgres
> [13309]    80 13309    47549     2555      34       95         -1000 postgres
> [13442]   500 13442   260391    19500     137        0           200 python
> [13447]    80 13447    49761    29978     101       83         -1000 postgres
> [13470]   500 13470   122584     8709     157        0             0 kio_http
> [13482]    80 13482    48989    18272      99       83         -1000 postgres
> [13501]   500 13501   122587     8695     157        0             0 kio_http
> [13505]   500 13505   122584     8710     157        0             0 kio_http
> [13512]   500 13512   122585     8694     157        0             0 kio_http
> [13523]   500 13523   122584     8686     157        0             0 kio_http
> [13526]   500 13526   122584     8702     157        0             0 kio_http
> [13788]   500 13788   115594    13290     145        0             0 net_applet
> [13815]   500 13815    40303     3579      48        0             0 netmon_cron_par
> [13830]   500 13830    23067     3637      47        0             0 content_index.p
> [13843]   500 13843    21772     3377      46        0             0 decode-hardware
> [13863]   500 13863    24181     3735      52        0             0 netmon_decode.p
> [13909]   500 13909    21912     3505      47        0             0 python
> [13914]   500 13914     2330      599      10        0             0 mails-process-n
> [13925]   500 13925    24138     3722      53        0             0 netmon_cflastse
> [13989]     0 13989     1975      438       9        0             0 agetty
> [13994]   500 13994    34668     3038      66        0             0 kcminit
> [13995]   500 13995    43712     4218      78        0             0 kcminit
> [13996]   500 13996     4623      678      13        0             0 setxkbmap
> [13997]     0 13997     4582      268      13        0             0 xkbcomp
> Out of memory: Kill process 13442 (python) score 208 or sacrifice child
> Killed process 13442 (python) total-vm:1041564kB, anon-rss:67284kB, file-rss:10716kB
> 
> SysRq : HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k)
> show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q)
> unraw(r) sync(s) show-task-states(t) unmount(u) force-fb(V) show-blocked-tasks(w) dump-ftrace-buffer(z)
> SysRq : HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k)
> show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q)
> unraw(r) sync(s) show-task-states(t) unmount(u) force-fb(V) show-blocked-tasks(w) dump-ftrace-buffer(z)
> SysRq : HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k)
> show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q)
> unraw(r) sync(s) show-task-states(t) unmount(u) force-fb(V) show-blocked-tasks(w) dump-ftrace-buffer(z)
> SysRq : Restore framebuffer console
> SysRq : HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k)
> show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q)
> unraw(r) sync(s) show-task-states(t) unmount(u) force-fb(V) show-blocked-tasks(w) dump-ftrace-buffer(z)
> SysRq : Keyboard mode set to system default
> 
> -- Reboot --
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: Early test: hangs in mm/compact.c w. Linus's 12d7aacab56e9ef185c
  2014-11-09  4:47 Hillf Danton
@ 2014-11-09  8:22 ` P. Christeas
  2014-11-09  9:35   ` Vlastimil Babka
  0 siblings, 1 reply; 20+ messages in thread
From: P. Christeas @ 2014-11-09  8:22 UTC (permalink / raw)
  To: Hillf Danton, linux-kernel; +Cc: 'Vlastimil Babka', linux-mm

On Sunday 09 November 2014, Hillf Danton wrote:
> -		return COMPACT_CONTINUE;
> +		return COMPACT_SKIPPED;

I guess this one would mitigate against Vlastmil's migration scanner issue, 
wouldn't it?

In that case, I should wait a bit[1] to try the first patch, then revert, try 
yours and (hopefully) have some results.

Then, apply both.

[1] trying to push the vm by loading memory-hungry apps and random load.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: Early test: hangs in mm/compact.c w. Linus's 12d7aacab56e9ef185c
  2014-11-08 22:18             ` Vlastimil Babka
@ 2014-11-09  8:27               ` Pavel Machek
  2014-11-09  9:43                 ` Vlastimil Babka
  2014-11-10  6:07               ` Joonsoo Kim
  2014-11-10  8:14               ` P. Christeas
  2 siblings, 1 reply; 20+ messages in thread
From: Pavel Machek @ 2014-11-09  8:27 UTC (permalink / raw)
  To: Vlastimil Babka
  Cc: P. Christeas, linux-mm, Joonsoo Kim, lkml, David Rientjes,
	Norbert Preining, Markus Trippelsdorf

[-- Attachment #1: Type: text/plain, Size: 2258 bytes --]

Hi!

> >> Oh and did I ask in this thread for /proc/zoneinfo yet? :)
> > 
> > Using that same kernel[1], got again into a race, gathered a few more data.
> > 
> > This time, I had 1x "urpmq" process [2] hung at 100% CPU , when "kwin" got 
> > apparently blocked (100% CPU, too) trying to resize a GUI window. I suppose 
> > the resizing operation would mean heavy memory alloc/free.
> > 
> > The rest of the system was responsive, I could easily get a console, login, 
> > gather the files.. Then, I have *killed* -9 the "urpmq" process, which solved 
> > the race and my system is still alive! "kwin" is still running, returned to 
> > regular CPU load.
> > 
> > Attached is traces from SysRq+l (pressed a few times, wanted to "snapshot" the 
> > stack) and /proc/zoneinfo + /proc/vmstat
> > 
> > Bisection is not yet meaningful, IMHO, because I cannot be sure that "good" 
> > points are really free from this issue. I'd estimate that each test would take 
> > +3days, unless I really find a deterministic way to reproduce the issue .
> 
> Hi,
> 
> I think I finally found the cause by staring into the code... CCing
> people from all 4 separate threads I know about this issue.
> The problem with finding the cause was that the first report I got from
> Markus was about isolate_freepages_block() overhead, and later Norbert
> reported that reverting a patch for isolate_freepages* helped. But the
> problem seems to be that although the loop in isolate_migratepages exits
> because the scanners almost meet (they are within same pageblock), they
> don't truly meet, therefore compact_finished() decides to continue, but
> isolate_migratepages() exits immediately... boom! But indeed e14c720efdd7
> made this situation possible, as free scaner pfn can now point to a
> middle of pageblock.

Ok, it seems it happened second time now, again shortly after
resume. I guess I should apply your patch after all.

(Or... instead it should go to Linus ASAP -- it fixes known problem
that is affected people, and we want it in soon in case it is not
complete fix.)

Dmesg is in the attachment, perhaps it helps.
									Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: delme.gz --]
[-- Type: application/gzip, Size: 18436 bytes --]

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: Early test: hangs in mm/compact.c w. Linus's 12d7aacab56e9ef185c
  2014-11-09  8:22 ` P. Christeas
@ 2014-11-09  9:35   ` Vlastimil Babka
  2014-11-10  3:23     ` Hillf Danton
  0 siblings, 1 reply; 20+ messages in thread
From: Vlastimil Babka @ 2014-11-09  9:35 UTC (permalink / raw)
  To: P. Christeas, Hillf Danton, linux-kernel; +Cc: linux-mm

On 11/09/2014 09:22 AM, P. Christeas wrote:
> On Sunday 09 November 2014, Hillf Danton wrote:
>> -		return COMPACT_CONTINUE;
>> +		return COMPACT_SKIPPED;
> 
> I guess this one would mitigate against Vlastmil's migration scanner issue, 
> wouldn't it?

Please no, that's a wrong fix. The purpose of compaction is to make the
high-order watermark meet, not give up.

> In that case, I should wait a bit[1] to try the first patch, then revert, try 
> yours and (hopefully) have some results.

I hope my patch will be enough,

> Then, apply both.
> 
> [1] trying to push the vm by loading memory-hungry apps and random load.

Maybe the tools/testing/selftests/vm/transhuge-stress could help

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: Early test: hangs in mm/compact.c w. Linus's 12d7aacab56e9ef185c
  2014-11-09  8:27               ` Pavel Machek
@ 2014-11-09  9:43                 ` Vlastimil Babka
  2014-11-09 22:32                   ` Norbert Preining
  0 siblings, 1 reply; 20+ messages in thread
From: Vlastimil Babka @ 2014-11-09  9:43 UTC (permalink / raw)
  To: Pavel Machek
  Cc: P. Christeas, linux-mm, Joonsoo Kim, lkml, David Rientjes,
	Norbert Preining, Markus Trippelsdorf

On 11/09/2014 09:27 AM, Pavel Machek wrote:
> Hi!
> 
>>>> Oh and did I ask in this thread for /proc/zoneinfo yet? :)
>>>
>>> Using that same kernel[1], got again into a race, gathered a few more data.
>>>
>>> This time, I had 1x "urpmq" process [2] hung at 100% CPU , when "kwin" got 
>>> apparently blocked (100% CPU, too) trying to resize a GUI window. I suppose 
>>> the resizing operation would mean heavy memory alloc/free.
>>>
>>> The rest of the system was responsive, I could easily get a console, login, 
>>> gather the files.. Then, I have *killed* -9 the "urpmq" process, which solved 
>>> the race and my system is still alive! "kwin" is still running, returned to 
>>> regular CPU load.
>>>
>>> Attached is traces from SysRq+l (pressed a few times, wanted to "snapshot" the 
>>> stack) and /proc/zoneinfo + /proc/vmstat
>>>
>>> Bisection is not yet meaningful, IMHO, because I cannot be sure that "good" 
>>> points are really free from this issue. I'd estimate that each test would take 
>>> +3days, unless I really find a deterministic way to reproduce the issue .
>>
>> Hi,
>>
>> I think I finally found the cause by staring into the code... CCing
>> people from all 4 separate threads I know about this issue.
>> The problem with finding the cause was that the first report I got from
>> Markus was about isolate_freepages_block() overhead, and later Norbert
>> reported that reverting a patch for isolate_freepages* helped. But the
>> problem seems to be that although the loop in isolate_migratepages exits
>> because the scanners almost meet (they are within same pageblock), they
>> don't truly meet, therefore compact_finished() decides to continue, but
>> isolate_migratepages() exits immediately... boom! But indeed e14c720efdd7
>> made this situation possible, as free scaner pfn can now point to a
>> middle of pageblock.
> 
> Ok, it seems it happened second time now, again shortly after
> resume. I guess I should apply your patch after all.

Thanks.

> (Or... instead it should go to Linus ASAP -- it fixes known problem
> that is affected people, and we want it in soon in case it is not
> complete fix.)

I don't want to send untested fix, and wasn't able to reproduce the bug
myself. I think Norbert could do it rather quickly so I hope he can tell
us soon.

> Dmesg is in the attachment, perhaps it helps.
> 									Pavel

It looks the same as before, so no surprises there, which is good.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: Early test: hangs in mm/compact.c w. Linus's 12d7aacab56e9ef185c
  2014-11-09  9:43                 ` Vlastimil Babka
@ 2014-11-09 22:32                   ` Norbert Preining
  0 siblings, 0 replies; 20+ messages in thread
From: Norbert Preining @ 2014-11-09 22:32 UTC (permalink / raw)
  To: Vlastimil Babka
  Cc: Pavel Machek, P. Christeas, linux-mm, Joonsoo Kim, lkml,
	David Rientjes, Markus Trippelsdorf

Hi Vlastimil, hi all,

On Sun, 09 Nov 2014, Vlastimil Babka wrote:
> I don't want to send untested fix, and wasn't able to reproduce the bug
> myself. I think Norbert could do it rather quickly so I hope he can tell
> us soon.

Sorry, weekend means I am away from my laptop for extended times,
and I wanted to give it a bit of stress testing.

No problems till now, no hangs, all working as expected with
your latest patch.

Thanks a lot

Norbert

------------------------------------------------------------------------
PREINING, Norbert                               http://www.preining.info
JAIST, Japan                                 TeX Live & Debian Developer
GPG: 0x860CDC13   fp: F7D8 A928 26E3 16A1 9FA0  ACF0 6CAC A448 860C DC13
------------------------------------------------------------------------

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: Early test: hangs in mm/compact.c w. Linus's 12d7aacab56e9ef185c
  2014-11-09  9:35   ` Vlastimil Babka
@ 2014-11-10  3:23     ` Hillf Danton
  0 siblings, 0 replies; 20+ messages in thread
From: Hillf Danton @ 2014-11-10  3:23 UTC (permalink / raw)
  To: 'Vlastimil Babka', 'P. Christeas',
	'linux-kernel'
  Cc: linux-mm

> >
> > I guess this one would mitigate against Vlastmil's migration scanner issue,
> > wouldn't it?
> 
Nope, I wanted to see  if free pages are low enough.

> Please no, that's a wrong fix. The purpose of compaction is to make the
> high-order watermark meet, not give up.
> 
Yupe, have to spin.

--- a/mm/compaction.c	Sun Nov  9 12:02:59 2014
+++ b/mm/compaction.c	Mon Nov 10 11:12:07 2014
@@ -1074,6 +1074,8 @@ static int compact_finished(struct zone 
 	watermark = low_wmark_pages(zone);
 	watermark += (1 << cc->order);
 
+	if (!zone_watermark_ok(zone, 0, watermark, 0, 0))
+		return COMPACT_SKIPPED;
 	if (!zone_watermark_ok(zone, cc->order, watermark, 0, 0))
 		return COMPACT_CONTINUE;
 
--


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: Early test: hangs in mm/compact.c w. Linus's 12d7aacab56e9ef185c
  2014-11-08 22:18             ` Vlastimil Babka
  2014-11-09  8:27               ` Pavel Machek
@ 2014-11-10  6:07               ` Joonsoo Kim
  2014-11-10  7:53                 ` Vlastimil Babka
  2014-11-10  8:14               ` P. Christeas
  2 siblings, 1 reply; 20+ messages in thread
From: Joonsoo Kim @ 2014-11-10  6:07 UTC (permalink / raw)
  To: Vlastimil Babka
  Cc: P. Christeas, linux-mm, lkml, David Rientjes, Norbert Preining,
	Markus Trippelsdorf, Pavel Machek

On Sat, Nov 08, 2014 at 11:18:37PM +0100, Vlastimil Babka wrote:
> On 11/08/2014 02:11 PM, P. Christeas wrote:
> > On Thursday 06 November 2014, Vlastimil Babka wrote:
> >>> On Wednesday 05 November 2014, Vlastimil Babka wrote:
> >>>> Can you please try the following patch?
> >>>> -			compaction_defer_reset(zone, order, false);
> >> Oh and did I ask in this thread for /proc/zoneinfo yet? :)
> > 
> > Using that same kernel[1], got again into a race, gathered a few more data.
> > 
> > This time, I had 1x "urpmq" process [2] hung at 100% CPU , when "kwin" got 
> > apparently blocked (100% CPU, too) trying to resize a GUI window. I suppose 
> > the resizing operation would mean heavy memory alloc/free.
> > 
> > The rest of the system was responsive, I could easily get a console, login, 
> > gather the files.. Then, I have *killed* -9 the "urpmq" process, which solved 
> > the race and my system is still alive! "kwin" is still running, returned to 
> > regular CPU load.
> > 
> > Attached is traces from SysRq+l (pressed a few times, wanted to "snapshot" the 
> > stack) and /proc/zoneinfo + /proc/vmstat
> > 
> > Bisection is not yet meaningful, IMHO, because I cannot be sure that "good" 
> > points are really free from this issue. I'd estimate that each test would take 
> > +3days, unless I really find a deterministic way to reproduce the issue .
> 
> Hi,
> 
> I think I finally found the cause by staring into the code... CCing
> people from all 4 separate threads I know about this issue.
> The problem with finding the cause was that the first report I got from
> Markus was about isolate_freepages_block() overhead, and later Norbert
> reported that reverting a patch for isolate_freepages* helped. But the
> problem seems to be that although the loop in isolate_migratepages exits
> because the scanners almost meet (they are within same pageblock), they
> don't truly meet, therefore compact_finished() decides to continue, but
> isolate_migratepages() exits immediately... boom! But indeed e14c720efdd7
> made this situation possible, as free scaner pfn can now point to a
> middle of pageblock.

Indeed.

> 
> So I hope the attached patch will fix the soft-lockup issues in
> compact_zone. Please apply on 3.18-rc3 or later without any other reverts,
> and test. It probably won't help Markus and his isolate_freepages_block()
> overhead though...

Yes, I found this bug too, but, it can't explain
isolate_freepages_block() overhead. Anyway, I can't find another bug
related to isolate_freepages_block(). :/

> Thanks,
> Vlastimil
> 
> ------8<------
> >From fbf8eb0bcd2897090312e23da6a31bad9cc6b337 Mon Sep 17 00:00:00 2001
> From: Vlastimil Babka <vbabka@suse.cz>
> Date: Sat, 8 Nov 2014 22:20:43 +0100
> Subject: [PATCH] mm, compaction: prevent endless loop in migrate scanner
> 
> ---
>  mm/compaction.c | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/compaction.c b/mm/compaction.c
> index ec74cf0..1b7a1be 100644
> --- a/mm/compaction.c
> +++ b/mm/compaction.c
> @@ -1029,8 +1029,12 @@ static isolate_migrate_t isolate_migratepages(struct zone *zone,
>  	}
>  
>  	acct_isolated(zone, cc);
> -	/* Record where migration scanner will be restarted */
> -	cc->migrate_pfn = low_pfn;
> +	/* 
> +	 * Record where migration scanner will be restarted. If we end up in
> +	 * the same pageblock as the free scanner, make the scanners fully
> +	 * meet so that compact_finished() terminates compaction.
> +	 */
> +	cc->migrate_pfn = (end_pfn <= cc->free_pfn) ? low_pfn : cc->free_pfn;
>  
>  	return cc->nr_migratepages ? ISOLATE_SUCCESS : ISOLATE_NONE;
>  }

IMHO, proper fix is not to change this logic, but, to change decision
logic in compact_finished() and in compact_zone(). Maybe helper
function would be good for readability.

Thanks.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: Early test: hangs in mm/compact.c w. Linus's 12d7aacab56e9ef185c
  2014-11-10  6:07               ` Joonsoo Kim
@ 2014-11-10  7:53                 ` Vlastimil Babka
  2014-11-10  8:05                   ` Joonsoo Kim
  0 siblings, 1 reply; 20+ messages in thread
From: Vlastimil Babka @ 2014-11-10  7:53 UTC (permalink / raw)
  To: Joonsoo Kim
  Cc: P. Christeas, linux-mm, lkml, David Rientjes, Norbert Preining,
	Markus Trippelsdorf, Pavel Machek

On 11/10/2014 07:07 AM, Joonsoo Kim wrote:
> On Sat, Nov 08, 2014 at 11:18:37PM +0100, Vlastimil Babka wrote:
>> On 11/08/2014 02:11 PM, P. Christeas wrote:
>>
>> Hi,
>>
>> I think I finally found the cause by staring into the code... CCing
>> people from all 4 separate threads I know about this issue.
>> The problem with finding the cause was that the first report I got from
>> Markus was about isolate_freepages_block() overhead, and later Norbert
>> reported that reverting a patch for isolate_freepages* helped. But the
>> problem seems to be that although the loop in isolate_migratepages exits
>> because the scanners almost meet (they are within same pageblock), they
>> don't truly meet, therefore compact_finished() decides to continue, but
>> isolate_migratepages() exits immediately... boom! But indeed e14c720efdd7
>> made this situation possible, as free scaner pfn can now point to a
>> middle of pageblock.
>
> Indeed.
>
>>
>> So I hope the attached patch will fix the soft-lockup issues in
>> compact_zone. Please apply on 3.18-rc3 or later without any other reverts,
>> and test. It probably won't help Markus and his isolate_freepages_block()
>> overhead though...
>
> Yes, I found this bug too, but, it can't explain
> isolate_freepages_block() overhead. Anyway, I can't find another bug
> related to isolate_freepages_block(). :/

Thanks for checking.

>> Thanks,
>> Vlastimil
>>
>> ------8<------
>> >From fbf8eb0bcd2897090312e23da6a31bad9cc6b337 Mon Sep 17 00:00:00 2001
>> From: Vlastimil Babka <vbabka@suse.cz>
>> Date: Sat, 8 Nov 2014 22:20:43 +0100
>> Subject: [PATCH] mm, compaction: prevent endless loop in migrate scanner
>>
>> ---
>>   mm/compaction.c | 8 ++++++--
>>   1 file changed, 6 insertions(+), 2 deletions(-)
>>
>> diff --git a/mm/compaction.c b/mm/compaction.c
>> index ec74cf0..1b7a1be 100644
>> --- a/mm/compaction.c
>> +++ b/mm/compaction.c
>> @@ -1029,8 +1029,12 @@ static isolate_migrate_t isolate_migratepages(struct zone *zone,
>>   	}
>>
>>   	acct_isolated(zone, cc);
>> -	/* Record where migration scanner will be restarted */
>> -	cc->migrate_pfn = low_pfn;
>> +	/*
>> +	 * Record where migration scanner will be restarted. If we end up in
>> +	 * the same pageblock as the free scanner, make the scanners fully
>> +	 * meet so that compact_finished() terminates compaction.
>> +	 */
>> +	cc->migrate_pfn = (end_pfn <= cc->free_pfn) ? low_pfn : cc->free_pfn;
>>
>>   	return cc->nr_migratepages ? ISOLATE_SUCCESS : ISOLATE_NONE;
>>   }
>
> IMHO, proper fix is not to change this logic, but, to change decision
> logic in compact_finished() and in compact_zone(). Maybe helper
> function would be good for readability.

OK but I would think that to fix 3.18 ASAP and not introduce more 
regressions, go with the patch above first, as it is the minimal fix and 
people already test it. Then we can implement your suggestion later as a 
cleanup for 3.19?

Vlastimil

> Thanks.
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: Early test: hangs in mm/compact.c w. Linus's 12d7aacab56e9ef185c
  2014-11-10  7:53                 ` Vlastimil Babka
@ 2014-11-10  8:05                   ` Joonsoo Kim
  0 siblings, 0 replies; 20+ messages in thread
From: Joonsoo Kim @ 2014-11-10  8:05 UTC (permalink / raw)
  To: Vlastimil Babka
  Cc: P. Christeas, linux-mm, lkml, David Rientjes, Norbert Preining,
	Markus Trippelsdorf, Pavel Machek

On Mon, Nov 10, 2014 at 08:53:38AM +0100, Vlastimil Babka wrote:
> On 11/10/2014 07:07 AM, Joonsoo Kim wrote:
> >On Sat, Nov 08, 2014 at 11:18:37PM +0100, Vlastimil Babka wrote:
> >>On 11/08/2014 02:11 PM, P. Christeas wrote:
> >>
> >>Hi,
> >>
> >>I think I finally found the cause by staring into the code... CCing
> >>people from all 4 separate threads I know about this issue.
> >>The problem with finding the cause was that the first report I got from
> >>Markus was about isolate_freepages_block() overhead, and later Norbert
> >>reported that reverting a patch for isolate_freepages* helped. But the
> >>problem seems to be that although the loop in isolate_migratepages exits
> >>because the scanners almost meet (they are within same pageblock), they
> >>don't truly meet, therefore compact_finished() decides to continue, but
> >>isolate_migratepages() exits immediately... boom! But indeed e14c720efdd7
> >>made this situation possible, as free scaner pfn can now point to a
> >>middle of pageblock.
> >
> >Indeed.
> >
> >>
> >>So I hope the attached patch will fix the soft-lockup issues in
> >>compact_zone. Please apply on 3.18-rc3 or later without any other reverts,
> >>and test. It probably won't help Markus and his isolate_freepages_block()
> >>overhead though...
> >
> >Yes, I found this bug too, but, it can't explain
> >isolate_freepages_block() overhead. Anyway, I can't find another bug
> >related to isolate_freepages_block(). :/
> 
> Thanks for checking.
> 
> >>Thanks,
> >>Vlastimil
> >>
> >>------8<------
> >>>From fbf8eb0bcd2897090312e23da6a31bad9cc6b337 Mon Sep 17 00:00:00 2001
> >>From: Vlastimil Babka <vbabka@suse.cz>
> >>Date: Sat, 8 Nov 2014 22:20:43 +0100
> >>Subject: [PATCH] mm, compaction: prevent endless loop in migrate scanner
> >>
> >>---
> >>  mm/compaction.c | 8 ++++++--
> >>  1 file changed, 6 insertions(+), 2 deletions(-)
> >>
> >>diff --git a/mm/compaction.c b/mm/compaction.c
> >>index ec74cf0..1b7a1be 100644
> >>--- a/mm/compaction.c
> >>+++ b/mm/compaction.c
> >>@@ -1029,8 +1029,12 @@ static isolate_migrate_t isolate_migratepages(struct zone *zone,
> >>  	}
> >>
> >>  	acct_isolated(zone, cc);
> >>-	/* Record where migration scanner will be restarted */
> >>-	cc->migrate_pfn = low_pfn;
> >>+	/*
> >>+	 * Record where migration scanner will be restarted. If we end up in
> >>+	 * the same pageblock as the free scanner, make the scanners fully
> >>+	 * meet so that compact_finished() terminates compaction.
> >>+	 */
> >>+	cc->migrate_pfn = (end_pfn <= cc->free_pfn) ? low_pfn : cc->free_pfn;
> >>
> >>  	return cc->nr_migratepages ? ISOLATE_SUCCESS : ISOLATE_NONE;
> >>  }
> >
> >IMHO, proper fix is not to change this logic, but, to change decision
> >logic in compact_finished() and in compact_zone(). Maybe helper
> >function would be good for readability.
> 
> OK but I would think that to fix 3.18 ASAP and not introduce more
> regressions, go with the patch above first, as it is the minimal fix
> and people already test it. Then we can implement your suggestion
> later as a cleanup for 3.19?

Yeap. Agreed.

Thanks.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: Early test: hangs in mm/compact.c w. Linus's 12d7aacab56e9ef185c
  2014-11-08 22:18             ` Vlastimil Babka
  2014-11-09  8:27               ` Pavel Machek
  2014-11-10  6:07               ` Joonsoo Kim
@ 2014-11-10  8:14               ` P. Christeas
  2 siblings, 0 replies; 20+ messages in thread
From: P. Christeas @ 2014-11-10  8:14 UTC (permalink / raw)
  To: Vlastimil Babka, Markus Trippelsdorf
  Cc: linux-mm, Joonsoo Kim, lkml, Pavel Machek

On Saturday 08 November 2014, Vlastimil Babka wrote:
> >From fbf8eb0bcd2897090312e23da6a31bad9cc6b337 Mon Sep 17 00:00:00 2001
> 
> From: Vlastimil Babka <vbabka@suse.cz>
> Date: Sat, 8 Nov 2014 22:20:43 +0100
> Subject: [PATCH] mm, compaction: prevent endless loop in migrate scanner

After 30hrs uptime, I also mark this test as PASSED .

:)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 20+ messages in thread

end of thread, other threads:[~2014-11-10  8:14 UTC | newest]

Thread overview: 20+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-11-04  7:26 Early test: hangs in mm/compact.c w. Linus's 12d7aacab56e9ef185c P. Christeas
2014-11-04  8:55 ` Vlastimil Babka
2014-11-04  9:36   ` P. Christeas
2014-11-05 15:26     ` Vlastimil Babka
2014-11-05 16:02       ` P. Christeas
2014-11-06 19:23       ` P. Christeas
2014-11-06 21:38         ` Vlastimil Babka
2014-11-08 13:11           ` P. Christeas
2014-11-08 22:18             ` Vlastimil Babka
2014-11-09  8:27               ` Pavel Machek
2014-11-09  9:43                 ` Vlastimil Babka
2014-11-09 22:32                   ` Norbert Preining
2014-11-10  6:07               ` Joonsoo Kim
2014-11-10  7:53                 ` Vlastimil Babka
2014-11-10  8:05                   ` Joonsoo Kim
2014-11-10  8:14               ` P. Christeas
  -- strict thread matches above, loose matches on Subject: below --
2014-11-09  4:47 Hillf Danton
2014-11-09  8:22 ` P. Christeas
2014-11-09  9:35   ` Vlastimil Babka
2014-11-10  3:23     ` Hillf Danton

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).