From: Dave Jones <davej@redhat.com>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: "Chris Mason" <clm@fb.com>,
"Mike Galbraith" <umgwanakikbuti@gmail.com>,
"Ingo Molnar" <mingo@kernel.org>,
"Peter Zijlstra" <peterz@infradead.org>,
"Dâniel Fraga" <fragabr@gmail.com>,
"Sasha Levin" <sasha.levin@oracle.com>,
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
"Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>,
"Suresh Siddha" <sbsiddha@gmail.com>,
"Oleg Nesterov" <oleg@redhat.com>,
"Peter Anvin" <hpa@linux.intel.com>
Subject: Re: frequent lockups in 3.18rc4
Date: Thu, 18 Dec 2014 00:13:27 -0500 [thread overview]
Message-ID: <20141218051327.GA31988@redhat.com> (raw)
In-Reply-To: <CA+55aFz6KEKdz4Nxkx2fa-FH3PZ+Aa51iA_nXobsQ-dDW5PGEg@mail.gmail.com>
On Mon, Dec 15, 2014 at 03:46:41PM -0800, Linus Torvalds wrote:
> On Mon, Dec 15, 2014 at 10:21 AM, Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> >
> > So let's just fix it. Here's a completely untested patch.
>
> So after looking at this more, I'm actually really convinced that this
> was a pretty nasty bug.
>
> I'm *not* convinced that it's necessarily *your* bug, but I still
> think it could be.
Bah, I was getting all optimistic.
I came home this evening to a locked up machine.
Serial console had a *lot* more traces than usual though.
Full log below. The 12xxx.xxxxxx traces we seemed to recover from,
followed by silence for a while, before the real fun begins at 157xx.xxxxxx
[12525.310743] NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [trinity-c60:28574]
[12525.311429] Modules linked in: bridge 8021q garp stp snd_seq_dummy dlci tun fuse rfcomm hidp bnep af_key llc2 nfnetlink can_bcm scsi_transport_iscsi can_raw sctp libcrc32c nfc caif_socket caif af_802154 ieee802154 phonet af_rxrpc bluetooth can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 usb_debug cfg80211 rfkill coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode serio_raw pcspkr snd_hda_intel snd_hda_controller snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp pps_core snd_timer snd soundcore shpchp nfsd auth_rpcgss oid_registry nfs_acl lockd grace sunrpc
[12525.315877] CPU: 2 PID: 28574 Comm: trinity-c60 Not tainted 3.18.0+ #106
[12525.318136] task: ffff880153a20000 ti: ffff8801c251c000 task.ti: ffff8801c251c000
[12525.318931] RIP: 0010:[<ffffffff81373805>] [<ffffffff81373805>] copy_user_enhanced_fast_string+0x5/0x10
[12525.319758] RSP: 0018:ffff8801c251fbe0 EFLAGS: 00010206
[12525.320555] RAX: 00007f9c44b07f1d RBX: 0000000000000003 RCX: 00000000000009c0
[12525.321403] RDX: 0000000000001000 RSI: 00007f9c44b0855d RDI: ffff88007cb1f640
[12525.322227] RBP: ffff8801c251fc28 R08: 0000000000000000 R09: 0000000000000001
[12525.323046] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff817cf22b
[12525.323854] R13: ffff8801c251fb68 R14: ffff880061dccd40 R15: ffff880061dcce60
[12525.324676] FS: 00007f9c4714c740(0000) GS:ffff880245200000(0000) knlGS:0000000000000000
[12525.325468] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12525.326262] CR2: 00007f9c45b905b8 CR3: 000000022a30b000 CR4: 00000000001407e0
[12525.327065] DR0: 00007f6c9030b000 DR1: 0000000000000000 DR2: 0000000000000000
[12525.327867] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[12525.328666] Stack:
[12525.329459] ffffffff8119fe56 ffff8801c251fbf8 140000010718f000 0000000000001000
[12525.330283] 140000010718f000 0000000000001000 ffff8801c251fd60 0000000000000000
[12525.331139] ffff880061dcd070 ffff8801c251fcc8 ffffffff81175fe7 ffff8801c251fc88
[12525.331969] Call Trace:
[12525.332801] [<ffffffff8119fe56>] ? iov_iter_copy_from_user_atomic+0x156/0x180
[12525.333642] [<ffffffff81175fe7>] generic_perform_write+0xf7/0x1f0
[12525.334475] [<ffffffff81178722>] __generic_file_write_iter+0x162/0x350
[12525.335315] [<ffffffff811e7940>] ? new_sync_read+0xd0/0xd0
[12525.336149] [<ffffffff8117894f>] generic_file_write_iter+0x3f/0xb0
[12525.336981] [<ffffffff81178910>] ? __generic_file_write_iter+0x350/0x350
[12525.337809] [<ffffffff811e7a88>] do_iter_readv_writev+0x78/0xc0
[12525.338630] [<ffffffff811e92b8>] do_readv_writev+0xd8/0x2a0
[12525.339435] [<ffffffff81178910>] ? __generic_file_write_iter+0x350/0x350
[12525.340238] [<ffffffff81178910>] ? __generic_file_write_iter+0x350/0x350
[12525.341068] [<ffffffff810c361f>] ? lock_release_holdtime.part.24+0xf/0x190
[12525.341865] [<ffffffff817cf310>] ? _raw_spin_unlock_irq+0x30/0x40
[12525.342664] [<ffffffff811e950c>] vfs_writev+0x3c/0x50
[12525.343486] [<ffffffff811e98d2>] SyS_pwritev+0xc2/0xf0
[12525.344292] [<ffffffff817cff12>] system_call_fastpath+0x12/0x17
[12525.345099] Code: 48 ff c6 48 ff c7 ff c9 75 f2 89 d1 c1 e9 03 83 e2 07 f3 48 a5 89 d1 f3 a4 31 c0 0f 1f 00 c3 0f 1f 80 00 00 00 00 0f 1f 00 89 d1 <f3> a4 31 c0 0f 1f 00 c3 90 90 90 0f 1f 00 83 fa 08 0f 82 95 00
[12525.346820] sending NMI to other CPUs:
[12525.347621] NMI backtrace for cpu 3
[12525.348409] CPU: 3 PID: 25560 Comm: trinity-c201 Not tainted 3.18.0+ #106
[12525.350733] task: ffff880095342da0 ti: ffff8801c048c000 task.ti: ffff8801c048c000
[12525.351503] RIP: 0010:[<ffffffff810fbc4e>] [<ffffffff810fbc4e>] generic_exec_single+0xee/0x1b0
[12525.352270] RSP: 0018:ffff8801c048f9f8 EFLAGS: 00000202
[12525.353017] RAX: 0000000000000008 RBX: ffff8801c048fa10 RCX: 0000000000000038
[12525.353757] RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000000
[12525.354479] RBP: ffff8801c048fa58 R08: ffff8802437418f0 R09: 0000000000000000
[12525.355187] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000002
[12525.355878] R13: 0000000000000001 R14: ffff880061e34fc0 R15: 0000000000000003
[12525.356558] FS: 00007f9c4714c740(0000) GS:ffff880245400000(0000) knlGS:0000000000000000
[12525.357236] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12525.357905] CR2: 0000000001ac9498 CR3: 00000002247cb000 CR4: 00000000001407e0
[12525.358577] DR0: 00007f6c9030b000 DR1: 0000000000000000 DR2: 0000000000000000
[12525.359246] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[12525.359911] Stack:
[12525.360568] 0000000000000000 0000000000000000 ffff8801c048fa78 0000000000000000
[12525.361250] ffffffff81048cd0 ffff8801c048fb08 0000000000000003 00000000639bd7ce
[12525.361928] ffff880095342da0 00000000ffffffff 0000000000000002 ffffffff81048cd0
[12525.362601] Call Trace:
[12525.363259] [<ffffffff81048cd0>] ? do_flush_tlb_all+0x60/0x60
[12525.363926] [<ffffffff81048cd0>] ? do_flush_tlb_all+0x60/0x60
[12525.364589] [<ffffffff810fbdb0>] smp_call_function_single+0x70/0xd0
[12525.365250] [<ffffffff81048cd0>] ? do_flush_tlb_all+0x60/0x60
[12525.365904] [<ffffffff810fc4a9>] smp_call_function_many+0x2b9/0x320
[12525.366556] [<ffffffff811a396d>] ? unmap_single_vma+0x50d/0x900
[12525.367204] [<ffffffff81049020>] flush_tlb_mm_range+0x90/0x1d0
[12525.367843] [<ffffffff811a25b2>] tlb_flush_mmu_tlbonly+0x42/0x50
[12525.368476] [<ffffffff811a2bac>] tlb_flush_mmu+0x1c/0x30
[12525.369102] [<ffffffff811a2bd4>] tlb_finish_mmu+0x14/0x40
[12525.369726] [<ffffffff811a3e78>] zap_page_range_single+0x118/0x160
[12525.370348] [<ffffffff811a4044>] unmap_mapping_range+0x134/0x190
[12525.370971] [<ffffffff81192a5d>] shmem_fallocate+0x4fd/0x520
[12525.371591] [<ffffffff810bcb77>] ? prepare_to_wait+0x27/0x90
[12525.372210] [<ffffffff811e6062>] do_fallocate+0x132/0x1d0
[12525.372825] [<ffffffff811b8558>] SyS_madvise+0x398/0x870
[12525.373439] [<ffffffff817cff12>] system_call_fastpath+0x12/0x17
[12525.374061] Code: 48 89 de 48 03 14 c5 a0 f2 d1 81 48 89 df e8 fa e0 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 18 01 74 0b 0f 1f 00 f3 90 f6 43 18 01 <75> f8 31 c0 48 8b 7d d8 65 48 33 3c 25 28 00 00 00 0f 85 98 00
[12525.375458] NMI backtrace for cpu 0
[12525.375565] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 27.836 msecs
[12525.376788] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.18.0+ #106
[12525.378827] task: ffffffff81c164c0 ti: ffffffff81c00000 task.ti: ffffffff81c00000
[12525.379514] RIP: 0010:[<ffffffff813d6cab>] [<ffffffff813d6cab>] intel_idle+0xdb/0x180
[12525.380215] RSP: 0018:ffffffff81c03e38 EFLAGS: 00000046
[12525.380905] RAX: 0000000000000032 RBX: 0000000000000010 RCX: 0000000000000001
[12525.381588] RDX: 0000000000000000 RSI: ffffffff81c03fd8 RDI: 0000000000000000
[12525.382254] RBP: ffffffff81c03e68 R08: 000000008baf8a0b R09: 0000000000000000
[12525.382911] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000005
[12525.383558] R13: 0000000000000032 R14: 0000000000000004 R15: ffffffff81c00000
[12525.384194] FS: 0000000000000000(0000) GS:ffff880244e00000(0000) knlGS:00000000000000[12525.398359] NMI backtrace for cpu 1
[12525.398368] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 50.720 msecs
[12525.399764] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 3.18.0+ #106
[12525.401932] task: ffff8802434b96d0 ti: ffff880243748000 task.ti: ffff880243748000
[12525.402687] RIP: 0010:[<ffffffff813d6cab>] [<ffffffff813d6cab>] intel_idle+0xdb/0x180
[12525.403457] RSP: 0018:ffff88024374be08 EFLAGS: 00000046
[12525.404241] RAX: 0000000000000032 RBX: 0000000000000010 RCX: 0000000000000001
[12525.405038] RDX: 0000000000000000 RSI: ffff88024374bfd8 RDI: 0000000000000001
[12525.405832] RBP: ffff88024374be38 R08: 000000008baf8a0b R09: 0000000000000000
[12525.406611] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000005
[12525.407377] R13: 0000000000000032 R14: 0000000000000004 R15: ffff880243748000
[12525.408131] FS: 0000000000000000(0000) GS:ffff880245000000(0000) knlGS:0000000000000000
[12525.408881] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12525.409601] CR2: 00000000000000d2 CR3: 0000000001c11000 CR4: 00000000001407e0
[12525.410310] DR0: 00007f6c9030b000 DR1: 0000000000000000 DR2: 0000000000000000
[12525.411011] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[12525.411694] Stack:
[12525.412363] 000000014374be38 b532cae23f43ed20 ffffe8ffff003200 0000000000000005
[12525.413051] ffffffff81cb1bc0 0000000000000001 ffff88024374be88 ffffffff8165f7b5
[12525.413744] 00000b65f893e397 ffffffff81cb1d90 ffffffff81cb1bc0 ffffffff81d215f0
[12525.414439] Call Trace:
[12525.415125] [<ffffffff8165f7b5>] cpuidle_enter_state+0x55/0x190
[12525.415824] [<ffffffff8165f9a7>] cpuidle_enter+0x17/0x20
[12525.416523] [<ffffffff810bd665>] cpu_startup_entry+0x355/0x410
[12525.417216] [<ffffffff8103015a>] start_secondary+0x1aa/0x230
[12525.417904] Code: 31 d2 65 48 8b 34 25 08 aa 00 00 48 8d 86 38 c0 ff ff 48 89 d1 0f 01 c8 48 8b 86 38 c0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <65> 48 8b 0c 25 08 aa 00 00 f0 80 a1 3a c0 ff ff df 0f ae f0 48
[12525.419435] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 71.790 msecs
[12549.296847] NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [trinity-c60:28574]
[12549.297585] Modules linked in: bridge 8021q garp stp snd_seq_dummy dlci tun fuse rfcomm hidp bnep af_key llc2 nfnetlink can_bcm scsi_transport_iscsi can_raw sctp libcrc32c nfc caif_socket caif af_802154 ieee802154 phonet af_rxrpc bluetooth can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 usb_debug cfg80211 rfkill coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode serio_raw pcspkr snd_hda_intel snd_hda_controller snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp pps_core snd_timer snd soundcore shpchp nfsd auth_rpcgss oid_registry nfs_acl lockd grace sunrpc
[12549.302426] CPU: 2 PID: 28574 Comm: trinity-c60 Tainted: G L 3.18.0+ #106
[12549.304874] task: ffff880153a20000 ti: ffff8801c251c000 task.ti: ffff8801c251c000
[12549.305730] RIP: 0010:[<ffffffff81373805>] [<ffffffff81373805>] copy_user_enhanced_fast_string+0x5/0x10
[12549.306609] RSP: 0018:ffff8801c251fbe0 EFLAGS: 00010206
[12549.307502] RAX: 00007f9c44b07f1d RBX: 0000000000000003 RCX: 00000000000009c0
[12549.308400] RDX: 0000000000001000 RSI: 00007f9c44b0855d RDI: ffff88007cb1f640
[12549.309284] RBP: ffff8801c251fc28 R08: 0000000000000000 R09: 0000000000000001
[12549.310159] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff817cf22b
[12549.311018] R13: ffff8801c251fb68 R14: ffff880061dccd40 R15: ffff880061dcce60
[12549.311862] FS: 00007f9c4714c740(0000) GS:ffff880245200000(0000) knlGS:0000000000000000
[12549.312716] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12549.313581] CR2: 00007f9c45b905b8 CR3: 000000022a30b000 CR4: 00000000001407e0
[12549.314418] DR0: 00007f6c9030b000 DR1: 0000000000000000 DR2: 0000000000000000
[12549.315239] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[12549.316051] Stack:
[12549.316861] ffffffff8119fe56 ffff8801c251fbf8 140000010718f000 0000000000001000
[12549.317679] 140000010718f000 0000000000001000 ffff8801c251fd60 0000000000000000
[12549.318514] ffff880061dcd070 ffff8801c251fcc8 ffffffff81175fe7 ffff8801c251fc88
[12549.319338] Call Trace:
[12549.320153] [<ffffffff8119fe56>] ? iov_iter_copy_from_user_atomic+0x156/0x180
[12549.320988] [<ffffffff81175fe7>] generic_perform_write+0xf7/0x1f0
[12549.321824] [<ffffffff81178722>] __generic_file_write_iter+0x162/0x350
[12549.322658] [<ffffffff811e7940>] ? new_sync_read+0xd0/0xd0
[12549.323484] [<ffffffff8117894f>] generic_file_write_iter+0x3f/0xb0
[12549.324306] [<ffffffff81178910>] ? __generic_file_write_iter+0x350/0x350
[12549.325133] [<ffffffff811e7a88>] do_iter_readv_writev+0x78/0xc0
[12549.325961] [<ffffffff811e92b8>] do_readv_writev+0xd8/0x2a0
[12549.326783] [<ffffffff81178910>] ? __generic_file_write_iter+0x350/0x350
[12549.327646] [<ffffffff81178910>] ? __generic_file_write_iter+0x350/0x350
[12549.328470] [<ffffffff810c361f>] ? lock_release_holdtime.part.24+0xf/0x190
[12549.329279] [<ffffffff817cf310>] ? _raw_spin_unlock_irq+0x30/0x40
[12549.330078] [<ffffffff811e950c>] vfs_writev+0x3c/0x50
[12549.330874] [<ffffffff811e98d2>] SyS_pwritev+0xc2/0xf0
[12549.331672] [<ffffffff817cff12>] system_call_fastpath+0x12/0x17
[12549.332471] Code: 48 ff c6 48 ff c7 ff c9 75 f2 89 d1 c1 e9 03 83 e2 07 f3 48 a5 89 d1 f3 a4 31 c0 0f 1f 00 c3 0f 1f 80 00 00 00 00 0f 1f 00 89 d1 <f3> a4 31 c0 0f 1f 00 c3 90 90 90 0f 1f 00 83 fa 08 0f 82 95 00
[12549.334222] sending NMI to other CPUs:
[12549.335023] NMI backtrace for cpu 3
[12549.335811] CPU: 3 PID: 25560 Comm: trinity-c201 Tainted: G L 3.18.0+ #106
[12549.338137] task: ffff880095342da0 ti: ffff8801c048c000 task.ti: ffff8801c048c000
[12549.338904] RIP: 0010:[<ffffffff810fbc4e>] [<ffffffff810fbc4e>] generic_exec_single+0xee/0x1b0
[12549.339670] RSP: 0018:ffff8801c048f9f8 EFLAGS: 00000202
[12549.340416] RAX: 0000000000000008 RBX: ffff8801c048fa10 RCX: 0000000000000038
[12549.341157] RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000000
[12549.341879] RBP: ffff8801c048fa58 R08: ffff8802437418f0 R09: 0000000000000000
[12549.342587] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000002
[12549.343280] R13: 0000000000000001 R14: ffff880061e34fc0 R15: 0000000000000003
[12549.343961] FS: 00007f9c4714c740(0000) GS:ffff880245400000(0000) knlGS:0000000000000000
[12549.344639] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12549.345309] CR2: 0000000001ac9498 CR3: 00000002247cb000 CR4: 00000000001407e0
[12549.345981] DR0: 00007f6c9030b000 DR1: 0000000000000000 DR2: 0000000000000000
[12549.346649] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[12549.347313] Stack:
[12549.347969] 0000000000000000 0000000000000000 ffff8801c048fa78 0000000000000000
[12549.348648] ffffffff81048cd0 ffff8801c048fb08 0000000000000003 00000000639bd7ce
[12549.349325] ffff880095342da0 00000000ffffffff 0000000000000002 ffffffff81048cd0
[12549.349997] Call Trace:
[12549.350653] [<ffffffff81048cd0>] ? do_flush_tlb_all+0x60/0x60
[12549.351319] [<ffffffff81048cd0>] ? do_flush_tlb_all+0x60/0x60
[12549.351983] [<ffffffff810fbdb0>] smp_call_function_single+0x70/0xd0
[12549.352644] [<ffffffff81048cd0>] ? do_flush_tlb_all+0x60/0x60
[12549.353298] [<ffffffff810fc4a9>] smp_call_function_many+0x2b9/0x320
[12549.353954] [<ffffffff811a396d>] ? unmap_single_vma+0x50d/0x900
[12549.354604] [<ffffffff81049020>] flush_tlb_mm_range+0x90/0x1d0
[12549.355245] [<ffffffff811a25b2>] tlb_flush_mmu_tlbonly+0x42/0x50
[12549.355881] [<ffffffff811a2bac>] tlb_flush_mmu+0x1c/0x30
[12549.356508] [<ffffffff811a2bd4>] tlb_finish_mmu+0x14/0x40
[12549.357131] [<ffffffff811a3e78>] zap_page_range_single+0x118/0x160
[12549.357756] [<ffffffff811a4044>] unmap_mapping_range+0x134/0x190
[12549.358379] [<ffffffff81192a5d>] shmem_fallocate+0x4fd/0x520
[12549.359001] [<ffffffff810bcb77>] ? prepare_to_wait+0x27/0x90
[12549.359620] [<ffffffff811e6062>] do_fallocate+0x132/0x1d0
[12549.360239] [<ffffffff811b8558>] SyS_madvise+0x398/0x870
[12549.360855] [<ffffffff817cff12>] system_call_fastpath+0x12/0x17
[12549.361476] Code: 48 89 de 48 03 14 c5 a0 f2 d1 81 48 89 df e8 fa e0 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 18 01 74 0b 0f 1f 00 f3 90 f6 43 18 01 <75> f8 31 c0 48 8b 7d d8 65 48 33 3c 25 28 00 00 00 0f 85 98 00
[12549.362876] NMI backtrace for cpu 1
[12549.363530] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G L 3.18.0+ #106
[12549.365548] task: ffff8802434b96d0 ti: ffff880243748000 task.ti: ffff880243748000
[12549.366222] RIP: 0010:[<ffffffff813d6cab>] [<ffffffff813d6cab>] intel_idle+0xdb/0x180
[12549.366902] RSP: 0018:ffff88024374be08 EFLAGS: 00000046
[12549.367578] RAX: 0000000000000032 RBX: 0000000000000010 RCX: 0000000000000001
[12549.368258] RDX: 0000000000000000 RSI: ffff88024374bfd8 RDI: 0000000000000001
[12549.368920] RBP: ffff88024374be38 R08: 000000008baf89f8 R09: 0000000000000000
[12549.369565] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000005
[12549.370201] R13: 0000000000000032 R14: 0000000000000004 R15: ffff880243748000
[12549.370828] FS: 0000000000000000(0000) GS:ffff880245000000(0000) knlGS:0000000000000000
[12549.371451] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12549.372069][12563.088868] INFO: rcu_sched self-detected stall on CPU
[12563.089596] 2: (1 GPs behind) idle=d4b/140000000000001/0 softirq=781780/781781
[12563.090304] (t=6000 jiffies g=382738 c=382737 q=0)
[12563.090996] Task dump for CPU 2:
[12563.091676] trinity-c60 R running task 12768 28574 19163 0x0000000c
[12563.092378] ffff880153a20000 000000005c7b7872 ffff880245203d68 ffffffff810a743c
[12563.093085] ffffffff810a73a2 0000000000000002 0000000000000004 0000000000000002
[12563.093810] ffffffff81c52380 0000000000000092 ffff880245203d88 ffffffff810ab4ed
[12563.094507] Call Trace:
[12563.095197] <IRQ> [<ffffffff810a743c>] sched_show_task+0x11c/0x190
[12563.095910] [<ffffffff810a73a2>] ? sched_show_task+0x82/0x190
[12563.096636] [<ffffffff810ab4ed>] dump_cpu_task+0x3d/0x50
[12563.097336] [<ffffffff810d9b90>] rcu_dump_cpu_stacks+0x90/0xd0
[12563.098019] [<ffffffff810e0723>] rcu_check_callbacks+0x503/0x770
[12563.098712] [<ffffffff8113058c>] ? acct_account_cputime+0x1c/0x20
[12563.099408] [<ffffffff810abe27>] ? account_system_time+0x97/0x180
[12563.100022] [<ffffffff810e63fb>] update_process_times+0x4b/0x80
[12563.100623] [<ffffffff810f6db3>] ? tick_sched_timer+0x23/0x1b0
[12563.101214] [<ffffffff810f6ddf>] tick_sched_timer+0x4f/0x1b0
[12563.101795] [<ffffffff810e729f>] __run_hrtimer+0xaf/0x240
[12563.102370] [<ffffffff810e75ab>] ? hrtimer_interrupt+0x8b/0x260
[12563.102941] [<ffffffff810f6d90>] ? tick_init_highres+0x20/0x20
[12563.103514] [<ffffffff810e7627>] hrtimer_interrupt+0x107/0x260
[12563.104086] [<ffffffff81031e9b>] local_apic_timer_interrupt+0x3b/0x70
[12563.104660] [<ffffffff817d29c5>] smp_apic_timer_interrupt+0x45/0x60
[12563.105234] [<ffffffff817d0daf>] apic_timer_interrupt+0x6f/0x80
[12563.105805] <EOI> [<ffffffff81373805>] ? copy_user_enhanced_fast_string+0x5/0x10
[12563.106387] [<ffffffff8119fe56>] ? iov_iter_copy_from_user_atomic+0x156/0x180
[12563.106965] [<ffffffff81175fe7>] generic_perform_write+0xf7/0x1f0
[12563.107545] [<ffffffff81178722>] __generic_file_write_iter+0x162/0x350
[12563.108128] [<ffffffff811e7940>] ? new_sync_read+0xd0/0xd0
[12563.108705] [<ffffffff8117894f>] generic_file_write_iter+0x3f/0xb0
[12563.109299] [<ffffffff81178910>] ? __generic_file_write_iter+0x350/0x350
[12563.109869] [<ffffffff811e7a88>] do_iter_readv_writev+0x78/0xc0
[12563.110440] [<ffffffff811e92b8>] do_readv_writev+0xd8/0x2a0
[12563.111003] [<ffffffff81178910>] ? __generic_file_write_iter+0x350/0x350
[12563.111566] [<ffffffff81178910>] ? __generic_file_write_iter+0x350/0x350
[12563.112125] [<ffffffff810c361f>] ? lock_release_holdtime.part.24+0xf/0x190
[12563.112686] [<ffffffff817cf310>] ? _raw_spin_unlock_irq+0x30/0x40
[12563.113248] [<ffffffff811e950c>] vfs_writev+0x3c/0x50
[12563.113810] [<ffffffff811e98d2>] SyS_pwritev+0xc2/0xf0
[12563.114373] [<ffffffff817cff12>] system_call_fastpath+0x12/0x17
[12577.290614] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [trinity-c201:25560]
[12577.291259] Modules linked in: bridge 8021q garp stp snd_seq_dummy dlci tun fuse rfcomm hidp bnep af_key llc2 nfnetlink can_bcm scsi_transport_iscsi can_raw sctp libcrc32c nfc caif_socket caif af_802154 ieee802154 phonet af_rxrpc bluetooth can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 usb_debug cfg80211 rfkill coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode serio_raw pcspkr snd_hda_intel snd_hda_controller snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp pps_core snd_timer snd soundcore shpchp nfsd auth_rpcgss oid_registry nfs_acl lockd grace sunrpc
[12577.295639] CPU: 3 PID: 25560 Comm: trinity-c201 Tainted: G L 3.18.0+ #106
[12577.297656] task: ffff880095342da0 ti: ffff8801c048c000 task.ti: ffff8801c048c000
[12577.298367] RIP: 0010:[<ffffffff810fbc4a>] [<ffffffff810fbc4a>] generic_exec_single+0xea/0x1b0
[12577.299091] RSP: 0018:ffff8801c048f9f8 EFLAGS: 00000202
[12577.299825] RAX: 0000000000000008 RBX: ffffffff817d0ae0 RCX: 0000000000000038
[12577.300553] RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000000
[12577.301296] RBP: ffff8801c048fa58 R08: ffff8802437418f0 R09: 0000000000000000
[12577.302007] R10: 0000000000000000 R11: 0000000000000000 R12: ffff8801c048f968
[12577.302726] R13: 0000000000001a4a R14: ffff8801c048c000 R15: ffff880095342da0
[12577.303444] FS: 00007f9c4714c740(0000) GS:ffff880245400000(0000) knlGS:0000000000000000
[12577.304169] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12577.304897] CR2: 0000000001ac9498 CR3: 00000002247cb000 CR4: 00000000001407e0
[12577.305636] DR0: 00007f6c9030b000 DR1: 0000000000000000 DR2: 0000000000000000
[12577.306386] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[12577.307114] Stack:
[12577.307830] 0000000000000000 0000000000000000 ffff8801c048fa78 0000000000000000
[12577.308585] ffffffff81048cd0 ffff8801c048fb08 0000000000000003 00000000639bd7ce
[12577.309355] ffff880095342da0 00000000ffffffff 0000000000000002 ffffffff81048cd0
[12577.310106] Call Trace:
[12577.310898] [<ffffffff81048cd0>] ? do_flush_tlb_all+0x60/0x60
[12577.311664] [<ffffffff81048cd0>] ? do_flush_tlb_all+0x60/0x60
[12577.312440] [<ffffffff810fbdb0>] smp_call_function_single+0x70/0xd0
[12577.313196] [<ffffffff81048cd0>] ? do_flush_tlb_all+0x60/0x60
[12577.313935] [<ffffffff810fc4a9>] smp_call_function_many+0x2b9/0x320
[12577.314684] [<ffffffff811a396d>] ? unmap_single_vma+0x50d/0x900
[12577.315429] [<ffffffff81049020>] flush_tlb_mm_range+0x90/0x1d0
[12577.316182] [<ffffffff811a25b2>] tlb_flush_mmu_tlbonly+0x42/0x50
[12577.316913] [<ffffffff811a2bac>] tlb_flush_mmu+0x1c/0x30
[12577.317664] [<ffffffff811a2bd4>] tlb_finish_mmu+0x14/0x40
[12577.318399] [<ffffffff811a3e78>] zap_page_range_single+0x118/0x160
[12577.319123] [<ffffffff811a4044>] unmap_mapping_range+0x134/0x190
[12577.319871] [<ffffffff81192a5d>] shmem_fallocate+0x4fd/0x520
[12577.320624] [<ffffffff810bcb77>] ? prepare_to_wait+0x27/0x90
[12577.321411] [<ffffffff811e6062>] do_fallocate+0x132/0x1d0
[12577.322149] [<ffffffff811b8558>] SyS_madvise+0x398/0x870
[12577.322868] [<ffffffff817cff12>] system_call_fastpath+0x12/0x17
[12577.323617] Code: c0 3a 1d 00 48 89 de 48 03 14 c5 a0 f2 d1 81 48 89 df e8 fa e0 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 18 01 74 0b 0f 1f 00 f3 90 <f6> 43 18 01 75 f8 31 c0 48 8b 7d d8 65 48 33 3c 25 28 00 00 00
[12577.325246] sending NMI to other CPUs:
[12577.325989] NMI backtrace for cpu 2
[12577.326717] CPU: 2 PID: 28574 Comm: trinity-c60 Tainted: G L 3.18.0+ #106
[12577.328845] task: ffff880153a20000 ti: ffff8801c251c000 task.ti: ffff8801c251c000
[12577.329548] RIP: 0010:[<ffffffff810c523d>] [<ffffffff810c523d>] __lock_acquire.isra.31+0x33d/0x9f0
[12577.330250] RSP: 0018:ffff880245203d88 EFLAGS: 00000097
[12577.330933] RAX: 0000000000000000 RBX: ffff880153a20000 RCX: ffff8802453cff98
[12577.331610] RDX: 0000000000000980 RSI: 0000000000000010 RDI: 0000000000000000
[12577.332272] RBP: ffff880245203df8 R08: 0000000000000001 R09: 0000000000000000
[12577.332924] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000097
[12577.333564] R13: 0000000000000002 R14: ffff8802453cc898 R15: ffff880153a207e0
[12577.334195] FS: 00007f9c4714c740(0000) GS:ffff880245200000(0000) knlGS:0000000000000000
[12577.334831] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12577.335469] CR2: 00007f9c45b905b8 CR3: 000000022a30b000 CR4: 00000000001407e0
[12577.336116] DR0: 00007f6c9030b000 DR1: 0000000000000000 DR2: 0000000000000000
[12577.336759] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[12577.337399] Stack:
[12577.338024] ffff880153a20000 0000000000000046 ffff880245203dd8 0000000000000092
[12577.338671] ffff8802453d2f18 ffff8802453d2f00 0000000000000002 0000000000000000
[12577.339313] ffff880245203dc8 0000000000000046 0000000000000000 0000000000000000
[12577.339955] Call Trace:
[12577.340585] <IRQ>
[12577.340593] [<ffffffff810c5fff>] lock_acquire+0x9f/0x120
[12577.341830] [<ffffffff810e72c6>] ? __run_hrtimer+0xd6/0x240
[12577.342450] [<ffffffff810f6db3>] ? tick_sched_timer+0x23/0x1b0
[12577.343065] [<ffffffff817cee9e>] _raw_spin_lock+0x3e/0x80
[12577.343673] [<ffffffff810e72c6>] ? __run_hrtimer+0xd6/0x240
[12577.344277] [<ffffffff810e72c6>] __run_hrtimer+0xd6/0x240
[12577.344886] [<ffffffff810e75ab>] ? hrtimer_interrupt+0x8b/0x260
[12577.345493] [<ffffffff810f6d90>] ? tick_init_highres+0x20/0x20
[12577.346099] [<ffffffff810e7627>] hrtimer_interrupt+0x107/0x260
[12577.346703] [<ffffffff81031e9b>] local_apic_timer_interrupt+0x3b/0x70
[12577.347309] [<ffffffff817d29c5>] smp_apic_timer_interrupt+0x45/0x60
[12577.347916] [<ffffffff817d0daf>] apic_timer_interrupt+0x6f/0x80
[12577.348522] <EOI>
[12577.348529] [<ffffffff81373805>] ? copy_user_enhanced_fast_string+0x5/0x10
[12577.349750] [<ffffffff8119fe56>] ? iov_iter_copy_from_user_atomic+0x156/0x180
[12577.350375] [<ffffffff81175fe7>] generic_perform_write+0xf7/0x1f0
[12577.351001] [<ffffffff81178722>] __generic_file_write_iter+0x162/0x350
[12577.351626] [<ffffffff811e7940>] ? new_sync_read+0xd0/0xd0
[12577.352253] [<ffffffff8117894f>] generic_file_write_iter+0x3f/0xb0
[12577.352883] [<ffffffff81178910>] ? __generic_file_write_iter+0x350/0x350
[12577.353515] [<ffffffff811e7a88>] do_iter_readv_writev+0x78/0xc0
[12577.354147] [<ffffffff811e92b8>] do_readv_writev+0xd8/0x2a0
[12577.354763] [<ffffffff81178910>] ? __generic_file_write_iter+0x350/0x350
[12577.355366] [<ffffffff81178910>] ? __generic_file_write_iter+0x350/0x350
[12577.355956] [<ffffffff810c361f>] ? lock_release_holdtime.part.24+0xf/0x190
[12577.356550] [<ffffffff817cf310>] ? _raw_spin_unlock_irq+0x30/0x40
[12577.357141] [<ffffffff811e950c>] vfs_writev+0x3c/0x50
[12577.357713] [<ffffffff811e98d2>] SyS_pwritev+0xc2/0xf0
[12577.358267] [<ffffffff817cff12>] system_call_fastpath+0x12/0x17
[12577.358815] Code: 00 00 48 83 c4 48 44 89 e8 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 2e 0f 1f 84 00 00 00 00 00 41 81 fc fe 1f 00 00 0f 87 34 04 00 00 <45> 85 ed 48 8b 83 60 07 00 00 75 09 48 85 c0 0f 85 df 03 00 00
[12577.360038] NMI backtrace for cpu 0
[12577.360598] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G L 3.18.0+ #106
[12577.381697] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G L 3.18.0+ #106
[12577.383862] task: ffff8802434b96d0 ti: ffff880243748000 task.ti: ffff880243748000
[12577.384585] RIP: 0010:[<ffffffff813d6cab>] [<ffffffff813d6cab>] intel_idle+0xdb/0x180
[12577.385332] RSP: 0018:ffff88024374be08 EFLAGS: 00000046
[12577.386066] RAX: 0000000000000032 RBX: 0000000000000010 RCX: 0000000000000001
[12577.386807] RDX: 0000000000000000 RSI: ffff88024374bfd8 RDI: 0000000000000001
[12577.387545] RBP: ffff88024374be38 R08: 000000008baf89e1 R09: 0000000000000000
[12577.388285] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000005
[12577.389028] R13: 0000000000000032 R14: 0000000000000004 R15: ffff880243748000
[12577.389752] FS: 0000000000000000(0000) GS:ffff880245000000(0000) knlGS:0000000000000000
[12577.390463] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12577.391170] CR2: 00007f5271c07000 CR3: 0000000001c11000 CR4: 00000000001407e0
[12577.391888] DR0: 00007f6c9030b000 DR1: 0000000000000000 DR2: 0000000000000000
[12577.392584] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[12577.393258] Stack:
[12577.393915] 000000014374be38 b532cae23f43ed20 ffffe8ffff003200 0000000000000005
[12577.394595] ffffffff81cb1bc0 0000000000000001 ffff88024374be88 ffffffff8165f7b5
[12577.395278] 00000b72149ea4e1 ffffffff81cb1d90 ffffffff81cb1bc0 ffffffff81d215f0
[12577.395972] Call Trace:
[12577.396643] [<ffffffff8165f7b5>] cpuidle_enter_state+0x55/0x190
[12577.397327] [<ffffffff8165f9a7>] cpuidle_enter+0x17/0x20
[12577.398010] [<ffffffff810bd665>] cpu_startup_entry+0x355/0x410
[12577.398691] [<ffffffff8103015a>] start_secondary+0x1aa/0x230
[12577.399364] Code: 31 d2 65 48 8b 34 25 08 aa 00 00 48 8d 86 38 c0 ff ff 48 89 d1 0f 01 c8 48 8b 86 38 c0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <65> 48 8b 0c 25 08 aa 00 00 f0 80 a1 3a c0 ff ff df 0f ae f0 48
[12577.400864] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 74.851 msecs
[12591.532403] sched: RT throttling activated
[12613.269774] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 21s! [trinity-c12:30629]
[12613.270528] Modules linked in: bridge 8021q garp stp snd_seq_dummy dlci tun fuse rfcomm hidp bnep af_key llc2 nfnetlink can_bcm scsi_transport_iscsi can_raw sctp libcrc32c nfc caif_socket caif af_802154 ieee802154 phonet af_rxrpc bluetooth can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 usb_debug cfg80211 rfkill coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode serio_raw pcspkr snd_hda_intel snd_hda_controller snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp pps_core snd_timer snd soundcore shpchp nfsd auth_rpcgss oid_registry nfs_acl lockd grace sunrpc
[12613.275557] CPU: 3 PID: 30629 Comm: trinity-c12 Tainted: G L 3.18.0+ #106
[12613.278212] task: ffff8802433b5b40 ti: ffff880180f34000 task.ti: ffff880180f34000
[12613.279152] RIP: 0010:[<ffffffff8117e2df>] [<ffffffff8117e2df>] free_hot_cold_page+0x11f/0x1a0
[12613.280117] RSP: 0018:ffff880180f37b38 EFLAGS: 00000202
[12613.281030] RAX: 0000000000000003 RBX: 0000000000000002 RCX: 0000000000000240
[12613.281939] RDX: ffff88024540d380 RSI: 0000000000000000 RDI: ffff88024e5d3f40
[12613.282817] RBP: ffff880180f37b78 R08: 0000000000000000 R09: 000000000024e600
[12613.283701] R10: 0000000000000002 R11: 0000000000000000 R12: ffff88024e5d3f40
[12613.284572] R13: 0000000180f37b08 R14: ffffffff8117df5b R15: ffff880180f37b28
[12613.285454] FS: 00007f688fc33740(0000) GS:ffff880245400000(0000) knlGS:0000000000000000
[12613.286408] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12613.287338] CR2: 00007f688baa9000 CR3: 00000002276f9000 CR4: 00000000001407e0
[12613.288228] DR0: 00007f688e6a9000 DR1: 0000000000000000 DR2: 0000000000000000
[12613.289102] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000070602
[12613.289957] Stack:
[12613.290804] 00000000001ddf58 ffff88024e5d3e80 ffff8802433b5b40 ffffea00076a8540
[12613.291675] ffffea000777d600 0000000000000000 ffffea00062ec580 ffff880180f37c00
[12613.292574] ffff880180f37bb8 ffffffff8117e3c5 000000000000000d ffffea00076a8540
[12613.293471] Call Trace:
[12613.294325] [<ffffffff8117e3c5>] free_hot_cold_page_list+0x65/0xd0
[12613.295179] [<ffffffff81184f2d>] release_pages+0x1bd/0x270
[12613.296027] [<ffffffff81185f63>] __pagevec_release+0x43/0x60
[12613.296869] [<ffffffff81191f80>] shmem_undo_range+0x460/0x710
[12613.297704] [<ffffffff81192248>] shmem_truncate_range+0x18/0x40
[12613.298541] [<ffffffff811924d6>] shmem_setattr+0x116/0x1a0
[12613.299383] [<ffffffff812075d1>] notify_change+0x241/0x390
[12613.300250] [<ffffffff811e5ad5>] do_truncate+0x75/0xc0
[12613.301103] [<ffffffff811e5e4a>] ? do_sys_ftruncate.constprop.14+0xda/0x160
[12613.301962] [<ffffffff811e5e7f>] do_sys_ftruncate.constprop.14+0x10f/0x160
[12613.302815] [<ffffffff81374f6e>] ? trace_hardirqs_on_thunk+0x3a/0x3f
[12613.303656] [<ffffffff811e5f0e>] SyS_ftruncate+0xe/0x10
[12613.304497] [<ffffffff817cff12>] system_call_fastpath+0x12/0x17
[12613.305362] Code: 8d 7b 20 e8 64 3b 20 00 41 8b 04 24 83 c0 01 41 3b 44 24 04 41 89 04 24 7d 52 41 f7 c6 00 02 00 00 74 31 e8 94 bf fc ff 41 56 9d <48> 83 c4 18 5b 41 5c 41 5d 41 5e 41 5f 5d c3 66 90 49 8d 34 04
[12613.307230] sending NMI to other CPUs:
[12613.308088] NMI backtrace for cpu 2
[12613.308871] CPU: 2 PID: 30778 Comm: trinity-c161 Tainted: G L 3.18.0+ #106
[12613.311213] task: ffff8800478eada0 ti: ffff8802251dc000 task.ti: ffff8802251dc000
[12613.311995] RIP: 0010:[<ffffffff810c96bd>] [<ffffffff810c96bd>] do_raw_spin_trylock+0x2d/0x50
[12613.312776] RSP: 0018:ffff8802251dfd88 EFLAGS: 00000246
[12613.313536] RAX: 0000000000008f8f RBX: ffff880235610238 RCX: 0000000000000000
[12613.314287] RDX: 0000000000008f8f RSI: 000000000000908f RDI: ffff880235610238
[12613.315022] RBP: ffff8802251dfd88 R08: 0000000000000000 R09: 0000000000000000
[12613.315743] R10: 0000000000000000 R11: 0000000000000000 R12: ffff880235610250
[12613.316446] R13: ffff88009f3cc640 R14: 0000000000000000 R15: ffff8802356103c0
[12613.317133] FS: 00007f688fc33740(0000) GS:ffff880245200000(0000) knlGS:0000000000000000
[12613.317813] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12613.318483] CR2: 000000288076098d CR3: 000000022a0d2000 CR4: 00000000001407e0
[12613.319155] DR0: 00007f688e6a9000 DR1: 0000000000000000 DR2: 0000000000000000
[12613.319819] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000070602
[12613.320475] Stack:
[12613.321122] ffff8802251dfdb8 ffffffff817ceea6 ffffffff81214c6a ffffffff817cf22b
[12613.321796] ffff8802356101b0 ffff880235610238 ffff8802251dff08 ffffffff81214c6a
[12613.322474] 00000000251dfde8 7fffffffffffffff ffff88009f3cc520 ffff88017b9a3c28
[12613.323153] Call Trace:
[12613.323815] [<ffffffff817ceea6>] _raw_spin_lock+0x46/0x80
[12613.324479] [<ffffffff81214c6a>] ? sync_inodes_sb+0x1ca/0x2b0
[12613.325138] [<ffffffff817cf22b>] ? _raw_spin_unlock+0x2b/0x40
[12613.325798] [<ffffffff81214c6a>] sync_inodes_sb+0x1ca/0x2b0
[12613.326458] [<ffffffff817cb01f>] ? wait_for_completion+0xff/0x130
[12613.327114] [<ffffffff8121c560>] ? vfs_fsync+0x40/0x40
[12613.327761] [<ffffffff8121c579>] sync_inodes_one_sb+0x19/0x20
[12613.328404] [<ffffffff811ec002>] iterate_supers+0xb2/0x110
[12613.329041] [<ffffffff8121c824>] sys_sync+0x44/0xb0
[12613.329669] [<ffffffff817cff12>] system_call_fastpath+0x12/0x17
[12613.330292] Code: 44 00 00 0f b7 17 55 31 c9 48 89 e5 38 d6 74 0e 89 c8 5d c3 66 2e 0f 1f 84 00 00 00 00 00 8d b2 00 01 00 00 89 d0 f0 66 0f b1 37 <66> 39 d0 75 e0 b1 01 5d 65 8b 04 25 2c a0 00 00 89 47 08 65 48
[12613.331704] NMI backtrace for cpu 0
[12613.332380] CPU: 0 PID: 1460 Comm: trinity-c155 Tainted: G L 3.18.0+ #106
[12613.334512] task: ffff880232a72da0 ti: ffff880232820000 task.ti: ffff880232820000
[12613.335239] RIP: 0033:[<000000336ee93d7d>] [<000000336ee93d7d>] 0x336ee93d7d
[12613.335984] RSP: 002b:00007fff9d870a18 EFLAGS: 00000206
[12613.336708] RAX: ffffff68f0ef8610 RBX: 0000000000000200 RCX: 00007fff9d870f00
[12613.337440] RDX: 00007fff9d871a00 RSI: 00007f688e769020 RDI: 00007fff9d870a20
[12613.338173] RBP: 00007fff9d871a60 R08: ffffff68f0ef8600 R09: ffffff68f0ef85f0
[12613.338905] R10: ffffff68f0ef85e0 R11: 0000000000000246 R12: 0000000000000129
[12613.339627] R13: 0000000000001000 R14: 0000000000001000 R15: 00007f688e6a9000
[12613.340332] FS: 00007f688fc33740(0000) GS:ffff880244e00000(0000) knlGS:0000000000000000
[12613.341040] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12613.341753] CR2: 00007f688e702000 CR3: 00000001c9675000 CR4: 00000000001407f0
[12613.342468] DR0: 00007f688e6a9000 DR1: 0000000000000000 DR2: 0000000000000000
[12613.343161] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000070602
[12613.343836]
[12613.344498] NMI backtrace for cpu 1
[12613.345187] CPU: 1 PID: 30783 Comm: trinity-c166 Tainted: G L 3.18.0+ #106
[12613.347247] task: ffff8801c059ada0 ti: ffff880227058000 task.ti: ffff880227058000
[12613.347978] RIP: 0033:[<000000336ee39bc0>] [<000000336ee39bc0>] 0x336ee39bc0
[12613.348715] RSP: 002b:00007fff9d8719d8 EFLAGS: 00000202
[12613.349440] RAX: e8ffff3573f747e9 RBX: 00007f688f22e000 RCX: 0000000000000000
[12613.350175] RDX: 8000000000000000 RSI: 00007fff9d8719cc RDI: 000000336f1b76e0
[12613.350903] RBP: 00007f688f22e068 R08: 000000336f1b7120 R09: 000000336f1b7140
[12613.351639] R10: ffffffffffff9f00 R11: 0000000000000202 R12: 00007f688fbb7528
[12613.352362] R13: 00007f688f22e068 R14: 0000000000000000 R15: 0000000000000000
[12613.353089] FS: 00007f688fc33740(0000) GS:ffff880245000000(0000) knlGS:0000000000000000
[12613.353814] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12613.354537] CR2: 0000000000000008 CR3: 0000000153ac1000 CR4: 00000000001407e0
[12613.355265] DR0: 00007f688e6a9000 DR1: 0000000000000000 DR2: 0000000000000000
[12613.355989] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000070602
[12613.356701]
[12641.253566] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [trinity-main:30108]
[12641.254492] Modules linked in: bridge 8021q garp stp snd_seq_dummy dlci tun fuse rfcomm hidp bnep af_key llc2 nfnetlink can_bcm scsi_transport_iscsi can_raw sctp libcrc32c nfc caif_socket caif af_802154 ieee802154 phonet af_rxrpc bluetooth can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 usb_debug cfg80211 rfkill coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode serio_raw pcspkr snd_hda_intel snd_hda_controller snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp pps_core snd_timer snd soundcore shpchp nfsd auth_rpcgss oid_registry nfs_acl lockd grace sunrpc
[12641.260655] CPU: 3 PID: 30108 Comm: trinity-main Tainted: G L 3.18.0+ #106
[12641.263755] task: ffff8800a0d15b40 ti: ffff880234ec4000 task.ti: ffff880234ec4000
[12641.264753] RIP: 0010:[<ffffffff811729d7>] [<ffffffff811729d7>] task_bp_pinned.isra.6.constprop.12+0x37/0x90
[12641.265768] RSP: 0018:ffff880234ec7a40 EFLAGS: 00000287
[12641.266763] RAX: ffff88023f199d18 RBX: 000000000000005c RCX: 00000000001ce490
[12641.267751] RDX: ffff8801c87c2fa8 RSI: ffff8801ad860000 RDI: 0000000000000007
[12641.268728] RBP: ffff880234ec7a70 R08: ffffffff81d215f8 R09: ffff880245dce490
[12641.269700] R10: 0000000000000000 R11: 0000000000000008 R12: ffff880234ec7a30
[12641.270657] R13: ffffffff810abaf5 R14: ffff880234ec79b0 R15: ffffea000252c000
[12641.271618] FS: 00007f688fc33740(0000) GS:ffff880245400000(0000) knlGS:0000000000000000
[12641.272585] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12641.273547] CR2: 000000336f1b8490 CR3: 0000000229035000 CR4: 00000000001407e0
[12641.274502] DR0: 00007f688e6a9000 DR1: 0000000000000000 DR2: 0000000000000000
[12641.275443] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 000000000017060a
[12641.276381] Stack:
[12641.277317] 0000000000000002 ffff8800a0d15b40 ffff880234ec7aa0 0000000000000001
[12641.278267] ffff880094b06618 0000000000000007 ffff880234ec7ad0 ffffffff81172af6
[12641.279214] ffffffff81c64868 0000000100000246 00000000001ce490 ffffffff81d215f8
[12641.280161] Call Trace:
[12641.281101] [<ffffffff81172af6>] toggle_bp_slot.constprop.7+0xc6/0x1b0
[12641.282055] [<ffffffff81172dfb>] __reserve_bp_slot+0x1eb/0x230
[12641.283017] [<ffffffff81172e77>] reserve_bp_slot+0x27/0x40
[12641.283972] [<ffffffff81172f68>] register_perf_hw_breakpoint+0x18/0x60
[12641.284884] [<ffffffff81172fdf>] hw_breakpoint_event_init+0x2f/0x50
[12641.285766] [<ffffffff8116fcfb>] perf_init_event+0x17b/0x1e0
[12641.286650] [<ffffffff8116fb80>] ? perf_bp_event+0xd0/0xd0
[12641.287512] [<ffffffff811700f8>] perf_event_alloc+0x398/0x440
[12641.288373] [<ffffffff810e3c1c>] ? do_init_timer+0x5c/0x60
[12641.289239] [<ffffffff81170e3e>] inherit_event.isra.90+0x8e/0x260
[12641.290095] [<ffffffff8117104e>] inherit_task_group.isra.92.part.93+0x3e/0xe0
[12641.290959] [<ffffffff811716e3>] perf_event_init_task+0x163/0x2e0
[12641.291812] [<ffffffff81075756>] copy_process.part.26+0x726/0x1a40
[12641.292664] [<ffffffff810abaf5>] ? local_clock+0x25/0x30
[12641.293514] [<ffffffff810c361f>] ? lock_release_holdtime.part.24+0xf/0x190
[12641.294347] [<ffffffff810424b2>] ? __do_page_fault+0x232/0x610
[12641.295158] [<ffffffff81076c37>] do_fork+0xe7/0x490
[12641.295969] [<ffffffff811ea0b1>] ? __fput+0x191/0x200
[12641.296791] [<ffffffff81374fe4>] ? lockdep_sys_exit_thunk+0x35/0x67
[12641.297597] [<ffffffff81077066>] SyS_clone+0x16/0x20
[12641.298395] [<ffffffff817d0299>] stub_clone+0x69/0x90
[12641.299173] [<ffffffff817cff12>] ? system_call_fastpath+0x12/0x17
[12641.299935] Code: 54 53 48 83 ec 18 48 8b 05 57 1f af 00 48 8d 98 c0 fe ff ff 48 3d 10 49 c6 81 74 4c 41 89 fd 45 31 e4 eb 17 48 8b 93 40 01 00 00 <48> 8d 9a c0 fe ff ff 48 81 fa 10 49 c6 81 74 39 48 3b b3 28 01
[12641.301567] sending NMI to other CPUs:
[12641.302311] NMI backtrace for cpu 1
[12641.303001] CPU: 1 PID: 3545 Comm: trinity-c7 Tainted: G L 3.18.0+ #106
[12641.305035] task: ffff8800963f96d0 ti: ffff880215b28000 task.ti: ffff880215b28000
[12641.305715] RIP: 0010:[<ffffffff810fc466>] [<ffffffff810fc466>] smp_call_function_many+0x276/0x320
[12641.306405] RSP: 0000:ffff880215b2baa8 EFLAGS: 00000202
[12641.307074] RAX: 0000000000000003 RBX: ffff8802451d3b00 RCX: ffff8802455d9488
[12641.307748] RDX: 0000000000000003 RSI: 0000000000000008 RDI: 0000000000000000
[12641.308423] RBP: ffff880215b2baf8 R08: ffff880243741e30 R09: 0000000100180011
[12641.309097] R10: ffff880244804240 R11: 0000000000000000 R12: 0000000000000003
[12641.309770] R13: 0000000000000000 R14: 0000000000000008 R15: 0000000000000008
[12641.310436] FS: 00007f688fc33740(0000) GS:ffff880245000000(0000) knlGS:0000000000000000
[12641.311099] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12641.311757] CR2: 0000000002258ff8 CR3: 0000000180fec000 CR4: 00000000001407e0
[12641.312420] DR0: 00007f688e6a9000 DR1: 0000000000000000 DR2: 0000000000000000
[12641.313082] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000070602
[12641.313744] Stack:
[12641.314390] ffff88024e5d4d00 0000000000000001 0000000100000003 00000000001d3ac0
[12641.315059] 0000004000000000 ffffffff82bea298 0000000000000001 ffffffff8117e150
[12641.315726] 0000000000000000 0000000000000000 ffff880215b2bb28 ffffffff810fc652
[12641.316393] Call Trace:
[12641.317045] [<ffffffff8117e150>] ? drain_pages+0xc0/0xc0
[12641.317698] [<ffffffff810fc652>] on_each_cpu_mask+0x32/0xa0
[12641.318351] [<ffffffff8117c371>] drain_all_pages+0x101/0x120
[12641.319003] [<ffffffff8118024f>] __alloc_pages_nodemask+0x7af/0xb40
[12641.319657] [<ffffffff811c8ebe>] alloc_pages_vma+0xee/0x1b0
[12641.320310] [<ffffffff811a416a>] ? do_wp_page+0xca/0x770
[12641.320959] [<ffffffff811a416a>] do_wp_page+0xca/0x770
[12641.321608] [<ffffffff811a69bb>] handle_mm_fault+0x6cb/0xe90
[12641.322258] [<ffffffff8104241d>] ? __do_page_fault+0x19d/0x610
[12641.322910] [<ffffffff8104248c>] __do_page_fault+0x20c/0x610
[12641.323557] [<ffffffff811aa2dc>] ? validate_mm+0x15c/0x2c0
[12641.324204] [<ffffffff810abaf5>] ? local_clock+0x25/0x30
[12641.324847] [<ffffffff810c361f>] ? lock_release_holdtime.part.24+0xf/0x190
[12641.325494] [<ffffffff81374fad>] ? trace_hardirqs_off_thunk+0x3a/0x3c
[12641.326142] [<ffffffff8104289c>] do_page_fault+0xc/0x10
[12641.326785] [<ffffffff817d1b32>] page_fault+0x22/0x30
[12641.327423] Code: 00 41 89 c4 39 f0 0f 8d 27 fe ff ff 48 63 d0 48 8b 0b 48 03 0c d5 a0 f2 d1 81 f6 41 18 01 74 14 0f 1f 44 00 00 f3 90 f6 41 18 01 <75> f8 48 63 35 05 63 c2 00 83 f8 ff 48 8b 7b 08 74 b0 39 c6 77
[12641.328871] NMI backtrace for cpu 0
[12641.329555] CPU: 0 PID: 30823 Comm: trinity-c206 Tainted: G L 3.18.0+ #106
[12641.331677] task: ffff8801ad862da0 ti: ffff88017d7d0000 task.ti: ffff88017d7d0000
[12641.332396] RIP: 0010:[<ffffffff810fc466>] [<ffffffff810fc466>] smp_call_function_many+0x276/0x320
[12641.333117] RSP: 0000:ffff88017d7d3a18 EFLAGS: 00000202
[12641.333827] RAX: 0000000000000003 RBX: ffff880244fd3b00 RCX: ffff8802455d7228
[12641.334546] RDX: 0000000000000003 RSI: 0000000000000008 RDI: 0000000000000000
[12641.335264] RBP: ffff88017d7d3a68 R08: ffff88024483b650 R09: 0000000100180011
[12641.335968] R10: ffff880244804240 R11: 0000000000000000 R12: 0000000000000003
[12641.336653] R13: 0000000000000000 R14: 0000000000000008 R15: 0000000000000008
[12641.337327] FS: 00007f688fc33740(0000) GS:ffff880244e00000(0000) knlGS:0000000000000000
[12641.338000] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12641.338665] CR2: 0000000002258ff8 CR3: 0000000232b93000 CR4: 00000000001407f0
[126[12641.356633] NMI backtrace for cpu 2
[12641.357349] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G L 3.18.0+ #106
[12641.359555] task: ffff8802434bc470 ti: ffff880243758000 task.ti: ffff880243758000
[12641.360314] RIP: 0010:[<ffffffff813d6cab>] [<ffffffff813d6cab>] intel_idle+0xdb/0x180
[12641.361070] RSP: 0000:ffff88024375be08 EFLAGS: 00000046
[12641.361812] RAX: 0000000000000000 RBX: 0000000000000002 RCX: 0000000000000001
[12641.362561] RDX: 0000000000000000 RSI: ffff88024375bfd8 RDI: 0000000000000002
[12641.363289] RBP: ffff88024375be38 R08: 000000008baf89af R09: 0000000000000000
[12641.363998] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000001
[12641.364680] R13: 0000000000000000 R14: 0000000000000001 R15: ffff880243758000
[12641.365350] FS: 0000000000000000(0000) GS:ffff880245200000(0000) knlGS:0000000000000000
[12641.366017] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12641.366678] CR2: 0000000002258ff8 CR3: 0000000097dd1000 CR4: 00000000001407e0
[12641.367347] DR0: 00007f688e6a9000 DR1: 0000000000000000 DR2: 0000000000000000
[12641.368016] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[12641.368680] Stack:
[12641.369340] 000000024375be38 5dccb806cd27b931 ffffe8ffff203200 0000000000000001
[12641.370021] ffffffff81cb1bc0 0000000000000002 ffff88024375be88 ffffffff8165f7b5
[12641.370710] 00000b80fc4e5dd1 ffffffff81cb1c30 ffffffff81cb1bc0 ffffffff81d215f0
[12641.371403] Call Trace:
[12641.372073] [<ffffffff8165f7b5>] cpuidle_enter_state+0x55/0x190
[12641.372758] [<ffffffff8165f9a7>] cpuidle_enter+0x17/0x20
[12641.373441] [<ffffffff810bd665>] cpu_startup_entry+0x355/0x410
[12641.374108] [<ffffffff8103015a>] start_secondary+0x1aa/0x230
[12641.374769] Code: 31 d2 65 48 8b 34 25 08 aa 00 00 48 8d 86 38 c0 ff ff 48 89 d1 0f 01 c8 48 8b 86 38 c0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <65> 48 8b 0c 25 08 aa 00 00 f0 80 a1 3a c0 ff ff df 0f ae f0 48
[12669.237353] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [trinity-c59:7816]
[12669.238086] Modules linked in: bridge 8021q garp stp snd_seq_dummy dlci tun fuse rfcomm hidp bnep af_key llc2 nfnetlink can_bcm scsi_transport_iscsi can_raw sctp libcrc32c nfc caif_socket caif af_802154 ieee802154 phonet af_rxrpc bluetooth can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 usb_debug cfg80211 rfkill coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode serio_raw pcspkr snd_hda_intel snd_hda_controller snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp pps_core snd_timer snd soundcore shpchp nfsd auth_rpcgss oid_registry nfs_acl lockd grace sunrpc
[12669.242957] CPU: 3 PID: 7816 Comm: trinity-c59 Tainted: G L 3.18.0+ #106
[12669.245468] task: ffff8802258796d0 ti: ffff88007df64000 task.ti: ffff88007df64000
[12669.246378] RIP: 0010:[<ffffffff810c6014>] [<ffffffff810c6014>] lock_acquire+0xb4/0x120
[12669.247281] RSP: 0018:ffff88007df67da8 EFLAGS: 00000246
[12669.248181] RAX: ffff8802258796d0 RBX: ffffffff810c512c RCX: ffff8802455cff98
[12669.249074] RDX: 00000000000001e0 RSI: 0000000000000010 RDI: 0000000000000000
[12669.249977] RBP: ffff88007df67e08 R08: 0000000000000000 R09: 0000000000000000
[12669.250881] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff810abaf5
[12669.251790] R13: ffff88007df67d18 R14: ffffffff810abaf5 R15: ffff88007df67d08
[12669.252693] FS: 00007f688fc33740(0000) GS:ffff880245400000(0000) knlGS:0000000000000000
[12669.253585] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12669.254486] CR2: 000000336f1b7740 CR3: 00000000a0ef4000 CR4: 00000000001407e0
[12669.255431] DR0: 00007f688e6a9000 DR1: 0000000000000000 DR2: 0000000000000000
[12669.256368] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000070602
[12669.257263] Stack:
[12669.258097] ffffffff81086b05 0000000000000000 0000000000000000 0000000000000246
[12669.258933] 0000000000000002 ffffffff81c50e20 ffff88007df67e48 00000000000024ca
[12669.259768] 0000000000000000 ffff88007df67eb0 ffff8800953896d0 0000000000000000
[12669.260638] Call Trace:
[12669.261471] [<ffffffff81086b05>] ? group_send_sig_info+0x5/0xc0
[12669.262318] [<ffffffff81086b48>] group_send_sig_info+0x48/0xc0
[12669.263170] [<ffffffff81086b05>] ? group_send_sig_info+0x5/0xc0
[12669.264028] [<ffffffff81086d05>] kill_pid_info+0x65/0xb0
[12669.264945] [<ffffffff81086ca5>] ? kill_pid_info+0x5/0xb0
[12669.265851] [<ffffffff81086e3c>] SYSC_kill+0xcc/0x240
[12669.266674] [<ffffffff81086df8>] ? SYSC_kill+0x88/0x240
[12669.267508] [<ffffffff8107aebb>] ? SyS_wait4+0x8b/0x110
[12669.268328] [<ffffffff81088cde>] SyS_kill+0xe/0x10
[12669.269145] [<ffffffff817cff12>] system_call_fastpath+0x12/0x17
[12669.269958] Code: d8 49 c1 e8 09 48 89 04 24 49 83 f0 01 41 83 e0 01 e8 01 ef ff ff 65 48 8b 04 25 00 aa 00 00 c7 80 6c 07 00 00 00 00 00 00 53 9d <48> 83 c4 38 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 44 00 00 65
[12669.271728] sending NMI to other CPUs:
[12669.272548] NMI backtrace for cpu 1
[12669.273314] CPU: 1 PID: 7878 Comm: trinity-c219 Tainted: G L 3.18.0+ #106
[12669.275686] task: ffff880096860000 ti: ffff880097f2c000 task.ti: ffff880097f2c000
[12669.276489] RIP: 0010:[<ffffffff811037c2>] [<ffffffff811037c2>] is_module_text_address+0x22/0x30
[12669.277296] RSP: 0018:ffff880097f2f9e8 EFLAGS: 00000046
[12669.278089] RAX: 0000000000000000 RBX: ffff880097f2fd70 RCX: 0000000000000004
[12669.278876] RDX: ffff880227e01b80 RSI: 0000000000000000 RDI: ffff880097f2fd70
[12669.279650] RBP: ffff880097f2fa00 R08: ffff880097f2fb40 R09: 0000000000000000
[12669.280408] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[12669.281147] R13: ffff880097f2fd80 R14: ffff880097f2fd50 R15: ffffffff81801b30
[12669.281874] FS: 00007f688fc33740(0000) GS:ffff880245000000(0000) knlGS:0000000000000000
[12669.282595] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12669.283299] CR2: 0000000001cb9a28 CR3: 00000000948d4000 CR4: 00000000001407e0
[12669.283997] DR0: 00007f688e6a9000 DR1: 0000000000000000 DR2: 0000000000000000
[12669.284677] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000070602
[12669.285348] Stack:
[12669.286003] ffffffff810973c8 0000000000000000 ffff880097f2fd70 ffff880097f2fa70
[12669.286678] ffffffff81006a7f ffff880097f2fa20 ffff880097f2fff8 ffff880097f2c000
[12669.287346] ffff880097f2fd80 ffff880097f2fb40 ffffffffffffc000 0000000000000000
[12669.288011] Call Trace:
[12669.288672] [<ffffffff810973c8>] ? __kernel_text_address+0x58/0x80
[12669.289343] [<ffffffff81006a7f>] print_context_stack+0x8f/0x100
[12669.290009] [<ffffffff8100559f>] dump_trace+0x16f/0x350
[12669.290668] [<ffffffff811b22a9>] ? anon_vma_clone+0x49/0x140
[12669.291321] [<ffffffff811b22a9>] ? anon_vma_clone+0x49/0x140
[12669.291963] [<ffffffff810135bf>] save_stack_trace+0x2f/0x50
[12669.292603] [<ffffffff811ce860>] set_track+0x70/0x140
[12669.293240] [<ffffffff817c26de>] alloc_debug_processing+0x92/0x118
[12669.293872] [<ffffffff817c33b2>] __slab_alloc+0x4da/0x58f
[12669.294498] [<ffffffff810abaf5>] ? local_clock+0x25/0x30
[12669.295119] [<ffffffff810c512c>] ? __lock_acquire.isra.31+0x22c/0x9f0
[12669.295745] [<ffffffff811b22a9>] ? anon_vma_clone+0x49/0x140
[12669.296362] [<ffffffff811b22a9>] ? anon_vma_clone+0x49/0x140
[12669.296969] [<ffffffff811d295b>] kmem_cache_alloc+0x1cb/0x1f0
[12669.297577] [<ffffffff811b22a9>] anon_vma_clone+0x49/0x140
[12669.298185] [<ffffffff811b23cd>] anon_vma_fork+0x2d/0x100
[12669.298790] [<ffffffff81076392>] copy_process.part.26+0x1362/0x1a40
[12669.299398] [<ffffffff810abaf5>] ? local_clock+0x25/0x30
[12669.300009] [<ffffffff81076c37>] do_fork+0xe7/0x490
[12669.300619] [<ffffffff810abaf5>] ? local_clock+0x25/0x30
[12669.301232] [<ffffffff81374fe4>] ? lockdep_sys_exit_thunk+0x35/0x67
[12669.301850] [<ffffffff81077066>] SyS_clone+0x16/0x20
[12669.302466] [<ffffffff817d0299>] stub_clone+0x69/0x90
[12669.303062] [<ffffffff817cff12>] ? system_call_fastpath+0x12/0x17
[12669.303646] Code: 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 65 ff 04 25 e0 a9 00 00 e8 4a b7 ff ff 65 ff 0c 25 e0 a9 00 00 48 85 c0 5d <0f> 95 c0 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48
[12669.304961] NMI backtrace for cpu 0
[12669.305579] CPU: 0 PID: 5863 Comm: trinity-c230 Tainted: G L 3.18.0+ #106
[12669.307401] task: ffff88023f830000 ti: ffff880096b74000 task.ti: ffff880096b74000
[12669.308023] RIP: 0010:[<ffffffff81191f24>] [<ffffffff81191f24>] shmem_undo_range+0x404/0x710
[12669.308649] RSP: 0000:ffff880096b77c78 EFLAGS: 00000246
[12669.309266] RAX: 002ffe000008003d RBX: 000000000000000c RCX: 0000000000000835
[12669.309888] RDX: ffffea0005e915c0 RSI: ffffea0005e91580 RDI:[12669.325471] NMI backtrace for cpu 2
[12669.326185] CPU: 2 PID: 7188 Comm: trinity-c210 Tainted: G L 3.18.0+ #106
[12669.328331] task: ffff8800a0d14470 ti: ffff8801c96d4000 task.ti: ffff8801c96d4000
[12669.329085] RIP: 0010:[<ffffffff810c4f21>] [<ffffffff810c4f21>] __lock_acquire.isra.31+0x21/0x9f0
[12669.329858] RSP: 0018:ffff8801c96d79b8 EFLAGS: 00000096
[12669.330627] RAX: ffffffff81185068 RBX: ffff8800a0d14470 RCX: 0000000000000000
[12669.331406] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff88024e5d3c58
[12669.332191] RBP: ffff8801c96d7a28 R08: 0000000000000001 R09: 0000000000000000
[12669.332984] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000001
[12669.333780] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[12669.334577] FS: 00007f688fc33740(0000) GS:ffff880245200000(0000) knlGS:0000000000000000
[12669.335370] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12669.336152] CR2: 00007f688d8a9000 CR3: 00000002271f7000 CR4: 00000000001407e0
[12669.336934] DR0: 00007f688e6a9000 DR1: 0000000000000000 DR2: 0000000000000000
[12669.337707] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000070602
[12669.338474] Stack:
[12669.339224] ffff8801c96d79f8 0000000093c05f9b 00000000000000d0 ffffffff82c00cc0
[12669.340023] ffff8800a0d14470 00000000000200da ffff8800a0d15780 0000000000000000
[12669.340803] ffff8801c96d7a08 0000000000000046 0000000000000000 0000000000000000
[12669.341541] Call Trace:
[12669.342270] [<ffffffff810c5fff>] lock_acquire+0x9f/0x120
[12669.342998] [<ffffffff81185068>] ? pagevec_lru_move_fn+0x88/0x100
[12669.343728] [<ffffffff817cf0a9>] _raw_spin_lock_irqsave+0x49/0x90
[12669.344455] [<ffffffff81185068>] ? pagevec_lru_move_fn+0x88/0x100
[12669.345190] [<ffffffff810c361f>] ? lock_release_holdtime.part.24+0xf/0x190
[12669.345916] [<ffffffff81185068>] pagevec_lru_move_fn+0x88/0x100
[12669.346640] [<ffffffff81185160>] ? __pagevec_lru_add+0x20/0x20
[12669.347374] [<ffffffff81185811>] __lru_cache_add+0x71/0x90
[12669.348097] [<ffffffff81185bf9>] lru_cache_add_anon+0x19/0x20
[12669.348811] [<ffffffff81190b08>] shmem_getpage_gfp+0x528/0x7a0
[12669.349559] [<ffffffff81190dc2>] shmem_write_begin+0x42/0x70
[12669.350260] [<ffffffff81175fc4>] generic_perform_write+0xd4/0x1f0
[12669.350957] [<ffffffff81178722>] __generic_file_write_iter+0x162/0x350
[12669.351655] [<ffffffff811e7940>] ? new_sync_read+0xd0/0xd0
[12669.352348] [<ffffffff8117894f>] generic_file_write_iter+0x3f/0xb0
[12669.353037] [<ffffffff81178910>] ? __generic_file_write_iter+0x350/0x350
[12669.353738] [<ffffffff811e7a88>] do_iter_readv_writev+0x78/0xc0
[12669.354434] [<ffffffff811e92b8>] do_readv_writev+0xd8/0x2a0
[12669.355126] [<ffffffff81178910>] ? __generic_file_write_iter+0x350/0x350
[12669.355828] [<ffffffff81178910>] ? __generic_file_write_iter+0x350/0x350
[12669.356531] [<ffffffff810c361f>] ? lock_release_holdtime.part.24+0xf/0x190
[12669.357228] [<ffffffff817cf310>] ? _raw_spin_unlock_irq+0x30/0x40
[12669.357921] [<ffffffff811e950c>] vfs_writev+0x3c/0x50
[12669.358596] [<ffffffff811e967c>] SyS_writev+0x5c/0x100
[12669.359258] [<ffffffff817cff12>] system_call_fastpath+0x12/0x17
[12669.359905] Code: 66 2e 0f 1f 84 00 00 00 00 00 55 48 89 e5 41 57 41 56 41 55 41 54 53 48 83 ec 48 44 8b 25 48 80 be 00 65 48 8b 1c 25 00 aa 00 00 <45> 85 e4 0f 84 ef 00 00 00 44 8b 1d e7 32 ab 01 49 89 fe 41 89
[12669.361354] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 88.798 msecs
[15739.449422] NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u16:3:14112]
[15739.450087] Modules linked in: bridge 8021q garp stp snd_seq_dummy dlci tun fuse rfcomm hidp bnep af_key llc2 nfnetlink can_bcm scsi_transport_iscsi can_raw sctp libcrc32c nfc caif_socket caif af_802154 ieee802154 phonet af_rxrpc bluetooth can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 usb_debug cfg80211 rfkill coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode serio_raw pcspkr snd_hda_intel snd_hda_controller snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp pps_core snd_timer snd soundcore shpchp nfsd auth_rpcgss oid_registry nfs_acl lockd grace sunrpc
[15739.454475] CPU: 2 PID: 14112 Comm: kworker/u16:3 Tainted: G L 3.18.0+ #106
[15739.456686] Workqueue: khelper __call_usermodehelper
[15739.457473] task: ffff8801c95f0000 ti: ffff880227eac000 task.ti: ffff880227eac000
[15739.458231] RIP: 0010:[<ffffffff817c3407>] [<ffffffff817c3407>] __slab_alloc+0x52f/0x58f
[15739.459015] RSP: 0018:ffff880227eaf8f8 EFLAGS: 00000246
[15739.459794] RAX: 0000000000000002 RBX: ffff8802304cf5c8 RCX: 00000000000002e0
[15739.460570] RDX: ffff88024520d7e0 RSI: 0000000000000000 RDI: ffff880244802000
[15739.461343] RBP: ffff880227eaf9e8 R08: 0000000000000000 R09: 0000000000000000
[15739.462113] R10: 0000000000000092 R11: 0000000000000000 R12: ffffffff810135bf
[15739.462885] R13: ffff880227eaf878 R14: 0000000100160015 R15: ffffffff8138278d
[15739.463648] FS: 0000000000000000(0000) GS:ffff880245200000(0000) knlGS:0000000000000000
[15739.464431] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15739.465203] CR2: 0000000000000008 CR3: 0000000225ab9000 CR4: 00000000001407e0
[15739.465985] DR0: 00007fbe591ef000 DR1: 0000000000000000 DR2: 0000000000000000
[15739.466766] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[15739.467542] Stack:
[15739.468308] 000000000000005c ffff880240f8d790 ffff880240f8d790 ffff880240f8dd00
[15739.469106] 0000000180230020 000000010000000f ffffffff8112ee12 0000000000000000
[15739.469912] ffff8802453d7260 000000020023001f ffff880227eaf968 ffffffff8138278d
[15739.470717] Call Trace:
[15739.471511] [<ffffffff8112ee12>] ? __delayacct_tsk_init+0x22/0x50
[15739.472325] [<ffffffff8138278d>] ? __debug_object_init+0x43d/0x450
[15739.473138] [<ffffffff811ce860>] ? set_track+0x70/0x140
[15739.473947] [<ffffffff811cf35d>] ? init_object+0x3d/0x70
[15739.474757] [<ffffffff8138278d>] ? __debug_object_init+0x43d/0x450
[15739.475571] [<ffffffff811d295b>] kmem_cache_alloc+0x1cb/0x1f0
[15739.476374] [<ffffffff8138278d>] __debug_object_init+0x43d/0x450
[15739.477174] [<ffffffff813827bb>] debug_object_init+0x1b/0x20
[15739.477983] [<ffffffff810e66d5>] hrtimer_init+0x25/0xb0
[15739.478781] [<ffffffff8109f069>] __sched_fork+0x99/0x230
[15739.479590] [<ffffffff810a59c9>] sched_fork+0x29/0x200
[15739.480387] [<ffffffff8107568c>] copy_process.part.26+0x65c/0x1a40
[15739.481184] [<ffffffff810abaf5>] ? local_clock+0x25/0x30
[15739.481983] [<ffffffff810c512c>] ? __lock_acquire.isra.31+0x22c/0x9f0
[15739.482797] [<ffffffff8108e340>] ? call_helper+0x20/0x20
[15739.483598] [<ffffffff81076c37>] do_fork+0xe7/0x490
[15739.484382] [<ffffffff810c512c>] ? __lock_acquire.isra.31+0x22c/0x9f0
[15739.485160] [<ffffffff81092f98>] ? process_one_work+0x178/0x550
[15739.485922] [<ffffffff81077006>] kernel_thread+0x26/0x30
[15739.486669] [<ffffffff8108e1b4>] __call_usermodehelper+0x64/0x80
[15739.487407] [<ffffffff8109301a>] process_one_work+0x1fa/0x550
[15739.488126] [<ffffffff81092f98>] ? process_one_work+0x178/0x550
[15739.488834] [<ffffffff8109348b>] worker_thread+0x11b/0x490
[15739.489529] [<ffffffff81093370>] ? process_one_work+0x550/0x550
[15739.490197] [<ffffffff81098c89>] kthread+0xf9/0x110
[15739.490845] [<ffffffff810abaf5>] ? local_clock+0x25/0x30
[15739.491474] [<ffffffff81098b90>] ? kthread_create_on_node+0x250/0x250
[15739.492092] [<ffffffff817cfe6c>] ret_from_fork+0x7c/0xb0
[15739.492689] [<ffffffff81098b90>] ? kthread_create_on_node+0x250/0x250
[15739.493288] Code: 00 02 00 00 49 c7 45 00 00 00 00 00 75 11 ff b5 78 ff ff ff 9d e8 7a 6d 98 ff 4c 89 e0 eb 0f e8 70 6e 98 ff ff b5 78 ff ff ff 9d <4c> 89 e0 48 8b 55 c8 65 48 33 14 25 28 00 00 00 74 3c e8 22 46
[15739.494617] sending NMI to other CPUs:
[15739.495221] NMI backtrace for cpu 3
[15739.495787] CPU: 3 PID: 1650 Comm: trinity-c76 Tainted: G L 3.18.0+ #106
[15739.497539] task: ffff8801adac4470 ti: ffff880180e04000 task.ti: ffff880180e04000
[15739.498153] RIP: 0010:[<ffffffff810c6014>] [<ffffffff810c6014>] lock_acquire+0xb4/0x120
[15739.498772] RSP: 0018:ffff880180e07dd8 EFLAGS: 00000246
[15739.499382] RAX: ffff8801adac4470 RBX: 0000000000000246 RCX: ffff8802455cff98
[15739.499994] RDX: 00000000000006a0 RSI: 0000000000000000 RDI: 0000000000000000
[15739.500609] RBP: ffff880180e07e38 R08: 0000000000000000 R09: 0000000000000000
[15739.501224] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[15739.501829] R13: 0000000000000000 R14: 0000000000000002 R15: 0000000000000000
[15739.502422] FS: 00007f79977d8740(0000) GS:ffff880245400000(0000) knlGS:0000000000000000
[15739.503021] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15739.503621] CR2: 0000000000000001 CR3: 00000001c9593000 CR4: 00000000001407e0
[15739.504217] DR0: 00007fbe591ef000 DR1: 0000000000000000 DR2: 0000000000000000
[15739.504812] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[15739.505403] Stack:
[15739.505988] ffffffff8107a8e9 0000000000000000 ffff8801adac4470 0000000000000246
[15739.506603] 0000000127cc4da0 ffffffff81c0a098 ffff8801adac4470 ffffffff81c0a080
[15739.507220] ffffffff81c0a098 ffff8801adac4470 ffff8801adac4470 ffff8801adac4470
[15739.507839] Call Trace:
[15739.508448] [<ffffffff8107a8e9>] ? do_wait+0xd9/0x280
[15739.509064] [<ffffffff817cf3d1>] _raw_read_lock+0x41/0x80
[15739.509678] [<ffffffff8107a8e9>] ? do_wait+0xd9/0x280
[15739.510293] [<ffffffff8107a8e9>] do_wait+0xd9/0x280
[15739.510907] [<ffffffff8107aeb0>] SyS_wait4+0x80/0x110
[15739.511518] [<ffffffff81078990>] ? task_stopped_code+0x60/0x60
[15739.512130] [<ffffffff817d0109>] tracesys_phase2+0xd4/0xd9
[15739.512739] Code: d8 49 c1 e8 09 48 89 04 24 49 83 f0 01 41 83 e0 01 e8 01 ef ff ff 65 48 8b 04 25 00 aa 00 00 c7 80 6c 07 00 00 00 00 00 00 53 9d <48> 83 c4 38 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 44 00 00 65
[15739.514121] NMI backtrace for cpu 1
[15739.514769] CPU: 1 PID: 2849 Comm: trinity-c95 Tainted: G L 3.18.0+ #106
[15739.516771] task: ffff880096b4c470 ti: ffff880153a4c000 task.ti: ffff880153a4c000
[15739.517467] RIP: 0010:[<ffffffff810961f9>] [<ffffffff810961f9>] find_pid_ns+0x39/0x90
[15739.518177] RSP: 0018:ffff880153a4fe78 EFLAGS: 00000207
[15739.518884] RAX: ffff88024e517120 RBX: 0000000000000d7f RCX: 0000000000000034
[15739.519599] RDX: ffff880094be6500 RSI: ffffffff81c486c0 RDI: 0000000000000d7f
[15739.520316] RBP: ffff880153a4fe78 R08: 0000000000000000 R09: 0000000000000000
[15739.521036] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[15739.521752] R13: ffff880096b4c470 R14: 0000000000000000 R15: 0000000000000000
[15739.522464] FS: 00007f79977d8740(0000) GS:ffff880245000000(0000) knlGS:0000000000000000
[15739.523181] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15739.523882] CR2: 00007f7996f24220 CR3: 00000002251b4000 CR4: 00000000001407e0
[15739.524580] DR0: 00007fbe591ef000 DR1: 0000000000000000 DR2: 0000000000000000
[15739.525269] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[15739.525957] Stack:
[15739.526632] ffff880153a4fe88 ffffffff8109627f ffff880153a4ff68 ffffffff81086e2a
[15739.527319] ffffffff81086df8 0000000000000000 ffff880096b4c840 0000000000000000
[15739.527992] ffff880100000000 000003e800000b21 ffff880094be64c0 00007fff123fb870
[15739.528656] Call Trace:
[15739.529301] [<ffffffff8109627f>] find_vpid+0x2f/0x50
[15739.529947] [<ffffffff81086e2a>] SYSC_kill+0xba/0x240
[15739.530587] [<ffffffff81086df8>] ? SYSC_kill+0x88/0x240
[15739[15763.435537] NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u16:3:14112]
[15763.436252] Modules linked in: bridge 8021q garp stp snd_seq_dummy dlci tun fuse rfcomm hidp bnep af_key llc2 nfnetlink can_bcm scsi_transport_iscsi can_raw sctp libcrc32c nfc caif_socket caif af_802154 ieee802154 phonet af_rxrpc bluetooth can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 usb_debug cfg80211 rfkill coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode serio_raw pcspkr snd_hda_intel snd_hda_controller snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp pps_core snd_timer snd soundcore shpchp nfsd auth_rpcgss oid_registry nfs_acl lockd grace sunrpc
[15763.441025] CPU: 2 PID: 14112 Comm: kworker/u16:3 Tainted: G L 3.18.0+ #106
[15763.443320] Workqueue: khelper __call_usermodehelper
[15763.444090] task: ffff8801c95f0000 ti: ffff880227eac000 task.ti: ffff880227eac000
[15763.444861] RIP: 0010:[<ffffffff817c3407>] [<ffffffff817c3407>] __slab_alloc+0x52f/0x58f
[15763.445654] RSP: 0018:ffff880227eaf8f8 EFLAGS: 00000246
[15763.446422] RAX: 0000000000000002 RBX: ffff8802304cf5c8 RCX: 00000000000002e0
[15763.447201] RDX: ffff88024520d7e0 RSI: 0000000000000000 RDI: ffff880244802000
[15763.447985] RBP: ffff880227eaf9e8 R08: 0000000000000000 R09: 0000000000000000
[15763.448764] R10: 0000000000000092 R11: 0000000000000000 R12: ffffffff810135bf
[15763.449545] R13: ffff880227eaf878 R14: 0000000100160015 R15: ffffffff8138278d
[15763.450319] FS: 0000000000000000(0000) GS:ffff880245200000(0000) knlGS:0000000000000000
[15763.451100] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15763.451876] CR2: 0000000000000008 CR3: 0000000225ab9000 CR4: 00000000001407e0
[15763.452662] DR0: 00007fbe591ef000 DR1: 0000000000000000 DR2: 0000000000000000
[15763.453451] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[15763.454230] Stack:
[15763.455003] 000000000000005c ffff880240f8d790 ffff880240f8d790 ffff880240f8dd00
[15763.455818] 0000000180230020 000000010000000f ffffffff8112ee12 0000000000000000
[15763.456608] ffff8802453d7260 000000020023001f ffff880227eaf968 ffffffff8138278d
[15763.457393] Call Trace:
[15763.458166] [<ffffffff8112ee12>] ? __delayacct_tsk_init+0x22/0x50
[15763.458960] [<ffffffff8138278d>] ? __debug_object_init+0x43d/0x450
[15763.459753] [<ffffffff811ce860>] ? set_track+0x70/0x140
[15763.460544] [<ffffffff811cf35d>] ? init_object+0x3d/0x70
[15763.461338] [<ffffffff8138278d>] ? __debug_object_init+0x43d/0x450
[15763.462127] [<ffffffff811d295b>] kmem_cache_alloc+0x1cb/0x1f0
[15763.462921] [<ffffffff8138278d>] __debug_object_init+0x43d/0x450
[15763.463707] [<ffffffff813827bb>] debug_object_init+0x1b/0x20
[15763.464503] [<ffffffff810e66d5>] hrtimer_init+0x25/0xb0
[15763.465294] [<ffffffff8109f069>] __sched_fork+0x99/0x230
[15763.466086] [<ffffffff810a59c9>] sched_fork+0x29/0x200
[15763.466875] [<ffffffff8107568c>] copy_process.part.26+0x65c/0x1a40
[15763.467664] [<ffffffff810abaf5>] ? local_clock+0x25/0x30
[15763.468458] [<ffffffff810c512c>] ? __lock_acquire.isra.31+0x22c/0x9f0
[15763.469251] [<ffffffff8108e340>] ? call_helper+0x20/0x20
[15763.470041] [<ffffffff81076c37>] do_fork+0xe7/0x490
[15763.470828] [<ffffffff810c512c>] ? __lock_acquire.isra.31+0x22c/0x9f0
[15763.471603] [<ffffffff81092f98>] ? process_one_work+0x178/0x550
[15763.472371] [<ffffffff81077006>] kernel_thread+0x26/0x30
[15763.473120] [<ffffffff8108e1b4>] __call_usermodehelper+0x64/0x80
[15763.473854] [<ffffffff8109301a>] process_one_work+0x1fa/0x550
[15763.474578] [<ffffffff81092f98>] ? process_one_work+0x178/0x550
[15763.475287] [<ffffffff8109348b>] worker_thread+0x11b/0x490
[15763.475975] [<ffffffff81093370>] ? process_one_work+0x550/0x550
[15763.476640] [<ffffffff81098c89>] kthread+0xf9/0x110
[15763.477285] [<ffffffff810abaf5>] ? local_clock+0x25/0x30
[15763.477915] [<ffffffff81098b90>] ? kthread_create_on_node+0x250/0x250
[15763.478532] [<ffffffff817cfe6c>] ret_from_fork+0x7c/0xb0
[15763.479131] [<ffffffff81098b90>] ? kthread_create_on_node+0x250/0x250
[15763.479741] Code: 00 02 00 00 49 c7 45 00 00 00 00 00 75 11 ff b5 78 ff ff ff 9d e8 7a 6d 98 ff 4c 89 e0 eb 0f e8 70 6e 98 ff ff b5 78 ff ff ff 9d <4c> 89 e0 48 8b 55 c8 65 48 33 14 25 28 00 00 00 74 3c e8 22 46
[15763.481087] sending NMI to other CPUs:
[15763.481693] NMI backtrace for cpu 3
[15763.482255] CPU: 3 PID: 1650 Comm: trinity-c76 Tainted: G L 3.18.0+ #106
[15763.484005] task: ffff8801adac4470 ti: ffff880180e04000 task.ti: ffff880180e04000
[15763.484618] RIP: 0010:[<ffffffff810c47f5>] [<ffffffff810c47f5>] lock_acquired+0x45/0x370
[15763.485234] RSP: 0018:ffff880180e07db8 EFLAGS: 00000046
[15763.485841] RAX: 0000000000000001 RBX: ffff880227cc4da0 RCX: 0000000000000001
[15763.486453] RDX: 000000000000dbdb RSI: ffffffff810bcc8d RDI: ffff880227cc4db8
[15763.487068] RBP: ffff880180e07df8 R08: 0000000000000000 R09: 0000000000000001
[15763.487687] R10: 0000000000000000 R11: 0000000000000000 R12: ffff8801adac4470
[15763.488303] R13: ffff880227cc4db8 R14: 0000000000000046 R15: ffff8801adac4460
[15763.488909] FS: 00007f79977d8740(0000) GS:ffff880245400000(0000) knlGS:0000000000000000
[15763.489522] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15763.490133] CR2: 0000000000000001 CR3: 00000001c9593000 CR4: 00000000001407e0
[15763.490747] DR0: 00007fbe591ef000 DR1: 0000000000000000 DR2: 0000000000000000
[15763.491356] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[15763.491964] Stack:
[15763.492564] 0000000100000002 ffffffff810bcc8d ffff880180e07dd8 ffff880227cc4da0
[15763.493194] ffff880227cc4db8 0000000000000292 ffff8801adac4470 ffff8801adac4460
[15763.493828] ffff880180e07e38 ffffffff817cf0d5 ffffffff810bcc8d 0000000000000296
[15763.494464] Call Trace:
[15763.495088] [<ffffffff810bcc8d>] ? remove_wait_queue+0x1d/0x40
[15763.495726] [<ffffffff817cf0d5>] _raw_spin_lock_irqsave+0x75/0x90
[15763.496364] [<ffffffff810bcc8d>] ? remove_wait_queue+0x1d/0x40
[15763.497005] [<ffffffff810bcc8d>] remove_wait_queue+0x1d/0x40
[15763.497645] [<ffffffff8107a95b>] do_wait+0x14b/0x280
[15763.498283] [<ffffffff8107aeb0>] SyS_wait4+0x80/0x110
[15763.498917] [<ffffffff81078990>] ? task_stopped_code+0x60/0x60
[15763.499551] [<ffffffff817d0109>] tracesys_phase2+0xd4/0xd9
[15763.500183] Code: b8 00 45 85 c9 0f 84 d8 00 00 00 65 4c 8b 24 25 00 aa 00 00 45 8b 84 24 6c 07 00 00 45 85 c0 0f 85 be 00 00 00 49 89 fd 9c 41 5e <fa> 8b 35 1c 3a ab 01 41 c7 84 24 6c 07 00 00 01 00 00 00 41 8b
[15763.501611] NMI backtrace for cpu 1
[15763.502274] CPU: 1 PID: 3298 Comm: trinity-c183 Tainted: G L 3.18.0+ #106
[15763.504345] task: ffff880227de16d0 ti: ffff880071060000 task.ti: ffff880071060000
[15763.505065] RIP: 0033:[<000000336eebc2fc>] [<000000336eebc2fc>] 0x336eebc2fc
[15763.505796] RSP: 002b:00007fff123fb868 EFLAGS: 00000246
[15763.506522] RAX: 0000000000000000 RBX: 0000000000000d7b RCX: ffffffffffffffff
[15763.507257] RDX: 000000000000000b RSI: 00007fff123fb870 RDI: 0000000000000d7b
[15763.507994] RBP: 0000000000000000 R08: 00007f79977d8740 R09: 0000000000000000
[15763.508729] R10: 0000000000000000 R11: 0000000000000246 R12: 00007f7996cf6000
[15763.509460] R13: 00007f7996cf6068 R14: 0000000000000000 R15: 0000000000000000
[15763.510188] FS: 00007f79977d8740(0000) GS:ffff880245000000(0000) knlGS:0000000000000000
[15763.510908] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15763.511612] CR2: 00007f7996f24220 CR3: 000000009a646000 CR4: 00000000001407e0
[15763.512319] DR0: 00007fbe591ef000 DR1: 0000000000000000 DR2: 0000000000000000
[15763.513027] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[15763.513727]
[15763.514403] NMI backtrace for cpu 0
[15763.515067] CPU: 0 PID: 1876 Comm: trinity-c189 Tainted: G L 3.18.0+ #106
[15763.517087] task: ffff880096b4ada0 ti: ffff8802253d0000 task.ti: ffff8802253d0000
[15763.517776] RIP: 0010:[<[15779.306349] INFO: rcu_sched detected stalls on CPUs/tasks:
[15779.307024] (detected by 0, t=6002 jiffies, g=481360, c=481359, q=0)
[15779.307662] INFO: Stall ended before state dump start
[15787.421647] NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u16:3:14112]
[15787.422333] Modules linked in: bridge 8021q garp stp snd_seq_dummy dlci tun fuse rfcomm hidp bnep af_key llc2 nfnetlink can_bcm scsi_transport_iscsi can_raw sctp libcrc32c nfc caif_socket caif af_802154 ieee802154 phonet af_rxrpc bluetooth can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 usb_debug cfg80211 rfkill coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode serio_raw pcspkr snd_hda_intel snd_hda_controller snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp pps_core snd_timer snd soundcore shpchp nfsd auth_rpcgss oid_registry nfs_acl lockd grace sunrpc
[15787.426946] CPU: 2 PID: 14112 Comm: kworker/u16:3 Tainted: G L 3.18.0+ #106
[15787.429181] Workqueue: khelper __call_usermodehelper
[15787.429945] task: ffff8801c95f0000 ti: ffff880227eac000 task.ti: ffff880227eac000
[15787.430719] RIP: 0010:[<ffffffff817c3407>] [<ffffffff817c3407>] __slab_alloc+0x52f/0x58f
[15787.431507] RSP: 0018:ffff880227eaf8f8 EFLAGS: 00000246
[15787.432297] RAX: 0000000000000002 RBX: ffff8802304cf5c8 RCX: 00000000000002e0
[15787.433094] RDX: ffff88024520d7e0 RSI: 0000000000000000 RDI: ffff880244802000
[15787.433886] RBP: ffff880227eaf9e8 R08: 0000000000000000 R09: 0000000000000000
[15787.434671] R10: 0000000000000092 R11: 0000000000000000 R12: ffffffff810135bf
[15787.435452] R13: ffff880227eaf878 R14: 0000000100160015 R15: ffffffff8138278d
[15787.436234] FS: 0000000000000000(0000) GS:ffff880245200000(0000) knlGS:0000000000000000
[15787.437028] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15787.437816] CR2: 0000000000000008 CR3: 0000000225ab9000 CR4: 00000000001407e0
[15787.438605] DR0: 00007fbe591ef000 DR1: 0000000000000000 DR2: 0000000000000000
[15787.439400] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[15787.440183] Stack:
[15787.440959] 000000000000005c ffff880240f8d790 ffff880240f8d790 ffff880240f8dd00
[15787.441792] 0000000180230020 000000010000000f ffffffff8112ee12 0000000000000000
[15787.442615] ffff8802453d7260 000000020023001f ffff880227eaf968 ffffffff8138278d
[15787.443422] Call Trace:
[15787.444217] [<ffffffff8112ee12>] ? __delayacct_tsk_init+0x22/0x50
[15787.445027] [<ffffffff8138278d>] ? __debug_object_init+0x43d/0x450
[15787.445833] [<ffffffff811ce860>] ? set_track+0x70/0x140
[15787.446647] [<ffffffff811cf35d>] ? init_object+0x3d/0x70
[15787.447449] [<ffffffff8138278d>] ? __debug_object_init+0x43d/0x450
[15787.448258] [<ffffffff811d295b>] kmem_cache_alloc+0x1cb/0x1f0
[15787.449067] [<ffffffff8138278d>] __debug_object_init+0x43d/0x450
[15787.449879] [<ffffffff813827bb>] debug_object_init+0x1b/0x20
[15787.450688] [<ffffffff810e66d5>] hrtimer_init+0x25/0xb0
[15787.451497] [<ffffffff8109f069>] __sched_fork+0x99/0x230
[15787.452307] [<ffffffff810a59c9>] sched_fork+0x29/0x200
[15787.453122] [<ffffffff8107568c>] copy_process.part.26+0x65c/0x1a40
[15787.453928] [<ffffffff810abaf5>] ? local_clock+0x25/0x30
[15787.454733] [<ffffffff810c512c>] ? __lock_acquire.isra.31+0x22c/0x9f0
[15787.455524] [<ffffffff8108e340>] ? call_helper+0x20/0x20
[15787.456299] [<ffffffff81076c37>] do_fork+0xe7/0x490
[15787.457062] [<ffffffff810c512c>] ? __lock_acquire.isra.31+0x22c/0x9f0
[15787.457826] [<ffffffff81092f98>] ? process_one_work+0x178/0x550
[15787.458587] [<ffffffff81077006>] kernel_thread+0x26/0x30
[15787.459336] [<ffffffff8108e1b4>] __call_usermodehelper+0x64/0x80
[15787.460078] [<ffffffff8109301a>] process_one_work+0x1fa/0x550
[15787.460794] [<ffffffff81092f98>] ? process_one_work+0x178/0x550
[15787.461495] [<ffffffff8109348b>] worker_thread+0x11b/0x490
[15787.462182] [<ffffffff81093370>] ? process_one_work+0x550/0x550
[15787.462850] [<ffffffff81098c89>] kthread+0xf9/0x110
[15787.463496] [<ffffffff810abaf5>] ? local_clock+0x25/0x30
[15787.464124] [<ffffffff81098b90>] ? kthread_create_on_node+0x250/0x250
[15787.464742] [<ffffffff817cfe6c>] ret_from_fork+0x7c/0xb0
[15787.465341] [<ffffffff81098b90>] ? kthread_create_on_node+0x250/0x250
[15787.465947] Code: 00 02 00 00 49 c7 45 00 00 00 00 00 75 11 ff b5 78 ff ff ff 9d e8 7a 6d 98 ff 4c 89 e0 eb 0f e8 70 6e 98 ff ff b5 78 ff ff ff 9d <4c> 89 e0 48 8b 55 c8 65 48 33 14 25 28 00 00 00 74 3c e8 22 46
[15787.467291] sending NMI to other CPUs:
[15787.467904] NMI backtrace for cpu 3
[15787.468463] CPU: 3 PID: 1650 Comm: trinity-c76 Tainted: G L 3.18.0+ #106
[15787.470211] task: ffff8801adac4470 ti: ffff880180e04000 task.ti: ffff880180e04000
[15787.470824] RIP: 0033:[<000000336eebc2fc>] [<000000336eebc2fc>] 0x336eebc2fc
[15787.471440] RSP: 002b:00007fff123fb868 EFLAGS: 00000246
[15787.472043] RAX: 0000000000000000 RBX: 0000000000000d7e RCX: ffffffffffffffff
[15787.472652] RDX: 000000000000000b RSI: 00007fff123fb870 RDI: 0000000000000d7e
[15787.473259] RBP: 0000000000000000 R08: 00007f79977d8740 R09: 0000000000000000
[15787.473868] R10: 0000000000000000 R11: 0000000000000246 R12: 00007f7997265000
[15787.474468] R13: 00007f7997265068 R14: 0000000000000000 R15: 0000000000000000
[15787.475063] FS: 00007f79977d8740(0000) GS:ffff880245400000(0000) knlGS:0000000000000000
[15787.475660] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15787.476254] CR2: 0000000000000001 CR3: 00000001c9593000 CR4: 00000000001407e0
[15787.476851] DR0: 00007fbe591ef000 DR1: 0000000000000000 DR2: 0000000000000000
[15787.477442] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[15787.478032]
[15787.478615] NMI backtrace for cpu 1
[15787.479205] CPU: 1 PID: 2849 Comm: trinity-c95 Tainted: G L 3.18.0+ #106
[15787.481052] task: ffff880096b4c470 ti: ffff880153a4c000 task.ti: ffff880153a4c000
[15787.481696] RIP: 0010:[<ffffffff817cfea0>] [<ffffffff817cfea0>] system_call+0x0/0x3
[15787.482351] RSP: 0018:00007fff123fb868 EFLAGS: 00000046
[15787.483007] RAX: 000000000000003d RBX: 0000000000000d7f RCX: 000000336eebc2fc
[15787.483674] RDX: 000000000000000b RSI: 00007fff123fb870 RDI: 0000000000000d7f
[15787.484338] RBP: 0000000000000000 R08: 00007f79977d8740 R09: 0000000000000000
[15787.485000] R10: 0000000000000000 R11: 0000000000000246 R12: 00007f799716e000
[15787.485658] R13: 00007f799716e068 R14: 0000000000000000 R15: 0000000000000000
[15787.486314] FS: 00007f79977d8740(0000) GS:ffff880245000000(0000) knlGS:0000000000000000
[15787.486979] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15787.487641] CR2: 00007f7996f24220 CR3: 00000002251b4000 CR4: 00000000001407e0
[15787.488311] DR0: 00007fbe591ef000 DR1: 0000000000000000 DR2: 0000000000000000
[15787.488982] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[15787.489650] Stack:
[15787.490312] 000000000041401b 00007f79976944e8 00007f799716e000 0000000000000001
[15787.490999] 00007f799716e07c 00007f799716e000 0000000000000000 00007f799716e07c
[15787.491690] 0000000000416c03 00000000000066b2 0000000000000155 00000000cccccccd
[15787.492383] Call Trace:
[15787.493063] <UNK>
[15787.493070] Code: 8b 3c 24 4c 8b 74 24 08 4c 8b 6c 24 10 4c 8b 64 24 18 48 8b 6c 24 20 48 8b 5c 24 28 48 83 c4 30 e9 74 02 00 00 66 0f 1f 44 00 00 <0f> 01 f8 65 48 89 24 25 80 a0 00 00 65 48 8b 24 25 08 aa 00 00
[15787.495270] NMI backtrace for cpu 0
[15787.495987] CPU: 0 PID: 1876 Comm: trinity-c189 Tainted: G L 3.18.0+ #106
[15787.498136] task: ffff880096b4ada0 ti: ffff8802253d0000 task.ti: ffff8802253d0000
[15787.498870] RIP: 0010:[<ffffffff810c9698>] [<ffffffff810c9698>] do_raw_spin_trylock+0x8/0x50
[15787.499618] RSP: 0018:ffff880244e03d00 EFLAGS: 00000092
[15787.500344] RAX: ffff880096b4ada0 RBX: ffff880240c51578 RCX: ffff880244fcff98
[15787.501064] RDX: 0000000000004a4a RSI: 0000000000000018 RDI: ffff880240c51578
[15787.501780] RBP: ffff880244e03d38 R08: 0000000000000001 R09: 0000000000000000
[15787.502487] R10: 0000000000000000 R11: 0000000000000000 R12: ffff880240c51590
[15787.503190] R13: 0000000000000092 R14: ff[15811.407761] NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u16:3:14112]
[15811.408334] Modules linked in: bridge 8021q garp stp snd_seq_dummy dlci tun fuse rfcomm hidp bnep af_key llc2 nfnetlink can_bcm scsi_transport_iscsi can_raw sctp libcrc32c nfc caif_socket caif af_802154 ieee802154 phonet af_rxrpc bluetooth can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 usb_debug cfg80211 rfkill coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode serio_raw pcspkr snd_hda_intel snd_hda_controller snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp pps_core snd_timer snd soundcore shpchp nfsd auth_rpcgss oid_registry nfs_acl lockd grace sunrpc
[15811.412296] CPU: 2 PID: 14112 Comm: kworker/u16:3 Tainted: G L 3.18.0+ #106
[15811.414324] Workqueue: khelper __call_usermodehelper
[15811.415025] task: ffff8801c95f0000 ti: ffff880227eac000 task.ti: ffff880227eac000
[15811.415733] RIP: 0010:[<ffffffff817c3407>] [<ffffffff817c3407>] __slab_alloc+0x52f/0x58f
[15811.416455] RSP: 0018:ffff880227eaf8f8 EFLAGS: 00000246
[15811.417171] RAX: 0000000000000002 RBX: ffff8802304cf5c8 RCX: 00000000000002e0
[15811.417913] RDX: ffff88024520d7e0 RSI: 0000000000000000 RDI: ffff880244802000
[15811.418645] RBP: ffff880227eaf9e8 R08: 0000000000000000 R09: 0000000000000000
[15811.419374] R10: 0000000000000092 R11: 0000000000000000 R12: ffffffff810135bf
[15811.420102] R13: ffff880227eaf878 R14: 0000000100160015 R15: ffffffff8138278d
[15811.420831] FS: 0000000000000000(0000) GS:ffff880245200000(0000) knlGS:0000000000000000
[15811.421572] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15811.422317] CR2: 0000000000000008 CR3: 0000000225ab9000 CR4: 00000000001407e0
[15811.423070] DR0: 00007fbe591ef000 DR1: 0000000000000000 DR2: 0000000000000000
[15811.423828] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[15811.424583] Stack:
[15811.425327] 000000000000005c ffff880240f8d790 ffff880240f8d790 ffff880240f8dd00
[15811.426099] 0000000180230020 000000010000000f ffffffff8112ee12 0000000000000000
[15811.426872] ffff8802453d7260 000000020023001f ffff880227eaf968 ffffffff8138278d
[15811.427657] Call Trace:
[15811.428437] [<ffffffff8112ee12>] ? __delayacct_tsk_init+0x22/0x50
[15811.429231] [<ffffffff8138278d>] ? __debug_object_init+0x43d/0x450
[15811.430025] [<ffffffff811ce860>] ? set_track+0x70/0x140
[15811.430814] [<ffffffff811cf35d>] ? init_object+0x3d/0x70
[15811.431600] [<ffffffff8138278d>] ? __debug_object_init+0x43d/0x450
[15811.432394] [<ffffffff811d295b>] kmem_cache_alloc+0x1cb/0x1f0
[15811.433180] [<ffffffff8138278d>] __debug_object_init+0x43d/0x450
[15811.433972] [<ffffffff813827bb>] debug_object_init+0x1b/0x20
[15811.434768] [<ffffffff810e66d5>] hrtimer_init+0x25/0xb0
[15811.435559] [<ffffffff8109f069>] __sched_fork+0x99/0x230
[15811.436349] [<ffffffff810a59c9>] sched_fork+0x29/0x200
[15811.437141] [<ffffffff8107568c>] copy_process.part.26+0x65c/0x1a40
[15811.437937] [<ffffffff810abaf5>] ? local_clock+0x25/0x30
[15811.438725] [<ffffffff810c512c>] ? __lock_acquire.isra.31+0x22c/0x9f0
[15811.439519] [<ffffffff8108e340>] ? call_helper+0x20/0x20
[15811.440307] [<ffffffff81076c37>] do_fork+0xe7/0x490
[15811.441094] [<ffffffff810c512c>] ? __lock_acquire.isra.31+0x22c/0x9f0
[15811.441871] [<ffffffff81092f98>] ? process_one_work+0x178/0x550
[15811.442637] [<ffffffff81077006>] kernel_thread+0x26/0x30
[15811.443385] [<ffffffff8108e1b4>] __call_usermodehelper+0x64/0x80
[15811.444121] [<ffffffff8109301a>] process_one_work+0x1fa/0x550
[15811.444838] [<ffffffff81092f98>] ? process_one_work+0x178/0x550
[15811.445538] [<ffffffff8109348b>] worker_thread+0x11b/0x490
[15811.446219] [<ffffffff81093370>] ? process_one_work+0x550/0x550
[15811.446883] [<ffffffff81098c89>] kthread+0xf9/0x110
[15811.447527] [<ffffffff810abaf5>] ? local_clock+0x25/0x30
[15811.448159] [<ffffffff81098b90>] ? kthread_create_on_node+0x250/0x250
[15811.448776] [<ffffffff817cfe6c>] ret_from_fork+0x7c/0xb0
[15811.449371] [<ffffffff81098b90>] ? kthread_create_on_node+0x250/0x250
[15811.449968] Code: 00 02 00 00 49 c7 45 00 00 00 00 00 75 11 ff b5 78 ff ff ff 9d e8 7a 6d 98 ff 4c 89 e0 eb 0f e8 70 6e 98 ff ff b5 78 ff ff ff 9d <4c> 89 e0 48 8b 55 c8 65 48 33 14 25 28 00 00 00 74 3c e8 22 46
[15811.451292] sending NMI to other CPUs:
[15811.451901] NMI backtrace for cpu 3
[15811.452459] CPU: 3 PID: 1650 Comm: trinity-c76 Tainted: G L 3.18.0+ #106
[15811.454206] task: ffff8801adac4470 ti: ffff880180e04000 task.ti: ffff880180e04000
[15811.454817] RIP: 0010:[<ffffffff810c511a>] [<ffffffff810c511a>] __lock_acquire.isra.31+0x21a/0x9f0
[15811.455437] RSP: 0018:ffff880180e07d18 EFLAGS: 00000002
[15811.456045] RAX: 0000000000000008 RBX: ffff8801adac4470 RCX: 0000000000000000
[15811.456660] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
[15811.457274] RBP: ffff880180e07d88 R08: 0000000000000001 R09: 0000000000000000
[15811.457889] R10: 0000000000000000 R11: 0000000000000000 R12: 000000000000014e
[15811.458493] R13: 0000000000000000 R14: ffff880227cc4db8 R15: ffff8801adac4be0
[15811.459092] FS: 00007f79977d8740(0000) GS:ffff880245400000(0000) knlGS:0000000000000000
[15811.459699] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15811.460303] CR2: 0000000000000001 CR3: 00000001c9593000 CR4: 00000000001407e0
[15811.460907] DR0: 00007fbe591ef000 DR1: 0000000000000000 DR2: 0000000000000000
[15811.461507] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[15811.462108] Stack:
[15811.462702] ffff880180e07d98 ffffffff810c512c 0000000000000102 0000000000000000
[15811.463325] ffff880180e07d48 ffffffff810abaf5 ffff880180e07dc8 0000000000000000
[15811.463945] ffff880180e07dd8 0000000000000046 0000000000000000 0000000000000000
[15811.464564] Call Trace:
[15811.465173] [<ffffffff810c512c>] ? __lock_acquire.isra.31+0x22c/0x9f0
[15811.465797] [<ffffffff810abaf5>] ? local_clock+0x25/0x30
[15811.466421] [<ffffffff810c5fff>] lock_acquire+0x9f/0x120
[15811.467047] [<ffffffff810bcc8d>] ? remove_wait_queue+0x1d/0x40
[15811.467672] [<ffffffff817cf0a9>] _raw_spin_lock_irqsave+0x49/0x90
[15811.468300] [<ffffffff810bcc8d>] ? remove_wait_queue+0x1d/0x40
[15811.468924] [<ffffffff810bcc8d>] remove_wait_queue+0x1d/0x40
[15811.469546] [<ffffffff8107a95b>] do_wait+0x14b/0x280
[15811.470166] [<ffffffff8107aeb0>] SyS_wait4+0x80/0x110
[15811.470782] [<ffffffff81078990>] ? task_stopped_code+0x60/0x60
[15811.471398] [<ffffffff817d0109>] tracesys_phase2+0xd4/0xd9
[15811.472011] Code: e0 7f 44 09 d0 41 88 47 31 41 0f b6 47 32 83 e0 f0 45 85 c0 0f 95 c2 09 c8 c1 e2 03 09 d0 41 88 47 32 0f b7 55 18 41 0f b7 47 32 <c1> e2 04 83 e0 0f 09 d0 66 41 89 47 32 e8 a4 69 fe ff 4c 8b 4d
[15811.473409] NMI backtrace for cpu 1
[15811.474061] CPU: 1 PID: 2849 Comm: trinity-c95 Tainted: G L 3.18.0+ #106
[15811.476098] task: ffff880096b4c470 ti: ffff880153a4c000 task.ti: ffff880153a4c000
[15811.476808] RIP: 0010:[<ffffffff810c63cf>] [<ffffffff810c63cf>] lock_release+0x1f/0x240
[15811.477535] RSP: 0018:ffff880153a4fe40 EFLAGS: 00000246
[15811.478253] RAX: ffff880096b4c470 RBX: 0000000000000000 RCX: 00000000000003a0
[15811.478982] RDX: ffffffff81086d0c RSI: 0000000000000001 RDI: ffffffff81c50e20
[15811.479707] RBP: ffff880153a4fe48 R08: 0000000000000000 R09: 0000000000000000
[15811.480419] R10: 0000000000000000 R11: 0000000000000000 R12: ffff880094be64c0
[15811.481115] R13: ffff880153a4feb0 R14: 0000000000000000 R15: 0000000000000000
[15811.481803] FS: 00007f79977d8740(0000) GS:ffff880245000000(0000) knlGS:0000000000000000
[15811.482504] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15811.483200] CR2: 00007f7996f24220 CR3: 00000002251b4000 CR4: 00000000001407e0
[15811.483890] DR0: 00007fbe591ef000 DR1: 0000000000000000 DR2: 0000000000000000
[15811.484562] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[15811.485221] Stack:
[15811.485862] 0000000000000000 ffff880153a4fe88 ffffffff81086d24 ffffffff81086ca5
[15811.486521] 0000000000000d7f 0000000000000d7f 0000000000000000 ffff880096[15835.393872] NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u16:3:14112]
[15835.394432] Modules linked in: bridge 8021q garp stp snd_seq_dummy dlci tun fuse rfcomm hidp bnep af_key llc2 nfnetlink can_bcm scsi_transport_iscsi can_raw sctp libcrc32c nfc caif_socket caif af_802154 ieee802154 phonet af_rxrpc bluetooth can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 usb_debug cfg80211 rfkill coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode serio_raw pcspkr snd_hda_intel snd_hda_controller snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp pps_core snd_timer snd soundcore shpchp nfsd auth_rpcgss oid_registry nfs_acl lockd grace sunrpc
[15835.398378] CPU: 2 PID: 14112 Comm: kworker/u16:3 Tainted: G L 3.18.0+ #106
[15835.400379] Workqueue: khelper __call_usermodehelper
[15835.401080] task: ffff8801c95f0000 ti: ffff880227eac000 task.ti: ffff880227eac000
[15835.401795] RIP: 0010:[<ffffffff817c3407>] [<ffffffff817c3407>] __slab_alloc+0x52f/0x58f
[15835.402534] RSP: 0018:ffff880227eaf8f8 EFLAGS: 00000246
[15835.403252] RAX: 0000000000000002 RBX: ffff8802304cf5c8 RCX: 00000000000002e0
[15835.403995] RDX: ffff88024520d7e0 RSI: 0000000000000000 RDI: ffff880244802000
[15835.404727] RBP: ffff880227eaf9e8 R08: 0000000000000000 R09: 0000000000000000
[15835.405456] R10: 0000000000000092 R11: 0000000000000000 R12: ffffffff810135bf
[15835.406185] R13: ffff880227eaf878 R14: 0000000100160015 R15: ffffffff8138278d
[15835.406916] FS: 0000000000000000(0000) GS:ffff880245200000(0000) knlGS:0000000000000000
[15835.407667] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15835.408410] CR2: 0000000000000008 CR3: 0000000225ab9000 CR4: 00000000001407e0
[15835.409165] DR0: 00007fbe591ef000 DR1: 0000000000000000 DR2: 0000000000000000
[15835.409924] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[15835.410679] Stack:
[15835.411433] 000000000000005c ffff880240f8d790 ffff880240f8d790 ffff880240f8dd00
[15835.412226] 0000000180230020 000000010000000f ffffffff8112ee12 0000000000000000
[15835.413002] ffff8802453d7260 000000020023001f ffff880227eaf968 ffffffff8138278d
[15835.413784] Call Trace:
[15835.414563] [<ffffffff8112ee12>] ? __delayacct_tsk_init+0x22/0x50
[15835.415358] [<ffffffff8138278d>] ? __debug_object_init+0x43d/0x450
[15835.416162] [<ffffffff811ce860>] ? set_track+0x70/0x140
[15835.416952] [<ffffffff811cf35d>] ? init_object+0x3d/0x70
[15835.417745] [<ffffffff8138278d>] ? __debug_object_init+0x43d/0x450
[15835.418533] [<ffffffff811d295b>] kmem_cache_alloc+0x1cb/0x1f0
[15835.419328] [<ffffffff8138278d>] __debug_object_init+0x43d/0x450
[15835.420114] [<ffffffff813827bb>] debug_object_init+0x1b/0x20
[15835.420904] [<ffffffff810e66d5>] hrtimer_init+0x25/0xb0
[15835.421701] [<ffffffff8109f069>] __sched_fork+0x99/0x230
[15835.422491] [<ffffffff810a59c9>] sched_fork+0x29/0x200
[15835.423282] [<ffffffff8107568c>] copy_process.part.26+0x65c/0x1a40
[15835.424077] [<ffffffff810abaf5>] ? local_clock+0x25/0x30
[15835.424865] [<ffffffff810c512c>] ? __lock_acquire.isra.31+0x22c/0x9f0
[15835.425658] [<ffffffff8108e340>] ? call_helper+0x20/0x20
[15835.426447] [<ffffffff81076c37>] do_fork+0xe7/0x490
[15835.427239] [<ffffffff810c512c>] ? __lock_acquire.isra.31+0x22c/0x9f0
[15835.428018] [<ffffffff81092f98>] ? process_one_work+0x178/0x550
[15835.428776] [<ffffffff81077006>] kernel_thread+0x26/0x30
[15835.429525] [<ffffffff8108e1b4>] __call_usermodehelper+0x64/0x80
[15835.430259] [<ffffffff8109301a>] process_one_work+0x1fa/0x550
[15835.430986] [<ffffffff81092f98>] ? process_one_work+0x178/0x550
[15835.431694] [<ffffffff8109348b>] worker_thread+0x11b/0x490
[15835.432374] [<ffffffff81093370>] ? process_one_work+0x550/0x550
[15835.433049] [<ffffffff81098c89>] kthread+0xf9/0x110
[15835.433694] [<ffffffff810abaf5>] ? local_clock+0x25/0x30
[15835.434328] [<ffffffff81098b90>] ? kthread_create_on_node+0x250/0x250
[15835.434947] [<ffffffff817cfe6c>] ret_from_fork+0x7c/0xb0
[15835.435543] [<ffffffff81098b90>] ? kthread_create_on_node+0x250/0x250
[15835.436148] Code: 00 02 00 00 49 c7 45 00 00 00 00 00 75 11 ff b5 78 ff ff ff 9d e8 7a 6d 98 ff 4c 89 e0 eb 0f e8 70 6e 98 ff ff b5 78 ff ff ff 9d <4c> 89 e0 48 8b 55 c8 65 48 33 14 25 28 00 00 00 74 3c e8 22 46
[15835.437494] sending NMI to other CPUs:
next prev parent reply other threads:[~2014-12-18 5:14 UTC|newest]
Thread overview: 488+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-11-14 21:31 frequent lockups in 3.18rc4 Dave Jones
2014-11-14 22:01 ` Linus Torvalds
2014-11-14 22:30 ` Dave Jones
2014-11-14 22:55 ` Thomas Gleixner
2014-11-14 23:32 ` Dave Jones
2014-11-15 0:36 ` Thomas Gleixner
2014-11-15 2:40 ` Dave Jones
2014-11-16 12:16 ` Thomas Gleixner
2014-11-15 1:59 ` Linus Torvalds
2014-11-17 21:22 ` Linus Torvalds
2014-11-17 22:31 ` Thomas Gleixner
2014-11-17 22:43 ` Thomas Gleixner
2014-11-17 22:58 ` Jens Axboe
2014-11-17 23:59 ` Linus Torvalds
2014-11-18 0:15 ` Thomas Gleixner
2014-11-17 23:04 ` Jens Axboe
2014-11-17 23:17 ` Thomas Gleixner
2014-11-18 2:23 ` Jens Axboe
2014-11-15 21:34 ` Dave Jones
2014-11-16 1:40 ` Dave Jones
2014-11-16 6:33 ` Linus Torvalds
2014-11-16 10:06 ` Markus Trippelsdorf
2014-11-16 18:33 ` Linus Torvalds
2014-11-17 17:03 ` Dave Jones
2014-11-17 19:59 ` Linus Torvalds
2014-11-18 2:09 ` Dave Jones
2014-11-18 2:21 ` Linus Torvalds
2014-11-18 2:39 ` Dave Jones
2014-11-18 2:51 ` Linus Torvalds
2014-11-18 14:52 ` Dave Jones
2014-11-18 17:20 ` Linus Torvalds
2014-11-18 19:28 ` Thomas Gleixner
2014-11-18 21:25 ` Don Zickus
2014-11-18 21:31 ` Dave Jones
2014-11-18 18:54 ` Thomas Gleixner
2014-11-18 21:55 ` Don Zickus
2014-11-18 22:02 ` Dave Jones
2014-11-19 14:41 ` Don Zickus
2014-11-19 15:03 ` Vivek Goyal
2014-11-19 15:38 ` Dave Jones
2014-11-19 16:28 ` Vivek Goyal
2014-11-20 16:10 ` Dave Jones
2014-11-20 16:48 ` Vivek Goyal
2014-11-20 17:38 ` Dave Jones
2014-11-21 9:46 ` Dave Young
2014-11-20 16:54 ` Vivek Goyal
2014-11-20 9:54 ` Dave Young
2014-11-19 2:19 ` Dave Jones
2014-11-19 4:40 ` Linus Torvalds
2014-11-19 4:59 ` Dave Jones
2014-11-19 5:15 ` Dave Jones
2014-11-20 14:36 ` Frederic Weisbecker
2014-11-19 14:59 ` Dave Jones
2014-11-19 17:22 ` Linus Torvalds
2014-11-19 17:40 ` Linus Torvalds
2014-11-19 19:02 ` Frederic Weisbecker
2014-11-19 19:03 ` Andy Lutomirski
2014-11-19 23:00 ` Frederic Weisbecker
2014-11-19 23:07 ` Andy Lutomirski
2014-11-19 23:13 ` Frederic Weisbecker
2014-11-19 21:56 ` Thomas Gleixner
2014-11-19 22:56 ` Frederic Weisbecker
2014-11-19 22:59 ` Andy Lutomirski
2014-11-19 23:07 ` Frederic Weisbecker
2014-11-19 23:09 ` Thomas Gleixner
2014-11-19 23:50 ` Frederic Weisbecker
2014-11-20 12:23 ` Tejun Heo
2014-11-20 21:58 ` Thomas Gleixner
2014-11-20 22:06 ` Andy Lutomirski
2014-11-20 22:11 ` Tejun Heo
2014-11-20 22:42 ` Thomas Gleixner
2014-11-20 23:05 ` Tejun Heo
2014-11-20 23:08 ` Andy Lutomirski
2014-11-20 23:34 ` Linus Torvalds
2014-11-20 23:39 ` Tejun Heo
2014-11-20 23:55 ` Andy Lutomirski
2014-11-21 16:27 ` Tejun Heo
2014-11-21 16:38 ` Andy Lutomirski
2014-11-21 16:48 ` Linus Torvalds
2014-11-21 17:08 ` Steven Rostedt
2014-11-21 17:19 ` Linus Torvalds
2014-11-21 17:22 ` Andy Lutomirski
2014-11-21 18:22 ` Linus Torvalds
2014-11-21 18:28 ` Andy Lutomirski
2014-11-21 19:06 ` Linus Torvalds
2014-11-21 19:23 ` Steven Rostedt
2014-11-21 19:34 ` Linus Torvalds
2014-11-21 19:46 ` Linus Torvalds
2014-11-21 19:52 ` Andy Lutomirski
2014-11-21 20:14 ` Josh Boyer
2014-11-21 20:16 ` Andy Lutomirski
2014-11-21 20:23 ` Josh Boyer
2014-11-24 18:48 ` Konrad Rzeszutek Wilk
2014-11-24 19:07 ` Josh Boyer
2014-11-25 5:36 ` Jürgen Groß
2014-11-25 17:22 ` Linus Torvalds
2014-11-21 20:00 ` Dave Jones
2014-11-21 20:02 ` Andy Lutomirski
2014-11-21 19:51 ` Thomas Gleixner
2014-11-21 20:00 ` Linus Torvalds
2014-11-21 20:16 ` Thomas Gleixner
2014-11-21 20:41 ` Linus Torvalds
2014-11-21 21:11 ` Thomas Gleixner
2014-11-21 22:55 ` Linus Torvalds
2014-11-21 23:03 ` Andy Lutomirski
2014-11-21 23:33 ` Linus Torvalds
2014-12-16 19:28 ` Peter Zijlstra
2014-12-16 20:46 ` Linus Torvalds
2014-12-16 21:19 ` Mel Gorman
2014-12-16 23:02 ` Peter Zijlstra
2014-12-17 0:00 ` Linus Torvalds
2014-12-17 0:41 ` Andy Lutomirski
2014-12-17 17:01 ` Konrad Rzeszutek Wilk
2014-12-17 17:14 ` Peter Zijlstra
2014-11-21 22:33 ` Konrad Rzeszutek Wilk
2014-11-22 1:17 ` Thomas Gleixner
2014-11-22 1:17 ` Thomas Gleixner
2014-11-21 22:33 ` Konrad Rzeszutek Wilk
2014-11-21 17:34 ` Steven Rostedt
2014-11-21 18:24 ` Linus Torvalds
2014-11-21 22:10 ` Frederic Weisbecker
2014-11-21 2:33 ` Steven Rostedt
2014-11-21 0:54 ` Thomas Gleixner
2014-11-21 14:13 ` Frederic Weisbecker
2014-11-21 16:25 ` Tejun Heo
2014-11-21 17:01 ` Steven Rostedt
2014-11-21 17:11 ` Steven Rostedt
2014-11-21 21:32 ` Frederic Weisbecker
2014-11-21 21:34 ` Andy Lutomirski
2014-11-21 21:50 ` Frederic Weisbecker
2014-11-21 22:45 ` Steven Rostedt
2014-11-21 21:44 ` Frederic Weisbecker
2014-11-22 0:11 ` Tejun Heo
2014-11-22 0:18 ` Linus Torvalds
2014-11-22 0:41 ` Andy Lutomirski
2014-11-19 23:54 ` Andy Lutomirski
2014-11-20 0:00 ` Thomas Gleixner
2014-11-20 0:30 ` Andy Lutomirski
2014-11-20 0:40 ` Linus Torvalds
2014-11-20 0:49 ` Andy Lutomirski
2014-11-20 1:07 ` Linus Torvalds
2014-11-20 1:16 ` Andy Lutomirski
2014-11-20 2:42 ` Linus Torvalds
2014-11-20 6:16 ` Andy Lutomirski
2014-11-19 19:15 ` Andy Lutomirski
2014-11-19 19:38 ` Linus Torvalds
2014-11-19 22:18 ` Dave Jones
2014-11-19 21:01 ` Andy Lutomirski
2014-11-19 21:47 ` Dave Jones
2014-11-19 21:58 ` Borislav Petkov
2014-11-19 22:18 ` Dave Jones
2014-11-20 10:33 ` Borislav Petkov
2014-11-19 21:56 ` [PATCH] x86, syscall: Fix _TIF_NOHZ handling in syscall_trace_enter_phase1 Andy Lutomirski
2014-11-19 22:13 ` Thomas Gleixner
2014-11-20 20:33 ` Linus Torvalds
2014-11-20 22:07 ` Thomas Gleixner
2014-11-20 22:04 ` [tip:x86/urgent] " tip-bot for Andy Lutomirski
2014-11-20 15:25 ` frequent lockups in 3.18rc4 Dave Jones
2014-11-20 19:43 ` Linus Torvalds
2014-11-20 20:06 ` Dave Jones
2014-11-20 20:37 ` Don Zickus
2014-11-20 20:51 ` Linus Torvalds
2014-11-21 6:37 ` Ingo Molnar
2014-11-21 14:50 ` Dave Jones
2014-11-25 12:22 ` Will Deacon
2014-12-01 11:48 ` Will Deacon
2014-12-01 17:05 ` Linus Torvalds
2014-12-01 17:10 ` Will Deacon
2014-12-01 17:53 ` Linus Torvalds
2014-12-01 18:25 ` Kirill A. Shutemov
2014-12-01 18:36 ` Linus Torvalds
2014-12-04 10:51 ` Will Deacon
2014-12-04 14:56 ` Dave Jones
2014-12-05 13:49 ` Will Deacon
2014-11-20 15:04 ` Frederic Weisbecker
2014-11-20 15:08 ` Frederic Weisbecker
2014-11-20 16:19 ` Dave Jones
2014-11-20 16:42 ` Frederic Weisbecker
2014-11-26 0:25 ` Dave Jones
2014-11-26 1:48 ` Linus Torvalds
2014-11-26 2:40 ` Dave Jones
2014-11-26 22:57 ` Dave Jones
2014-11-27 0:46 ` Linus Torvalds
2014-11-27 19:17 ` Linus Torvalds
2014-11-27 22:56 ` Dave Jones
2014-11-29 20:38 ` Dâniel Fraga
2014-11-30 20:45 ` Linus Torvalds
2014-11-30 21:21 ` Dâniel Fraga
2014-12-01 0:21 ` Linus Torvalds
2014-12-01 1:02 ` Dâniel Fraga
2014-12-01 19:14 ` Paul E. McKenney
2014-12-01 20:28 ` Dâniel Fraga
2014-12-01 20:36 ` Linus Torvalds
2014-12-01 23:08 ` Chris Mason
2014-12-01 23:25 ` Linus Torvalds
2014-12-01 23:44 ` Chris Mason
2014-12-02 0:39 ` Linus Torvalds
2014-12-02 14:13 ` Mike Galbraith
2014-12-02 16:33 ` Linus Torvalds
2014-12-02 17:14 ` Chris Mason
2014-12-03 18:41 ` Dave Jones
2014-12-03 18:45 ` Linus Torvalds
2014-12-03 19:00 ` Dave Jones
2014-12-03 19:25 ` Linus Torvalds
2014-12-03 19:30 ` Dave Jones
2014-12-03 19:48 ` Linus Torvalds
2014-12-03 20:09 ` Dave Jones
2014-12-03 20:37 ` Linus Torvalds
2014-12-03 20:55 ` Thomas Gleixner
2014-12-03 21:14 ` Linus Torvalds
2014-12-03 22:19 ` Thomas Gleixner
2014-12-03 23:21 ` Dave Jones
2014-12-03 23:49 ` Thomas Gleixner
2014-12-04 0:19 ` Linus Torvalds
2014-12-04 1:02 ` Thomas Gleixner
2014-12-04 0:20 ` Dave Jones
2014-12-04 0:59 ` Thomas Gleixner
2014-12-04 1:32 ` Dave Jones
2014-12-04 3:45 ` Dave Jones
2014-12-03 19:56 ` John Stultz
2014-12-03 20:37 ` Thomas Gleixner
2014-12-03 20:44 ` Dave Jones
2014-12-03 20:59 ` Thomas Gleixner
2014-12-03 21:05 ` Dave Jones
2014-12-03 21:48 ` Thomas Gleixner
2014-12-03 20:39 ` Thomas Gleixner
2014-12-04 3:15 ` Chris Mason
2014-12-04 5:49 ` Linus Torvalds
2014-12-04 14:57 ` Chris Mason
2014-12-04 15:22 ` Dave Hansen
2014-12-04 15:30 ` Chris Mason
2014-12-03 19:59 ` Chris Mason
2014-12-03 20:11 ` Dave Jones
2014-12-03 20:56 ` Chris Mason
2014-12-04 0:27 ` Dave Jones
2014-12-05 17:15 ` Dave Jones
2014-12-05 18:38 ` Linus Torvalds
2014-12-05 18:48 ` Dave Jones
2014-12-05 19:31 ` Linus Torvalds
2014-12-05 19:37 ` Dave Jones
2014-12-06 22:38 ` Thomas Gleixner
2014-12-06 9:37 ` Chuck Ebbert
2014-12-06 16:22 ` Martin van Es
2014-12-06 20:09 ` Linus Torvalds
2014-12-06 20:41 ` Linus Torvalds
2014-12-06 21:14 ` Martin van Es
2014-12-12 12:58 ` Martin van Es
2014-12-15 12:07 ` Martin van Es
2014-12-06 22:14 ` Thomas Gleixner
2014-12-05 19:04 ` Chris Mason
2014-12-05 19:29 ` Linus Torvalds
2014-12-11 14:54 ` Dave Jones
2014-12-11 21:49 ` Linus Torvalds
2014-12-11 21:52 ` Sasha Levin
2014-12-11 21:57 ` Chris Mason
2014-12-11 22:00 ` Sasha Levin
2014-12-11 22:36 ` Linus Torvalds
2014-12-11 22:57 ` Sasha Levin
2014-12-12 6:54 ` Ingo Molnar
2014-12-12 23:54 ` Sasha Levin
2014-12-13 0:23 ` Linus Torvalds
2014-12-13 0:34 ` Sasha Levin
2014-12-13 0:44 ` Linus Torvalds
2014-12-13 16:28 ` Jeff Chua
2014-12-13 2:32 ` Dave Jones
2014-12-11 21:57 ` Borislav Petkov
2014-12-12 3:03 ` Dave Jones
2014-12-12 4:45 ` Dave Jones
2014-12-12 14:38 ` Dave Jones
2014-12-12 18:24 ` Paul E. McKenney
2014-12-12 18:10 ` Paul E. McKenney
2014-12-12 18:42 ` Dave Jones
2014-12-12 18:54 ` Dave Jones
2014-12-12 19:14 ` Linus Torvalds
2014-12-12 19:23 ` Dave Jones
2014-12-12 19:58 ` David Lang
2014-12-12 20:20 ` Linus Torvalds
2014-12-13 7:43 ` Ingo Molnar
2014-12-12 20:34 ` Paul E. McKenney
2014-12-12 21:23 ` Sasha Levin
2014-12-13 0:58 ` Paul E. McKenney
2014-12-13 12:08 ` Paul E. McKenney
2014-12-13 8:30 ` Ingo Molnar
2014-12-13 15:53 ` Sasha Levin
2014-12-13 18:07 ` Paul E. McKenney
2014-12-14 17:50 ` Paul E. McKenney
2014-12-14 23:46 ` Sasha Levin
2014-12-15 0:11 ` Paul E. McKenney
2014-12-15 1:20 ` Sasha Levin
2014-12-15 6:33 ` Paul E. McKenney
2014-12-15 12:56 ` Paul E. McKenney
2014-12-15 13:16 ` Sasha Levin
2014-12-16 3:40 ` Paul E. McKenney
2014-12-13 7:36 ` [PATCH] sched: Fix lost reschedule in __cond_resched() Ingo Molnar
2014-12-14 18:04 ` Frederic Weisbecker
2014-12-14 19:43 ` Ingo Molnar
2014-12-14 19:50 ` Linus Torvalds
2014-12-14 20:30 ` Frederic Weisbecker
2014-12-13 8:19 ` frequent lockups in 3.18rc4 Ingo Molnar
2014-12-13 8:27 ` Ingo Molnar
2014-12-13 14:15 ` Sasha Levin
2014-12-13 16:59 ` Dave Jones
2014-12-13 18:04 ` Paul E. McKenney
2014-12-13 20:41 ` Dave Jones
2014-12-14 4:04 ` Paul E. McKenney
2014-12-13 22:36 ` Dave Jones
2014-12-13 22:40 ` Linus Torvalds
2014-12-13 22:59 ` Linus Torvalds
2014-12-13 23:09 ` Linus Torvalds
2014-12-13 23:35 ` Al Viro
2014-12-13 23:38 ` Linus Torvalds
2014-12-13 23:47 ` Al Viro
2014-12-14 0:14 ` Linus Torvalds
2014-12-14 0:33 ` Al Viro
2014-12-14 1:35 ` Linus Torvalds
2014-12-14 3:14 ` Al Viro
2014-12-15 0:18 ` Al Viro
2014-12-13 23:39 ` Al Viro
2014-12-14 23:46 ` Dave Jones
2014-12-15 0:38 ` Linus Torvalds
2014-12-15 0:42 ` Dave Jones
2014-12-15 5:47 ` Linus Torvalds
2014-12-15 5:57 ` Dave Jones
2014-12-15 18:21 ` Linus Torvalds
2014-12-15 23:46 ` Linus Torvalds
2014-12-18 2:42 ` Sasha Levin
2014-12-18 2:45 ` Linus Torvalds
2014-12-18 5:13 ` Dave Jones [this message]
2014-12-18 15:54 ` Chris Mason
2014-12-18 16:12 ` Dave Jones
2014-12-19 2:45 ` Dave Jones
2014-12-19 3:49 ` Linus Torvalds
2014-12-19 3:58 ` Dave Jones
2014-12-19 4:03 ` Dave Jones
2014-12-19 4:48 ` Linus Torvalds
2014-12-19 11:35 ` Peter Zijlstra
2014-12-19 14:55 ` Dave Jones
2014-12-19 15:14 ` Chris Mason
2014-12-19 19:15 ` Linus Torvalds
2014-12-19 19:44 ` Peter Zijlstra
2014-12-19 19:51 ` Linus Torvalds
2014-12-19 20:46 ` Linus Torvalds
2014-12-19 20:54 ` Dave Jones
2014-12-19 22:05 ` Linus Torvalds
2014-12-20 16:49 ` Dave Jones
2014-12-19 20:31 ` Chris Mason
2014-12-19 20:36 ` Dave Jones
2014-12-19 23:22 ` Thomas Gleixner
2014-12-20 0:12 ` Chris Mason
2014-12-20 1:06 ` Thomas Gleixner
2014-12-19 23:14 ` Thomas Gleixner
2014-12-19 23:55 ` Linus Torvalds
2014-12-20 1:00 ` Thomas Gleixner
2014-12-20 1:57 ` Linus Torvalds
2014-12-20 18:25 ` Linus Torvalds
2014-12-20 21:16 ` Linus Torvalds
2014-12-21 3:52 ` Paul E. McKenney
2014-12-21 21:22 ` Linus Torvalds
2014-12-21 22:19 ` Linus Torvalds
2014-12-21 22:32 ` Dave Jones
2014-12-21 23:58 ` Linus Torvalds
2014-12-22 0:41 ` Linus Torvalds
2014-12-22 0:52 ` Linus Torvalds
2014-12-22 1:22 ` Dave Jones
2014-12-22 3:11 ` Paul E. McKenney
2014-12-22 19:47 ` Linus Torvalds
2014-12-22 20:06 ` Linus Torvalds
2014-12-22 22:57 ` Dave Jones
2014-12-22 23:59 ` Linus Torvalds
2014-12-23 14:56 ` Dave Jones
2014-12-24 13:58 ` Sasha Levin
2014-12-24 3:01 ` Dave Jones
2014-12-26 16:34 ` Dave Jones
2014-12-26 18:12 ` Dave Jones
2014-12-26 20:57 ` Linus Torvalds
2014-12-26 21:20 ` Dave Jones
2014-12-26 22:57 ` Dave Jones
2014-12-26 23:16 ` Linus Torvalds
2014-12-27 0:36 ` Dave Jones
2014-12-27 3:14 ` Linus Torvalds
2014-12-27 16:48 ` Dave Jones
2014-12-26 23:30 ` Linus Torvalds
2014-12-27 0:39 ` Dave Jones
2014-12-27 2:53 ` Dave Jones
2015-01-03 0:27 ` John Stultz
2015-01-03 14:58 ` Sasha Levin
2015-01-04 19:46 ` Linus Torvalds
2015-01-06 1:17 ` John Stultz
2015-01-06 1:25 ` Linus Torvalds
2015-01-06 2:05 ` John Stultz
2014-12-22 23:59 ` John Stultz
2014-12-23 0:46 ` Linus Torvalds
2014-12-27 20:33 ` Paul E. McKenney
2015-01-12 10:05 ` Thomas Gleixner
2014-12-19 14:30 ` Chris Mason
2014-12-19 15:12 ` Dave Jones
2014-12-18 18:54 ` Linus Torvalds
2014-12-15 14:00 ` Borislav Petkov
2014-12-18 21:17 ` save_xstate_sig (Re: frequent lockups in 3.18rc4) Andy Lutomirski
2014-12-18 21:34 ` Linus Torvalds
2014-12-18 21:41 ` Andy Lutomirski
2014-12-18 21:37 ` Dave Jones
2014-12-17 18:22 ` frequent lockups in 3.18rc4 Dave Jones
2014-12-17 18:57 ` Dave Jones
2014-12-17 19:24 ` Dave Jones
2014-12-17 19:51 ` Linus Torvalds
2014-12-17 20:16 ` Dave Jones
2014-12-17 19:41 ` Linus Torvalds
2014-12-06 5:04 ` Gene Heskett
2014-12-02 17:47 ` Mike Galbraith
2014-12-13 8:11 ` Ingo Molnar
2014-12-13 9:57 ` Mike Galbraith
2014-12-17 11:13 ` Peter Zijlstra
2014-12-02 19:32 ` Dave Jones
2014-12-02 23:32 ` Sasha Levin
2014-12-03 0:09 ` Linus Torvalds
2014-12-03 0:25 ` Sasha Levin
2014-12-05 5:00 ` Sasha Levin
2014-12-05 6:38 ` Linus Torvalds
2014-12-05 15:03 ` Sasha Levin
2014-12-05 18:15 ` Linus Torvalds
2014-12-07 14:58 ` Sasha Levin
2014-12-07 18:24 ` Paul E. McKenney
2014-12-07 19:43 ` Paul E. McKenney
2014-12-07 23:28 ` Sasha Levin
2014-12-08 5:20 ` Paul E. McKenney
2014-12-08 14:33 ` Sasha Levin
2014-12-08 15:28 ` Sasha Levin
2014-12-08 15:57 ` Paul E. McKenney
2014-12-08 16:34 ` Sasha Levin
2014-12-08 15:56 ` Paul E. McKenney
2014-12-07 23:53 ` Linus Torvalds
2014-12-02 19:31 ` Dave Jones
2014-12-02 21:17 ` Linus Torvalds
2014-12-02 20:30 ` Dave Jones
2014-12-02 20:48 ` Paul E. McKenney
2014-12-01 23:08 ` Paul E. McKenney
2014-12-02 16:43 ` Dâniel Fraga
2014-12-02 17:04 ` Paul E. McKenney
2014-12-02 17:14 ` Dâniel Fraga
2014-12-02 18:42 ` Paul E. McKenney
2014-12-02 18:47 ` Dâniel Fraga
2014-12-02 19:11 ` Paul E. McKenney
2014-12-02 19:24 ` Dâniel Fraga
2014-12-02 20:56 ` Paul E. McKenney
2014-12-02 22:01 ` Dâniel Fraga
2014-12-02 22:10 ` Paul E. McKenney
2014-12-02 22:18 ` Dâniel Fraga
2014-12-02 22:35 ` Paul E. McKenney
2014-12-02 22:10 ` Linus Torvalds
2014-12-02 22:16 ` Dâniel Fraga
2014-12-03 3:21 ` Dâniel Fraga
2014-12-03 4:14 ` Linus Torvalds
2014-12-03 4:51 ` Dâniel Fraga
2014-12-03 6:02 ` Chris Rorvick
2014-12-03 15:22 ` Linus Torvalds
2014-12-04 8:43 ` Dâniel Fraga
2014-12-04 16:18 ` Linus Torvalds
2014-12-04 16:52 ` Frederic Weisbecker
2014-12-04 17:25 ` Dâniel Fraga
2014-12-04 17:47 ` Linus Torvalds
2014-12-04 18:07 ` Dâniel Fraga
2014-12-03 14:54 ` Tejun Heo
2014-12-02 18:09 ` Paul E. McKenney
2014-12-02 18:41 ` Dâniel Fraga
2014-12-02 17:08 ` Linus Torvalds
2014-12-02 17:16 ` Dâniel Fraga
2014-12-02 8:40 ` Lai Jiangshan
2014-12-02 16:58 ` Paul E. McKenney
2014-12-02 16:58 ` Dâniel Fraga
2014-12-02 17:17 ` Paul E. McKenney
2014-12-03 2:03 ` Lai Jiangshan
2014-12-03 5:22 ` Paul E. McKenney
2014-12-01 16:56 ` Don Zickus
2014-11-26 4:39 ` Jürgen Groß
[not found] ` <CA+55aFx1SiFBzmA=k9jHxi3cZE3Ei_+2NHepujgf86KEvkz8eQ@mail.gmail.com>
2014-11-26 5:11 ` Dave Jones
2014-11-26 5:24 ` Juergen Gross
2014-11-26 5:52 ` Linus Torvalds
2014-11-26 6:21 ` Linus Torvalds
2014-11-26 6:52 ` Juergen Gross
2014-11-26 9:44 ` Juergen Gross
2014-11-26 14:34 ` Dave Jones
2014-11-26 17:37 ` Linus Torvalds
2014-11-20 15:28 ` Frederic Weisbecker
2014-11-17 15:07 ` Don Zickus
-- strict thread matches above, loose matches on Subject: below --
2014-12-16 3:04 Hillf Danton
2015-02-12 11:09 Martin van Es
2015-02-12 16:01 ` Linus Torvalds
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20141218051327.GA31988@redhat.com \
--to=davej@redhat.com \
--cc=clm@fb.com \
--cc=fragabr@gmail.com \
--cc=hpa@linux.intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=oleg@redhat.com \
--cc=paulmck@linux.vnet.ibm.com \
--cc=peterz@infradead.org \
--cc=sasha.levin@oracle.com \
--cc=sbsiddha@gmail.com \
--cc=torvalds@linux-foundation.org \
--cc=umgwanakikbuti@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.