From: Dave Jones <davej@redhat.com>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: "Chris Mason" <clm@fb.com>,
"Mike Galbraith" <umgwanakikbuti@gmail.com>,
"Ingo Molnar" <mingo@kernel.org>,
"Peter Zijlstra" <peterz@infradead.org>,
"Dâniel Fraga" <fragabr@gmail.com>,
"Sasha Levin" <sasha.levin@oracle.com>,
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
"Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>
Subject: Re: frequent lockups in 3.18rc4
Date: Wed, 3 Dec 2014 14:00:45 -0500 [thread overview]
Message-ID: <20141203190045.GB32005@redhat.com> (raw)
In-Reply-To: <CA+55aFzLprvtdLGDXgRr=k3QqO824uQSzbxT-b4vu_4pryMtSA@mail.gmail.com>
On Wed, Dec 03, 2014 at 10:45:57AM -0800, Linus Torvalds wrote:
> On Wed, Dec 3, 2014 at 10:41 AM, Dave Jones <davej@redhat.com> wrote:
> >
> > I've been stuck on this kernel for a few days now trying to prove it
> > good/bad one way or the other, and I'm leaning towards good, given
> > that it recovers, even though the traces look similar.
>
> Ugh. But this does *not* happen with 3.16, right? Even the non-fatal case?
correct. at least not in any of the runs that I did to date.
> If so, I'd be inclined to call it "bad". But there might well be two
> bugs: one that makes that NMI watchdog trigger, and another one that
> then makes it be a hard lockup. I'd think it would be good to figure
> out the "NMI watchdog starts triggering" one first, though.
I think you're right.
So right after sending my last mail, I rebooted, and restarted the run
on the same kernel again.
As I was writing this mail, this happened.
[ 524.420897] NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [trinity-c178:20182]
and that's all that made it over the console. I couldn't log in via ssh,
and thought "ah-ha, so it IS bad". I walked over to reboot it, and
found I could actually log in on the console. check out this dmesg..
[ 503.683055] Clocksource tsc unstable (delta = -95946009388 ns)
[ 503.692038] Switched to clocksource hpet
[ 524.420897] NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [trinity-c178:20182]
[ 524.420972] Modules linked in: fuse tun rfcomm llc2 af_key nfnetlink scsi_transport_iscsi can_bcm bnep can_raw nfc caif_socket caif af_802154 ieee802154 phonet af_rxrpc bluetooth can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 cfg80211 rfkill coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm nfsd auth_rpcgss oid_registry nfs_acl lockd sunrpc snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_controller snd_hda_codec crct10dif_pclmul crc32c_intel ghash_clmulni_intel e1000e snd_hwdep snd_seq snd_seq_device snd_pcm snd_timer microcode snd serio_raw pcspkr usb_debug ptp pps_core shpchp soundcore
[ 524.421288] CPU: 0 PID: 20182 Comm: trinity-c178 Not tainted 3.17.0-rc1+ #112
[ 524.421351] task: ffff8801cd63c4d0 ti: ffff8801d2138000 task.ti: ffff8801d2138000
[ 524.421377] RIP: 0010:[<ffffffff8136968d>] [<ffffffff8136968d>] copy_user_handle_tail+0x6d/0x90
[ 524.421411] RSP: 0018:ffff8801d213bf00 EFLAGS: 00000202
[ 524.421430] RAX: 000000000007a8d9 RBX: ffffffff817b2c64 RCX: 0000000000000000
[ 524.421455] RDX: 0000000000056ddc RSI: ffff88023412baf5 RDI: ffff88023412baf4
[ 524.421480] RBP: ffff8801d213bf00 R08: 0000000000000000 R09: 0000000000000000
[ 524.421504] R10: 0000000000000100 R11: 0000000000000000 R12: ffffffff817b92d0
[ 524.421528] R13: 00007f6a24fe0000 R14: ffff8801d2138000 R15: 0000000000000001
[ 524.421552] FS: 00007f6a24fd0740(0000) GS:ffff880244000000(0000) knlGS:0000000000000000
[ 524.421579] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 524.421600] CR2: 00007f6a24fe0000 CR3: 00000002053c8000 CR4: 00000000001407f0
[ 524.421624] DR0: 00007f6670c66000 DR1: 0000000000000000 DR2: 0000000000000000
[ 524.421648] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[ 524.421672] Stack:
[ 524.421683] ffff8801d213bf78 ffffffff812e4a85 00007f6a2452f068 0000000000000000
[ 524.421716] ffffffff3fffffff 00000000000003e8 00007f6a2452f000 00000000000000f8
[ 524.421748] 0000000000000000 00000000a417dc9d 00000000000000f8 00007f6a2452f000
[ 524.421781] Call Trace:
[ 524.422754] [<ffffffff812e4a85>] SyS_add_key+0xd5/0x240
[ 524.423736] [<ffffffff817b2264>] tracesys+0xdd/0xe2
[ 524.424713] Code: c0 74 d3 85 d2 89 d0 74 39 85 c9 74 35 45 31 c0 eb 0c 0f 1f 40 00 83 ea 01 74 17 48 89 f7 48 8d 77 01 44 89 c1 0f 1f 00 c6 07 00 <0f> 1f 00 85 c9 74 e4 0f 1f 00 5d c3 0f 1f 80 00 00 00 00 31 c0
[ 524.426861] sending NMI to other CPUs:
[ 524.427867] NMI backtrace for cpu 3
[ 524.428868] CPU: 3 PID: 20165 Comm: trinity-c161 Not tainted 3.17.0-rc1+ #112
[ 524.430914] task: ffff8801fe67dbc0 ti: ffff8801fe70c000 task.ti: ffff8801fe70c000
[ 524.431951] RIP: 0010:[<ffffffff810f99da>] [<ffffffff810f99da>] generic_exec_single+0xea/0x1a0
[ 524.433004] RSP: 0018:ffff8801fe70fc40 EFLAGS: 00000202
[ 524.434051] RAX: 0000000000000000 RBX: ffff8801fe70fc40 RCX: 0000000000000038
[ 524.435108] RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000000
[ 524.436165] RBP: ffff8801fe70fc90 R08: ffff880242bfa3f0 R09: 0000000000000000
[ 524.437221] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[ 524.438278] R13: 0000000000000001 R14: ffff880238ef1290 R15: ffffffff8115fd30
[ 524.439339] FS: 00007f6a24fd0740(0000) GS:ffff880244600000(0000) knlGS:0000000000000000
[ 524.440415] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 524.441494] CR2: 00007f6a23446001 CR3: 00000001cd627000 CR4: 00000000001407e0
[ 524.442579] DR0: 00007f6670c66000 DR1: 0000000000000000 DR2: 0000000000000000
[ 524.443665] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[ 524.444749] Stack:
[ 524.445825] 0000000000000000 ffffffff8115fd30 ffff880238ef1290 0000000000000003
[ 524.446926] 00000000af3b5f31 00000000ffffffff 0000000000000000 ffffffff8115fd30
[ 524.448028] ffff880238ef1290 0000000000000001 ffff8801fe70fcd0 ffffffff810f9b5a
[ 524.449115] Call Trace:
[ 524.450179] [<ffffffff8115fd30>] ? perf_duration_warn+0x70/0x70
[ 524.451259] [<ffffffff8115fd30>] ? perf_duration_warn+0x70/0x70
[ 524.452334] [<ffffffff810f9b5a>] smp_call_function_single+0x6a/0xe0
[ 524.453411] [<ffffffff810a2dbb>] ? preempt_count_sub+0x6b/0xf0
[ 524.454483] [<ffffffff8116036a>] perf_event_read+0xca/0xd0
[ 524.455549] [<ffffffff81160400>] perf_event_read_value+0x90/0xe0
[ 524.456618] [<ffffffff81161b1e>] perf_read+0x20e/0x360
[ 524.457689] [<ffffffff810bffc6>] ? lock_release_holdtime.part.24+0xe6/0x160
[ 524.458761] [<ffffffff81161910>] ? cpu_clock_event_init+0x40/0x40
[ 524.459827] [<ffffffff811dfdd3>] do_loop_readv_writev+0x63/0x90
[ 524.460887] [<ffffffff81161910>] ? cpu_clock_event_init+0x40/0x40
[ 524.461944] [<ffffffff811e1c97>] do_readv_writev+0x267/0x280
[ 524.462996] [<ffffffff81375a47>] ? debug_smp_processor_id+0x17/0x20
[ 524.464029] [<ffffffff810bffc6>] ? lock_release_holdtime.part.24+0xe6/0x160
[ 524.465040] [<ffffffff810a2c5d>] ? get_parent_ip+0xd/0x50
[ 524.466027] [<ffffffff810a2dbb>] ? preempt_count_sub+0x6b/0xf0
[ 524.466995] [<ffffffff817b1377>] ? _raw_spin_unlock_irq+0x37/0x60
[ 524.467940] [<ffffffff810e672a>] ? do_setitimer+0x1ca/0x250
[ 524.468874] [<ffffffff811e1ce9>] vfs_readv+0x39/0x50
[ 524.469791] [<ffffffff811e1dac>] SyS_readv+0x5c/0x100
[ 524.470689] [<ffffffff817b2264>] tracesys+0xdd/0xe2
[ 524.471568] Code: 00 4c 1d 00 48 89 de 48 03 14 c5 e0 af d1 81 48 89 df e8 5a 47 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 18 01 74 0b 0f 1f 00 f3 90 <f6> 43 18 01 75 f8 31 c0 48 8b 4d d0 65 48 33 0c 25 28 00 00 00
[ 524.473500] NMI backtrace for cpu 1
[ 524.473584] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 45.633 msecs
[ 524.475347] CPU: 1 PID: 20241 Comm: trinity-c237 Not tainted 3.17.0-rc1+ #112
[ 524.477246] task: ffff8801e32c16f0 ti: ffff88008696c000 task.ti: ffff88008696c000
[ 524.478212] RIP: 0010:[<ffffffff810f99d8>] [<ffffffff810f99d8>] generic_exec_single+0xe8/0x1a0
[ 524.479189] RSP: 0018:ffff88008696fc40 EFLAGS: 00000202
[ 524.480152] RAX: ffff880062d6bc00 RBX: ffff88008696fc40 RCX: ffff880062d6bc40
[ 524.481130] RDX: ffff8802441d4c00 RSI: ffff88008696fc40 RDI: ffff88008696fc40
[ 524.482108] RBP: ffff88008696fc90 R08: 0000000000000001 R09: 0000000000000001
[ 524.483084] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[ 524.484051] R13: 0000000000000001 R14: ffff880238ef1bd8 R15: ffffffff8115fd30
[ 524.485022] FS: 00007f6a24fd0740(0000) GS:ffff880244200000(0000) knlGS:0000000000000000
[ 524.486001] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 524.486975] CR2: 0000000000000000 CR3: 00000000869c1000 CR4: 00000000001407e0
[ 524.487952] DR0: 00007f6670c66000 DR1: 0000000000000000 DR2: 0000000000000000
[ 524.488935] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[ 524.489891] Stack:
[ 524.490817] ffff880062d6bc40 ffffffff8115fd30 ffff880238ef1bd8 0000000000000003
[ 524.491770] 00000000d88cff08 00000000ffffffff 0000000000000000 ffffffff8115fd30
[ 524.492729] ffff880238ef1bd8 0000000000000001 ffff88008696fcd0 ffffffff810f9b5a
[ 524.493684] Call Trace:
[ 524.494611] [<ffffffff8115fd30>] ? perf_duration_warn+0x70/0x70
[ 524.495530] [<ffffffff8115fd30>] ? perf_duration_warn+0x70/0x70
[ 524.496433] [<ffffffff810f9b5a>] smp_call_function_single+0x6a/0xe0
[ 524.497334] [<ffffffff810a2dbb>] ? preempt_count_sub+0x6b/0xf0
[ 524.498227] [<ffffffff8116036a>] perf_event_read+0xca/0xd0
[ 524.499117] [<ffffffff81160400>] perf_event_read_value+0x90/0xe0
[ 524.500007] [<ffffffff81161b1e>] perf_read+0x20e/0x360
[ 524.500895] [<ffffffff810bffc6>] ? lock_release_holdtime.part.24+0xe6/0x160
[ 524.501786] [<ffffffff81161910>] ? cpu_clock_event_init+0x40/0x40
[ 524.502668] [<ffffffff811dfdd3>] do_loop_readv_writev+0x63/0x90
[ 524.503548] [<ffffffff81161910>] ? cpu_clock_event_init+0x40/0x40
[ 524.504419] [<ffffffff811e1c97>] do_readv_writev+0x267/0x280
[ 524.505295] [<ffffffff81375a47>] ? debug_smp_processor_id+0x17/0x20
[ 524.506172] [<ffffffff810bffc6>] ? lock_release_holdtime.part.24+0xe6/0x160
[ 524.507045] [<ffffffff810a2c5d>] ? get_parent_ip+0xd/0x50
[ 524.507911] [<ffffffff810a2dbb>] ? preempt_count_sub+0x6b/0xf0
[ 524.508774] [<ffffffff817b1377>] ? _raw_spin_unlock_irq+0x37/0x60
[ 524.509634] [<ffffffff810e672a>] ? do_setitimer+0x1ca/0x250
[ 524.510488] [<ffffffff811e1ce9>] vfs_readv+0x39/0x50
[ 524.511330] [<ffffffff811e1dac>] SyS_readv+0x5c/0x100
[ 524.512180] [<ffffffff817b2264>] tracesys+0xdd/0xe2
[ 524.513026] Code: c7 c2 00 4c 1d 00 48 89 de 48 03 14 c5 e0 af d1 81 48 89 df e8 5a 47 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 18 01 74 0b 0f 1f 00 <f3> 90 f6 43 18 01 75 f8 31 c0 48 8b 4d d0 65 48 33 0c 25 28 00
[ 524.514900] NMI backtrace for cpu 2
[ 524.514903] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 87.033 msecs
[ 524.516709] CPU: 2 PID: 20160 Comm: trinity-c156 Not tainted 3.17.0-rc1+ #112
[ 524.518568] task: ffff88006b945bc0 ti: ffff880062d68000 task.ti: ffff880062d68000
[ 524.519521] RIP: 0010:[<ffffffff810f99de>] [<ffffffff810f99de>] generic_exec_single+0xee/0x1a0
[ 524.520488] RSP: 0018:ffff880062d6bc40 EFLAGS: 00000202
[ 524.521456] RAX: ffff8801fe70fc00 RBX: ffff880062d6bc40 RCX: ffff8801fe70fc40
[ 524.522424] RDX: ffff8802441d4c00 RSI: ffff880062d6bc40 RDI: ffff880062d6bc40
[ 524.523394] RBP: ffff880062d6bc90 R08: 0000000000000001 R09: 0000000000000001
[ 524.524360] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[ 524.525325] R13: 0000000000000001 R14: ffff880238ef5cd0 R15: ffffffff8115fd30
[ 524.526293] FS: 00007f6a24fd0740(0000) GS:ffff880244400000(0000) knlGS:0000000000000000
[ 524.527269] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 524.528248] CR2: 00000000019ca288 CR3: 000000007a7bb000 CR4: 00000000001407e0
[ 524.529238] DR0: 00007f6670c66000 DR1: 0000000000000000 DR2: 0000000000000000
[ 524.530230] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[ 524.531200] Stack:
[ 524.532141] ffff8801fe70fc40 ffffffff8115fd30 ffff880238ef5cd0 0000000000000003
[ 524.533104] 000000008b5d668d 00000000ffffffff 0000000000000000 ffffffff8115fd30
[ 524.534065] ffff880238ef5cd0 0000000000000001 ffff880062d6bcd0 ffffffff810f9b5a
[ 524.535014] Call Trace:
[ 524.535938] [<ffffffff8115fd30>] ? perf_duration_warn+0x70/0x70
[ 524.536854] [<ffffffff8115fd30>] ? perf_duration_warn+0x70/0x70
[ 524.537756] [<ffffffff810f9b5a>] smp_call_function_single+0x6a/0xe0
[ 524.538648] [<ffffffff810a2dbb>] ? preempt_count_sub+0x6b/0xf0
[ 524.539536] [<ffffffff8116036a>] perf_event_read+0xca/0xd0
[ 524.540425] [<ffffffff81160400>] perf_event_read_value+0x90/0xe0
[ 524.541317] [<ffffffff81161b1e>] perf_read+0x20e/0x360
[ 524.542202] [<ffffffff810bffc6>] ? lock_release_holdtime.part.24+0xe6/0x160
[ 524.543089] [<ffffffff81161910>] ? cpu_clock_event_init+0x40/0x40
[ 524.543967] [<ffffffff811dfdd3>] do_loop_readv_writev+0x63/0x90
[ 524.544840] [<ffffffff81161910>] ? cpu_clock_event_init+0x40/0x40
[ 524.545709] [<ffffffff811e1c97>] do_readv_writev+0x267/0x280
[ 524.546581] [<ffffffff81375a47>] ? debug_smp_processor_id+0x17/0x20
[ 524.547455] [<ffffffff810bffc6>] ? lock_release_holdtime.part.24+0xe6/0x160
[ 524.548327] [<ffffffff810a2c5d>] ? get_parent_ip+0xd/0x50
[ 524.549188] [<ffffffff810a2dbb>] ? preempt_count_sub+0x6b/0xf0
[ 524.550046] [<ffffffff817b1377>] ? _raw_spin_unlock_irq+0x37/0x60
[ 524.550905] [<ffffffff810e672a>] ? do_setitimer+0x1ca/0x250
[ 524.551755] [<ffffffff811e1ce9>] vfs_readv+0x39/0x50
[ 524.552600] [<ffffffff811e1dac>] SyS_readv+0x5c/0x100
[ 524.553444] [<ffffffff817b2264>] tracesys+0xdd/0xe2
[ 524.554281] Code: 48 89 de 48 03 14 c5 e0 af d1 81 48 89 df e8 5a 47 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 18 01 74 0b 0f 1f 00 f3 90 f6 43 18 01 <75> f8 31 c0 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 8e 00
[ 524.556148] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 128.279 msecs
[ 548.406844] NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [trinity-c178:20182]
[ 548.407766] Modules linked in: fuse tun rfcomm llc2 af_key nfnetlink scsi_transport_iscsi can_bcm bnep can_raw nfc caif_socket caif af_802154 ieee802154 phonet af_rxrpc bluetooth can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 cfg80211 rfkill coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm nfsd auth_rpcgss oid_registry nfs_acl lockd sunrpc snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_controller snd_hda_codec crct10dif_pclmul crc32c_intel ghash_clmulni_intel e1000e snd_hwdep snd_seq snd_seq_device snd_pcm snd_timer microcode snd serio_raw pcspkr usb_debug ptp pps_core shpchp soundcore
[ 548.412860] CPU: 0 PID: 20182 Comm: trinity-c178 Tainted: G L 3.17.0-rc1+ #112
[ 548.414955] task: ffff8801cd63c4d0 ti: ffff8801d2138000 task.ti: ffff8801d2138000
[ 548.416020] RIP: 0010:[<ffffffff8136968d>] [<ffffffff8136968d>] copy_user_handle_tail+0x6d/0x90
[ 548.417110] RSP: 0018:ffff8801d213bf00 EFLAGS: 00000202
[ 548.418188] RAX: 000000000007a8d9 RBX: ffffffff817b2c64 RCX: 0000000000000000
[ 548.419278] RDX: 0000000000056ddc RSI: ffff88023412baf5 RDI: ffff88023412baf4
[ 548.420372] RBP: ffff8801d213bf00 R08: 0000000000000000 R09: 0000000000000000
[ 548.421473] R10: 0000000000000100 R11: 0000000000000000 R12: ffffffff817b92d0
[ 548.422576] R13: 00007f6a24fe0000 R14: ffff8801d2138000 R15: 0000000000000001
[ 548.423662] FS: 00007f6a24fd0740(0000) GS:ffff880244000000(0000) knlGS:0000000000000000
[ 548.424731] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 548.425801] CR2: 00007f6a24fe0000 CR3: 00000002053c8000 CR4: 00000000001407f0
[ 548.426870] DR0: 00007f6670c66000 DR1: 0000000000000000 DR2: 0000000000000000
[ 548.427923] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[ 548.428956] Stack:
[ 548.429968] ffff8801d213bf78 ffffffff812e4a85 00007f6a2452f068 0000000000000000
[ 548.430992] ffffffff3fffffff 00000000000003e8 00007f6a2452f000 00000000000000f8
[ 548.432009] 0000000000000000 00000000a417dc9d 00000000000000f8 00007f6a2452f000
[ 548.433023] Call Trace:
[ 548.434031] [<ffffffff812e4a85>] SyS_add_key+0xd5/0x240
[ 548.435046] [<ffffffff817b2264>] tracesys+0xdd/0xe2
[ 548.436057] Code: c0 74 d3 85 d2 89 d0 74 39 85 c9 74 35 45 31 c0 eb 0c 0f 1f 40 00 83 ea 01 74 17 48 89 f7 48 8d 77 01 44 89 c1 0f 1f 00 c6 07 00 <0f> 1f 00 85 c9 74 e4 0f 1f 00 5d c3 0f 1f 80 00 00 00 00 31 c0
[ 548.438271] sending NMI to other CPUs:
[ 548.439311] NMI backtrace for cpu 3
[ 548.440341] CPU: 3 PID: 20165 Comm: trinity-c161 Tainted: G L 3.17.0-rc1+ #112
[ 548.442472] task: ffff8801fe67dbc0 ti: ffff8801fe70c000 task.ti: ffff8801fe70c000
[ 548.443553] RIP: 0010:[<ffffffff810f99de>] [<ffffffff810f99de>] generic_exec_single+0xee/0x1a0
[ 548.444639] RSP: 0018:ffff8801fe70fc40 EFLAGS: 00000202
[ 548.445718] RAX: 0000000000000000 RBX: ffff8801fe70fc40 RCX: 0000000000000038
[ 548.446803] RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000000
[ 548.447882] RBP: ffff8801fe70fc90 R08: ffff880242bfa3f0 R09: 0000000000000000
[ 548.448957] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[ 548.450039] R13: 0000000000000001 R14: ffff880238ef1290 R15: ffffffff8115fd30
[ 548.451120] FS: 00007f6a24fd0740(0000) GS:ffff880244600000(0000) knlGS:0000000000000000
[ 548.452211] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 548.453305] CR2: 00007f6a23446001 CR3: 00000001cd627000 CR4: 00000000001407e0
[ 548.454411] DR0: 00007f6670c66000 DR1: 0000000000000000 DR2: 0000000000000000
[ 548.455515] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[ 548.456617] Stack:
[ 548.457711] 0000000000000000 ffffffff8115fd30 ffff880238ef1290 0000000000000003
[ 548.458831] 00000000af3b5f31 00000000ffffffff 0000000000000000 ffffffff8115fd30
[ 548.459954] ffff880238ef1290 0000000000000001 ffff8801fe70fcd0 ffffffff810f9b5a
[ 548.461078] Call Trace:
[ 548.462187] [<ffffffff8115fd30>] ? perf_duration_warn+0x70/0x70
[ 548.463310] [<ffffffff8115fd30>] ? perf_duration_warn+0x70/0x70
[ 548.464425] [<ffffffff810f9b5a>] smp_call_function_single+0x6a/0xe0
[ 548.465541] [<ffffffff810a2dbb>] ? preempt_count_sub+0x6b/0xf0
[ 548.466656] [<ffffffff8116036a>] perf_event_read+0xca/0xd0
[ 548.467772] [<ffffffff81160400>] perf_event_read_value+0x90/0xe0
[ 548.468887] [<ffffffff81161b1e>] perf_read+0x20e/0x360
[ 548.470003] [<ffffffff810bffc6>] ? lock_release_holdtime.part.24+0xe6/0x160
[ 548.471127] [<ffffffff81161910>] ? cpu_clock_event_init+0x40/0x40
[ 548.472230] [<ffffffff811dfdd3>] do_loop_readv_writev+0x63/0x90
[ 548.473310] [<ffffffff81161910>] ? cpu_clock_event_init+0x40/0x40
[ 548.474375] [<ffffffff811e1c97>] do_readv_writev+0x267/0x280
[ 548.475432] [<ffffffff81375a47>] ? debug_smp_processor_id+0x17/0x20
[ 548.476470] [<ffffffff810bffc6>] ? lock_release_holdtime.part.24+0xe6/0x160
[ 548.477486] [<ffffffff810a2c5d>] ? get_parent_ip+0xd/0x50
[ 548.478479] [<ffffffff810a2dbb>] ? preempt_count_sub+0x6b/0xf0
[ 548.479452] [<ffffffff817b1377>] ? _raw_spin_unlock_irq+0x37/0x60
[ 548.480402] [<ffffffff810e672a>] ? do_setitimer+0x1ca/0x250
[ 548.481346] [<ffffffff811e1ce9>] vfs_readv+0x39/0x50
[ 548.482266] [<ffffffff811e1dac>] SyS_readv+0x5c/0x100
[ 548.483164] [<ffffffff817b2264>] tracesys+0xdd/0xe2
[ 548.484050] Code: 48 89 de 48 03 14 c5 e0 af d1 81 48 89 df e8 5a 47 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 18 01 74 0b 0f 1f 00 f3 90 f6 43 18 01 <75> f8 31 c0 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 8e 00
[ 548.485991] NMI backtrace for cpu 2
[ 548.486903] CPU: 2 PID: 20160 Comm: trinity-c156 Tainted: G L 3.17.0-rc1+ #112
[ 548.488787] task: ffff88006b945bc0 ti: ffff880062d68000 task.ti: ffff880062d68000
[ 548.489743] RIP: 0010:[<ffffffff810f99da>] [<ffffffff810f99da>] generic_exec_single+0xea/0x1a0
[ 548.490709] RSP: 0018:ffff880062d6bc40 EFLAGS: 00000202
[ 548.491667] RAX: ffff8801fe70fc00 RBX: ffff880062d6bc40 RCX: ffff8801fe70fc40
[ 548.492629] RDX: ffff8802441d4c00 RSI: ffff880062d6bc40 RDI: ffff880062d6bc40
[ 548.493590] RBP: ffff880062d6bc90 R08: 0000000000000001 R09: 0000000000000001
[ 548.494553] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[ 548.495511] R13: 0000000000000001 R14: ffff880238ef5cd0 R15: ffffffff8115fd30
[ 548.496462] FS: 00007f6a24fd0740(0000) GS:ffff880244400000(0000) knlGS:0000000000000000
[ 548.497423] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 548.498382] CR2: 00000000019ca288 CR3: 000000007a7bb000 CR4: 00000000001407e0
[ 548.499347] DR0: 00007f6670c66000 DR1: 0000000000000000 DR2: 0000000000000000
[ 548.500305] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[ 548.501264] Stack:
[ 548.502193] ffff8801fe70fc40 ffffffff8115fd30 ffff880238ef5cd0 0000000000000003
[ 548.503131] 000000008b5d668d 00000000ffffffff 0000000000000000 ffffffff8115fd30
[ 548.504073] ffff880238ef5cd0 0000000000000001 ffff880062d6bcd0 ffffffff810f9b5a
[ 548.505015] Call Trace:
[ 548.505940] [<ffffffff8115fd30>] ? perf_duration_warn+0x70/0x70
[ 548.506857] [<ffffffff8115fd30>] ? perf_duration_warn+0x70/0x70
[ 548.507753] [<ffffffff810f9b5a>] smp_call_function_single+0x6a/0xe0
[ 548.508639] [<ffffffff810a2dbb>] ? preempt_count_sub+0x6b/0xf0
[ 548.509518] [<ffffffff8116036a>] perf_event_read+0xca/0xd0
[ 548.510391] [<ffffffff81160400>] perf_event_read_value+0x90/0xe0
[ 548.511261] [<ffffffff81161b1e>] perf_read+0x20e/0x360
[ 548.512130] [<ffffffff810bffc6>] ? lock_release_holdtime.part.24+0xe6/0x160
[ 548.513005] [<ffffffff81161910>] ? cpu_clock_event_init+0x40/0x40
[ 548.513877] [<ffffffff811dfdd3>] do_loop_readv_writev+0x63/0x90
[ 548.514740] [<ffffffff81161910>] ? cpu_clock_event_init+0x40/0x40
[ 548.515600] [<ffffffff811e1c97>] do_readv_writev+0x267/0x280
[ 548.516451] [<ffffffff81375a47>] ? debug_smp_processor_id+0x17/0x20
[ 548.517313] [<ffffffff810bffc6>] ? lock_release_holdtime.part.24+0xe6/0x160
[ 548.518177] [<ffffffff810a2c5d>] ? get_parent_ip+0xd/0x50
[ 548.519031] [<ffffffff810a2dbb>] ? preempt_count_sub+0x6b/0xf0
[ 548.519878] [<ffffffff817b1377>] ? _raw_spin_unlock_irq+0x37/0x60
[ 548.520719] [<ffffffff810e672a>] ? do_setitimer+0x1ca/0x250
[ 548.521563] [<ffffffff811e1ce9>] vfs_readv+0x39/0x50
[ 548.522397] [<ffffffff811e1dac>] SyS_readv+0x5c/0x100
[ 548.523224] [<ffffffff817b2264>] tracesys+0xdd/0xe2
[ 548.524052] Code: 00 4c 1d 00 48 89 de 48 03 14 c5 e0 af d1 81 48 89 df e8 5a 47 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 18 01 74 0b 0f 1f 00 f3 90 <f6> 43 18 01 75 f8 31 c0 48 8b 4d d0 65 48 33 0c 25 28 00 00 00
[ 548.525897] NMI backtrace for cpu 1
[ 548.526765] CPU: 1 PID: 20241 Comm: trinity-c237 Tainted: G L 3.17.0-rc1+ #112
[ 548.528575] task: ffff8801e32c16f0 ti: ffff88008696c000 task.ti: ffff88008696c000
[ 548.529499] RIP: 0010:[<ffffffff810f99de>] [<ffffffff810f99de>] generic_exec_single+0xee/0x1a0
[ 548.530433] RSP: 0018:ffff88008696fc40 EFLAGS: 00000202
[ 548.531363] RAX: ffff880062d6bc00 RBX: ffff88008696fc40 RCX: ffff880062d6bc40
[ 548.532305] RDX: ffff8802441d4c00 RSI: ffff88008696fc40 RDI: ffff88008696fc40
[ 548.533244] RBP: ffff88008696fc90 R08: 0000000000000001 R09: 0000000000000001
[ 548.534181] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[ 548.535114] R13: 0000000000000001 R14: ffff880238ef1bd8 R15: ffffffff8115fd30
[ 548.536048] FS: 00007f6a24fd0740(0000) GS:ffff880244200000(0000) knlGS:0000000000000000
[ 548.536995] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 548.537936] CR2: 0000000000000000 CR3: 00000000869c1000 CR4: 00000000001407e0
[ 548.538888] DR0: 00007f6670c66000 DR1: 0000000000000000 DR2: 0000000000000000
[ 548.539842] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[ 548.540795] Stack:
[ 548.541745] ffff880062d6bc40 ffffffff8115fd30 ffff880238ef1bd8 0000000000000003
[ 548.542699] 00000000d88cff08 00000000ffffffff 0000000000000000 ffffffff8115fd30
[ 548.543640] ffff880238ef1bd8 0000000000000001 ffff88008696fcd0 ffffffff810f9b5a
[ 548.544574] Call Trace:
[ 548.545491] [<ffffffff8115fd30>] ? perf_duration_warn+0x70/0x70
[ 548.546404] [<ffffffff8115fd30>] ? perf_duration_warn+0x70/0x70
[ 548.547297] [<ffffffff810f9b5a>] smp_call_function_single+0x6a/0xe0
[ 548.548179] [<ffffffff810a2dbb>] ? preempt_count_sub+0x6b/0xf0
[ 548.549053] [<ffffffff8116036a>] perf_event_read+0xca/0xd0
[ 548.549920] [<ffffffff81160400>] perf_event_read_value+0x90/0xe0
[ 548.550789] [<ffffffff81161b1e>] perf_read+0x20e/0x360
[ 548.551651] [<ffffffff810bffc6>] ? lock_release_holdtime.part.24+0xe6/0x160
[ 548.552520] [<ffffffff81161910>] ? cpu_clock_event_init+0x40/0x40
[ 548.553388] [<ffffffff811dfdd3>] do_loop_readv_writev+0x63/0x90
[ 548.554247] [<ffffffff81161910>] ? cpu_clock_event_init+0x40/0x40
[ 548.555103] [<ffffffff811e1c97>] do_readv_writev+0x267/0x280
[ 548.555952] [<ffffffff81375a47>] ? debug_smp_processor_id+0x17/0x20
[ 548.556809] [<ffffffff810bffc6>] ? lock_release_holdtime.part.24+0xe6/0x160
[ 548.557665] [<ffffffff810a2c5d>] ? get_parent_ip+0xd/0x50
[ 548.558511] [<ffffffff810a2dbb>] ? preempt_count_sub+0x6b/0xf0
[ 548.559352] [<ffffffff817b1377>] ? _raw_spin_unlock_irq+0x37/0x60
[ 548.560189] [<ffffffff810e672a>] ? do_setitimer+0x1ca/0x250
[ 548.561021] [<ffffffff811e1ce9>] vfs_readv+0x39/0x50
[ 548.561847] [<ffffffff811e1dac>] SyS_readv+0x5c/0x100
[ 548.562664] [<ffffffff817b2264>] tracesys+0xdd/0xe2
[ 548.563483] Code: 48 89 de 48 03 14 c5 e0 af d1 81 48 89 df e8 5a 47 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 18 01 74 0b 0f 1f 00 f3 90 f6 43 18 01 <75> f8 31 c0 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 8e 00
[ 564.237567] INFO: rcu_preempt self-detected stall on CPU
[ 564.238434] 0: (23495594 ticks this GP) idle=ee5/140000000000001/0 softirq=31067/31067
[ 564.239318] (t=6000 jiffies g=12425 c=12424 q=0)
[ 564.240203] Task dump for CPU 0:
[ 564.241078] trinity-c178 R running task 13424 20182 19467 0x10000008
[ 564.241975] ffff8801cd63c4d0 00000000a417dc9d ffff880244003dc8 ffffffff810a4406
[ 564.242880] ffffffff810a4372 0000000000000000 ffffffff81c50240 0000000000000086
[ 564.243789] ffff880244003de0 ffffffff810a83e9 0000000000000001 ffff880244003e10
[ 564.244697] Call Trace:
[ 564.245597] <IRQ> [<ffffffff810a4406>] sched_show_task+0x116/0x180
[ 564.246511] [<ffffffff810a4372>] ? sched_show_task+0x82/0x180
[ 564.247420] [<ffffffff810a83e9>] dump_cpu_task+0x39/0x40
[ 564.248337] [<ffffffff810d6360>] rcu_dump_cpu_stacks+0xa0/0xe0
[ 564.249247] [<ffffffff810ddba3>] rcu_check_callbacks+0x503/0x810
[ 564.250155] [<ffffffff81375a63>] ? __this_cpu_preempt_check+0x13/0x20
[ 564.251069] [<ffffffff810e5c93>] ? hrtimer_run_queues+0x43/0x130
[ 564.251985] [<ffffffff810e43e7>] update_process_times+0x47/0x70
[ 564.252902] [<ffffffff810f4c8a>] tick_sched_timer+0x4a/0x1a0
[ 564.253796] [<ffffffff810e4a71>] ? __run_hrtimer+0x81/0x250
[ 564.254671] [<ffffffff810e4a71>] __run_hrtimer+0x81/0x250
[ 564.255541] [<ffffffff810f4c40>] ? tick_init_highres+0x20/0x20
[ 564.256404] [<ffffffff810e5697>] hrtimer_interrupt+0x107/0x260
[ 564.257251] [<ffffffff81031cc4>] local_apic_timer_interrupt+0x34/0x60
[ 564.258086] [<ffffffff817b4b8f>] smp_apic_timer_interrupt+0x3f/0x60
[ 564.258907] [<ffffffff817b2faf>] apic_timer_interrupt+0x6f/0x80
[ 564.259721] <EOI> [<ffffffff817b2c64>] ? retint_restore_args+0xe/0xe
[ 564.260541] [<ffffffff8136968d>] ? copy_user_handle_tail+0x6d/0x90
[ 564.261360] [<ffffffff812e4a85>] SyS_add_key+0xd5/0x240
[ 564.262176] [<ffffffff817b2264>] tracesys+0xdd/0xe2
[ 572.432766] NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [trinity-c156:20160]
[ 572.433593] Modules linked in: fuse tun rfcomm llc2 af_key nfnetlink scsi_transport_iscsi can_bcm bnep can_raw nfc caif_socket caif af_802154 ieee802154 phonet af_rxrpc bluetooth can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 cfg80211 rfkill coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm nfsd auth_rpcgss oid_registry nfs_acl lockd sunrpc snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_controller snd_hda_codec crct10dif_pclmul crc32c_intel ghash_clmulni_intel e1000e snd_hwdep snd_seq snd_seq_device snd_pcm snd_timer microcode snd serio_raw pcspkr usb_debug ptp pps_core shpchp soundcore
[ 572.438194] CPU: 2 PID: 20160 Comm: trinity-c156 Tainted: G L 3.17.0-rc1+ #112
[ 572.440070] task: ffff88006b945bc0 ti: ffff880062d68000 task.ti: ffff880062d68000
[ 572.441020] RIP: 0010:[<ffffffff810f99de>] [<ffffffff810f99de>] generic_exec_single+0xee/0x1a0
[ 572.441982] RSP: 0018:ffff880062d6bc40 EFLAGS: 00000202
[ 572.442935] RAX: ffff8801fe70fc00 RBX: ffffffff817b2c64 RCX: ffff8801fe70fc40
[ 572.443898] RDX: ffff8802441d4c00 RSI: ffff880062d6bc40 RDI: ffff880062d6bc40
[ 572.444868] RBP: ffff880062d6bc90 R08: 0000000000000001 R09: 0000000000000001
[ 572.445836] R10: 0000000000000000 R11: 0000000000000000 R12: ffff880062d6bbb8
[ 572.446808] R13: 0000000000406040 R14: ffff880062d68000 R15: ffff88006b945bc0
[ 572.447777] FS: 00007f6a24fd0740(0000) GS:ffff880244400000(0000) knlGS:0000000000000000
[ 572.448759] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 572.449742] CR2: 00000000019ca288 CR3: 000000007a7bb000 CR4: 00000000001407e0
[ 572.450733] DR0: 00007f6670c66000 DR1: 0000000000000000 DR2: 0000000000000000
[ 572.451722] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[ 572.452710] Stack:
[ 572.453692] ffff8801fe70fc40 ffffffff8115fd30 ffff880238ef5cd0 0000000000000003
[ 572.454698] 000000008b5d668d 00000000ffffffff 0000000000000000 ffffffff8115fd30
[ 572.455712] ffff880238ef5cd0 0000000000000001 ffff880062d6bcd0 ffffffff810f9b5a
[ 572.456721] Call Trace:
[ 572.457721] [<ffffffff8115fd30>] ? perf_duration_warn+0x70/0x70
[ 572.458732] [<ffffffff8115fd30>] ? perf_duration_warn+0x70/0x70
[ 572.459736] [<ffffffff810f9b5a>] smp_call_function_single+0x6a/0xe0
[ 572.460745] [<ffffffff810a2dbb>] ? preempt_count_sub+0x6b/0xf0
[ 572.461748] [<ffffffff8116036a>] perf_event_read+0xca/0xd0
[ 572.462763] [<ffffffff81160400>] perf_event_read_value+0x90/0xe0
[ 572.463776] [<ffffffff81161b1e>] perf_read+0x20e/0x360
[ 572.464765] [<ffffffff810bffc6>] ? lock_release_holdtime.part.24+0xe6/0x160
[ 572.465738] [<ffffffff81161910>] ? cpu_clock_event_init+0x40/0x40
[ 572.466709] [<ffffffff811dfdd3>] do_loop_readv_writev+0x63/0x90
[ 572.467669] [<ffffffff81161910>] ? cpu_clock_event_init+0x40/0x40
[ 572.468627] [<ffffffff811e1c97>] do_readv_writev+0x267/0x280
[ 572.469586] [<ffffffff81375a47>] ? debug_smp_processor_id+0x17/0x20
[ 572.470543] [<ffffffff810bffc6>] ? lock_release_holdtime.part.24+0xe6/0x160
[ 572.471493] [<ffffffff810a2c5d>] ? get_parent_ip+0xd/0x50
[ 572.472442] [<ffffffff810a2dbb>] ? preempt_count_sub+0x6b/0xf0
[ 572.473386] [<ffffffff817b1377>] ? _raw_spin_unlock_irq+0x37/0x60
[ 572.474322] [<ffffffff810e672a>] ? do_setitimer+0x1ca/0x250
[ 572.475260] [<ffffffff811e1ce9>] vfs_readv+0x39/0x50
[ 572.476186] [<ffffffff811e1dac>] SyS_readv+0x5c/0x100
[ 572.477112] [<ffffffff817b2264>] tracesys+0xdd/0xe2
[ 572.478038] Code: 48 89 de 48 03 14 c5 e0 af d1 81 48 89 df e8 5a 47 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 18 01 74 0b 0f 1f 00 f3 90 f6 43 18 01 <75> f8 31 c0 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 8e 00
[ 572.480077] sending NMI to other CPUs:
[ 572.481045] NMI backtrace for cpu 1
[ 572.482011] CPU: 1 PID: 20241 Comm: trinity-c237 Tainted: G L 3.17.0-rc1+ #112
[ 572.484001] task: ffff8801e32c16f0 ti: ffff88008696c000 task.ti: ffff88008696c000
[ 572.485020] RIP: 0010:[<ffffffff810f99de>] [<ffffffff810f99de>] generic_exec_single+0xee/0x1a0
[ 572.486053] RSP: 0018:ffff88008696fc40 EFLAGS: 00000202
[ 572.487084] RAX: ffff880062d6bc00 RBX: ffff88008696fc40 RCX: ffff880062d6bc40
[ 572.488124] RDX: ffff8802441d4c00 RSI: ffff88008696fc40 RDI: ffff88008696fc40
[ 572.489163] RBP: ffff88008696fc90 R08: 0000000000000001 R09: 0000000000000001
[ 572.490199] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[ 572.491231] R13: 0000000000000001 R14: ffff880238ef1bd8 R15: ffffffff8115fd30
[ 572.492262] FS: 00007f6a24fd0740(0000) GS:ffff880244200000(0000) knlGS:0000000000000000
[ 572.493299] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 572.494343] CR2: 0000000000000000 CR3: 00000000869c1000 CR4: 00000000001407e0
[ 572.495380] DR0: 00007f6670c66000 DR1: 0000000000000000 DR2: 0000000000000000
[ 572.496396] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[ 572.497393] Stack:
[ 572.498357] ffff880062d6bc40 ffffffff8115fd30 ffff880238ef1bd8 0000000000000003
[ 572.499325] 00000000d88cff08 00000000ffffffff 0000000000000000 ffffffff8115fd30
[ 572.500276] ffff880238ef1bd8 0000000000000001 ffff88008696fcd0 ffffffff810f9b5a
[ 572.501223] Call Trace:
[ 572.502140] [<ffffffff8115fd30>] ? perf_duration_warn+0x70/0x70
[ 572.503053] [<ffffffff8115fd30>] ? perf_duration_warn+0x70/0x70
[ 572.503947] [<ffffffff810f9b5a>] smp_call_function_single+0x6a/0xe0
[ 572.504837] [<ffffffff810a2dbb>] ? preempt_count_sub+0x6b/0xf0
[ 572.505720] [<ffffffff8116036a>] perf_event_read+0xca/0xd0
[ 572.506600] [<ffffffff81160400>] perf_event_read_value+0x90/0xe0
[ 572.507477] [<ffffffff81161b1e>] perf_read+0x20e/0x360
[ 572.508350] [<ffffffff810bffc6>] ? lock_release_holdtime.part.24+0xe6/0x160
[ 572.509229] [<ffffffff81161910>] ? cpu_clock_event_init+0x40/0x40
[ 572.510102] [<ffffffff811dfdd3>] do_loop_readv_writev+0x63/0x90
[ 572.510968] [<ffffffff81161910>] ? cpu_clock_event_init+0x40/0x40
[ 572.511828] [<ffffffff811e1c97>] do_readv_writev+0x267/0x280
[ 572.512689] [<ffffffff81375a47>] ? debug_smp_processor_id+0x17/0x20
[ 572.513556] [<ffffffff810bffc6>] ? lock_release_holdtime.part.24+0xe6/0x160
[ 572.514413] [<ffffffff810a2c5d>] ? get_parent_ip+0xd/0x50
[ 572.515267] [<ffffffff810a2dbb>] ? preempt_count_sub+0x6b/0xf0
[ 572.516114] [<ffffffff817b1377>] ? _raw_spin_unlock_irq+0x37/0x60
[ 572.516958] [<ffffffff810e672a>] ? do_setitimer+0x1ca/0x250
[ 572.517794] [<ffffffff811e1ce9>] vfs_readv+0x39/0x50
[ 572.518627] [<ffffffff811e1dac>] SyS_readv+0x5c/0x100
[ 572.519462] [<ffffffff817b2264>] tracesys+0xdd/0xe2
[ 572.520291] Code: 48 89 de 48 03 14 c5 e0 af d1 81 48 89 df e8 5a 47 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 18 01 74 0b 0f 1f 00 f3 90 f6 43 18 01 <75> f8 31 c0 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 8e 00
[ 572.522141] NMI backtrace for cpu 3
[ 572.523018] CPU: 3 PID: 20165 Comm: trinity-c161 Tainted: G L 3.17.0-rc1+ #112
[ 572.524827] task: ffff8801fe67dbc0 ti: ffff8801fe70c000 task.ti: ffff8801fe70c000
[ 572.525748] RIP: 0010:[<ffffffff810f99de>] [<ffffffff810f99de>] generic_exec_single+0xee/0x1a0
[ 572.526677] RSP: 0018:ffff8801fe70fc40 EFLAGS: 00000202
[ 572.527605] RAX: 0000000000000000 RBX: ffff8801fe70fc40 RCX: 0000000000000038
[ 572.528545] RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000000
[ 572.529480] RBP: ffff8801fe70fc90 R08: ffff880242bfa3f0 R09: 0000000000000000
[ 572.530416] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[ 572.531350] R13: 0000000000000001 R14: ffff880238ef1290 R15: ffffffff8115fd30
[ 572.532287] FS: 00007f6a24fd0740(0000) GS:ffff880244600000(0000) knlGS:0000000000000000
[ 572.533230] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 572.534174] CR2: 00007f6a23446001 CR3: 00000001cd627000 CR4: 00000000001407e0
[ 572.535133] DR0: 00007f6670c66000 DR1: 0000000000000000 DR2: 0000000000000000
[ 572.536091] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[ 572.537048] Stack:
[ 572.537979] 0000000000000000 ffffffff8115fd30 ffff880238ef1290 0000000000000003
[ 572.538915] 00000000af3b5f31 00000000ffffffff 0000000000000000 ffffffff8115fd30
[ 572.539850] ffff880238ef1290 0000000000000001 ffff8801fe70fcd0 ffffffff810f9b5a
[ 572.540790] Call Trace:
[ 572.541711] [<ffffffff8115fd30>] ? perf_duration_warn+0x70/0x70
[ 572.542629] [<ffffffff8115fd30>] ? perf_duration_warn+0x70/0x70
[ 572.543522] [<ffffffff810f9b5a>] smp_call_function_single+0x6a/0xe0
[ 572.544408] [<ffffffff810a2dbb>] ? preempt_count_sub+0x6b/0xf0
[ 572.545285] [<ffffffff8116036a>] perf_event_read+0xca/0xd0
[ 572.546154] [<ffffffff81160400>] perf_event_read_value+0x90/0xe0
[ 572.547027] [<ffffffff81161b1e>] perf_read+0x20e/0x360
[ 572.547890] [<ffffffff810bffc6>] ? lock_release_holdtime.part.24+0xe6/0x160
[ 572.548765] [<ffffffff81161910>] ? cpu_clock_event_init+0x40/0x40
[ 572.549637] [<ffffffff811dfdd3>] do_loop_readv_writev+0x63/0x90
[ 572.550497] [<ffffffff81161910>] ? cpu_clock_event_init+0x40/0x40
[ 572.551353] [<ffffffff811e1c97>] do_readv_writev+0x267/0x280
[ 572.552210] [<ffffffff81375a47>] ? debug_smp_processor_id+0x17/0x20
[ 572.553066] [<ffffffff810bffc6>] ? lock_release_holdtime.part.24+0xe6/0x160
[ 572.553924] [<ffffffff810a2c5d>] ? get_parent_ip+0xd/0x50
[ 572.554775] [<ffffffff810a2dbb>] ? preempt_count_sub+0x6b/0xf0
[ 572.555618] [<ffffffff817b1377>] ? _raw_spin_unlock_irq+0x37/0x60
[ 572.556453] [<ffffffff810e672a>] ? do_setitimer+0x1ca/0x250
[ 572.557284] [<ffffffff811e1ce9>] vfs_readv+0x39/0x50
[ 572.558111] [<ffffffff811e1dac>] SyS_readv+0x5c/0x100
[ 572.558934] [<ffffffff817b2264>] tracesys+0xdd/0xe2
[ 572.559756] Code: 48 89 de 48 03 14 c5 e0 af d1 81 48 89 df e8 5a 47 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 18 01 74 0b 0f 1f 00 f3 90 f6 43 18 01 <75> f8 31 c0 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 8e 00
[ 572.561589] NMI backtrace for cpu 0
[ 572.562451] CPU: 0 PID: 20182 Comm: trinity-c178 Tainted: G L 3.17.0-rc1+ #112
[ 572.564252] task: ffff8801cd63c4d0 ti: ffff8801d2138000 task.ti: ffff8801d2138000
[ 572.565166] RIP: 0010:[<ffffffff8103cc16>] [<ffffffff8103cc16>] read_hpet+0x16/0x20
[ 572.566090] RSP: 0018:ffff880244003e70 EFLAGS: 00000046
[ 572.567011] RAX: 00000000e8dd201c RBX: 000000000001cc86 RCX: ffff8802441d1118
[ 572.567939] RDX: 0000000000010001 RSI: ffffffff81a86870 RDI: ffffffff81c28680
[ 572.568869] RBP: ffff880244003e70 R08: 0000000000000000 R09: 0000000000000000
[ 572.569802] R10: 0000000000000000 R11: 0000000000000000 R12: 000000854428057e
[ 572.570732] R13: ffff8802441ce060 R14: ffff8802441cda80 R15: 0000008baf2377a5
[ 572.571665] FS: 00007f6a24fd0740(0000) GS:ffff880244000000(0000) knlGS:0000000000000000
[ 572.572608] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 572.573555] CR2: 00007f6a24fe0000 CR3: 00000002053c8000 CR4: 00000000001407f0
[ 572.574508] DR0: 00007f6670c66000 DR1: 0000000000000000 DR2: 0000000000000000
[ 572.575459] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[ 572.576414] Stack:
[ 572.577362] ffff880244003e98 ffffffff810eb574 ffffffff810f4c63 ffff8801d213be58
[ 572.578323] ffff880244003f40 ffff880244003ec8 ffffffff810f4c63 ffff8802441cda80
[ 572.579255] ffff8802441ce060 ffff880244003f40 ffff8802441cdb08 ffff880244003f08
[ 572.580181] Call Trace:
[ 572.581092] <IRQ>
[ 572.581988] [<ffffffff810eb574>] ktime_get+0x94/0x120
[ 572.582865] [<ffffffff810f4c63>] ? tick_sched_timer+0x23/0x1a0
[ 572.583735] [<ffffffff810f4c63>] tick_sched_timer+0x23/0x1a0
[ 572.584592] [<ffffffff810e4a71>] __run_hrtimer+0x81/0x250
[ 572.585447] [<ffffffff810f4c40>] ? tick_init_highres+0x20/0x20
[ 572.586297] [<ffffffff810e5697>] hrtimer_interrupt+0x107/0x260
[ 572.587148] [<ffffffff81031cc4>] local_apic_timer_interrupt+0x34/0x60
[ 572.588004] [<ffffffff817b4b8f>] smp_apic_timer_interrupt+0x3f/0x60
[ 572.588859] [<ffffffff817b2faf>] apic_timer_interrupt+0x6f/0x80
[ 572.589705] <EOI>
[ 572.590542] [<ffffffff817b2c64>] ? retint_restore_args+0xe/0xe
[ 572.591379] [<ffffffff8136968d>] ? copy_user_handle_tail+0x6d/0x90
[ 572.592225] [<ffffffff812e4a85>] SyS_add_key+0xd5/0x240
[ 572.593071] [<ffffffff817b2264>] tracesys+0xdd/0xe2
[ 572.593901] Code: 00 29 c7 ba 00 00 00 00 b8 c2 ff ff ff 83 ff 7f 5d 0f 4f c2 c3 0f 1f 44 00 00 55 48 8b 05 a3 0c 0b 01 48 89 e5 8b 80 f0 00 00 00 <89> c0 5d c3 66 0f 1f 44 00 00 0f 1f 44 00 00 8b 0d d9 0b 0b 01
[ 599.566886] [sched_delayed] sched: RT throttling activated
[ 599.573324] end_request: I/O error, dev sda, sector 0
[ 624.402393] NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [swapper/2:0]
[ 624.403521] Modules linked in: fuse tun rfcomm llc2 af_key nfnetlink scsi_transport_iscsi can_bcm bnep can_raw nfc caif_socket caif af_802154 ieee802154 phonet af_rxrpc bluetooth can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 cfg80211 rfkill coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm nfsd auth_rpcgss oid_registry nfs_acl lockd sunrpc snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_controller snd_hda_codec crct10dif_pclmul crc32c_intel ghash_clmulni_intel e1000e snd_hwdep snd_seq snd_seq_device snd_pcm snd_timer microcode snd serio_raw pcspkr usb_debug ptp pps_core shpchp soundcore
[ 624.409477] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G L 3.17.0-rc1+ #112
[ 624.411931] task: ffff880242b744d0 ti: ffff880242414000 task.ti: ffff880242414000
[ 624.413170] RIP: 0010:[<ffffffff81645849>] [<ffffffff81645849>] cpuidle_enter_state+0x79/0x1c0
[ 624.414396] RSP: 0018:ffff880242417e60 EFLAGS: 00000246
[ 624.415614] RAX: 0000000000000000 RBX: ffff880242b744d0 RCX: 0000000000000019
[ 624.416843] RDX: 20c49ba5e353f7cf RSI: 000000000003cd60 RDI: 002e512580cfca6e
[ 624.418074] RBP: ffff880242417e98 R08: 000000008bafc4e0 R09: 0000000000000000
[ 624.419397] R10: 0000000000000000 R11: 0000000000000000 R12: ffff880242417df0
[ 624.420626] R13: ffffffff810bfc5e R14: ffff880242417dd0 R15: 0000000000000210
[ 624.421864] FS: 0000000000000000(0000) GS:ffff880244400000(0000) knlGS:0000000000000000
[ 624.423132] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 624.424380] CR2: 00007f15cfbaf000 CR3: 0000000001c11000 CR4: 00000000001407e0
[ 624.425640] DR0: 00007f6670c66000 DR1: 0000000000000000 DR2: 0000000000000000
[ 624.426904] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[ 624.428164] Stack:
[ 624.429393] 0000009177337316 ffffffff81cae7f0 ffffffff81d1d290 ffffe8ffff204da8
[ 624.430630] ffff880242414000 ffffffff81cae620 ffff880242414000 ffff880242417ea8
[ 624.431865] ffffffff81645a47 ffff880242417f10 ffffffff810b9fb4 ffff880242417fd8
[ 624.433085] Call Trace:
[ 624.434242] [<ffffffff81645a47>] cpuidle_enter+0x17/0x20
[ 624.435413] [<ffffffff810b9fb4>] cpu_startup_entry+0x384/0x410
[ 624.436588] [<ffffffff8102ff37>] start_secondary+0x237/0x340
[ 624.437761] Code: d0 48 89 df ff 50 48 41 89 c5 e8 b3 5c aa ff 44 8b 63 04 49 89 c7 0f 1f 44 00 00 e8 a2 19 b0 ff fb 48 ba cf f7 53 e3 a5 9b c4 20 <4c> 2b 7d c8 4c 89 f8 49 c1 ff 3f 48 f7 ea b8 ff ff ff 7f 48 c1
[ 624.440251] sending NMI to other CPUs:
[ 624.441500] NMI backtrace for cpu 1
[ 624.442624] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G L 3.17.0-rc1+ #112
[ 624.444912] task: ffff880242b716f0 ti: ffff88024240c000 task.ti: ffff88024240c000
[ 624.446070] RIP: 0010:[<ffffffff813c9e65>] [<ffffffff813c9e65>] intel_idle+0xd5/0x180
[ 624.447236] RSP: 0018:ffff88024240fe20 EFLAGS: 00000046
[ 624.448397] RAX: 0000000000000032 RBX: 0000000000000010 RCX: 0000000000000001
[ 624.449570] RDX: 0000000000000000 RSI: ffff88024240ffd8 RDI: 0000000000000001
[ 624.450735] RBP: ffff88024240fe50 R08: 000000008bafc4e0 R09: 0000000000000000
[ 624.451894] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000005
[ 624.453053] R13: 0000000000000032 R14: 0000000000000004 R15: ffff88024240c000
[ 624.454213] FS: 0000000000000000(0000) GS:ffff880244200000(0000) knlGS:0000000000000000
[ 624.455369] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 624.456531] CR2: 0000000000497120 CR3: 0000000001c11000 CR4: 00000000001407e0
[ 624.457707] DR0: 00007f6670c66000 DR1: 0000000000000000 DR2: 0000000000000000
[ 624.458881] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[ 624.460056] Stack:
[ 624.461220] 000000014240c000 0d2996f8aba02290 ffffe8ffff004da8 0000000000000005
[ 624.462418] ffffffff81cae620 0000000000000001 ffff88024240fe98 ffffffff81645825
[ 624.463622] 000000917994f078 ffffffff81cae7f0 ffffffff81d1d290 ffffe8ffff004da8
[ 624.464826] Call Trace:
[ 624.466015] [<ffffffff81645825>] cpuidle_enter_state+0x55/0x1c0
[ 624.467220] [<ffffffff81645a47>] cpuidle_enter+0x17/0x20
[ 624.468415] [<ffffffff810b9fb4>] cpu_startup_entry+0x384/0x410
[ 624.469607] [<ffffffff8102ff37>] start_secondary+0x237/0x340
[ 624.470800] Code: 31 d2 65 48 8b 34 25 08 ba 00 00 48 8d 86 38 c0 ff ff 48 89 d1 0f 01 c8 48 8b 86 38 c0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <65> 48 8b 0c 25 08 ba 00 00 f0 80 a1 3a c0 ff ff df 0f ae f0 48
[ 624.473394] NMI backtrace for cpu 0
[ 624.474616] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G L 3.17.0-rc1+ #112
[ 624.477061] task: ffffffff81c164c0 ti: ffffffff81c00000 task.ti: ffffffff81c00000
[ 624.478299] RIP: 0010:[<ffffffff813c9e65>] [<ffffffff813c9e65>] intel_idle+0xd5/0x180
[ 624.479544] RSP: 0018:ffffffff81c03e68 EFLAGS: 00000046
[ 624.480766] RAX: 0000000000000032 RBX: 0000000000000010 RCX: 0000000000000001
[ 624.481964] RDX: 0000000000000000 RSI: ffffffff81c03fd8 RDI: 0000000000000000
[ 624.483137] RBP: ffffffff81c03e98 R08: 000000008bafc4e0 R09: 0000000000000000
[ 624.484284] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000005
[ 624.485403] R13: 0000000000000032 R14: 0000000000000004 R15: ffffffff81c00000
[ 624.486516] FS: 0000000000000000(0000) GS:ffff880244000000(0000) knlGS:0000000000000000
[ 624.487613] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 624.488690] CR2: 00007fd405db1000 CR3: 0000000001c11000 CR4: 00000000001407f0
[ 624.489768] DR0: 00007f6670c66000 DR1: 0000000000000000 DR2: 0000000000000000
[ 624.490837] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[ 624.491894] Stack:
[ 624.492941] 0000000081c00000 b6e98a804d03933a ffffe8fffee04da8 0000000000000005
[ 624.494017] ffffffff81cae620 0000000000000000 ffffffff81c03ee0 ffffffff81645825
[ 624.495096] 000000917a0eca81 ffffffff81cae7f0 ffffffff81d1d290 ffffe8fffee04da8
[ 624.496174] Call Trace:
[ 624.497229] [<ffffffff81645825>] cpuidle_enter_state+0x55/0x1c0
[ 624.498294] [<ffffffff81645a47>] cpuidle_enter+0x17/0x20
[ 624.499348] [<ffffffff810b9fb4>] cpu_startup_entry+0x384/0x410
[ 624.500403] [<ffffffff8179d7a0>] rest_init+0xc0/0xd0
[ 624.501458] [<ffffffff8179d6e5>] ? rest_init+0x5/0xd0
[ 624.502507] [<ffffffff81eff009>] start_kernel+0x475/0x496
[ 624.503549] [<ffffffff81efe98d>] ? set_init_arg+0x53/0x53
[ 624.504585] [<ffffffff81efe57b>] x86_64_start_reservations+0x2a/0x2c
[ 624.505627] [<ffffffff81efe66e>] x86_64_start_kernel+0xf1/0xf4
[ 624.506655] Code: 31 d2 65 48 8b 34 25 08 ba 00 00 48 8d 86 38 c0 ff ff 48 89 d1 0f 01 c8 48 8b 86 38 c0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <65> 48 8b 0c 25 08 ba 00 00 f0 80 a1 3a c0 ff ff df 0f ae f0 48
[ 624.508927] NMI backtrace for cpu 3
[ 624.510061] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G L 3.17.0-rc1+ #112
[ 624.512303] task: ffff880242b72de0 ti: ffff880242418000 task.ti: ffff880242418000
[ 624.513284] RIP: 0010:[<ffffffff813c9e65>] [<ffffffff813c9e65>] intel_idle+0xd5/0x180
[ 624.514281] RSP: 0000:ffff88024241be20 EFLAGS: 00000046
[ 624.515289] RAX: 0000000000000032 RBX: 0000000000000010 RCX: 0000000000000001
[ 624.516286] RDX: 0000000000000000 RSI: ffff88024241bfd8 RDI: 0000000000000003
[ 624.517264] RBP: ffff88024241be50 R08: 000000008bafc4e0 R09: 0000000000000000
[ 624.518235] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000005
[ 624.519205] R13: 0000000000000032 R14: 0000000000000004 R15: ffff880242418000
[ 624.520169] FS: 0000000000000000(0000) GS:ffff880244600000(0000) knlGS:0000000000000000
[ 624.521142] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 624.522100] CR2: 00000000013a4738 CR3: 0000000001c11000 CR4: 00000000001407e0
[ 624.523105] DR0: 00007f6670c66000 DR1: 0000000000000000 DR2: 0000000000000000
[ 624.524063] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[ 624.525039] Stack:
[ 624.526003] 0000000342418000 1e8c3d4850bfa337 ffffe8ffff404da8 0000000000000005
[ 624.526991] ffffffff81cae620 0000000000000003 ffff88024241be98 ffffffff81645825
[ 624.527953] 000000917994f2a7 ffffffff81cae7f0 ffffffff81d1d290 ffffe8ffff404da8
[ 624.528926] Call Trace:
[ 624.529911] [<ffffffff81645825>] cpuidle_enter_state+0x55/0x1c0
[ 624.530911] [<ffffffff81645a47>] cpuidle_enter+0x17/0x20
[ 624.531883] [<ffffffff810b9fb4>] cpu_startup_entry+0x384/0x410
[ 624.532905] [<ffffffff8102ff37>] start_secondary+0x237/0x340
[ 624.533877] Code: 31 d2 65 48 8b 34 25 08 ba 00 00 48 8d 86 38 c0 ff ff 48 89 d1 0f 01 c8 48 8b 86 38 c0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <65> 48 8b 0c 25 08 ba 00 00 f0 80 a1 3a c0 ff ff df 0f ae f0 48
[ 652.386003] NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [swapper/2:0]
[ 652.388580] Modules linked in: fuse tun rfcomm llc2 af_key nfnetlink scsi_transport_iscsi can_bcm bnep can_raw nfc caif_socket caif af_802154 ieee802154 phonet af_rxrpc bluetooth can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 cfg80211 rfkill coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm nfsd auth_rpcgss oid_registry nfs_acl lockd sunrpc snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_controller snd_hda_codec crct10dif_pclmul crc32c_intel ghash_clmulni_intel e1000e snd_hwdep snd_seq snd_seq_device snd_pcm snd_timer microcode snd serio_raw pcspkr usb_debug ptp pps_core shpchp soundcore
[ 652.394750] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G L 3.17.0-rc1+ #112
[ 652.397225] task: ffff880242b744d0 ti: ffff880242414000 task.ti: ffff880242414000
[ 652.398457] RIP: 0010:[<ffffffff81645849>] [<ffffffff81645849>] cpuidle_enter_state+0x79/0x1c0
[ 652.399711] RSP: 0018:ffff880242417e60 EFLAGS: 00000246
[ 652.400953] RAX: 0000000000000000 RBX: ffff880242b744d0 RCX: 0000000000000019
[ 652.402203] RDX: 20c49ba5e353f7cf RSI: 0000000000039e2e RDI: 002e6b0e8bd4c66e
[ 652.403446] RBP: ffff880242417e98 R08: 000000008bafc4e0 R09: 0000000000000000
[ 652.404695] R10: 0000000000000000 R11: 0000000000000000 R12: ffff880242417df0
[ 652.405953] R13: ffffffff810bfc5e R14: ffff880242417dd0 R15: 00000000000001da
[ 652.407176] FS: 0000000000000000(0000) GS:ffff880244400000(0000) knlGS:0000000000000000
[ 652.408441] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 652.409679] CR2: 00007f15cfbaf000 CR3: 0000000001c11000 CR4: 00000000001407e0
[ 652.410923] DR0: 00007f6670c66000 DR1: 0000000000000000 DR2: 0000000000000000
[ 652.412173] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[ 652.413415] Stack:
[ 652.414652] 00000097fc215a08 ffffffff81cae7f0 ffffffff81d1d290 ffffe8ffff204da8
[ 652.415906] ffff880242414000 ffffffff81cae620 ffff880242414000 ffff880242417ea8
[ 652.417138] ffffffff81645a47 ffff880242417f10 ffffffff810b9fb4 ffff880242417fd8
[ 652.418379] Call Trace:
[ 652.419637] [<ffffffff81645a47>] cpuidle_enter+0x17/0x20
[ 652.420917] [<ffffffff810b9fb4>] cpu_startup_entry+0x384/0x410
[ 652.422192] [<ffffffff8102ff37>] start_secondary+0x237/0x340
[ 652.423462] Code: d0 48 89 df ff 50 48 41 89 c5 e8 b3 5c aa ff 44 8b 63 04 49 89 c7 0f 1f 44 00 00 e8 a2 19 b0 ff fb 48 ba cf f7 53 e3 a5 9b c4 20 <4c> 2b 7d c8 4c 89 f8 49 c1 ff 3f 48 f7 ea b8 ff ff ff 7f 48 c1
[ 652.426104] sending NMI to other CPUs:
[ 652.427302] NMI backtrace for cpu 0
[ 652.428449] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G L 3.17.0-rc1+ #112
[ 652.430736] task: ffffffff81c164c0 ti: ffffffff81c00000 task.ti: ffffffff81c00000
[ 652.431882] RIP: 0010:[<ffffffff813c9e65>] [<ffffffff813c9e65>] intel_idle+0xd5/0x180
[ 652.433036] RSP: 0018:ffffffff81c03e68 EFLAGS: 00000046
[ 652.434181] RAX: 0000000000000032 RBX: 0000000000000010 RCX: 0000000000000001
[ 652.435341] RDX: 0000000000000000 RSI: ffffffff81c03fd8 RDI: 0000000000000000
[ 652.436491] RBP: ffffffff81c03e98 R08: 000000008bafc4e0 R09: 0000000000000000
[ 652.437642] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000005
[ 652.438797] R13: 0000000000000032 R14: 0000000000000004 R15: ffffffff81c00000
[ 652.439945] FS: 0000000000000000(0000) GS:ffff880244000000(0000) knlGS:0000000000000000
[ 652.441093] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 652.442234] CR2: 00007f00bcc2c000 CR3: 0000000001c11000 CR4: 00000000001407f0
[ 652.443388] DR0: 00007f6670c66000 DR1: 0000000000000000 DR2: 0000000000000000
[ 652.444540] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[ 652.445680] Stack:
[ 652.446808] 0000000081c00000 b6e98a804d03933a ffffe8fffee04da8 0000000000000005
[ 652.447959] ffffffff81cae620 0000000000000000 ffffffff81c03ee0 ffffffff81645825
[ 652.449111] 00000097ff1f0b6f ffffffff81cae7f0 ffffffff81d1d290 ffffe8fffee04da8
[ 652.450261] Call Trace:
[ 652.451392] [<ffffffff81645825>] cpuidle_enter_state+0x55/0x1c0
[ 652.452539] [<ffffffff81645a47>] cpuidle_enter+0x17/0x20
[ 652.453682] [<ffffffff810b9fb4>] cpu_startup_entry+0x384/0x410
[ 652.454825] [<ffffffff8179d7a0>] rest_init+0xc0/0xd0
[ 652.455942] [<ffffffff8179d6e5>] ? rest_init+0x5/0xd0
[ 652.457036] [<ffffffff81eff009>] start_kernel+0x475/0x496
[ 652.458123] [<ffffffff81efe98d>] ? set_init_arg+0x53/0x53
[ 652.459218] [<ffffffff81efe57b>] x86_64_start_reservations+0x2a/0x2c
[ 652.460320] [<ffffffff81efe66e>] x86_64_start_kernel+0xf1/0xf4
[ 652.461419] Code: 31 d2 65 48 8b 34 25 08 ba 00 00 48 8d 86 38 c0 ff ff 48 89 d1 0f 01 c8 48 8b 86 38 c0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <65> 48 8b 0c 25 08 ba 00 00 f0 80 a1 3a c0 ff ff df 0f ae f0 48
[ 652.463812] NMI backtrace for cpu 3
[ 652.464973] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G L 3.17.0-rc1+ #112
[ 652.467190] task: ffff880242b72de0 ti: ffff880242418000 task.ti: ffff880242418000
[ 652.468248] RIP: 0010:[<ffffffff813c9e65>] [<ffffffff813c9e65>] intel_idle+0xd5/0x180
[ 652.469309] RSP: 0018:ffff88024241be20 EFLAGS: 00000046
[ 652.470336] RAX: 0000000000000032 RBX: 0000000000000010 RCX: 0000000000000001
[ 652.471368] RDX: 0000000000000000 RSI: ffff88024241bfd8 RDI: 0000000000000003
[ 652.472378] RBP: ffff88024241be50 R08: 000000008bafc4e0 R09: 0000000000000000
[ 652.473383] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000005
[ 652.474378] R13: 0000000000000032 R14: 0000000000000004 R15: ffff880242418000
[ 652.475366] FS: 0000000000000000(0000) GS:ffff880244600000(0000) knlGS:0000000000000000
[ 652.476385] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 652.477389] CR2: 00000000013a4738 CR3: 0000000001c11000 CR4: 00000000001407e0
[ 652.478379] DR0: 00007f6670c66000 DR1: 0000000000000000 DR2: 0000000000000000
[ 652.479371] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[ 652.480366] Stack:
[ 652.481327] 0000000342418000 1e8c3d4850bfa337 ffffe8ffff404da8 0000000000000005
[ 652.482326] ffffffff81cae620 0000000000000003 ffff88024241be98 ffffffff81645825
[ 652.483326] 00000097ff1b9892 ffffffff81cae7f0 ffffffff81d1d290 ffffe8ffff404da8
[ 652.484332] Call Trace:
[ 652.485323] [<ffffffff81645825>] cpuidle_enter_state+0x55/0x1c0
[ 652.486345] [<ffffffff81645a47>] cpuidle_enter+0x17/0x20
[ 652.487338] [<ffffffff810b9fb4>] cpu_startup_entry+0x384/0x410
[ 652.488335] [<ffffffff8102ff37>] start_secondary+0x237/0x340
[ 652.489322] Code: 31 d2 65 48 8b 34 25 08 ba 00 00 48 8d 86 38 c0 ff ff 48 89 d1 0f 01 c8 48 8b 86 38 c0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <65> 48 8b 0c 25 08 ba 00 00 f0 80 a1 3a c0 ff ff df 0f ae f0 48
[ 652.491493] NMI backtrace for cpu 1
[ 652.492553] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G L 3.17.0-rc1+ #112
[ 652.494553] task: ffff880242b716f0 ti: ffff88024240c000 task.ti: ffff88024240c000
[ 652.495559] RIP: 0010:[<ffffffff813c9e65>] [<ffffffff813c9e65>] intel_idle+0xd5/0x180
[ 652.496585] RSP: 0018:ffff88024240fe20 EFLAGS: 00000046
[ 652.497569] RAX: 0000000000000032 RBX: 0000000000000010 RCX: 0000000000000001
[ 652.498545] RDX: 0000000000000000 RSI: ffff88024240ffd8 RDI: 0000000000000001
[ 652.499502] RBP: ffff88024240fe50 R08: 000000008bafc4e0 R09: 0000000000000000
[ 652.500452] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000005
[ 652.501398] R13: 0000000000000032 R14: 0000000000000004 R15: ffff88024240c000
[ 652.502342] FS: 0000000000000000(0000) GS:ffff880244200000(0000) knlGS:0000000000000000
[ 652.503290] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 652.504236] CR2: 00007fecce638ab8 CR3: 0000000001c11000 CR4: 00000000001407e0
[ 652.505195] DR0: 00007f6670c66000 DR1: 0000000000000000 DR2: 0000000000000000
[ 652.506202] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[ 652.507177] Stack:
[ 652.508136] 000000014240c000 0d2996f8aba02290 ffffe8ffff004da8 0000000000000005
[ 652.509122] ffffffff81cae620 0000000000000001 ffff88024240fe98 ffffffff81645825
[ 652.510113] 00000097ff1b96a9 ffffffff81cae7f0 ffffffff81d1d290 ffffe8ffff004da8
[ 652.511130] Call Trace:
[ 652.512121] [<ffffffff81645825>] cpuidle_enter_state+0x55/0x1c0
[ 652.513102] [<ffffffff81645a47>] cpuidle_enter+0x17/0x20
[ 652.514086] [<ffffffff810b9fb4>] cpu_startup_entry+0x384/0x410
[ 652.515078] [<ffffffff8102ff37>] start_secondary+0x237/0x340
[ 652.516085] Code: 31 d2 65 48 8b 34 25 08 ba 00 00 48 8d 86 38 c0 ff ff 48 89 d1 0f 01 c8 48 8b 86 38 c0 ff ff a8 08 75 08 b1 01 4c 89 e8 0f 01 c9 <65> 48 8b 0c 25 08 ba 00 00 f0 80 a1 3a c0 ff ff df 0f ae f0 48
It kept spewing lockups over and over.
Something weird that jumped out to me was this:
[ 599.573324] end_request: I/O error, dev sda, sector 0
User trinity was running under did not have permission
to read block device directly, so that's just.. creepy.
Hopefully not a sign of impending disk death.
Dave
next prev parent reply other threads:[~2014-12-03 19:01 UTC|newest]
Thread overview: 488+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-11-14 21:31 frequent lockups in 3.18rc4 Dave Jones
2014-11-14 22:01 ` Linus Torvalds
2014-11-14 22:30 ` Dave Jones
2014-11-14 22:55 ` Thomas Gleixner
2014-11-14 23:32 ` Dave Jones
2014-11-15 0:36 ` Thomas Gleixner
2014-11-15 2:40 ` Dave Jones
2014-11-16 12:16 ` Thomas Gleixner
2014-11-15 1:59 ` Linus Torvalds
2014-11-17 21:22 ` Linus Torvalds
2014-11-17 22:31 ` Thomas Gleixner
2014-11-17 22:43 ` Thomas Gleixner
2014-11-17 22:58 ` Jens Axboe
2014-11-17 23:59 ` Linus Torvalds
2014-11-18 0:15 ` Thomas Gleixner
2014-11-17 23:04 ` Jens Axboe
2014-11-17 23:17 ` Thomas Gleixner
2014-11-18 2:23 ` Jens Axboe
2014-11-15 21:34 ` Dave Jones
2014-11-16 1:40 ` Dave Jones
2014-11-16 6:33 ` Linus Torvalds
2014-11-16 10:06 ` Markus Trippelsdorf
2014-11-16 18:33 ` Linus Torvalds
2014-11-17 17:03 ` Dave Jones
2014-11-17 19:59 ` Linus Torvalds
2014-11-18 2:09 ` Dave Jones
2014-11-18 2:21 ` Linus Torvalds
2014-11-18 2:39 ` Dave Jones
2014-11-18 2:51 ` Linus Torvalds
2014-11-18 14:52 ` Dave Jones
2014-11-18 17:20 ` Linus Torvalds
2014-11-18 19:28 ` Thomas Gleixner
2014-11-18 21:25 ` Don Zickus
2014-11-18 21:31 ` Dave Jones
2014-11-18 18:54 ` Thomas Gleixner
2014-11-18 21:55 ` Don Zickus
2014-11-18 22:02 ` Dave Jones
2014-11-19 14:41 ` Don Zickus
2014-11-19 15:03 ` Vivek Goyal
2014-11-19 15:38 ` Dave Jones
2014-11-19 16:28 ` Vivek Goyal
2014-11-20 16:10 ` Dave Jones
2014-11-20 16:48 ` Vivek Goyal
2014-11-20 17:38 ` Dave Jones
2014-11-21 9:46 ` Dave Young
2014-11-20 16:54 ` Vivek Goyal
2014-11-20 9:54 ` Dave Young
2014-11-19 2:19 ` Dave Jones
2014-11-19 4:40 ` Linus Torvalds
2014-11-19 4:59 ` Dave Jones
2014-11-19 5:15 ` Dave Jones
2014-11-20 14:36 ` Frederic Weisbecker
2014-11-19 14:59 ` Dave Jones
2014-11-19 17:22 ` Linus Torvalds
2014-11-19 17:40 ` Linus Torvalds
2014-11-19 19:02 ` Frederic Weisbecker
2014-11-19 19:03 ` Andy Lutomirski
2014-11-19 23:00 ` Frederic Weisbecker
2014-11-19 23:07 ` Andy Lutomirski
2014-11-19 23:13 ` Frederic Weisbecker
2014-11-19 21:56 ` Thomas Gleixner
2014-11-19 22:56 ` Frederic Weisbecker
2014-11-19 22:59 ` Andy Lutomirski
2014-11-19 23:07 ` Frederic Weisbecker
2014-11-19 23:09 ` Thomas Gleixner
2014-11-19 23:50 ` Frederic Weisbecker
2014-11-20 12:23 ` Tejun Heo
2014-11-20 21:58 ` Thomas Gleixner
2014-11-20 22:06 ` Andy Lutomirski
2014-11-20 22:11 ` Tejun Heo
2014-11-20 22:42 ` Thomas Gleixner
2014-11-20 23:05 ` Tejun Heo
2014-11-20 23:08 ` Andy Lutomirski
2014-11-20 23:34 ` Linus Torvalds
2014-11-20 23:39 ` Tejun Heo
2014-11-20 23:55 ` Andy Lutomirski
2014-11-21 16:27 ` Tejun Heo
2014-11-21 16:38 ` Andy Lutomirski
2014-11-21 16:48 ` Linus Torvalds
2014-11-21 17:08 ` Steven Rostedt
2014-11-21 17:19 ` Linus Torvalds
2014-11-21 17:22 ` Andy Lutomirski
2014-11-21 18:22 ` Linus Torvalds
2014-11-21 18:28 ` Andy Lutomirski
2014-11-21 19:06 ` Linus Torvalds
2014-11-21 19:23 ` Steven Rostedt
2014-11-21 19:34 ` Linus Torvalds
2014-11-21 19:46 ` Linus Torvalds
2014-11-21 19:52 ` Andy Lutomirski
2014-11-21 20:14 ` Josh Boyer
2014-11-21 20:16 ` Andy Lutomirski
2014-11-21 20:23 ` Josh Boyer
2014-11-24 18:48 ` Konrad Rzeszutek Wilk
2014-11-24 19:07 ` Josh Boyer
2014-11-25 5:36 ` Jürgen Groß
2014-11-25 17:22 ` Linus Torvalds
2014-11-21 20:00 ` Dave Jones
2014-11-21 20:02 ` Andy Lutomirski
2014-11-21 19:51 ` Thomas Gleixner
2014-11-21 20:00 ` Linus Torvalds
2014-11-21 20:16 ` Thomas Gleixner
2014-11-21 20:41 ` Linus Torvalds
2014-11-21 21:11 ` Thomas Gleixner
2014-11-21 22:55 ` Linus Torvalds
2014-11-21 23:03 ` Andy Lutomirski
2014-11-21 23:33 ` Linus Torvalds
2014-12-16 19:28 ` Peter Zijlstra
2014-12-16 20:46 ` Linus Torvalds
2014-12-16 21:19 ` Mel Gorman
2014-12-16 23:02 ` Peter Zijlstra
2014-12-17 0:00 ` Linus Torvalds
2014-12-17 0:41 ` Andy Lutomirski
2014-12-17 17:01 ` Konrad Rzeszutek Wilk
2014-12-17 17:14 ` Peter Zijlstra
2014-11-21 22:33 ` Konrad Rzeszutek Wilk
2014-11-22 1:17 ` Thomas Gleixner
2014-11-22 1:17 ` Thomas Gleixner
2014-11-21 22:33 ` Konrad Rzeszutek Wilk
2014-11-21 17:34 ` Steven Rostedt
2014-11-21 18:24 ` Linus Torvalds
2014-11-21 22:10 ` Frederic Weisbecker
2014-11-21 2:33 ` Steven Rostedt
2014-11-21 0:54 ` Thomas Gleixner
2014-11-21 14:13 ` Frederic Weisbecker
2014-11-21 16:25 ` Tejun Heo
2014-11-21 17:01 ` Steven Rostedt
2014-11-21 17:11 ` Steven Rostedt
2014-11-21 21:32 ` Frederic Weisbecker
2014-11-21 21:34 ` Andy Lutomirski
2014-11-21 21:50 ` Frederic Weisbecker
2014-11-21 22:45 ` Steven Rostedt
2014-11-21 21:44 ` Frederic Weisbecker
2014-11-22 0:11 ` Tejun Heo
2014-11-22 0:18 ` Linus Torvalds
2014-11-22 0:41 ` Andy Lutomirski
2014-11-19 23:54 ` Andy Lutomirski
2014-11-20 0:00 ` Thomas Gleixner
2014-11-20 0:30 ` Andy Lutomirski
2014-11-20 0:40 ` Linus Torvalds
2014-11-20 0:49 ` Andy Lutomirski
2014-11-20 1:07 ` Linus Torvalds
2014-11-20 1:16 ` Andy Lutomirski
2014-11-20 2:42 ` Linus Torvalds
2014-11-20 6:16 ` Andy Lutomirski
2014-11-19 19:15 ` Andy Lutomirski
2014-11-19 19:38 ` Linus Torvalds
2014-11-19 22:18 ` Dave Jones
2014-11-19 21:01 ` Andy Lutomirski
2014-11-19 21:47 ` Dave Jones
2014-11-19 21:58 ` Borislav Petkov
2014-11-19 22:18 ` Dave Jones
2014-11-20 10:33 ` Borislav Petkov
2014-11-19 21:56 ` [PATCH] x86, syscall: Fix _TIF_NOHZ handling in syscall_trace_enter_phase1 Andy Lutomirski
2014-11-19 22:13 ` Thomas Gleixner
2014-11-20 20:33 ` Linus Torvalds
2014-11-20 22:07 ` Thomas Gleixner
2014-11-20 22:04 ` [tip:x86/urgent] " tip-bot for Andy Lutomirski
2014-11-20 15:25 ` frequent lockups in 3.18rc4 Dave Jones
2014-11-20 19:43 ` Linus Torvalds
2014-11-20 20:06 ` Dave Jones
2014-11-20 20:37 ` Don Zickus
2014-11-20 20:51 ` Linus Torvalds
2014-11-21 6:37 ` Ingo Molnar
2014-11-21 14:50 ` Dave Jones
2014-11-25 12:22 ` Will Deacon
2014-12-01 11:48 ` Will Deacon
2014-12-01 17:05 ` Linus Torvalds
2014-12-01 17:10 ` Will Deacon
2014-12-01 17:53 ` Linus Torvalds
2014-12-01 18:25 ` Kirill A. Shutemov
2014-12-01 18:36 ` Linus Torvalds
2014-12-04 10:51 ` Will Deacon
2014-12-04 14:56 ` Dave Jones
2014-12-05 13:49 ` Will Deacon
2014-11-20 15:04 ` Frederic Weisbecker
2014-11-20 15:08 ` Frederic Weisbecker
2014-11-20 16:19 ` Dave Jones
2014-11-20 16:42 ` Frederic Weisbecker
2014-11-26 0:25 ` Dave Jones
2014-11-26 1:48 ` Linus Torvalds
2014-11-26 2:40 ` Dave Jones
2014-11-26 22:57 ` Dave Jones
2014-11-27 0:46 ` Linus Torvalds
2014-11-27 19:17 ` Linus Torvalds
2014-11-27 22:56 ` Dave Jones
2014-11-29 20:38 ` Dâniel Fraga
2014-11-30 20:45 ` Linus Torvalds
2014-11-30 21:21 ` Dâniel Fraga
2014-12-01 0:21 ` Linus Torvalds
2014-12-01 1:02 ` Dâniel Fraga
2014-12-01 19:14 ` Paul E. McKenney
2014-12-01 20:28 ` Dâniel Fraga
2014-12-01 20:36 ` Linus Torvalds
2014-12-01 23:08 ` Chris Mason
2014-12-01 23:25 ` Linus Torvalds
2014-12-01 23:44 ` Chris Mason
2014-12-02 0:39 ` Linus Torvalds
2014-12-02 14:13 ` Mike Galbraith
2014-12-02 16:33 ` Linus Torvalds
2014-12-02 17:14 ` Chris Mason
2014-12-03 18:41 ` Dave Jones
2014-12-03 18:45 ` Linus Torvalds
2014-12-03 19:00 ` Dave Jones [this message]
2014-12-03 19:25 ` Linus Torvalds
2014-12-03 19:30 ` Dave Jones
2014-12-03 19:48 ` Linus Torvalds
2014-12-03 20:09 ` Dave Jones
2014-12-03 20:37 ` Linus Torvalds
2014-12-03 20:55 ` Thomas Gleixner
2014-12-03 21:14 ` Linus Torvalds
2014-12-03 22:19 ` Thomas Gleixner
2014-12-03 23:21 ` Dave Jones
2014-12-03 23:49 ` Thomas Gleixner
2014-12-04 0:19 ` Linus Torvalds
2014-12-04 1:02 ` Thomas Gleixner
2014-12-04 0:20 ` Dave Jones
2014-12-04 0:59 ` Thomas Gleixner
2014-12-04 1:32 ` Dave Jones
2014-12-04 3:45 ` Dave Jones
2014-12-03 19:56 ` John Stultz
2014-12-03 20:37 ` Thomas Gleixner
2014-12-03 20:44 ` Dave Jones
2014-12-03 20:59 ` Thomas Gleixner
2014-12-03 21:05 ` Dave Jones
2014-12-03 21:48 ` Thomas Gleixner
2014-12-03 20:39 ` Thomas Gleixner
2014-12-04 3:15 ` Chris Mason
2014-12-04 5:49 ` Linus Torvalds
2014-12-04 14:57 ` Chris Mason
2014-12-04 15:22 ` Dave Hansen
2014-12-04 15:30 ` Chris Mason
2014-12-03 19:59 ` Chris Mason
2014-12-03 20:11 ` Dave Jones
2014-12-03 20:56 ` Chris Mason
2014-12-04 0:27 ` Dave Jones
2014-12-05 17:15 ` Dave Jones
2014-12-05 18:38 ` Linus Torvalds
2014-12-05 18:48 ` Dave Jones
2014-12-05 19:31 ` Linus Torvalds
2014-12-05 19:37 ` Dave Jones
2014-12-06 22:38 ` Thomas Gleixner
2014-12-06 9:37 ` Chuck Ebbert
2014-12-06 16:22 ` Martin van Es
2014-12-06 20:09 ` Linus Torvalds
2014-12-06 20:41 ` Linus Torvalds
2014-12-06 21:14 ` Martin van Es
2014-12-12 12:58 ` Martin van Es
2014-12-15 12:07 ` Martin van Es
2014-12-06 22:14 ` Thomas Gleixner
2014-12-05 19:04 ` Chris Mason
2014-12-05 19:29 ` Linus Torvalds
2014-12-11 14:54 ` Dave Jones
2014-12-11 21:49 ` Linus Torvalds
2014-12-11 21:52 ` Sasha Levin
2014-12-11 21:57 ` Chris Mason
2014-12-11 22:00 ` Sasha Levin
2014-12-11 22:36 ` Linus Torvalds
2014-12-11 22:57 ` Sasha Levin
2014-12-12 6:54 ` Ingo Molnar
2014-12-12 23:54 ` Sasha Levin
2014-12-13 0:23 ` Linus Torvalds
2014-12-13 0:34 ` Sasha Levin
2014-12-13 0:44 ` Linus Torvalds
2014-12-13 16:28 ` Jeff Chua
2014-12-13 2:32 ` Dave Jones
2014-12-11 21:57 ` Borislav Petkov
2014-12-12 3:03 ` Dave Jones
2014-12-12 4:45 ` Dave Jones
2014-12-12 14:38 ` Dave Jones
2014-12-12 18:24 ` Paul E. McKenney
2014-12-12 18:10 ` Paul E. McKenney
2014-12-12 18:42 ` Dave Jones
2014-12-12 18:54 ` Dave Jones
2014-12-12 19:14 ` Linus Torvalds
2014-12-12 19:23 ` Dave Jones
2014-12-12 19:58 ` David Lang
2014-12-12 20:20 ` Linus Torvalds
2014-12-13 7:43 ` Ingo Molnar
2014-12-12 20:34 ` Paul E. McKenney
2014-12-12 21:23 ` Sasha Levin
2014-12-13 0:58 ` Paul E. McKenney
2014-12-13 12:08 ` Paul E. McKenney
2014-12-13 8:30 ` Ingo Molnar
2014-12-13 15:53 ` Sasha Levin
2014-12-13 18:07 ` Paul E. McKenney
2014-12-14 17:50 ` Paul E. McKenney
2014-12-14 23:46 ` Sasha Levin
2014-12-15 0:11 ` Paul E. McKenney
2014-12-15 1:20 ` Sasha Levin
2014-12-15 6:33 ` Paul E. McKenney
2014-12-15 12:56 ` Paul E. McKenney
2014-12-15 13:16 ` Sasha Levin
2014-12-16 3:40 ` Paul E. McKenney
2014-12-13 7:36 ` [PATCH] sched: Fix lost reschedule in __cond_resched() Ingo Molnar
2014-12-14 18:04 ` Frederic Weisbecker
2014-12-14 19:43 ` Ingo Molnar
2014-12-14 19:50 ` Linus Torvalds
2014-12-14 20:30 ` Frederic Weisbecker
2014-12-13 8:19 ` frequent lockups in 3.18rc4 Ingo Molnar
2014-12-13 8:27 ` Ingo Molnar
2014-12-13 14:15 ` Sasha Levin
2014-12-13 16:59 ` Dave Jones
2014-12-13 18:04 ` Paul E. McKenney
2014-12-13 20:41 ` Dave Jones
2014-12-14 4:04 ` Paul E. McKenney
2014-12-13 22:36 ` Dave Jones
2014-12-13 22:40 ` Linus Torvalds
2014-12-13 22:59 ` Linus Torvalds
2014-12-13 23:09 ` Linus Torvalds
2014-12-13 23:35 ` Al Viro
2014-12-13 23:38 ` Linus Torvalds
2014-12-13 23:47 ` Al Viro
2014-12-14 0:14 ` Linus Torvalds
2014-12-14 0:33 ` Al Viro
2014-12-14 1:35 ` Linus Torvalds
2014-12-14 3:14 ` Al Viro
2014-12-15 0:18 ` Al Viro
2014-12-13 23:39 ` Al Viro
2014-12-14 23:46 ` Dave Jones
2014-12-15 0:38 ` Linus Torvalds
2014-12-15 0:42 ` Dave Jones
2014-12-15 5:47 ` Linus Torvalds
2014-12-15 5:57 ` Dave Jones
2014-12-15 18:21 ` Linus Torvalds
2014-12-15 23:46 ` Linus Torvalds
2014-12-18 2:42 ` Sasha Levin
2014-12-18 2:45 ` Linus Torvalds
2014-12-18 5:13 ` Dave Jones
2014-12-18 15:54 ` Chris Mason
2014-12-18 16:12 ` Dave Jones
2014-12-19 2:45 ` Dave Jones
2014-12-19 3:49 ` Linus Torvalds
2014-12-19 3:58 ` Dave Jones
2014-12-19 4:03 ` Dave Jones
2014-12-19 4:48 ` Linus Torvalds
2014-12-19 11:35 ` Peter Zijlstra
2014-12-19 14:55 ` Dave Jones
2014-12-19 15:14 ` Chris Mason
2014-12-19 19:15 ` Linus Torvalds
2014-12-19 19:44 ` Peter Zijlstra
2014-12-19 19:51 ` Linus Torvalds
2014-12-19 20:46 ` Linus Torvalds
2014-12-19 20:54 ` Dave Jones
2014-12-19 22:05 ` Linus Torvalds
2014-12-20 16:49 ` Dave Jones
2014-12-19 20:31 ` Chris Mason
2014-12-19 20:36 ` Dave Jones
2014-12-19 23:22 ` Thomas Gleixner
2014-12-20 0:12 ` Chris Mason
2014-12-20 1:06 ` Thomas Gleixner
2014-12-19 23:14 ` Thomas Gleixner
2014-12-19 23:55 ` Linus Torvalds
2014-12-20 1:00 ` Thomas Gleixner
2014-12-20 1:57 ` Linus Torvalds
2014-12-20 18:25 ` Linus Torvalds
2014-12-20 21:16 ` Linus Torvalds
2014-12-21 3:52 ` Paul E. McKenney
2014-12-21 21:22 ` Linus Torvalds
2014-12-21 22:19 ` Linus Torvalds
2014-12-21 22:32 ` Dave Jones
2014-12-21 23:58 ` Linus Torvalds
2014-12-22 0:41 ` Linus Torvalds
2014-12-22 0:52 ` Linus Torvalds
2014-12-22 1:22 ` Dave Jones
2014-12-22 3:11 ` Paul E. McKenney
2014-12-22 19:47 ` Linus Torvalds
2014-12-22 20:06 ` Linus Torvalds
2014-12-22 22:57 ` Dave Jones
2014-12-22 23:59 ` Linus Torvalds
2014-12-23 14:56 ` Dave Jones
2014-12-24 13:58 ` Sasha Levin
2014-12-24 3:01 ` Dave Jones
2014-12-26 16:34 ` Dave Jones
2014-12-26 18:12 ` Dave Jones
2014-12-26 20:57 ` Linus Torvalds
2014-12-26 21:20 ` Dave Jones
2014-12-26 22:57 ` Dave Jones
2014-12-26 23:16 ` Linus Torvalds
2014-12-27 0:36 ` Dave Jones
2014-12-27 3:14 ` Linus Torvalds
2014-12-27 16:48 ` Dave Jones
2014-12-26 23:30 ` Linus Torvalds
2014-12-27 0:39 ` Dave Jones
2014-12-27 2:53 ` Dave Jones
2015-01-03 0:27 ` John Stultz
2015-01-03 14:58 ` Sasha Levin
2015-01-04 19:46 ` Linus Torvalds
2015-01-06 1:17 ` John Stultz
2015-01-06 1:25 ` Linus Torvalds
2015-01-06 2:05 ` John Stultz
2014-12-22 23:59 ` John Stultz
2014-12-23 0:46 ` Linus Torvalds
2014-12-27 20:33 ` Paul E. McKenney
2015-01-12 10:05 ` Thomas Gleixner
2014-12-19 14:30 ` Chris Mason
2014-12-19 15:12 ` Dave Jones
2014-12-18 18:54 ` Linus Torvalds
2014-12-15 14:00 ` Borislav Petkov
2014-12-18 21:17 ` save_xstate_sig (Re: frequent lockups in 3.18rc4) Andy Lutomirski
2014-12-18 21:34 ` Linus Torvalds
2014-12-18 21:41 ` Andy Lutomirski
2014-12-18 21:37 ` Dave Jones
2014-12-17 18:22 ` frequent lockups in 3.18rc4 Dave Jones
2014-12-17 18:57 ` Dave Jones
2014-12-17 19:24 ` Dave Jones
2014-12-17 19:51 ` Linus Torvalds
2014-12-17 20:16 ` Dave Jones
2014-12-17 19:41 ` Linus Torvalds
2014-12-06 5:04 ` Gene Heskett
2014-12-02 17:47 ` Mike Galbraith
2014-12-13 8:11 ` Ingo Molnar
2014-12-13 9:57 ` Mike Galbraith
2014-12-17 11:13 ` Peter Zijlstra
2014-12-02 19:32 ` Dave Jones
2014-12-02 23:32 ` Sasha Levin
2014-12-03 0:09 ` Linus Torvalds
2014-12-03 0:25 ` Sasha Levin
2014-12-05 5:00 ` Sasha Levin
2014-12-05 6:38 ` Linus Torvalds
2014-12-05 15:03 ` Sasha Levin
2014-12-05 18:15 ` Linus Torvalds
2014-12-07 14:58 ` Sasha Levin
2014-12-07 18:24 ` Paul E. McKenney
2014-12-07 19:43 ` Paul E. McKenney
2014-12-07 23:28 ` Sasha Levin
2014-12-08 5:20 ` Paul E. McKenney
2014-12-08 14:33 ` Sasha Levin
2014-12-08 15:28 ` Sasha Levin
2014-12-08 15:57 ` Paul E. McKenney
2014-12-08 16:34 ` Sasha Levin
2014-12-08 15:56 ` Paul E. McKenney
2014-12-07 23:53 ` Linus Torvalds
2014-12-02 19:31 ` Dave Jones
2014-12-02 21:17 ` Linus Torvalds
2014-12-02 20:30 ` Dave Jones
2014-12-02 20:48 ` Paul E. McKenney
2014-12-01 23:08 ` Paul E. McKenney
2014-12-02 16:43 ` Dâniel Fraga
2014-12-02 17:04 ` Paul E. McKenney
2014-12-02 17:14 ` Dâniel Fraga
2014-12-02 18:42 ` Paul E. McKenney
2014-12-02 18:47 ` Dâniel Fraga
2014-12-02 19:11 ` Paul E. McKenney
2014-12-02 19:24 ` Dâniel Fraga
2014-12-02 20:56 ` Paul E. McKenney
2014-12-02 22:01 ` Dâniel Fraga
2014-12-02 22:10 ` Paul E. McKenney
2014-12-02 22:18 ` Dâniel Fraga
2014-12-02 22:35 ` Paul E. McKenney
2014-12-02 22:10 ` Linus Torvalds
2014-12-02 22:16 ` Dâniel Fraga
2014-12-03 3:21 ` Dâniel Fraga
2014-12-03 4:14 ` Linus Torvalds
2014-12-03 4:51 ` Dâniel Fraga
2014-12-03 6:02 ` Chris Rorvick
2014-12-03 15:22 ` Linus Torvalds
2014-12-04 8:43 ` Dâniel Fraga
2014-12-04 16:18 ` Linus Torvalds
2014-12-04 16:52 ` Frederic Weisbecker
2014-12-04 17:25 ` Dâniel Fraga
2014-12-04 17:47 ` Linus Torvalds
2014-12-04 18:07 ` Dâniel Fraga
2014-12-03 14:54 ` Tejun Heo
2014-12-02 18:09 ` Paul E. McKenney
2014-12-02 18:41 ` Dâniel Fraga
2014-12-02 17:08 ` Linus Torvalds
2014-12-02 17:16 ` Dâniel Fraga
2014-12-02 8:40 ` Lai Jiangshan
2014-12-02 16:58 ` Paul E. McKenney
2014-12-02 16:58 ` Dâniel Fraga
2014-12-02 17:17 ` Paul E. McKenney
2014-12-03 2:03 ` Lai Jiangshan
2014-12-03 5:22 ` Paul E. McKenney
2014-12-01 16:56 ` Don Zickus
2014-11-26 4:39 ` Jürgen Groß
[not found] ` <CA+55aFx1SiFBzmA=k9jHxi3cZE3Ei_+2NHepujgf86KEvkz8eQ@mail.gmail.com>
2014-11-26 5:11 ` Dave Jones
2014-11-26 5:24 ` Juergen Gross
2014-11-26 5:52 ` Linus Torvalds
2014-11-26 6:21 ` Linus Torvalds
2014-11-26 6:52 ` Juergen Gross
2014-11-26 9:44 ` Juergen Gross
2014-11-26 14:34 ` Dave Jones
2014-11-26 17:37 ` Linus Torvalds
2014-11-20 15:28 ` Frederic Weisbecker
2014-11-17 15:07 ` Don Zickus
-- strict thread matches above, loose matches on Subject: below --
2014-12-16 3:04 Hillf Danton
2015-02-12 11:09 Martin van Es
2015-02-12 16:01 ` Linus Torvalds
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20141203190045.GB32005@redhat.com \
--to=davej@redhat.com \
--cc=clm@fb.com \
--cc=fragabr@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=paulmck@linux.vnet.ibm.com \
--cc=peterz@infradead.org \
--cc=sasha.levin@oracle.com \
--cc=torvalds@linux-foundation.org \
--cc=umgwanakikbuti@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.