Re: Kernel: BUG: soft lockup - CPU#0 stuck for 22s! on many machines at the same time

[Stefan Priebe <s.priebe@profihost.ag>]

Some of the guest machines also say:

[1450155.131037] INFO: rcu_sched detected stalls on CPUs/tasks: {} 
(detected by 1, t=60008 jiffies, g=109736865, c=109736864, q=0)
[1450155.131653] All QSes seen, last rcu_sched kthread activity 60005 
(5744764007-5744704002), jiffies_till_next_fqs=3, root ->qsmask 0x0
[1450155.131878] swapper/1       R  running task        0     0      1 
0x00080000
[1450155.132068]  ffffffff84c64040 ffff880331643d98 ffffffff840ac79f 
ffff880331657040
[1450155.132199]  ffffffff84c64040 ffff880331643e28 ffffffff840dc46f 
0000000000000000
[1450155.132331]  ffffffff84096364 ffff880331656240 0000000000000000 
ffff88032f660000
[1450155.132487] Call Trace:
[1450155.132953]  <IRQ>  [<ffffffff840ac79f>] sched_show_task+0xaf/0x120
[1450155.133134]  [<ffffffff840dc46f>] rcu_check_callbacks+0x7ff/0x810
[1450155.133252]  [<ffffffff84096364>] ? insert_work+0x74/0xc0
[1450155.133376]  [<ffffffff840e21b8>] update_process_times+0x38/0x70
[1450155.133491]  [<ffffffff840f1e1e>] tick_sched_handle.isra.14+0x2e/0x70
[1450155.133608]  [<ffffffff840f21eb>] tick_sched_timer+0x4b/0x80
[1450155.137916]  [<ffffffff840e2ab9>] __run_hrtimer+0x79/0x1c0
[1450155.139284]  [<ffffffff840f21a0>] ? tick_sched_do_timer+0x60/0x60
[1450155.140722]  [<ffffffff840e327b>] hrtimer_interrupt+0xfb/0x220
[1450155.142183]  [<ffffffff8404a18e>] ? kvm_clock_read+0x1e/0x20
[1450155.143502]  [<ffffffff8403e109>] local_apic_timer_interrupt+0x39/0x60
[1450155.144813]  [<ffffffff8403e605>] smp_apic_timer_interrupt+0x45/0x60
[1450155.146152]  [<ffffffff846a3dfe>] apic_timer_interrupt+0x6e/0x80
[1450155.147449]  <EOI>  [<ffffffff8404a566>] ? native_safe_halt+0x6/0x10
[1450155.148718]  [<ffffffff840d95d8>] ? rcu_eqs_enter+0x68/0x90
[1450155.150154]  [<ffffffff8400e8be>] default_idle+0x1e/0xc0
[1450155.151439]  [<ffffffff8400f2bf>] arch_cpu_idle+0xf/0x20
[1450155.152720]  [<ffffffff840c372c>] cpu_startup_entry+0x30c/0x3b0
[1450155.154031]  [<ffffffff8403c3e2>] start_secondary+0x182/0x1b0
[1450155.155301] rcu_sched kthread starved for 60005 jiffies!

Am 01.03.2016 um 11:58 schrieb Stefan Priebe - Profihost AG:
> Hello List,
>
> while running Kernel 4.4.2 on the host and Kernel 4.1.18 in guests we're
> seeing frequent hangs inside our guests.
>
> This happens for all guests at the same time.
>
> All of them show a trace + soft lockup while seeing no message at the host.
>
> Is this a known bug? I expect this one inside the kernel kvm module as
> it happens at the same minute on all guests.
>
> Greets Stefan
>