rcu_sched_state detected stall on CPU 6

rcu_sched_state detected stall on CPU 6

[Harald Dunkel]

[-- Attachment #1.1: Type: text/plain, Size: 4016 bytes --]

Hi folks,

I received this for 3.0.4 (amd64) (see attachment for the
complete kern.log).

Sep 30 10:09:03 cecil kernel: [ 9345.593031] INFO: rcu_sched_state detected stall on CPU 6 (t=0 jiffies)
Sep 30 10:09:03 cecil kernel: [ 9345.593035] sending NMI to all CPUs:
Sep 30 10:09:03 cecil kernel: [ 9345.593040] NMI backtrace for cpu 6
Sep 30 10:09:03 cecil kernel: [ 9345.593043] CPU 6
:
:
ep 30 10:09:03 cecil kernel: [ 9345.593194] Call Trace:
Sep 30 10:09:03 cecil kernel: [ 9345.593195]  <IRQ>
Sep 30 10:09:03 cecil kernel: [ 9345.593200]  [<ffffffff810179e2>] ? arch_trigger_all_cpu_backtrace+0x59/0x67
Sep 30 10:09:03 cecil kernel: [ 9345.593204]  [<ffffffff81076033>] ? __rcu_pending+0x7e/0x337
Sep 30 10:09:03 cecil kernel: [ 9345.593208]  [<ffffffff8105580c>] ? tick_nohz_handler+0xcd/0xcd
Sep 30 10:09:03 cecil kernel: [ 9345.593211]  [<ffffffff81076e45>] ? rcu_check_callbacks+0xbb/0x10a
Sep 30 10:09:03 cecil kernel: [ 9345.593214]  [<ffffffff8103dea5>] ? update_process_times+0x31/0x63
Sep 30 10:09:03 cecil kernel: [ 9345.593217]  [<ffffffff81055873>] ? tick_sched_timer+0x67/0x8d
Sep 30 10:09:03 cecil kernel: [ 9345.593219]  [<ffffffff8104bec1>] ? __run_hrtimer.isra.28+0x4f/0xa8
Sep 30 10:09:03 cecil kernel: [ 9345.593222]  [<ffffffff8104c47f>] ? hrtimer_interrupt+0xd5/0x1a1
Sep 30 10:09:03 cecil kernel: [ 9345.593225]  [<ffffffff8101701f>] ? smp_apic_timer_interrupt+0x7d/0x8f
Sep 30 10:09:03 cecil kernel: [ 9345.593229]  [<ffffffff812ac48e>] ? apic_timer_interrupt+0xe/0x20
Sep 30 10:09:03 cecil kernel: [ 9345.593231]  [<ffffffff812ac493>] ? apic_timer_interrupt+0x13/0x20
Sep 30 10:09:03 cecil kernel: [ 9345.593232]  <EOI>
Sep 30 10:09:03 cecil kernel: [ 9345.593236]  [<ffffffff81175830>] ? intel_idle+0xd1/0xed
Sep 30 10:09:03 cecil kernel: [ 9345.593239]  [<ffffffff8117580c>] ? intel_idle+0xad/0xed
Sep 30 10:09:03 cecil kernel: [ 9345.593242]  [<ffffffff8120a273>] ? cpuidle_idle_call+0x82/0xb9
Sep 30 10:09:03 cecil kernel: [ 9345.593245]  [<ffffffff8100112d>] ? cpu_idle+0x4e/0x99
Sep 30 10:09:03 cecil kernel: [ 9345.593246] Code: 0f 1f 84 00 00 00 00 00 48 ff c8 75 fb 48 ff c8 c3 48 8b 05 c8 40 2d 00 ff e0 48 8d 04 bd 00 00 00 00 65 48 8b 14 25 98 02 01 00
Sep 30 10:09:03 cecil kernel: [ 9345.593271] Call Trace:
Sep 30 10:09:03 cecil kernel: [ 9345.593272]  <IRQ>  [<ffffffff810179e2>] ? arch_trigger_all_cpu_backtrace+0x59/0x67
Sep 30 10:09:03 cecil kernel: [ 9345.593276]  [<ffffffff81076033>] ? __rcu_pending+0x7e/0x337
Sep 30 10:09:03 cecil kernel: [ 9345.593279]  [<ffffffff8105580c>] ? tick_nohz_handler+0xcd/0xcd
Sep 30 10:09:03 cecil kernel: [ 9345.593281]  [<ffffffff81076e45>] ? rcu_check_callbacks+0xbb/0x10a
Sep 30 10:09:03 cecil kernel: [ 9345.593284]  [<ffffffff8103dea5>] ? update_process_times+0x31/0x63
Sep 30 10:09:03 cecil kernel: [ 9345.593286]  [<ffffffff81055873>] ? tick_sched_timer+0x67/0x8d
Sep 30 10:09:03 cecil kernel: [ 9345.593288]  [<ffffffff8104bec1>] ? __run_hrtimer.isra.28+0x4f/0xa8
Sep 30 10:09:03 cecil kernel: [ 9345.593291]  [<ffffffff8104c47f>] ? hrtimer_interrupt+0xd5/0x1a1
Sep 30 10:09:03 cecil kernel: [ 9345.593293]  [<ffffffff8101701f>] ? smp_apic_timer_interrupt+0x7d/0x8f
Sep 30 10:09:03 cecil kernel: [ 9345.593296]  [<ffffffff812ac48e>] ? apic_timer_interrupt+0xe/0x20
Sep 30 10:09:03 cecil kernel: [ 9345.593298]  [<ffffffff812ac493>] ? apic_timer_interrupt+0x13/0x20
Sep 30 10:09:03 cecil kernel: [ 9345.593300]  <EOI>  [<ffffffff81175830>] ? intel_idle+0xd1/0xed
Sep 30 10:09:03 cecil kernel: [ 9345.593304]  [<ffffffff8117580c>] ? intel_idle+0xad/0xed
Sep 30 10:09:03 cecil kernel: [ 9345.593307]  [<ffffffff8120a273>] ? cpuidle_idle_call+0x82/0xb9
Sep 30 10:09:03 cecil kernel: [ 9345.593309]  [<ffffffff8100112d>] ? cpu_idle+0x4e/0x99
:

When the problem occured I was running an rsync of some huge
files (>4 GByte each) from one USB device to another. It was
already running for more than 2 hours.

Please mail if I can help to track this down.


Regards

Harri


[-- Attachment #1.2: kern.log.gz --]
[-- Type: application/x-tar, Size: 3864 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: rcu_sched_state detected stall on CPU 6

[Justin Piszcz]

On Fri, 30 Sep 2011, Harald Dunkel wrote:

> Hi folks,
>

This is a known bug, upgrade to 3.1-rc8 and your problem should go away, I 
had the same problem with 3.0.4 (amd64)

kern.log:Sep 11 18:11:43 p34 kernel: [26902.879347] INFO: rcu_sched_state 
detected stall on CPU 20 (t=60000 jiffies)

Justin.