* 2.6.31.4 panic: CRED: put_cred_rcu() sees ffff880204e58c00 with usage 82150912 @ 2009-10-30 11:06 Nikola Ciprich 2009-10-30 11:15 ` Nikola Ciprich 0 siblings, 1 reply; 5+ messages in thread From: Nikola Ciprich @ 2009-10-30 11:06 UTC (permalink / raw) To: Linux kernel list; +Cc: nikola.ciprich Hi, some time ago, I updated my KVM hosting machine to 2.6.31.1 and it just died horribly: Oct 30 10:45:17 vbox [706369.133516] Kernel panic - not syncing: CRED: put_cred_rcu() sees ffff880204e58c00 with usage 82150912 Oct 30 10:45:17 vbox [706369.133519] Oct 30 10:45:17 vbox [706369.144990] Pid: 19, comm: ksoftirqd/5 Not tainted 2.6.31lb.02 #1 Oct 30 10:45:17 vbox [706369.151554] Call Trace: Oct 30 10:45:17 vbox [706369.154332] <IRQ> Oct 30 10:45:17 vbox [<ffffffff8104c1fa>] panic+0xaa/0x180 Oct 30 10:45:17 vbox [706369.160280] [<ffffffff81322b90>] ? _spin_unlock+0x30/0x60 Oct 30 10:45:17 vbox [706369.166256] [<ffffffff810f5671>] ? add_partial+0x21/0x90 Oct 30 10:45:17 vbox [706369.172155] [<ffffffff810f6a92>] ? __slab_free+0x92/0x3c0 Oct 30 10:45:17 vbox [706369.178127] [<ffffffff81102317>] ? file_free_rcu+0x37/0x50 Oct 30 10:45:17 vbox [706369.184198] [<ffffffff8106c655>] put_cred_rcu+0x75/0x80 Oct 30 10:45:17 vbox [706369.190008] [<ffffffff810a2525>] __rcu_process_callbacks+0x125/0x250 Oct 30 10:45:17 vbox [706369.197020] [<ffffffff810a2689>] rcu_process_callbacks+0x39/0x60 Oct 30 10:45:17 vbox [706369.203624] [<ffffffff81052a61>] __do_softirq+0xc1/0x250 Oct 30 10:45:17 vbox [706369.209506] [<ffffffff81053860>] ? ksoftirqd+0x0/0x1a0 Oct 30 10:45:17 vbox [706369.215182] [<ffffffff8100c4dc>] call_softirq+0x1c/0x30 Oct 30 10:45:17 vbox [706369.220986] [<ffffffff8100e46d>] do_softirq+0x3d/0x80 Oct 30 10:45:17 vbox [706369.227317] [<ffffffff81053860>] ? ksoftirqd+0x0/0x1a0 Oct 30 10:45:17 vbox [706369.233020] [<ffffffff810538e4>] ksoftirqd+0x84/0x1a0 Oct 30 10:45:17 vbox [706369.238622] [<ffffffff81066686>] kthread+0xa6/0xb0 Oct 30 10:45:17 vbox [706369.243956] [<ffffffff8100c3da>] child_rip+0xa/0x20 Oct 30 10:45:17 vbox [706369.249390] [<ffffffff810665e0>] ? kthread+0x0/0xb0 Oct 30 10:45:17 vbox [706369.254806] [<ffffffff8100c3d0>] ? child_rip+0x0/0x20 Oct 30 10:45:17 vbox [706369.260454] Rebooting in 10 seconds.. (trace is obtained from netconsole, so hopefully it's not mangled). The machine was running ~30 KVM guests, it's 8CPU 16GB x86_64, when it crashed, it was only moderately loaded. Never had this (or any other) kind of crash on it before. I know there's 2.6.31.5 already out, but I'm not sure if some related problem has been reported/fixed and I'm obviously not able to quicky test/reproduce it with latest kernel, so I'm rather reporting. Should more information/testing/etc be required, I'll be glad to help regards nik -- ------------------------------------- Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax: +420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: servis@linuxbox.cz ------------------------------------- ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: 2.6.31.4 panic: CRED: put_cred_rcu() sees ffff880204e58c00 with usage 82150912 2009-10-30 11:06 2.6.31.4 panic: CRED: put_cred_rcu() sees ffff880204e58c00 with usage 82150912 Nikola Ciprich @ 2009-10-30 11:15 ` Nikola Ciprich 2009-10-30 18:20 ` Rafael J. Wysocki 2009-11-03 14:46 ` Marcelo Tosatti 0 siblings, 2 replies; 5+ messages in thread From: Nikola Ciprich @ 2009-10-30 11:15 UTC (permalink / raw) To: Linux kernel list; +Cc: nikola.ciprich, KVM list Ouch, typo in subject, it's 2.6.31.1 of course. sorry about that. also CCing kvm. n. On Fri, Oct 30, 2009 at 12:06:32PM +0100, Nikola Ciprich wrote: > Hi, > some time ago, I updated my KVM hosting machine to 2.6.31.1 and it just died horribly: > Oct 30 10:45:17 vbox [706369.133516] Kernel panic - not syncing: CRED: put_cred_rcu() sees ffff880204e58c00 with usage 82150912 > Oct 30 10:45:17 vbox [706369.133519] > Oct 30 10:45:17 vbox [706369.144990] Pid: 19, comm: ksoftirqd/5 Not tainted 2.6.31lb.02 #1 > Oct 30 10:45:17 vbox [706369.151554] Call Trace: > Oct 30 10:45:17 vbox [706369.154332] <IRQ> > Oct 30 10:45:17 vbox [<ffffffff8104c1fa>] panic+0xaa/0x180 > Oct 30 10:45:17 vbox [706369.160280] [<ffffffff81322b90>] ? _spin_unlock+0x30/0x60 > Oct 30 10:45:17 vbox [706369.166256] [<ffffffff810f5671>] ? add_partial+0x21/0x90 > Oct 30 10:45:17 vbox [706369.172155] [<ffffffff810f6a92>] ? __slab_free+0x92/0x3c0 > Oct 30 10:45:17 vbox [706369.178127] [<ffffffff81102317>] ? file_free_rcu+0x37/0x50 > Oct 30 10:45:17 vbox [706369.184198] [<ffffffff8106c655>] put_cred_rcu+0x75/0x80 > Oct 30 10:45:17 vbox [706369.190008] [<ffffffff810a2525>] __rcu_process_callbacks+0x125/0x250 > Oct 30 10:45:17 vbox [706369.197020] [<ffffffff810a2689>] rcu_process_callbacks+0x39/0x60 > Oct 30 10:45:17 vbox [706369.203624] [<ffffffff81052a61>] __do_softirq+0xc1/0x250 > Oct 30 10:45:17 vbox [706369.209506] [<ffffffff81053860>] ? ksoftirqd+0x0/0x1a0 > Oct 30 10:45:17 vbox [706369.215182] [<ffffffff8100c4dc>] call_softirq+0x1c/0x30 > Oct 30 10:45:17 vbox [706369.220986] [<ffffffff8100e46d>] do_softirq+0x3d/0x80 > Oct 30 10:45:17 vbox [706369.227317] [<ffffffff81053860>] ? ksoftirqd+0x0/0x1a0 > Oct 30 10:45:17 vbox [706369.233020] [<ffffffff810538e4>] ksoftirqd+0x84/0x1a0 > Oct 30 10:45:17 vbox [706369.238622] [<ffffffff81066686>] kthread+0xa6/0xb0 > Oct 30 10:45:17 vbox [706369.243956] [<ffffffff8100c3da>] child_rip+0xa/0x20 > Oct 30 10:45:17 vbox [706369.249390] [<ffffffff810665e0>] ? kthread+0x0/0xb0 > Oct 30 10:45:17 vbox [706369.254806] [<ffffffff8100c3d0>] ? child_rip+0x0/0x20 > Oct 30 10:45:17 vbox [706369.260454] Rebooting in 10 seconds.. > (trace is obtained from netconsole, so hopefully it's not mangled). > The machine was running ~30 KVM guests, it's 8CPU 16GB x86_64, when it crashed, it was only > moderately loaded. Never had this (or any other) kind of crash on it before. > I know there's 2.6.31.5 already out, but I'm not sure if some related problem has been > reported/fixed and I'm obviously not able to quicky test/reproduce it with latest kernel, > so I'm rather reporting. > Should more information/testing/etc be required, I'll be glad to help > regards > nik > > -- > ------------------------------------- > Nikola CIPRICH > LinuxBox.cz, s.r.o. > 28. rijna 168, 709 01 Ostrava > > tel.: +420 596 603 142 > fax: +420 596 621 273 > mobil: +420 777 093 799 > www.linuxbox.cz > > mobil servis: +420 737 238 656 > email servis: servis@linuxbox.cz > ------------------------------------- > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > -- ------------------------------------- Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax: +420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: servis@linuxbox.cz ------------------------------------- ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: 2.6.31.4 panic: CRED: put_cred_rcu() sees ffff880204e58c00 with usage 82150912 2009-10-30 11:15 ` Nikola Ciprich @ 2009-10-30 18:20 ` Rafael J. Wysocki 2009-11-03 14:46 ` Marcelo Tosatti 1 sibling, 0 replies; 5+ messages in thread From: Rafael J. Wysocki @ 2009-10-30 18:20 UTC (permalink / raw) To: Nikola Ciprich; +Cc: Linux kernel list, nikola.ciprich, KVM list On Friday 30 October 2009, Nikola Ciprich wrote: > Ouch, typo in subject, it's 2.6.31.1 of course. sorry about that. > also CCing kvm. What about later kernels? Mainline in particular? Rafael ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: 2.6.31.4 panic: CRED: put_cred_rcu() sees ffff880204e58c00 with usage 82150912 2009-10-30 11:15 ` Nikola Ciprich 2009-10-30 18:20 ` Rafael J. Wysocki @ 2009-11-03 14:46 ` Marcelo Tosatti 2009-11-03 21:12 ` Nikola Ciprich 1 sibling, 1 reply; 5+ messages in thread From: Marcelo Tosatti @ 2009-11-03 14:46 UTC (permalink / raw) To: Nikola Ciprich; +Cc: Linux kernel list, nikola.ciprich, KVM list On Fri, Oct 30, 2009 at 12:15:34PM +0100, Nikola Ciprich wrote: > Ouch, typo in subject, it's 2.6.31.1 of course. sorry about that. > also CCing kvm. > n. > > On Fri, Oct 30, 2009 at 12:06:32PM +0100, Nikola Ciprich wrote: > > Hi, > > some time ago, I updated my KVM hosting machine to 2.6.31.1 and it just died horribly: Nikola, Upgraded from what? Did you see experience the crash again? > > Oct 30 10:45:17 vbox [706369.133516] Kernel panic - not syncing: CRED: put_cred_rcu() sees ffff880204e58c00 with usage 82150912 > > Oct 30 10:45:17 vbox [706369.133519] > > Oct 30 10:45:17 vbox [706369.144990] Pid: 19, comm: ksoftirqd/5 Not tainted 2.6.31lb.02 #1 > > Oct 30 10:45:17 vbox [706369.151554] Call Trace: > > Oct 30 10:45:17 vbox [706369.154332] <IRQ> > > Oct 30 10:45:17 vbox [<ffffffff8104c1fa>] panic+0xaa/0x180 > > Oct 30 10:45:17 vbox [706369.160280] [<ffffffff81322b90>] ? _spin_unlock+0x30/0x60 > > Oct 30 10:45:17 vbox [706369.166256] [<ffffffff810f5671>] ? add_partial+0x21/0x90 > > Oct 30 10:45:17 vbox [706369.172155] [<ffffffff810f6a92>] ? __slab_free+0x92/0x3c0 > > Oct 30 10:45:17 vbox [706369.178127] [<ffffffff81102317>] ? file_free_rcu+0x37/0x50 > > Oct 30 10:45:17 vbox [706369.184198] [<ffffffff8106c655>] put_cred_rcu+0x75/0x80 > > Oct 30 10:45:17 vbox [706369.190008] [<ffffffff810a2525>] __rcu_process_callbacks+0x125/0x250 > > Oct 30 10:45:17 vbox [706369.197020] [<ffffffff810a2689>] rcu_process_callbacks+0x39/0x60 > > Oct 30 10:45:17 vbox [706369.203624] [<ffffffff81052a61>] __do_softirq+0xc1/0x250 > > Oct 30 10:45:17 vbox [706369.209506] [<ffffffff81053860>] ? ksoftirqd+0x0/0x1a0 > > Oct 30 10:45:17 vbox [706369.215182] [<ffffffff8100c4dc>] call_softirq+0x1c/0x30 > > Oct 30 10:45:17 vbox [706369.220986] [<ffffffff8100e46d>] do_softirq+0x3d/0x80 > > Oct 30 10:45:17 vbox [706369.227317] [<ffffffff81053860>] ? ksoftirqd+0x0/0x1a0 > > Oct 30 10:45:17 vbox [706369.233020] [<ffffffff810538e4>] ksoftirqd+0x84/0x1a0 > > Oct 30 10:45:17 vbox [706369.238622] [<ffffffff81066686>] kthread+0xa6/0xb0 > > Oct 30 10:45:17 vbox [706369.243956] [<ffffffff8100c3da>] child_rip+0xa/0x20 > > Oct 30 10:45:17 vbox [706369.249390] [<ffffffff810665e0>] ? kthread+0x0/0xb0 > > Oct 30 10:45:17 vbox [706369.254806] [<ffffffff8100c3d0>] ? child_rip+0x0/0x20 > > Oct 30 10:45:17 vbox [706369.260454] Rebooting in 10 seconds.. > > (trace is obtained from netconsole, so hopefully it's not mangled). > > The machine was running ~30 KVM guests, it's 8CPU 16GB x86_64, when it crashed, it was only > > moderately loaded. Never had this (or any other) kind of crash on it before. > > I know there's 2.6.31.5 already out, but I'm not sure if some related problem has been > > reported/fixed and I'm obviously not able to quicky test/reproduce it with latest kernel, > > so I'm rather reporting. > > Should more information/testing/etc be required, I'll be glad to help > > regards > > nik ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: 2.6.31.4 panic: CRED: put_cred_rcu() sees ffff880204e58c00 with usage 82150912 2009-11-03 14:46 ` Marcelo Tosatti @ 2009-11-03 21:12 ` Nikola Ciprich 0 siblings, 0 replies; 5+ messages in thread From: Nikola Ciprich @ 2009-11-03 21:12 UTC (permalink / raw) To: Marcelo Tosatti; +Cc: Nikola Ciprich, Linux kernel list, KVM list [-- Attachment #1: Type: text/plain, Size: 3925 bytes --] Hello Marcelo, well, my report might have been a bit misleading, 2.6.31.2 has been running there for almost 3 weeks. So the problem didn't occur JUST after the upgrade. But it never occured before it, while we were using older kernels (varius 2.6.30.x, 2.6.29.x and older). I've updated machine to 2.6.31.5, and it's been running without problem since then, but it's been just few days, so we'll see. I'll report if the problem appears again. Thanks. regards nik On Tue, Nov 03, 2009 at 12:46:38PM -0200, Marcelo Tosatti wrote: > On Fri, Oct 30, 2009 at 12:15:34PM +0100, Nikola Ciprich wrote: > > Ouch, typo in subject, it's 2.6.31.1 of course. sorry about that. > > also CCing kvm. > > n. > > > > On Fri, Oct 30, 2009 at 12:06:32PM +0100, Nikola Ciprich wrote: > > > Hi, > > > some time ago, I updated my KVM hosting machine to 2.6.31.1 and it just died horribly: > > Nikola, > > Upgraded from what? Did you see experience the crash again? > > > > Oct 30 10:45:17 vbox [706369.133516] Kernel panic - not syncing: CRED: put_cred_rcu() sees ffff880204e58c00 with usage 82150912 > > > Oct 30 10:45:17 vbox [706369.133519] > > > Oct 30 10:45:17 vbox [706369.144990] Pid: 19, comm: ksoftirqd/5 Not tainted 2.6.31lb.02 #1 > > > Oct 30 10:45:17 vbox [706369.151554] Call Trace: > > > Oct 30 10:45:17 vbox [706369.154332] <IRQ> > > > Oct 30 10:45:17 vbox [<ffffffff8104c1fa>] panic+0xaa/0x180 > > > Oct 30 10:45:17 vbox [706369.160280] [<ffffffff81322b90>] ? _spin_unlock+0x30/0x60 > > > Oct 30 10:45:17 vbox [706369.166256] [<ffffffff810f5671>] ? add_partial+0x21/0x90 > > > Oct 30 10:45:17 vbox [706369.172155] [<ffffffff810f6a92>] ? __slab_free+0x92/0x3c0 > > > Oct 30 10:45:17 vbox [706369.178127] [<ffffffff81102317>] ? file_free_rcu+0x37/0x50 > > > Oct 30 10:45:17 vbox [706369.184198] [<ffffffff8106c655>] put_cred_rcu+0x75/0x80 > > > Oct 30 10:45:17 vbox [706369.190008] [<ffffffff810a2525>] __rcu_process_callbacks+0x125/0x250 > > > Oct 30 10:45:17 vbox [706369.197020] [<ffffffff810a2689>] rcu_process_callbacks+0x39/0x60 > > > Oct 30 10:45:17 vbox [706369.203624] [<ffffffff81052a61>] __do_softirq+0xc1/0x250 > > > Oct 30 10:45:17 vbox [706369.209506] [<ffffffff81053860>] ? ksoftirqd+0x0/0x1a0 > > > Oct 30 10:45:17 vbox [706369.215182] [<ffffffff8100c4dc>] call_softirq+0x1c/0x30 > > > Oct 30 10:45:17 vbox [706369.220986] [<ffffffff8100e46d>] do_softirq+0x3d/0x80 > > > Oct 30 10:45:17 vbox [706369.227317] [<ffffffff81053860>] ? ksoftirqd+0x0/0x1a0 > > > Oct 30 10:45:17 vbox [706369.233020] [<ffffffff810538e4>] ksoftirqd+0x84/0x1a0 > > > Oct 30 10:45:17 vbox [706369.238622] [<ffffffff81066686>] kthread+0xa6/0xb0 > > > Oct 30 10:45:17 vbox [706369.243956] [<ffffffff8100c3da>] child_rip+0xa/0x20 > > > Oct 30 10:45:17 vbox [706369.249390] [<ffffffff810665e0>] ? kthread+0x0/0xb0 > > > Oct 30 10:45:17 vbox [706369.254806] [<ffffffff8100c3d0>] ? child_rip+0x0/0x20 > > > Oct 30 10:45:17 vbox [706369.260454] Rebooting in 10 seconds.. > > > (trace is obtained from netconsole, so hopefully it's not mangled). > > > The machine was running ~30 KVM guests, it's 8CPU 16GB x86_64, when it crashed, it was only > > > moderately loaded. Never had this (or any other) kind of crash on it before. > > > I know there's 2.6.31.5 already out, but I'm not sure if some related problem has been > > > reported/fixed and I'm obviously not able to quicky test/reproduce it with latest kernel, > > > so I'm rather reporting. > > > Should more information/testing/etc be required, I'll be glad to help > > > regards > > > nik > -- ------------------------------------- Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax: +420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: servis@linuxbox.cz ------------------------------------- [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2009-11-03 21:30 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2009-10-30 11:06 2.6.31.4 panic: CRED: put_cred_rcu() sees ffff880204e58c00 with usage 82150912 Nikola Ciprich 2009-10-30 11:15 ` Nikola Ciprich 2009-10-30 18:20 ` Rafael J. Wysocki 2009-11-03 14:46 ` Marcelo Tosatti 2009-11-03 21:12 ` Nikola Ciprich
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox