-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 01/23/2014 08:33 PM, Dave Hansen wrote: > On 01/23/2014 10:55 AM, Dave Hansen wrote: >> On 01/21/2014 08:38 AM, Toralf Förster wrote: >>> Jan 21 17:18:57 n22 kernel: INFO: rcu_sched self-detected stall on CPU { 2} (t=60001 jiffies g=18494 c=18493 q=183951) >>> Jan 21 17:18:57 n22 kernel: sending NMI to all CPUs: >>> Jan 21 17:18:57 n22 kernel: NMI backtrace for cpu 2 >>> Jan 21 17:18:57 n22 kernel: CPU: 2 PID: 6779 Comm: qemu-system-x86 Not tainted 3.13.0 #3 >>> Jan 21 17:18:57 n22 kernel: Hardware name: LENOVO 4180F65/4180F65, BIOS 83ET75WW (1.45 ) 05/10/2013 >>> Jan 21 17:18:57 n22 kernel: task: e921c370 ti: e5f36000 task.ti: e5f36000 >> >> I'm seeing a very similar hang with an ubuntu guest and a custom kernel. >> I'm on commit 0dc3fd0249a, and it's 100% reproducible every time I run KVM. > > Did a little more LKML digging and found this: > > http://marc.info/?l=linux-kernel&m=139038631607917&q=raw > > Peter's fix works for me. I'm also running a CONFIG_PREEMPT_VOLUNTARY=y > config. > > Hhhm, that fix seems not to be applicable for 3.13.x With 3.13.2 I removed multimedia, sound and wlan and few file systems from the kernel's config, now at least the kernel doesn't hang, but I do still get while trying to start a virtual machine (and please note, I run into a similar issue while just umounting a btrfs file system) : Feb 9 17:14:59 n22 polkitd[4129]: Operator of unix-session:/org/freedesktop/ConsoleKit/Session1 successfully authenticated as unix-user:root to gain TEMPORARY authorization for action org.libvirt.unix.manage for unix-process:6544:23328 [/usr/bin/python2.7 /usr/share/virt-manager/virt-manager] (owned by unix-user:tfoerste) Feb 9 17:15:01 n22 kernel: type=1006 audit(1391962501.815:5): pid=6601 uid=0 old auid=4294967295 new auid=0 old ses=4294967295 new ses=4 res=1 Feb 9 17:15:01 n22 crond[6601]: pam_unix(crond:session): session opened for user root by (uid=0) Feb 9 17:15:01 n22 CROND[6602]: (root) CMD (/usr/lib/sa/sa1 60 15 ) Feb 9 17:15:05 n22 kernel: NET: Registered protocol family 17 Feb 9 17:15:05 n22 kernel: device vnet0 entered promiscuous mode Feb 9 17:15:05 n22 kernel: br0: port 4(vnet0) entered forwarding state Feb 9 17:15:05 n22 kernel: br0: port 4(vnet0) entered forwarding state Feb 9 17:15:05 n22 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): br0: link becomes ready Feb 9 17:15:05 n22 kernel: cgroup: libvirtd (5677) created nested cgroup for controller "memory" which has incomplete hierarchy support. Nested cgroups may change behavior in the future. Feb 9 17:15:05 n22 kernel: cgroup: "memory" requires setting use_hierarchy to 1 on the root. Feb 9 17:15:08 n22 ntpd[5582]: Listen normally on 5 br0 fe80::acb9:fbff:fea0:57f9 UDP 123 Feb 9 17:15:08 n22 ntpd[5582]: Listen normally on 6 vnet0 fe80::fc54:ff:fe46:1fc4 UDP 123 Feb 9 17:15:08 n22 ntpd[5582]: peers refreshed Feb 9 17:16:57 n22 kernel: INFO: rcu_sched self-detected stall on CPU { 0} (t=60000 jiffies g=16324 c=16323 q=11304) Feb 9 17:16:57 n22 kernel: sending NMI to all CPUs: Feb 9 17:16:57 n22 kernel: NMI backtrace for cpu 0 Feb 9 17:16:57 n22 kernel: CPU: 0 PID: 6617 Comm: qemu-system-x86 Not tainted 3.13.2 #17 Feb 9 17:16:57 n22 kernel: Hardware name: LENOVO 4180F65/4180F65, BIOS 83ET75WW (1.45 ) 05/10/2013 Feb 9 17:16:57 n22 kernel: task: e70e5b40 ti: e736e000 task.ti: e736e000 Feb 9 17:16:57 n22 kernel: EIP: 0060:[] EFLAGS: 00000006 CPU: 0 Feb 9 17:16:57 n22 kernel: EIP is at __const_udelay+0xd/0x20 Feb 9 17:16:57 n22 kernel: EAX: 01062560 EBX: 00002710 ECX: c1599460 EDX: 00278bf9 Feb 9 17:16:57 n22 kernel: ESI: c159c580 EDI: f3650c40 EBP: e736fca8 ESP: e736fca8 Feb 9 17:16:57 n22 kernel: DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068 Feb 9 17:16:57 n22 kernel: CR0: 80050033 CR2: 091441f4 CR3: 00110000 CR4: 000427f0 Feb 9 17:16:57 n22 kernel: Stack: Feb 9 17:16:57 n22 kernel: e736fcb8 c102ba30 c1500d39 c159c580 e736fcfc c107febc c150cd0c 0000ea60 Feb 9 17:16:57 n22 kernel: 00003fc4 00003fc3 00002c28 c10606d8 00000001 c159c580 c15c4a84 f3650c40 Feb 9 17:16:57 n22 kernel: 00002c28 00000000 e70e5b40 00000000 00000000 e736fd10 c1044126 f3650e60 Feb 9 17:16:57 n22 kernel: Call Trace: Feb 9 17:16:57 n22 kernel: [] arch_trigger_all_cpu_backtrace+0x50/0x70 Feb 9 17:16:57 n22 kernel: [] rcu_check_callbacks+0x31c/0x520 Feb 9 17:16:57 n22 kernel: [] ? account_system_time+0xb8/0x170 Feb 9 17:16:57 n22 kernel: [] update_process_times+0x36/0x70 Feb 9 17:16:57 n22 kernel: [] tick_sched_handle.isra.13+0x2e/0x30 Feb 9 17:16:57 n22 kernel: [] tick_sched_timer+0x3b/0x70 Feb 9 17:16:57 n22 kernel: [] ? __remove_hrtimer+0x42/0xa0 Feb 9 17:16:57 n22 kernel: [] __run_hrtimer.isra.29+0x3c/0xb0 Feb 9 17:16:57 n22 kernel: [] hrtimer_interrupt+0x1e5/0x270 Feb 9 17:16:57 n22 kernel: [] local_apic_timer_interrupt+0x2a/0x50 Feb 9 17:16:57 n22 kernel: [] ? irq_enter+0x10/0x60 Feb 9 17:16:57 n22 kernel: [] smp_apic_timer_interrupt+0x2e/0x50 Feb 9 17:16:57 n22 kernel: [] apic_timer_interrupt+0x34/0x3c Feb 9 17:16:57 n22 kernel: [] ? __srcu_read_unlock+0xb/0x20 Feb 9 17:16:57 n22 kernel: [] kvm_arch_vcpu_ioctl_run+0x8c0/0xf30 [kvm] Feb 9 17:16:57 n22 kernel: [] kvm_vcpu_ioctl+0x40b/0x4a0 [kvm] Feb 9 17:16:57 n22 kernel: [] ? put_prev_task_fair+0x5c/0x3e0 Feb 9 17:16:57 n22 kernel: [] ? ttwu_do_activate.constprop.82+0x53/0x60 Feb 9 17:16:57 n22 kernel: [] ? __dequeue_entity+0x20/0x40 Feb 9 17:16:57 n22 kernel: [] ? vcpu_put+0x20/0x20 [kvm] Feb 9 17:16:57 n22 kernel: [] do_vfs_ioctl+0x6a/0x540 Feb 9 17:16:57 n22 kernel: [] ? __schedule+0x20e/0x660 Feb 9 17:16:57 n22 kernel: [] SyS_ioctl+0x40/0x70 Feb 9 17:16:57 n22 kernel: [] sysenter_do_call+0x12/0x22 Feb 9 17:16:57 n22 kernel: [] ? dns_resolver_instantiate+0x400/0x480 Feb 9 17:16:57 n22 kernel: Code: fd 48 5d c3 8d 76 00 8d bc 27 00 00 00 00 55 89 e5 ff 15 6c ea 5a c1 5d c3 90 8d 74 26 00 55 c1 e0 02 89 e5 64 8b 15 5c 7d 62 c1 <69> d2 fa 00 00 00 f7 e2 8d 42 01 ff 15 6c ea 5a c1 5d c3 55 69 Feb 9 17:16:57 n22 kernel: NMI backtrace for cpu 1 Feb 9 17:16:57 n22 kernel: CPU: 1 PID: 5818 Comm: period_search_1 Not tainted 3.13.2 #17 Feb 9 17:16:57 n22 kernel: Hardware name: LENOVO 4180F65/4180F65, BIOS 83ET75WW (1.45 ) 05/10/2013 Feb 9 17:16:57 n22 kernel: task: f00716d0 ti: ef3be000 task.ti: ef3be000 Feb 9 17:16:57 n22 kernel: EIP: 0073:[<08050915>] EFLAGS: 00000246 CPU: 1 Feb 9 17:16:57 n22 kernel: EIP is at 0x8050915 Feb 9 17:16:57 n22 kernel: EAX: 000000ac EBX: 0000004d ECX: 00000120 EDX: 0000000e Feb 9 17:16:57 n22 kernel: ESI: 00000980 EDI: 0a846fc8 EBP: bfcd34b8 ESP: bfcc9dc0 Feb 9 17:16:57 n22 kernel: DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b Feb 9 17:16:57 n22 kernel: Feb 9 17:16:57 n22 kernel: NMI backtrace for cpu 2 Feb 9 17:16:57 n22 kernel: CPU: 2 PID: 5817 Comm: period_search_1 Not tainted 3.13.2 #17 Feb 9 17:16:57 n22 kernel: Hardware name: LENOVO 4180F65/4180F65, BIOS 83ET75WW (1.45 ) 05/10/2013 Feb 9 17:16:57 n22 kernel: task: f0075fd0 ti: f12ca000 task.ti: f12ca000 Feb 9 17:16:57 n22 kernel: EIP: 0073:[<08050d94>] EFLAGS: 00000287 CPU: 2 Feb 9 17:16:57 n22 kernel: EIP is at 0x8050d94 Feb 9 17:16:57 n22 kernel: EAX: 00000128 EBX: 0858a720 ECX: bfcb42e0 EDX: 00000180 Feb 9 17:16:57 n22 kernel: ESI: bfcb3320 EDI: 000001f8 EBP: bfcbc058 ESP: bfcb2960 Feb 9 17:16:57 n22 kernel: DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b Feb 9 17:16:57 n22 kernel: Feb 9 17:16:57 n22 kernel: NMI backtrace for cpu 3 Feb 9 17:16:57 n22 kernel: CPU: 3 PID: 5814 Comm: einstein_S6CasA Not tainted 3.13.2 #17 Feb 9 17:16:57 n22 kernel: Hardware name: LENOVO 4180F65/4180F65, BIOS 83ET75WW (1.45 ) 05/10/2013 Feb 9 17:16:57 n22 kernel: task: f0073b50 ti: ef722000 task.ti: ef722000 Feb 9 17:16:57 n22 kernel: EIP: 0073:[<080558e1>] EFLAGS: 00000202 CPU: 3 Feb 9 17:16:57 n22 kernel: EIP is at 0x80558e1 Feb 9 17:16:57 n22 kernel: EAX: b3214120 EBX: bfc8d160 ECX: b4589b98 EDX: 00010450 Feb 9 17:16:57 n22 kernel: ESI: 00010450 EDI: 00016cf1 EBP: bfc8d2f8 ESP: bfc8caa0 Feb 9 17:16:57 n22 kernel: DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b Feb 9 17:16:57 n22 kernel: Feb 9 17:17:53 n22 kernel: br0: port 4(vnet0) entered disabled state Feb 9 17:17:53 n22 kernel: device vnet0 left promiscuous mode Feb 9 17:17:53 n22 kernel: br0: port 4(vnet0) entered disabled state Feb 9 17:17:54 n22 ntpd[5582]: Deleting interface #6 vnet0, fe80::fc54:ff:fe46:1fc4#123, interface stats: received=0, sent=0, dropped=0, active_time=166 secs Feb 9 17:17:54 n22 ntpd[5582]: peers refreshed Feb 9 17:18:25 n22 polkitd[4129]: Unregistered Authentication Agent for unix-session:/org/freedesktop/ConsoleKit/Session1 (system bus name :1.18, object path /org/kde/PolicyKit1/AuthenticationAgent, locale en_US.utf8) Feb 9 17:18:26 n22 kdm: :0[4888]: pam_unix(kde:session): session closed for user tfoerste Feb 9 17:18:26 n22 shutdown[7138]: shutting down for system reboot Feb 9 17:18:27 n22 init: Switching to runlevel: 6 Feb 9 17:18:28 n22 thinkfan: Caught deadly signal. F - -- MfG/Sincerely Toralf Förster pgp finger print:1A37 6F99 4A9D 026F 13E2 4DCF C4EA CDDE 0076 E94E -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iF4EAREIAAYFAlL3rx4ACgkQxOrN3gB26U4uDwD+MofngLtD/et921CrLyVrAFr+ 1QkIVk0cyUFIAp4986wA/2d5zZ1jRj61uXnGidDcxCq2AyIl85Iqy0KkNkDGQ/FM =W0Ci -----END PGP SIGNATURE-----