From mboxrd@z Thu Jan 1 00:00:00 1970 From: Scot Doyle Subject: Re: Kernel panic when using bridge Date: Fri, 08 Apr 2011 23:51:10 -0500 Message-ID: <4D9FE5BE.6060600@scotdoyle.com> References: <4D9E62D9.5010400@scotdoyle.com> <20110408121700.0aad53fe@nehalam> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: netdev@vger.kernel.org To: Stephen Hemminger Return-path: Received: from smtp.scotdoyle.com ([74.207.249.244]:37225 "EHLO smtp.scotdoyle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751196Ab1DIEvN (ORCPT ); Sat, 9 Apr 2011 00:51:13 -0400 In-Reply-To: <20110408121700.0aad53fe@nehalam> Sender: netdev-owner@vger.kernel.org List-ID: On 04/08/2011 02:17 PM, Stephen Hemminger wrote: > Please reproduce with exactly 2.6.39-rc2 there were some bug fixes > to make sure that header was initialized. Hi Stephen, here's another panic with 2.6.39-rc2 (git commit bb3c90f0de7b34995b5e35cf5dc97a3d428b3761) using default kernel config options. # sysctl -a | grep bridge net.bridge.bridge-nf-call-arptables = 1 net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-filter-vlan-tagged = 0 net.bridge.bridge-nf-filter-pppoe-tagged = 0 # /etc/network/interfaces auto lo iface lo inet loopback auto br0 iface br0 inet static address x.y.z.237 netmask 255.255.255.224 gateway x.y.z.225 bridge_ports eth3 bridge_stp off bridge_maxwait 0 bridge_fd 0 auto br0:1 iface br0:1 inet static address 10.0.0.1 netmask 255.255.255.0 auto br0:2 iface br0:2 inet static address 10.0.1.1 netmask 255.255.255.0 ------ [ 1691.681069] BUG: unable to handle kernel NULL pointer dereference at 00000000000000cc [ 1691.688879] IP: [] ip_options_compile+0x1c1/0x435 [ 1691.695126] PGD 0 [ 1691.697131] Oops: 0000 [#1] SMP [ 1691.700357] last sysfs file: /sys/devices/virtual/misc/kvm/uevent [ 1691.706418] CPU 0 [ 1691.708241] Modules linked in: kvm_intel kvm bridge stp loop snd_pcm snd_timer snd soundcore snd_page_alloc tpm_tis i7core_edac psmouse ghes tpm evdev edac_core pcspkr serio_raw processor tpm_bios button dcdbas thermal_sys hed power_meter ext2 mbcache dm_mod raid1 md_mod sd_mod crc_t10dif usb_storage uas uhci_hcd mpt2sas scsi_transport_sas raid_class ehci_hcd igb scsi_mod usbcore dca bnx2 [last unloaded: scsi_wait_scan] [ 1691.745849] [ 1691.747330] Pid: 0, comm: swapper Not tainted 2.6.39-rc2+ #3 Dell Inc. PowerEdge R510/0DPRKF [ 1691.755752] RIP: 0010:[] [] ip_options_compile+0x1c1/0x435 [ 1691.764418] RSP: 0018:ffff88042f203af0 EFLAGS: 00010286 [ 1691.769702] RAX: 0000000000000024 RBX: ffff88041c9fa900 RCX: ffff880403466865 [ 1691.776800] RDX: 0000000000000027 RSI: 0000000000000000 RDI: ffffffff817e6100 [ 1691.783899] RBP: ffff880403466863 R08: ffffffffa01ade89 R09: ffff88042f203c58 [ 1691.790997] R10: ffffe1c4ff103b40 R11: 0000000000000004 R12: ffff88041c9fa928 [ 1691.798095] R13: 0000000000000027 R14: ffff88040346684e R15: 0000000000000027 [ 1691.805194] FS: 0000000000000000(0000) GS:ffff88042f200000(0000) knlGS:0000000000000000 [ 1691.813245] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 1691.818960] CR2: 00000000000000cc CR3: 0000000001603000 CR4: 00000000000006f0 [ 1691.826058] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1691.833156] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1691.840254] Process swapper (pid: 0, threadinfo ffffffff81600000, task ffffffff8160b020) [ 1691.848303] Stack: [ 1691.850300] ffff88042ec02900 ffff8804051ac740 0000000000000000 ffffffff817e6100 [ 1691.857693] 0000000000000282 ffffffff810ec848 0000000000000282 ffff88041c9fa928 [ 1691.865085] ffff88041c9fa900 ffff8804038e8000 ffff88040346684e ffff8804038e8000 [ 1691.872480] Call Trace: [ 1691.874910] [ 1691.877005] [] ? __slab_free+0x80/0x14a [ 1691.882465] [] ? br_parse_ip_options+0x133/0x1a0 [bridge] [ 1691.889480] [] ? br_nf_pre_routing+0x348/0x3cb [bridge] [ 1691.896324] [] ? cpumask_next_and+0x2b/0x3a [ 1691.902127] [] ? nf_iterate+0x41/0x7e [ 1691.907413] [] ? NF_HOOK.clone.4+0x56/0x56 [bridge] [ 1691.913908] [] ? NF_HOOK.clone.4+0x56/0x56 [bridge] [ 1691.920402] [] ? nf_hook_slow+0x73/0x114 [ 1691.925947] [] ? NF_HOOK.clone.4+0x56/0x56 [bridge] [ 1691.932442] [] ? NF_HOOK.clone.4+0x56/0x56 [bridge] [ 1691.938937] [] ? NF_HOOK.clone.4+0x3c/0x56 [bridge] [ 1691.945432] [] ? __kmalloc_node_track_caller+0xd4/0x10d [ 1691.952274] [] ? br_handle_frame+0x195/0x1ac [bridge] [ 1691.958942] [] ? br_handle_frame_finish+0x1c7/0x1c7 [bridge] [ 1691.966217] [] ? __netif_receive_skb+0x2a7/0x450 [ 1691.972452] [] ? netif_receive_skb+0x52/0x58 [ 1691.978340] [] ? napi_gro_receive+0x1f/0x2f [ 1691.984143] [] ? napi_skb_finish+0x1c/0x31 [ 1691.989862] [] ? igb_poll+0x6d9/0x9ee [igb] [ 1691.995666] [] ? try_to_wake_up+0x16a/0x17c [ 1692.001470] [] ? handle_irq_event+0x40/0x55 [ 1692.007275] [] ? arch_local_irq_save+0x14/0x1d [ 1692.013338] [] ? net_rx_action+0xa4/0x1b1 [ 1692.018971] [] ? __do_softirq+0xb8/0x176 [ 1692.024516] [] ? call_softirq+0x1c/0x30 [ 1692.029973] [] ? do_softirq+0x3f/0x84 [ 1692.035257] [] ? irq_exit+0x3f/0x8f [ 1692.040368] [] ? do_IRQ+0x85/0x9e [ 1692.045308] [] ? common_interrupt+0x13/0x13 [ 1692.051110] [ 1692.053204] [] ? enqueue_hrtimer+0x3f/0x53 [ 1692.058922] [] ? arch_local_irq_enable+0x7/0x8 [processor] [ 1692.066021] [] ? acpi_idle_enter_bm+0x218/0x250 [processor] [ 1692.073208] [] ? menu_select+0x169/0x296 [ 1692.078752] [] ? cpuidle_idle_call+0xf4/0x17e [ 1692.084727] [] ? cpu_idle+0xa2/0xc4 [ 1692.089838] [] ? start_kernel+0x3b9/0x3c4 [ 1692.095469] [] ? x86_64_start_kernel+0x102/0x10f [ 1692.101703] Code: 4d 02 3c 03 0f 86 59 02 00 00 0f b6 d0 44 39 ea 7f 32 83 c2 03 44 39 ea 0f 8f 45 02 00 00 48 85 db 74 18 48 8b 74 24 10 0f b6 c0 <8b> 96 cc 00 00 00 89 54 05 ff 41 80 4c 24 08 04 80 01 04 41 80 [ 1692.121051] RIP [] ip_options_compile+0x1c1/0x435 [ 1692.127382] RSP [ 1692.130850] CR2: 00000000000000cc [ 1692.134470] ---[ end trace 0afda543b32ed72b ]--- [ 1692.139064] BUG: scheduling while atomic: swapper/0/0x10000100 [ 1692.144866] Modules linked in: kvm_intel kvm bridge stp loop snd_pcm snd_timer snd soundcore snd_page_alloc tpm_tis i7core_edac psmouse ghes tpm evdev edac_core pcspkr serio_raw processor tpm_bios button dcdbas thermal_sys hed power_meter ext2 mbcache dm_mod raid1 md_mod sd_mod crc_t10dif usb_storage uas uhci_hcd mpt2sas scsi_transport_sas raid_class ehci_hcd igb scsi_mod usbcore dca bnx2 [last unloaded: scsi_wait_scan] [ 1692.182294] CPU 0 [ 1692.184119] Modules linked in: kvm_intel kvm bridge stp loop snd_pcm snd_timer snd soundcore snd_page_alloc tpm_tis i7core_edac psmouse ghes tpm evdev edac_core pcspkr serio_raw processor tpm_bios button dcdbas thermal_sys hed power_meter ext2 mbcache dm_mod raid1 md_mod sd_mod crc_t10dif usb_storage uas uhci_hcd mpt2sas scsi_transport_sas raid_class ehci_hcd igb scsi_mod usbcore dca bnx2 [last unloaded: scsi_wait_scan] [ 1692.221718] [ 1692.223199] Pid: 0, comm: swapper Tainted: G D 2.6.39-rc2+ #3 Dell Inc. PowerEdge R510/0DPRKF [ 1692.232487] RIP: 0010:[] [] arch_local_irq_enable+0x7/0x8 [processor] [ 1692.242105] RSP: 0018:ffffffff81601eb0 EFLAGS: 00000292 [ 1692.247389] RAX: 000000000003fce5 RBX: ffffffff81061348 RCX: 00000000000003e8 [ 1692.254489] RDX: 00000000000000c5 RSI: 0000000225c17d03 RDI: 000000000f93df4d [ 1692.261588] RBP: ffff880405349800 R08: 00000000fffffffd R09: 0000000000000000 [ 1692.268689] R10: ffff88042f210ac0 R11: 0000000000000040 R12: ffffffff8132cace [ 1692.275790] R13: ffff88042f20feb0 R14: ffffffff811a2ed2 R15: ffff88042f20fdc8 [ 1692.282892] FS: 0000000000000000(0000) GS:ffff88042f200000(0000) knlGS:0000000000000000 [ 1692.290943] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 1692.296660] CR2: 00000000000000cc CR3: 0000000001603000 CR4: 00000000000006f0 [ 1692.303759] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1692.310860] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1692.317959] Process swapper (pid: 0, threadinfo ffffffff81600000, task ffffffff8160b020) [ 1692.326010] Stack: [ 1692.328008] ffffffffa032cfdf 0000000000011140 ffffffff8125df49 000000010005525f [ 1692.335406] ffff880405349820 ffff8804053498f0 0000000000000002 ffffffffffffffff [ 1692.342803] ffffffff8125d059 0000000000000000 ffffffff81600000 ffffffff816812d0 [ 1692.350197] Call Trace: [ 1692.352630] [] ? acpi_idle_enter_bm+0x218/0x250 [processor] [ 1692.359818] [] ? menu_select+0x169/0x296 [ 1692.365362] [] ? cpuidle_idle_call+0xf4/0x17e [ 1692.371339] [] ? cpu_idle+0xa2/0xc4 [ 1692.376452] [] ? start_kernel+0x3b9/0x3c4 [ 1692.382084] [] ? x86_64_start_kernel+0x102/0x10f [ 1692.388319] Code: 63 1c fb 48 83 c4 38 89 e8 5b 5d 41 5c 41 5d 41 5e 41 5f c3 0f 09 0f 1f 44 00 00 c3 fa 66 0f 1f 44 00 00 c3 fb 66 0f 1f 44 00 00 48 8b 15 81 20 3f e1 48 8d 42 fd 48 83 f8 01 0f 96 c0 48 ff [ 1692.407673] Call Trace: [ 1692.410105] [] ? acpi_idle_enter_bm+0x218/0x250 [processor] [ 1692.417294] [] ? menu_select+0x169/0x296 [ 1692.422838] [] ? cpuidle_idle_call+0xf4/0x17e [ 1692.428815] [] ? cpu_idle+0xa2/0xc4 [ 1692.433928] [] ? start_kernel+0x3b9/0x3c4 [ 1692.439560] [] ? x86_64_start_kernel+0x102/0x10f ^M ^GMessage from[ 1692.446160] BUG: unable to handle kernel NULL pointer dereference at 00000000000000e0 [ 1692.455313] IP: [] igb_poll+0x3b4/0x9ee [igb] [ 1692.461214] PGD 404c2e067 PUD 402966067 PMD 0 [ 1692.465660] Oops: 0000 [#2] SMP [ 1692.468887] last sysfs file: /sys/devices/virtual/misc/kvm/uevent [ 1692.474947] CPU 0 [ 1692.476772] Modules linked in: kvm_intel kvm bridge stp loop snd_pcm snd_timer snd soundcore snd_page_alloc tpm_tis i7core_edac psmouse ghes tpm evdev edac_core pcspkr serio_raw processor tpm_bios button dcdbas thermal_sys hed power_meter ext2 mbcache dm_mod raid1 md_mod sd_mod crc_t10dif usb_storage uas uhci_hcd mpt2sas scsi_transport_sas raid_class ehci_hcd igb scsi_mod usbcore dca bnx2 [last unloaded: scsi_wait_scan] [ 1692.514362] [ 1692.515843] Pid: 1740, comm: rsyslogd Tainted: G D 2.6.39-rc2+ #3 Dell Inc. PowerEdge R510/0DPRKF [ 1692.525475] RIP: 0010:[] [] igb_poll+0x3b4/0x9ee [igb] [ 1692.533796] RSP: 0018:ffff880403081b90 EFLAGS: 00010203 [ 1692.539078] RAX: ffff880404f24e58 RBX: ffff880404f25a40 RCX: 0000000000000000 [ 1692.546176] RDX: 0000000000000040 RSI: 0000000000001043 RDI: ffff880404f24e58 [ 1692.553273] RBP: 0000000000000000 R08: 00000000f352249a R09: ffff88042f20f470 [ 1692.560373] R10: 0000000000000000 R11: 0000000000000293 R12: 000000000000000c [ 1692.567470] R13: ffffc9001267b320 R14: 0000000000000000 R15: ffff880403ff3140 [ 1692.574571] FS: 00007f4ce614f700(0000) GS:ffff88042f200000(0000) knlGS:0000000000000000 [ 1692.582619] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 1692.588334] CR2: 00000000000000e0 CR3: 0000000403c96000 CR4: 00000000000006f0 [ 1692.595434] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1692.602532] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1692.609630] Process rsyslogd (pid: 1740, threadinfo ffff880403080000, task ffff880413b8a170) [ 1692.618025] Stack: [ 1692.620024] 0000000000000000 ffffffff8105b633 0000000000000082 ffffffff8103eb92 [ 1692.627419] ffffffff8132b45a 0000000000000082 0000000000000000 000010431d887e00 [ 1692.634813] 0000000100000015 ffff880404f24e40 ffff880400000000 ffff8804038e8740 [ 1692.642208] Call Trace: [ 1692.644640] [] ? wq_worker_waking_up+0x8/0x20 [ 1692.650615] [] ? try_to_wake_up+0x16a/0x17c [ 1692.656418] [] ? schedule+0x56e/0x585 [ 1692.661703] [] ? _raw_spin_lock_irq+0xd/0x1a [ 1692.667592] [] ? net_rx_action+0xa4/0x1b1 [ 1692.673221] [] ? __do_softirq+0xb8/0x176 [ 1692.678765] [] ? call_softirq+0x1c/0x30 [ 1692.684224] [] ? do_softirq+0x3f/0x84 [ 1692.689507] [] ? irq_exit+0x3f/0x8f [ 1692.694618] [] ? do_IRQ+0x85/0x9e [ 1692.699556] [] ? common_interrupt+0x13/0x13 [ 1692.705360] [] ? kmsg_poll+0x3a/0x3a [ 1692.710559] [] ? spin_unlock_irq.clone.1+0xe/0x10 [ 1692.716882] [] ? do_syslog+0x1e2/0x430 [ 1692.722253] [] ? kmsg_read+0x46/0x50 [ 1692.727451] [] ? proc_reg_read+0x6f/0x88 [ 1692.732995] [] ? vfs_read+0x9f/0xf2 [ 1692.738106] [] ? sys_read+0x45/0x6b [ 1692.743218] [] ? system_call_fastpath+0x16/0x1b [ 1692.749364] Code: 89 74 24 3c e9 78 03 00 00 8b 54 24 70 39 54 24 44 0f 8d 75 03 00 00 ff 44 24 44 0f ae e8 49 8b 6d 00 ff 44 24 40 b9 00 00 00 00 [ 1692.762725] 8b 85 e0 00 00 00 49 c7 45 00 00 00 00 00 41 8b 77 0c 0f 18 [ 1692.769844] RIP [] igb_poll+0x3b4/0x9ee [igb] [ 1692.775829] RSP [ 1692.779295] CR2: 00000000000000e0 syslogd@r510-1 [ 1692.782628] ---[ end trace 0afda543b32ed72c ]--- [ 1692.788575] BUG: scheduling while atomic: rsyslogd/1740/0x10000100 at Apr 8 20:34:[ 1692.794754] Modules linked in: kvm_intel04 ...^M kernel kvm bridge:[ 1691.697131] stp loopOops: 0000 [#1] snd_pcm snd_timerSMP ^M^M ^GMessa snd soundcorege from syslogd@ snd_page_alloc tpm_tisr510-1 at Apr 8 i7core_edac psmouse 20:34:04 ...^M ghes tpm kernel:[ 1691.7 evdev edac_core00357] last sysf pcspkr serio_raws file: /sys/dev processor tpm_biosices/virtual/mis button dcdbasc/kvm/uevent ^M^M thermal_sys hed ^GMessage from power_meter ext2syslogd@r510-1 a mbcache dm_modt Apr 8 20:34:0 raid1 md_mod4 ...^M kernel: sd_mod crc_t10dif[ 1691.848303] S usb_storage uastack: ^M^M ^GMess uhci_hcd mpt2sasage from syslogd scsi_transport_sas raid_class@r510-1 at Apr ehci_hcd igb8 20:34:04 ...^M^M scsi_mod usbcore kernel:[ 1691. dca bnx2872480] Call Tra [last unloaded: scsi_wait_scan] ce: ^M^M ^GMessag[ 1692.865388] CPU 0 [ 1692.868564] Modules linked in:e from syslogd@r kvm_intel kvm510-1 at Apr 8 bridge stp20:34:04 ...^M loop snd_pcmkernel:[ 1691.87 snd_timer snd4910] ^M soundcore snd_page_alloc tpm_tis i7core_edac psmouse ghes tpm evdev edac_core pcspkr serio_raw processor tpm_bios button dcdbas thermal_sys hed power_meter ext2 mbcache dm_mod raid1 md_mod sd_mod crc_t10dif usb_storage uas uhci_hcd mpt2sas scsi_transport_sas raid_class ehci_hcd igb scsi_mod usbcore dca bnx2 [last unloaded: scsi_wait_scan] [ 1692.913092] [ 1692.914574] Pid: 1740, comm: rsyslogd Tainted: G D 2.6.39-rc2+ #3 Dell Inc. PowerEdge R510/0DPRKF [ 1692.924208] RIP: 0010:[] [] spin_unlock_irq.clone.1+0xe/0x10 [ 1692.933051] RSP: 0018:ffff880403081e30 EFLAGS: 00000286 [ 1692.938336] RAX: 000000000001455c RBX: ffff880403afecc0 RCX: ffffffff817518a0 [ 1692.945435] RDX: 0000000000014520 RSI: 0000000000000003 RDI: ffffffff81751888 [ 1692.952536] RBP: 0000000000000fff R08: ffffffff811439a0 R09: 0000000000000000 [ 1692.959636] R10: 0000000000000000 R11: 0000000000000293 R12: ffffffff8132cad3 [ 1692.966737] R13: 0000000000000e33 R14: 00007f4ce7983693 R15: ffff880403081da8 [ 1692.973837] FS: 00007f4ce614f700(0000) GS:ffff88042f200000(0000) knlGS:0000000000000000 [ 1692.981890] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 1692.987607] CR2: 00000000000000e0 CR3: 0000000403c96000 CR4: 00000000000006f0 [ 1692.994706] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1693.001805] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1693.008907] Process rsyslogd (pid: 1740, threadinfo ffff880403080000, task ffff880413b8a170) [ 1693.017303] Stack: [ 1693.019301] ffffffff81045e01 0000000003afed20 ffffffff00000020 0000000000000000 [ 1693.026696] 0000000000000001 ffff880403081fd8 0000000000000000 0002000003081e60 [ 1693.034093] ffff880404f9b090 0000000100000000 00007f4ce7982860 0000000000000fff [ 1693.041490] Call Trace: [ 1693.043922] [] ? do_syslog+0x1e2/0x430 [ 1693.049295] [] ? kmsg_read+0x46/0x50 [ 1693.054493] [] ? proc_reg_read+0x6f/0x88 [ 1693.060039] [] ? vfs_read+0x9f/0xf2 [ 1693.065153] [] ? sys_read+0x45/0x6b [ 1693.070266] [] ? system_call_fastpath+0x16/0x1b [ 1693.076415] Code: 00 00 75 14 c7 05 48 be 72 00 01 00 00 00 c7 05 42 be 72 00 01 00 00 00 48 83 c4 08 c3 66 ff 05 ea bc 70 00 fb 66 0f 1f 44 00 00 c3 48 83 ec 08 48 c7 c2 a8 38 61 81 48 c7 c6 3e 2b 4c 81 48 [ 1693.095772] Call Trace: [ 1693.098205] [] ? do_syslog+0x1e2/0x430 [ 1693.103579] [] ? kmsg_read+0x46/0x50 [ 1693.108778] [] ? proc_reg_read+0x6f/0x88 [ 1693.114323] [] ? vfs_read+0x9f/0xf2 [ 1693.119435] [] ? sys_read+0x45/0x6b [ 1693.124547] [] ? system_call_fastpath+0x16/0x1b [ 1693.130704] Kernel panic - not syncing: Fatal exception in interrupt [ 1693.137028] Pid: 1740, comm: rsyslogd Tainted: G D 2.6.39-rc2+ #3 [ 1693.143870] Call Trace: [ 1693.146304] [] ? panic+0x92/0x1a1 [ 1693.151244] [] ? oops_end+0xa9/0xb6 [ 1693.156359] [] ? no_context+0x1ed/0x1fa [ 1693.161819] [] ? do_page_fault+0x16b/0x308 [ 1693.167537] [] ? page_fault+0x25/0x30 [ 1693.172824] [] ? igb_poll+0x3b4/0x9ee [igb] [ 1693.178629] [] ? wq_worker_waking_up+0x8/0x20 [ 1693.184606] [] ? try_to_wake_up+0x16a/0x17c [ 1693.190410] [] ? schedule+0x56e/0x585 [ 1693.195697] [] ? _raw_spin_lock_irq+0xd/0x1a [ 1693.201588] [] ? net_rx_action+0xa4/0x1b1 [ 1693.207219] [] ? __do_softirq+0xb8/0x176 [ 1693.212764] [] ? call_softirq+0x1c/0x30 [ 1693.218222] [] ? do_softirq+0x3f/0x84 [ 1693.223507] [] ? irq_exit+0x3f/0x8f [ 1693.228619] [] ? do_IRQ+0x85/0x9e [ 1693.233560] [] ? common_interrupt+0x13/0x13 [ 1693.239365] [] ? kmsg_poll+0x3a/0x3a [ 1693.244564] [] ? spin_unlock_irq.clone.1+0xe/0x10 [ 1693.250888] [] ? do_syslog+0x1e2/0x430 [ 1693.256259] [] ? kmsg_read+0x46/0x50 [ 1693.261459] [] ? proc_reg_read+0x6f/0x88 [ 1693.267005] [] ? vfs_read+0x9f/0xf2 [ 1693.272117] [] ? sys_read+0x45/0x6b [ 1693.277231] [] ? system_call_fastpath+0x16/0x1b