From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dave Jones Subject: Re: tc related lockdep warning. Date: Tue, 26 Sep 2006 17:20:34 -0400 Message-ID: <20060926212034.GA3134@redhat.com> References: <20060925124352.GA1592@ff.dom.local> <1159188473.5301.68.camel@jzny2> <4517D9A6.70307@trash.net> <45195219.7050105@trash.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: hadi@cyberus.ca, Jarek Poplawski , netdev@vger.kernel.org, davem@davemloft.net Return-path: Received: from mx1.redhat.com ([66.187.233.31]:27058 "EHLO mx1.redhat.com") by vger.kernel.org with ESMTP id S932419AbWIZVVA (ORCPT ); Tue, 26 Sep 2006 17:21:00 -0400 To: Patrick McHardy Content-Disposition: inline In-Reply-To: <45195219.7050105@trash.net> Sender: netdev-owner@vger.kernel.org List-Id: netdev.vger.kernel.org On Tue, Sep 26, 2006 at 06:15:21PM +0200, Patrick McHardy wrote: > Patrick McHardy wrote: > > jamal wrote: > > > >>Yes, that looks plausible. Can you try making those changes and see if > >>the warning is gone? > > > > > > I think this points to a bigger brokeness caused by the move of > > dev->qdisc to RCU. It means destruction of filters and actions doesn't > > necessarily happens in user-context and thus not protected by the rtnl > > anymore. > > I looked into this and we indeed still have lots of problems from that > broken RCU patch. Basically all locking (qdiscs, classifiers, actions, > estimators) assumes that updates are only done in process context and > thus read_lock doesn't need bottem half protection. Quite a few things > also assume that updates only happen under the RTNL and don't need > any further protection if not used during packet processing. > > Instead of "fixing" all this I suggest something like this (untested) > patch instead. Since only the dev->qdisc pointer is protected by RCU, > but enqueue and the qdisc tree are still protected by dev->qdisc_lock, > we can perform destruction of the tree immediately and only do the > final free in the rcu callback, as long as we make sure not to enqueue > anything to a half-way destroyed qdisc. With this patch, I get no lockdep warnings, but the machine locks up completely. I hooked up a serial console, and found this.. u32 classifier Performance counters on input device check on Actions configured BUG: warning at net/sched/sch_htb.c:395/htb_safe_rb_erase() Call Trace: [] show_trace+0xae/0x336 [] dump_stack+0x15/0x17 [] :sch_htb:htb_safe_rb_erase+0x3b/0x55 [] :sch_htb:htb_deactivate_prios+0x173/0x1cd [] :sch_htb:htb_dequeue+0x4d0/0x856 [] __qdisc_run+0x3f/0x1ca [] dev_queue_xmit+0x137/0x268 [] neigh_resolve_output+0x249/0x27e [] ip_output+0x210/0x25a [] ip_push_pending_frames+0x37c/0x45b [] icmp_push_reply+0x13b/0x148 [] icmp_send+0x366/0x3d3 [] udp_rcv+0x53d/0x556 [] ip_local_deliver+0x1a3/0x26b [] ip_rcv+0x4b9/0x501 [] netif_receive_skb+0x33d/0x3c9 [] :e1000:e1000_clean_rx_irq+0x450/0x4fe [] :e1000:e1000_clean+0x88/0x17d [] net_rx_action+0xac/0x1d1 [] __do_softirq+0x68/0xf5 [] call_softirq+0x1c/0x28 DWARF2 unwinder stuck at call_softirq+0x1c/0x28 Leftover inexact backtrace: [] do_softirq+0x39/0x9f [] irq_exit+0x57/0x59 [] do_IRQ+0xfd/0x107 [] mwait_idle+0x0/0x54 [] ret_from_intr+0x0/0xf [] __sched_text_start+0xaa6/0xadd [] mwait_idle+0x3f/0x54 [] mwait_idle+0x9/0x54 [] cpu_idle+0xa2/0xc5 [] rest_init+0x2b/0x2d [] start_kernel+0x24a/0x24c [] _sinittext+0x28b/0x292 BUG: warning at net/sched/sch_htb.c:395/htb_safe_rb_erase() Call Trace: [] show_trace+0xae/0x336 [] dump_stack+0x15/0x17 [] :sch_htb:htb_safe_rb_erase+0x3b/0x55 [] :sch_htb:htb_deactivate_prios+0x173/0x1cd [] :sch_htb:htb_dequeue+0x4d0/0x856 [] __qdisc_run+0x3f/0x1ca [] dev_queue_xmit+0x137/0x268 [] neigh_resolve_output+0x249/0x27e [] ip_output+0x210/0x25a [] ip_push_pending_frames+0x37c/0x45b [] icmp_push_reply+0x13b/0x148 [] icmp_send+0x366/0x3d3 [] udp_rcv+0x53d/0x556 [] ip_local_deliver+0x1a3/0x26b [] ip_rcv+0x4b9/0x501 [] netif_receive_skb+0x33d/0x3c9 [] :e1000:e1000_clean_rx_irq+0x450/0x4fe [] :e1000:e1000_clean+0x88/0x17d [] net_rx_action+0xac/0x1d1 [] __do_softirq+0x68/0xf5 [] call_softirq+0x1c/0x28 DWARF2 unwinder stuck at call_softirq+0x1c/0x28 Leftover inexact backtrace: [] do_softirq+0x39/0x9f [] irq_exit+0x57/0x59 [] do_IRQ+0xfd/0x107 [] mwait_idle+0x0/0x54 [] ret_from_intr+0x0/0xf [] __sched_text_start+0xaa6/0xadd [] mwait_idle+0x3f/0x54 [] mwait_idle+0x9/0x54 [] cpu_idle+0xa2/0xc5 [] rest_init+0x2b/0x2d [] start_kernel+0x24a/0x24c [] _sinittext+0x28b/0x292 BUG: soft lockup detected on CPU#0! Call Trace: [] show_trace+0xae/0x336 [] dump_stack+0x15/0x17 [] softlockup_tick+0xd5/0xea [] run_local_timers+0x13/0x15 [] update_process_times+0x4c/0x79 [] smp_local_timer_interrupt+0x2b/0x50 [] smp_apic_timer_interrupt+0x58/0x62 [] apic_timer_interrupt+0x6b/0x70 DWARF2 unwinder stuck at apic_timer_interrupt+0x6b/0x70 Leftover inexact backtrace: [] dev_queue_xmit+0x107/0x268 [] dev_queue_xmit+0x107/0x268 [] __rb_rotate_left+0x5f/0x60 [] rb_insert_color+0xb9/0xe3 [] :sch_htb:htb_add_to_id_tree+0x64/0x66 [] :sch_htb:htb_activate_prios+0xd6/0xe7 [] :sch_htb:htb_enqueue+0x139/0x194 [] dev_queue_xmit+0x119/0x268 [] neigh_resolve_output+0x249/0x27e [] ip_output+0x210/0x25a [] ip_push_pending_frames+0x37c/0x45b [] icmp_push_reply+0x13b/0x148 [] icmp_send+0x366/0x3d3 [] udp_rcv+0x53d/0x556 [] ip_local_deliver+0x1a3/0x26b [] ip_rcv+0x4b9/0x501 [] netif_receive_skb+0x33d/0x3c9 [] :e1000:e1000_clean_rx_irq+0x450/0x4fe [] :e1000:e1000_clean+0x88/0x17d [] net_rx_action+0xac/0x1d1 [] __do_softirq+0x54/0xf5 [] __do_softirq+0x68/0xf5 [] call_softirq+0x1c/0x28 [] do_softirq+0x39/0x9f [] irq_exit+0x57/0x59 [] do_IRQ+0xfd/0x107 [] mwait_idle+0x0/0x54 [] ret_from_intr+0x0/0xf [] __sched_text_start+0xaa6/0xadd [] mwait_idle+0x3f/0x54 [] mwait_idle+0x9/0x54 [] cpu_idle+0xa2/0xc5 [] rest_init+0x2b/0x2d [] start_kernel+0x24a/0x24c [] _sinittext+0x28b/0x292 BUG: soft lockup detected on CPU#3! Call Trace: [] show_trace+0xae/0x336 [] dump_stack+0x15/0x17 [] softlockup_tick+0xd5/0xea [] run_local_timers+0x13/0x15 [] update_process_times+0x4c/0x79 [] smp_local_timer_interrupt+0x2b/0x50 [] smp_apic_timer_interrupt+0x58/0x62 [] apic_timer_interrupt+0x6b/0x70 DWARF2 unwinder stuck at apic_timer_interrupt+0x6b/0x70 Leftover inexact backtrace: [] :sch_htb:htb_rate_timer+0x21/0xd8 [] :sch_htb:htb_rate_timer+0x21/0xd8 [] __delay+0xa/0x15 [] _raw_spin_lock+0x8c/0x104 [] :sch_htb:htb_rate_timer+0x21/0xd8 [] _spin_lock_bh+0x32/0x36 [] :sch_htb:htb_rate_timer+0x21/0xd8 [] run_timer_softirq+0x14c/0x1d5 [] __do_softirq+0x68/0xf5 [] call_softirq+0x1c/0x28 [] do_softirq+0x39/0x9f [] irq_exit+0x57/0x59 [] smp_apic_timer_interrupt+0x5d/0x62 [] mwait_idle+0x0/0x54 [] apic_timer_interrupt+0x6b/0x70 [] __sched_text_start+0xaa6/0xadd [] mwait_idle+0x3f/0x54 [] mwait_idle+0x9/0x54 [] cpu_idle+0xa2/0xc5 [] start_secondary+0x468/0x477 BUG: soft lockup detected on CPU#0! Call Trace: [] show_trace+0xae/0x336 [] dump_stack+0x15/0x17 [] softlockup_tick+0xd5/0xea [] run_local_timers+0x13/0x15 [] update_process_times+0x4c/0x79 [] smp_local_timer_interrupt+0x2b/0x50 [] smp_apic_timer_interrupt+0x58/0x62 [] apic_timer_interrupt+0x6b/0x70 DWARF2 unwinder stuck at apic_timer_interrupt+0x6b/0x70 Leftover inexact backtrace: [] dev_queue_xmit+0x107/0x268 [] dev_queue_xmit+0x107/0x268 [] __rb_rotate_left+0x3d/0x60 [] rb_insert_color+0xb9/0xe3 [] :sch_htb:htb_add_to_id_tree+0x64/0x66 [] :sch_htb:htb_activate_prios+0xd6/0xe7 [] :sch_htb:htb_enqueue+0x139/0x194 [] dev_queue_xmit+0x119/0x268 [] neigh_resolve_output+0x249/0x27e [] ip_output+0x210/0x25a [] ip_push_pending_frames+0x37c/0x45b [] icmp_push_reply+0x13b/0x148 [] icmp_send+0x366/0x3d3 [] udp_rcv+0x53d/0x556 [] ip_local_deliver+0x1a3/0x26b [] ip_rcv+0x4b9/0x501 [] netif_receive_skb+0x33d/0x3c9 [] :e1000:e1000_clean_rx_irq+0x450/0x4fe [] :e1000:e1000_clean+0x88/0x17d [] net_rx_action+0xac/0x1d1 [] __do_softirq+0x54/0xf5 [] __do_softirq+0x68/0xf5 [] call_softirq+0x1c/0x28 [] do_softirq+0x39/0x9f [] irq_exit+0x57/0x59 [] do_IRQ+0xfd/0x107 [] mwait_idle+0x0/0x54 [] ret_from_intr+0x0/0xf [] __sched_text_start+0xaa6/0xadd [] mwait_idle+0x3f/0x54 [] mwait_idle+0x9/0x54 [] cpu_idle+0xa2/0xc5 [] rest_init+0x2b/0x2d [] start_kernel+0x24a/0x24c [] _sinittext+0x28b/0x292 BUG: soft lockup detected on CPU#3! Call Trace: [] show_trace+0xae/0x336 [] dump_stack+0x15/0x17 [] softlockup_tick+0xd5/0xea [] run_local_timers+0x13/0x15 [] update_process_times+0x4c/0x79 [] smp_local_timer_interrupt+0x2b/0x50 [] smp_apic_timer_interrupt+0x58/0x62 [] apic_timer_interrupt+0x6b/0x70 DWARF2 unwinder stuck at apic_timer_interrupt+0x6b/0x70 Leftover inexact backtrace: [] :sch_htb:htb_rate_timer+0x21/0xd8 [] :sch_htb:htb_rate_timer+0x21/0xd8 [] __delay+0xa/0x15 [] _raw_spin_lock+0x8c/0x104 [] :sch_htb:htb_rate_timer+0x21/0xd8 [] _spin_lock_bh+0x32/0x36 [] :sch_htb:htb_rate_timer+0x21/0xd8 [] run_timer_softirq+0x14c/0x1d5 [] __do_softirq+0x68/0xf5 [] call_softirq+0x1c/0x28 [] do_softirq+0x39/0x9f [] irq_exit+0x57/0x59 [] smp_apic_timer_interrupt+0x5d/0x62 [] mwait_idle+0x0/0x54 [] apic_timer_interrupt+0x6b/0x70 [] __sched_text_start+0xaa6/0xadd [] mwait_idle+0x3f/0x54 [] mwait_idle+0x9/0x54 [] cpu_idle+0xa2/0xc5 [] start_secondary+0x468/0x477 BUG: soft lockup detected on CPU#0! Call Trace: [] show_trace+0xae/0x336 [] dump_stack+0x15/0x17 [] softlockup_tick+0xd5/0xea [] run_local_timers+0x13/0x15 [] update_process_times+0x4c/0x79 [] smp_local_timer_interrupt+0x2b/0x50 [] smp_apic_timer_interrupt+0x58/0x62 [] apic_timer_interrupt+0x6b/0x70 DWARF2 unwinder stuck at apic_timer_interrupt+0x6b/0x70 Leftover inexact backtrace: [] dev_queue_xmit+0x107/0x268 [] dev_queue_xmit+0x107/0x268 [] __rb_rotate_right+0x41/0x60 [] rb_insert_color+0x68/0xe3 [] :sch_htb:htb_add_to_id_tree+0x64/0x66 [] :sch_htb:htb_activate_prios+0xd6/0xe7 [] :sch_htb:htb_enqueue+0x139/0x194 [] dev_queue_xmit+0x119/0x268 [] neigh_resolve_output+0x249/0x27e [] ip_output+0x210/0x25a [] ip_push_pending_frames+0x37c/0x45b [] icmp_push_reply+0x13b/0x148 [] icmp_send+0x366/0x3d3 [] udp_rcv+0x53d/0x556 [] ip_local_deliver+0x1a3/0x26b [] ip_rcv+0x4b9/0x501 [] netif_receive_skb+0x33d/0x3c9 [] :e1000:e1000_clean_rx_irq+0x450/0x4fe [] :e1000:e1000_clean+0x88/0x17d [] net_rx_action+0xac/0x1d1 [] __do_softirq+0x54/0xf5 [] __do_softirq+0x68/0xf5 [] call_softirq+0x1c/0x28 [] do_softirq+0x39/0x9f [] irq_exit+0x57/0x59 [] do_IRQ+0xfd/0x107 [] mwait_idle+0x0/0x54 [] ret_from_intr+0x0/0xf [] __sched_text_start+0xaa6/0xadd [] mwait_idle+0x3f/0x54 [] mwait_idle+0x9/0x54 [] cpu_idle+0xa2/0xc5 [] rest_init+0x2b/0x2d [] start_kernel+0x24a/0x24c [] _sinittext+0x28b/0x292 BUG: soft lockup detected on CPU#3! Call Trace: [] show_trace+0xae/0x336 [] dump_stack+0x15/0x17 [] softlockup_tick+0xd5/0xea [] run_local_timers+0x13/0x15 [] update_process_times+0x4c/0x79 [] smp_local_timer_interrupt+0x2b/0x50 [] smp_apic_timer_interrupt+0x58/0x62 [] apic_timer_interrupt+0x6b/0x70 DWARF2 unwinder stuck at apic_timer_interrupt+0x6b/0x70 Leftover inexact backtrace: [] :sch_htb:htb_rate_timer+0x21/0xd8 [] :sch_htb:htb_rate_timer+0x21/0xd8 [] __delay+0x6/0x15 [] _raw_spin_lock+0x8c/0x104 [] :sch_htb:htb_rate_timer+0x21/0xd8 [] _spin_lock_bh+0x32/0x36 [] :sch_htb:htb_rate_timer+0x21/0xd8 [] run_timer_softirq+0x14c/0x1d5 [] __do_softirq+0x68/0xf5 [] call_softirq+0x1c/0x28 [] do_softirq+0x39/0x9f [] irq_exit+0x57/0x59 [] smp_apic_timer_interrupt+0x5d/0x62 [] mwait_idle+0x0/0x54 [] apic_timer_interrupt+0x6b/0x70 [] __sched_text_start+0xaa6/0xadd [] mwait_idle+0x3f/0x54 [] mwait_idle+0x9/0x54 [] cpu_idle+0xa2/0xc5 [] start_secondary+0x468/0x477 BUG: soft lockup detected on CPU#0! Call Trace: [] show_trace+0xae/0x336 [] dump_stack+0x15/0x17 [] softlockup_tick+0xd5/0xea [] run_local_timers+0x13/0x15 [] update_process_times+0x4c/0x79 [] smp_local_timer_interrupt+0x2b/0x50 [] smp_apic_timer_interrupt+0x58/0x62 [] apic_timer_interrupt+0x6b/0x70 DWARF2 unwinder stuck at apic_timer_interrupt+0x6b/0x70 Leftover inexact backtrace: [] dev_queue_xmit+0x107/0x268 [] dev_queue_xmit+0x107/0x268 [] rb_insert_color+0x68/0xe3 [] rb_insert_color+0x68/0xe3 [] :sch_htb:htb_add_to_id_tree+0x64/0x66 [] :sch_htb:htb_activate_prios+0xd6/0xe7 [] :sch_htb:htb_enqueue+0x139/0x194 [] dev_queue_xmit+0x119/0x268 [] neigh_resolve_output+0x249/0x27e [] ip_output+0x210/0x25a [] ip_push_pending_frames+0x37c/0x45b [] icmp_push_reply+0x13b/0x148 [] icmp_send+0x366/0x3d3 [] udp_rcv+0x53d/0x556 [] ip_local_deliver+0x1a3/0x26b [] ip_rcv+0x4b9/0x501 [] netif_receive_skb+0x33d/0x3c9 [] :e1000:e1000_clean_rx_irq+0x450/0x4fe [] :e1000:e1000_clean+0x88/0x17d [] net_rx_action+0xac/0x1d1 [] __do_softirq+0x54/0xf5 [] __do_softirq+0x68/0xf5 [] call_softirq+0x1c/0x28 [] do_softirq+0x39/0x9f [] irq_exit+0x57/0x59 [] do_IRQ+0xfd/0x107 [] mwait_idle+0x0/0x54 [] ret_from_intr+0x0/0xf [] __sched_text_start+0xaa6/0xadd [] mwait_idle+0x3f/0x54 [] mwait_idle+0x9/0x54 [] cpu_idle+0xa2/0xc5 [] rest_init+0x2b/0x2d [] start_kernel+0x24a/0x24c [] _sinittext+0x28b/0x292 BUG: soft lockup detected on CPU#3! Call Trace: [] show_trace+0xae/0x336 [] dump_stack+0x15/0x17 [] softlockup_tick+0xd5/0xea [] run_local_timers+0x13/0x15 [] update_process_times+0x4c/0x79 [] smp_local_timer_interrupt+0x2b/0x50 [] smp_apic_timer_interrupt+0x58/0x62 [] apic_timer_interrupt+0x6b/0x70 DWARF2 unwinder stuck at apic_timer_interrupt+0x6b/0x70 Leftover inexact backtrace: [] :sch_htb:htb_rate_timer+0x21/0xd8 [] :sch_htb:htb_rate_timer+0x21/0xd8 [] __delay+0x6/0x15 [] _raw_spin_lock+0x8c/0x104 [] :sch_htb:htb_rate_timer+0x21/0xd8 [] _spin_lock_bh+0x32/0x36 [] :sch_htb:htb_rate_timer+0x21/0xd8 [] run_timer_softirq+0x14c/0x1d5 [] __do_softirq+0x68/0xf5 [] call_softirq+0x1c/0x28 [] do_softirq+0x39/0x9f [] irq_exit+0x57/0x59 [] smp_apic_timer_interrupt+0x5d/0x62 [] mwait_idle+0x0/0x54 [] apic_timer_interrupt+0x6b/0x70 [] __sched_text_start+0xaa6/0xadd [] mwait_idle+0x3f/0x54 [] mwait_idle+0x9/0x54 [] cpu_idle+0xa2/0xc5 [] start_secondary+0x468/0x477 BUG: soft lockup detected on CPU#0! Call Trace: [] show_trace+0xae/0x336 [] dump_stack+0x15/0x17 [] softlockup_tick+0xd5/0xea [] run_local_timers+0x13/0x15 [] update_process_times+0x4c/0x79 [] smp_local_timer_interrupt+0x2b/0x50 [] smp_apic_timer_interrupt+0x58/0x62 [] apic_timer_interrupt+0x6b/0x70 DWARF2 unwinder stuck at apic_timer_interrupt+0x6b/0x70 Leftover inexact backtrace: [] dev_queue_xmit+0x107/0x268 [] dev_queue_xmit+0x107/0x268 [] rb_insert_color+0xc3/0xe3 [] rb_insert_color+0x68/0xe3 [] :sch_htb:htb_add_to_id_tree+0x64/0x66 [] :sch_htb:htb_activate_prios+0xd6/0xe7 [] :sch_htb:htb_enqueue+0x139/0x194 [] dev_queue_xmit+0x119/0x268 [] neigh_resolve_output+0x249/0x27e [] ip_output+0x210/0x25a [] ip_push_pending_frames+0x37c/0x45b [] icmp_push_reply+0x13b/0x148 [] icmp_send+0x366/0x3d3 [] udp_rcv+0x53d/0x556 [] ip_local_deliver+0x1a3/0x26b [] ip_rcv+0x4b9/0x501 [] netif_receive_skb+0x33d/0x3c9 [] :e1000:e1000_clean_rx_irq+0x450/0x4fe [] :e1000:e1000_clean+0x88/0x17d [] net_rx_action+0xac/0x1d1 [] __do_softirq+0x54/0xf5 [] __do_softirq+0x68/0xf5 [] call_softirq+0x1c/0x28 [] do_softirq+0x39/0x9f [] irq_exit+0x57/0x59 [] do_IRQ+0xfd/0x107 [] mwait_idle+0x0/0x54 [] ret_from_intr+0x0/0xf [] __sched_text_start+0xaa6/0xadd [] mwait_idle+0x3f/0x54 [] mwait_idle+0x9/0x54 [] cpu_idle+0xa2/0xc5 [] rest_init+0x2b/0x2d [] start_kernel+0x24a/0x24c [] _sinittext+0x28b/0x292 BUG: soft lockup detected on CPU#3! Call Trace: [] show_trace+0xae/0x336 [] dump_stack+0x15/0x17 [] softlockup_tick+0xd5/0xea [] run_local_timers+0x13/0x15 [] update_process_times+0x4c/0x79 [] smp_local_timer_interrupt+0x2b/0x50 [] smp_apic_timer_interrupt+0x58/0x62 [] apic_timer_interrupt+0x6b/0x70 DWARF2 unwinder stuck at apic_timer_interrupt+0x6b/0x70 Leftover inexact backtrace: [] :sch_htb:htb_rate_timer+0x21/0xd8 [] :sch_htb:htb_rate_timer+0x21/0xd8 [] __delay+0xa/0x15 [] _raw_spin_lock+0x8c/0x104 [] :sch_htb:htb_rate_timer+0x21/0xd8 [] _spin_lock_bh+0x32/0x36 [] :sch_htb:htb_rate_timer+0x21/0xd8 [] run_timer_softirq+0x14c/0x1d5 [] __do_softirq+0x68/0xf5 [] call_softirq+0x1c/0x28 [] do_softirq+0x39/0x9f [] irq_exit+0x57/0x59 [] smp_apic_timer_interrupt+0x5d/0x62 [] mwait_idle+0x0/0x54 [] apic_timer_interrupt+0x6b/0x70 [] __sched_text_start+0xaa6/0xadd [] mwait_idle+0x3f/0x54 [] mwait_idle+0x9/0x54 [] cpu_idle+0xa2/0xc5 [] start_secondary+0x468/0x477 BUG: soft lockup detected on CPU#1! Call Trace: [] show_trace+0xae/0x336 [] dump_stack+0x15/0x17 [] softlockup_tick+0xd5/0xea [] run_local_timers+0x13/0x15 [] update_process_times+0x4c/0x79 [] smp_local_timer_interrupt+0x2b/0x50 [] smp_apic_timer_interrupt+0x58/0x62 [] apic_timer_interrupt+0x6b/0x70 DWARF2 unwinder stuck at apic_timer_interrupt+0x6b/0x70 Leftover inexact backtrace: [] dev_queue_xmit+0x107/0x268 [] dev_queue_xmit+0x107/0x268 [] __delay+0xc/0x15 [] _raw_spin_lock+0x8c/0x104 [] dev_queue_xmit+0x107/0x268 [] _spin_lock+0x2d/0x31 [] dev_queue_xmit+0x107/0x268 [] ip_output+0x210/0x25a [] ip_push_pending_frames+0x37c/0x45b [] ip_generic_getfrag+0x0/0x98 [] udp_push_pending_frames+0x251/0x278 [] udp_sendmsg+0x4b8/0x5db [] inet_sendmsg+0x46/0x53 [] sock_sendmsg+0x111/0x130 [] autoremove_wake_function+0x0/0x38 [] _spin_unlock_irq+0x2b/0x31 [] trace_hardirqs_on_thunk+0x35/0x37 [] trace_hardirqs_on_thunk+0x35/0x37 [] sys_sendto+0x106/0x12d [] sys_rt_sigreturn+0x28f/0x360 [] sys_rt_sigreturn+0x32e/0x360 [] system_call+0x7e/0x83 BUG: soft lockup detected on CPU#0! Call Trace: [] show_trace+0xae/0x336 [] dump_stack+0x15/0x17 [] softlockup_tick+0xd5/0xea [] run_local_timers+0x13/0x15 [] update_process_times+0x4c/0x79 [] smp_local_timer_interrupt+0x2b/0x50 [] smp_apic_timer_interrupt+0x58/0x62 [] apic_timer_interrupt+0x6b/0x70 DWARF2 unwinder stuck at apic_timer_interrupt+0x6b/0x70 Leftover inexact backtrace: [] dev_queue_xmit+0x107/0x268 [] dev_queue_xmit+0x107/0x268 [] rb_insert_color+0x54/0xe3 [] rb_insert_color+0xb9/0xe3 [] :sch_htb:htb_add_to_id_tree+0x64/0x66 [] :sch_htb:htb_activate_prios+0xd6/0xe7 [] :sch_htb:htb_enqueue+0x139/0x194 [] dev_queue_xmit+0x119/0x268 [] neigh_resolve_output+0x249/0x27e [] ip_output+0x210/0x25a [] ip_push_pending_frames+0x37c/0x45b [] icmp_push_reply+0x13b/0x148 [] icmp_send+0x366/0x3d3 [] udp_rcv+0x53d/0x556 [] ip_local_deliver+0x1a3/0x26b [] ip_rcv+0x4b9/0x501 [] netif_receive_skb+0x33d/0x3c9 [] :e1000:e1000_clean_rx_irq+0x450/0x4fe [] :e1000:e1000_clean+0x88/0x17d [] net_rx_action+0xac/0x1d1 [] __do_softirq+0x54/0xf5 [] __do_softirq+0x68/0xf5 [] call_softirq+0x1c/0x28 [] do_softirq+0x39/0x9f [] irq_exit+0x57/0x59 [] do_IRQ+0xfd/0x107 [] mwait_idle+0x0/0x54 [] ret_from_intr+0x0/0xf [] __sched_text_start+0xaa6/0xadd [] mwait_idle+0x3f/0x54 [] mwait_idle+0x9/0x54 [] cpu_idle+0xa2/0xc5 [] rest_init+0x2b/0x2d [] start_kernel+0x24a/0x24c [] _sinittext+0x28b/0x292 BUG: soft lockup detected on CPU#3! Call Trace: [] show_trace+0xae/0x336 [] dump_stack+0x15/0x17 [] softlockup_tick+0xd5/0xea [] run_local_timers+0x13/0x15 [] update_process_times+0x4c/0x79 [] smp_local_timer_interrupt+0x2b/0x50 [] smp_apic_timer_interrupt+0x58/0x62 [] apic_timer_interrupt+0x6b/0x70 DWARF2 unwinder stuck at apic_timer_interrupt+0x6b/0x70 Leftover inexact backtrace: [] :sch_htb:htb_rate_timer+0x21/0xd8 [] :sch_htb:htb_rate_timer+0x21/0xd8 [] __delay+0xa/0x15 [] _raw_spin_lock+0x8c/0x104 [] :sch_htb:htb_rate_timer+0x21/0xd8 [] _spin_lock_bh+0x32/0x36 [] :sch_htb:htb_rate_timer+0x21/0xd8 [] run_timer_softirq+0x14c/0x1d5 [] __do_softirq+0x68/0xf5 [] call_softirq+0x1c/0x28 [] do_softirq+0x39/0x9f [] irq_exit+0x57/0x59 [] smp_apic_timer_interrupt+0x5d/0x62 [] mwait_idle+0x0/0x54 [