From mboxrd@z Thu Jan 1 00:00:00 1970 From: Chris Madden Subject: Oops in filter add Date: Mon, 19 Mar 2007 16:10:29 -0400 Message-ID: <45FEEE35.6090606@reflexsecurity.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: jamal , tgraf@suug.ch, davem@davemloft.net To: netdev@vger.kernel.org Return-path: Received: from crown.reflexsecurity.com ([72.54.139.163]:55824 "EHLO crown.reflexsecurity.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753193AbXCSUlt (ORCPT ); Mon, 19 Mar 2007 16:41:49 -0400 Sender: netdev-owner@vger.kernel.org List-Id: netdev.vger.kernel.org Hi- We're experiencing an oops when we attempt to add a filter to an ingress qdisc while heavy traffic is passing through the device being manipulated. Currently, we're using 2.6.20.3. Here is the setup: We have traffic coming in a device, say, eth1. While traffic comes in, we use tc to add an ingress queuing discipline with "tc qdisc add dev eth1 handle ffff: ingress". This works, and a "tc qdisc show" confirms that it is in place. Its hard to boil down exactly where things are going wrong, but it seems to center around adding a basic match filter to the ingress path we created above. I did some digging, and it appears the filter add isn't mutexed right. Inside net/core/dev.c, ing_filter, I see: spin_lock(&dev->ingress_lock); if ((q = dev->qdisc_ingress) != NULL) result = q->enqueue(skb, q); spin_unlock(&dev->ingress_lock); And unless I'm missing something, this is the only place this lock is used ( other than initialization ). In net/sched/cls_api.c, I see we do qdisc_lock_tree/qdisc_unlock_tree (which locks dev->queue_lock). As near as I can tell, this is our problem ( our mutexes don't prohibit manipulation while packets are flowing ). It doesn't happen every time, but if we have a script that adds/removes qdiscs and filters, and run a few hundred megabit through the interface, it dies eventually. I've been trying to puzzle out the locking myself, but I fear I've been less than successful. Oops below: [] tc_classify+0x34/0xbc [] ingress_enqueue+0x16/0x55 [sch_ingress] [] netif_receive_skb+0x1de/0x2bf [] e1000_clean_rx_irq+0x35b/0x42c [e1000] [] e1000_clean_rx_irq+0x0/0x42c [e1000] [] e1000_clean+0x6e/0x23d [e1000] [] net_rx_action+0xd5/0x1c6 [] __do_softirq+0x5d/0xba [] do_softirq+0x59/0xa9 [] basic_dump+0x0/0x115 [cls_basic] [] tcf_em_tree_dump+0x14d/0x2ff [] handle_fasteoi_irq+0x0/0xa0 [] do_IRQ+0xa1/0xb9 [] common_interrupt+0x23/0x28 [] basic_change+0x167/0x370 [cls_basic] [] tc_ctl_tfilter+0x3ec/0x469 [] tc_ctl_tfilter+0x0/0x469 [] rtnetlink_rcv_msg+0x1b3/0x1d8 [] tc_dump_qdisc+0x0/0xfe [] netlink_run_queue+0x50/0xbe [] rtnetlink_rcv_msg+0x0/0x1d8 [] rtnetlink_rcv+0x25/0x3d [] netlink_data_ready+0x12/0x4c [] netlink_sendskb+0x19/0x30 [] netlink_sendmsg+0x242/0x24e [] sock_sendmsg+0xbc/0xd4 [] autoremove_wake_function+0x0/0x35 [] autoremove_wake_function+0x0/0x35 [] verify_iovec+0x3e/0x70 [] sys_sendmsg+0x194/0x1f9 [] __wake_up+0x32/0x43 [] netlink_insert+0x106/0x110 [] netlink_autobind+0xc7/0xe3 [] netlink_bind+0x8d/0x127 [] do_wp_page+0x149/0x36c [] d_alloc+0x138/0x17a [] __handle_mm_fault+0x756/0x7a6 [] sys_socketcall+0x223/0x242 [] do_page_fault+0x0/0x525 [] syscall_call+0x7/0xb Chris Madden Reflex Security