Kernel WARNING: at net/core/dev.c:1330 __netif_schedule+0x2c/0x98()

Larry Finger
2008-07-22T16:39:54Z

David and Patrick,

Here is the latest on this problem.

I pulled from Linus's tree this morning and now have git-05752-g93ded9b. The
kernel WARNING from __netif_schedule and the lockdep warning are present with or
without the patches from yesterday.

As I stated earlier, the kernel WARNING (it was a BUG then) was introduced in
commit 37437bb2 when the BUG statement was entered.

The lockdep warning started with the next commit (16361127).

I am not using any network traffic shaping. Is it correct that the faulty
condition is not that q == &noop_qdisc, but that __netif_schedule was called
when that condition exists?

The lockdep warning is:

=============================================
[ INFO: possible recursive locking detected ]
2.6.26-Linus-git-05752-g93ded9b #49
---------------------------------------------
NetworkManager/2611 is trying to acquire lock:
(&dev->addr_list_lock){-...}, at: [<ffffffff803a2ad1>] dev_mc_sync+0x19/0x57

but task is already holding lock:
(&dev->addr_list_lock){-...}, at: [<ffffffff8039e909>] dev_set_rx_mode+0x19/0x2e

other info that might help us debug this:
2 locks held by NetworkManager/2611:
#0: (rtnl_mutex){--..}, at: [<ffffffff803a8488>] rtnetlink_rcv+0x12/0x27
#1: (&dev->addr_list_lock){-...}, at: [<ffffffff8039e909>]
dev_set_rx_mode+0x19/0x2e

stack backtrace:
Pid: 2611, comm: NetworkManager Not tainted 2.6.26-Linus-git-05752-g93ded9b #49

Call Trace:
[<ffffffff80251b02>] __lock_acquire+0xb7b/0xecc
[<ffffffff80251ea4>] lock_acquire+0x51/0x6a
[<ffffffff803a2ad1>] dev_mc_sync+0x19/0x57
[<ffffffff8040b3fc>] _spin_lock_bh+0x23/0x2c
[<ffffffff803a2ad1>] dev_mc_sync+0x19/0x57
[<ffffffff8039e911>] dev_set_rx_mode+0x21/0x2e
[<ffffffff803a04da>] dev_open+0x8e/0xb0
[<ffffffff8039fe84>] dev_change_flags+0xa6/0x163
[<ffffffff803a7591>] do_setlink+0x286/0x349
[<ffffffff803a849d>] rtnetlink_rcv_msg+0x0/0x1ec
[<ffffffff803a849d>] rtnetlink_rcv_msg+0x0/0x1ec
[<ffffffff803a849d>] rtnetlink_rcv_msg+0x0/0x1ec
[<ffffffff803a77de>] rtnl_setlink+0x10b/0x10d
[<ffffffff803a849d>] rtnetlink_rcv_msg+0x0/0x1ec
[<ffffffff803b416f>] netlink_rcv_skb+0x34/0x7d
[<ffffffff803a8497>] rtnetlink_rcv+0x21/0x27
[<ffffffff803b3c6f>] netlink_unicast+0x1f0/0x261
[<ffffffff8039a58d>] __alloc_skb+0x66/0x12a
[<ffffffff803b3f48>] netlink_sendmsg+0x268/0x27b
[<ffffffff80393cb9>] sock_sendmsg+0xcb/0xe3
[<ffffffff80246aab>] autoremove_wake_function+0x0/0x2e
[<ffffffff8039b2d8>] verify_iovec+0x46/0x82
[<ffffffff80393ee8>] sys_sendmsg+0x217/0x28a
[<ffffffff80393642>] sockfd_lookup_light+0x1a/0x52
[<ffffffff80250990>] trace_hardirqs_on_caller+0xef/0x113
[<ffffffff8040af74>] trace_hardirqs_on_thunk+0x3a/0x3f
[<ffffffff8020be9b>] system_call_after_swapgs+0x7b/0x80
========================================================

The logged data for the WARNING is as follows:

------------[ cut here ]------------
WARNING: at net/core/dev.c:1330 __netif_schedule+0x2c/0x98()
Modules linked in: af_packet nfs lockd nfs_acl rfkill_input sunrpc
cpufreq_conservative cpufreq_userspace cpufreq_powersave powernow_k8 fuse loop
dm_mod arc4 ecb crypto_blkcipher b43 firmware_class rfkill mac80211 cfg80211
led_class input_polldev battery ac button ssb serio_raw forcedeth sr_mod cdrom
k8temp hwmon sg sd_mod ehci_hcd ohci_hcd usbcore edd fan thermal processor ext3
mbcache jbd pata_amd ahci libata scsi_mod dock
Pid: 1990, comm: b43 Not tainted 2.6.26-Linus-git-05752-g93ded9b #49

Call Trace:
[<ffffffff80233f6d>] warn_on_slowpath+0x51/0x8c
[<ffffffff8039d937>] __netif_schedule+0x2c/0x98
[<ffffffffa015445d>] ieee80211_scan_completed+0x26b/0x2f1 [mac80211]
[<ffffffffa01546de>] ieee80211_sta_scan_work+0x0/0x1b8 [mac80211]
[<ffffffff8024325e>] run_workqueue+0xf0/0x1f2
[<ffffffff8024343b>] worker_thread+0xdb/0xea
[<ffffffff80246aab>] autoremove_wake_function+0x0/0x2e
[<ffffffff80243360>] worker_thread+0x0/0xea
[<ffffffff8024678b>] kthread+0x47/0x73
[<ffffffff8040af74>] trace_hardirqs_on_thunk+0x3a/0x3f
[<ffffffff8020cea9>] child_rip+0xa/0x11
[<ffffffff8020c4df>] restore_args+0x0/0x30
[<ffffffff8024671f>] kthreadd+0x188/0x1ad
[<ffffffff80246744>] kthread+0x0/0x73
[<ffffffff8020ce9f>] child_rip+0x0/0x11

---[ end trace 42d234b678daea7a ]---

Other info I have found. The call to __netif_schedule from
ieee80211_scan_completed is through the following code from
include/linux/netdevice.h:

/**
* netif_wake_queue - restart transmit
* @dev: network device
*
* Allow upper layers to call the device hard_start_xmit routine.
* Used for flow control when transmit resources are available.
*/
static inline void netif_tx_wake_queue(struct netdev_queue *dev_queue)
{
#ifdef CONFIG_NETPOLL_TRAP
if (netpoll_trap()) {
clear_bit(__QUEUE_STATE_XOFF, &dev_queue->state);
return;
}
#endif
if (test_and_clear_bit(__QUEUE_STATE_XOFF, &dev_queue->state))
__netif_schedule(dev_queue->qdisc);
}

It doesn't make any difference if CONFIG_NETPOLL_TRAP is defined or not.

Please let me know if I can provide any further information,

Larry