From: Larry Finger <Larry.Finger@lwfinger.net>
To: Arjan van de Ven <arjan@linux.intel.com>
Cc: Joseph Jezak <josejx@gentoo.org>, netdev@vger.kernel.org
Subject: Re: [patch] work around/fix deadlock in the bcm43xx driver by making netlink irq safe
Date: Sat, 08 Jul 2006 12:59:14 -0500 [thread overview]
Message-ID: <44AFF272.8000109@lwfinger.net> (raw)
In-Reply-To: <44A53918.2020700@linux.intel.com>
Arjan van de Ven wrote:
> Joseph Jezak wrote:
>> Can you provide the details to the list? I'll look into getting
>> SoftMAC fixed if you do.
>>
>
> sure
> the basic issue is that bcm43xx does it's rx processing in a softirq,
> and holds the bcm->irq_lock during that time. The rx processing calls
> into the softmac layer, which in turn calls into netlink.
>
> With this you can get a deadlock that looks like this
> cpu 0: user context |cpu1: softirq context
> netlink_table_grab takes nl_table_lock as |take bcm->irq_lock in
> write_lock_bh, but leaves irqs enabled |bcm43xx_interrupt_tasklet()
> |which then in a few steps
> |leads to a call to
> |bcm43xx_rx
>
>
> hardirq comes in and the isr tries to take |in bcm43xx_rx, call
> bcm->irq_lock but has to wait on cpu 1 |ieee80211_rx_mgt which
> |leads to a call to
> |wireless_send_event which
> |tries to take nl_table_lock
> |for read but has to wait
> |for cpu0
>
> according to Michael Buesch, the softmac layer should queue the packet
> internally for another softirq, similar to what DeviceScape does, so
> that the rx softirq can just drop all packets quickly and drop its locks.
I think the deadlock dump shown below is related; however, since I have a uniprocessor system and
the deadlock is not exactly the same, I'll include it here. This is using v2.6.18-rc1 from Linus's tree.
kernel: -> (af_callback_keys + sk->sk_family#2){-.-?} ops: 431 {
kernel: initial-use at:
kernel: [<c0135d48>] lock_acquire+0x68/0x90
kernel: [<c030c195>] _read_lock+0x45/0x60
kernel: [<c02a786f>] sock_def_readable+0x1f/0x90
kernel: [<c02bf072>] netlink_broadcast+0x282/0x320
kernel: [<c01ef236>] kobject_uevent+0x366/0x4c0
kernel: [<c01eed08>] kobject_register+0x48/0x60
kernel: [<c013ce89>] sys_init_module+0x1439/0x1870
kernel: [<c01031cd>] sysenter_past_esp+0x56/0x8d
kernel: hardirq-on-W at:
kernel: [<c0135d48>] lock_acquire+0x68/0x90
kernel: [<c030c37a>] _write_lock_bh+0x4a/0x60
kernel: [<c02beba3>] netlink_release+0xe3/0x330
kernel: [<c02a439d>] sock_release+0x1d/0xf0
kernel: [<c02a44a7>] sock_close+0x37/0x60
kernel: [<c0163688>] __fput+0xd8/0x210
kernel: [<c01637d8>] fput+0x18/0x20
kernel: [<c0160574>] filp_close+0x54/0x80
kernel: [<c011a5df>] put_files_struct+0x7f/0xd0
kernel: [<c011b6cc>] do_exit+0x12c/0x9a0
kernel: [<c011bf7d>] do_group_exit+0x3d/0xa0
kernel: [<c011bff5>] sys_exit_group+0x15/0x20
kernel: [<c01031cd>] sysenter_past_esp+0x56/0x8d
kernel: in-softirq-R at:
kernel: [<c0135d48>] lock_acquire+0x68/0x90
kernel: [<c030c195>] _read_lock+0x45/0x60
kernel: [<c02a786f>] sock_def_readable+0x1f/0x90
kernel: [<c02bf072>] netlink_broadcast+0x282/0x320
kernel: [<c02bb6e4>] wireless_send_event+0x244/0x3b0
kernel: [<e4a2c586>] ieee80211softmac_call_events_locked+0x86/0x140
[ieee80211softmac]
kernel: [<e4a2c674>] ieee80211softmac_call_events+0x34/0x6f [ieee80211softmac]
kernel: [<e4a28faf>] ieee80211softmac_auth_resp+0x19f/0x620 [ieee80211softmac]
kernel: [<e4a1e413>] ieee80211_rx_mgt+0x543/0x810 [ieee80211]
kernel: [<e4a7ea2b>] bcm43xx_rx+0x34b/0x980 [bcm43xx]
kernel: [<e4a820bc>] bcm43xx_dma_rx+0x23c/0x550 [bcm43xx]
kernel: [<e4a6751e>] bcm43xx_interrupt_tasklet+0x38e/0x970 [bcm43xx]
kernel: [<c011e4de>] tasklet_action+0x4e/0x90
kernel: [<c011ecc2>] __do_softirq+0x62/0xe0
kernel: [<c01055cb>] do_softirq+0x9b/0xf0
kernel: softirq-on-R at:
kernel: [<c0135d48>] lock_acquire+0x68/0x90
kernel: [<c030c195>] _read_lock+0x45/0x60
kernel: [<c02a786f>] sock_def_readable+0x1f/0x90
kernel: [<c02bf072>] netlink_broadcast+0x282/0x320
kernel: [<c01ef236>] kobject_uevent+0x366/0x4c0
kernel: [<c01eed08>] kobject_register+0x48/0x60
kernel: [<c013ce89>] sys_init_module+0x1439/0x1870
kernel: [<c01031cd>] sysenter_past_esp+0x56/0x8d
kernel: hardirq-on-R at:
kernel: [<c0135d48>] lock_acquire+0x68/0x90
kernel: [<c030c195>] _read_lock+0x45/0x60
kernel: [<c02a786f>] sock_def_readable+0x1f/0x90
kernel: [<c02bf072>] netlink_broadcast+0x282/0x320
kernel: [<c01ef236>] kobject_uevent+0x366/0x4c0
kernel: [<c01eed08>] kobject_register+0x48/0x60
kernel: [<c013ce89>] sys_init_module+0x1439/0x1870
kernel: [<c01031cd>] sysenter_past_esp+0x56/0x8d
kernel: }
kernel ... key at: [<c05bad60>] af_callback_keys+0x80/0x100
kernel:
kernel: stack backtrace:
kernel: [<c0103d1d>] show_trace_log_lvl+0x13d/0x160
kernel: [<c010525b>] show_trace+0x1b/0x20
kernel: [<c0105286>] dump_stack+0x26/0x30
kernel: [<c0133f7d>] check_usage+0x26d/0x280
kernel: [<c013536f>] __lock_acquire+0x77f/0xdd0
kernel: [<c0135d48>] lock_acquire+0x68/0x90
kernel: [<c030c195>] _read_lock+0x45/0x60
kernel: [<c02a786f>] sock_def_readable+0x1f/0x90
kernel: [<c02bf072>] netlink_broadcast+0x282/0x320
kernel: [<c02bb6e4>] wireless_send_event+0x244/0x3b0
kernel: [<e4a2c586>] ieee80211softmac_call_events_locked+0x86/0x140 [ieee80211softmac]
kernel: [<e4a2c674>] ieee80211softmac_call_events+0x34/0x6f [ieee80211softmac]
kernel: [<e4a28faf>] ieee80211softmac_auth_resp+0x19f/0x620 [ieee80211softmac]
kernel: [<e4a1e413>] ieee80211_rx_mgt+0x543/0x810 [ieee80211]
kernel: [<e4a7ea2b>] bcm43xx_rx+0x34b/0x980 [bcm43xx]
kernel: [<e4a820bc>] bcm43xx_dma_rx+0x23c/0x550 [bcm43xx]
kernel: [<e4a6751e>] bcm43xx_interrupt_tasklet+0x38e/0x970 [bcm43xx]
kernel: [<c011e4de>] tasklet_action+0x4e/0x90
kernel: [<c011ecc2>] __do_softirq+0x62/0xe0
kernel: [<c01055cb>] do_softirq+0x9b/0xf0
kernel: [<c01056d1>] do_IRQ+0xb1/0x110
kernel: [<c0103439>] common_interrupt+0x25/0x2c
kernel: [<c015e01e>] kmem_cache_free+0x6e/0xa0
kernel: [<c019631d>] proc_destroy_inode+0x1d/0x20
kernel: [<c017d7eb>] destroy_inode+0x2b/0x60
kernel: [<c017e753>] generic_delete_inode+0xb3/0x100
kernel: [<c017d8fd>] iput+0x6d/0x80
kernel: [<c017b79b>] dentry_iput+0x7b/0xd0
kernel: [<c017bee4>] dput+0x84/0x190
kernel: [<c0172194>] path_release+0x14/0x30
kernel: [<c017295a>] __link_path_walk+0x3ea/0xef0
kernel: [<c01734b4>] link_path_walk+0x54/0xf0
kernel: [<c017394e>] do_path_lookup+0xae/0x260
kernel: [<c017403a>] __path_lookup_intent_open+0x4a/0x90
kernel: [<c017410a>] path_lookup_open+0x2a/0x30
kernel: [<c01743a7>] open_namei+0x77/0x6d0
kernel: [<c0161898>] do_filp_open+0x38/0x60
kernel: [<c016190b>] do_sys_open+0x4b/0x100
kernel: [<c0161a17>] sys_open+0x27/0x30
kernel: [<c01031cd>] sysenter_past_esp+0x56/0x8d
kernel: [<b7fb9410>] 0xb7fb9410
kernel: SoftMAC: sent association request!
kernel: SoftMAC: associated!
kernel: SoftMAC: Scanning finished
So far, this situation has only occurred during the initial association/authorization steps during
bootup.
Larry
next prev parent reply other threads:[~2006-07-08 17:59 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2006-06-30 14:24 [patch] work around/fix deadlock in the bcm43xx driver by making netlink irq safe Arjan van de Ven
[not found] ` <44A536BE.6020209@gentoo.org>
2006-06-30 14:45 ` Arjan van de Ven
2006-07-08 17:59 ` Larry Finger [this message]
2006-07-08 18:32 ` Michael Buesch
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=44AFF272.8000109@lwfinger.net \
--to=larry.finger@lwfinger.net \
--cc=arjan@linux.intel.com \
--cc=josejx@gentoo.org \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).