rt61pci current wireless testing git gives kernel oops

rt61pci current wireless testing git gives kernel oops

[Bas Hulsken]

Hi Ivo,

I recently obtained an rt61pci NIC, and in an attempt to find out
whether my recent hostapd problems were rt2500pci specific (remember
that?) I plugged it into the server.

the interface goes up well, but I get an oops as soon as I run hostapd,
seems something with debugfs. This shows up in dmesg, let me know if you
need any more info.

best regards,
Bas

attached: dmesg, modinfo, lspci

dmesg -----------------------------------------------------------------
Unable to handle kernel NULL pointer dereference at 0000000000000218
RIP: 
 [<ffffffff8824bd9d>] :mac80211:ieee80211_debugfs_key_add_default
+0x24/0x55
PGD bad8c067 PUD b702d067 PMD 0 
Oops: 0000 [1] SMP 
CPU 2 
Modules linked in: nfsd lockd nfs_acl auth_rpcgss exportfs autofs4 fuse
rfcomm l2cap sunrpc rt2500pci(U) ipt_REJECT xt_multiport iptable_filter
ipt_MASQUERADE ipt_REDIRECT iptable_nat nf_nat nf_conntrack_ipv4 ipt_TOS
iptable_mangle ip_tables nf_conntrack_ipv6 xt_state nf_conntrack
xt_tcpudp ip6t_ipv6header ip6t_REJECT ip6table_filter ip6_tables
x_tables ipv6 cpufreq_ondemand acpi_cpufreq ext2 dm_mirror dm_multipath
dm_mod wm8775 cx25840 msp3400 arc4 ecb blkcipher snd_hda_intel saa7115
snd_seq_dummy snd_seq_oss snd_seq_midi_event tuner snd_seq rt61pci(U)
rt2x00pci(U) snd_seq_device tea5767 tda8290 tuner_simple rt2x00lib(U)
mt20xx snd_pcm_oss tea5761 snd_mixer_oss snd_pcm rfkill input_polldev
rtl8187(U) mac80211(U) ivtv snd_timer snd_page_alloc snd_hwdep
cfg80211(U) i2c_algo_bit snd i2c_i801 cx2341x tveeprom videodev
firewire_ohci firewire_core crc_itu_t r8169 eeprom_93cx6(U) v4l2_common
v4l1_compat iTCO_wdt hci_usb sr_mod lirc_atiusb button i2c_core pcspkr
soundcore lirc_dev iTCO_vendor_support bluetooth sky2 cdrom sg floppy
ahci pata_jmicron ata_generic ata_piix pata_acpi libata sd_mod scsi_mod
raid456 async_xor async_memcpy async_tx xor ext3 jbd mbcache uhci_hcd
ohci_hcd ehci_hcd
Pid: 17, comm: events/2 Not tainted 2.6.24.4-64.fc8 #1
RIP: 0010:[<ffffffff8824bd9d>]
[<ffffffff8824bd9d>] :mac80211:ieee80211_debugfs_key_add_default
+0x24/0x55
RSP: 0018:ffff81012fb2de10  EFLAGS: 00010286
RAX: 0000000000000000 RBX: ffff8100bac66400 RCX: 00000000000000b6
RDX: ffff81012a443ea0 RSI: ffffffff88258eb7 RDI: ffff81012fb2de10
RBP: ffff81012a811700 R08: ffff81012fc09c30 R09: 0000000000000036
R10: ffffffff810b09b9 R11: ffffffff810fb05d R12: ffff81012fa53d40
R13: ffffffffffffffff R14: 0000000000000036 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff81012fc01b00(0000)
knlGS:0000000000000000
CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 0000000000000218 CR3: 00000000cf9fc000 CR4: 00000000000006a0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process events/2 (pid: 17, threadinfo ffff81012fb2c000, task
ffff81012fb248b0)
Stack:  0000000000000036 ffff81012a443ea0 ffff81012a443ea0
ffff81012a811700
 ffff81012dfda500 ffffffff810b479f ffff810100000000 ffff81012a811700
 ffffffff8824722f ffff8100bac66400 ffffffff8824722f ffffffff88247027
Call Trace:
 [<ffffffff810b479f>] mntput_no_expire+0x1c/0x87
 [<ffffffff8824722f>] :mac80211:key_todo+0x0/0x5
 [<ffffffff8824722f>] :mac80211:key_todo+0x0/0x5
 [<ffffffff88247027>] :mac80211:__ieee80211_key_todo+0x76/0x1ed
 [<ffffffff8824722f>] :mac80211:key_todo+0x0/0x5
 [<ffffffff88247228>] :mac80211:ieee80211_key_todo+0xe/0x15
 [<ffffffff81045625>] run_workqueue+0x7f/0x10b
 [<ffffffff81045f2d>] worker_thread+0x0/0xe4
 [<ffffffff81046007>] worker_thread+0xda/0xe4
 [<ffffffff81048f1d>] autoremove_wake_function+0x0/0x2e
 [<ffffffff81048dee>] kthread+0x47/0x75
 [<ffffffff8100cca8>] child_rip+0xa/0x12
 [<ffffffff81048da7>] kthread+0x0/0x75
 [<ffffffff8100cc9e>] child_rip+0x0/0x12


Code: 8b 90 18 02 00 00 31 c0 e8 43 c0 ed f8 48 8b b5 40 04 00 00 
RIP  [<ffffffff8824bd9d>] :mac80211:ieee80211_debugfs_key_add_default
+0x24/0x55
 RSP <ffff81012fb2de10>
CR2: 0000000000000218
---[ end trace 5bf48e6dc8139959 ]---

modinfo-----------------------------------------------------------------
filename:       /lib/modules/2.6.24.4-64.fc8/updates/rt2x00/rt61pci.ko
license:        GPL
firmware:       rt2661.bin
firmware:       rt2561s.bin
firmware:       rt2561.bin
description:    Ralink RT61 PCI & PCMCIA Wireless LAN driver.
version:        2.1.4
author:         http://rt2x00.serialmonkey.com
srcversion:     FCC52B48B3074E83A77D602
alias:          pci:v00001814d00000401sv*sd*bc*sc*i*
alias:          pci:v00001814d00000302sv*sd*bc*sc*i*
alias:          pci:v00001814d00000301sv*sd*bc*sc*i*
depends:        rt2x00lib,rt2x00pci,crc-itu-t,eeprom_93cx6
vermagic:       2.6.24.4-64.fc8 SMP mod_unload 

lspci-----------------------------------------------------------------
05:01.0 Network controller: RaLink RT2561/RT61 802.11g PCI
        Subsystem: Linksys WMP54G ver 4.1
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop-
ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=slow >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 64, Cache Line Size: 32 bytes
        Interrupt: pin A routed to IRQ 17
        Region 0: Memory at febf8000 (32-bit, non-prefetchable)
[size=32K]
        Capabilities: [40] Power Management version 2
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 PME-Enable- DSel=0 DScale=0 PME-
        Kernel driver in use: rt61pci
        Kernel modules: rt61pci

Re: rt61pci current wireless testing git gives kernel oops

[Ivo van Doorn]

Hi,

> I recently obtained an rt61pci NIC, and in an attempt to find out
> whether my recent hostapd problems were rt2500pci specific (remember
> that?) I plugged it into the server.

Yeah, I know the issue, I still have it on my todo list but haven't had much
time to take a real close look at it.

> the interface goes up well, but I get an oops as soon as I run hostapd,
> seems something with debugfs. This shows up in dmesg, let me know if you
> need any more info.

Johannes, this issue might have been introduced in your patch
"[PATCH] mac80211: fix key vs. sta locking problems"
do you know about this issue?

Ivo

> dmesg -----------------------------------------------------------------
> Unable to handle kernel NULL pointer dereference at 0000000000000218
> RIP: 
>  [<ffffffff8824bd9d>] :mac80211:ieee80211_debugfs_key_add_default
> +0x24/0x55
> PGD bad8c067 PUD b702d067 PMD 0 
> Oops: 0000 [1] SMP 
> CPU 2 
> Modules linked in: nfsd lockd nfs_acl auth_rpcgss exportfs autofs4 fuse
> rfcomm l2cap sunrpc rt2500pci(U) ipt_REJECT xt_multiport iptable_filter
> ipt_MASQUERADE ipt_REDIRECT iptable_nat nf_nat nf_conntrack_ipv4 ipt_TOS
> iptable_mangle ip_tables nf_conntrack_ipv6 xt_state nf_conntrack
> xt_tcpudp ip6t_ipv6header ip6t_REJECT ip6table_filter ip6_tables
> x_tables ipv6 cpufreq_ondemand acpi_cpufreq ext2 dm_mirror dm_multipath
> dm_mod wm8775 cx25840 msp3400 arc4 ecb blkcipher snd_hda_intel saa7115
> snd_seq_dummy snd_seq_oss snd_seq_midi_event tuner snd_seq rt61pci(U)
> rt2x00pci(U) snd_seq_device tea5767 tda8290 tuner_simple rt2x00lib(U)
> mt20xx snd_pcm_oss tea5761 snd_mixer_oss snd_pcm rfkill input_polldev
> rtl8187(U) mac80211(U) ivtv snd_timer snd_page_alloc snd_hwdep
> cfg80211(U) i2c_algo_bit snd i2c_i801 cx2341x tveeprom videodev
> firewire_ohci firewire_core crc_itu_t r8169 eeprom_93cx6(U) v4l2_common
> v4l1_compat iTCO_wdt hci_usb sr_mod lirc_atiusb button i2c_core pcspkr
> soundcore lirc_dev iTCO_vendor_support bluetooth sky2 cdrom sg floppy
> ahci pata_jmicron ata_generic ata_piix pata_acpi libata sd_mod scsi_mod
> raid456 async_xor async_memcpy async_tx xor ext3 jbd mbcache uhci_hcd
> ohci_hcd ehci_hcd
> Pid: 17, comm: events/2 Not tainted 2.6.24.4-64.fc8 #1
> RIP: 0010:[<ffffffff8824bd9d>]
> [<ffffffff8824bd9d>] :mac80211:ieee80211_debugfs_key_add_default
> +0x24/0x55
> RSP: 0018:ffff81012fb2de10  EFLAGS: 00010286
> RAX: 0000000000000000 RBX: ffff8100bac66400 RCX: 00000000000000b6
> RDX: ffff81012a443ea0 RSI: ffffffff88258eb7 RDI: ffff81012fb2de10
> RBP: ffff81012a811700 R08: ffff81012fc09c30 R09: 0000000000000036
> R10: ffffffff810b09b9 R11: ffffffff810fb05d R12: ffff81012fa53d40
> R13: ffffffffffffffff R14: 0000000000000036 R15: 0000000000000000
> FS:  0000000000000000(0000) GS:ffff81012fc01b00(0000)
> knlGS:0000000000000000
> CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
> CR2: 0000000000000218 CR3: 00000000cf9fc000 CR4: 00000000000006a0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Process events/2 (pid: 17, threadinfo ffff81012fb2c000, task
> ffff81012fb248b0)
> Stack:  0000000000000036 ffff81012a443ea0 ffff81012a443ea0
> ffff81012a811700
>  ffff81012dfda500 ffffffff810b479f ffff810100000000 ffff81012a811700
>  ffffffff8824722f ffff8100bac66400 ffffffff8824722f ffffffff88247027
> Call Trace:
>  [<ffffffff810b479f>] mntput_no_expire+0x1c/0x87
>  [<ffffffff8824722f>] :mac80211:key_todo+0x0/0x5
>  [<ffffffff8824722f>] :mac80211:key_todo+0x0/0x5
>  [<ffffffff88247027>] :mac80211:__ieee80211_key_todo+0x76/0x1ed
>  [<ffffffff8824722f>] :mac80211:key_todo+0x0/0x5
>  [<ffffffff88247228>] :mac80211:ieee80211_key_todo+0xe/0x15
>  [<ffffffff81045625>] run_workqueue+0x7f/0x10b
>  [<ffffffff81045f2d>] worker_thread+0x0/0xe4
>  [<ffffffff81046007>] worker_thread+0xda/0xe4
>  [<ffffffff81048f1d>] autoremove_wake_function+0x0/0x2e
>  [<ffffffff81048dee>] kthread+0x47/0x75
>  [<ffffffff8100cca8>] child_rip+0xa/0x12
>  [<ffffffff81048da7>] kthread+0x0/0x75
>  [<ffffffff8100cc9e>] child_rip+0x0/0x12
> 
> 
> Code: 8b 90 18 02 00 00 31 c0 e8 43 c0 ed f8 48 8b b5 40 04 00 00 
> RIP  [<ffffffff8824bd9d>] :mac80211:ieee80211_debugfs_key_add_default
> +0x24/0x55
>  RSP <ffff81012fb2de10>
> CR2: 0000000000000218
> ---[ end trace 5bf48e6dc8139959 ]---
> 
> modinfo-----------------------------------------------------------------
> filename:       /lib/modules/2.6.24.4-64.fc8/updates/rt2x00/rt61pci.ko
> license:        GPL
> firmware:       rt2661.bin
> firmware:       rt2561s.bin
> firmware:       rt2561.bin
> description:    Ralink RT61 PCI & PCMCIA Wireless LAN driver.
> version:        2.1.4
> author:         http://rt2x00.serialmonkey.com
> srcversion:     FCC52B48B3074E83A77D602
> alias:          pci:v00001814d00000401sv*sd*bc*sc*i*
> alias:          pci:v00001814d00000302sv*sd*bc*sc*i*
> alias:          pci:v00001814d00000301sv*sd*bc*sc*i*
> depends:        rt2x00lib,rt2x00pci,crc-itu-t,eeprom_93cx6
> vermagic:       2.6.24.4-64.fc8 SMP mod_unload 
> 
> lspci-----------------------------------------------------------------
> 05:01.0 Network controller: RaLink RT2561/RT61 802.11g PCI
>         Subsystem: Linksys WMP54G ver 4.1
>         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop-
> ParErr- Stepping- SERR- FastB2B- DisINTx-
>         Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=slow >TAbort-
> <TAbort- <MAbort- >SERR- <PERR- INTx-
>         Latency: 64, Cache Line Size: 32 bytes
>         Interrupt: pin A routed to IRQ 17
>         Region 0: Memory at febf8000 (32-bit, non-prefetchable)
> [size=32K]
>         Capabilities: [40] Power Management version 2
>                 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
> PME(D0-,D1-,D2-,D3hot-,D3cold-)
>                 Status: D0 PME-Enable- DSel=0 DScale=0 PME-
>         Kernel driver in use: rt61pci
>         Kernel modules: rt61pci
> 
> 
> 

Re: rt61pci current wireless testing git gives kernel oops

[Bas Hulsken]

Hi,

> > the interface goes up well, but I get an oops as soon as I run hostapd,
> > seems something with debugfs. This shows up in dmesg, let me know if you
> > need any more info.
> 
> Johannes, this issue might have been introduced in your patch
> "[PATCH] mac80211: fix key vs. sta locking problems"
> do you know about this issue?

might be possible, I tried with
mac80211-v2.6.25_rc9_5745_g58908b9_080418-1 (wirelesstesting cheched out
at the 18th of April) when this patch was not yet committed, and the
oops does not occur. But then again, I don't get any traffic on the
monitoring interface with this version, so it might also just not get
triggered. 

Bas

Re: rt61pci current wireless testing git gives kernel oops

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 489 bytes --]

Hi,

> > the interface goes up well, but I get an oops as soon as I run hostapd,
> > seems something with debugfs. This shows up in dmesg, let me know if you
> > need any more info.
> 
> Johannes, this issue might have been introduced in your patch
> "[PATCH] mac80211: fix key vs. sta locking problems"
> do you know about this issue?

Huh. I'll take a look but I've run hostapd before w/o problems. I might
not have had encryption enabled though, don't remember.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: rt61pci current wireless testing git gives kernel oops

[Johannes Berg]

> > the interface goes up well, but I get an oops as soon as I run hostapd,
> > seems something with debugfs. This shows up in dmesg, let me know if you
> > need any more info.

Bas, can you reproduce this issue? Could you try the patch below?

johannes

---
 net/mac80211/debugfs_key.c |   21 ++++++++++++++++++---
 1 file changed, 18 insertions(+), 3 deletions(-)

--- everything.orig/net/mac80211/debugfs_key.c	2008-04-28 20:15:42.000000000 +0200
+++ everything/net/mac80211/debugfs_key.c	2008-04-28 20:15:44.000000000 +0200
@@ -255,14 +255,29 @@ void ieee80211_debugfs_key_remove(struct
 void ieee80211_debugfs_key_add_default(struct ieee80211_sub_if_data *sdata)
 {
 	char buf[50];
+	struct ieee80211_key *key;
 
 	if (!sdata->debugfsdir)
 		return;
 
-	sprintf(buf, "../keys/%d", sdata->default_key->debugfs.cnt);
-	sdata->debugfs.default_key =
-		debugfs_create_symlink("default_key", sdata->debugfsdir, buf);
+	rcu_read_lock();
+	key = rcu_dereference(sdata->default_key);
+	if (key) {
+		sprintf(buf, "../keys/%d", key->debugfs.cnt);
+		sdata->debugfs.default_key =
+			debugfs_create_symlink("default_key",
+					       sdata->debugfsdir, buf);
+	} else {
+		/*
+		 * Oops. No default key, let's remove the debugfs entry
+		 * This can happen if the workqueue is too slow.
+		 */
+		debugfs_remove(sdata->debugfs.default_key);
+		sdata->debugfs.default_key = NULL;
+	}
+	rcu_read_unlock();
 }
+
 void ieee80211_debugfs_key_remove_default(struct ieee80211_sub_if_data *sdata)
 {
 	if (!sdata)

Re: rt61pci current wireless testing git gives kernel oops

[Bas Hulsken]

Hi,

On Mon, 2008-04-28 at 20:18 +0200, Johannes Berg wrote:
> > > the interface goes up well, but I get an oops as soon as I run hostapd,
> > > seems something with debugfs. This shows up in dmesg, let me know if you
> > > need any more info.
> 
> Bas, can you reproduce this issue? Could you try the patch below?
> 
> johannes

with your patch applied I can no longer reproduce this issue, and I've
tried running hostapd at least 5 times. Without your patch, I could
reproduce it two out of two times.

thanks,
Bas

Re: rt61pci current wireless testing git gives kernel oops

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 701 bytes --]

On Wed, 2008-04-30 at 11:45 +0200, Bas Hulsken wrote:
> Hi,
> 
> On Mon, 2008-04-28 at 20:18 +0200, Johannes Berg wrote:
> > > > the interface goes up well, but I get an oops as soon as I run hostapd,
> > > > seems something with debugfs. This shows up in dmesg, let me know if you
> > > > need any more info.
> > 
> > Bas, can you reproduce this issue? Could you try the patch below?
> > 
> > johannes
> 
> with your patch applied I can no longer reproduce this issue, and I've
> tried running hostapd at least 5 times. Without your patch, I could
> reproduce it two out of two times.

Thanks for testing. That patch causes another problem but now I know
where to look.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 828 bytes --]