Problem with 8922ae

Problem with 8922ae

[Larry Finger]

Ping-Ke,

I have been busy with the 8821au driver and had not done anything with the 
8922ae driver in the meantime. Now, I am getting some warhings in the logs:

[ 8783.319543] [  T839] rtw89_8922ae 0000:02:00.0: FW status = 0xb6001220
[ 8783.319552] [  T839] rtw89_8922ae 0000:02:00.0: FW BADADDR = 0x0
[ 8783.319557] [  T839] rtw89_8922ae 0000:02:00.0: FW EPC/RA = 0x20136928
[ 8783.319562] [  T839] rtw89_8922ae 0000:02:00.0: FW MISC = 0x0
[ 8783.319566] [  T839] rtw89_8922ae 0000:02:00.0: R_AX_HALT_C2H = 0x30000007
[ 8783.319571] [  T839] rtw89_8922ae 0000:02:00.0: R_AX_SER_DBG_INFO = 0x0
[ 8783.319575] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8783.319597] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8783.319612] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8783.319627] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8783.319641] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8783.319656] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8783.319671] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8783.319686] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8783.319700] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8783.319715] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8783.319730] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8783.319745] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8783.319759] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8783.319774] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8783.319789] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8783.319800] [  T839] rtw89_8922ae 0000:02:00.0: SER catches error: 0x4000
[ 8783.326029] [T23006] ieee80211 phy1: Hardware restart was requested
[ 8867.101365] [  T839] rtw89_8922ae 0000:02:00.0: FW status = 0xdd001220
[ 8867.101375] [  T839] rtw89_8922ae 0000:02:00.0: FW BADADDR = 0x0
[ 8867.101381] [  T839] rtw89_8922ae 0000:02:00.0: FW EPC/RA = 0x20136928
[ 8867.101386] [  T839] rtw89_8922ae 0000:02:00.0: FW MISC = 0x0
[ 8867.101391] [  T839] rtw89_8922ae 0000:02:00.0: R_AX_HALT_C2H = 0x30000007
[ 8867.101396] [  T839] rtw89_8922ae 0000:02:00.0: R_AX_SER_DBG_INFO = 0x0
[ 8867.101401] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8867.101423] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8867.101439] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8867.101455] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8867.101470] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8867.101485] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8867.101500] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8867.101515] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8867.101529] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8867.101545] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8867.101560] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8867.101575] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8867.101590] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8867.101605] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8867.101620] [  T839] rtw89_8922ae 0000:02:00.0: [ERR]fw PC = 0x2001b780
[ 8867.101632] [  T839] rtw89_8922ae 0000:02:00.0: SER catches error: 0x4000
[ 8867.108241] [T23006] ieee80211 phy1: Hardware restart was requested
[ 8867.108256] [T22691] ------------[ cut here ]------------
[ 8867.108257] [T22691] ieee80211_restart_work called with hardware scan in progress
[ 8867.108285] [T22691] WARNING: CPU: 1 PID: 22691 at net/mac80211/main.c:467 
ieee80211_restart_work+0x136/0x140 [mac80211]
[ 8867.108353] [T22691] Modules linked in: vboxnetadp(OE) vboxnetflt(OE) 
vboxdrv(OE) rfcomm(E) snd_seq_dummy(E) snd_hrtimer(E) snd_seq(E) 
snd_seq_device(E) ccm(E) rpcsec_gss_krb5(E) auth_rpcgss(E) nfsv4(E) 
dns_resolver(E) nfs(E) lockd(E) grace(E) sunrpc(E) af_packet(E) cmac(E) 
algif_hash(E) algif_skcipher(E) af_alg(E) qrtr(E) bnep(E) btusb(E) btrtl(E) 
btintel(E) btbcm(E) btmtk(E) bluetooth(E) ecdh_generic(E) binfmt_misc(E) 
nls_iso8859_1(E) nls_cp437(E) rtw89_8922ae(E) vfat(E) rtw89_8922a(E) fat(E) 
rtw89_pci(E) snd_hda_codec_realtek(E) iwlmvm(E) snd_hda_codec_generic(E) 
rtw89_core(E) intel_rapl_msr(E) snd_hda_scodec_component(E) intel_rapl_common(E) 
x86_pkg_temp_thermal(E) iTCO_wdt(E) intel_powerclamp(E) spi_intel_platform(E) 
snd_hda_codec_hdmi(E) mac80211(E) mei_hdcp(E) spi_intel(E) mei_pxp(E) 
coretemp(E) intel_pmc_bxt(E) iTCO_vendor_support(E) snd_hda_intel(E) 
kvm_intel(E) snd_intel_dspcfg(E) iwlwifi(E) snd_hda_codec(E) snd_hwdep(E) 
snd_hda_core(E) libarc4(E) kvm(E) pcspkr(E) cfg80211(E) snd_pcm(E) wmi_bmof(E) 
toshiba_acpi(E)
[ 8867.108409] [T22691]  snd_timer(E) e1000e(E) lpc_ich(E) mei_me(E) 
sparse_keymap(E) snd(E) mei(E) soundcore(E) toshiba_bluetooth(E) industrialio(E) 
thermal(E) tiny_power_button(E) tpm_infineon(E) battery(E) button(E) ac(E) 
joydev(E) fuse(E) loop(E) efi_pstore(E) dm_mod(E) configfs(E) dmi_sysfs(E) 
ext4(E) mbcache(E) jbd2(E) crct10dif_pclmul(E) crc32_pclmul(E) crc32c_intel(E) 
polyval_clmulni(E) polyval_generic(E) ghash_clmulni_intel(E) sha512_ssse3(E) 
sha256_ssse3(E) sha1_ssse3(E) i915(E) i2c_algo_bit(E) drm_buddy(E) 
drm_display_helper(E) drm_kms_helper(E) sr_mod(E) cec(E) xhci_pci(E) 
xhci_pci_renesas(E) aesni_intel(E) video(E) rtsx_pci_sdmmc(E) mmc_core(E) 
crypto_simd(E) cryptd(E) cdrom(E) ehci_pci(E) ehci_hcd(E) ttm(E) xhci_hcd(E) 
wmi(E) drm(E) rtsx_pci(E) usbcore(E) serio_raw(E) msr(E)
[ 8867.108461] [T22691] Unloaded tainted modules: acpi_cpufreq(E):1 
ie31200_edac(E):1
[ 8867.108466] [T22691] CPU: 1 PID: 22691 Comm: kworker/1:0 Tainted: G 
OE      6.9.0-rc6-rtw-gdfddbd1faef0 #2 22389bc84921f116568819a110243f5d34dca438
[ 8867.108470] [T22691] Hardware name: TOSHIBA TECRA A50-A/TECRA A50-A, BIOS 
Version 4.50   09/29/2014
[ 8867.108472] [T22691] Workqueue: events_freezable ieee80211_restart_work 
[mac80211]
[ 8867.108521] [T22691] RIP: 0010:ieee80211_restart_work+0x136/0x140 [mac80211]
[ 8867.108566] [T22691] Code: 48 8b bd b8 ea ff ff e8 18 33 c6 ff 5b 5d 41 5c 41 
5d e9 cd a4 37 d3 48 c7 c6 d0 5d 28 c1 48 c7 c7 30 e9 28 c1 e8 4a 16 b0 d2 <0f> 
0b e9 12 ff ff ff 0f 1f 00 0f 1f 44 00 00 53 48 89 fb 48 81 c7
[ 8867.108568] [T22691] RSP: 0018:ffffa68008177e58 EFLAGS: 00010282
[ 8867.108570] [T22691] RAX: 0000000000000000 RBX: ffff915aae0bf3c0 RCX: 
0000000000000027
[ 8867.108572] [T22691] RDX: ffff915b232a3808 RSI: 0000000000000001 RDI: 
ffff915b232a3800
[ 8867.108573] [T22691] RBP: ffff915907b01e88 R08: 0000000000000000 R09: 
ffffa68008177d08
[ 8867.108575] [T22691] R10: ffffa68008177d00 R11: 0000000000000003 R12: 
ffff915b232b5800
[ 8867.108576] [T22691] R13: ffff915907b00900 R14: ffff915907b01e88 R15: 
0000000000000000
[ 8867.108578] [T22691] FS:  0000000000000000(0000) GS:ffff915b23280000(0000) 
knlGS:0000000000000000
[ 8867.108580] [T22691] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8867.108582] [T22691] CR2: 00007f4f247054f8 CR3: 0000000066235006 CR4: 
00000000001706f0
[ 8867.108584] [T22691] Call Trace:
[ 8867.108586] [T22691]  <TASK>
[ 8867.108587] [T22691]  ? ieee80211_restart_work+0x136/0x140 [mac80211 
8391c747405e6378a75b7e6b3ede48d8663d2f79]
[ 8867.108629] [T22691]  ? __warn.cold+0xa8/0x102
[ 8867.108634] [T22691]  ? ieee80211_restart_work+0x136/0x140 [mac80211 
8391c747405e6378a75b7e6b3ede48d8663d2f79]
[ 8867.108675] [T22691]  ? report_bug+0xd0/0x140
[ 8867.108680] [T22691]  ? handle_bug+0x38/0x70
[ 8867.108684] [T22691]  ? exc_invalid_op+0x13/0x60
[ 8867.108687] [T22691]  ? asm_exc_invalid_op+0x16/0x20
[ 8867.108691] [T22691]  ? ieee80211_restart_work+0x136/0x140 [mac80211 
8391c747405e6378a75b7e6b3ede48d8663d2f79]
[ 8867.108734] [T22691]  ? ieee80211_restart_work+0x136/0x140 [mac80211 
8391c747405e6378a75b7e6b3ede48d8663d2f79]
[ 8867.108778] [T22691]  process_one_work+0x17a/0x330
[ 8867.108781] [T22691]  worker_thread+0x2e9/0x400
[ 8867.108784] [T22691]  ? rescuer_thread+0x3e0/0x3e0
[ 8867.108786] [T22691]  kthread+0xcb/0x100
[ 8867.108790] [T22691]  ? kthread_complete_and_exit+0x20/0x20
[ 8867.108793] [T22691]  ret_from_fork+0x2d/0x50
[ 8867.108796] [T22691]  ? kthread_complete_and_exit+0x20/0x20
[ 8867.108799] [T22691]  ret_from_fork_asm+0x11/0x20
[ 8867.108804] [T22691]  </TASK>
[ 8867.108805] [T22691] ---[ end trace 0000000000000000 ]---

This warning came while running a kernel generated from rtw-next with commit 
28818b4d871bc93.

Thanks,

Larry

RE: Problem with 8922ae

[Ping-Ke Shih]

Larry Finger <larry.finger@gmail.com> wrote:
> 
> I have been busy with the 8821au driver and had not done anything with the
> 8922ae driver in the meantime. Now, I am getting some warhings in the logs:
> 
> This warning came while running a kernel generated from rtw-next with commit
> 28818b4d871bc93.
> 

The log said SER occurs. I will check this internally.

Question: How frequent you met this warning? and is there a method you can
always reproduce this symptom?

Ping-Ke

Re: Problem with 8922ae

[Larry Finger]

On 5/15/24 7:40 PM, Ping-Ke Shih wrote:
> Larry Finger <larry.finger@gmail.com> wrote:
>>
>> I have been busy with the 8821au driver and had not done anything with the
>> 8922ae driver in the meantime. Now, I am getting some warhings in the logs:
>>
>> This warning came while running a kernel generated from rtw-next with commit
>> 28818b4d871bc93.
>>
> 
> The log said SER occurs. I will check this internally.
> 
> Question: How frequent you met this warning? and is there a method you can
> always reproduce this symptom?

I have had 43 of them in about 7 hours of running time. Unfortunately, I do not 
know how to reproduce them.

I will try to bisect the problem as they were not present earlier.

Larry

RE: Problem with 8922ae

[Ping-Ke Shih]

Larry Finger <larry.finger@gmail.com> wrote:
> On 5/15/24 7:40 PM, Ping-Ke Shih wrote:
> > Larry Finger <larry.finger@gmail.com> wrote:
> >>
> >> I have been busy with the 8821au driver and had not done anything with the
> >> 8922ae driver in the meantime. Now, I am getting some warhings in the logs:
> >>
> >> This warning came while running a kernel generated from rtw-next with commit
> >> 28818b4d871bc93.
> >>
> >
> > The log said SER occurs. I will check this internally.
> >
> > Question: How frequent you met this warning? and is there a method you can
> > always reproduce this symptom?
> 
> I have had 43 of them in about 7 hours of running time. Unfortunately, I do not
> know how to reproduce them.
> 
> I will try to bisect the problem as they were not present earlier.

Recently we added WoWLAN and BT-coexistence for 8922AE. I think you didn't use
WoWLAN features, so cause might be BT-coexistence. Maybe another try is to turn
off BT if you are using. 

I will setup a PC to see if I can reproduce this problem.

Ping-Ke 

Re: Problem with 8922ae

[Larry Finger]

On 5/15/24 8:16 PM, Ping-Ke Shih wrote:
> Recently we added WoWLAN and BT-coexistence for 8922AE. I think you didn't use
> WoWLAN features, so cause might be BT-coexistence. Maybe another try is to turn
> off BT if you are using.
> 
> I will setup a PC to see if I can reproduce this problem.
> 

Ping-Ke,

I am not using BT. I unloaded the BT drivers, and still got the warning.

Larry

RE: Problem with 8922ae

[Ping-Ke Shih]

Larry Finger <larry.finger@gmail.com> wrote:
> On 5/15/24 8:16 PM, Ping-Ke Shih wrote:
> > Recently we added WoWLAN and BT-coexistence for 8922AE. I think you didn't use
> > WoWLAN features, so cause might be BT-coexistence. Maybe another try is to turn
> > off BT if you are using.
> >
> > I will setup a PC to see if I can reproduce this problem.
> >
> 
> Ping-Ke,
> 
> I am not using BT. I unloaded the BT drivers, and still got the warning.
> 

Thanks for the test. Please attach warnings you met to me. I will pass them
to internal experts to analyze. 

In my setup, which connects to 2GHz AP and plays youtube, I still can't see
warning for about 1 day. I will keep test over weekend.

Ping-Ke