Persistent `iwlwifi` Queue Stuck and `rcu_preempt` Detected Stalls on AX210

Persistent `iwlwifi` Queue Stuck and `rcu_preempt` Detected Stalls on AX210

[chris]

Dear Linux Wireless Community and Gregory Greenman,

I’m writing to bring to your attention a persistent issue affecting Intel AX210 
Wi-Fi adapter users. Myself and others across different Linux distributions, 
including NixOS and Fedora, have encountered regular system crashes 
characterized by `iwlwifi` queue stuck log entries, followed by `rcu_preempt 
detected stalls`. 

From discussions on kernel bug trackers and with other users facing the same 
problem, it seems this issue might be related to the Wi-Fi adapter halting the 
PCIe bus, leading to a system-wide freeze necessitating a hard restart.

I haven't observed any specific circumstances leading to the crash which occurs 
irregularly. On Fedora Adam seems to have observed it is the downloading of 
large files that is causing it.

Below a brief summary of my system setup and error logs:

```
$ hostnamectl 
 Static hostname: nixos
       Icon name: computer-desktop
         Chassis: desktop 🖥
Operating System: NixOS 24.05 (Uakari)            
          Kernel: Linux 6.7.1
    Architecture: x86-64
 Hardware Vendor: ASUS
  Hardware Model: ROG STRIX X670E-E GAMING WIFI
Firmware Version: 1807
   Firmware Date: Thu 2023-12-14
```

```
$ journalctl -b 0 | grep "firmware version"
Jan 30 06:49:29 nixos kernel: iwlwifi 0000:08:00.0: loaded firmware version 
86.fb5c9aeb.0 ty-a0-gf-a0-86.ucode op_mode iwlmvm
```

```
$ journalctl -b -1
...
Jan 26 03:24:55 nixos kernel: iwlwifi 0000:08:00.0: Queue 4 is stuck 39108 
39109
Jan 26 03:25:00 nixos kernel: sched: RT throttling activated
Jan 26 03:25:11 nixos bluetoothd[1059]: profiles/audio/avdtp.c:cancel_request() 
Suspend: Connection timed out (110)
Jan 26 03:25:16 nixos kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/
tasks:
Jan 26 03:25:16 nixos kernel: rcu:         0-....: (8 GPs behind) idle=d6bc/
1/0x4000000000000002 softirq=3077933/3077934 fqs=5080
Jan 26 03:25:16 nixos kernel: rcu:         (detected by 3, t=21002 jiffies, 
g=14745593, q=886 ncpus=12)
```

Here error log from Adam experiencing same system crash:

```
$ egrep -i 'queue.*is stuck|rcu_preempt' fedora_journalctl_20240116
Jan 15 10:31:58 fedora kernel: iwlwifi 0000:0a:00.0: Queue 4 is stuck 12684 
12704
Jan 15 10:32:50 fedora kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/
tasks:
```
For additional information and undertaken troubleshooting steps:
https://bugzilla.kernel.org/show_bug.cgi?id=217990[1]
https://community.intel.com/t5/Wireless/AX210-System-Crash-after-iwlwifi-Queue-Stuck-Error-Issue/m-p/1562147[2]
https://bugzilla.redhat.com/show_bug.cgi?id=2262044[3]

Any guidance, insights, or suggestions for troubleshooting this issue further 
would be greatly appreciated. 

Thank you for your time and dedication to the Linux wireless ecosystem.

Sincerely,  
Chris 

--------
[1] https://bugzilla.kernel.org/show_bug.cgi?id=217990
[2] https://community.intel.com/t5/Wireless/AX210-System-Crash-after-iwlwifi-Queue-Stuck-Error-Issue/m-p/1562147
[3] https://bugzilla.redhat.com/show_bug.cgi?id=2262044