rtw88: alloc_skb(32768, GFP_ATOMIC) fails, driver gets stuck

[Bitterblue Smith <rtl8821cerfe2@gmail.com>]

On 25/09/2024 14:46, Bitterblue Smith wrote:
> Hi,
> 
> I have this problem with RTL8811CU, RTL8723DU, RTL8811AU, RTL8812AU.
> I assume all USB devices are affected. If I have qBittorrent running,
> the wifi stops working after a few hours:
> 
> Sep 24 00:48:21 ideapad2 wpa_supplicant[1290]: wlp3s0f3u2i2: CTRL-EVENT-BEACON-LOSS
> Sep 24 00:48:21 ideapad2 kernel: rtw_8723du 1-2:1.2: failed to get tx report from firmware
> Sep 24 00:48:23 ideapad2 wpa_supplicant[1290]: wlp3s0f3u2i2: CTRL-EVENT-BEACON-LOSS
> Sep 24 00:48:23 ideapad2 kernel: rtw_8723du 1-2:1.2: failed to get tx report from firmware
> Sep 24 00:48:25 ideapad2 wpa_supplicant[1290]: wlp3s0f3u2i2: CTRL-EVENT-BEACON-LOSS
> Sep 24 00:48:25 ideapad2 kernel: rtw_8723du 1-2:1.2: failed to get tx report from firmware
> Sep 24 00:48:27 ideapad2 wpa_supplicant[1290]: wlp3s0f3u2i2: CTRL-EVENT-BEACON-LOSS
> Sep 24 00:48:27 ideapad2 kernel: rtw_8723du 1-2:1.2: failed to get tx report from firmware
> Sep 24 00:48:29 ideapad2 wpa_supplicant[1290]: wlp3s0f3u2i2: CTRL-EVENT-BEACON-LOSS
> Sep 24 00:48:29 ideapad2 kernel: rtw_8723du 1-2:1.2: failed to get tx report from firmware
> Sep 24 00:48:31 ideapad2 wpa_supplicant[1290]: wlp3s0f3u2i2: CTRL-EVENT-BEACON-LOSS
> Sep 24 00:48:31 ideapad2 kernel: rtw_8723du 1-2:1.2: failed to get tx report from firmware
> Sep 24 00:48:33 ideapad2 wpa_supplicant[1290]: wlp3s0f3u2i2: CTRL-EVENT-BEACON-LOSS
> Sep 24 00:48:33 ideapad2 kernel: rtw_8723du 1-2:1.2: failed to get tx report from firmware
> Sep 24 00:48:35 ideapad2 wpa_supplicant[1290]: wlp3s0f3u2i2: CTRL-EVENT-BEACON-LOSS
> Sep 24 00:48:35 ideapad2 kernel: rtw_8723du 1-2:1.2: failed to get tx report from firmware
> Sep 24 00:48:37 ideapad2 wpa_supplicant[1290]: wlp3s0f3u2i2: CTRL-EVENT-BEACON-LOSS
> Sep 24 00:48:37 ideapad2 kernel: rtw_8723du 1-2:1.2: failed to get tx report from firmware
> Sep 24 00:48:39 ideapad2 wpa_supplicant[1290]: wlp3s0f3u2i2: CTRL-EVENT-BEACON-LOSS
> Sep 24 00:48:39 ideapad2 kernel: rtw_8723du 1-2:1.2: failed to get tx report from firmware
> Sep 24 00:48:41 ideapad2 wpa_supplicant[1290]: wlp3s0f3u2i2: CTRL-EVENT-BEACON-LOSS
> Sep 24 00:48:41 ideapad2 kernel: rtw_8723du 1-2:1.2: failed to get tx report from firmware
> Sep 24 00:48:42 ideapad2 wpa_supplicant[1290]: wlp3s0f3u2i2: CTRL-EVENT-DISCONNECTED bssid=... reason=4 locally_generated=1
> Sep 24 00:48:42 ideapad2 wpa_supplicant[1290]: wlp3s0f3u2i2: Added BSSID ... into ignore list, ignoring for 10 seconds
> Sep 24 00:48:42 ideapad2 NetworkManager[433]: <info>  [1727128122.0377] device (wlp3s0f3u2i2): supplicant interface state: completed -> disconnected
> Sep 24 00:48:45 ideapad2 NetworkManager[433]: <info>  [1727128125.6030] device (wlp3s0f3u2i2): supplicant interface state: disconnected -> scanning
> Sep 24 00:48:47 ideapad2 wpa_supplicant[1290]: wlp3s0f3u2i2: Removed BSSID ... from ignore list (clear)
> Sep 24 00:48:47 ideapad2 wpa_supplicant[1290]: wlp3s0f3u2i2: SME: Trying to authenticate with ... (SSID='...' freq=2472 MHz)
> Sep 24 00:48:50 ideapad2 kernel: wlp3s0f3u2i2: authenticate with ... (local address=,,,)
> Sep 24 00:48:51 ideapad2 NetworkManager[433]: <info>  [1727128131.2488] device (wlp3s0f3u2i2): supplicant interface state: scanning -> authenticating
> Sep 24 00:48:51 ideapad2 kernel: wlp3s0f3u2i2: send auth to ... (try 1/3)
> Sep 24 00:48:51 ideapad2 kernel: rtw_8723du 1-2:1.2: failed to get tx report from firmware
> Sep 24 00:48:52 ideapad2 kernel: wlp3s0f3u2i2: send auth to ... (try 2/3)
> Sep 24 00:48:52 ideapad2 kernel: rtw_8723du 1-2:1.2: failed to get tx report from firmware
> Sep 24 00:48:53 ideapad2 kernel: wlp3s0f3u2i2: send auth to ... (try 3/3)
> Sep 24 00:48:53 ideapad2 kernel: rtw_8723du 1-2:1.2: failed to get tx report from firmware
> Sep 24 00:48:54 ideapad2 kernel: wlp3s0f3u2i2: authentication with ... timed out
> 
> After this all scans return nothing. The chip is still alive,
> though. The LED blinks during the scans (it's hardware-controlled)
> and another device in monitor mode can see the probe requests.
> 
> I confirmed that even C2H stop coming. I used aireplay-ng to send
> some authentication or association frames (can't remember) which
> require TX ACK report. I saw "failed to get tx report from firmware"
> and no C2H.
> 
> While qBittorrent is needed to trigger this bug, simply downloading
> a random Linux iso did not do the job. "Other" torrents did. It's
> unclear why. Maybe it's uploading that triggers the bug.
> 
> I left iperf3 running all day and nothing happened. Only qBittorrent
> can break it.
> 
> RTL8822CE doesn't have this problem. I can use qBittorrent with it
> just fine.
> 
> I mounted debugfs and dumped the MAC registers during a scan using
> this command:
> 
> for i in {00..20}; do sleep 0.5; cat /sys/kernel/debug/ieee80211/phy2/rtw88/mac_{0..7} > dead-$i.txt; done
> 
> I thought maybe some RX URBs failed silently and rtw88 stopped
> sending them to the device (== stopped requesting data from it),
> but that's not the case. [1]
> 
> I have the device in this state right now. Is there anything else
> I should look at?
> 
> 
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/drivers/net/wireless/realtek/rtw88/usb.c?h=v6.10.11&id=25eaef533bf3ccc6fee5067aac16f41f280e343e#n641

I still don't know why qBittorrent is required to trigger this bug,
but I found the problem.

alloc_skb fails (silently) therefore the RX URB is not submitted
ever again. There are only 4 RX URBs.

static void rtw_usb_rx_resubmit(struct rtw_usb *rtwusb, struct rx_usb_ctrl_block *rxcb)
{
	struct rtw_dev *rtwdev = rtwusb->rtwdev;
	int error;

	rxcb->rx_skb = alloc_skb(RTW_USB_MAX_RECVBUF_SZ, GFP_ATOMIC);
	if (!rxcb->rx_skb)
		return;

	usb_fill_bulk_urb(rxcb->rx_urb, rtwusb->udev,
			  usb_rcvbulkpipe(rtwusb->udev, rtwusb->pipe_in),
			  rxcb->rx_skb->data, RTW_USB_MAX_RECVBUF_SZ,
			  rtw_usb_read_port_complete, rxcb);

	error = usb_submit_urb(rxcb->rx_urb, GFP_ATOMIC);

I added an error message there:

	rxcb->rx_skb = alloc_skb(RTW_USB_MAX_RECVBUF_SZ, GFP_ATOMIC);
	if (!rxcb->rx_skb) {
		rtw_err(rtwdev, "failed to allocate rx_skb\n");
		return;
	}

Now I get these before it stops receiving:

Sep 30 22:34:38 ideapad2 kernel: rtw_8723du 1-2:1.2: failed to allocate rx_skb
Sep 30 22:35:03 ideapad2 kernel: rtw_8723du 1-2:1.2: failed to allocate rx_skb
Sep 30 22:35:03 ideapad2 kernel: rtw_8723du 1-2:1.2: failed to allocate rx_skb
Sep 30 22:35:03 ideapad2 kernel: rtw_8723du 1-2:1.2: failed to allocate rx_skb
Sep 30 22:35:05 ideapad2 wpa_supplicant[1287]: wlp3s0f3u2i2: CTRL-EVENT-BEACON-LOSS                                                                                        Sep 30 22:35:05 ideapad2 kernel: rtw_8723du 1-2:1.2: failed to get tx report from firmware                                                                              
Sep 30 22:35:07 ideapad2 wpa_supplicant[1287]: wlp3s0f3u2i2: CTRL-EVENT-BEACON-LOSS                                                                                        
Sep 30 22:35:07 ideapad2 kernel: rtw_8723du 1-2:1.2: failed to get tx report from firmware                                                                                 
Sep 30 22:35:08 ideapad2 wpa_supplicant[1287]: wlp3s0f3u2i2: CTRL-EVENT-DISCONNECTED bssid=... reason=4 locally_generated=1                                  

What to do about it?