RE: [BUG] rtw88 8821ce: connection wedges (100% loss until reboot) with station power save

[Ping-Ke Shih <pkshih@realtek.com>]

Abdurrahman Karadag <abdurrahmankaradag19@gmail.com> wrote:
> > Is there are version it works well on your platform?
> > I'm thinking bisect is a method to find cause.
> 
> Yes - and I have to correct my first mail, which said "also seen on
> 7.0.12". Going back through the system journal and pacman log:
> 
>   - 7.0.12 (Arch) ran on this laptop from 2026-06-16 to 2026-08-23. The
>     14 boots the journal holds from that period show no wedge episode
>     (defined as: DHCP lease acquired, then the gateway unreachable / DNS
>     failing continuously until reboot). The few candidates are either
>     sessions caught in the separate 63 s regulatory reconnect loop, or
>     sporadic DNS timeouts spread over hours-long otherwise-working
>     sessions (14-27 timeouts in 0.8-3.5 h), plus one 176 s blip - none is
>     the 100%-loss-until-reboot pattern.
>   - On 2026-08-23 18:41 pacman upgraded linux 7.0.12 -> 7.1.9 (in the same
>     transaction: linux-firmware-realtek 20260519 -> 20260810, whose
>     rtw8821c_fw.bin is byte-identical by sha256, and systemd 260.2 ->
>     261.2 a few minutes earlier).
>   - The first boot on 7.1.9 (2026-08-23 23:37) logged 12 wedge episodes,
>     and it has recurred on most days since.
> 
> So for a bisect: good = 7.0.12, bad = 7.1.9 (with the caveat that
> systemd changed in the same upgrade; I don't think networkd can explain
> a per-AC L2 TX failure, but I mention it for completeness). I still have
> the 7.0.12 package and can confirm by running it again for a few days,
> and I'm happy to bisect the rtw88/mac80211 range between the two if you
> think that's the right next step. (Under Windows the same laptop has
> never shown this.)

Checking commits between 7.0.12 and 7.1.9, the only related commit might be
   c95323ea9dfb ("wifi: rtw88: coex: Solve LE-HID lag & update coex version to 26020420")

You can revert the patch from 7.1.9 to see if it becomes normal.

Another simple way is to use 7.0.12 kernel + 7.1.9 rtw88 driver and
opposite combination to address the cause, like

     kernel         rtw88 driver         result
     ------         ------------         ------
     7.0.12         (built-in)           Good
     7.0.12         7.1.9
     7.1.9          (built-in)           NG
     7.1.9          7.0.12 

This can also bisect if the cause is driver or mac80211.

> 
> > I'm not sure why the size can affect the result. Normally large size
> > is harder to transmit basically though.
> 
> You are right, and I withdraw the size theory. Re-examining the same
> capture by 802.11 access category instead of size explains every frame
> with no exceptions:
> 
>   - every laptop TX frame that provably reached the AP carries IP DSCP
>     0xc0 (CS6 -> UP 6 -> AC_VO): the DHCP DISCOVER/REQUEST frames
>     (systemd-networkd's DHCP client marks them CS6) and IGMPv3 reports;
>   - every laptop TX frame that provably never arrived is AC_BE:
>     89 ARP requests (no IP header -> BE), 13 unicast ARP replies,
>     83 DNS queries (0 answers), 2 TCP SYNs (no SYN-ACK), mDNS/LLMNR;
>   - sizes overlap the wrong way for a size theory: BE frames of 42-201 B
>     all fail, VO frames of 54-345 B all pass.
> 
> So the wedge looks like "AC_BE TX dead, AC_VO TX alive", RX intact, and
> it persists across disconnect/reconnect and across AP changes (so not
> per-association state), cleared only by a reboot. It happened with
> power save fully off.
> 
> As far as I can tell from reading the driver, BE and VO take different
> paths (separate PCIe TX rings per AC, and on 8821C BE/BK on the LOW
> TX-FIFO queue vs VO/VI on NORMAL), so a per-AC stall - e.g. the BE ring's
> mac80211 queue left stopped, or the LOW queue paused/page-starved - would
> match what I see (BE frames silently aged out, VO flowing, no driver
> message). Does that sound plausible to you, or is there a more likely
> place for a per-AC TX stall on this chip? The monitor-mode capture
> should at least show whether BE frames reach the air at all.

I think BE and VO should almost the same, but as your perspective
BE and VO go via different paths. It is worth to do an experiment
to let all packets (by driver modification) go via VO queue.

> 
> When it wedges, besides the air capture, I plan to dump before rebooting:
> /sys/kernel/debug/ieee80211/phy0/queues (per-hw-queue stop reasons),
> stations/<ap>/aqm (per-TID backlog), and via rtw88 debugfs read_reg:
> REG_TXPAUSE (0x522), REG_TXDMA_STATUS (0x210), REG_FIFOPAGE_INFO_2/3
> (0x234/0x238) and the BE/VO TXBD indices (0x3A8/0x3A0), compared with a
> healthy baseline. If there are better registers or a debugfs page for
> the BE ring / LOW queue state on 8821C, please tell me and I'll dump
> those instead.

Before checking these values, let's bisect the code, check sniffer, and 
do experiment first.