RE: [BUG] wifi: rtw88: Hard system freeze on RTL8821CE when power_save is enabled (LPS/ASPM conflict)

[Ping-Ke Shih <pkshih@realtek.com>]

LB F <goainwo@gmail.com> wrote:
> Hi Ping-Ke,
> 
> I successfully applied your patch out-of-tree and performed rigorous
> testing on the host machine.
> 
> I can officially confirm that the patch works flawlessly. The DMI
> quirk triggered correctly and successfully prevented the
> hardware-level PCIe bus lockups on my HP P3S95EA#ACB.

Thanks for your quickly test with my patch. :)

> 
> Testing Environment & Methodology:
> - Kernel: CachyOS Linux 6.19.6-2-cachyos x86_64
> - Toolchain: Clang/LLVM 21.1.8 (`make CC=clang LLVM=1 modules`)
> - Extraction: We fetched the strict
> `drivers/net/wireless/realtek/rtw88` sub-tree out of the
> torvalds/linux `v6.19` tree utilizing `git sparse-checkout` to cleanly
> apply the patch without having to compile the entire 2.5GB+ kernel.
> - The resulting `.ko` object files were compressed to `.zst` and
> installed successfully over the generic CachyOS system driver objects.
> 
> Verification Conditions:
> - Removed ALL local workarounds. `disable_aspm=Y` is no longer forced
> via `/etc/modprobe.d/` overrides.
> - Power saving remains natively ON `wifi.powersave = 3` (managed by
> NetworkManager).
> - Left the laptop in multiple 5-10 minute complete idle states to
> enforce sleep modes.
> 
> Post-Boot Log Analysis & Potential Improvement Proposition:
> The system remained 100% stable without any kernel panics or UI freezes.
> However, I continuously monitored the `dmesg` ring buffer and noticed
> an intriguing behavior. While the laptop sits completely idle
> (NetworkManager connected, but no active traffic), the `rtw88` driver
> starts flooded the logs with thousands of firmware errors:
> 
> [ 1084.746485] rtw88_8821ce 0000:13:00.0: firmware failed to leave lps state
> [ 1084.749662] rtw88_8821ce 0000:13:00.0: failed to send h2c command
> [ 1084.752895] rtw88_8821ce 0000:13:00.0: failed to send h2c command
> 
> If my understanding of this architecture is correct, previously, when
> ASPM wasn't disabled, this exact failure of the adapter firmare inside
> `LPS_DEEP_MODE_LCLK` would violently lock up the PCIe bus and crash
> the host. Now, thanks to your DMI ASPM quirk at the `rtw88_pci` level,
> the host PCIe controller doesn't enter `L1` and is perfectly shielded
> from the adapter locking itself up! The OS handles the timeouts
> gracefully and driver recovery prevents a hard freeze.

I'm really not sure how/why kernel becomes frozen. As I mentioned before
it might because of received malformed data and no complete validation
before reporting RX packet to mac80211.

Not sure if you can try to dig and add some validation?

(Current DMI patch is fine to me.)

> 
> A question for your consideration: Given the immense volume of these
> `h2c` timeout errors (and the underlying firmware's fundamental
> inability to cleanly enter/exit its own sleep states without L1
> participation on this HP model), do you think it would be beneficial
> to *also* dynamically disable LPS Deep sleep when this specific ASPM
> quirk is triggered?
> 
> For example, dynamically forcing `rtwdev->lps_conf.deep_mode =
> LPS_DEEP_MODE_NONE` when the DMI ASPM flag is active, strictly to
> prevent the firmware from attempting a sleep cycle that is doomed to
> fail and polluting the queues and logs? Perhaps this might also save
> microscopic CPU interrupts from continuous H2C polling timeouts?

Are the 'h2c' timeout messages flooding? or appears periodically? 
Does it really affect connection stable?

If you change another AP or connection on 5GHz band, does the messages
still present? 

I think it isn't easy to find out the cause without measuring hardware
signals, since I saw the message very very rare. So, I'd adopt your
suggestion (dynamic LPS_DEEP_MODE_NONE) if the test is positive.

> 
> If you believe that simply letting the driver recover and tolerating
> the error spam in `dmesg` is the preferred/safer upstream approach, I
> am perfectly happy. The patch functions as advertised and system
> stability is unequivocally restored!
> 
> Thank you immensely for your rapid debugging and definitive patch for
> this long-standing issue and for bringing stability to this model.
> 
> Tested-by: Oleksandr Havrylov <goainwo@gmail.com>

I will add this to my patch then.

> 
> *(Note: I was a bit unsure which of the two active mailing list
> threads was the most appropriate place for this final report — the
> original bug discussion or the new RFT patch submission thread — so I
> replied to both just to ensure it is correctly attached to the patch.
> Apologies for the duplicate email!)*
> 

Let's discuss in this thread. For RFT patch, I suppose you only reply
me about the test result and give me Tested-by tag if it works. 

By the way, your this reply is top posting that mailing list isn't
preferred, so I delete old discussion. Please avoid this in the future.

Ping-Ke