Linux wireless drivers development
 help / color / mirror / Atom feed
* Re: [REGRESSION] mt7925: MLO connectivity silently stalls with 6GHz link active
@ 2026-08-19  9:03 Jonas Hort
  0 siblings, 0 replies; 8+ messages in thread
From: Jonas Hort @ 2026-08-19  9:03 UTC (permalink / raw)
  To: Devin Wittmayer
  Cc: Thorsten Leemhuis, regressions, linux-wireless, Felix Fietkau,
	lorenzo.bianconi83

Thanks for the detailed breakdown.

I'll build v7.0 vanilla and test it, as suggested. Fair warning
though: I'm on vacation this week, so I'll pick this up next week.
I've also never compiled a kernel before, so it'll likely take me a
bit of trial and error the first time around - please bear with me
if it takes a little longer than expected.

Will report back once I have results.

Thanks again,
Jonas

19.08.2026 03:18:34 Devin Wittmayer <lucid_duck@justthetip.ca>:

> Thank you very much, that answers both things.
> 
> The ROC tracing is the more useful of the two even though it came back
> negative. Two of the three freezes have no ROC activity in them at all, so a
> link switch that never finished cannot be what starts this. The middle one
> does have rocabort, mloroc and rocwork in it, but one out of three makes
> that look like the exception rather than the pattern. So the area I sent you
> looking at is out, and that is worth knowing before you spend nights on
> builds.
> 
> One other thing worth saying first. There is a five patch mt76 series on the
> list at the moment and two of the patches look like they were written for
> exactly this bug. I do not think they were, and it is your own numbers that
> show it. The failure 4/5 fixes stops mt76_txq_schedule_list from servicing
> the queue, and the one 5/5 fixes stops mt76_txq_send_burst once the non-AQL
> count reaches its cap. Both of those keep frames from ever reaching the
> hardware, so if either were your problem head would be sitting still
> alongside tail. Yours does the opposite. Head climbs 390 to 408 while tail
> stays at 260, so the frames are getting into the ring and nothing is
> finishing them, which is the far end of the same path. 2/5 is a use after
> free when an interface goes away, so it does not fit either. I would not
> expect that series to change what you see.
> 
> On the bisect I would build v7.0 next. There are 32 mt7925 commits between
> 7.0 and 7.1 and 19 of them are one run of work from Sean Wang, reworking how
> the driver tracks the per link mlink and WCID for an MLO station. That is
> the kind of change that fits a bug only showing up with two links up. If
> v7.0 comes back clean, that series is where I would look. If v7.0 is already
> broken then it is off the hook and 6.19 becomes the next split. The mt76
> core and mac80211 both moved in the same window, so mt7925 is where I would
> look first rather than the only place worth looking.
> 
> Devin
> 
> Am 17.08.26 um 23:34 schrieb Jonas Hort:
> 
>> Quick follow-up: managed to confirm 6.18 as a clean baseline on my
>> own hardware now (not just secondhand from others in the forum
>> thread) - running Linux 6.18.42-1-cachyos-lts with MLO active
>> (5GHz+6GHz, same FritzBox 5690 Pro) for 3 hours straight, no freeze
>> at all.
> 
> Am 17.08.26 um 16:25 schrieb Jonas Hort:
> 
>> One correction to how I described this earlier: the connection does
>> NOT reliably self-heal on its own. I have manually intervened every
>> single time to restore connectivity [...]
>> 
>> Update on the ROC tracing: three real freezes captured now with the
>> kprobes active (all confirmed via the WFDMA0 tail-frozen signature).
>> 
>> - Freeze #1 (15:37): no ROC activity in the trace.
>> - Freeze #2 (15:43): [...] The trace shows several ROC events
>>   (rocabort, mloroc, rocwork) clustered together.
>> - Freeze #3 (16:15): no ROC activity again.
>> 
>> Uploaded all three logs to the bugzilla ticket if useful:
>> https://bugzilla.kernel.org/show_bug.cgi?id=221884

^ permalink raw reply	[flat|nested] 8+ messages in thread
* Re: [REGRESSION] mt7925: MLO connectivity silently stalls with 6GHz link active
@ 2026-08-19  9:03 Jonas Hort
  0 siblings, 0 replies; 8+ messages in thread
From: Jonas Hort @ 2026-08-19  9:03 UTC (permalink / raw)
  To: Devin Wittmayer
  Cc: Thorsten Leemhuis, regressions, linux-wireless, Felix Fietkau,
	lorenzo.bianconi83

Thanks for the detailed breakdown.

I'll build v7.0 vanilla and test it, as suggested. Fair warning
though: I'm on vacation this week, so I'll pick this up next week.
I've also never compiled a kernel before, so it'll likely take me a
bit of trial and error the first time around - please bear with me
if it takes a little longer than expected.

Will report back once I have results.

Thanks again,
Jonas

19.08.2026 03:18:34 Devin Wittmayer <lucid_duck@justthetip.ca>:

> Thank you very much, that answers both things.
> 
> The ROC tracing is the more useful of the two even though it came back
> negative. Two of the three freezes have no ROC activity in them at all, so a
> link switch that never finished cannot be what starts this. The middle one
> does have rocabort, mloroc and rocwork in it, but one out of three makes
> that look like the exception rather than the pattern. So the area I sent you
> looking at is out, and that is worth knowing before you spend nights on
> builds.
> 
> One other thing worth saying first. There is a five patch mt76 series on the
> list at the moment and two of the patches look like they were written for
> exactly this bug. I do not think they were, and it is your own numbers that
> show it. The failure 4/5 fixes stops mt76_txq_schedule_list from servicing
> the queue, and the one 5/5 fixes stops mt76_txq_send_burst once the non-AQL
> count reaches its cap. Both of those keep frames from ever reaching the
> hardware, so if either were your problem head would be sitting still
> alongside tail. Yours does the opposite. Head climbs 390 to 408 while tail
> stays at 260, so the frames are getting into the ring and nothing is
> finishing them, which is the far end of the same path. 2/5 is a use after
> free when an interface goes away, so it does not fit either. I would not
> expect that series to change what you see.
> 
> On the bisect I would build v7.0 next. There are 32 mt7925 commits between
> 7.0 and 7.1 and 19 of them are one run of work from Sean Wang, reworking how
> the driver tracks the per link mlink and WCID for an MLO station. That is
> the kind of change that fits a bug only showing up with two links up. If
> v7.0 comes back clean, that series is where I would look. If v7.0 is already
> broken then it is off the hook and 6.19 becomes the next split. The mt76
> core and mac80211 both moved in the same window, so mt7925 is where I would
> look first rather than the only place worth looking.
> 
> Devin
> 
> Am 17.08.26 um 23:34 schrieb Jonas Hort:
> 
>> Quick follow-up: managed to confirm 6.18 as a clean baseline on my
>> own hardware now (not just secondhand from others in the forum
>> thread) - running Linux 6.18.42-1-cachyos-lts with MLO active
>> (5GHz+6GHz, same FritzBox 5690 Pro) for 3 hours straight, no freeze
>> at all.
> 
> Am 17.08.26 um 16:25 schrieb Jonas Hort:
> 
>> One correction to how I described this earlier: the connection does
>> NOT reliably self-heal on its own. I have manually intervened every
>> single time to restore connectivity [...]
>> 
>> Update on the ROC tracing: three real freezes captured now with the
>> kprobes active (all confirmed via the WFDMA0 tail-frozen signature).
>> 
>> - Freeze #1 (15:37): no ROC activity in the trace.
>> - Freeze #2 (15:43): [...] The trace shows several ROC events
>>   (rocabort, mloroc, rocwork) clustered together.
>> - Freeze #3 (16:15): no ROC activity again.
>> 
>> Uploaded all three logs to the bugzilla ticket if useful:
>> https://bugzilla.kernel.org/show_bug.cgi?id=221884

^ permalink raw reply	[flat|nested] 8+ messages in thread
* [REGRESSION] mt7925: MLO connectivity silently stalls with 6GHz link active
@ 2026-08-14 14:04 Jonas Hort
  2026-08-15  5:51 ` Thorsten Leemhuis
  0 siblings, 1 reply; 8+ messages in thread
From: Jonas Hort @ 2026-08-14 14:04 UTC (permalink / raw)
  To: regressions; +Cc: linux-wireless, lorenzo.bianconi83

Filed as: https://bugzilla.kernel.org/show_bug.cgi?id=221884


Summary: WiFi 7 MLO (5GHz+6GHz) on MT7925 silently stops passing traffic
after a few minutes, while the driver continues to report a fully
healthy link. Root cause appears to be a stalled WFDMA0 TX hardware
queue. Reproduced on both linux 7.1.8 (stable) and 7.2.0-rc7 (mainline
pre-release), not fixed as of rc7. Disabling the 6GHz radio on the
access point eliminates the freeze entirely in my testing.

This is a regression: reported by multiple users on the CachyOS forum
as broken starting with the 7.1.x kernel line; earlier kernels
(6.18.x longterm) are reported to work fine. I have not yet performed
a commit-level bisection but can if guided on the best approach given
the AP-side 6GHz dependency.

--- Detailed description ---

Symptom:
When connected via MLO (Multi-Link Operation, EHT/WiFi 7) with two
active links - one on 5GHz and one on 6GHz - network traffic silently
stops after a few minutes of normal use (web browsing, etc). The
connection continues to show as healthy:

- `iw link` / `iw station dump` report the connection as associated,
  authenticated, good signal (-60 to -65 dBm), high negotiated
  bitrate (800-1700 Mbit/s), 0 tx retries, 0 tx failed.
- Nothing is logged to dmesg/journalctl at the time of the freeze.
- Ping to the gateway shows 100% packet loss for several minutes,
  until the connection eventually recovers on its own.

I built a small watchdog script (ping-based failure detection + live
kernel log monitoring) to catch the freeze automatically and sample
mt76 debugfs state repeatedly across the event. This revealed the
actual mechanism:

`/sys/kernel/debug/ieee80211/phy0/mt76/xmit-queues` shows the WFDMA0
hardware TX queue's `tail` pointer completely frozen while `head`
keeps advancing - i.e. packets keep getting enqueued but the firmware
stops draining the queue. RX stops in the same instant (rx byte/packet
counters in `iw station dump` freeze completely). No tx retries or tx
failures are ever reported by the driver, so it does not appear to
notice the stall itself.

Example from one incident (kernel 7.2.0-rc7):
  t+1s:  WFDMA0: queued=130 head=390 tail=260   rx bytes=184791894
  t+9s:  WFDMA0: queued=148 head=408 tail=260   rx bytes=184791894 (unchanged)

`tail` never advances during the whole stall window while `head`
keeps growing - the queue is being filled but never drained.

Reproduced this exact signature four times total (twice on 7.1.8-1,
twice on 7.2.0-rc7-1), always with MLO 5GHz+6GHz active, always
within 3-5 minutes of normal use.

6GHz correlation:
My MLO links are 5GHz (5180 MHz) + 6GHz (varies, e.g. 6135 MHz).
Fully disabling the 6GHz radio on the access point (FritzBox 5690
Pro, FRITZ!OS 8.25 - not just client-side band restriction via
NetworkManager, which does not reliably suppress the second MLO
link) results in 30+ minutes of clean operation with no freeze.
Re-enabling 6GHz reproduces the freeze again within minutes.

Regression info:
Multiple users on the CachyOS forum report this started with the
7.1.x kernel line; the previous longterm kernel (6.18.x) is reported
to work fine by another user with different affected hardware
(different AP). I have not personally tested 6.18.x myself, and have
not performed a commit-level bisection yet - happy to do so if
pointed toward the most likely area of the driver, given the AP-side
6GHz dependency makes a fully automated bisection awkward.

Steps to reproduce:
1. Connect to an AP advertising WiFi 7 MLO with 5GHz+6GHz link
   options, so the client negotiates both links.
2. Use the connection normally (web browsing is sufficient).
3. Within roughly 3-5 minutes, new connections start hanging;
   existing traffic stops.
4. Check `iw link` - link still reports as connected/healthy.
5. Check `/sys/kernel/debug/ieee80211/phy*/mt76/xmit-queues` -
   WFDMA0 tail pointer frozen while head continues to advance.

Environment:
- Kernel: linux-cachyos 7.1.8-1 and linux-cachyos-rc 7.2.0-rc7-1
  (CachyOS, near-vanilla Arch-based build)
- cat /proc/version:
  Linux version 7.2.0-rc7-1-cachyos-rc (linux-cachyos-rc@cachyos)
  (clang version 22.1.8, LLD 22.1.8) #1 SMP PREEMPT_DYNAMIC
  Mon, 10 Aug 2026 20:51:00 +0000
- Distribution: CachyOS
- Architecture: x86_64
- Kernel tainted: no (/proc/sys/kernel/tainted = 0)
- WiFi hardware: MediaTek MT7925 (mt7925e driver, mt76 core),
  WM Firmware Build 20260605184805, ASIC revision 79250000
- Access point: AVM FritzBox 5690 Pro, FRITZ!OS 8.25

Related community discussion:
https://discuss.cachyos.org/t/mt7925-wifi-7-mlo-breaks-connectivity-on-7-1-x-mt76-regression/31972
(I am the thread starter; several other users report matching
symptoms with different hardware/APs)

Possibly related (different symptom, same 6GHz/MT7925 area, worth
checking for a common root cause):
Bug 221627 - MT7925E - System locks up with flashing caps lock
(comment 1 there: "caused by switching between AP's with the same
6GHz wifi name")

Attachments (will add to this ticket):
- Full incident logs for all four reproductions (dmesg + journalctl
  + mt76 debugfs time series across each freeze)
- iw station dump / iw link output during freeze
- Kernel .config

Note: checked bug 220424 (mlo_sta_cmd/sta_cmd regression) before filing -
different symptom (persistent near-zero throughput vs. our intermittent
full stall with healthy link stats) and different root cause (broadcast
wtbl hdr_trans_tlv handling). That fix landed in 6.16.6/6.17-rc5, well
before the kernel versions tested here, so it is not the cause of this
issue.


^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2026-08-19  9:03 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-08-19  9:03 [REGRESSION] mt7925: MLO connectivity silently stalls with 6GHz link active Jonas Hort
  -- strict thread matches above, loose matches on Subject: below --
2026-08-19  9:03 Jonas Hort
2026-08-14 14:04 Jonas Hort
2026-08-15  5:51 ` Thorsten Leemhuis
2026-08-16 20:48   ` Devin Wittmayer
2026-08-17 14:25     ` Jonas Hort
2026-08-17 21:34       ` Jonas Hort
2026-08-19  1:18         ` Devin Wittmayer

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox