Netdev List
 help / color / mirror / Atom feed
* [PATCH net v7 0/6] net: wwan: t7xx: fix TX push kthread soft lockup on suspend and RX stop issues
@ 2026-11-20  1:55 Tim JH Chen
  2026-11-20  1:55 ` [PATCH net v7 1/6] net: wwan: t7xx: fix TX push kthread soft lockup on system suspend Tim JH Chen
                   ` (6 more replies)
  0 siblings, 7 replies; 8+ messages in thread
From: Tim JH Chen @ 2026-11-20  1:55 UTC (permalink / raw)
  To: netdev
  Cc: chandrashekar.devegowda, haijun.liu, ricardo.martinez,
	loic.poulain, ryazanov.s.a, johannes, andrew+netdev, davem,
	edumazet, kuba, pabeni, ilpo.jarvinen, sreehari.kancharla,
	m.chetan.kumar, horms, linux-kernel, tim.jh.chen, Chih.Hung.Huang,
	Tim JH Chen

net: wwan: t7xx: fix NAPI completion and rx_stop ordering

Two fixes in t7xx_hif_dpmaif_rx.c.

Patch 1 completes the NAPI on the not-started early return in
t7xx_dpmaif_napi_rx_poll().

Patch 2 fixes the memory ordering between t7xx_dpmaif_rx_stop() and
t7xx_dpmaif_napi_rx_poll().

v7:
 - Drop v6 patches 1-3. They need rework based on the review and will be
   sent separately.
 - Patch 1 (v6 4/4): change Fixes: to 5545b7b9f294; remove the code
   comment.
 - Patch 2: new.

v6: https://lore.kernel.org/netdev/20261002014638.47981-1-tim770802@gmail.com/

v6 -> v7:
  - Patch 1 (was 3/4): drop the explicit quiesce added to
    t7xx_dpmaif_suspend() in v6 (interrupt mask reorder,
    cancel_work_sync() of the TX-done workers and of bat_release_work,
    hardware stop moved last). The review found it incomplete without
    synchronize_irq(), losing the UL-DONE unmask and able to deadlock
    runtime suspend. Also drop the freezer: the kthread now skips the
    push and waits on tx_wq when it finds the TX queue stopped after
    the runtime PM resume, which also covers a resume that does not
    restart the queues (modem exception, shutdown), where the freezer
    does not apply. Commit message rewritten around the mechanism
    visible in the captured soft lockup; subject changed accordingly.
  - Patch 2 (was 1/4): use pm_runtime_get_noresume() +
    pm_runtime_resume() at every site instead of per-site flags, which
    also covers t7xx_dpmaif_irq_rx_done()/t7xx_dpmaif_napi_rx_poll().
    State when -EACCES is reachable.
  - Patch 3 (new): t7xx_cldma_send_skb() returned -EACCES for a
    successful send, so the caller freed an skb that was already on
    the ring; same reference handling as patch 2.
  - Patch 4 (was 2/4): retry every second with the first failure of a
    streak logged, instead of every 20 ms; Reported-by/Closes for the
    field report on the v6 thread.
  - Patch 5 (was 4/4): Fixes: changed to 5545b7b9f294 ("net: wwan:
    t7xx: Add NAPI support"); code comment removed; user-visible
    impact added to the commit message.
  - Patch 6 (new): fix the memory ordering between
    t7xx_dpmaif_rx_stop() and t7xx_dpmaif_napi_rx_poll(), found by the
    v6 review, and keep the pending PIT release count of a poll that
    is in flight when the queue is stopped.
  - v6: https://lore.kernel.org/netdev/20261002014638.47981-1-tim770802@gmail.com/
v5 -> v6:
  - Drop the "freezer as a global quiesce" direction: remove every
    WQ_FREEZABLE annotation added in v4/v5. Marking the data-plane
    workqueues freezable can deadlock the suspend, because a
    flush_work()/cancel_work_sync() issued from a context the freezer
    does not freeze (the FSM kthread, or an unbind holding device_lock)
    blocks until thaw_workqueues(). (Reported on v5 review.)
  - Instead quiesce the DPMAIF data plane explicitly in the system
    suspend callback: mask interrupts, cancel_work_sync() the TX-done
    workers, call t7xx_dpmaif_rx_stop() to drain the in-flight NAPI RX
    poll (the freezer cannot park a softirq), then cancel
    bat_release_work and stop the hardware last.
  - Keep the PM freezer only for the lone TX push kthread, and use
    kthread_freezable_should_stop() instead of try_to_freeze().
  - Fold in the pre-existing fixes previously deferred to a separate
    series: the -EACCES runtime PM usage-count underflow (patch 1), the
    TX push kthread self-exit use-after-free (patch 2), and a NAPI that
    is never completed on the not-started RX poll early return (patch
    4). This revision is therefore a 4-patch series.
  - Do not touch t7xx_hif_cldma.c. Restrict the commit messages to the
    DPMAIF data path; the CLDMA control path uses a different mechanism
    and is called out as out of scope.
v4 -> v5:
  - Fix freeze deadlock in t7xx_do_tx_hw_push(): when the TX-done
    workqueue (WQ_FREEZABLE) is frozen first it stops draining the DRB
    ring; the kthread then loops indefinitely in the ring-full retry
    branch and never reaches try_to_freeze(), causing a freezer timeout
    and suspend abort. (Simon Horman)
  - Extend WQ_FREEZABLE to the BAT-release and CLDMA TX/RX workqueues.
    (Simon Horman) [reverted in v6, see above]
  - Note the -EACCES underflow and the stale kthread pointer as
    pre-existing issues to be addressed separately. [done in v6, patches 1-2]
v3 -> v4:
  - Drop the tx_pm_lock / state-snapshot approach entirely and use the PM
    freezer instead. The previous approach deadlocked through the runtime
    PM wait queue and opened ISR windows by writing dpmaif_ctrl->state in
    suspend/resume.
v2 -> v3: process fixes (Fixes tag, changelog placement).
v1 -> v2: save/restore pre-suspend state; wrap pm_runtime with a mutex.

Tim JH Chen (6):
  net: wwan: t7xx: fix TX push kthread soft lockup on system suspend
  net: wwan: t7xx: fix runtime PM usage count underflow on -EACCES
  net: wwan: t7xx: fix skb double free when CLDMA send sees -EACCES
  net: wwan: t7xx: do not exit the TX push kthread on resume failure
  net: wwan: t7xx: complete NAPI on the not-started RX poll early return
  net: wwan: t7xx: fix memory ordering in t7xx_dpmaif_rx_stop()

 drivers/net/wwan/t7xx/t7xx_hif_cldma.c     | 12 +++++-
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c | 35 ++++++++++-----
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c | 50 ++++++++++++++++++----
 3 files changed, 75 insertions(+), 22 deletions(-)

-- 
2.43.0


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH net v7 1/6] net: wwan: t7xx: fix TX push kthread soft lockup on system suspend
  2026-11-20  1:55 [PATCH net v7 0/6] net: wwan: t7xx: fix TX push kthread soft lockup on suspend and RX stop issues Tim JH Chen
@ 2026-11-20  1:55 ` Tim JH Chen
  2026-11-20  1:55 ` [PATCH net v7 2/6] net: wwan: t7xx: fix runtime PM usage count underflow on -EACCES Tim JH Chen
                   ` (5 subsequent siblings)
  6 siblings, 0 replies; 8+ messages in thread
From: Tim JH Chen @ 2026-11-20  1:55 UTC (permalink / raw)
  To: netdev
  Cc: chandrashekar.devegowda, haijun.liu, ricardo.martinez,
	loic.poulain, ryazanov.s.a, johannes, andrew+netdev, davem,
	edumazet, kuba, pabeni, ilpo.jarvinen, sreehari.kancharla,
	m.chetan.kumar, horms, linux-kernel, tim.jh.chen, Chih.Hung.Huang,
	Tim JH Chen

t7xx_dpmaif_suspend() stops the TX queues (que_started = false) but
leaves dpmaif_ctrl->state at DPMAIF_STATE_PWRON. The TX push kthread
only sleeps while the TX lists are empty or state != PWRON, so when
skbs are still queued at the time the suspend callback runs its wait
condition stays true and every iteration completes without doing
anything and without sleeping: pm_runtime_resume_and_get() returns 0
without resuming anything (the PCI core resumed the device before the
suspend callback, and once runtime PM is disabled in the late phase an
active device is reported as already resumed); t7xx_pci_disable_sleep()
completes at once because md_pm_state is no longer MTK_PM_RESUMED;
t7xx_do_tx_hw_push() returns immediately because no queue is started.
There is no scheduling point in that loop, so the kthread spins on its
CPU for the rest of the suspend/resume cycle.

With s2idle and ASPM L1 enabled this was seen as a soft lockup after
about a thousand suspend/resume cycles (the 981st in the capture below).
The lockup was reported at different points of the suspend: while the
device callbacks were still running, right after this driver's own
suspend callback had stopped the queues, and after the wakeup interrupt
but before any resume callback had run. Captured with kdump on an
unmodified 6.19 kernel:

  mtk_t7xx 0000:56:00.0: PM: pci_pm_suspend returned 0 after 39260 usecs
  ...
  watchdog: BUG: soft lockup - CPU#4 stuck for 26s! [dpmaif_tx_hw_pu:764]
  RIP: 0010:ktime_get_mono_fast_ns+0x2e/0xe0
  Call Trace:
   ? t7xx_pci_enable_sleep+0x44/0x90 [mtk_t7xx]
   t7xx_dpmaif_tx_hw_push_thread+0x1f9/0x490 [mtk_t7xx]
   kthread+0x10b/0x220
   ret_from_fork+0x159/0x1d0
   ret_from_fork_asm+0x1a/0x30

Other captures sample the same loop in t7xx_pci_disable_sleep() or in
the runtime PM put; all other CPUs are idle.

The same loop runs after a resume that does not restart the queues (a
modem exception found in the PM resume state, or shutdown) until the
FSM takes the DPMAIF out of DPMAIF_STATE_PWRON.

Once pm_runtime_resume_and_get() has returned, a runtime suspended
device has been resumed and its queues restarted, so a TX queue that is
still stopped at that point was stopped by a system suspend or by an
exception. Skip the push in that case and wait on tx_wq until the queue
is restarted or the thread is stopped; t7xx_dpmaif_resume() and
t7xx_dpmaif_start() already wake tx_wq after restarting the queues.

The check is not folded into the thread's main wait condition on
purpose: a runtime suspended device also has its queues stopped and
relies on this thread calling pm_runtime_resume_and_get() to be
resumed when skbs are queued.

Fixes: 46e8f49ed7b3 ("net: wwan: t7xx: Introduce power management")
Signed-off-by: Tim JH Chen <tim770802@gmail.com>
---
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c | 20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)

diff --git a/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c b/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c
index 236d632cf591..798e4e95004f 100644
--- a/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c
+++ b/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c
@@ -445,6 +445,7 @@ static void t7xx_do_tx_hw_push(struct dpmaif_ctrl *dpmaif_ctrl)
 static int t7xx_dpmaif_tx_hw_push_thread(void *arg)
 {
 	struct dpmaif_ctrl *dpmaif_ctrl = arg;
+	bool txq_stopped;
 	int ret;
 
 	while (!kthread_should_stop()) {
@@ -464,10 +465,23 @@ static int t7xx_dpmaif_tx_hw_push_thread(void *arg)
 		if (ret < 0 && ret != -EACCES)
 			return ret;
 
-		t7xx_pci_disable_sleep(dpmaif_ctrl->t7xx_dev);
-		t7xx_do_tx_hw_push(dpmaif_ctrl);
-		t7xx_pci_enable_sleep(dpmaif_ctrl->t7xx_dev);
+		/* A runtime suspended device was resumed above and had its queues
+		 * restarted, so a stopped queue here belongs to a system suspend
+		 * or an exception: wait for t7xx_dpmaif_resume() or
+		 * t7xx_dpmaif_start() to restart it instead of spinning.
+		 */
+		txq_stopped = !t7xx_select_tx_queue(dpmaif_ctrl);
+		if (!txq_stopped) {
+			t7xx_pci_disable_sleep(dpmaif_ctrl->t7xx_dev);
+			t7xx_do_tx_hw_push(dpmaif_ctrl);
+			t7xx_pci_enable_sleep(dpmaif_ctrl->t7xx_dev);
+		}
 		pm_runtime_put_autosuspend(dpmaif_ctrl->dev);
+
+		if (txq_stopped)
+			wait_event_interruptible(dpmaif_ctrl->tx_wq,
+						 t7xx_select_tx_queue(dpmaif_ctrl) ||
+						 kthread_should_stop());
 	}
 
 	return 0;
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH net v7 2/6] net: wwan: t7xx: fix runtime PM usage count underflow on -EACCES
  2026-11-20  1:55 [PATCH net v7 0/6] net: wwan: t7xx: fix TX push kthread soft lockup on suspend and RX stop issues Tim JH Chen
  2026-11-20  1:55 ` [PATCH net v7 1/6] net: wwan: t7xx: fix TX push kthread soft lockup on system suspend Tim JH Chen
@ 2026-11-20  1:55 ` Tim JH Chen
  2026-11-20  1:55 ` [PATCH net v7 3/6] net: wwan: t7xx: fix skb double free when CLDMA send sees -EACCES Tim JH Chen
                   ` (4 subsequent siblings)
  6 siblings, 0 replies; 8+ messages in thread
From: Tim JH Chen @ 2026-11-20  1:55 UTC (permalink / raw)
  To: netdev
  Cc: chandrashekar.devegowda, haijun.liu, ricardo.martinez,
	loic.poulain, ryazanov.s.a, johannes, andrew+netdev, davem,
	edumazet, kuba, pabeni, ilpo.jarvinen, sreehari.kancharla,
	m.chetan.kumar, horms, linux-kernel, tim.jh.chen, Chih.Hung.Huang,
	Tim JH Chen

t7xx_dpmaif_tx_hw_push_thread(), t7xx_dpmaif_tx_done(),
t7xx_dpmaif_bat_release_work() and t7xx_dpmaif_irq_rx_done() treat
-EACCES from pm_runtime_resume_and_get() as success and proceed to
access the hardware. That is intentional, but
pm_runtime_resume_and_get() has already dropped the usage count it
took before returning -EACCES, so the pm_runtime_put_autosuspend() at
the end of each context (in t7xx_dpmaif_napi_rx_poll() for the RX
interrupt) drops a reference that was never held.

Take the reference with pm_runtime_get_noresume() and resume with
pm_runtime_resume(), so that exactly one reference is held on every
path that touches the hardware, and drop it only on the errors that
abort the operation.

-EACCES is only returned while runtime PM is disabled and the device is
not active. Not observed; found by review.

Fixes: 46e8f49ed7b3 ("net: wwan: t7xx: Introduce power management")
Signed-off-by: Tim JH Chen <tim770802@gmail.com>
---
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c | 11 ++++++++---
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c | 14 ++++++++++----
 2 files changed, 18 insertions(+), 7 deletions(-)

diff --git a/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c b/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c
index 5af90ca6e063..77770869d575 100644
--- a/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c
+++ b/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c
@@ -911,8 +911,10 @@ void t7xx_dpmaif_irq_rx_done(struct dpmaif_ctrl *dpmaif_ctrl, const unsigned int
 	 * which can't be done from softirq context the poll function
 	 * is running in.
 	 */
-	ret = pm_runtime_resume_and_get(ctrl->dev);
+	pm_runtime_get_noresume(ctrl->dev);
+	ret = pm_runtime_resume(ctrl->dev);
 	if (ret < 0 && ret != -EACCES) {
+		pm_runtime_put_noidle(ctrl->dev);
 		dev_err(ctrl->dev, "Failed to resume device: %d\n", ret);
 		return;
 	}
@@ -1068,9 +1070,12 @@ static void t7xx_dpmaif_bat_release_work(struct work_struct *work)
 	struct dpmaif_rx_queue *rxq;
 	int ret;
 
-	ret = pm_runtime_resume_and_get(dpmaif_ctrl->dev);
-	if (ret < 0 && ret != -EACCES)
+	pm_runtime_get_noresume(dpmaif_ctrl->dev);
+	ret = pm_runtime_resume(dpmaif_ctrl->dev);
+	if (ret < 0 && ret != -EACCES) {
+		pm_runtime_put_noidle(dpmaif_ctrl->dev);
 		return;
+	}
 
 	t7xx_pci_disable_sleep(dpmaif_ctrl->t7xx_dev);
 
diff --git a/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c b/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c
index 798e4e95004f..48558b98e7c6 100644
--- a/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c
+++ b/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c
@@ -162,9 +162,12 @@ static void t7xx_dpmaif_tx_done(struct work_struct *work)
 	struct dpmaif_hw_info *hw_info;
 	int ret;
 
-	ret = pm_runtime_resume_and_get(dpmaif_ctrl->dev);
-	if (ret < 0 && ret != -EACCES)
+	pm_runtime_get_noresume(dpmaif_ctrl->dev);
+	ret = pm_runtime_resume(dpmaif_ctrl->dev);
+	if (ret < 0 && ret != -EACCES) {
+		pm_runtime_put_noidle(dpmaif_ctrl->dev);
 		return;
+	}
 
 	/* The device may be in low power state. Disable sleep if needed */
 	t7xx_pci_disable_sleep(dpmaif_ctrl->t7xx_dev);
@@ -461,9 +464,12 @@ static int t7xx_dpmaif_tx_hw_push_thread(void *arg)
 				break;
 		}
 
-		ret = pm_runtime_resume_and_get(dpmaif_ctrl->dev);
-		if (ret < 0 && ret != -EACCES)
+		pm_runtime_get_noresume(dpmaif_ctrl->dev);
+		ret = pm_runtime_resume(dpmaif_ctrl->dev);
+		if (ret < 0 && ret != -EACCES) {
+			pm_runtime_put_noidle(dpmaif_ctrl->dev);
 			return ret;
+		}
 
 		/* A runtime suspended device was resumed above and had its queues
 		 * restarted, so a stopped queue here belongs to a system suspend
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH net v7 3/6] net: wwan: t7xx: fix skb double free when CLDMA send sees -EACCES
  2026-11-20  1:55 [PATCH net v7 0/6] net: wwan: t7xx: fix TX push kthread soft lockup on suspend and RX stop issues Tim JH Chen
  2026-11-20  1:55 ` [PATCH net v7 1/6] net: wwan: t7xx: fix TX push kthread soft lockup on system suspend Tim JH Chen
  2026-11-20  1:55 ` [PATCH net v7 2/6] net: wwan: t7xx: fix runtime PM usage count underflow on -EACCES Tim JH Chen
@ 2026-11-20  1:55 ` Tim JH Chen
  2026-11-20  1:55 ` [PATCH net v7 4/6] net: wwan: t7xx: do not exit the TX push kthread on resume failure Tim JH Chen
                   ` (3 subsequent siblings)
  6 siblings, 0 replies; 8+ messages in thread
From: Tim JH Chen @ 2026-11-20  1:55 UTC (permalink / raw)
  To: netdev
  Cc: chandrashekar.devegowda, haijun.liu, ricardo.martinez,
	loic.poulain, ryazanov.s.a, johannes, andrew+netdev, davem,
	edumazet, kuba, pabeni, ilpo.jarvinen, sreehari.kancharla,
	m.chetan.kumar, horms, linux-kernel, tim.jh.chen, Chih.Hung.Huang,
	Tim JH Chen

t7xx_cldma_send_skb() tolerates -EACCES from
pm_runtime_resume_and_get() and goes on to queue the skb on the GPD
ring, but it keeps that -EACCES in ret across a successful send and
returns it. The callers treat a non-zero return as "not sent" and free
the skb (t7xx_port_fastboot_write(), t7xx_port_wwan_write()), while
t7xx_cldma_gpd_tx_collect() frees it again once the hardware is done
with it: a double free.

The same path also drops a runtime PM reference that was never taken,
since pm_runtime_resume_and_get() releases its reference before
returning -EACCES.

Take the reference with pm_runtime_get_noresume() and resume with
pm_runtime_resume(), so that one reference is always held across the
send, and clear ret after the resume so that only a failure of the send
itself is reported.

-EACCES is only returned while runtime PM is disabled and the device is
not active. Not observed; found by review.

Fixes: 46e8f49ed7b3 ("net: wwan: t7xx: Introduce power management")
Signed-off-by: Tim JH Chen <tim770802@gmail.com>
---
 drivers/net/wwan/t7xx/t7xx_hif_cldma.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/drivers/net/wwan/t7xx/t7xx_hif_cldma.c b/drivers/net/wwan/t7xx/t7xx_hif_cldma.c
index e1b37deb6d9b..5b4b9ed4a95b 100644
--- a/drivers/net/wwan/t7xx/t7xx_hif_cldma.c
+++ b/drivers/net/wwan/t7xx/t7xx_hif_cldma.c
@@ -926,9 +926,17 @@ int t7xx_cldma_send_skb(struct cldma_ctrl *md_ctrl, int qno, struct sk_buff *skb
 	if (qno >= CLDMA_TXQ_NUM)
 		return -EINVAL;
 
-	ret = pm_runtime_resume_and_get(md_ctrl->dev);
-	if (ret < 0 && ret != -EACCES)
+	pm_runtime_get_noresume(md_ctrl->dev);
+	ret = pm_runtime_resume(md_ctrl->dev);
+	if (ret < 0 && ret != -EACCES) {
+		pm_runtime_put_noidle(md_ctrl->dev);
 		return ret;
+	}
+
+	/* ret holds the resume status (1 for an active device, -EACCES with
+	 * runtime PM disabled), not the result of the send below.
+	 */
+	ret = 0;
 
 	t7xx_pci_disable_sleep(md_ctrl->t7xx_dev);
 	queue = &md_ctrl->txq[qno];
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH net v7 4/6] net: wwan: t7xx: do not exit the TX push kthread on resume failure
  2026-11-20  1:55 [PATCH net v7 0/6] net: wwan: t7xx: fix TX push kthread soft lockup on suspend and RX stop issues Tim JH Chen
                   ` (2 preceding siblings ...)
  2026-11-20  1:55 ` [PATCH net v7 3/6] net: wwan: t7xx: fix skb double free when CLDMA send sees -EACCES Tim JH Chen
@ 2026-11-20  1:55 ` Tim JH Chen
  2026-11-20  1:55 ` [PATCH net v7 5/6] net: wwan: t7xx: complete NAPI on the not-started RX poll early return Tim JH Chen
                   ` (2 subsequent siblings)
  6 siblings, 0 replies; 8+ messages in thread
From: Tim JH Chen @ 2026-11-20  1:55 UTC (permalink / raw)
  To: netdev
  Cc: chandrashekar.devegowda, haijun.liu, ricardo.martinez,
	loic.poulain, ryazanov.s.a, johannes, andrew+netdev, davem,
	edumazet, kuba, pabeni, ilpo.jarvinen, sreehari.kancharla,
	m.chetan.kumar, horms, linux-kernel, tim.jh.chen, Chih.Hung.Huang,
	Tim JH Chen, Andreas Haerter

t7xx_dpmaif_tx_hw_push_thread() returns, ending the kthread, when the
runtime PM resume fails with anything other than -EACCES. kthread_run()
keeps no extra reference to the task, so the task_struct can be reaped
or reused while dpmaif_ctrl->tx_thread still points at it. A later
t7xx_dpmaif_tx_thread_rel() then calls kthread_stop() on the stale
pointer. It also stops TX permanently and silently.

This was hit in the field after a runtime suspend failure ("SAP
suspend error: -110") left dev->power.runtime_error set: every resume
returned -EINVAL, the thread exited, and a later write to the PCI
remove attribute crashed in kthread_stop() with a NULL
worker_private. The remove then hung in D state holding
pci_rescan_remove_lock and the machine could not shut down cleanly.

Log the failure and retry every second instead of exiting. Only the
first failure of a streak is logged. The wait ends early on
kthread_stop(), so the thread stays alive until
t7xx_dpmaif_tx_thread_rel() tears it down. Recovering from a sticky
runtime_error is a separate problem and is not attempted here.

Fixes: 46e8f49ed7b3 ("net: wwan: t7xx: Introduce power management")
Reported-by: Andreas Haerter <ah@foundata.com>
Closes: https://lore.kernel.org/netdev/20261106054532.201311-1-ah@foundata.com/
Signed-off-by: Tim JH Chen <tim770802@gmail.com>
---
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c b/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c
index 48558b98e7c6..00e456bdd808 100644
--- a/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c
+++ b/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c
@@ -448,6 +448,7 @@ static void t7xx_do_tx_hw_push(struct dpmaif_ctrl *dpmaif_ctrl)
 static int t7xx_dpmaif_tx_hw_push_thread(void *arg)
 {
 	struct dpmaif_ctrl *dpmaif_ctrl = arg;
+	bool resume_failed = false;
 	bool txq_stopped;
 	int ret;
 
@@ -468,8 +469,19 @@ static int t7xx_dpmaif_tx_hw_push_thread(void *arg)
 		ret = pm_runtime_resume(dpmaif_ctrl->dev);
 		if (ret < 0 && ret != -EACCES) {
 			pm_runtime_put_noidle(dpmaif_ctrl->dev);
-			return ret;
+			/* Do not exit: t7xx_dpmaif_tx_thread_rel() kthread_stop()s
+			 * this task. Retry until the resume works or we are stopped.
+			 */
+			if (!resume_failed)
+				dev_err(dpmaif_ctrl->dev,
+					"Failed to resume for TX push: %d\n", ret);
+			resume_failed = true;
+			wait_event_interruptible_timeout(dpmaif_ctrl->tx_wq,
+							 kthread_should_stop(),
+							 HZ);
+			continue;
 		}
+		resume_failed = false;
 
 		/* A runtime suspended device was resumed above and had its queues
 		 * restarted, so a stopped queue here belongs to a system suspend
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH net v7 5/6] net: wwan: t7xx: complete NAPI on the not-started RX poll early return
  2026-11-20  1:55 [PATCH net v7 0/6] net: wwan: t7xx: fix TX push kthread soft lockup on suspend and RX stop issues Tim JH Chen
                   ` (3 preceding siblings ...)
  2026-11-20  1:55 ` [PATCH net v7 4/6] net: wwan: t7xx: do not exit the TX push kthread on resume failure Tim JH Chen
@ 2026-11-20  1:55 ` Tim JH Chen
  2026-11-20  1:55 ` [PATCH net v7 6/6] net: wwan: t7xx: fix memory ordering in t7xx_dpmaif_rx_stop() Tim JH Chen
  2026-11-20  2:03 ` [PATCH net v7 0/6] net: wwan: t7xx: fix TX push kthread soft lockup on suspend and RX stop issues netdev-bot+sinfo
  6 siblings, 0 replies; 8+ messages in thread
From: Tim JH Chen @ 2026-11-20  1:55 UTC (permalink / raw)
  To: netdev
  Cc: chandrashekar.devegowda, haijun.liu, ricardo.martinez,
	loic.poulain, ryazanov.s.a, johannes, andrew+netdev, davem,
	edumazet, kuba, pabeni, ilpo.jarvinen, sreehari.kancharla,
	m.chetan.kumar, horms, linux-kernel, tim.jh.chen, Chih.Hung.Huang,
	Tim JH Chen

When the RX queue is not started, t7xx_dpmaif_napi_rx_poll() returns 0
without calling napi_complete_done(). A poll that returns less than the
budget must complete the NAPI itself, so NAPI_STATE_SCHED stays set:
every later napi_schedule() from t7xx_dpmaif_irq_rx_done() is a no-op
and leaks the runtime PM reference it just took, no further RX poll
runs on that queue, and a later napi_synchronize() in
t7xx_ccmni_disable_napi() does not return.

This early return is taken after t7xx_dpmaif_rx_stop() clears
que_started, for example on the modem exception path.

Call napi_complete_done() before returning.

Fixes: 5545b7b9f294 ("net: wwan: t7xx: Add NAPI support")
Signed-off-by: Tim JH Chen <tim770802@gmail.com>
---
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c b/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c
index 77770869d575..a7d6289415b1 100644
--- a/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c
+++ b/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c
@@ -848,6 +848,7 @@ int t7xx_dpmaif_napi_rx_poll(struct napi_struct *napi, const int budget)
 		atomic_set(&rxq->rx_processing, 0);
 		pm_runtime_put_autosuspend(rxq->dpmaif_ctrl->dev);
 		dev_err(rxq->dpmaif_ctrl->dev, "Work RXQ: %d has not been started\n", rxq->index);
+		napi_complete_done(napi, work_done);
 		return work_done;
 	}
 
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH net v7 6/6] net: wwan: t7xx: fix memory ordering in t7xx_dpmaif_rx_stop()
  2026-11-20  1:55 [PATCH net v7 0/6] net: wwan: t7xx: fix TX push kthread soft lockup on suspend and RX stop issues Tim JH Chen
                   ` (4 preceding siblings ...)
  2026-11-20  1:55 ` [PATCH net v7 5/6] net: wwan: t7xx: complete NAPI on the not-started RX poll early return Tim JH Chen
@ 2026-11-20  1:55 ` Tim JH Chen
  2026-11-20  2:03 ` [PATCH net v7 0/6] net: wwan: t7xx: fix TX push kthread soft lockup on suspend and RX stop issues netdev-bot+sinfo
  6 siblings, 0 replies; 8+ messages in thread
From: Tim JH Chen @ 2026-11-20  1:55 UTC (permalink / raw)
  To: netdev
  Cc: chandrashekar.devegowda, haijun.liu, ricardo.martinez,
	loic.poulain, ryazanov.s.a, johannes, andrew+netdev, davem,
	edumazet, kuba, pabeni, ilpo.jarvinen, sreehari.kancharla,
	m.chetan.kumar, horms, linux-kernel, tim.jh.chen, Chih.Hung.Huang,
	Tim JH Chen

t7xx_dpmaif_napi_rx_poll() sets rx_processing, issues smp_mb(), then
reads que_started. t7xx_dpmaif_rx_stop() reads rx_processing first and
clears que_started afterwards. With this order the stop can read
rx_processing == 0 while a new poll reads que_started == true, and the
poll runs after t7xx_dpmaif_rx_stop() returns.

Clear que_started before waiting for rx_processing, so the two sides use
the same store, smp_mb(), load order. Use WRITE_ONCE()/READ_ONCE() on
the two accesses that form this handshake; the other accesses of
que_started are on the same side of the barrier or on paths that
t7xx_dpmaif_rx_stop() serialises through rx_processing.

With the new order a poll that is already past its que_started check
when the queue is stopped finishes its pass with que_started == false,
which used to be rare (only after the stop timed out). In that state
t7xx_dpmaifq_release_pit_entry() returned 0 without touching the
hardware and the caller zeroed pit_remain_release_cnt, so the entries
consumed by that pass would never be released to the hardware. Move
the check to t7xx_dpmaif_pit_release_and_add() and keep the pending
count, so the entries are released by the first poll after the queue
is restarted. The hardware is still up during that pass because
t7xx_dpmaif_rx_stop() waits for it before the hardware is stopped.

Fixes: d642b012df70 ("net: wwan: t7xx: Add data path interface")
Signed-off-by: Tim JH Chen <tim770802@gmail.com>
---
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c | 23 ++++++++++++++--------
 1 file changed, 15 insertions(+), 8 deletions(-)

diff --git a/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c b/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c
index a7d6289415b1..d70ef315f5f5 100644
--- a/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c
+++ b/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c
@@ -240,9 +240,6 @@ static int t7xx_dpmaifq_release_pit_entry(struct dpmaif_rx_queue *rxq,
 	unsigned int old_rel_idx, new_rel_idx, hw_wr_idx;
 	int ret;
 
-	if (!rxq->que_started)
-		return 0;
-
 	if (rel_entry_num >= rxq->pit_size_cnt) {
 		dev_err(rxq->dpmaif_ctrl->dev, "Invalid PIT release index\n");
 		return -EINVAL;
@@ -565,6 +562,14 @@ static int t7xx_dpmaif_pit_release_and_add(struct dpmaif_rx_queue *rxq)
 	if (rxq->pit_remain_release_cnt < DPMAIF_PIT_CNT_THRESHOLD)
 		return 0;
 
+	/* The queue is stopped: keep the count, the entries are released by
+	 * the first poll after the queue has been restarted.
+	 * t7xx_dpmaif_rx_stop() waits for this poll before the hardware is
+	 * stopped.
+	 */
+	if (!READ_ONCE(rxq->que_started))
+		return 0;
+
 	ret = t7xx_dpmaifq_release_pit_entry(rxq, rxq->pit_remain_release_cnt);
 	if (ret)
 		return ret;
@@ -844,7 +849,7 @@ int t7xx_dpmaif_napi_rx_poll(struct napi_struct *napi, const int budget)
 	/* Ensure rx_processing is changed to 1 before actually begin RX flow */
 	smp_mb();
 
-	if (!rxq->que_started) {
+	if (!READ_ONCE(rxq->que_started)) {
 		atomic_set(&rxq->rx_processing, 0);
 		pm_runtime_put_autosuspend(rxq->dpmaif_ctrl->dev);
 		dev_err(rxq->dpmaif_ctrl->dev, "Work RXQ: %d has not been started\n", rxq->index);
@@ -1127,14 +1132,16 @@ void t7xx_dpmaif_rx_stop(struct dpmaif_ctrl *dpmaif_ctrl)
 		struct dpmaif_rx_queue *rxq = &dpmaif_ctrl->rxq[i];
 		int timeout, value;
 
+		WRITE_ONCE(rxq->que_started, false);
+		/* Order the que_started store before the rx_processing load.
+		 * Pairs with the smp_mb() in t7xx_dpmaif_napi_rx_poll().
+		 */
+		smp_mb();
+
 		timeout = readx_poll_timeout_atomic(atomic_read, &rxq->rx_processing, value,
 						    !value, 0, DPMAIF_CHECK_INIT_TIMEOUT_US);
 		if (timeout)
 			dev_err(dpmaif_ctrl->dev, "Stop RX SW failed\n");
-
-		/* Ensure RX processing has stopped before we set rxq->que_started to false */
-		smp_mb();
-		rxq->que_started = false;
 	}
 }
 
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [PATCH net v7 0/6] net: wwan: t7xx: fix TX push kthread soft lockup on suspend and RX stop issues
  2026-11-20  1:55 [PATCH net v7 0/6] net: wwan: t7xx: fix TX push kthread soft lockup on suspend and RX stop issues Tim JH Chen
                   ` (5 preceding siblings ...)
  2026-11-20  1:55 ` [PATCH net v7 6/6] net: wwan: t7xx: fix memory ordering in t7xx_dpmaif_rx_stop() Tim JH Chen
@ 2026-11-20  2:03 ` netdev-bot+sinfo
  6 siblings, 0 replies; 8+ messages in thread
From: netdev-bot+sinfo @ 2026-11-20  2:03 UTC (permalink / raw)
  To: Tim JH Chen
  Cc: netdev, chandrashekar.devegowda, haijun.liu, ricardo.martinez,
	loic.poulain, ryazanov.s.a, johannes, andrew+netdev, davem,
	edumazet, kuba, pabeni, ilpo.jarvinen, sreehari.kancharla,
	m.chetan.kumar, horms, linux-kernel, tim.jh.chen, Chih.Hung.Huang

Hi!

This is an automated message. This series looks like a fix, but its
commit messages seem to be missing some information:

 - Whether the issue was actually triggered, or is only theoretical
   (e.g. found by LLM and not proven at runtime). If it was triggered
   please include the symptoms, like the stack trace or kernel messages.

Please do not repost the series just to address the above. Instead,
reply to this email with the missing information, so that reviewers
can take it into account.

*IMPORTANT*: If you do repost (for other reasons) - please include
the relevant info in the cover letter or commit messages.
-- 
The evaluation is done by an LLM so it may be wrong, if you think
that is the case please reply and explain.

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2026-11-20  2:03 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-20  1:55 [PATCH net v7 0/6] net: wwan: t7xx: fix TX push kthread soft lockup on suspend and RX stop issues Tim JH Chen
2026-11-20  1:55 ` [PATCH net v7 1/6] net: wwan: t7xx: fix TX push kthread soft lockup on system suspend Tim JH Chen
2026-11-20  1:55 ` [PATCH net v7 2/6] net: wwan: t7xx: fix runtime PM usage count underflow on -EACCES Tim JH Chen
2026-11-20  1:55 ` [PATCH net v7 3/6] net: wwan: t7xx: fix skb double free when CLDMA send sees -EACCES Tim JH Chen
2026-11-20  1:55 ` [PATCH net v7 4/6] net: wwan: t7xx: do not exit the TX push kthread on resume failure Tim JH Chen
2026-11-20  1:55 ` [PATCH net v7 5/6] net: wwan: t7xx: complete NAPI on the not-started RX poll early return Tim JH Chen
2026-11-20  1:55 ` [PATCH net v7 6/6] net: wwan: t7xx: fix memory ordering in t7xx_dpmaif_rx_stop() Tim JH Chen
2026-11-20  2:03 ` [PATCH net v7 0/6] net: wwan: t7xx: fix TX push kthread soft lockup on suspend and RX stop issues netdev-bot+sinfo

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox