All of lore.kernel.org
 help / color / mirror / Atom feed
From: Tim JH Chen <tim770802@gmail.com>
To: netdev@vger.kernel.org
Cc: chandrashekar.devegowda@intel.com, haijun.liu@mediatek.com,
	ricardo.martinez@linux.intel.com, loic.poulain@oss.qualcomm.com,
	ryazanov.s.a@gmail.com, johannes@sipsolutions.net,
	andrew+netdev@lunn.ch, davem@davemloft.net, edumazet@kernel.org,
	kuba@kernel.org, pabeni@redhat.com,
	ilpo.jarvinen@linux.intel.com,
	sreehari.kancharla@linux.intel.com,
	m.chetan.kumar@linux.intel.com, horms@kernel.org,
	linux-kernel@vger.kernel.org, tim.jh.chen@wnc.com.tw,
	Chih.Hung.Huang@wnc.com.tw, Tim JH Chen <tim770802@gmail.com>
Subject: [PATCH net v7 1/6] net: wwan: t7xx: fix TX push kthread soft lockup on system suspend
Date: Fri, 20 Nov 2026 09:55:51 +0800	[thread overview]
Message-ID: <20261120015556.131547-2-tim770802@gmail.com> (raw)
In-Reply-To: <20261120015556.131547-1-tim770802@gmail.com>

t7xx_dpmaif_suspend() stops the TX queues (que_started = false) but
leaves dpmaif_ctrl->state at DPMAIF_STATE_PWRON. The TX push kthread
only sleeps while the TX lists are empty or state != PWRON, so when
skbs are still queued at the time the suspend callback runs its wait
condition stays true and every iteration completes without doing
anything and without sleeping: pm_runtime_resume_and_get() returns 0
without resuming anything (the PCI core resumed the device before the
suspend callback, and once runtime PM is disabled in the late phase an
active device is reported as already resumed); t7xx_pci_disable_sleep()
completes at once because md_pm_state is no longer MTK_PM_RESUMED;
t7xx_do_tx_hw_push() returns immediately because no queue is started.
There is no scheduling point in that loop, so the kthread spins on its
CPU for the rest of the suspend/resume cycle.

With s2idle and ASPM L1 enabled this was seen as a soft lockup after
about a thousand suspend/resume cycles (the 981st in the capture below).
The lockup was reported at different points of the suspend: while the
device callbacks were still running, right after this driver's own
suspend callback had stopped the queues, and after the wakeup interrupt
but before any resume callback had run. Captured with kdump on an
unmodified 6.19 kernel:

  mtk_t7xx 0000:56:00.0: PM: pci_pm_suspend returned 0 after 39260 usecs
  ...
  watchdog: BUG: soft lockup - CPU#4 stuck for 26s! [dpmaif_tx_hw_pu:764]
  RIP: 0010:ktime_get_mono_fast_ns+0x2e/0xe0
  Call Trace:
   ? t7xx_pci_enable_sleep+0x44/0x90 [mtk_t7xx]
   t7xx_dpmaif_tx_hw_push_thread+0x1f9/0x490 [mtk_t7xx]
   kthread+0x10b/0x220
   ret_from_fork+0x159/0x1d0
   ret_from_fork_asm+0x1a/0x30

Other captures sample the same loop in t7xx_pci_disable_sleep() or in
the runtime PM put; all other CPUs are idle.

The same loop runs after a resume that does not restart the queues (a
modem exception found in the PM resume state, or shutdown) until the
FSM takes the DPMAIF out of DPMAIF_STATE_PWRON.

Once pm_runtime_resume_and_get() has returned, a runtime suspended
device has been resumed and its queues restarted, so a TX queue that is
still stopped at that point was stopped by a system suspend or by an
exception. Skip the push in that case and wait on tx_wq until the queue
is restarted or the thread is stopped; t7xx_dpmaif_resume() and
t7xx_dpmaif_start() already wake tx_wq after restarting the queues.

The check is not folded into the thread's main wait condition on
purpose: a runtime suspended device also has its queues stopped and
relies on this thread calling pm_runtime_resume_and_get() to be
resumed when skbs are queued.

Fixes: 46e8f49ed7b3 ("net: wwan: t7xx: Introduce power management")
Signed-off-by: Tim JH Chen <tim770802@gmail.com>
---
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c | 20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)

diff --git a/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c b/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c
index 236d632cf591..798e4e95004f 100644
--- a/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c
+++ b/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c
@@ -445,6 +445,7 @@ static void t7xx_do_tx_hw_push(struct dpmaif_ctrl *dpmaif_ctrl)
 static int t7xx_dpmaif_tx_hw_push_thread(void *arg)
 {
 	struct dpmaif_ctrl *dpmaif_ctrl = arg;
+	bool txq_stopped;
 	int ret;
 
 	while (!kthread_should_stop()) {
@@ -464,10 +465,23 @@ static int t7xx_dpmaif_tx_hw_push_thread(void *arg)
 		if (ret < 0 && ret != -EACCES)
 			return ret;
 
-		t7xx_pci_disable_sleep(dpmaif_ctrl->t7xx_dev);
-		t7xx_do_tx_hw_push(dpmaif_ctrl);
-		t7xx_pci_enable_sleep(dpmaif_ctrl->t7xx_dev);
+		/* A runtime suspended device was resumed above and had its queues
+		 * restarted, so a stopped queue here belongs to a system suspend
+		 * or an exception: wait for t7xx_dpmaif_resume() or
+		 * t7xx_dpmaif_start() to restart it instead of spinning.
+		 */
+		txq_stopped = !t7xx_select_tx_queue(dpmaif_ctrl);
+		if (!txq_stopped) {
+			t7xx_pci_disable_sleep(dpmaif_ctrl->t7xx_dev);
+			t7xx_do_tx_hw_push(dpmaif_ctrl);
+			t7xx_pci_enable_sleep(dpmaif_ctrl->t7xx_dev);
+		}
 		pm_runtime_put_autosuspend(dpmaif_ctrl->dev);
+
+		if (txq_stopped)
+			wait_event_interruptible(dpmaif_ctrl->tx_wq,
+						 t7xx_select_tx_queue(dpmaif_ctrl) ||
+						 kthread_should_stop());
 	}
 
 	return 0;
-- 
2.43.0


  reply	other threads:[~2026-11-20  1:56 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-20  1:55 [PATCH net v7 0/6] net: wwan: t7xx: fix TX push kthread soft lockup on suspend and RX stop issues Tim JH Chen
2026-11-20  1:55 ` Tim JH Chen [this message]
2026-11-20  1:55 ` [PATCH net v7 2/6] net: wwan: t7xx: fix runtime PM usage count underflow on -EACCES Tim JH Chen
2026-11-20  1:55 ` [PATCH net v7 3/6] net: wwan: t7xx: fix skb double free when CLDMA send sees -EACCES Tim JH Chen
2026-11-20  1:55 ` [PATCH net v7 4/6] net: wwan: t7xx: do not exit the TX push kthread on resume failure Tim JH Chen
2026-11-20  1:55 ` [PATCH net v7 5/6] net: wwan: t7xx: complete NAPI on the not-started RX poll early return Tim JH Chen
2026-11-20  1:55 ` [PATCH net v7 6/6] net: wwan: t7xx: fix memory ordering in t7xx_dpmaif_rx_stop() Tim JH Chen
2026-11-20  2:03 ` [PATCH net v7 0/6] net: wwan: t7xx: fix TX push kthread soft lockup on suspend and RX stop issues netdev-bot+sinfo
2026-11-20  2:42   ` Tim JH Chen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261120015556.131547-2-tim770802@gmail.com \
    --to=tim770802@gmail.com \
    --cc=Chih.Hung.Huang@wnc.com.tw \
    --cc=andrew+netdev@lunn.ch \
    --cc=chandrashekar.devegowda@intel.com \
    --cc=davem@davemloft.net \
    --cc=edumazet@kernel.org \
    --cc=haijun.liu@mediatek.com \
    --cc=horms@kernel.org \
    --cc=ilpo.jarvinen@linux.intel.com \
    --cc=johannes@sipsolutions.net \
    --cc=kuba@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=loic.poulain@oss.qualcomm.com \
    --cc=m.chetan.kumar@linux.intel.com \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=ricardo.martinez@linux.intel.com \
    --cc=ryazanov.s.a@gmail.com \
    --cc=sreehari.kancharla@linux.intel.com \
    --cc=tim.jh.chen@wnc.com.tw \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.