From: Thinh Tran <thinhtr@linux.vnet.ibm.com>
To: netdev@vger.kernel.org
Cc: siva.kallam@broadcom.com, prashant@broadcom.com,
mchan@broadcom.com, drc@linux.vnet.ibm.com,
pavan.chebbi@broadcom.com,
Thinh Tran <thinhtr@linux.vnet.ibm.com>,
Venkata Sai Duggi <venkata.sai.duggi@ibm.com>
Subject: [PATCH] net/tg3: fix race condition in tg3_reset_task_cancel()
Date: Mon, 2 Oct 2023 13:55:10 -0500 [thread overview]
Message-ID: <20231002185510.1488-1-thinhtr@linux.vnet.ibm.com> (raw)
during the EEH error injection tests on the 4-port 1 GbE NetXtreme
BCM5719 Gigabit Ethernet PCIe adapter, a race condition was observed in
the process of resetting and setting the driver flag to
TX_RECOVERY_PENDING between tg3_reset_task_cancel() and tg3_tx_recover().
As a result, it occasionally leads to transmit timeouts and the
subsequent disabling of all the driver's interfaces.
[12046.886221] NETDEV WATCHDOG: eth16 (tg3): transmit queue 0 timed out
[12046.886238] WARNING: CPU: 7 PID: 0 at ../net/sched/sch_generic.c:478
dev_watchdog+0x42c/0x440
[12046.886247] Modules linked in: tg3 libphy nfsv3 nfs_acl .......
..........
[12046.886571] tg3 0021:01:00.0 eth16: transmit timed out, resetting
...........
[12046.966175] tg3 0021:01:00.1 eth15: transmit timed out, resetting
...........
[12046.981584] tg3 0021:01:00.2 eth14: transmit timed out, resetting
...........
[12047.056165] tg3 0021:01:00.3 eth13: transmit timed out, resetting
Fixing this issue by taking the spinlock when modifying the driver flag
Fixes: 6c4ca03bd890 ("net/tg3: resolve deadlock in tg3_reset_task() during EEH")
Signed-off-by: Thinh Tran <thinhtr@linux.vnet.ibm.com>
Tested-by: Venkata Sai Duggi <venkata.sai.duggi@ibm.com>
---
drivers/net/ethernet/broadcom/tg3.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/drivers/net/ethernet/broadcom/tg3.c b/drivers/net/ethernet/broadcom/tg3.c
index 14b311196b8f..f4558762f9de 100644
--- a/drivers/net/ethernet/broadcom/tg3.c
+++ b/drivers/net/ethernet/broadcom/tg3.c
@@ -6507,7 +6507,9 @@ static void tg3_tx_recover(struct tg3 *tp)
"Please report the problem to the driver maintainer "
"and include system chipset information.\n");
+ tg3_full_lock(tp, 0);
tg3_flag_set(tp, TX_RECOVERY_PENDING);
+ tg3_full_unlock(tp);
}
static inline u32 tg3_tx_avail(struct tg3_napi *tnapi)
@@ -7210,7 +7212,10 @@ static inline void tg3_reset_task_cancel(struct tg3 *tp)
{
if (test_and_clear_bit(TG3_FLAG_RESET_TASK_PENDING, tp->tg3_flags))
cancel_work_sync(&tp->reset_task);
+
+ tg3_full_lock(tp, 0);
tg3_flag_clear(tp, TX_RECOVERY_PENDING);
+ tg3_full_unlock(tp);
}
static int tg3_poll_msix(struct napi_struct *napi, int budget)
--
2.25.1
next reply other threads:[~2023-10-02 18:55 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-10-02 18:55 Thinh Tran [this message]
2023-10-03 4:34 ` [PATCH] net/tg3: fix race condition in tg3_reset_task_cancel() Pavan Chebbi
2023-10-31 23:18 ` Thinh Tran
2023-10-03 9:37 ` Michael Chan
2023-10-03 22:05 ` Thinh Tran
2023-11-02 16:02 ` Thinh Tran
2023-11-02 16:12 ` [PATCH v2] net/tg3: fix race condition in tg3_reset_task() Thinh Tran
2023-11-02 17:27 ` Michael Chan
2023-11-02 20:37 ` Thinh Tran
2023-11-14 17:39 ` Thinh Tran
2023-11-14 21:03 ` Michael Chan
2023-11-15 18:23 ` Thinh Tran
2023-11-15 18:56 ` Michael Chan
2023-11-16 14:41 ` Thinh Tran
2023-11-16 15:18 ` [PATCH v3] " Thinh Tran
2023-11-16 21:34 ` Michael Chan
2023-11-17 16:19 ` Thinh Tran
2023-11-17 18:31 ` Michael Chan
2023-11-30 22:29 ` Thinh Tran
2023-12-01 0:19 ` [PATCH v4] " Thinh Tran
2023-12-01 16:50 ` Michael Chan
2023-12-02 0:40 ` patchwork-bot+netdevbpf
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20231002185510.1488-1-thinhtr@linux.vnet.ibm.com \
--to=thinhtr@linux.vnet.ibm.com \
--cc=drc@linux.vnet.ibm.com \
--cc=mchan@broadcom.com \
--cc=netdev@vger.kernel.org \
--cc=pavan.chebbi@broadcom.com \
--cc=prashant@broadcom.com \
--cc=siva.kallam@broadcom.com \
--cc=venkata.sai.duggi@ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).