* [PATCH net v2 0/3] igb: fix runtime PM resume issues
@ 2026-11-16 15:30 Antonio Quartulli
2026-11-16 15:30 ` [PATCH net v2 1/3] igb: fix RTNL assertion when resuming from runtime suspend Antonio Quartulli
` (2 more replies)
0 siblings, 3 replies; 5+ messages in thread
From: Antonio Quartulli @ 2026-11-16 15:30 UTC (permalink / raw)
To: intel-wired-lan
Cc: Antonio Quartulli, Tony Nguyen, Przemek Kitszel, Andrew Lunn,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Heiner Kallweit, Kurt Kanzenbach, Joe Damato, Yan, Zheng, netdev,
linux-kernel
igb can be runtime suspended when the link goes down (with
power/control set to auto). Resuming it calls functions that need RTNL
without holding it (patches 1 and 2), and devices that can't signal a
wake-up are suspended anyway, so their port stays down when the link
comes back (patch 3).
Tested with the 82576 emulated by QEMU.
Changes in v2:
- patch 1: keep setting the queue counts in __igb_open(), only skipping
them on runtime resume (Sashiko)
- mention the tested hardware in the commit messages
v1: https://lore.kernel.org/netdev/20261113105835.3821433-1-antonio@mandelbit.com/
Antonio Quartulli (3):
igb: fix RTNL assertion when resuming from runtime suspend
igb: link queues to NAPIs under RTNL after runtime resume
igb: only allow runtime PM if the device can wake up
drivers/net/ethernet/intel/igb/igb.h | 1 +
drivers/net/ethernet/intel/igb/igb_main.c | 69 ++++++++++++++++++-----
2 files changed, 57 insertions(+), 13 deletions(-)
--
2.56.0
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH net v2 1/3] igb: fix RTNL assertion when resuming from runtime suspend
2026-11-16 15:30 [PATCH net v2 0/3] igb: fix runtime PM resume issues Antonio Quartulli
@ 2026-11-16 15:30 ` Antonio Quartulli
2026-11-17 15:31 ` sashiko-bot
2026-11-16 15:30 ` [PATCH net v2 2/3] igb: link queues to NAPIs under RTNL after runtime resume Antonio Quartulli
2026-11-16 15:30 ` [PATCH net v2 3/3] igb: only allow runtime PM if the device can wake up Antonio Quartulli
2 siblings, 1 reply; 5+ messages in thread
From: Antonio Quartulli @ 2026-11-16 15:30 UTC (permalink / raw)
To: intel-wired-lan
Cc: Antonio Quartulli, Tony Nguyen, Przemek Kitszel, Andrew Lunn,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Heiner Kallweit, netdev, linux-kernel, stable
Since commit ac8c58f5b535 ("igb: fix deadlock caused by taking RTNL in
RPM resume path") __igb_resume() doesn't take RTNL for runtime resume,
but __igb_open() still calls netif_set_real_num_{tx,rx}_queues(), which
require it. An asynchronous runtime resume (e.g. after the link comes
back) then triggers "RTNL: assertion failed".
Skip them when resuming. The counts are set again on every regular open,
which happens under RTNL; the only case left out is a fallback from
MSI-X to MSI during a resume, which leaves the stack with more queues
than the device uses until the next open.
Tested with the 82576 emulated by QEMU.
Fixes: ac8c58f5b535 ("igb: fix deadlock caused by taking RTNL in RPM resume path")
Cc: stable@vger.kernel.org
Closes: https://bugzilla.ipfire.org/show_bug.cgi?id=13562
Signed-off-by: Antonio Quartulli <antonio@mandelbit.com>
---
v2:
- keep setting the queue counts in __igb_open() after igb_request_irq(),
only skipping them on runtime resume: setting them in igb_open()
before the device was resumed broke the MSI-X to MSI fallback and XDP
setup on a runtime suspended device (Sashiko)
- drop the reference to igc, which has the same ordering problem
Note: igc sets the queue counts in igc_open() before igc_request_irq(),
which can fall back to MSI, since 6f31d6b643a3 ("igc: Refactor runtime
power management flow"). We didn't have igc hardware to test a fix with,
so no igc patch is sent for now.
drivers/net/ethernet/intel/igb/igb_main.c | 22 +++++++++++++---------
1 file changed, 13 insertions(+), 9 deletions(-)
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index 78f0786..4447546 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -4178,16 +4178,20 @@ static int __igb_open(struct net_device *netdev, bool resuming)
if (err)
goto err_req_irq;
- /* Notify the stack of the actual queue counts. */
- err = netif_set_real_num_tx_queues(adapter->netdev,
- adapter->num_tx_queues);
- if (err)
- goto err_set_queues;
+ /* Notify the stack of the actual queue counts. This needs RTNL,
+ * which isn't held when resuming from runtime suspend.
+ */
+ if (!resuming) {
+ err = netif_set_real_num_tx_queues(adapter->netdev,
+ adapter->num_tx_queues);
+ if (err)
+ goto err_set_queues;
- err = netif_set_real_num_rx_queues(adapter->netdev,
- adapter->num_rx_queues);
- if (err)
- goto err_set_queues;
+ err = netif_set_real_num_rx_queues(adapter->netdev,
+ adapter->num_rx_queues);
+ if (err)
+ goto err_set_queues;
+ }
/* From here on the code is the same as igb_up() */
clear_bit(__IGB_DOWN, &adapter->state);
--
2.56.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH net v2 2/3] igb: link queues to NAPIs under RTNL after runtime resume
2026-11-16 15:30 [PATCH net v2 0/3] igb: fix runtime PM resume issues Antonio Quartulli
2026-11-16 15:30 ` [PATCH net v2 1/3] igb: fix RTNL assertion when resuming from runtime suspend Antonio Quartulli
@ 2026-11-16 15:30 ` Antonio Quartulli
2026-11-16 15:30 ` [PATCH net v2 3/3] igb: only allow runtime PM if the device can wake up Antonio Quartulli
2 siblings, 0 replies; 5+ messages in thread
From: Antonio Quartulli @ 2026-11-16 15:30 UTC (permalink / raw)
To: intel-wired-lan
Cc: Antonio Quartulli, Tony Nguyen, Przemek Kitszel, Andrew Lunn,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Kurt Kanzenbach, Joe Damato, netdev, linux-kernel, stable
__igb_open() links queues to NAPIs with netif_queue_set_napi(), which
requires RTNL, but RTNL isn't held when resuming from runtime suspend:
this triggers "RTNL: assertion failed".
Don't link them in __igb_open() when resuming: do it right away on
system resume (RTNL held) and from a work item taking RTNL on runtime
resume. The links can't be kept across a suspend, because
__igb_shutdown() frees the q_vectors. These are only freed under RTNL
with __IGB_DOWN set, so the work item checks __IGB_DOWN under RTNL.
Tested with the 82576 emulated by QEMU.
Fixes: b75a1dea500f ("igb: Link queues to NAPI instances")
Cc: stable@vger.kernel.org # 6.16+
Closes: https://bugzilla.ipfire.org/show_bug.cgi?id=13562
Signed-off-by: Antonio Quartulli <antonio@mandelbit.com>
---
v2: no code changes; mention the tested hardware.
Note: igc may have the same issue: __igc_open() also links queues to NAPIs
(igc_set_queue_napi()) on the RTNL-less runtime resume path. We didn't
have igc hardware to test a fix with, so no igc patch is sent for now.
drivers/net/ethernet/intel/igb/igb.h | 1 +
drivers/net/ethernet/intel/igb/igb_main.c | 38 +++++++++++++++++++++--
2 files changed, 37 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/intel/igb/igb.h b/drivers/net/ethernet/intel/igb/igb.h
index 0fff1df..ea2d739 100644
--- a/drivers/net/ethernet/intel/igb/igb.h
+++ b/drivers/net/ethernet/intel/igb/igb.h
@@ -585,6 +585,7 @@ struct igb_adapter {
struct work_struct reset_task;
struct work_struct watchdog_task;
+ struct work_struct queue_napi_task;
bool fc_autoneg;
u8 tx_timeout_factor;
struct timer_list blink_timer;
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index 4447546..894eedb 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -2115,6 +2115,30 @@ void igb_set_queue_napi(struct igb_adapter *adapter, int vector,
NETDEV_QUEUE_TYPE_TX, napi);
}
+static void igb_link_queue_napis(struct igb_adapter *adapter)
+{
+ int i;
+
+ for (i = 0; i < adapter->num_q_vectors; i++)
+ igb_set_queue_napi(adapter, i, &adapter->q_vector[i]->napi);
+}
+
+/* Linking queues to NAPIs requires RTNL, which isn't held when resuming
+ * from runtime suspend: do it from here instead. The q_vectors are only
+ * freed under RTNL with __IGB_DOWN set, so checking it under RTNL is safe.
+ */
+static void igb_queue_napi_task(struct work_struct *work)
+{
+ struct igb_adapter *adapter = container_of(work, struct igb_adapter,
+ queue_napi_task);
+
+ rtnl_lock();
+ if (!test_bit(__IGB_DOWN, &adapter->state) &&
+ netif_running(adapter->netdev))
+ igb_link_queue_napis(adapter);
+ rtnl_unlock();
+}
+
/**
* igb_up - Open the interface and prepare it to handle traffic
* @adapter: board private structure
@@ -3426,6 +3450,7 @@ static int igb_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
INIT_WORK(&adapter->reset_task, igb_reset_task);
INIT_WORK(&adapter->watchdog_task, igb_watchdog_task);
+ INIT_WORK(&adapter->queue_napi_task, igb_queue_napi_task);
/* Initialize link properties that are user-changeable */
adapter->fc_autoneg = true;
@@ -3890,6 +3915,7 @@ static void igb_remove(struct pci_dev *pdev)
cancel_work_sync(&adapter->reset_task);
cancel_work_sync(&adapter->watchdog_task);
+ cancel_work_sync(&adapter->queue_napi_task);
#ifdef CONFIG_IGB_DCA
if (adapter->flags & IGB_FLAG_DCA_ENABLED) {
@@ -4199,7 +4225,8 @@ static int __igb_open(struct net_device *netdev, bool resuming)
for (i = 0; i < adapter->num_q_vectors; i++) {
napi = &adapter->q_vector[i]->napi;
napi_enable(napi);
- igb_set_queue_napi(adapter, i, napi);
+ if (!resuming)
+ igb_set_queue_napi(adapter, i, napi);
}
/* Clear any pending interrupts. */
@@ -9637,8 +9664,15 @@ static int __igb_resume(struct device *dev, bool rpm)
if (!rpm)
rtnl_lock();
- if (!err && netif_running(netdev))
+ if (!err && netif_running(netdev)) {
err = __igb_open(netdev, true);
+ if (!err) {
+ if (rpm)
+ schedule_work(&adapter->queue_napi_task);
+ else
+ igb_link_queue_napis(adapter);
+ }
+ }
if (!err)
netif_device_attach(netdev);
--
2.56.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH net v2 3/3] igb: only allow runtime PM if the device can wake up
2026-11-16 15:30 [PATCH net v2 0/3] igb: fix runtime PM resume issues Antonio Quartulli
2026-11-16 15:30 ` [PATCH net v2 1/3] igb: fix RTNL assertion when resuming from runtime suspend Antonio Quartulli
2026-11-16 15:30 ` [PATCH net v2 2/3] igb: link queues to NAPIs under RTNL after runtime resume Antonio Quartulli
@ 2026-11-16 15:30 ` Antonio Quartulli
2 siblings, 0 replies; 5+ messages in thread
From: Antonio Quartulli @ 2026-11-16 15:30 UTC (permalink / raw)
To: intel-wired-lan
Cc: Antonio Quartulli, Tony Nguyen, Przemek Kitszel, Andrew Lunn,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Yan, Zheng, netdev, linux-kernel, stable
igb lets the device be runtime suspended even if it can't signal a
wake-up from D3. Such a device doesn't notice the link coming back, so
the port stays down until it is resumed from user space.
Allow runtime PM only if pci_dev_run_wake() is true, as e1000e and
r8169 do. Devices that can't wake up no longer runtime suspend, even
with power/control set to auto.
Tested with the 82576 emulated by QEMU.
Fixes: 749ab2cd1270 ("igb: add basic runtime PM support")
Cc: stable@vger.kernel.org
Closes: https://bugzilla.ipfire.org/show_bug.cgi?id=13562
Signed-off-by: Antonio Quartulli <antonio@mandelbit.com>
---
v2: no code changes; mention the tested hardware.
Note: igc may have the same issue: igc_probe() also allows runtime PM
regardless of pci_dev_run_wake(). We didn't have igc hardware to test a
fix with, so no igc patch is sent for now.
drivers/net/ethernet/intel/igb/igb_main.c | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index 894eedb..9046611 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -3677,7 +3677,11 @@ static int igb_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
dev_pm_set_driver_flags(&pdev->dev, DPM_FLAG_NO_DIRECT_COMPLETE);
- pm_runtime_put_noidle(&pdev->dev);
+ /* A device that can't wake up wouldn't notice the link coming back
+ * while runtime suspended
+ */
+ if (pci_dev_run_wake(pdev))
+ pm_runtime_put_noidle(&pdev->dev);
return 0;
err_register:
@@ -3900,7 +3904,8 @@ static void igb_remove(struct pci_dev *pdev)
struct igb_adapter *adapter = netdev_priv(netdev);
struct e1000_hw *hw = &adapter->hw;
- pm_runtime_get_noresume(&pdev->dev);
+ if (pci_dev_run_wake(pdev))
+ pm_runtime_get_noresume(&pdev->dev);
#ifdef CONFIG_IGB_HWMON
igb_sysfs_exit(adapter);
#endif
--
2.56.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH net v2 1/3] igb: fix RTNL assertion when resuming from runtime suspend
2026-11-16 15:30 ` [PATCH net v2 1/3] igb: fix RTNL assertion when resuming from runtime suspend Antonio Quartulli
@ 2026-11-17 15:31 ` sashiko-bot
0 siblings, 0 replies; 5+ messages in thread
From: sashiko-bot @ 2026-11-17 15:31 UTC (permalink / raw)
To: Antonio Quartulli, intel-wired-lan
Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] igb: stale real queue counts after resume MSI fallback in
__igb_open()
--
commit 6c8e67bb8e01233489112f0bf629f1764d87d2c6
Author: Antonio Quartulli <antonio@mandelbit.com>
igb: fix RTNL assertion when resuming from runtime suspend
This commit skips calling netif_set_real_num_tx_queues() and
netif_set_real_num_rx_queues() in __igb_open() when resuming is true to
avoid triggering an RTNL assertion during runtime resume.
> diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
> index 131df3e09ba14..bf8acac72b324 100644
> --- a/drivers/net/ethernet/intel/igb/igb_main.c
> +++ b/drivers/net/ethernet/intel/igb/igb_main.c
> @@ -4178,16 +4178,20 @@ static int __igb_open(struct net_device *netdev, bool resuming)
> if (err)
> goto err_req_irq;
>
> - /* Notify the stack of the actual queue counts. */
> - err = netif_set_real_num_tx_queues(adapter->netdev,
> - adapter->num_tx_queues);
> - if (err)
> - goto err_set_queues;
> + /* Notify the stack of the actual queue counts. This needs RTNL,
> + * which isn't held when resuming from runtime suspend.
> + */
> + if (!resuming) {
> + err = netif_set_real_num_tx_queues(adapter->netdev,
> + adapter->num_tx_queues);
> + if (err)
> + goto err_set_queues;
>
> - err = netif_set_real_num_rx_queues(adapter->netdev,
> - adapter->num_rx_queues);
> - if (err)
> - goto err_set_queues;
> + err = netif_set_real_num_rx_queues(adapter->netdev,
> + adapter->num_rx_queues);
> + if (err)
> + goto err_set_queues;
> + }
>
> /* From here on the code is the same as igb_up() */
> clear_bit(__IGB_DOWN, &adapter->state);
[Severity: Medium]
Can leaving netdev->real_num_tx_queues and netdev->real_num_rx_queues stale
when MSI-X falls back to MSI during resume cause concurrent unlocked TX on
adapter->tx_ring[0] and a NULL pointer dereference in igb_xsk_pool_disable()?
During suspend, __igb_shutdown() -> igb_clear_interrupt_scheme() frees all
q_vectors and clears adapter->tx_ring[] and adapter->rx_ring[]. If MSI-X
allocation fails on resume (including system resume via igb_resume() ->
__igb_resume(dev, false), which already holds rtnl_lock() but still passes
resuming = true), the driver falls back to MSI with adapter->num_tx_queues
and adapter->num_rx_queues set to 1, allocating only index 0 while
netdev->real_num_tx_queues and netdev->real_num_rx_queues remain at N > 1.
In the TX path, the stack selects subqueues across 0 .. real_num_tx_queues - 1,
each serialized only by its own netdev->_tx[q]._xmit_lock, but
igb_tx_queue_mapping() folds all of them onto adapter->tx_ring[0]:
drivers/net/ethernet/intel/igb/igb_main.c:igb_tx_queue_mapping() {
unsigned int r_idx = skb->queue_mapping;
if (r_idx >= adapter->num_tx_queues)
r_idx = r_idx % adapter->num_tx_queues;
return adapter->tx_ring[r_idx];
}
CPU0 (subqueue 0) CPU1 (subqueue 1)
netdev_start_xmit() netdev_start_xmit()
HARD_TX_LOCK(dev, txq0) HARD_TX_LOCK(dev, txq1)
igb_xmit_frame() igb_xmit_frame()
igb_tx_queue_mapping() igb_tx_queue_mapping()
-> adapter->tx_ring[0] -> adapter->tx_ring[0]
igb_xmit_frame_ring(tx_ring[0]) igb_xmit_frame_ring(tx_ring[0])
first = &tx_ring-> first = &tx_ring->
tx_buffer_info[next_to_use] tx_buffer_info[next_to_use]
<-- concurrent ring corruption
Multiple CPUs transmitting on different subqueues concurrently run
igb_xmit_frame_ring() and igb_tx_map() on adapter->tx_ring[0], corrupting
tx_ring->next_to_use, tx_buffer_info, hardware descriptors, and BQL state.
In addition, igb_xsk_pool_disable() checks xsk_get_pool_from_qid(), which
validates qid against real_num_rx_queues and real_num_tx_queues; disabling an
AF_XDP pool bound to qid >= 1 after a resume fallback passes qid to
igb_txrx_ring_disable(), dereferencing NULL adapter->tx_ring[qid] and
adapter->rx_ring[qid].
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261116153041.3002666-1-antonio@mandelbit.com?part=1
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2026-11-17 15:31 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-16 15:30 [PATCH net v2 0/3] igb: fix runtime PM resume issues Antonio Quartulli
2026-11-16 15:30 ` [PATCH net v2 1/3] igb: fix RTNL assertion when resuming from runtime suspend Antonio Quartulli
2026-11-17 15:31 ` sashiko-bot
2026-11-16 15:30 ` [PATCH net v2 2/3] igb: link queues to NAPIs under RTNL after runtime resume Antonio Quartulli
2026-11-16 15:30 ` [PATCH net v2 3/3] igb: only allow runtime PM if the device can wake up Antonio Quartulli
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox